Mark McLaren's Weblog

Arthur C. Clarke's Third Law was related to me recently in the context of Google Suggest, it goes something like:

I first saw Google Suggest around the 17th December 2004. At the time I had no idea how it worked, it was as magic to me. In the intervening period the AJAX phenomenon has grown, especially following the arrival of Google Maps. In November last year, I put together a modest Google Suggest clone of my own using Periodic Table data (see: Yet another Google Suggest Clone). This was largely based upon the ObjectGraph Dictionary, accompanying explanation and articles from phpRiot and XmlProNews. My initial implementation was a poor imitation of the rich interface that Google Suggest offers.

I have since improved upon that effort to a point where it is almost usable (bear with me self deprecation is a proud British tradition)! The following can be said about almost any human endeavour but I still feel the need to say it anyway. This effort represents the combination of ideas and techniques gathered, stolen and borrowed from several other authors. Although, I do hope that it is sufficiently novel in its own right as to merit some attention.

A valuable reference for this was Chris Justus' Google Suggest Dissected. Google Suggest has been explored by many others. My primary aim was to learn about how Google Suggest works to try and replicate its functionality for myself. Some methods I have found concentrate too much on the use of particular library or backend technology (e.g. JSP, ColdFusion, ASP, PHP) as an integral part of the solution. Google Suggest is all about JavaScript, how the backend is produced should be largely immaterial. I have opted to use JSON as the backend data format. JSON can apparently be produced using a wide range of technologies including some that are relatively obscure, such as Chicken Scheme and Squeak.

The evolution of my Google Suggest clone

Starting with my Periodic Table Suggest application the first step I took was to fix the CSS in order to stop the screen jumping about as a result of the appearing and vanishing "dropdown" suggestion box.

The original version used crude text separators as a backend data format rather than any recognised format. I have modified it to reply with JSON format data (this is easily modified to output larger arrays of data for more complicated functionality).

In the original application the onkeyup event triggered a new query, meaning a query was sent after every key press. I modified the application to use the submission throttling approach, this polls the textbox obtaining a value periodically rather than after every keypress.

A colleague of mine had rewritten my initial simple example to allow the use of the up and down keys to navigate between the suggestion options. I took his code on board and with the help of another example I found (here, found via Find a U.S. City - Suggest Two) I reintegrated this code into my original suggest application.

Inspired by Creating an Autosuggest Textbox with JavaScript, Part 1, I added autocomplete functionality. I refactored the behaviour to make it as simpler and removed duplicate functionality where I could. Finally, I wrapped the whole thing inside object oriented JavaScript (in a vain attempt that I could run two suggest windows on a single page, needs more work!).

It is still not perfect yet but AFAICT it pretty much emulates Google Suggest behaviour. The example in Creating an Autosuggest Textbox with JavaScript, Part 2 achieves the same thing and probably does a better job of it than I do! At least I have the satisfaction of having gone through each step and as a result I have a clear idea of what is going on. I have created an example application using the Cities table from the MONDIAL database (a larger dataset than the elements of the periodic table that I previously used). This can be seen here:

An example application: City Suggest

For a finishing touch I created a Google Suggest style logo using the logo maker tool at http://googlefor.com/.

How my suggest implementation works:

The JavaScript behind this application can be accessed here. Once the JavaScript Suggest class is instantiated, the program continually checks for textbox changes, key presses and mouse events.

• onkeydown events trigger suggestion highlighting and setValues()
• onmousemove events trigger highlight() suggestion
• onmousedown events trigger setValues()
• onkeyup events trigger an autosuggest option refresh

It is all encapsulated inside a class called Suggest. An instance of Suggest is created (with suitable parameters) and then the startup() method is invoked.

startup()

assigns an onkeydown event handler (keypressHandler() - for the up/down/enter key functionality) and an onkeyup event handler (keyupHandler() - for the autocomplete functionality). It invokes an infinite polling loop (requestLoop()).

requestLoop()

each iteration of this loop it first obtains the "unselected" portion of the textbox (using query()) and if the query has changed since the last request it makes a new request for data from the backend (using sendQuery()).

query()

retrieves unselected portion of the textbox

sendQuery()

calls initialize() in order to establish a usable XMLHttpRequest, on retrieving a result it passes it to process()

initialize()

establish XMLHttpRequest

process()

obtains JSON object and makes a call to htmlFormat() to render the result or reports an error

htmlFormat()

renders the suggestion box which includes adding listeners for onmousemove (to enable mouse driven suggestion highlighting - calling highlight()) and onmousedown (to enable mouse driven suggestion selection - calling setValues()). At the end of the rendering a function call is made that assembles suitable autocomplete options (requestSuggestions()).

keyupHandler()

handles keyup events, principally used for autocomplete functionality

keypressHandler()

handles keydown events, principally for key up, key down and enter key

highlight()

invoked by onmousemove event, highlights current selected suggestion

setValues()

invoked by onmousedown and onkeydown, sets textbox value to currently selected suggestion

showAutocompleteDiv()

shows autocomplete div

hideAutocompleteDiv()

hides autocomplete div

requestSuggestions()

builds an array of currently matching autocomplete options, calls autosuggest

autosuggest()

calls typeAhead if a non-empty autocomplete array exists

typeAhead()

inserts a suggestion into the textbox, highlighting the suggested part of the text.

selectRange()

selects a range of text in the textbox.

trim()

trims whitespace from a given string

I have been doing some further work on my simple Google Suggest clone recently (I've not finished doing this yet, more on this another time). A colleague of mine brought the autocomplete or (type ahead) behaviour of Google Suggest to my attention. I have to admit that I'd never really noticed this before, it is so easy to take clever AJAX behaviour for granted (one of the reasons for AJAX's success is it just feels natural). Whilst trying to figure out how it works I tripped over a JavaScript feature that I never knew existed before. If you are viewing this as a standalone entry you will be able to see an example of it working below (fixed! I think my blog front page is too JavaScript laden for it to work there). The highlighting is achieved via JavaScript. JavaScript is also used to tell you what text is highlighted and what text is not highlighted. Here is the JavaScript.

The mechanism used (createTextRange/setSelectionRange) differs between IE and Firefox. Whilst investigating this I found the following references useful:

I'm famous...well my blog is! It was recently referenced in the recently published O'Reilly book called Google Maps Hacks by Rich Gibson, Schuyler Erle (January 2006).

In section 4.10.6:

Hack 37. View Your GPS Tracklogs in Google Maps

My blog is referenced as:

How to use Google Maps' XSLT voodoo to process the GPX file

How cool is that?!!

The Spring Framework is my new favourite thing! I toyed with LDAP and Spring recently. There is a Sourceforge project called LDAPTemplate maintained by Mattias Arthursson and Ulrik Sandberg of Jayway which looks to be very impressive. It is built on the same principles as Spring JDBCTemplate and other DAO templates. Unfortunately since I am working with LDAP version 2 (rather than version 3) I couldn't get it to work but the guys are working on fixing this for a future release. I can see this project becoming part of the core Spring framework in the not too distant future.

I did however manage to get an earlier version of the code from LDAPTemplate written by Olivier Jolly working with the following example.

Spring Config file (applicationContext.xml):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">

<beans>
    <bean id="rootContextSource"
        class="org.springframework.ldap.support.UrlContextSource">
        <property name="ldapUrl" value="ldap://hostname:389/o=Blah, c=GB"/>
    </bean>
    <bean id="ldapTemplate"
        class="org.springframework.ldap.core.LdapTemplate">
        <property name="contextSource">
            <ref bean="rootContextSource" />
        </property>
    </bean>
</beans>

LDAPTest.java

import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import javax.naming.NamingEnumeration;
import javax.naming.directory.Attributes;
import javax.naming.directory.SearchControls;
import javax.naming.directory.SearchResult;
import org.springframework.beans.factory.xml.XmlBeanFactory;
import org.springframework.core.io.ClassPathResource;
import org.springframework.ldap.core.LdapTemplate;
import org.springframework.ldap.core.SearchResultCallbackHandler;

public class LDAPTest {

    public static void main(String[] args) throws Exception {

        // Manually create the Spring Bean Factory
        // Note: The factory is driven by the Bean definitions held in ApplicationContext
        // Note: There are also other implementations of Resource and Factory, but
        //       the following combination work well for standalone apps.

        ClassPathResource res = new ClassPathResource("applicationContext.xml");
        XmlBeanFactory factory = new XmlBeanFactory(res);
        {
            SearchControls ctls = new SearchControls();
            ctls.setSearchScope(SearchControls.SUBTREE_SCOPE);
            // Specify the ids of the attributes to return
            String[] attrIDs = {"sn", "mail", "uid", "cn", "ou", "postalAddress", "postalCode"};
            ctls.setReturningAttributes(attrIDs);

            LdapTemplate ldapTemplate = (LdapTemplate)factory.getBean("ldapTemplate");
            List l = (List) ldapTemplate.search(
                    "",
                    "(sn=McLaren)",
                    ctls, new SearchResultCallbackHandler() {
                LinkedList list = new LinkedList();

                public void processSearchResult(SearchResult searchResult) throws javax.naming.NamingException {
                    Attributes attrs = searchResult.getAttributes();
                    list.add(attrs);
                }

                public Object getResult() {
                    return list;
                }
            });
            Iterator i = l.iterator();
            while(i.hasNext()) {
                Attributes o = (Attributes) i.next();
                NamingEnumeration ids = o.getIDs();
                while(ids.hasMore()){
                    String att = (String) ids.next();
                    System.out.println(att + ":");
                    System.out.println(o.get(att).get());
                }
                System.out.println();
            }
        }
        System.out.println("Done");
    }

}

What is it about Spring that makes me use anonymous classes all the time?