Mark McLaren's Weblog

I'm now very enthusiastic, some may say obsessed, about using the Spring Framework. Spring is certainly making me more productive. The following example made me feel as if I'd switched from Java to pure XML as my core programming language. I'm embedding some closed-source java classes into one of my web applications. (Actually, I'm embedding the Dwarf IMAP and SMTP servers into a web application, this is insane huh?, anyway that bit is not important).

My closed source bean has a setter something like this:

setFile(java.io.File file)

Since this is to run embedded inside my web application I want set to my file with a path relative to where the web application is installed. Therefore, I think I need to locate the file resource using ServletContextResource and access the file by using MethodInvokingFactoryBean. In my Spring ApplicationContext I now have the following XML:

<bean class="closed.source.bean" id="someBean">
    <property name="file">
        <bean class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
            <property name="targetObject">                    
                <bean class="org.springframework.web.context.support.ServletContextResource">
                    <constructor-arg index="0">
                        <bean class="org.springframework.web.context.support.ServletContextFactoryBean"/>
                    </constructor-arg>
                    <constructor-arg index="1">
                        <value>/WEB-INF/somefile</value>
                    </constructor-arg> 
                </bean>
            </property>
            <property name="targetMethod">
                <value>getFile</value>
            </property>
        </bean>
    </property>
</bean>

It works but it just feels wrong.

A short while ago, Amit posted a question on my blog asking how he could render road data from a Google Earth KML file onto the web based Google Maps. I wasn't really sure what to suggest at the time and the best I could come up with was to create custom tiles. After a little experimentation I've now come with a different solution.

There have been many mashups using the Google Maps API, as requirements become more sophisticated and the dataset increases in size you start to find that a smattering of trigonometry is no longer sufficient or efficient.

I'm not a expert on Geographical Information Systems (GIS) but I do find the subject very interesting. A few weeks ago, I discovered that some databases have geographically ("spatial") aware extensions. The idea being that a database could be extended to support native spatial data types (co-ordinates, points, linestrings etc) and also support common GIS functions. These are some "GIS aware database" implementations that I found:

Commercial GIS aware database offerings

• DB2 Spatial Extender
• Spatial Oracle

Open source GIS aware database offerings

• SpatialDerby - GIS extenstion to Apache Derby
• MySQL has a set of spatial extensions
• PostGIS adds support for geographic objects to the PostgreSQL

I also have to mention JTS which is not a database but is a handy Java class library of GIS functions. Now from my perspective it would be ideal if there were a mature 100% Java database with a geographical extension but at the moment my preference from the free offerings that I found is PostGIS.

I will now describe the process I took from KML file to a fully working (if not quite production ready) Google Maps powered road renderer. It is not a particularly difficult process but there are quite a few steps involved. Whilst conducting my investigations I found that somebody had achieved something similar to this using PHP and output to SVG format (see: Dynamic Loading of Vector Geodata for SVG Mapping Applications Using Postgis, PHP and getURL()/XMLHttpRequest()).

I made use of the following tools

• PostGIS (and underlying PostgreSQL with pgAdmin III)
• TextPad for text file editing
• Google Earth for testing the KML file
• Quantum GIS for testing my GML file
• ogr2ogr (from FWTools) to import my GML file into PostGIS
• NetBeans to create my Java servlets and validate my XML documents (this includes a bundled version of Tomcat 5 to test my Java servlets)
• Google Maps API

KML to GML

Amit's KML file contained around 3MB of road data. The first stage in getting this data into the PostGIS database was to convert it from KML into GML. Since both KML and GML are both XML formats there are a couple of XSL styles floating around on the net that I could have used (e.g. Styling KML to GML). What I actually ended up doing was loading the KML into TextPad with the help of a couple of example GML files that I found on the web I set about performing some "Search and Replace" surgery until I got the KML formatted to look like the GML that I wanted. At this stage it is important to test the resulting GML file to make sure the rendered GML resembles the output of the original KML. GML is a little bit of a pig to validate, as it seems that GML is mostly intended to be embedded into other documents. This means that you may have to write your own DTD or XML Schema to get your GML to validate (yuck!). Once you have something that validates, then you can load it up into a GML aware renderer and see what you get. Quantum GIS (QGIS) is free and is able to render GML files.

Google Earth showing the road network as rendered via Amit's original KML file

Quantum GIS showing the same road network but this time rendered from the newly created GML file

GML to PostGIS

We have our GML file and have checked that the rendered version resembles the rendered version of the original KML file. The next step is to get the GML into the PostGIS database. There is a GIS toolkit called FWTools which include a utility called ogr2ogr which can be used to convert between different GIS formats (much like GPSBabel does for GPS systems). One really nice feature of ogr2ogr is that it can directly import data from GML files into PostGIS databases. I used this tool to import my GML data, invoking it using something like this:

ogr2ogr -f "PostgreSQL" "PG:dbname=postgis user=postgres password=postgres host=localhost port=5432" roads.gml

I could check on the data import and also tweak the table and column names with PostreSQL's pgAdmin III utility.

PostGIS to dynamically served GML and then to Google Maps API

Right, the road data is now in my PostGIS database. The next step is to create a servlet that will serve up only the relevant portions of the road data for Google Maps to render. This is a very common requirement of systems like PostGIS and since I am relatively unfamiliar with GIS jargon it took me a little while to pinpoint how exactly to do it. Apparently what I wanted to do was perform a "frame-based" query, the chapter 4 of the PostGIS documentation helpfully provides an example of how to do this (if I'd only known it was called this sooner!). Generally there is a lot of GIS jargon that is quite academic, mathematical and disconcerting for the uninitiated (e.g. convex hulls) but if you keep looking long enough you'll eventually find what you need!

As you'd expect the construction of my servlet was actually conducted in parallel with the creation of my Google Maps API HTML and JavaScript. I started my servlet with the examples of using Java clients with PostGIS. For the Google Maps part I modified the Event Listeners example from the Google Maps API Documentation to pass the bounding box co-ordinates of the current browser view.

After a little experimentation with other approaches (including using JSON) I chose to output GML format XML from my servlet. It just seems more straightforward to me to do it this way. I also noticed that IE seems to be particularly choosy about the name of the servlet (it seems to need to have the .xml extension and output the text/xml content type).

Google Maps API rendering part of the road network in Firefox, the JavaScript processes the servlet generated GML

Quantum GIS rendering of the same part of the road network as shown above, again using the same servlet generated GML

Static Demo

Here is a static version of of my Google Maps powered road renderer. I don't want to serve up the complete servlet backed application from my blog server without doing some further optimisation.

Resources

GML generating servlet source code: GoogleMapsServlet.java
Google Maps HTML/JavaScript file: RoadsRender.html [Note: this is not a live example as the background servlet is not running].

A copy of Amit's original KML file: Amit-Uttaranchal.kml [zipped 858 KB, unzipped 3.62 MB]
GML road network file derived from the above KML file: roads.gml [zipped 983 KB, unzipped 4.32 MB]
XML Schema for the above GML file: roads.xsd

I've recently been working on integrating a webmail client with our uPortal installation. The consensus is that the best IMAP webmail clients seem to be written in PHP (SquirrelMail and Horde IMP).

The idea is that we achieve single sign-on between our portal and webmail client via the JASIG CAS authentication broker. I'm planning to use ESUP's phpCAS.

My development environment is currently Win2K, I discovered a really nice set of 100% Java e-mail servers (part of the Dwarf Server Framework) these are ideal for IMAP/SMTP testing purposes.

Usually to CASify a proper Unix IMAP server you need to install a PAM CAS module. Although there are some free windows IMAP/SMTP servers around, from what I have seen, most of them don't support pluggable authentication mechanisms. The idea of porting a Unix IMAP server with PAM authentication to windows via Cygwin is enough to give me nightmares!

I had a play with the Dwarf IMAP Server and I have found that it more than suits my purposes for development. Obviously I won't be recommending that we move to using the Dwarf IMAP server in production; I'd no more suggest this than I'd install my washing machine onto the top shelf of my bookcase. Platform portability has it's place but for mission critical systems like e-mail then native C IMAP servers are still essential! The Dwarf Server Framework is not open source but it is free to use. The nice thing about the Dwarf IMAP server is that since it is written in Java I can fairly easily add my own authentication mechanisms, I found Dwarf very easy to CASify.

Until the Apache James project comes up with a stable and easy to install open source IMAP server, Dwarf will do very nicely.

A little while ago I followed a Google Ads link and signed up for a free subscription to Visual Systems Journal (a UK based publication). The reason I did this was because the text of the Google Ad proudly proclaimed something along the lines of "Now with HANDS-ON JAVA section". Cool, I thought, a free UK magazine with some Java coverage.

I’ve now received a couple of issues and I can't help feeling a little duped. In the HANDS-ON JAVA section over the last few issues I've seen articles on Python,Ruby on Rails and an article from Bruce Tate entitled "Pushing for pensioning Java" which seemed to be calling for the demise of Java. Don't get me wrong there is some really good stuff about Java on the VSJ website and the Bruce Tate article made some very good points (it was just there was little in the way of counterargument in the magazine).

Call me an old cynic if you like; I think the reason that the Hands-On Java section in the printed VSJ magazine gives such a negative view of Java is because all their advertisers are promoting Microsoft related technologies! I should probably stick to buying JDJ from Borders at highly inflated prices.

I've been upgrading various pieces of server software. I've also upgraded my blog software to the latest Pebble 2.0.0-M1 incarnation. Gone are the categories, these are replaced with a shiny new Web 2.0 style tag cloud. I also added a table of contents page. So now there are numerous ways to navigate my blog, search, calendar, tag and content based.

I also took advantage of the new Pebble skin to overhaul the design. I'm not really much of a designer so I based my new design on the Deliciously Blue design from the Open Source Web Design collection.

Incidentally, the fabric style tile shown below was used as the background to my previous blog design. It is from SquidFingers which has many other excellent free background tiles. I always meant to reference it but I had lost the link until recently.