Saturday, May 27, 2006

Rendering roads on Google Maps using Java and PostGIS

A short while ago, Amit posted a question on my blog asking how he could render road data from a Google Earth KML file onto the web based Google Maps. I wasn't really sure what to suggest at the time and the best I could come up with was to create custom tiles. After a little experimentation I've now come with a different solution.

There have been many mashups using the Google Maps API, as requirements become more sophisticated and the dataset increases in size you start to find that a smattering of trigonometry is no longer sufficient or efficient.

I'm not a expert on Geographical Information Systems (GIS) but I do find the subject very interesting. A few weeks ago, I discovered that some databases have geographically ("spatial") aware extensions. The idea being that a database could be extended to support native spatial data types (co-ordinates, points, linestrings etc) and also support common GIS functions. These are some "GIS aware database" implementations that I found:

Commercial GIS aware database offerings

Open source GIS aware database offerings

I also have to mention JTS which is not a database but is a handy Java class library of GIS functions. Now from my perspective it would be ideal if there were a mature 100% Java database with a geographical extension but at the moment my preference from the free offerings that I found is PostGIS.

I will now describe the process I took from KML file to a fully working (if not quite production ready) Google Maps powered road renderer. It is not a particularly difficult process but there are quite a few steps involved. Whilst conducting my investigations I found that somebody had achieved something similar to this using PHP and output to SVG format (see: Dynamic Loading of Vector Geodata for SVG Mapping Applications Using Postgis, PHP and getURL()/XMLHttpRequest()).

I made use of the following tools

KML to GML

Amit's KML file contained around 3MB of road data. The first stage in getting this data into the PostGIS database was to convert it from KML into GML. Since both KML and GML are both XML formats there are a couple of XSL styles floating around on the net that I could have used (e.g. Styling KML to GML). What I actually ended up doing was loading the KML into TextPad with the help of a couple of example GML files that I found on the web I set about performing some "Search and Replace" surgery until I got the KML formatted to look like the GML that I wanted. At this stage it is important to test the resulting GML file to make sure the rendered GML resembles the output of the original KML. GML is a little bit of a pig to validate, as it seems that GML is mostly intended to be embedded into other documents. This means that you may have to write your own DTD or XML Schema to get your GML to validate (yuck!). Once you have something that validates, then you can load it up into a GML aware renderer and see what you get. Quantum GIS (QGIS) is free and is able to render GML files.

Google Earth showing the road network as rendered via Amit's original KML file

Quantum GIS showing the same road network but this time rendered from the newly created GML file



GML to PostGIS

We have our GML file and have checked that the rendered version resembles the rendered version of the original KML file. The next step is to get the GML into the PostGIS database. There is a GIS toolkit called FWTools which include a utility called ogr2ogr which can be used to convert between different GIS formats (much like GPSBabel does for GPS systems). One really nice feature of ogr2ogr is that it can directly import data from GML files into PostGIS databases. I used this tool to import my GML data, invoking it using something like this:

ogr2ogr -f "PostgreSQL" "PG:dbname=postgis user=postgres password=postgres host=localhost port=5432" roads.gml

I could check on the data import and also tweak the table and column names with PostreSQL's pgAdmin III utility.

PostGIS to dynamically served GML and then to Google Maps API

Right, the road data is now in my PostGIS database. The next step is to create a servlet that will serve up only the relevant portions of the road data for Google Maps to render. This is a very common requirement of systems like PostGIS and since I am relatively unfamiliar with GIS jargon it took me a little while to pinpoint how exactly to do it. Apparently what I wanted to do was perform a "frame-based" query, the chapter 4 of the PostGIS documentation helpfully provides an example of how to do this (if I'd only known it was called this sooner!). Generally there is a lot of GIS jargon that is quite academic, mathematical and disconcerting for the uninitiated (e.g. convex hulls) but if you keep looking long enough you'll eventually find what you need!

As you'd expect the construction of my servlet was actually conducted in parallel with the creation of my Google Maps API HTML and JavaScript. I started my servlet with the examples of using Java clients with PostGIS. For the Google Maps part I modified the Event Listeners example from the Google Maps API Documentation to pass the bounding box co-ordinates of the current browser view.

After a little experimentation with other approaches (including using JSON) I chose to output GML format XML from my servlet. It just seems more straightforward to me to do it this way. I also noticed that IE seems to be particularly choosy about the name of the servlet (it seems to need to have the .xml extension and output the text/xml content type).

Google Maps API rendering part of the road network in Firefox, the JavaScript processes the servlet generated GML

Quantum GIS rendering of the same part of the road network as shown above, again using the same servlet generated GML

Static Demo

Here is a static version of of my Google Maps powered road renderer. I don't want to serve up the complete servlet backed application from my blog server without doing some further optimisation.

Resources

GML generating servlet source code: GoogleMapsServlet.java
Google Maps HTML/JavaScript file: RoadsRender.html [Note: this is not a live example as the background servlet is not running].

A copy of Amit's original KML file: Amit-Uttaranchal.kml [zipped 858 KB, unzipped 3.62 MB]
GML road network file derived from the above KML file: roads.gml [zipped 983 KB, unzipped 4.32 MB]
XML Schema for the above GML file: roads.xsd

7 comments:

Mark McLaren said...

Mark you are simply awesome.. thank you so much..
Note: Comment imported. Original by Amit at 2006-05-28 00:32

Mark McLaren said...

Mark:



Good stuff spreading the word about using PostGIS in concert with Google interfaces. We put up a site recently (www.gulfimpact.com) that uses PostGIS to allow the user to build buffers and do spatial intersections using Google Earth. There's a wide range of interesting apps possible once you can start harnessing all of that geoprocessing power over the web...
Note: Comment imported. Original by Brian Timoney website: http://www.gulfimpact.com at 2006-05-29 06:35

Mark McLaren said...

You could also just use GeoServer to do the PostGIS -> GML step (using WFS a standard protocol for getting GML). GeoServer can also overlay on google maps, as well as produce KML from the same database. You can also get data from Oracle, DB2, ArcSDE, Shapefiles, ect. and produce it as GML (and it'll automatically make the right XML schema for you). Perhaps soon someone will contribute a KML datastore, and it could do all this automatically.
Note: Comment imported. Original by Chris Holmes website: http://cholmes.wordpress.com at 2006-06-02 23:15

Mark McLaren said...

I'm trying to follow what you've done and then adapt it for Microsoft Virtual Earth, but unfortunately for me I'm new to Servlets and XML and I'm getting quite confused about eveything oustide of the PostGIS and HTML stages.. I'd really appreciate any help you could offer me.

Thanks


Note: Comment imported. Original by Jason at 2007-02-27 12:18

Mark McLaren said...

Hi Jason,



I would love to help but I know absolutely nothing about Virtual Earth. The reason this works in Google Maps is because there is a means to pass how the map is moving to a backend service (in my case a servlet). (E.g. Google Maps API Event Listeners example).





You can also use the Google Maps API to supply "bounding box" information to the backend service so that I the servlet can provide data specifically for the visible map.





Can this be done using Virtual Earth? I had a very quick look and I could not fine any decent documentation or examples of this (although you would think it would be possible).


Note: Comment imported. Original by markmc website: http://content.mark-mclaren.info/ at 2007-02-27 15:33

Mark McLaren said...

Hi Mark, Yes It can be done with Virtual Earth although the code looks a little different.. you can download the documentation for MSVE v4 from this http..

http://www.microsoft.com/downloads/details.aspx?FamilyId=121CDAE7-EA23-4634-B815-4300EB98EE88&displaylang=en



actually my problems have been with getting the Html to talk to the servlet and that to talk to the databases.. Its the first time i've tried to do anyhting like this so Its a bit mind-boggling.. Thanks for your help though!



Cheers

Jason
Note: Comment imported. Original by Jason at 2007-02-27 17:47

Mark McLaren said...

Mark



A really useful explanation, it points me in the right direction to serve data out of Oracle spatial onto Google maps. Thanks for your work on this.



Nigel
Note: Comment imported. Original by Nigel at 2007-02-27 18:45