A short while ago, Amit posted a question on my blog asking how he could render road data from a Google Earth KML file onto the web based Google Maps. I wasn't really sure what to suggest at the time and the best I could come up with was to create custom tiles. After a little experimentation I've now come with a different solution.
There have been many mashups using the Google Maps API, as requirements become more sophisticated and the dataset increases in size you start to find that a smattering of trigonometry is no longer sufficient or efficient.
I'm not a expert on Geographical Information Systems (GIS) but I do find the subject very interesting. A few weeks ago, I discovered that some databases have geographically ("spatial") aware extensions. The idea being that a database could be extended to support native spatial data types (co-ordinates, points, linestrings etc) and also support common GIS functions. These are some "GIS aware database" implementations that I found:
Commercial GIS aware database offerings
Open source GIS aware database offerings
I also have to mention JTS which is not a database but is a handy Java class library of GIS functions. Now from my perspective it would be ideal if there were a mature 100% Java database with a geographical extension but at the moment my preference from the free offerings that I found is PostGIS.
I will now describe the process I took from KML file to a fully working (if not quite production ready) Google Maps powered road renderer. It is not a particularly difficult process but there are quite a few steps involved. Whilst conducting my investigations I found that somebody had achieved something similar to this using PHP and output to SVG format (see: Dynamic Loading of Vector Geodata for SVG Mapping Applications Using Postgis, PHP and getURL()/XMLHttpRequest()).
I made use of the following tools
KML to GML
Amit's KML file contained around 3MB of road data. The first stage in getting this data into the PostGIS database was to convert it from KML into GML. Since both KML and GML are both XML formats there are a couple of XSL styles floating around on the net that I could have used (e.g. Styling KML to GML). What I actually ended up doing was loading the KML into TextPad with the help of a couple of example GML files that I found on the web I set about performing some "Search and Replace" surgery until I got the KML formatted to look like the GML that I wanted. At this stage it is important to test the resulting GML file to make sure the rendered GML resembles the output of the original KML. GML is a little bit of a pig to validate, as it seems that GML is mostly intended to be embedded into other documents. This means that you may have to write your own DTD or XML Schema to get your GML to validate (yuck!). Once you have something that validates, then you can load it up into a GML aware renderer and see what you get. Quantum GIS (QGIS) is free and is able to render GML files.
Google Earth showing the road network as rendered via Amit's original KML file
Quantum GIS showing the same road network but this time rendered from the newly created GML file
GML to PostGIS
We have our GML file and have checked that the rendered version resembles the rendered version of the original KML file. The next step is to get the GML into the PostGIS database. There is a GIS toolkit called FWTools which include a utility called ogr2ogr which can be used to convert between different GIS formats (much like GPSBabel does for GPS systems). One really nice feature of ogr2ogr is that it can directly import data from GML files into PostGIS databases. I used this tool to import my GML data, invoking it using something like this:
ogr2ogr -f "PostgreSQL" "PG:dbname=postgis user=postgres password=postgres host=localhost port=5432" roads.gml
I could check on the data import and also tweak the table and column names with PostreSQL's pgAdmin III utility.
PostGIS to dynamically served GML and then to Google Maps API
Right, the road data is now in my PostGIS database. The next step is to create a servlet that will serve up only the relevant portions of the road data for Google Maps to render. This is a very common requirement of systems like PostGIS and since I am relatively unfamiliar with GIS jargon it took me a little while to pinpoint how exactly to do it. Apparently what I wanted to do was perform a "frame-based" query, the chapter 4 of the PostGIS documentation helpfully provides an example of how to do this (if I'd only known it was called this sooner!). Generally there is a lot of GIS jargon that is quite academic, mathematical and disconcerting for the uninitiated (e.g. convex hulls) but if you keep looking long enough you'll eventually find what you need!
As you'd expect the construction of my servlet was actually conducted in parallel with the creation of my Google Maps API HTML and JavaScript. I started my servlet with the examples of using Java clients with PostGIS. For the Google Maps part I modified the Event Listeners example from the Google Maps API Documentation to pass the bounding box co-ordinates of the current browser view.
After a little experimentation with other approaches (including using JSON) I chose to output GML format XML from my servlet. It just seems more straightforward to me to do it this way. I also noticed that IE seems to be particularly choosy about the name of the servlet (it seems to need to have the .xml extension and output the text/xml content type).
Google Maps API rendering part of the road network in Firefox, the JavaScript processes the servlet generated GML
Quantum GIS rendering of the same part of the road network as shown above, again using the same servlet generated GML
Static Demo
Here is a static version of of my Google Maps powered road renderer. I don't want to serve up the complete servlet backed application from my blog server without doing some further optimisation.
Resources
GML generating servlet source code: GoogleMapsServlet.java
Google Maps HTML/JavaScript file: RoadsRender.html [Note: this is not a live example as the background servlet is not running].
A copy of Amit's original KML file: Amit-Uttaranchal.kml [zipped 858 KB, unzipped 3.62 MB]
GML road network file derived from the above KML file: roads.gml [zipped 983 KB, unzipped 4.32 MB]
XML Schema for the above GML file: roads.xsd