Saturday, January 21, 2006

Combining AJAX, Canvas tag and SVG

Dave Hooper's recent article, All Aboard AJAX, HTML Canvas, and the Supertrain, on XML.com describes how to combine AJAX with the <canvas> tag. It is a very exciting article and trying out the example for yourself is a must. This is my first actual exposure to Ruby (I've known of it's existence for 5 years), now I just need to find out what all the fuss surrounding Ruby on Rails is about!

Dave also linked to another exciting example of the AJAX and <canvas> combination in his Lightweight Visual Thesaurus. Now if I were a Macromedia Flash devotee then I would probably find these developments slightly disturbing. However, surely a WYSIWYG editor for producing this kind of thing is a long way off, well perhaps it isn't that far off after all.

Firefox 1.5 included support for some shiny new features, okay I've gone on about E4X enough already but it also supports WHATWG's <canvas> tag (also used in Apple Safari) and has built-in support for SVG as a document format.

SVG and <canvas> both render graphics. I haven't looked at the Firefox source code but you would think that since SVG is rendered directly from an XML format file, I understand this is based on Cairo, and the <canvas> tag is JavaScript engine driven that their rendering mechanisms would actually have little in common. Indeed, if you look at <canvas> tag documentation and compare it directly to some examples of SVG they are quite dissimilar technologies. For example, how do you draw an elliptical arc using the <canvas> tag?

Well, programming with the <canvas> tag reminds me of the Logo programming language that they taught me at school but if I remember rightly when they showed me Logo at school they were trying to teach me mathematics! I'm far too lazy now a days as I'm used to drawing shapes in CorelDraw, PowerPoint etc. and not plotting co-ordinates by hand! I would expect that drawing any sort of complex graphic using the <canvas> tag would not be a painless experience. Granted SVG is a quite mathematical XML format but at least there are WYSIWYG graphics packages for producing SVG (Inkscape [open source vector graphics editor] looks particularly promising).

The future will include more about AJAX, <canvas> and SVG

Dave's article got me thinking that there must be some way of utilising SVG in JavaScript and perhaps using the <canvas> tag into the bargain. My first thought was to create an XSL stylesheet that could create <canvas> code from a SVG source but this seems like it would be a lot of work.

I then thought wouldn't it be great if I could load SVG files dynamically using AJAX and display them in my web pages. According to SVG in Firefox 1.5, Firefox handles SVG as entire documents or when referenced by embed/object/iframe. It cannot currently be used as source for a html/xhtml img element or for CSS properties that take an image reference.

However, I then found Antoine Quint's CanvaSVG which is described as a static SVG Tiny 1.2 implementation using the <canvas> APIs. Wow, a way to convert SVG format files into <canvas> examples and although not supporting the full SVG file format there are some very impressive examples (Tiger, Lion).

Great stuff, I just need to tweak it slightly to produce standards compliant XHTML and load the SVG from an external file using the XMLHttpRequest and I have the beginning of SVG file loading implemented using AJAX.

Click on the following images to see live examples of using CanvaSVG to load external SVG Tiny files using AJAX technology (requires Firefox 1.5).

Whilst messing with this example I noticed something a little strange. XUL programmers are probably way ahead of me on this one but I noticed that SVG seems to be a special format to Firefox 1.5. For example, using XMLHttpRequest req.responseXML.documentElement actually returns an SVGSVGElement javascript object rather than a straight forward XML Element object that I was expecting. However the full significance of this is actually unclear to me but I'm sure it is an area worthy of further exploration.

I think this technique has a lot of potential especially if you have a ready source of SVG Tiny documents to hand (For example, I believe GeoServer can produce SVG output). Combine this with other web services (Google Maps API, Yahoo web services etc) and who knows what mashup miracles you could come up with!

Friday, January 06, 2006

XML and the Dynamic Script Tag: Easy, XML Web Services for JavaScript

JSON has caught my eye recently, it has apparently been around since 2002 but with its recent association with AJAX it has come to the fore again. Also there have been a number of high profile sightings of it on the web recently.

Yahoo's Jason talks JSON...

Jason Levitt has recently published some very interesting articles concerning AJAX and JSON on XML.com. I have mild concerns about what in retrospect appears to be a very biased perspective and as a result I feel these articles suffer a little by only telling half the story.

In the first article, Fixing AJAX: XMLHttpRequest Considered Harmful, he rightly pointed out some of the shortcomings of the AJAX approach, the need for proxies to load remote XML sources and he proposes a number of solutions including something called On-Demand Javascript.

In his more recent article, JSON and the Dynamic Script Tag: Easy, XML-less Web Services for JavaScript, Jason talks about using JSON as an alternative approach to XML in AJAX style applications. As an example Jason uses Yahoo's Geocode web service. He also suggests that this is achieved with JSON and via something he now calls the dynamic script tag approach. In his XML.com profile Jason is described as a Technical Evangelist for the Yahoo Developer Network. So it is only right and proper that he should be promoting Yahoo's technical agenda.

What I personally found slightly misleading is the second article, it implied that JSON was easier than XML and I'm not convinced about that (I'm not alone: JSON not so great..., E4X and JSON). Okay, I'm being facetious but why was this published on XML.com instead of on JSON.com?! ;). Another issue I had with the second article was that JSON was being touted as part of the magic recipe underlying the dynamic script tag approach. One point that was omitted was that the JSON as a web service output format is currently pretty much only available via the Yahoo web services family (which thanks to Yahoo's recent acquisition also includes the del.icio.us API).

I'm not saying JSON is bad, just that I can't quite see the reason why it is easier than XML (but then I enjoy XSL programming so I would say that!). If you think that JSON is much easier than XML then I say go right ahead and use it. The article fails to mention that for security reasons you should really use the JSON parser in your JSON script to avoid cross scripting attack issues. Adding the JSON parser has the added disadvantage of making the JSON approach somewhat slower (one of the arguments I've seen for using JSON over XML is speed). AJAX XMLHttpRequest based methods also usually include some request exception handling (for 404 errors and such) which Yahoo's jsr_class.js does not include. The ability to use XPath is another benefit of the XML format approach, I'm not entirely sure how I would traverse a JSON object of un-predetermined structure. So I would conclude that JSON is only useful if you have a very good idea of the nature of the object you are getting returned.

To my eye, the dynamic script tag approach is a variant of the On-Demand Javascript approach. Clever though the dynamic script tag approach is, it doesn't really need to include JSON at all. Indeed the javascript JSONscriptRequest class can be reused quite easily and comfortably with other data formats (including XML, shock, horror).

Dynamic Script Tag Approach using XML

The following three examples will illustrate how you could use the the dynamic script tag approach in other ways; using the browser's built in XML parser (cross browser compatible), using a third party javascript XML parsing class library (XML for <Script>) (cross browser compatible) and finally using E4X (standard compliant browser compatible ;)). At least armed with a more complete picture you will be able to make a more informed comparison of the techniques so you can decide which is easiest for yourself.

Firstly I slightly modified the JSONscriptRequest class javascript to remove the noCacheIE related code. For these examples I am not actually implementing the callback mechanism that Yahoo uses, in fact for these purposes each file in the following example use a static file so I don't need the noCacheIE functionality code inside the library function. In fact including the noCacheIE stops the scripts working but since the Yahoo code has no request exception handling capability it is not immediately obviously why the scripts are not working unless you dig a little deeper, take my word for it, for these examples you don't need the noCacheIE stuff.

For the first two examples our "remote web service" URL is a static file containing (or click here):


getGeo(
"<ResultSet>" +
"<Result precision=\"zip\">" +
"<Latitude>37.7668</Latitude>" +
"<Longitude>-122.3959</Longitude>" +
"<Address></Address> "+
"<City>SAN FRANCISCO</City> "+
"<State>CA</State>" +
"<Zip>94107</Zip>" +
"<Country>US</Country>" +
"</Result>" +
"</ResultSet>");

Example 1 - Using the built in XML parsing capabilities of the browser (cross browser).

There are cross browser issues for this but they are no different to the issues which face users of Google Maps API. Indeed this will probably be familiar to people who have experimented with Google Maps API. The Javascript object being returned via the callback function in this example is an XML string.


<html>
<body>
<!-- Include the JSONscriptRequest class -->
<script type="text/javascript" src="jsr_class.js"> </script>
<script type="text/javascript">

// Define the callback function
function getGeo(xmlString) {

if (document.implementation.createDocument){
// Mozilla, create a new DOMParser
var parser = new DOMParser();
myDocument = parser.parseFromString(xmlString, "text/xml");
} else if (window.ActiveXObject){
// Internet Explorer, create a new XML document using ActiveX
// and use loadXML as a DOM parser.
myDocument = new ActiveXObject("Microsoft.XMLDOM")
myDocument.async="false";
myDocument.loadXML(xmlString);
}

var docRoot = myDocument.documentElement;

//get the first "Latitude" element
var latitude = docRoot.getElementsByTagName("Latitude")[0].firstChild.data;

//get the first "Logitude" element
var longitude = docRoot.getElementsByTagName("Longitude")[0].firstChild.data;

alert('Latitude = ' + latitude + ' Longitude = ' + longitude);

bObj.removeScriptTag();
}

// The web service call
var req = 'noScriptObject.js';

// Create a new request object
bObj = new JSONscriptRequest(req);
// Build the dynamic script tag
bObj.buildScriptTag();
// Add the script tag to the page
bObj.addScriptTag();

</script>
</body>
</html>

See Example 1 in action here.

With all the browser specific code going on this example is a bit yuck, but I would contend that it is hardly horrific.

Example 2 - Using a third party XML parsing javascript library [XML for <Script>] (cross browser).

Again the JavaScript object returned via the callback method is an XML string. Although this time we are using the same DOM parsing code for both browsers with no apparent browser specific code (at least on the surface of things).


<html>
<body>

<!-- Include the XML Script javascript classes -->
<script type="text/javascript" src="tinyxmlw3cdom.js"> </script>
<script type="text/javascript" src="tinyxmlsax.js"> </script>

<!-- Include the JSONscriptRequest class -->
<script type="text/javascript" src="jsr_class.js"> </script>
<script type="text/javascript">

// Define the callback function
function getGeo(xml) {

//instantiate the W3C DOM Parser
var parser = new DOMImplementation();

//load the XML into the parser and get the DOMDocument
var domDoc = parser.loadXML(xml);

//get the root node (in this case, it is ResultSet)
var docRoot = domDoc.getDocumentElement();

//get the first "Latitude" element
var latitude = docRoot.getElementsByTagName("Latitude").item(0);

//get the first "Logitude" element
var longitude = docRoot.getElementsByTagName("Longitude").item(0);

alert('Latitude = ' + latitude.getFirstChild().getNodeValue() + ' Longitude = ' + longitude.getFirstChild().getNodeValue());
bObj.removeScriptTag();
}


// The web service call
var req = 'xmlScriptObject.js';

// Create a new request object
bObj = new JSONscriptRequest(req);
// Build the dynamic script tag
bObj.buildScriptTag();
// Add the script tag to the page
bObj.addScriptTag();

</script>
</body>
</html>

See Example 2 in action here.

Example 3 - Using E4X

This example will only work in E4X supporting browsers (at the time of writing Firefox 1.5 and Mozilla 1.8). This time we pass a new type of JavaScript object in the callback method, namely the XML type introduced by E4X (see here).


getGeo(new XML(
"<ResultSet>" +
"<Result precision=\"zip\">" +
"<Latitude>37.7668</Latitude>" +
"<Longitude>-122.3959</Longitude>" +
"<City>SAN FRANCISCO</City> "+
"<State>CA</State>" +
"<Zip>94107</Zip>" +
"<Country>US</Country>" +
"</Result>" +
"</ResultSet>"));

Although this is not currently cross browser compatible, it could be argued that this approach is even simpler than the JSON approach illustrated in the Jason's JSON article.


<html>
<body>
<!-- Include the JSONscriptRequest class -->
<script type="text/javascript;e4x=1" src="jsr_class.js"> </script>
<script type="text/javascript;e4x=1">

// Define the callback function
function getGeo(ResultSet) {
alert('Latitude = ' + ResultSet.Result[0].Latitude + ' Longitude = ' + ResultSet.Result[0].Longitude);
bObj.removeScriptTag();
}


// The web service call
var req = 'e4xObject.js';
// Create a new request object
bObj = new JSONscriptRequest(req);
// Build the dynamic script tag
bObj.buildScriptTag();
// Add the script tag to the page
bObj.addScriptTag();

</script>
</body>
</html>

See Example 3 in action here.

So now that you have a more complete picture you can hopefully make a more informed decision about what is easiest method.

Microsoft Visio communicates happily with Apache Derby databases via ODBC

For those who like to cut the bulldust, the key to achieving this can be found in this developerWorks article: ODBC programming using Apache Derby. For those who are interested in my inane asshat ramblings, read on...

I've been working on an application with a database backend of late and one thing I find quite frustrating is the lack of a decent ERD / database reverse engineering diagramming tool.

Of course if I had oodles of cash I could buy something like CA's AllFusion ERwin package and I'm sure I'd be quite happy, however, I want something free or at least at negligible cost. I don't like the Eclipse IDE so I won't use their plugins. Also, I'm not sure why but I've been having difficulty installing anything that uses the install4j installer ever since I upgraded my JVM, so Visual Paradigm's community offerings and similar are not currently an option for me to try.

Some time ago I tried dia, which has some associated SQL related tools but I thought it was a little too much work for too little result. What I really wanted was Microsoft Visio compatibility. Visio is quite happy to talk to the expensive commercial databases from Microsoft, IBM or Oracle so this would be very easy if I were using one of those. However, I'm currently using an open source Java database (at least for development). Some time ago I looked into Visio and Java database interaction and it looked like I needed to buy some expensive looking ODBC-JDBC bridge software for my Visio to JDBC connections so I thought all was lost but I now find that this is not necessarily so.

I generally develop web applications for an Oracle database but I like to use Java databases in development (I admire PostgreSQL and MySQL but feel they are just too much work for development purposes). I like the idea of a Java database as I'm generally happy with Java as a portable technology and have more idea what is happening when exceptions occur etc.. The embedded Java databases are particularly useful as they mean I can transport a database inside my WAR files. Even when I run a Java database in standalone mode via a console window they are easy enough to invoke from Ant scripts.

Previously I have mostly used HSQLDB (formerly Hypersonic), mainly because this is what came with the uPortal quickstart. I have recently been experimenting with another Java database, Apache Derby (which was formerly IBM's Cloudspace and has been donated to the Apache Foundation). The people behind HSQLDB have recently produced a new database called H2 which claims limited ODBC support. This might possibly provide a solution to allow connection with Visio and other Microsoft products. Although I decided against delving too far into H2 as it is so very new that I think I would be better off looking elsewhere for my ODBC compatibility.

After my initial impressions with Apache Derby it now seems a little odd to me that IBM opted to support MySQL, especially since they already have DB2 and Cloudscape/Derby (See also Cloudscape Versus MySQL). Also there is some talk of DB2 becoming a free product so almost everything in the IBM database line could soon be free software (well "free" in some form or other). To be fair Oracle already make almost everything they release free for development purposes.

I must admit I had previously viewed Java databases with some suspicion; they somehow seemed more like toy solutions than other more serious database offerings. My mind is changing in the light of recent experiences. Serious things do happen in the Java database arena, as examples, OpenOffice have chosen HSQLDB as it's core database and Sleepycat produce a Java version of their Berkeley DB.

Since Derby is a direct descendant of IBM's Cloudscape it was once a stable mate of DB2. It turns out that due to its IBM heritage, Apache Derby, like DB2, uses the distributed relation database architecture (DRDA) communication protocol. This apparently means you can use the DB2 Runtime Client software with Apache Derby as a means to provide ODBC compatibility and "hey presto", I can therefore configure my Derby database as an ODBC source (bar a few minor technical difficulties due to my inexperience of DB2).

After a little configuration, Visio will quite happily communicate with my new ODBC source and reverse engineer my database to give me a pretty ERD. Visio obviously has no idea that it is talking to an open source Java database else I'm sure it would rise up and rebel against me!

The other implication is that I expect other Microsoft and Windows products will be equally happy to talk to my Apache Derby Database. Therefore I could create Java/JSP web applications running on Tomcat that could be using the same datasource as Access or Excel. Okay, I can already do that using a commercial database (like Oracle or SQL Server) but the difference with this approach is that it could be done by almost any Java propeller head rather than just by the more DBA minded.