Friday, July 27, 2007

Using Java JAI to stitch images together

I share this code which can be used to stitch image tiles together. I do this because I found that producing this code was quite a painful process. I am sure that there are other Java developers out there with similar requirements who, like me (an occasional Paint Shop Pro user), do not want to learn all the intricacies of image processing but want something quick and easy.

Every so often I need to stitch several images together. I am not a graphics designer so I do not have access to Adobe Photoshop and even if I did I would not know how to get it to amalgamate images. So invariably whenever I need to produce a montage of images I end up trawling the web for some freeware package, preferably something I can run from the command line and almost always I end up at ImageMagick. ImageMagick is good because I can run it from the command line and therefore script it. However, if you have other procedures that precede and follow the image processing stage then making a call out to a command line tool can seem a little too much like hard work.

With Java you can pretty much turn you hand to anything so why not this?Prior to image processing I had written the code to obtain the images I need from the net (using HttpClient). So I have my graphics files and a list of their file names.

The Java Advanced Imaging API (JAI) is supposed to make image processing easy. JAI does make life easier but IMHO it is quite poorly documented for beginners like me to use and there are not enough easy examples to copy.

My situation is that I have N PNG graphic images, arranged in an X x Y grid that I want to combine into a single image. The tiles are all the same size but (I don't know the technical term for this) have optimised palettes. By optimised palette, I mean to say that the palette of colours in the image only contains colours that are used in that image. From an efficiency standpoint, optimised palettes make sense (less colours, smaller files) but this is not particularly helpful when you need to combine several images all with different colour palettes.

So I wrote a program that positions each tile (what JAI calls a translate), converts the image palette into 24 bit true colour bitmap (at least I think that is what it does!) and finally combines all the images into a single image. Sounds easy but it took me quite a while to locate all the right code!


import java.awt.image.IndexColorModel;
import java.awt.image.RenderedImage;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.Vector;
import javax.imageio.ImageIO;
import javax.media.jai.LookupTableJAI;
import javax.media.jai.RenderedOp;
import javax.media.jai.operator.FileLoadDescriptor;
import javax.media.jai.operator.LookupDescriptor;
import javax.media.jai.operator.MosaicDescriptor;
import javax.media.jai.operator.TranslateDescriptor;

public class ImageMontageMaker {

public static void main(String[] args) throws FileNotFoundException, IOException {
int columnTotal = 1, rowTotal = 3;
int index = 0, col = 0, row = 0, height = 0, width = 0;
String[] files = {"x1y1.png","x1y2.png","x1y3.png"};
Vector renderedOps = new Vector();
RenderedOp op;

while(col<columnTotal){
row=0;
while(row<rowTotal){
System.out.println(index + ":" + files[index]);
System.out.println("c:" + col + "r:" + row);
// Load image
op = FileLoadDescriptor.create(files[index], null,null,null);
if (index == 0){
width = op.getWidth();
height = op.getHeight();
} else {
// TRANSLATE
// Translate source images to correct places in the mosaic.
op = TranslateDescriptor.create(op,(float)(width * col),(float)(height * row),null,null);
}
renderedOps.add(convert(op));
row++;
index++;
}
col++;
}
RenderedOp finalImage = MosaicDescriptor.create(
(RenderedImage[]) renderedOps.toArray(new RenderedOp[renderedOps.size()]),
MosaicDescriptor.MOSAIC_TYPE_OVERLAY,null,null,null,null,null
);
ImageIO.write(finalImage, "png", new File("out.png"));
}


public static RenderedOp convert(RenderedOp image){
// If the source image is colormapped, convert it to 3-band RGB.
if(image.getColorModel() instanceof IndexColorModel) {
// Retrieve the IndexColorModel
IndexColorModel icm = (IndexColorModel)image.getColorModel();
// Cache the number of elements in each band of the colormap.
int mapSize = icm.getMapSize();
// Allocate an array for the lookup table data.
byte[][] lutData = new byte[3][mapSize];
// Load the lookup table data from the IndexColorModel.
icm.getReds(lutData[0]);
icm.getGreens(lutData[1]);
icm.getBlues(lutData[2]);
// Create the lookup table object.
LookupTableJAI lut = new LookupTableJAI(lutData);
// Replace the original image with the 3-band RGB image.
image = LookupDescriptor.create(image,lut,null);
}
return image;
}
}

If you recognise any of this code then the chances are it probably is your code, I cobbled it together from several (open) sources!

Thursday, July 19, 2007

GME, Google Calendar to hCalendar format mashup

I have created another GME mashup. As with my GeoURL/Google Maps GME mashup this does not make use of GME in what I would expect to be a typical way. Due to the buzzword friendly technologies involved I would hope this mashup would score highly on the geek appeal scale!

http://hcalendar.googlemashups.com/

Wednesday, July 18, 2007

A GME, GeoURL and Google Maps mashup

I recently started playing around with that Google Mashup Editor (GME). In my last blog entry I promised to share the experience of my first mashup, so here is my account.

Further information about Google Mashup Editor (GME)

The Katamari Framework was Google's internal codename for the Google Mashup Editor. In essence, GME is an application server extension to the Google Web Toolkit. I also recently discovered that the attractive syntax highlighting in the Web IDE is provided courtesy of the Open Source Codepress project.

The analogy between Tomcat and the GME platform continues. Where JSP pages have application and session scope, GME has two built-in data feeds (${app}, ${user}) that allow users to read and write data specific to the application or to the user.

My GME mashup: GeoURL to Google Maps

http://geourl.googlemashups.com/

The source code for this mashup is available here and here.

My application would probably be considered "very advanced" rather than an exemplar of typical GME usage. What follows are some developer notes concerning the JavaScript approaches I took with this mashup, I hope these insights are useful to somebody. There is always room for improvement but I am happy with my mashup and think it works well enough.

The GME makes it very easy to produce certain types of application, especially when interfacing with Google's own applications. I have found that GME can be used to host very generic applications, these need not make any use of Google's applications (e.g. you could use Yahoo Maps for example and you get bonus points for using MapStraction!).

My intention was to recreate my ancient GeoURL to Google Maps experiment but host nothing myself with everything necessary running on Google's server.

On GME what makes generic application development possible is that GME can act as an application proxy for RSS/Atom content and you can host JavaScript on it. So if you can acquire your data in RSS/Atom or as XML/JSON using a Web API (e.g. via on demand callback methods) then you can host an application that uses that data on GME. There are many, many Web APIs out there for you to play with (See programmable web apis).

As I had done previously, I wanted to create a streaming display of GeoURL sites as I moved around a Google map.

My initial attempt only used GME tag functionality. This mashup didn't really do what I wanted it to very well and I felt a little restricted in controlling how it behaved so dispensed with using the gm:map tag in favour of hand coding against the Google Maps API myself.

Submission throttling

Rather than attempt to access the GeoURL feed at the end of every mouse move (as is usual with Google Maps stuff), I chose to use the submission throttling technique of only attempting to fetch new feed data after every 4 seconds. This should reduce the load on the backend feed, reduce client side processing and ultimately improve the user experience.

BitPerfects Google Maps "getOverlays" extension

I have methods that add markers and methods that remove markers. Removing markers from Google Maps is a little difficult as by default there is no way to get an array of all the visible markers. I experimented with BitPerfects extension to the Google Maps API that allows better management of markers but I was still having problems because I had methods simultaneously trying to add and delete markers to my array.

Ensuring mutual exclusive array access

My problem is that sometimes I am trying to add/delete and iterate through an array simultaneously. What I needed was some sort of mechanism by which only one process is allowed to act on my array at any one time. I first experimented with a mutually exclusive (mutex for short) access mechanism that implements something called the Wallace Variation to Lamport's bakery algorithm by Bruce Wallace (basically, you take a number and wait in line). This looked just the ticket (groan!) but I figured that for my mashup I didn't need to maintain a queue of waiting methods. In my mashup it would be sufficient to allow the most recent request to take precedence and throw away any previous requests. So inspired by Bruce's mutex algorithm I implemented something that dynamically defines a function and runs it. That function could be changed by each of the various array accessing processes, the last method to redefine the function is the one that actually gets called (assuming it is not already running).

Feedback welcome

Friday, July 13, 2007

Google Mashups Editor: The Google Application Server

I have recently been playing around with the Google Mashup Editor (GME). Like all good Web2.0 products GME is in Beta and user access is limited at the moment but IMHO GME is really *really* good! Warning, I am going to be saying Google rather a lot in the next few paragraphs!

Imagine something like Tomcat with remotely hosted editable JSP/JSTL pages and a web based IDE interface and you get close to what GME is about. Everything is hosted remotely and the web based IDE editor is surprisingly versatile all considered. You can even host your code projects on Google Code which is Google's version of Sourceforge and this includes a Subversion repository. Your GME applications can be deployed as standalone web pages or as Google Gadgets for iGoogle and Google Desktop (sort of portlets for Google's personalised portals).

The available GME tag library is squarely aimed at creating Google flavoured mashups with Google's existing offerings (Google Maps, Google Calendar, Google Search, Google Base etc.).

For more generic applications development GME provides support for importing blogs/RSS/XML data (all feeds are converted into Atom format). You can access your data using XPath. You can protect your applications behind Google Authentication. You can also embed any JavaScript and CSS that you see fit. It really is an extremely powerful product.

Since it is hosted on Google's own servers, there is no need to apply for messy API keys, everything you need is already there and ready to go.

So you want an application that accesses remote feeds and creates Google Maps but you don't have your own application server? Now you can host it on GME!!!

Behind the scenes Google Mashups is hosted on a server identifying itself as "Google Fronted" and the GML pages are compiled (analogous to JSP) using the Google Web Toolkit. I don't know a great deal about the Google Web Toolkit but essentially it lets you write applications using Java and compile these into JavaScript applications. So it appears that like a JSP page is compiled into a servlet, GML pages (Oi Google! The TLA GML is already spoken for) are compiled into JavaScript. There is obviously more going in the background of this process than what is currently provided by the public version of the Google Web Toolkit. Somewhere inside there must be an application server of some sort.

In one CSS file there is a tantalising mention of something called the "Katamari Framework". I have no found an explanation for what this is yet (I have just posted a question to the GME forum asking about it).

It looks like Google's tag library could be made to work on Tomcat?It looks like there is a GWT powered application server under the hood.

Are these things and other bits and pieces going to emerge as open source into the public domain at some point? We shall have to wait and see.

I have been experimenting with GME, Google Maps API and GeoURL again! I may share the results of my experiments in a later blog entry.