Friday, July 27, 2007

Using Java JAI to stitch images together

I share this code which can be used to stitch image tiles together. I do this because I found that producing this code was quite a painful process. I am sure that there are other Java developers out there with similar requirements who, like me (an occasional Paint Shop Pro user), do not want to learn all the intricacies of image processing but want something quick and easy.

Every so often I need to stitch several images together. I am not a graphics designer so I do not have access to Adobe Photoshop and even if I did I would not know how to get it to amalgamate images. So invariably whenever I need to produce a montage of images I end up trawling the web for some freeware package, preferably something I can run from the command line and almost always I end up at ImageMagick. ImageMagick is good because I can run it from the command line and therefore script it. However, if you have other procedures that precede and follow the image processing stage then making a call out to a command line tool can seem a little too much like hard work.

With Java you can pretty much turn you hand to anything so why not this?Prior to image processing I had written the code to obtain the images I need from the net (using HttpClient). So I have my graphics files and a list of their file names.

The Java Advanced Imaging API (JAI) is supposed to make image processing easy. JAI does make life easier but IMHO it is quite poorly documented for beginners like me to use and there are not enough easy examples to copy.

My situation is that I have N PNG graphic images, arranged in an X x Y grid that I want to combine into a single image. The tiles are all the same size but (I don't know the technical term for this) have optimised palettes. By optimised palette, I mean to say that the palette of colours in the image only contains colours that are used in that image. From an efficiency standpoint, optimised palettes make sense (less colours, smaller files) but this is not particularly helpful when you need to combine several images all with different colour palettes.

So I wrote a program that positions each tile (what JAI calls a translate), converts the image palette into 24 bit true colour bitmap (at least I think that is what it does!) and finally combines all the images into a single image. Sounds easy but it took me quite a while to locate all the right code!


import java.awt.image.IndexColorModel;
import java.awt.image.RenderedImage;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.Vector;
import javax.imageio.ImageIO;
import javax.media.jai.LookupTableJAI;
import javax.media.jai.RenderedOp;
import javax.media.jai.operator.FileLoadDescriptor;
import javax.media.jai.operator.LookupDescriptor;
import javax.media.jai.operator.MosaicDescriptor;
import javax.media.jai.operator.TranslateDescriptor;

public class ImageMontageMaker {

public static void main(String[] args) throws FileNotFoundException, IOException {
int columnTotal = 1, rowTotal = 3;
int index = 0, col = 0, row = 0, height = 0, width = 0;
String[] files = {"x1y1.png","x1y2.png","x1y3.png"};
Vector renderedOps = new Vector();
RenderedOp op;

while(col<columnTotal){
row=0;
while(row<rowTotal){
System.out.println(index + ":" + files[index]);
System.out.println("c:" + col + "r:" + row);
// Load image
op = FileLoadDescriptor.create(files[index], null,null,null);
if (index == 0){
width = op.getWidth();
height = op.getHeight();
} else {
// TRANSLATE
// Translate source images to correct places in the mosaic.
op = TranslateDescriptor.create(op,(float)(width * col),(float)(height * row),null,null);
}
renderedOps.add(convert(op));
row++;
index++;
}
col++;
}
RenderedOp finalImage = MosaicDescriptor.create(
(RenderedImage[]) renderedOps.toArray(new RenderedOp[renderedOps.size()]),
MosaicDescriptor.MOSAIC_TYPE_OVERLAY,null,null,null,null,null
);
ImageIO.write(finalImage, "png", new File("out.png"));
}


public static RenderedOp convert(RenderedOp image){
// If the source image is colormapped, convert it to 3-band RGB.
if(image.getColorModel() instanceof IndexColorModel) {
// Retrieve the IndexColorModel
IndexColorModel icm = (IndexColorModel)image.getColorModel();
// Cache the number of elements in each band of the colormap.
int mapSize = icm.getMapSize();
// Allocate an array for the lookup table data.
byte[][] lutData = new byte[3][mapSize];
// Load the lookup table data from the IndexColorModel.
icm.getReds(lutData[0]);
icm.getGreens(lutData[1]);
icm.getBlues(lutData[2]);
// Create the lookup table object.
LookupTableJAI lut = new LookupTableJAI(lutData);
// Replace the original image with the 3-band RGB image.
image = LookupDescriptor.create(image,lut,null);
}
return image;
}
}

If you recognise any of this code then the chances are it probably is your code, I cobbled it together from several (open) sources!

17 comments:

ismjml said...

THANK YOU! This is very helpful
Note: Comment imported. Original by Anonymous at 2007-07-31 14:53

ismjml said...

Thank you, this is extremely useful! It's hard to find a ready-made program that does the job.
Note: Comment imported. Original by Anonymous website: http://infofries.freehostia.com/ at 2008-01-30 05:11

ismjml said...

:) ... woah, and i thought i was the only person on the web with this damned IndexColorModel problem. Thanks!
Note: Comment imported. Original by Jonas at 2008-02-20 21:00

ismjml said...

how do u use the program? just copy and paste into, say, a myspace page? or do i have to add the url for every image into the program?



it is very helpful, just confusing on how to use it. thanks!
Note: Comment imported. Original by Anonymous at 2008-02-21 05:36

ismjml said...

It is a Java program (not JavaScript). You need to compile it (with javac) and run it. It currently works from files but it could work from URLs with a bit of adjustment.
Note: Comment imported. Original by markmc website: http://content.mark-mclaren.info/ at 2008-02-21 07:59

ismjml said...

Thank you very much. This example has been really helpful for me.



I'm wondering, do you have any tips for avoiding OutOfMemoryError when stitching together a whole bunch of, say, small JPEG source images with the above method?



I don't want to clutter the comments with my stack trace, but I can put together a test case if that would be helpful.



The FileLoadDescriptor seems to be not very good at managing the overall pool---at least not out of the box... ?
Note: Comment imported. Original by David Holstius at 2008-05-05 14:27

ismjml said...

I have to admit that I don't use this very often and, if I remember rightly, I had a few memory issues myself. I've just had and quick Google about and you could try increasing the maximum heap size of the JVM (-Xmx). Also, you probably need to use TileCache.setMemoryCapacity() as the default size is only 16Mbyte.





see also here


Note: Comment imported. Original by markmc website: http://content.mark-mclaren.info/ at 2008-05-06 08:20

ismjml said...

GREAT! THANKS!
Note: Comment imported. Original by Anonymous at 2008-12-17 12:53

ismjml said...

Hi,



Good Day,



i want to ask about stitching tool, is there any tool that provide SDK or DLL or component that i can use to develop a program in .Net platform (especially in Visuat Studio 2008), kindly please help me if any one of you know about stitching tool,



Thanks in advance for your help,



Thanks and Best Regards,

Michael
Note: Comment imported. Original by Michael website: http://www.azimuthip.com at 2009-01-22 09:06

ismjml said...

.NET is not my area of expertise. Apparently ImageMagick can be wrapped inside .NET. See: ImageMagick in VB.NET.



HTH



Mark
Note: Comment imported. Original by markmc website: http://content.mark-mclaren.info/ at 2009-01-24 20:40

ismjml said...

Hi,

This is a really nice example. Just wondering, how you would handle stitching 2 images, avoiding overlapping pixels between them? Cutting out overlaps to merge images making the new image seamless. Are you aware of how to do that?
Note: Comment imported. Original by Anonymous at 2009-10-13 05:08

ismjml said...

Determining which parts of an image overlap potentially sounds like it could be a fairly computationally expensive calculation. You would need to do some kind of "cross correlation". I would also expect you could use some kind of clever "trick" in order to reduce the amount of processing you would need to do.





See e.g. Re: cross correlation of images using Java JAI (Java Advanced Imaging). There is some source code there too!



Also it is not Java but this MatLab example might also give you some ideas: Registering an Image Using Normalized Cross-Correlation





HTH



Mark
Note: Comment imported. Original by markmc website: http://content.mark-mclaren.info/ at 2009-10-14 22:18

ismjml said...

Code is working fine but its takes saveral times to paint the images.


Note: Comment imported. Original by Jai at 2010-01-20 13:58

ismjml said...

Thanks for the code its working fine.

But Its take several times to print the images.I have 3 png files.
Note: Comment imported. Original by Anonymous at 2010-01-20 14:01

James said...

Thanks for putting this together. However, there is one slight bug (and it's more of a bugish type thing with JAI). You are using the FileLoadDescriptor with JAI, which implicitly opens a Stream. Thus, if you try and delete the image after using FileLoadDescriptor, there is usually an open handle on it.

So, if you care about deleting the image at some point, you need to use explicit streams. The JAI FAQ actually has some info about this:

http://java.sun.com/products/java-media/jai/forDevelopers/jaifaq.html#delete

Sir Tran said...

Greate code !
Thanks for share ! Please visit my blog about example code with OOP

prathap kumar said...

Interesting Article

Java Training in CHennai | Online Java Training