|
The Java 2D API offers several powerful facilities to perform image processing. Using the built-in BufferedImageOp classes, you can write your own custom filters very quickly
October 19, 2007
URL:http://drdobbs.com/jvm/custom-image-processing-in-java/202404964
Chet and Romain are authors of Filthy Rich Clients: Developing Animated and Graphical Effects for Desktop Java Applications, on which this article is based. Published by Addison-Wesley Professional, ISBN 0132413930; Copyright 2008 Sun Microsystems, Inc.
Image-processing tools such as Adobe Photoshop and The GIMP offer a wide variety of filters you can apply on your pictures to create various special effects. When you are designing a user interface, it is very tempting to use those effects. For instance, you could use a filter to blur an out-of- focus element in the UI. You could also increase the brightness of an image as the user moves the mouse over a component.
Despite the impressive-looking results, image processing is not a difficult task to implement. Processing an image, or applying a filter, is just a matter of calculating a new color for each pixel of a source image. The information required to compute the new pixels varies greatly from one filter to another. Some filters, a grayscale filter for instance, need only the current color of a pixel; other filters, such as a sharpening filter, may also need the color of the surrounding pixels; still other filters, such as a rotation filter, may need additional parameters.
Since the introduction of Java 2D in J2SE 1.2, Java programmers have access to a straightforward image-processing model. You might have learned or read about the old producer-consumer model of Java 1.1. If you did, forget everything you know about it because the new model is much easier and more versatile. Java 2D’s image-processing model revolves around the java.awt.image.BufferedImage class and the java.awt.image.BufferedImageOp interface.
A BufferedImageOp implementation takes a BufferedImage as input, called the source, and outputs another BufferedImage, called the destination, which is altered according to specific rules. Figure 1 shows how a blur filter produces the final image.
While the JDK does not offer concrete image filters, it does provide the foundations for you to create your own. If you need a sharpening or blurring filter, for example, you must know how to provide parameters to a ConvolveOp filter. We teach you such techniques in Filthy Rich Clients. Before we delve further into image processing theory, let’s see how we can use a BufferedImageOp to process an image.
Filtering a BufferedImage can be done onscreen at painting time or offscreen.
In both cases, you need a source image and an operation, an instance of BufferedImageOp. Processing the image at painting time is the easiest approach; here is how you might do it:
// createImageOp returns a useful image filter
BufferedImageOp op = createImageOp();
// loadSourceImage returns a valid image
BufferedImage sourceImage = loadSourceImage();
@Override
protected void paintComponent(Graphics g) {
Graphics2D g2 = (Graphics2D) g;
// Filter the image with a BufferedImageOp, then draw it
g2.drawImage(sourceImage, op, 0, 0);
}
You can filter an image at painting time by invoking the drawImage(BufferedImage, BufferedImageOp, int, int) method in Graphics2D that filters the source image and draws it at the specified location.
Here is an example of how to preprocess an image by doing all the operations offscreen:
BufferedImageOp op = createImageOp(); BufferedImage sourceImage = loadSourceImage(); BufferedImage destination; destination = op.filter(sourceImage, null);
Calling the filter() method on a BufferedImageOp triggers the processing of the source image and the generation of the destination image. The second parameter, set to null here, is actually the destination image, which, when set to null, tells the filter() method to create a new image of the appropriate size.
You can, instead, pass a non-null BufferedImage object as this parameter to avoid creating a new one on each invocation. Doing so can save performance by reducing costly image creations.
The following code example shows how you can optimize a routine applying the same filter on several images of the same size:
BufferedImageOp op = createImageOp();
BufferedImage[] sourceImagesArray = loadImages();
BufferedImage destination = null;
for (BufferedImage sourceImage : sourceImagesArray) {
// on the first pass, destination is null
// so we need to retrieve the reference to
// the newly created BufferedImage
destination = op.filter(sourceImage, destination);
saveImage(destination);
}
After the first pass in the loop, the destination will be non-null and filter() will not create a new BufferedImage when invoked. By doing so, we also make sure that the destination is in a format optimized for the filter, as it is created by the filter itself.
Processing an image with Java 2D is an easy task. No matter which method you choose, you will need to write only one line of code. But we haven’t seen any concrete BufferedImageOp yet and have just used an imaginary createImageOp() method that was supposedly returning a useful filter. As of Java SE 6, the JDK contains five implementations of BufferedImageOp we can rely on to write our own filters: AffineTransformOp, ColorConvertOp, ConvolveOp, LookupOp, and RescaleOp.
You can also write your own implementation of BufferedImageOp from scratch if the JDK does not fulfill your needs.
Creating a new filter from scratch is not a very complicated task. To prove it, we show you how to implement a color tint filter. This kind of filter can be used to mimic the effect of the colored filters photographers screw in front of their lenses. For instance, an orange color tint filter gives a sunset mood to a scene, while a blue filter cools down the tones in the picture.
You first need to create a new class that implements the BufferedImageOp interface and its five methods. To make the creation of several filters easier, we first define a new abstract class entitled AbstractFilter. As you will soon discover, all filters based on this class are nonspatial, linear color filters. That means that they will not affect the geometry of the source image and that they assume the destination image has the same size as the source image.
The complete source code of our custom BufferedImage is available online.
AbstractFilter implements all the methods from BufferedImageOp except for filter(), which actually processes the source image into the destination and hence belongs in the subclasses:
public abstract class AbstractFilter
implements BufferedImageOp {
public abstract BufferedImage filter(
BufferedImage src, BufferedImage dest);
public Rectangle2D getBounds2D(BufferedImage src) {
return new Rectangle(0, 0, src.getWidth(),
src.getHeight());
}
public BufferedImage createCompatibleDestImage(
BufferedImage src, ColorModel destCM) {
if (destCM == null) {
destCM = src.getColorModel();
}
return new BufferedImage(destCM,
destCM.createCompatibleWritableRaster(
src.getWidth(), src.getHeight()),
destCM.isAlphaPremultiplied(), null);
}
public Point2D getPoint2D(Point2D srcPt,
Point2D dstPt) {
return (Point2D) srcPt.clone();
}
public RenderingHints getRenderingHints() {
return null;
}
}
The getRenderingHints() method must return a set of RenderingHints when the image filter relies on rendering hints. Since this will probably not be the case for our custom filters, the abstract class simply returns null.
The two methods getBounds2D() and getPoint2D() are very important for spatial filters, such as AffineTransformOp. The first method, getBounds2D(), returns the bounding box of the filtered image. If your custom filter modifies the dimension of the source image, you must implement this method accordingly.
The implementation proposed here makes the assumption that the filtered image will have the same size as the source image.
The other method, getPoint2D(), returns the corresponding destination point given a location in the source image. As for getBounds2D(), AbstractFilter makes the assumption that no geometry transformation will be applied to the image, and the returned location is therefore the source location.
AbstractFilter also assumes that the only data needed to compute the pixel for (x, y) in the destination is the pixel for (x, y) in the source.
The last implemented method is createCompatibleDestImage(). Its role is to produce an image with the correct size and number of color components to contain the filtered image. The implementation shown in the previous source code creates an empty clone of the source image; it has the same size and the same color model regardless of the source image type.
The color tint filter, cleverly named ColorTintFilter, extends AbstractFilter and implements filter(), the only method left from the BufferedImageOp interface. Before we delve into the source code, we must first define the operation that the filter will perform on the source image. A color tint filter mixes every pixel from the source image with a given color. The strength of the mix is defined by a mix value. A mix value of 0 means that all of the pixels remain the same, whereas a mix value of 1 means that all of the source pixels are replaced by the tinting color. Given those two parameters, a color and a mix percentage, we can compute the color value of the destination pixels:
dstR = srcR * (1 – mixValue) + mixR * mixValue dstG = srcG * (1 – mixValue) + mixG * mixValue dstB = srcB * (1 – mixValue) + mixB * mixValue
If you tint a picture with 40 percent white, the filter will retain 60 percent (1 or 1 – mixValue) of the source pixel color values to preserve the overall luminosity of the picture.
The following source code shows the skeleton of ColorTintFilter, an immutable class.
public class ColorTintFilter extends AbstractFilter {
private final Color mixColor;
private final float mixValue;
public ColorTintFilter(Color mixColor, float mixValue) {
if (mixColor == null) {
throw new IllegalArgumentException(
"mixColor cannot be null");
}
this.mixColor = mixColor;
if (mixValue < 0.0f) {
mixValue = 0.0f;
} else if (mixValue > 1.0f) {
mixValue = 1.0f;
}
this.mixValue = mixValue;
}
public float getMixValue() {
return mixValue;
}
public Color getMixColor() {
return mixColor;
}
@Override
public BufferedImage filter(BufferedImage src,
BufferedImage dst) {
// filters src into dst
}
}
The most interesting part of this class is the implementation of the filter() method:
@Override
public BufferedImage filter(BufferedImage src,
BufferedImage dst) {
if (dst == null) {
dst = createCompatibleDestImage(src, null);
}
int width = src.getWidth();
int height = src.getHeight();
int[] pixels = new int[width * height];
GraphicsUtilities.getPixels(src, 0, 0, width,
height, pixels);
mixColor(pixels);
GraphicsUtilities.setPixels(dst, 0, 0, width,
height, pixels);
return dst;
}
The first few lines of this method create an acceptable destination image when the caller provides none. The javadoc of the BufferedImageOp interface dictates this behavior: “If the destination image is null, a BufferedImage with an appropriate ColorModel is created."
Instead of working directly on the source and destination images, the color tint filter reads all the pixels of the source image into an array of integers. The implications are threefold. First, all of the color values are stored on four ARGB 8-bit components packed as an integer. Then, the source and the destination can be the same, since all work will be performed on the array of integers. Finally, despite the increased memory usage, it is faster to perform one read and one write operation on the images rather than reading and writing pixel by pixel. Before we take a closer look at mixColor(), where the bulk of the work is done, here is the code used to read all the pixels at once into a single array of integers:
public static int[] getPixels(BufferedImage img,
int x, int y,
int w, int h,
int[] pixels) {
if (w == 0 || h == 0) {
return new int[0];
}
if (pixels == null) {
pixels = new int[w * h];
} else if (pixels.length < w * h) {
throw new IllegalArgumentException(
"pixels array must have a length >= w*h");
}
int imageType = img.getType();
if (imageType == BufferedImage.TYPE_INT_ARGB ||
imageType == BufferedImage.TYPE_INT_RGB) {
Raster raster = img.getRaster();
return (int[]) raster.getDataElements(x, y, w, h, pixels);
}
return img.getRGB(x, y, w, h, pixels, 0, w);
}
There are two different code paths, depending on the nature of the image from which the pixels are read. When the image is of type INT_ARGB or INT_RGB, we know for sure that the data elements composing the image are integers. We can therefore call Raster.getDataElements() and cast the result to an array of integers. This solution is not only fast but preserves all the optimizations of managed images performed by Java 2D.
When the image is of another type, for instance TYPE_3BYTE_BGR, as is often the case with JPEG pictures loaded from disk, the pixels are read by calling the BufferedImage.getRGB(int, int, int, int, int[], int, int) method. This invocation has two major problems. First, it needs to convert all the data elements into integers, which can take quite some time for large images. Second, it throws away all the optimizations made by Java 2D, resulting in slower painting operations, for instance. The picture is then said to be unmanaged. To learn more details about managed images, please refer to the Chapter 5, “Performance."
int[] getRGB(int startX, int startY, int w, int h, int[] rgbArray, int offset, int scansize)
This method is used to retrieve an array of pixels at once, and invoking it will punt the optimizations made by Java 2D. Consider the second variant of getRGB():
int getRGB(int x, int y)
This method is used to retrieve a single pixel and does not throw away the optimizations made by Java 2D. Be very careful about which one of these methods you decide to use.
The setPixels() method is very similar to getPixels():
public static void setPixels(BufferedImage img,
int x, int y,
int w, int h,
int[] pixels) {
if (pixels == null || w == 0 || h == 0) {
return;
} else if (pixels.length < w * h) {
throw new IllegalArgumentException(
"pixels array must have a length >= w*h");
}
int imageType = img.getType();
if (imageType == BufferedImage.TYPE_INT_ARGB ||
imageType == BufferedImage.TYPE_INT_RGB) {
WritableRaster raster = img.getRaster();
raster.setDataElements(x, y, w, h, pixels);
} else {
img.setRGB(x, y, w, h, pixels, 0, w);
}
}
Reading and writing pixels from and to images would be completely useless if we did not process them in between operations. The implementation of the color tint equations is straightforward:
private void mixColor(int[] inPixels) {
int mix_a = mixColor.getAlpha();
int mix_r = mixColor.getRed();
int mix_b = mixColor.getBlue();
int mix_g = mixColor.getGreen();
for (int i = 0; i < inPixels.length; i++) {
int argb = inPixels[i];
int a = argb & 0xFF000000;
int r = (argb >> 16) & 0xFF;
int g = (argb >> 8) & 0xFF;
int b = (argb ) & 0xFF;
r = (int) (r * (1.0f - mixValue) + mix_r * mixValue);
g = (int) (g * (1.0f - mixValue) + mix_g * mixValue);
b = (int) (b * (1.0f - mixValue) + mix_b * mixValue);
inPixels[i] = a < < 24 | r < < 16 | g < < 8 | b;
}
}
Before applying the equations, we must split the pixels into their four color components.
Some bit shifting and masking is all you need in this situation. Once each color component has been filtered, the destination pixel is computed by packing the four modified color components into a single integer. Figure 2 shows a picture tinted with 50 percent red.
The above implementation works well but can be vastly improved performancewise.
The ColorTintFilter class in the CustomImageOp project online offers a better implementation that uses a few tricks to avoid doing all of the computations in the loop.
Image filters perform a lot of operations on images, and performance can easily degrade if you do not pay attention to a few details. Whenever you write a filter assuming the source image will be of type INT_RGB or INT_ARGB, make sure the source image is actually of that type.
Usually, compatible images (images created with GraphicsConfiguration.createCompatibleImage()), which are designed to be in the same format as the screen, are stored as integers. It is often the case that the user’s display is in 32- bit format and not the older 8-, 16-, and 24-bit formats. Therefore, it is a good idea to always load your images as compatible images.
The CustomImageOp demo loads a JPEG picture, which would normally be of type 3BYTE_BGR, and turns it into a compatible image of type INT_RGB. You can look for the call to GraphicsUtilities.loadCompatibleImage() in the source code of the demo and replace it with ImageIO.read() to see the difference when moving the sliders of the user interface. As a rule of thumb, do not hesitate to use the various methods from the GraphicsUtilities class to always use compatible images.
Java 2D offers several powerful facilities to perform image processing on your pictures. The built-in BufferedImageOp implementations let you write your own custom filters very quickly. And if you need more flexibility, you can even create a new BufferedImageOp implementation from scratch.
Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.