Overview

Adding a custom image renderer to JavaFX 8

6 Comments

Out of the box, JavaFX 8 supports JPEG, PNG, GIF and BMP images, which should be sufficient for most use-cases. By additionally using the JavaFX WebView to display images, you can even extend the list of supported image formats for example by SVG. This might however be insufficient, since some JavaFX components require an Image object, whereas you usually cannot use WebViews when defining images using CSS. If you want to use SVG as a button graphic or as a background image using CSS, you thus need to teach JavaFX how to create Image objects from SVG files.

In this blog post, I describe how to add a custom image renderer to JavaFX 8 for SVG. With the resulting code, you can use SVG images anywhere in your project just like any already supported image format. For the sake of brevity, I focus on the most interesting code sections. Additionally, I created a complete example on GitHub that you can directly use in your own project.

JavaFX manages all supported image formats within the ImageStorage class. Adding a new format is supported by adding a respective ImageLoaderFactory using the following static method:

public static void addImageLoaderFactory(ImageLoaderFactory factory);

Unfortunately, this method is not part of the official JavaFX public API, which may result in an discouraged access warning when using it. The ImageLoaderFactory that needs to be provided has two main responsibilities, i.e. describing the supported image file format and converting the raw image data into a JavaFX intermediate representation. The former is done using a ImageFormatDescription class and the latter requires an implementation of the ImageLoader interface.

In order to determine whether a particular factory can be used to create images from a given input stream, the ImageFormatDescription is used to compare the the first few bytes of an image file with a set of signatures. It is interesting to note that JavaFX only uses magic bytes to determine the image format and does not care about the actual file ending. Since the image format description was designed to match binary files, the used signatures consist of a static byte sequence. For SVG, we can use these two signatures:

"<svg".getBytes()
"<?xml".getBytes()

However, this signature matching is rather inflexible and not well suited to match text based image files like SVG. For example, it does not allow for matching files starting with whitespaces or comments. Unfortunately, subclassing the utilized Signature class is not permitted by JavaFX, so we cannot easily alter the signature matching behavior. As a result, we’ll leave it at that for now as it is probably easier to just trim the SVG images files than to hook into the signature matching routines.

Now, as JavaFX knows that it should use the added custom ImageLoaderFactory for files starting with the provided signatures, we implement the actual ImageLoader. This class is instantiated with an input stream of the image file and provides a means to transforms this stream into an ImageFrame object. The core function that needs to be implemented has the following signature:

public ImageFrame load(int imageIndex, int width, int height, 
    boolean preserveAspectRatio, boolean smooth) throws IOException;

The imageIndex parameter is used to identify a frame number in animated images. Since there is no method to determine the total amount of frames, JavaFX calls this method multiple times with increasing indexes until the method returns null. For static images like SVG, an ImageFrame should only be returned for imageIndex == 0. Width and height may be zero if no image dimensions are explicitly defined within the JavaFX application. In this case, the image should be loaded using its actual size. The final smooth parameter indicates whether or not a smooth downscaling algorithm should be used.

As the ImageFrame object that needs to be returned requires a pixel-based representation of the SVG image, the SVG image first needs to be transcoded. For the actual rendering, we can use Apache Batik, which provides a simple means to transcode SVG images into BufferedImage objects by using the BufferedImageTranscoder. To be able to subsequently transform the BufferedImage object into a JavaFX image, we use a BufferedImage of type BufferedImage.TYPE_INT_ARGB. By calling

int[] rgb = bufferedImage.getRGB(0, 0, bufferedImage.getWidth(),
bufferedImage.getHeight(), null, 0, bufferedImage.getWidth());

we get an array containing the entire pixel data of the transcoded image. This one-dimensional array consists of a concatenation of all lines within the image, each with a length of bufferedImage.getWidth(). This amount of array elements used for a single line is also referred to as scanline stride. Each integer value represents one pixel, in which the first byte indicates the alpha value followed by three bytes for red, green and blue as depicted below.

+------+------+------+
| ARGB | ARGB | .... |
+------+------+------+

The pixel representation in ImageFrame is slightly different, because it consists of a byte array with a variable amount of bytes per pixel. Since we are going to use the image type ImageStorage.ImageType.RGBA, each pixel is represented by 4 consecutive bytes. In contrast to the BufferedImage representation, each pixel starts with three bytes for RGB followed by one byte for alpha as depicted below.

+---+---+---+---+---+---+---+---+---+
| R | G | B | A | R | G | B | A |...|
+---+---+---+---+---+---+---+---+---+

After transforming the integer array of BufferedImage to the required byte array, we can construct the final ImageFrame object as shown in the following snippet. The getStride method is used to determine the scanline stride of the transcoded image which is equal to four bytes per pixel multiplied by the width of the image.

new ImageFrame(ImageStorage.ImageType.RGBA, imageData, bufferedImage.getWidth(),
    bufferedImage.getHeight(), getStride(bufferedImage), null, null);

And that is it! Whenever an image is created, JavaFX iterats the image factories available to find a suitable image loader. This image loader then creates the aforementioned ImageFrame object which is subsequently converted to a JavaFX Image object. This process is the same for explicitly creating Image objects from code or when specifying images from FXML or CSS. As a result, the newly added image format can be used in exactly the same way as natively supported image formats.

I hope this article provides you with the basic information required to create your own image loader for JavaFX. Feel free to contact me directly if you have any further questions or leave a comment right below this post.

Kommentare

  • 16. March 2015 von Oliver Krylow

    Thanks for the excellent insights into JavaFX images.

    I wonder about your implementation though, will the loaded image be twice as big as necessary in memory, because of the BufferedImage?

    And if so, how could we improve it?

  • 16. March 2015 von Jan Gassen

    I guess you’re right. The biggest problem of the current implementation is that the pixel array of the BufferedImage needs to be transformed to an RGBA byte array resulting in a (temporary) twice as big memory usage.

    One optimization could probably be to use a BufferedImage with type TYPE_4BYTE_ABGR. It should then be possible to reuse the byte array by just reordering the bytes in place instead of creating a new byte array for the ImageFrame object.

  • Well there are classes to handle the conversion.. and return an Image class (minor mods) all from the fx stack..

    I would suggest looking thru the glass, com.sun.javafx.tk, and quantum pkgs..

    Pixels class is a good start with glass/../Application.getApplication ().createRobot ()… etc.. can even do full screenshots not just sceneshots..

    • 21. March 2015 von Jan Gassen

      Hi jdub! I know that there are a few methods that can do the entire transformation from BufferedImage to Image. However, the problem with that is that we need an ImageFrame object. You can of course use a PixelReader to read the pixel data from the Image object but that would still require a manual conversion.

      Unfortunately, the ImageFrame object only supports three RGB types, namely RGB, RGBA and RGBA_PRE whereas the PixelReader supports ARGB and BGRA. Thus, I thought it would be easier to directly use the ARGB pixel data from BufferedImage.

      I also haven’t found any method allowing for the direct conversion between BufferedImage and ImageFrame but maybe I’ve missed that. If you have anything in particular in mind, please let me know!

  • 22. March 2015 von jdub1581

    Well, take a look at ../prism/Image class .. It has a ton of conversions within..
    Including ImageFrame conversions..

    This is how I did Screenshots:

    public static Image fullScreenCapture(Screen screen) {
    return pixelsToImage(getRobot().getScreenCapture(
    (int) screen.getBounds().getMinX(),
    (int) screen.getBounds().getMinY(),
    (int) screen.getBounds().getWidth(),
    (int) screen.getBounds().getHeight(),
    true)
    );
    }

    public static Image pixelsToImage(Pixels pix) {
    Buffer pixbuf = pix.getPixels();
    if (pix.getBytesPerComponent() == 1) {
    ByteBuffer buf = ByteBuffer.allocateDirect(pixbuf.capacity());
    buf.put((ByteBuffer) pixbuf);
    buf.rewind();
    WritableImage byteBgra = new WritableImage(pix.getWidth(), pix.getHeight());
    byteBgra.getPixelWriter().setPixels(
    0, 0,
    pix.getWidth(), pix.getHeight(),
    PixelFormat.getByteBgraPreInstance(),
    buf.array(), 0,
    (pix.getWidth() * 4)
    );
    return byteBgra;
    }
    if (pix.getBytesPerComponent() == 4) {
    IntBuffer buf = IntBuffer.allocate(pixbuf.capacity());
    buf.put((IntBuffer) pixbuf);
    buf.rewind();
    WritableImage intArgb = new WritableImage(pix.getWidth(), pix.getHeight());
    intArgb.getPixelWriter().setPixels(
    0, 0,
    pix.getWidth(), pix.getHeight(),
    PixelFormat.getIntArgbPreInstance(),
    buf.array(), 0,
    (pix.getWidth() * 4)
    );
    return intArgb;
    }
    throw new IllegalArgumentException(“unhandled pixel buffer: ” + pixbuf.getClass().getName());
    }

    public static final Robot getRobot() {
    return com.sun.glass.ui.Application.GetApplication().createRobot();
    }

    I had to manually copy the pixelsToImage method as it was in a private class..

    • 31. March 2015 von Jan Gassen

      I looked through the classes, and com.sun.javafx.image.impl contains a variety of converters for different pixel data formats.

      It is astonishing, but even there is no direct conversion from one of the BufferedImage formats to BYTE_RGBA. Since the current conversion is really simple, I rather leave it at that than converting everything to one or more intermediate formats.

Comment

Your email address will not be published. Required fields are marked *