JXL Art — A talk with Jon Sneyers
As some may be aware, there is a corner of the Internet where people have been generating intriguing pictures by entering parameters in a program (see gallery below).
I talked with Jon Sneyers, one of the main developers.
- Do you want to introduce yourself?
Jon: I am Jon Sneyers, chair of the JPEG XL adhoc group of the JPEG committee, one of the developers of libjxl, and project lead and editor of the JPEG XL standard (ISO/IEC 18181).
I work at Cloudinary where I do research related to image processing and image compression.
- Why did you choose _wb_ as your handle?
Jon: wb is just an old nickname I use on discord, it's short for wildebeest which is a synonym of gnu, which refers to GNU/Linux. Not so important 🙂
- Is wildebeest a Belgian word? You grew up in Belgium right?
Jon: I grew up and live in Belgium. My native language is Dutch (or Flemish if you like). Wildebeest is actually an English word but I think it probably originates from Afrikaans, which originates from Dutch.
- What is JPEG XL?
Jon: JPEG XL is the next-generation image codec that offers state-of-the-art compression for both lossy and lossless image compression. It can also losslessly recompress existing JPEG images.
- What is jxl-from-tree?
Jon: jxl_from_tree originated as a debug/test tool to produce handcrafted JPEG XL bitstreams directly (as opposed to using the libjxl encoder in the usual way by giving it pixels as input). These handcrafted bitstreams are very small since they consist of just a context model for the entropy coding (this context model is described by means of a MA tree, hence the name jxl_from_tree), with the actual entropy-coded residuals simply being all zeroes, which compress to zero bits.
- When the JPEG XL format was defined, did the developers predict it would be (ab)used to produce electronic art?
Jon: The MA tree context model in JXL was based on that of FLIF/FUIF, but more expressivity was added to it: while in FLIF, the predictor is fixed and context nodes only provide different contexts for the entropy coder, in JXL the predictor itself can be different in each context node, and there is also a multiplier and offset in each context node. Also JXL has many more predictors than FLIF had: predictors from lossless WebP and lossless Pik were added.
These changes were only motivated by trying to improve lossless compression. Even in FLIF, the context model was already expressive enough to represent simple cellular automata that could be used to encode e.g. a Sierpinski triangle very effectively. So we did anticipate that in JXL there would be even more potential for such ‘trickery’.
But it wasn’t until we made jxl_from_tree and started playing with it (which happened after the bitstream was already defined and finalized) that we realized exactly how much potential there was to produce really interesting images with very small JXL files.
- What is a MA tree?
Jon: MA stands for “meta-adaptive” and it refers to the fact that we are not using a fixed context model (which is the usual approach) but a context model that can adapt to the image contents itself. The context model itself is signaled in the bitstream, allowing an encoder to use a context model that works well for the image it is encoding. Context modeling itself is “adaptive” in the sense that it adapts entropy coding to local context; by allowing the context model itself to be changed too, we made something “meta”.
FLIF was the first format to use MA trees for context modeling; JPEG XL built upon that and improved it.
- Can you tell me about this picture?
Jon: This was an early “jxl art” I made that demonstrates three different variants of Sierpinski triangles (one in each RGB channel).
- What about this picture?
Jon: This “jxl art” uses quite a few more tricks. It uses the so-called Self-Correcting (Weighted) Predictor (which originates from lossless Pik) to produce cloud-like textures; this is possible since it has a complicated internal feedback loop that effectively makes it act like a kind of non-uniform pseudorandom number generator.
The image is actually generated from right to left, which is top to bottom in the unoriented image (so orientation is used here to change the direction of the “automaton evolution”). The “initialization pattern” is cropped away by using a frame that is larger than the image and positioned so the initialization is not visible.
The image also uses the YCoCg reversible color transform in order to produce nice chroma gradients to emulate a sunset (this corresponds to a simple gradient in the Co channel). The “mountains” on the horizon are obtained by taking advantage of the fact that luma sample values can go negative (“blacker than black”), allowing to create this ‘silhouette’ effect.
- Going back to JPEG XL, can my existing JPEGs be converted without loss of quality?
Jon: Yes, since the VarDCT mode of JPEG XL is a superset of what JPEG can do, existing JPEG files can be represented exactly in JPEG XL. They will still benefit from the improved entropy coding of JPEG XL, reducing the file size by about 20%.
When encoding from pixels and using the full capabilities of JPEG XL (not just the subset that is JPEG), better compression can be obtained: about 50–60% smaller than JPEG at visually equivalent quality.
Get Denis Bredelet’s stories in your inbox
Join Medium for free to get updates from this writer.
But when the original pixels are not saved and only a JPEG file is available, JPEG XL allows you to still get some compression improvement without any loss. This is different from other codecs, where you can of course re-encode an existing JPEG but it will be a lossy operation that causes generation loss, and there is no guarantee that you will even end up with a smaller file (if you set the encode quality too high, the result can end up being larger than the JPEG; if you set the quality too low, generation loss will be even more of a problem).
- Is that what sets the format apart from others?
Jon: There are several things unique to JPEG XL compared to other recent codecs. The main thing is that it was designed from the start to be an image codec, while most of the other recent codecs (e.g. WebP, HEIC, AVIF) were designed originally as video codecs. That means that JPEG XL is designed for high-fidelity image compression, while video codecs are typically aiming at a lower fidelity (video frames get displayed for only 40 milliseconds anyway, so there is little time to notice artifacts).
JPEG XL supports progressive rendering (just like JPEG, but also improved), while video-based image codecs either don’t support this at all, or only at a cost of increased file size — in video codecs it is not useful to be able to render a single frame progressively since video streams need to be buffered before playback anyway.
JPEG XL is the first codec to be state-of-the-art in both lossless and lossy compression — other modern codecs do support lossless but they typically do not replace all use cases of PNG. JPEG XL seamlessly supports HDR and wide gamut and is future proof regarding future needs of image formats (e.g. very high precision is possible, new kinds of channels besides alpha, etc). And it’s also the only codec that can fully and safely replace JPEG thanks to its capability to losslessly recompress existing JPEG files.
- Is it complex to add to software and to cameras?
Jon: Adding JPEG XL support in software is not that hard to do: you just have to integrate libjxl, which has a modern and relatively easy to use API. It has already been done for a lot of software — many image viewers, image libraries (see https://github.com/libjxl/libjxl/blob/main/doc/software_support.md for a list).
Adding JPEG XL support to cameras will take more time. We are currently investigating a potential hardware implementation that could be implemented in cameras and phones, but the timeline for that will be measured in years, not months like it is with software.
- Which is your Discord server?
Jon: Here’s the link to the JPEG XL Discord, where you can have an informal chat with many of the JXL devs, application developers who are integrating JXL support, and many image compression enthusiasts:
Image Gallery
