Tutorial: Error Level Analysis

Error Level Analysis (ELA) permits identifying areas within an image that are at different compression levels. With JPEG images, the entire picture should be at roughly the same level. If a section of the image is at a significantly different error level, then it likely indicates a digital modification.

What To Look For

ELA highlights differences in the JPEG compression rate. Regions with uniform coloring, like a solid blue sky or a white wall, will likely have a lower ELA result (darker color) than high-contrast edges. The things to look for:

Edges	Similar edges should have similar brightness in the ELA result. All high-contrast edges should look similar to each other, and all low-contrast edges should look similar. With an original photo, low-contrast edges should be almost as bright as high-contrast edges.
Textures	Similar textures should have similar coloring under ELA. Areas with more surface detail, such as a close-up of a basketball, will likely have a higher ELA result than a smooth surface.
Surfaces	Regardless of the actual color of the surface, all flat surfaces should have about the same coloring under ELA.

Look around the picture and identify the different high-contrast edges, low-contrast edges, surfaces, and textures. Compare those areas with the ELA results. If there are significant differences, then it identifies suspicious areas that may have been digitally altered.

Resaving a JPEG removes high-frequencies and results in less differences between high-contrast edges, textures, and surfaces. A very low quality JPEG will appear very dark.

Scaling a picture smaller can boost high-contrast edges, making them brighter under ELA. Similarly, saving a JPEG with an Adobe product will automatically sharpen high-contrast edges and textures, making them appear much brighter than low-texture surfaces.

Lossy & Lossless

There are many different picture file formats. Some formats are lossy, while others are lossless.

With lossy picture formats, the first time an image is saved causes a significant amount of color loss. However, loading the picture and then encoding it again in the same lossy format will result in less additional color degradation. The ELA results highlight the areas in the image that are most prone to color degradation during a resave. Edits typically stand out as being a region with a higher degradation potential compared to the rest of the image.

File Formats

There are a wide range of file formats. Some are lossy, while others are lossless. The following table identifies some of the common image formats:

In addition to lossy and lossless, there are some file formats that only support . Full color file formats support at least 3 bytes per pixel (1 byte for red, 1 byte for green, and 1 byte for blue). This results in over 16 million possible colors per pixel, and each pixel is independent. In contrast, reduced color formats must alter the picture's color range to something less than full color. For example:

Some file formats include options to encode pictures in both lossy and lossless modes. For example:

The conversion to a reduced color file format typically results in a significant alteration to the pixel values. For this reason, ELA is not applicable on severely reduced color images like GIF. However, ELA is explicitly designed for use with JPEG.

Applying ELA to Lossy Images

JPEG images use a lossy compression system. Each re-encoding (resave) of the image adds more quality loss to the image. Specifically, the JPEG algorithm operates on an 8x8 pixel grid. Each 8x8 square is compressed independently. If the image is completely unmodified, then all 8x8 squares should have similar error potentials. If the image is unmodified and resaved, then every square should degrade at approximately the same rate.

ELA saves the image at a specified JPEG quality level. This resave introduces a known amount of error across the entire image. The resaved image is then compared against the original image.

If an image is modified, then every 8x8 square that was touched by the modification should be at a higher error potential than the rest of the image. Modified areas will appear with a higher potential error level.

JPEG stores colors using the YUV color space. 'Y' is the luminance, or gray-scale intensity of the image, 'U' and 'V' are the chrominance-blue and chrominance-red color portions. For displaying, the JPEG decoder converts the image from YUV to RGB. JPEG always encodes luminance with an 8x8 grid. However, chrominance may be encoded using 8x8, 8x16, 16x8, or 16x16. The chrominance subsampling is a JPEG encoding option. Depending on the selected chrominance subsampling, each 8x8, 8x16, 16x8, 16x16 grid will be independently encoded.

When a file is converted from a lossy file format to a lossless format, resave artifacts are retained. This permits ELA to highlight alterations made to a JPEG image that was converted to a PNG.

Not all lossy file formats are compatible. For example, the lossy WebP, HEIC, and HD Photo (JPEG XR) formats use different compression algorithms than JPEG. Even if a JPEG image has been repeatedly saved, it may still result in first-time-saved artifacts if it is converted to these other file formats. This happens because the WebP, HEIC, and HD Photo artifacts are applied for the first time.

FotoForensics can apply ELA to all supported file formats. However, ELA assumes that non-JPEG files were converted from JPEG images. (This is a reasonable assumption since there are no commercially available cameras that natively capture pictures in PNG, WebP, HD Photo, etc.) As a result, all ELA tests are compared against JPEG lossy compression.

Applying ELA to Lossless Images

Lossless file formats do not alter the colors in a picture. When a lossy picture is converted to a lossless format, all of the lossy artifacts are retained. This permits identifying specific types of alterations, such as:

Evaluating ELA

With ELA, every grid that is not optimized for the quality level will show grid squares that change during a resave. For example, digital cameras do not optimize images for the specified camera quality level (high, medium, low, etc.). Original pictures from digital cameras should have a high degree of change during any resave (high ELA values). Each subsequent resave will lower the error level potential, yielding a darker ELA result. With enough resaves, the grid square will eventually reach its minimum error level, where it will not change anymore.

It is important to recognize that high frequency areas, such as edges along objects, will usually have higher ELA values than the rest of the picture. For example, the text on the books stands out because the light/dark contrast creates a high frequency edge. In general, you should compare edges with edges and surfaces with surfaces. If all surfaces except one have similar ELA values, then the outlier should be suspect.

Rainbowing

Rather than saving colors by their red, green, and blue components, JPEG separates colors into luminance and chrominance channels. The luminance is effectively the gray-scale intensitity of the image. The chrominance-red and chrominance-blue components identify the amount of coloring, independent of the full color's intensity.

With ELA and resaved images, there may be a visible separation between the luminance and chrominance channels as a blue/purple/red coloring called rainbowing. Drawing tools such as Photoshop can introduce a distinct rainbowing pattern surfaces that have near-uniform coloring.

In general, Photoshop and other Adobe products generate a large amount of rainbowing. However, rainbowing is not an exclusive artifact to Adobe products. For example, the open source GIMP program generates little rainbowing and some high-quality camera photos may also include rainbowing along uniform-colored surfaces, such as white walls or blue skies. Some drawing tools, such as Microsoft's Paint, do not generate rainbowing.

The strong presence of rainbowing only suggests that an Adobe product, like Photoshop or Lightroom, was used to save the image. It does not identify intentional modifications.

Some digital cameras can produce rainbowing. However, there is an easy way to distinguish a camera's rainbowing from Photoshop. With a digital camera, the rainbowing is not restricted to the JPEG grid. The edges of a camera's rainbowing area will appear to have smooth contours. With Photoshop and other graphics applications, rainbowing is stictly limited to the JPEG grid. If the edges of the rainbowing area appear blocky in 8x8 or 16x16 chunks, then the rainbowing is likely caused by a graphics program such as Photoshop.

Improving Results

The results from ELA are directly dependent on the image quality. You may want to know if something was added, but if the picture is a copy of a copy of a copy, then ELA may only permit detecting the resaves. Try to find the best quality version of the picture.

For example, many pictures are hosted at Flickr. Flickr provides small, medium, large, and original images. The small, medium, and large are derivative images (resaves) created by Flickr. The "original" is whatever the user sent to Flickr, so the original will be the best quality. Similarly, pictures on news sites are usually resaved. If they have a tagline like "Source: AP Images", then go to the source and use that picture instead. News sites typically recolor, resize, and crop images before saving them at a very low quality. Go for the original source (or get as close as you can to the original source) to improve the image's quality and the ELA results.

Two easy ways to tell that the image is not an original are to look at the image size and attributions. In general, digital cameras do not generate small pictures. Pictures that are sized for the web are likely resized from other pictures, and even those may not be camera-original. Also, many web sites add their logo or URL to a corner of the picture. That means the base picture was resaved and the last modification was likely the addition of the attribution.

For best results, try to find the source picture. If you don't know where to start, then try TinEye. Many pictures on the web are resaved as they pass from user to user. TinEye doesn't know every picture on the web, but it knows many pictures. If the picture is being passed around, then TinEye can help find the source (or at least a better copy of the image). In general, the biggest image is usually the best quality. (But some sites do scale images larger...)

Advanced Uses

With training and practice, ELA users can also learn to identify image scaling, quality, cropping, and resave transformations. For example, if a non-JPEG image contains visible grid lines (1-pixel wide in 8x8 squares), then it means the picture started as a JPEG and was converted to the non-JPEG (e.g., PNG) format. If some areas of the picture lack grid lines or the grid lines shift, then it denotes a splice or drawn portion in the non-JPEG image.

As another example, PNG files are a lossless file format. If a picture is an original PNG, then ELA should produce very high values for edges and textures. However, if ELA generates weak results (dark or black coloring) along edges and textures, then the PNG was likely created from a JPEG. This is because the conversion process from JPEG to PNG is lossless and will retain JPEG artifacts.

When combined with other algorithms, ELA becomes a very powerful evaluation tool.

Caveats

While ELA is an excellent tool for helping detect modifications, there are a number of caveats:

ELA is only one algorithm. The interpretation of results may be inconclusive. It is important to validate findings with other analysis techniques and algorithms.

Sample Image: Oriental Dancer Smiling

I was looking for a public domain picture that would make a good ELA example when I came across "Oriental Dancer Smiling" by Petr Kratochvil.

According to the web site, this picture was captured with a Canon EOS 50D.

There is no mention that this picture was touched up. But what exactly was done to it? To answer that question, let's apply Error Level Analysis (ELA).

ELA

ELA shows the amount of difference that occurs during a JPEG resave. More white means more change, and black indicates no change.

A real, camera-original picture, should have a lot of white, almost like noise, over the entire picture. As the picture is repeatedly resaved (not copied, but actually loaded into a program and saved again as a JPEG), high frequencies and fine details are removed. With each resave, more frequencies/details are lost until the picture cannot get any worse (returning a black ELA picture).

In this picture, the background is completely black, but the person is not. That means that the background is at a different quality level compared to the rest of the picture; the background was digitally modified. In this case, it was enhanced to make the white look brighter.

Seeing Red

With ELA, all edges should be at about the same error level. Similar surfaces should all have similar error levels, similar coloring should have similar error levels, similar patterns, etc. But that isn't the case here. For example, her red headdress has a different intensity compared to the back of her dress, and she has one red strap that is much brighter than anything else. Even the red dangles from the hat are much brighter than the hat itself. The hat dangles appear brighter than the armband (photo lower left). Interestingly, the red bead in her hair (middle of her back) has a much lower ELA value.

Based on this, we can tell that the red in the hat was brightened, the dangles and one strap were reworked, and the original red color was probably darker -- like the red bead seen in the middle of her back.

We can also tell that her eyelashes were touched up. For example, her faint lower eyelashes cast a very strong ELA value -- stronger than the edges of her nose and lips. And the lines of her cheeks have just as much contrast as her lower eyelashes, but they have no noticeable ELA variation. This means that her eyelashes were enhanced, while her cheeks were not. (Well, not significantly.)

For bonus points, look at the bright ELA spots in the middle of her eyes. Is that a brightened specular reflection, or were her eyes originally looking in a different direction? We know that her eyes were touched up because we can observe that the sclera (whites of her eyes) contains no red veins. (All eyes have veins, and at this resolution they should be visible.) The veins were removed when the eyes were enhanced.

There may have been other changes made to this picture, but these are the things that ELA can readily identify.

End of the Rainbow

The final item that we can tell from this picture comes from the faint blue and red patches. For example, her left arm (photo left-bottom corner) has a faint blue patch. There is a red patch on her back near her right armpit (lower-right corner). In fact, her face has many small blue patches and only a few small red patches.

The red/blue patches are rainbowing. They are a tell-tail sign that an Adobe product was used on this picture. This is confirmed from the metadata: this picture was last saved with Adobe Photoshop 'Save-for-Web' (and not 'Save As') and the user selected a quality level of 60%.