A Free Web App for Image Colour Matching

by T Johnson and M Renzullo

Introduction

This article introduces a free web application that allows you to adjust the colours of one photo to match those of another.

The screenshot above shows the Web App in operation and the photographic images demonstrate the principle of image colour transfer. In this case, the left-hand image known as the ‘target image’ shows a flowering plant. The middle image known as the ‘palette image’ defines the colours that should be applied to the target image. The right-hand image shows the ‘output image’ which is a re-coloured version of the target image.

To reproduce the processing shown in the screenshot, follow these steps:

1. Visit www.dustfreesolutions.com/CT/CT.html to access the Web App.
2. From the drop-down menu labeled ‘Samples,’ choose the ‘Flowering Plant’ option.
3. Click on the ‘Generate output image’ button.

In this case, the Web App provides a menu of pre-supplied image pairs for processing. However, you can also choose your own images by either dragging and dropping them onto the target image panel or the palette image panel. Alternatively, you can click on these panels and browse your computer to select the image files you want to use.

Further examples of image re-colouring are shown below.

All the output images you’ve seen so far were created using a technique called ‘Enhanced Image Colour Transfer’ with default settings. This method is an improvement over the ‘Match Colour’ feature commonly found in standalone photo editing software.

The Web App allows you to adjust processing parameters to customize your results. To fully understand these options, it’s helpful to grasp some basic principles of image colour transfer processing.

Some Basics of Image Colour Transfer Processing

Suppose we have a target image and a palette image, and we want the colours of the target image to more closely match those of the palette image. One approach is as follows: we break down each image into its red, green, and blue color components. Then, we adjust each colour channel of the target image to align with the corresponding channel of the palette image.

For example, we might shift the red channel values of each pixel in the target image by a certain amount so that the average level of red matches that of the palette image. Additionally, we could adjust the spread of red values around the average to match the spread seen in the palette image. Similar adjustments could be made for the green and blue channels.

This process is known as color matching in the RGB (Red, Green, Blue) color space. It causes the colours in the target image to more closely resemble those in the palette image. However, pioneering work by Reinhard and colleagues suggests that even better color matching can be achieved by working in a different color space than RGB.

The Reinhard method of colour matching, therefore, performs mean-and-spread colour matching on the three channels in a colour space known as the Ruderman colour space. This is a perceptual colour space based on the Long, Medium and Short (LMS) cone responses of human retina. In the Ruderman model, one channel denotes luminance (brightness), while the other two channels represent chromaticity along the red-green and yellow-blue axes.

In the context of the Web App being discussed, users can initiate Reinhard processing by selecting the ‘Reset to Reinhard processing parameters’ button. The accuracy of this implementation has been confirmed through comparisons with similar implementations available on GitHub.

Some implementations on GitHub purport to implement the Reinhard method but use the CIELab colour space rather than the Ruderman colour space. This choice is primarily driven by the better support for CIELab in software libraries.

The Web App itself offers users the flexibility to choose between different colour spaces. In many cases, the choice between Ruderman and CIELab does not significantly affect results, and sometimes CIELab can even produce slightly more agreeable images than Ruderman. However, there are instances where the CIELab colour space performs poorly compared to Ruderman, particularly with certain types of images. The opposite scenario, where Ruderman performs noticeably worse than CIELab, does not seem to happen.

An example illustrating this issue is provided below.

Enhancements of Image Colour Transfer Processing

The Web App discussed here uses a method called Enhanced Image Colour Transfer by default, which improves upon the Reinhard method for processing selected pairs of images. This method includes a refinement known as cross-correlation processing.

Imagine a palette image that is mainly shades of purple. In such an image, there would typically be a strong relationship between the red and blue color channels. For instance, a bright purple pixel would have high values in both the red and blue channels, while a darker purple pixel would have lower values in both. If we were to apply simple colour adjustments independently to each colour channel (red, green, blue) of the target image in the RGB colour space, it wouldn’t necessarily result in a predominantly purple output. Some pixels might appear more red, others more blue, with no guarantee that their combination would create a mostly purple image. To ensure a predominantly purple outcome, we need processing that considers the correlation between the red and blue channel values.

Enhanced Image Colour Transfer processing improves upon the Reinhard method by incorporating cross-correlation adjustments. These adjustments are made in the Ruderman color space and account for any correlation between the two chromatic channels. The goal is to reproduce in the output image the colour correlations present in the palette image.

Whereas, the original Reinhard method assumes no correlation between the chromatic channels of the palette image, the Enhanced Image Colour Transfer method can accommodate positive or negative correlations of up to 50%.

Another refinement in Enhanced Image Colour Transfer is the option for ‘Extra shading’, also known as ‘shading adjustment’. This feature takes into account the fact that our eyes perceive brightness differently for different colours. For instance, green appears brighter than red, which appears brighter than blue at the same intensity level.

When colours within a pixel are altered, the perceived brightness can change, even if the average pixel value does not. In the Ruderman colour space, the luminance channel might stay unchanged for a pixel, but altering the colour balance can redistribute energy, affecting how bright the pixel appears.

For example, shifting from blue to green within the chromatic channels can increase perceived brightness. Shading adjustment processing aims to compensate for any perceived change in brightness caused by colour modification.

The figure below demonstrates the impact of applying Reinhard processing to the ‘Flowering Plant’ image pair. It then illustrates the results of adding extra shading, correlation processing, and both options together. The refinements in Enhanced Image Colour Transfer provide noticeable benefits. The detailed theory behind extra shading and correlation processing is explained here.

Further Processing Options

The term ‘colour transfer’ is often used to describe the method developed by Reinhard and his colleagues, but it actually involves the transfer of both colour and shading. With the Enhanced Image Colour Transfer Web App, setting the ‘PercentShadingShift’ slider to 100% achieves ‘full colour and shading transfer’. Conversely, setting the slider to 0% achieves ‘pure colour transfer’.

The figure below illustrates these two options. Users can adjust the shading setting based on their requirements, with the default setting at 50%, which applies shading that is intermediate between the target and palette images.

Slider options are also provided to specify partial adjustments to the processed image. One option allows you to blend the processed image with the original target image in different proportions. Another option enables the creation of a partly tinted image by mixing the processed image with a greyscale version. The figure below illustrates these different possibilities.

Some slider controls allow users to select values beyond the typical range of 0% to 100%, offering additional interesting effects.

Other sliders in the Web App enable users to adjust image saturation, set limits for cross-correlation processing, and control iterations for matching the distribution of output image channels to those of a palette image. These options are designed for advanced users and are described in detail here.

The Web App described here is the result of a collaboration between the author Terry Johnson, who developed the methodology, and Michele Renzullo who adapted it for implementation as a Web App. Their respective GitHub accounts are available here and here.

The Web App has been designed to primarily operate on a desktop computer, the speed and practicality of processing will be dependent upon image size and the computing power of a particular device. The size of the output image size corresponds to that of the target image. The palette image does not need to match that of the target image. Indeed, a smaller palette image can facilitate processing.

Applications of Image Colour Transfer Processing

When image colour transfer was initially developed, it was primarily used for aligning colours between images before merging them in photo editing. For instance, if two images were taken in sequence under slightly different lighting conditions and then merged into a single scene, inconsistencies in color and shading could be noticeable at the join. To mitigate this, one image could be adjusted to match the colour and tone of the other using a basic colour transfer technique.

Early software implementations of color transfer were rudimentary, restricting their practicality and limiting interest in further development. However, the method implemented here introduces a more refined and adaptable approach and opens up new possibilities for a broader range of applications.

One novel application of image colour transfer is indirect colour shading. Here the palette image is a modified version of the target image. The figure below shows an example of this.

Another interesting technique is to re-colour a photograph using a cropped version of itself in order to emphasize particular colours.

Commercial photo processing applications offer options such as ‘Solarise’, ‘Gradient Mapping’ and ‘Posterize’ which can transform images to achieve multicolour or psychedelic outcomes. Image colour transfer can also offer the possibility of colourful images but with the possibility of greater subtlety and control.

An intriguing aspect of image color transfer is that while the content of the target image might be copyrighted, the colours extracted from a palette image cannot be subject to copyright. This allows for the use of color schemes created by others. The figure below illustrates an image that has been re-coloured using a Claude Monet painting as the palette image.

A Comparison of Colour Transfer Methods

To conclude our exploration of Enhanced Image Color Processing, we will compare the results produced by the Web App with those from other tools.

Below are image sequences, each containing six images per row:

1. The first image in each row is the target image.
2. The second image is the palette image.
3. The third image shows the outcome when the image pair is processed by the Web App using default settings.
4. The fourth image displays the result using Photoshop’s ‘Match Color’ function under default settings.
5. The fifth image presents the outcome using a Photoshop Neural Filter.
6. The sixth image showcases the result of applying ‘Adaptive Recoloring’ to the image pair using default settings. This method is a new method that is proprietary to the author.

The comparative approach aims to highlight the strengths and nuances of each method’s capability.

Photoshop Neural Filtering is mostly applied as ‘Colour Transfer’ with the exception of the first row where it is applied as ‘Landscape Mixer’. It can be seen that the Neural Filter offers the best outcome in the first row but it does so at the expense of modifying the sky formation which may not be desirable under some circumstances. The Neural Filter options here are those provided under Photoshop Version 23.1 and are specified as beta versions.

Adaptive Recolouring appears to clearly outperform the other methods in the sixth row. Match Colour processing appears to be the least consistent.

The most interesting aspect of the last figure is the variety of outcomes achieved. Each method has its strengths and weaknesses. Perhaps the best overall outcome might be achieved by employing an AI supervision function to select the best processing method and processing parameters for any given image pair.

Image colour transfer is a relatively neglected area of image processing. A lack of development has given rise to a lack of interest and vice-versa. This article has shown that there is there is the potential for further development.

The images processed in this article may be accessed from here.