A Free-to-Use Web App for Image Colour Transfer Processing
This article provides a guide to a free-to-use browser-based application that allows a photo to be colour graded to match the colour of another photo.
The screenshot above shows the Web App in operation and the photographic images demonstrate the principle of image colour transfer. In this case, the left-hand image known as the ‘target image’ shows a flowering plant. The middle image known as the ‘palette image’ defines the colour palette that should be applied to the target image. The right-hand image shows the ‘output image’ which is a recoloured version of the target image.
The processing shown in the screenshot can be reproduced as follows. Call up the Web App at www.dustfreesolutions.com/CT/CT.html. Select the ‘Flowering Plant’ option from the drop-down ‘Samples’ menu. Click the ‘Generate output image’ button.
In this case the images for processing are selected from a menu of supplied image pairs, but the user can otherwise select images of their own choice either by dragging and dropping image files onto the target image panel and palette image panel or by clicking those panels and navigating to the required image files.
Further examples of image recolouring are shown below.
All of the output images shown so far have been generated using a method known as ‘Enhanced Image Colour Transfer’ with default processing parameter settings. This is an enhancement of the ‘Match Colour’ feature often in found standalone photo editing applications. The Web App offers facilities to adjust the processing parameters. In order to understand the options for processing, it is first necessary to understand some of the basic principles of image colour transfer processing.
Some Basics of Image Colour Transfer Processing
Suppose we have a target image and a palette image and we wish the colours of the target image to better match those of the palette image. One possible approach could be as follows. We separate each of the two images into their red, green and blue colour channels and we adjust the characteristics of each target image colour channel to match the characteristics of the corresponding palette image colour channel. So, for each pixel in the target image we might shift the values in the red channel by a fixed amount so that the mean level of red in the target image matches the mean level of red in the palette image. Additionally, we might then expand or compress the distribution of the red values about the mean so that the spread of the values in the red channel of the target image matches the spread of the values in the red channel of the palette image. We could apply similar processing to the green and blue channels. The processing just described can be said to implement colour matching in the RGB (Red, Green, Blue) colour space. The processing would cause the colouration of the target image to more closely resemble the colouration of the palette image. However, the pioneers of colour match processing, Reinhard and his colleagues, determined that better colour matching can be achieved if processing is performed not in the RGB colour space but in a different colour space.
The so-called Reinhard method of colour matching, therefore, performs mean-and-spread colour matching on the three channels of a colour space known as the Ruderman colour space. This is a perceptual colour space based on the Long, Medium and Short (LMS) cone responses of human retina. The Ruderman colour space utilises one luminance channel and two chromatic channels. The former channel specifies the brightness of a particular pixel and the latter two channels specify the colouration of the pixel on a red-green axis and a yellow-blue axis. The Web App under discussion here can be reset to implement Reinhard processing by selection of the button ‘Reset to Reinhard processing parameters’. The accuracy of this implementation has been verified by comparison with other implementations on GitHub.
Some implementations on GitHub claim to be an implementation of the Reinhard method but they utilise the CIELab colour space rather than the Ruderman colour space. This is because the former colour space is better supported in software libraries. The Web App allows a choice of colour spaces. Often it makes little difference which is used and sometimes the former can produce more accurate images than the latter. But in the case of certain images, the CIElab colour space can perform badly in comparison to the Ruderman colour space. The reverse does not appear to be true. An example of a poor outcome is shown below.
Enhancements of Image Colour Transfer Processing
As previously mentioned, the Web App under discussion here by default implements a processing method known as Enhanced Image Colour Transfer to a selected image pair. This method is a refinement of the Reinhard method. One part of the refinement is known as cross correlation processing.
Consider, a palette image which is predominantly shades of purple. In such an image, we might expect to see significant correlation between the red and blue channels. A bright purple pixel would have high values in both the red and blue channels. A dull purple pixel would have low values in both the red and blue channels. If colour matching were to be applied in the RGB colour space, by applying independent adjustments to each of the colour channels of the target image, then this would not necessarily provide an output image predominant in purple, some pixels might appear red some might appear blue but there is no guarantee that the red and blue pixel values would co-ordinate to produce a mostly purple outcome. A purple outcome can only be guaranteed if processing is applied which takes account of the correlation between the red and blue channel values. Enhanced Image Colour Transfer processing applies a cross correlation adjustment in addition to the simple independent channel processing of the Reinhard method. This processing is applied in the Ruderman colour space and it takes account of any correlation between the two chromatic channels. The processing attempts to mimic in the output image any colour correlation that is present in the palette image. Reinhard processing effectively assumes zero correlation between the chromatic channels of the palette image, Enhanced Image Colour Transfer processing by default allows for positive or negative correlations of up to 50% between the chromatic channels.
A further refinement of Enhanced Image Colour Transfer is an option for ‘Extra shading’ also known as ‘shading adjustment. Shading adjustment allows for the fact that the eye has a different luminance response to different colours. A given level of green looks brighter than a given level of red which looks brighter than a given level of blue. If the colours within a pixel are changed then the perceived brightness may change even if the average level is unchanged. In the Ruderman colour space the so-called luminance channel could be unchanged but the perceived brightness could be modified because of a redistribution of energy between the two the chromatic channels. So, for example, a change in the chromatic channel values which caused a shift from blue to green would cause an increase in perceptual brightness. The processing for the shading adjustment attempts to compensate for any perceived change in luminance due to colour modification.
The figure below shows the effect of applying Reinhard processing to the ‘Flowering Plant’ image pair. It subsequently shows the outcome of additionally applying extra shading, of additionally applying correlation processing and of additionally applying both options. The additional refinements associated with Enhanced Image Colour Transfer show a clear advantage. The theory for extra shading and correlation processing is described in detail here.
Further Processing Options
The phrase ‘colour transfer’ is often used to describe the processing developed by Reinhard and his colleagues, but in practice the processing method transfers both colour and shading. To implement full ‘colour and shading transfer’ on the Enhanced Image Colour Transfer Web App the ‘PercentShadingShift’ slider must be set to 100%. To implement ‘pure colour transfer’ the slider must be set to 0%. The figure below illustrates these two options. The user can choose a setting according to needs but the default setting is 50% which implements a degree of shading that is midway between that of the target and palette images.
Slider options are additionally provided to implement a partial processing modification in which the processed image is proportionally remixed with the original target image and to implement a partly tinted image where the processed image is partially remixed with grey shade monochrome. The figure below illustrates these different possibilities.
Some slider controls allow the user to select values outside the range 0% to 100% to achieve additional interesting effects.
Other sliders provided by the Web App allow control of output image saturation, limit setting for cross correlation processing and iteration control for shape matching of output image channel distributions to palette image channel distributions. These are options for the advanced users which are described in detail here.
The Web App described here is the result of a collaboration between the author Terry Johnson, who has formulated the underlying methodology, and Michele Renzullo who has adapted the methodology for implementation as a Web App. Their respective GitHub accounts are available here and here. The Web App has been designed to primarily operate on a desktop computer, the speed and practicality of processing will be dependent upon image size and the computing power of a particular device. The output image size is determined by the size of the target image. The palette image need not be the same size as the target image. Indeed, a smaller palette image can facilitate processing.
Applications of Image Colour Transfer Processing
When image colour transfer was first developed it was a mostly seen as a technique for matching images prior to photo merging. If two images are captured in sequence under slightly different lighting conditions and then merged to form a continuous scene, then the image join might be visible when colour and shading are not exactly matched. To overcome this, one photo can be matched to the other using a fairly rudimentary image colour transfer method. Because the initial implementation of the method in early software packages was fairly crude, colour matching was seen as a technique with fairly limited application. Consequently, little effort was made to improve the processing method. The implementation here offers a more refined and flexible approach to colour match processing and so offers the chance to address a wider range of potential applications.
One novel application of image colour transfer is indirect colour shading. Here the palette image is a modified version of the target image. The figure below shows an example of this.
Another interesting technique is to recolour a photograph using a cropped version of itself in order to emphasise particular colours.
Commercial photo processing applications offer options such as ‘Solarise’, ‘Gradient Mapping’ and ‘Posterize’ which can transform images to achieve multicolour or psychedelic outcomes. Image colour transfer can also offer the possibility of colourful images but with the possibility of greater subtlety and control.
An interesting aspect of image colour transfer is that the target image content may be subject to copyright but the colours within the palette image cannot be subject to copyright. This offers a chance to use colour schemes devised by other people. The figure below shows an image that has been recoloured using a Claude Monet painting as the palette image.
A Comparison of Colour Transfer Methods
To finish this exploration of the Enhanced Image Colour Processing the processing outcomes from the Web App will be compared to the outcomes from other facilities. The following image sequences show six images in each row. The first image in each row is a target image and the second is a palette image. The third image shows the outcome when the image pair is processed by the Web App under the default settings. The fourth image shows the outcome using the Photoshop ‘Match Colour’ function under default settings. The fifth image shows the outcome using a Photoshop Neural Filter. The sixth image shows the outcome from applying ‘Adaptive Recolouring’ to the image pair under default settings. Adaptive Recolouring is a new processing method that is proprietary to the author.
Photoshop Neural Filtering is mostly applied as ‘Colour Transfer’ with the exception of the first row where it is applied as ‘Landscape Mixer’. It can be seen that the Neural Filter offers the best outcome in the first row but it does so at the expense of modifying the sky formation which may not be desirable under some circumstances. The Neural Filter options here are those provided under Photoshop Version 23.1 and are specified as beta versions.
Adaptive Recolouring appears to clearly outperform the other methods in the fourth and sixth rows. Match Colour processing appears to be the least consistent.
The most interesting aspect of the last figure is the variety of outcomes achieved. Each method has its strengths and weaknesses. Perhaps the best overall outcome might be achieved by utilising an AI supervision function to select the best processing method and processing parameters for any given image pair.
Image colour transfer is a relatively neglected area of image processing. A lack of development has given rise to a lack of interest and vice-versa. This article has shown that there is there is the potential for further development.
The images processed in this article may be accessed from here.