Background Colour Detection using OpenCV and Python
EDIT (2020/06): I’ve updated the code samples to remove some bugs and bring compatibility forward to python 3
A recent programming problem at university, required us to analyse a screenshot of an Android application and retrieve as much data from it as possible. In this post I will outline the general process that we have taken to gather background colour from a given image using the OpenCV libraries and Python.
Theory
OpenCV allows us to open an image and store it in a 3 dimensional array or matrix where the x and y axis designate the location of the pixel in the image and the z axis designates the RGB colour channel. Each RGB pixel contains an 8 bit red component, an 8 bit green component, and an 8 bit blue component. When combined they give us the colour of the pixel in question. An RGB value of (255, 255, 255) means that the intensity of each colour component in this pixel is 255, which is the maximum intensity that the 8 bit value can be in decimal. The RGB value (255,255,255) designates the colour white as our eyes would see. At the other end of the RGB spectrum when all RGB components are at intensity 0 i.e (0,0,0) the screen would render a black pixel.
Approaches
First Approach
Our goal in the first approach is to see if we could find out what colour occurred the most in the sample image. To achieve this, we begin by splitting the image into it’s three colour channels and then flattening each of those colour channels into a 1 dimensional array. We are now left with three 1D arrays (1 for each colour) which can be iterated over all at once. The results are stored as Key : Value pairs where the key is the RGB stored as a tuple and the value is the frequency. The dictionary is sorted by value and printed to give a list of colours from least frequent to most frequent and their respective frequency count.
Second Approach
The second iteration of this solution focuses more on what would happen with non-ideal real world samples, for example if the image background is a gradient instead of a solid single image. From the first approach we can observe some interesting data about different types of images to lead us in the right direction. If we take a single colour background image such as the action bar from the Android Dropbox app (figure 1) and run it through the program, we can see that an exact shade of blue occurs in around 88% of the pixels in the image. If we compare that to what we find when we analyse this Facebook action bar (figure 2), which has a gradient background we find that no colour in the Facebook action bar has more than ~3% occurrence in the image. This conclusively tells us that there’s a huge statistical difference between the two situations and allows us to continue to the next solution.
To detect the solid colour background we merely need to apply solution 1 in cases where the most frequently occurring pixel, occurs more than around 50% of the time (value is estimated based on heuristic analysis of a few samples). If we pick the most frequently occurring colour then we have our background colour in most of these cases. If we want to get meaningful data from the gradient background image we need to do things a bit differently. For the purpose of this university project we don’t need to reproduce the gradient background but can settle with an average value of the gradient colours. To do this we can take the top 10 most common colours in the image and then take the average of the 3 colour channels to get an average colour value for the background of the image. You may also notice that the original code to count the frequencies has been refined slightly to reduce the need to split the image into channels and flatten them.
Results
Solution 1
Solution 2
The colour it finds for the Dropbox background is RGB(0,125,227)
And the average gradient colour it found in the Facebook Action Bar is RGB(91,115,166)
Some people have requested a version of the second approach that actually opens a window and shows the average colour so I created this modified gist to do that.
Feel free to ask any questions or suggest corrections to this post in the comments
Originally published at danielparker.com.au on September 4, 2013.
