Color Segmentation using OpenCV

Back in the September of 2019, one of the first few tasks I took up after starting my higher studies, was to identify co-ordinates for some underwater buoys which led me to huge new domain of Computer Sciences namely Image Processing.

One of the most prominent names in this trade is OpenCV, a python library written for the sole purpose of taking up Computer Vision challenges.

Of the several tasks like filtering, transforming, resizing and what not, segmentation is a rather important task that can be done using OpenCV which is a step towards advanced concepts like classification and detection. Segmentation is the task of dividing different objects in sections corresponding to either a same object type or color.

There are majorly 3 different types of segmentation in computer vision:-

1. Color Segmentation or Thresholding Segmentation
2. Semantic Segmentation
3. Edge Detection

As suggested in the title I am going to be talking about color segmentation particularly using OpenCV. You might ask why use OpenCV a 21 year old library when we have tools like Caffe and Keras at our disposal.

OpenCV however lagging in terms of accuracy is a much faster method as compared to the modern SOTA DL methods like Caffe and Keras.

CPU performance comparison across frameworks on Image Classification tasks

This celerity of OpenCV doesn’t stop here, one of the most famous neural network framework also somewhat lags in terms of object detection namely Object detection using YOLOv3. Where Darknet compiled with OpenMP (an application programming interface) took almost 18 times the time taken by OpenCV.

CPU performance training YOLOv3 on OpenCV vs Darknet

Color Segmentation can be used to detect bodily tumors, extracting images of wildlife from the uniform jungle or ocean backgrounds and other colorful objects from uniform background images.

Color Segmentation in Medical sciences

Examples of Color Segmentation

As you can see in these given examples although OpenCV is a faster approach but its not the most competent one.

Let us move onto the code for Color Segmentation using OpenCV:-

For our example we will be taking the following picture and try to extract just the bird from the picture.

Let us import all the required libraries and the image itself for the task:-

import cv2 as cv
import matplotlib.pyplot as plt
from PIL import Image
!wget -nv https://static.independent.co.uk/s3fs-public/thumbnails/image/2018/04/10/19/pinyon-jay-bird.jpg -O bird.png
img = Image.open('./bird.png')

Next step in the task is to pass the image through some filters which help reduce the traffic or the small discrepancies in the image, or rather blurs them out. I will be passing the image through 4 inbuilt filters, however there is not a need for that many filters in the image we have selected but its an advantage that can be used by opting OpenCV:-

blur = cv.blur(img,(5,5))
blur0=cv.medianBlur(blur,5)
blur1= cv.GaussianBlur(blur0,(5,5),0)
blur2= cv.bilateralFilter(blur1,9,75,75)

Blurred Image

You can read more about filtering here :- https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_filtering/py_filtering

Next step is to transform the image from BGR (blue green red) to HSV (hue saturation value). One might think that wouldn’t BGR give us a better description of the pixels in the image, as it turns out that B, G and R values of the pixel are correlated with the light falling on the object thus correlated with one another and fail to give an accurate description of the pixel, that’s where HSV shines by giving an accurate description of the brightness, saturation and chroma of a pixel.

hsv = cv.cvtColor(blur2, cv.COLOR_BGR2HSV)

However small this step seems it makes our life much easier when we try to find the threshold values or the range of the pixels we want to extract.

Next stage of segmentation is the most important step of Color Segmentation is Thresholding which is also from where this procedure gets its second name “Thresholding Segmentation” from.

You guessed right now we are going to decide and settle on the threshold or the range values for all the pixels we want to extract.

low_blue = np.array([55, 0, 0])
high_blue = np.array([118, 255, 255])
mask = cv.inRange(hsv, low_blue, high_blue)

Thresholding with being the most integral step of Color segmentation using OpenCV can be a rather tedious task but once you get the idea of the whereabouts of the pixel values by maybe using a color picker tool there is still the task of hit and trialing to get all the desired pixels into account and can sometimes be a challenging task.

Mask

The “mask” in the last line of the above code is basically us putting a mask over all the other pixels that do not lie in our described range of pixels.

Now lets run this last bit of code to show the image bounded by the mask.

res = cv.bitwise_and(img,img, mask= mask)

Extracted Image from Color Segmentation

Image Segmentation serves as a stepping stone to other advanced Computer Vision tasks such as Object Classification and Object Detection using concepts like Contours and Bounding Boxes which result in amazing feats that might not have seemed achievable when books like I-Robot was written.