Introduction to OpenCV with Python part II
This is a second part tutorial introducing some basics of OpenCV with problems proposed by my digital image processing engineering class.
Image histograms
Histogram is a representative way of showing numerical data based on its distribution through a range.
For images, a histogram represents how the pixels are distributed by its colors, giving a probabilistic idea of how frequent some color are in the image. Histograms can be used in automatic image segmentation, movement detection and granulometry, for example.
In openCV with python we have a bult-in function to do the job for us:
cv2.calcHist([image], channels, mask, histSize, ranges[, hist[, accumulate]])
- Image: Source image of type uint8 or float32. It is expected to be in square brackets.
- Channels: Also given in square brackets, it’s the index of channel for which we calculate histogram. For example, if the input image is gray-scaled, its value is [0]. For color image, you can pass [0],[1] or [2] to calculate histogram of blue,green and red channel respectively.
- Mask: To find a histogram of the full image we pass “None”, but if you want to calculate an histogram for a particular region its needed a mask image representing the region.
- HistSize: Here we pass the bins. Bins basically stands for the blocks to represent the data plotted. Also given in square brackets, for full scale we pass [256].
- Ranges: Range of the plot. Normally [0,256].
For this tutorial we are going to be equalizing the picture’s (gray-scaled) histogram. By that I mean to redistributes the image’s pixels intensity distributions in order to obtain a more stretched histogram of the picture. In summary it improves the image contrast.
In openCV we have a function also to make the equalization of an image, called equalizeHist, which has only the image to be equalized as parameter. So now, by calling the two functions on an image we can calculate the histogram for the original picture and then for the equalized one.
Proceeding into the code, we may want to see how the histograms look like for both pictures. The histogram calculation made by the function calcHist() does not return an image, instead it returns plottable data. For plotting the histograms we use the matplotlib.
As a result we see how the pixels get more spread out through the range.
Motion Detector with Histograms
Let’s suppose now that we have a continuous capture of a scenario and we keep calculating the histogram for the frames. If the histogram changes it means something new showed up, if the new histogram is way different we can make assumptions or set an alarm.
Although we are going to be using the last program as a support, for this problem it’s going to be introduced the webcam function to capture images continuously. Likewise all functions used until now, openCV has a built-in class for this called VideoCapture. The object we create then can read frames from a camera, usually for tests the webcam.
To keep capturing the frames from the camera we are going loop over the camera read and consequently calculate its histograms. This is not an scalable way to do it, since we are processing too much frames, we could improve it to process one frame in four for example. Anyways, to make the problem easier it’s done in gray-scaled, converting the image from the camera using the openCV function cvtColor.
To compare the histograms was used the Chi-square distance, one of the distance measures that can be used as a measure of dissimilarity between two histograms. You don’t need to perfectly understand this concept, we are just worried in realizing changes in a determined scene, for this metric the less the result, the better the match of the two histograms (For a numerical example check the openCV explanation).
The numerical difference adopted was 2000, based on the openCV example that is not a drastic change but something clearly changed. So, if we get something bigger than 2000 on the difference, act!!! To get a visible change, was implemented a border change to all white. While the difference keep on bigger than the maximum determined, we have images with white borders shown.
It’s needed to release the camera object after done capturing, so the camera closes properly and won’t crash the execution.
Digital Convolution and Filtering
Let’s introduce now an important concept for when processing digital images: digital convolution. Digital convolution can be defined as
being I(x, y) and K(x, y) the image and the kernel respectively. The kernel is a matrix usually composed of odd symmetrical dimensions (3x3, 5x5) and integer values. The values of the matrix will determine how the image will look like after the constitutional operation on the image.
The image above describes well how the convolution works, the kernel chosen will be placed upon each pixel of the image and a summation of products between the matrices will become a single pixel value on the new image. The mask then keeps moving and calculating the new pixels until we have filled a whole new picture.
This process of convolution can be understood as filtering, with the kernels being filters. There are many existing established filters to different goals, such as borders detection, noise reduction, blur, etc.
Now, we are going to implement a python code to filter an image given some options to the user.
The filters are defined as shown below besides the “gaussian laplacian”, which is going to be implemented by applying a filter on a filtered image as we will show further ahead. Yet, the values need the be defined as the type float due to the fact that the operations will face floating point numbers.
Some masks need to do some standard operations over the filters, while others don’t. Here we are going to show the mean mask being generated from an scale add between the mean filter multiplied by 1/9 and summed with an all zero matrix. The perks of each mask generation won’t be treated on this tutorial, only it’s usage.
Now that we have a mask let’s start manipulating images. We are going to define a bool variable for the absolute filter, present the menu to the user and load a gray-scaled image for tests. The image has to be converted into a same type as the mask for the operation be possible.
Inside an infinite loop we can iterate over the picture loaded and choose which mask we will pass over the image. To get started we have already calculated the mean mask and set the absolute to True, so let’s start applying that and see the result.
The openCV filter function filter2D is the responsible to passing the mask itself over the image. Setting the source image, the depth (-1 same as the source, usually the desired one), the mask and anchor [default value (-1,-1) means that the anchor is at the kernel center] is enough to get a new filtered image.
Transforming the matrix values from float to int to generate an image, we have as result
Now we have the option treatment for the user. The waitKey function now does not expect any key anymore, how it has been so far on the tutorials, it compares the key pressed to some result. Then, for each key we calculate a new mask to apply on the filter2D function. Using Lenna’s picture as usual, we find some interesting results.
Now to implement a combination of filters, called gaussian laplacian we first apply the gauss filter through the filter2D and call the variable mask to have the laplacian filter, so the next filter2D will apply it. This way we make sure the laplacian was done after the gauss.
This examples used the Lenna picture but could have been done through the webcam how it was mentioned before. Here a quick adaptation to how it would be to make it more interesting applying the masks in real time.
We would not need to call the Lenna picture before the loop only the camera object. Inside the loop we need then to read the frames and transform it to gray-scaled in float32 format. It’s done!! Just don’t forget to release the camera.
That’s it, on this part I showed some work with image filtering and hopefully was convincing about its importance for image processing. :)
Thank you for reading, to read more go:
1. Introduction to OpenCV with Python
3. Lighting exposure improvement using Fourier Transform with OpenCV
All the codes can be found at my public repository at GitHub.