Let’s play with image borders on OpenCV
This is a part four series that I’ve been doing for my graduation computer vision class.
Canny’s Algorithm
For this particular one, we are going to be working with the most used algorithm for border detection and also create pointillism based paintings. Canny’s algorithm for border detection is known as one of the most efficient and fast methods to find discontinuity in digital images, producing as the processing result a binary image, which pixels are white for borders and black for everything else.
Just as a brief introduction to the algorithm, it searches for gradient’s local maximums for the image. As my professor pointed out here, or OpenCV’s documentation, Canny’s can be summarized in a few steps:
- Convolution with the Gaussian filter (part 2), calculate the magnitude and gradient’s angle.
- Sharpening gradient’s large crests
- Points classification as the orientations horizontal/vertical as+45°/-45° (±22.5° intervals).
- Check neighbors gradients for the determined orientation.
- Suppression of non maximums: If the gradient’s magnitude M(x,y) is less than any of your neighbors, do g(x,y)=0; otherwise, do g(x,y)=M(x,y) (g(x,y) is the image with suppression).
3. Hysteresis thresholding usage for breaking contour.
- Used T1 e T2 as variables. T1>T2 (small for threshold 1 and 2).
- If the pixel value has g(x,y)≥T1, this can be seen as a strong border point.
- For the other pixels, g(x,y)≥T2, are weak border points.
4. All the weak border points’ neighbors search in its 8-neighbors if there is any strong border point. If so, it is treated like part of the frontier.
5. Canny’s parameters suggestions: H/TL=3/1TH/TL=3/1, ou TH/TL=2/1
Ok ok, since this summary won’t be implemented, openCV gives it all ready, it serves as an orientation or an idea of how things happen. If you get interested in going deeper, I suggest you to go for a book reading or a better source; now we are starting to program!!!
An example of the Canny’s detection can be seen on the documentation I linked above, however it’s a fixed border detection. For our implementation, we will have a trackbar to change the parameter T1 and see how it affects the detection.
The openCV function that does the Canny’s detection can be simply called by cv2.canny(src, output image, T1, T2, apertureSize, L2gradient). For the implementation here we will only be using the mandatory parameters, and changing only T1 (T2 assumed as 3xT1).
We want the bar to change from 0–200 and we initialize the T1 variable as 10. So, after reading Lenna’s picture in b&w we can already call the Canny’s and apply the trackbar to the result. The function setT1 is responsible to keep T1 from going lower than 10 (you will understand in the results), and applying the canny’s to the new values read.
Visualizing we can better understand better how the parameters work on the function and can generate a customized border detection, going from many borders to almost none (weakest to strongest on the picture).
Pointilism Art Application
Ok great, now for a minute we will introduce another implementation that, for this part’s general purpose, will be merged with the canny’s on the last one. The idea for this one is to paint an image the same color as the one we load in, but with dots. A reference to this idea is the work from the known artist Georges Seurat, reference in this technique.
Although the art came from creating new images with the dots, we are going to represent a known image in this effect. The solution proposed by my professor isn’t to just make a dot on every pixel position, but to have a certain randomness to where it should be placed and also skip some pixels so the frame background color can be perceived and help the illusion.
For that, first let’s create arrays representing dimensions that will help us skip some pixels, not use the original height and width of the image. The arrays for this are set with dimensions divided by 5. So we are only painting one fifth of the dimension (1 pixel for 5X5 = 25 pixels).
These arrays are then filled with increment values before having a STEP gain and a tilt to STEP/2.
Xrange and Yrange have then its sequences randomized, so when we start to paint it doesn’t follow any logical sequence. Before drawing the circle yet we apply some more randomness to the values in the arrays and with that generate the coordinate values for x and y.
The built-in function cv2.circle(output, where, radius, color, padding, type of line) is used then to create circle at the x and y that we have in the color of the image we loaded in on that specific spot. The result is printed on a same size image created previously with all white pixels.
The pointillism effect get’s really interesting because the randomness of the circle drawings together with the quantity of circles allow the background to be perceived and make allusion to hand painted on a real frame. However, the resolution isn’t the better.
Now, around that, we are going to merge both implementations and try to improve the pointillism resolution. One of the ideas purposed was to apply Canny’s border detection to the pointillism created and for each point of border create new circles; more circles leading to more resolution.
To start off, we get the same pointillism code, and apply and just call the Canny’s for the result right before applying the pointillism again in the same result but now only on the borders detected.
Since the Canny’s generates a binary result, we just need to get the pixels that are white and get the gray corresponding to that location in Lenna’s original image to then create new circles upon those pixels in the pointillism image.
The code to do that shown above should not be a challenge to be understood if it was clear to you both codes separated. The trackbar was not used here since we would only apply more and more circles to the same image and get a disrupted result.
The result then is as expected better for when we apply Canny’s with a lower value of T1, detecting more borders, so more circles are drawn; The white background gets almost eradicate. The radius was changed to 2 just to get cleaner result.
That’s it!! On this part we worked with border detection and had a little fun creating art. A cool challenge would be to adapt this code for doing the same for colorful pictures and maybe print it out and expose in a living room as a cool concept. :)
Thank you for reading, to read more go:
1. Introduction to OpenCV with Python
2. Introduction to OpenCV with Python part II
3. Lighting exposure improvement using Fourier Transform with OpenCV
All the codes can be found at my public repository at GitHub.