Filtering and Morphological Operations
Once upon a time, my brother had to scan one of his favorite baby pictures of me for his school project. After his project was a triumph in school, I immediately requested for it to return safely to the family album. However, he completely forgot until the day I found it sitting on his table, damaged with tea marks. Sadly, my baby photo lost so many details with the damage done. How do I wish I could retract the damage? Fortunately, image processing techniques could do the job.
In this article, we will find out how we could filter, detect, clean, uncover, or preserve non-linear objects in digital images. We will be using two techniques — spatial filters and morphological operations.
Let us start with Spatial Filters. This technique creates pixels based on its neighboring pixel and is represented as matrices called kernels. It applies convolution operations on images. Below are filters that can be used in Python.
import numpy as np# Sobel Operators
# Horizontal Sobel Filter
kernel1 = np.array([[1, 2, 1],
[0, 0, 0],
[-1, -2, -1]])# Vertical Sobel Filter
kernel2 = np.array([[1, 0, -1],
[2, 0, -2],
[1, 0, -1]])# Left Diagonal Filter
kernel3 = np.array([[1, -1, -1],
[-1, 1, -1],
[-1, -1, 1]])# Right Diagonal Filter
kernel4 = np.array([[-1, -1, 1],
[-1, 1, -1],
[1, -1, -1]])# Edge Detection
kernel5 = np.array([[-1, -1, -1],
[-1, 8, -1],
[-1, -1, -1]])# Sharpen
kernel6 = np.array([[0, -1, 0],
[-1, 5, -1],
[0, -1, 0]])# Box Blur
kernel7 = (1 / 9.0) * np.array([[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.]])# Gaussian Blur
kernel8 = (1 / 16.0) * np.array([[1., 2., 1.],
[2., 4., 2.],
[1., 2., 1.]])
We can try applying the horizontal sobel and vertical sobel filters for our sample image.
import matplotlib.pyplot as plt
from scipy.signal import convolve2d# Horizontal Sobel
conv_im1 = convolve2d(lena, kernel1, 'valid')
fig, ax = plt.subplots(1,3, gridspec_kw=dict(width_ratios=(8,1,20), wspace=0.5))
kern = ax[0].imshow(kernel1, cmap='gray')
fig.colorbar(kern, cax=ax[1])
ax[2].imshow(abs(conv_im1), cmap='gray')
ax[2].set_xticks([]);
ax[2].set_yticks([]);# Vertical Sobel
conv_im1 = convolve2d(lena, kernel2, 'valid')
fig, ax = plt.subplots(1,3, gridspec_kw=dict(width_ratios=(8,1,20), wspace=0.5))
kern = ax[0].imshow(kernel2, cmap='gray')
fig.colorbar(kern, cax=ax[1])
ax[2].imshow(abs(conv_im1), cmap='gray')
ax[2].set_xticks([]);
ax[2].set_yticks([]);
One thing we can notice between the two filtered photos is the nose detail. We can see the nose bridge on the right photo while we can see the nose holes on the left photo. Moreover, the lips are more prominent in the left photo compared to the right photo. Lastly, the hat feather details differ in directions for both photos.
One thing we have to realize that some photos may need more cleaning up in terms of “correctness” or “completeness” of our objects especially when dealing with geometric objects. This is where Morphological Operations come in to enhance the image.
Morphological operations transform each pixel based on neighboring pixels that are selected through structuring element. These operations make further image processing easier.
We will use the two most basic morphological operations are erosion and dilation. We will define a circular image to demonstrate erosion and dilation.
from skimage.draw import circlecirc_image = np.zeros((25, 25))
circ_image[circle(12, 12, 10)] = 1
imshow(circ_image);
Next, we should define our structuring element before we do any operations.
selem_circ = np.array([[0,1,0],
[1,1,1],
[0,1,0]])
imshow(selem_circ, cmap='gray');
We can now apply the erosions and dilations.
# Erosion
from skimage.morphology import erosion
imshow(erosion(circ_image, selem_circ));# Dilation
from skimage.morphology import dilation
imshow(dilation(circ_image, selem_circ));
With erosion, the circular image edges are smoother and finer compared to that of the original. With dilation, the circular images increased in the area, but the diagonal edges are not that smooth compared to the left one.
Although we tried out two morphological operations, there are other operations such as opening, closing that can be also used.
# Opening
from skimage.morphology import opening
imshow(opening(circ_image, selem_circ));# Closing
from skimage.morphology import closing
imshow(closing(circ_image, selem_circ));
With the opening, the circular image does not have even diagonal edges and the area may be almost the same as the original image. With the closing, the circular image does have even and smoother diagonal edges and has a smaller area compared to the left image.
Performing morphological operations and spatial filters on much more complex images can be trickier and may be confusing, but hang in there. It is very useful especially when we need to enhance images or perform image segmentation. With my baby photo dilemma, we can definitely consider more advanced spatial filters and morphological operations to recover and preserve details.
