Preprocessing with Computer Vision Part VII: Morphological Operations
“Go down deep into anything and you will find mathematics.” — Dean Schlicter
Through this article, we discuss morphological operators and their usefulness towards extracting the most accurate shape of the underlying object/element, with the examples of digits represented in four different languages.
Dataset Details
The samples from four language classes are all consolidated in a single dataset with the following features :- (The displayed images are in the sequence as below)
1. English-MNIST with 60,000 training samples on the 10 digit classes (1, 2, 3, 4, 5, 6, 7, 8, 9, 0)
2. Arabic MADBase with 60,000 training samples represented as 10 classes(1: ١, 2: ٢, 3: ٣, 4: ٤, 5: ٥, 6: ٦, 7: ٧, 8: ٨, 9: ٩,0: ٠ )
3. Japanese-MNIST with 60,000 training samples classified as 11 digit classes(1:一,2: 二, 3: 三, 4: 四, 5: 五, 6: 六, 7: 七, 8: 八, 9: 九, 10: 十, 0: 零)
4. Chinese-MNIST with 15,000 training samples and having 15 unique classes(1: 零, 2: 一, 3: 三, 4: 四, 5: 五, 6: 六, 7: 七, 8: 八, 9: 九, 10: 十, 0: 零, 100: 百, 1,000: 千, 10,000: 万, 100,000,000: 億 )
Introduction to Morphology in Computer Vision
Morphological operations are a set of mathematical functions, known as non-linear filters in image processing that process the image on the basis of morphology or shape. These nonlinear filters depend on the relative positions of the pixels as the value of the pixel is changed according to the neighboring pixels. This makes them suitable for binary image processing. The changes in the images are done by a small binary filter or kernel known as a structuring element.
Structuring Elements
A structuring element is a nxn matrix consisting of ‘0’s and ‘1’s. The dimensions of the matrix determine the size of the element while the arrangement of ‘0’s and ‘1’s in the matrix determines the shape of the element. The origin of the element is generally preferred to be at the center of the matrix though it can also reside outside the element. The size and shape of the kernel depends upon the target shape we need to extract from the processing image. Structuring elements can be created with numpy libraries as well as OpenCV in python. OpenCV provides functions for different shapes as well while numpy can be used for square shaped structuring elements.
#Structuring element with numpy and opencv# Create a structuring element with numpy
se_numpy = np.ones((3,3), np.uint8)# Create a structuring element with opencv
se_cv2 = cv2.getStructuringElement(cv2.MORPH_RECT,(3,3))
Morphological Operations
Erosion and dilation are two main and opposite techniques under morphological processing. Other techniques are also derived with the combination of these techniques depending upon whether they fit or hit the image. If the value of each pixel is 1 in the original image corresponding to each 1 in the element, the element is said to fit the image while if there is at least 1 pixel matching the above condition, the element is said to hit the image.
Erosion
As the same suggests, erosion washes away the structure of the foreground image making the boundaries thinner. This is achieved with the convolution of the structuring element with the original image, If the structuring element fits the image at a pixel, then the value of pixel in the original image is set to 1 else 0.
# erode(img, kernel, iterations): returns an eroded image with the kernel size with specified number of iterations
im1_erosion = cv2.erode(img1, se_cv2, iterations=1)
Dilation
Dilation is the opposite technique that of erosion. It adds structure to the original image depending on the convolution of the structuring element with the original image. If the structuring element hits the original image at pixel p, the value is set to 1 else 0.
# dilate(img, kernel, iterations): returns an eroded image with the kernel size with specified number of iterations
im1_dilate = cv2.dilate(img1, se_cv2, iterations=1)
Opening
Opening is the technique with erosion followed by dilation. Opening is used to separate two structures in an image with a thin gap of the boundaries.
# morphologyEx(img, operator, kernel): returns resulting image depending upon the operator specified
#MORPH_OPEN: for open operation
#MORPH_CLOSE: for close operation
#MORPH_TOPHAT: for top hat operation
#MORPH_BLACKHAT: for black hat operation
#MORPH_GRADIENT: for morphological gradient#opening operation
im1_open = cv2.morphologyEx(img1, cv2.MORPH_OPEN, se_cv2)
Closing
Closing creates the opposite effect to that of opening by erosion followed by dilation. Closing operation fills the small gaps in different structures making it close.
#closing operation
im1_close = cv2.morphologyEx(img1, cv2.MORPH_CLOSE, se_cv2)
Morphological Gradient
Morphological gradient creates an outline of an image by subtracting the result of erosion from dilation.
#gradient operation
im1_gradient = cv2.morphologyEx(img1, cv2.MORPH_GRADIENT, se_cv2)
Top Hat Transformation
Top hat is the result of subtracting the opened image and original image. This is used for extracting minute details from the structures in the original image, specially enhancing the dark objects in bright background.
#top hat operation
im1_top = cv2.morphologyEx(img1, cv2.MORPH_TOPHAT, se_cv2)
Black Hat Transformation
Black hat is the result of subtracting the closed image and original image. This is used for extracting minute details from the structures in the original image, specially enhancing the bright objects in dark background.
#black hat operation
im1_black = cv2.morphologyEx(img1, cv2.MORPH_BLACKHAT, se_cv2)
The jupyter notebook for all the morphological operations can be found here.
Takeaways
We invite the users to collect, combine and augment the dataset by including digits from their country’s native language.
Get in touch!
Reach out to us at perspectivesondatascience@gmail.com for any question and we will be happy to answer!
