Canny edge detector

Let’s implement a canny edge detector and remove borders of the input images in Python.

Let’s dive right into implementation as we have already discussed the basics of edge detection in the article. Do give a read for a better understanding of the process.

Data visualization

Input image loading and visualization

We are using Matplotlib, NumPy, OS and CV2 libraries for the visualisation.

The given function takes input a given directory and returns an array of images which were present in the directory. Here, we have loaded the array in the variable ‘plates’.

def load_data(dir_name = 'plates'):
    imgs=[]
    for root,dirs,files in os.walk(dir_name):
        for filename in files:
            if filename.endswith('.png'):
                img=cv2.imread(os.path.join(root,filename),0)
                imgs.append(img)
    return np.array(imgs)

plates = load_data()

Now we will visualize the loaded images. The auxiliary function ‘visualize()’ displays the images given as argument.

def visualize(imgs, format=None):
    plt.figure(figsize=(20, 40))
    for i, img in enumerate(imgs):
        if img.shape[0] == 3:
            img = img.transpose(1,2,0)
        plt_idx = i+1
        plt.subplot(3, 3, plt_idx)    
        plt.imshow(img, cmap=format)
    plt.show()

visualize(plates, 'gray')

The output of the ‘visualize’ function

Canny detector implementation

Canny detection algorithm

1. Noise reduction: To remove noise, the image is smoothed by Gaussian blur with the kernel of size 5 X 5 and sigma = 1.4. Since the sum of the elements in the Gaussian kernel equals 1, the kernel should be normalized before the convolution.
2. Calculating gradients: When the image I is smoothed, the derivatives Ix and Iy w.r.t x and y are calculated. They can be implemented by convolving image I with Sobel kernels Kx and Ky, respectively. Then, the magnitude G and slope theta are calculated.
3. Non-maximum suppression. For each pixel find two neighbours (in the positive and negative gradient directions, supposing that each neighbour occupies the angle of pi/4, and 0 is the direction straight to the right). If the magnitude of the current pixel is greater than the magnitudes of the neighbours, nothing changes, otherwise, the magnitude of the current pixel is set to zero.
4. Double threshold. The gradient magnitudes are compared with two specified threshold values, the first one is less than the second. The gradients that are smaller than the low threshold value are suppressed; the gradients higher than the high threshold value are marked as strong ones and the corresponding pixels are included in the final edge map. All the rest gradients are marked as weak ones and pixels corresponding to these gradients are considered in the next step.
5. Edge tracking by hysteresis. Since a weak edge pixel caused from true edges will be connected to a strong edge pixel, pixel w with weak gradient is marked as edge and included in the final edge map if and only if it is involved in the same blob (connected component) as some pixel s with strong gradient. In other words, there should be a chain of neighbour weak pixels connecting w and s(the neighbours are 8 pixels around the considered one).

Let’s define the function using this algorithm.

from  skimage.feature import canny
from skimage import dtype_limits
from scipy import ndimage
from scipy.ndimage.filters import convolve, gaussian_filter


def Canny_detector(img,highThreshold=91,lowThreshold=31):
  img=np.array(img,dtype=float) #convert to float to prevent clipping values

  #step1: Noise reduction
  img=gaussian_filter(img,sigma=1.4,truncate=3.3)

  #step2: Calculating gradients
  Kx=[[-1,0,1],[-2,0,2],[-1,0,1]]  #Sobel filters
  Ky=[[1,2,1],[0,0,0],[-1,-2,-1]]
  Ix=convolve(img,Kx)
  Iy=convolve(img,Ky)
  grad=np.hypot(Ix,Iy) #magnitude of gradient
  theta=np.arctan2(Iy,Ix) #slope theta of gradient

  thetaQ=(np.round(theta*(5.0/np.pi))+5)%5 #Quantize direction

  #step3: Non-maximum suppression
  gradS=grad.copy()
  for r in range(img.shape[0]):
    for c in range(img.shape[1]):
      #suppress pixels at the image edge
      if r==0 or r==img.shape[0]-1 or c==0 or c==img.shape[1]-1:
        gradS[r,c]=0
        continue
      tq=thetaQ[r,c] % 4

      if tq==0: #0 is E-W (horizontal)
        if grad[r,c] <= grad[r,c-1] or grad[r,c]<=grad[r,c+1]:
          gradS[r,c]=0
      if tq==1: #1 is NE-SW
        if grad[r,c] <= grad[r-1,c+1] or grad[r,c]<=grad[r+1,c-1]:
          gradS[r,c]=0
      if tq==2: #2 is N-S (vertical)
        if grad[r,c] <= grad[r-1,c] or grad[r,c]<=grad[r+1,c]:
          gradS[r,c]=0
      if tq==3: #3 is NW-SE
        if grad[r,c] <= grad[r-1,c-1] or grad[r,c]<=grad[r+1,c+1]:
          gradS[r,c]=0

    #step4: Double threshold
    strongEdges=(gradS>highThreshold)
    #strong has value 2, weak has value 1
    thresholdEdges=np.array(strongEdges,dtype=np.uint8)+ (gradS>lowThreshold)

    #step5: Edge tracking by hysterisis
    #Find weak edge pixels near strong edge pixels
    finalEdges=strongEdges.copy()
    currentPixels=[]
    for r in range(1,img.shape[0]-1):
      for c in range(1,img.shape[1]-1):
        if thresholdEdges[r,c]!=1:
          continue #Not a weak pixel
          
        #get a 3X3 patch
        localPatch=thresholdEdges[r-1:r+2,c-1:c+2]
        patchMax=localPatch.max()
        if patchMax==2:
          currentPixels.append((r,c))
          finalEdges[r,c]=1
        
    #Extend strong edges based on current pixels
    while len(currentPixels) > 0:
      newPixels=[]
      for r,c in currentPixels:
        for dr in range(-1,2):
            for dc in range(-1,2):
              if dr==0 and dc==0:
                continue
              r2=r+dr
              c2=c+dc
              if thresholdEdges[r2,c2]==1 and finalEdges[r2,c2]==0:
                #copy this weak pixel to final result
                newPixels.append((r2,c2))
                finalEdges[r2,c2]=1
      currentPixels= newPixels
        
    return finalEdges

Now let’s apply the function on the input images.

canny_imgs = []
for img in plates:
    canny_img = Canny_detector(img)
    canny_imgs.append(canny_img)
    
visualize(canny_imgs, 'gray')

The output of the ‘visualize’ function depicting canny edge transform

The borders removal

It is worth noting that there is a framing from all sides in most of the images. This framing can appreciably worsen the quality of channels alignment. Here, we find the borders on the plates using the Canny edge detector, and crop the images according to these edges.

Borders removal algorithm

The borders can be removed in the following way:

• Apply the Canny edge detector to the image (already applied).
• Find the rows and columns of the frame pixels. For example, in case of upper bound we will search for the row in the neighbourhood of the upper edge of the image (e.g. 5% of its height).
• For each row let us count the number of edge pixels (obtained using Canny detector) it contains. Having these number let us find two maximums among them. Two rows corresponding to these maximums are edge rows.
• As there are two colour changes in the frame (firstly, from light scanner background to the dark tape and then from the tape to the image), we need the second maximum that is further from the image border.
• The row corresponding to this maximum is the crop border. In order not to find two neighbouring peaks, non-maximum suppression should be implemented: the rows next to the first maximum are set to zero, and after that, the second maximum is searched for.

Let’s define the function using this algorithm.

def remove_borders(img, canny_img):   
    dx = int(img.shape[1] * 0.05) 
    return img[dx : -dx, dx : -dx]

Now let’s apply the function on canny images obtained earlier.

cropped_imgs = []
#crop borders
for i, img in enumerate(plates):
    cropped_imgs.append(remove_borders(img, canny_imgs[i]))

visualize(cropped_imgs, 'gray')

The output of ‘visualize’ function depicting border removal

Let’s compare all the images :

Input images (left), Canny edge detector transformed image (centre), Borders removed from images (right)

Thank you, for reading!

Happy learning!