Crop Row Detection using Python and OpenCV

The complete automation of farming is inevitable. Farmers have been adopting each new wave of technology since the invention of the wheel. In this series we’ll learn how to take advantage of this, using the advancements in computer vision and machine learning to provide farmers with new precision agricultural tools.

Today we’ll be looking at a new use case: Autonomous Agricultural Vehicles (AAV).

Our AAV poster child.

What’s interesting about an AAV is the fact that existing Ag. vehicles can be outfitted with sensors, actuators, and R/C controls to become fully autonomous! But, what makes these high-tech tractors tick? And how can we ensure the safety, reliability, and usability of these AAV? We’ll first look at how they remain on a designated course.

Detecting rows of crops is important for the vehicle to make decisions concerning its path, speed, and effectiveness of completing its mission. To accomplish this goal we’ll use Python and OpenCV. The algorithms we’ll be exploring today include:

• Color-shifting
• Canny edge detection
• Skeletonization
• Hough line transformation

Aerial image of coffee rows.

#import dependencies
import cv2
import numpy as np
import matplotlib.pyplot as plt
#load image using cv's imread(nameoffile)
img = cv2.imread(trees1.png)

All we did was the normal practice of importing dependencies and loading our image as a numpy array, which we assign to a variable ‘img’.

Next, we’ll begin the pre-processing phase to extract the information we want from the picture. The typical workflow for pre-processing depends on what you start with. For our use-case, we want to be sure the computer is focusing on this info:

1. The color green (Most plants are green).
2. The edges of the objects which are green (The rows of plants).

#split the image into blue, green, and red channels
b,g,r = cv2.split(img) 
#here we 'amplify' the color green to stand out, without red/blue
gscale = 2*g-r-b #we are going to refer to this as our grayscale img
#Canny edge detection 
gscale = cv2.Canny(gscale,280,290,apertureSize = 3)
#checking the results (good practice)
plt.figure()
plt.plot(), plt.imshow(gscale)
plt.title('Canny Edge-Detection Results')
plt.xticks([]), plt.yticks([])
plt.show()

Results of edge detecting green areas.

If that was confusing, let me know in the comments. The difficult part that I found was adjusting the min and max arguments in the cv2.Canny() function. One more pre-processing step and we’ll be able to find find each row of coffee.

Skeletonization is the process of thinning our regions of interest to their binary constituents. This makes it easier for us to perform pattern recognition. The previous steps were necessary to acheive this:

size = np.size(gscale) #returns the product of the array dimensions
skel = np.zeros(gscale.shape,np.uint8) #array of zeros
ret,gscale = cv2.threshold(gscale,128,255,0) #thresholding the image
element = cv2.getStructuringElement(cv2.MORPH_CROSS,(3,3))
done = False
while( not done):
    eroded = cv2.erode(gscale,element)
    temp = cv2.dilate(eroded,element)
    temp = cv2.subtract(gscale,temp)
    skel = cv2.bitwise_or(skel,temp)
    gscale = eroded.copy()
zeros = size - cv2.countNonZero(gscale)
    if zeros==size:
        done = True

Now, we have a stripped down version of our previously processed grayscale image. This means the computer should have an easier time performing the Hough lines algorithm that gives us the boundaries of each coffee row.

lines = cv2.HoughLines(skel,1,np.pi/180,130)
a,b,c = lines.shape
for i in range(a):
    rho = lines[i][0][0]
    theta = lines[i][0][1]    
    a = np.cos(theta)
    b = np.sin(theta)
    x0 = a*rho
    y0 = b*rho
    x1 = int(x0 + 1000*(-b))
    y1 = int(y0 + 1000*(a))
    x2 = int(x0 - 1000*(-b))
    y2 = int(y0 - 1000*(a))
    cv2.line(img,(x1,y1),(x2,y2),(0,0,255),2, cv2.LINE_AA)
#showing the results:
plt.subplot(121)
#OpenCV reads images as BGR, this corrects so it is displayed as RGB
plt.plot(),plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB)) 
plt.title('Row Detection'), plt.xticks([]), plt.yticks([])
plt.subplot(122)
plt.plot(),plt.imshow(skel,cmap='gray')
plt.title('Skeletal Image'), plt.xticks([]), plt.yticks([])
plt.show()

Results of row detection.

Row detection results (cont.).

As you can see the program we made did quite well on its own, without the need for ML classifier models or other pre-trained data. Of course, we were only able to detect most of the rows in the field. Why is this?

Areas for improvement might include iterations in color-shifts and skeletonization. This could allow for less false positives, and greater accuracy in the program’s detection.

If you liked the article, give it a clap! Visit my website https://jamesthesken.github.io to get a hold of me or to keep up with my latest happenings.

If you hated it, let me know what I could do better!

-James