Hand Detection and Finger Counting Using OpenCV-Python
Implementing hand detection using OpenCV-Python with Cosine Theorem for Finger Counting Problem.
By seeing above image now you are very excited for implement it (like me). So not wasting too much time let’s jump to the code.
OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
- cv2: opencv [pip install opencv]
- numpy: for handling arrays as well as for math [pip install numpy]
import cv2 as cv
import numpy as np
img_path = "data/palm.jpg"
img = cv.imread(img_path)
It is used for highlighting specific color on image.
- hsvim : Change BGR (blue, green, red) image to HSV (hue, saturation, value).
- lower : lower range of skin color in HSV.
- upper : upper range of skin color in HSV.
- skinRegionHSV : Detect skin on the range of lower and upper pixel values in the HSV colorspace.
- blurred: bluring image to improve masking.
- thresh : applying threshing.
hsvim = cv.cvtColor(img, cv.COLOR_BGR2HSV)
lower = np.array([0, 48, 80], dtype = "uint8")
upper = np.array([20, 255, 255], dtype = "uint8")
skinRegionHSV = cv.inRange(hsvim, lower, upper)
blurred = cv.blur(skinRegionHSV, (2,2))
ret,thresh = cv.threshold(blurred,0,255,cv.THRESH_BINARY)
Now let’s finding contours on the image.
contours, hierarchy = cv.findContours(thresh, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
contours = max(contours, key=lambda x: cv.contourArea(x))
cv.drawContours(img, [contours], -1, (255,255,0), 2)
hull = cv.convexHull(contours)
cv.drawContours(img, [hull], -1, (0, 255, 255), 2)
Any deviation of the object from this hull can be considered as convexity defect.
hull = cv.convexHull(contours, returnPoints=False)
defects = cv.convexityDefects(contours, hull)
Now, this is Math time! Let’s understand cosine theorem.
In trigonometry, the law of cosines relates the lengths of the sides of a triangle to the cosine of one of its angles. Using notation as in Fig. 1, the law of cosines states where γ denotes the angle contained between sides of lengths a and b and opposite the side of length c.
By seeing this formula now we understand that if we have; a,b and gama then we also find c as well as if we have; a,b,c then we also find gamma (vice-versa)
For finding gamma this formula is used:
Using Cosine theorem to recognize fingers
Sorry! for this dirty MS-Paint I am not a artist.
In Fig. 2, I am draw a Side: a,b,c and angle: gamma. Now this gamma is always less than 90 degree, So we can say: If gamma is less than 90 degree or pi/2 we consider it as a finger.
Note: if you not familiar with Convexity Defects please go and read this article by opencv docs. click here
Convexity Defects returns an array where each row contains these values :
- start point
- end point
- farthest point
- approximate distance to farthest point
By, this point we can easily derive Sides: a,b,c (see CODE) and from cosine theorem we can also derive gamma or angle between two finger. As you read earlier, if gamma is less than 90 degree we treated it as a finger. After knowing gamma we just draw circle with radius 4 in approximate distance to farthest point. And after we just simple put text in images we represent finger counts (cnt).
if defects is not None:
cnt = 0
for i in range(defects.shape): # calculate the angle
s, e, f, d = defects[i]
start = tuple(contours[s])
end = tuple(contours[e])
far = tuple(contours[f])
a = np.sqrt((end - start) ** 2 + (end - start) ** 2)
b = np.sqrt((far - start) ** 2 + (far - start) ** 2)
c = np.sqrt((end - far) ** 2 + (end - far) ** 2)
angle = np.arccos((b ** 2 + c ** 2 - a ** 2) / (2 * b * c)) # cosine theorem
if angle <= np.pi / 2: # angle less than 90 degree, treat as fingers
cnt += 1
cv.circle(img, far, 4, [0, 0, 255], -1)
if cnt > 0:
cnt = cnt+1
cv.putText(img, str(cnt), (0, 50), cv.FONT_HERSHEY_SIMPLEX,1, (255, 0, 0) , 2, cv.LINE_AA)
Let’s see our final result
You can also do it for Videos, just by calling “cv.VideoCapture()”. If You want the code you can get in my GitHub: finger_counting_video.py