Hand Detection and Finger Counting Using OpenCV-Python

Implementing hand detection using OpenCV-Python with Cosine Theorem for Finger Counting Problem.

Madhav Mishra
Aug 15, 2020 · 5 min read
Image for post
Image for post
Hand Detection and Finger Counting

By seeing above image now you are very excited for implement it (like me). So not wasting too much time let’s jump to the code.

OpenCV

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.

Importing Libraries

  • cv2: opencv [pip install opencv]
  • numpy: for handling arrays as well as for math [pip install numpy]
import cv2 as cv
import numpy as np

Reading Image

img_path = "data/palm.jpg"
img = cv.imread(img_path)
cv.imshow('palm image',img)
Image for post
Image for post
palm image

SkinMask

It is used for highlighting specific color on image.

  • hsvim : Change BGR (blue, green, red) image to HSV (hue, saturation, value).
  • lower : lower range of skin color in HSV.
  • upper : upper range of skin color in HSV.
  • skinRegionHSV : Detect skin on the range of lower and upper pixel values in the HSV colorspace.
  • blurred: bluring image to improve masking.
  • thresh : applying threshing.
hsvim = cv.cvtColor(img, cv.COLOR_BGR2HSV)
lower = np.array([0, 48, 80], dtype = "uint8")
upper = np.array([20, 255, 255], dtype = "uint8")
skinRegionHSV = cv.inRange(hsvim, lower, upper)
blurred = cv.blur(skinRegionHSV, (2,2))
ret,thresh = cv.threshold(blurred,0,255,cv.THRESH_BINARY)
cv.imshow("thresh", thresh)
Image for post
Image for post
thresh

Contours

Now let’s finding contours on the image.

contours, hierarchy = cv.findContours(thresh, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
contours = max(contours, key=lambda x: cv.contourArea(x))
cv.drawContours(img, [contours], -1, (255,255,0), 2)
cv.imshow("contours", img)
Image for post
Image for post
contours

Convex Hull

hull = cv.convexHull(contours)
cv.drawContours(img, [hull], -1, (0, 255, 255), 2)
cv.imshow("hull", img)
Image for post
Image for post
hull

Convexity Defects

Any deviation of the object from this hull can be considered as convexity defect.

hull = cv.convexHull(contours, returnPoints=False)
defects = cv.convexityDefects(contours, hull)
Image for post
Image for post
Convexity Defects Example

Cosine Theorem

Now, this is Math time! Let’s understand cosine theorem.

In trigonometry, the law of cosines relates the lengths of the sides of a triangle to the cosine of one of its angles. Using notation as in Fig. 1, the law of cosines states where γ denotes the angle contained between sides of lengths a and b and opposite the side of length c.

Image for post
Image for post
Fig. 1

Formula

Image for post
Image for post

By seeing this formula now we understand that if we have; a,b and gama then we also find c as well as if we have; a,b,c then we also find gamma (vice-versa)

For finding gamma this formula is used:

Image for post
Image for post

Using Cosine theorem to recognize fingers

Image for post
Image for post
Fig. 2

Sorry! for this dirty MS-Paint I am not a artist.

In Fig. 2, I am draw a Side: a,b,c and angle: gamma. Now this gamma is always less than 90 degree, So we can say: If gamma is less than 90 degree or pi/2 we consider it as a finger.

Counting Finger

Note: if you not familiar with Convexity Defects please go and read this article by opencv docs. click here

Convexity Defects returns an array where each row contains these values :

  • start point
  • end point
  • farthest point
  • approximate distance to farthest point

By, this point we can easily derive Sides: a,b,c (see CODE) and from cosine theorem we can also derive gamma or angle between two finger. As you read earlier, if gamma is less than 90 degree we treated it as a finger. After knowing gamma we just draw circle with radius 4 in approximate distance to farthest point. And after we just simple put text in images we represent finger counts (cnt).

if defects is not None:
cnt = 0
for i in range(defects.shape[0]): # calculate the angle
s, e, f, d = defects[i][0]
start = tuple(contours[s][0])
end = tuple(contours[e][0])
far = tuple(contours[f][0])
a = np.sqrt((end[0] - start[0]) ** 2 + (end[1] - start[1]) ** 2)
b = np.sqrt((far[0] - start[0]) ** 2 + (far[1] - start[1]) ** 2)
c = np.sqrt((end[0] - far[0]) ** 2 + (end[1] - far[1]) ** 2)
angle = np.arccos((b ** 2 + c ** 2 - a ** 2) / (2 * b * c)) # cosine theorem
if angle <= np.pi / 2: # angle less than 90 degree, treat as fingers
cnt += 1
cv.circle(img, far, 4, [0, 0, 255], -1)
if cnt > 0:
cnt = cnt+1
cv.putText(img, str(cnt), (0, 50), cv.FONT_HERSHEY_SIMPLEX,1, (255, 0, 0) , 2, cv.LINE_AA)

Let’s see our final result

cv.imshow('final_result',img)
Image for post
Image for post

You can also do it for Videos, just by calling “cv.VideoCapture()”. If You want the code you can get in my GitHub: finger_counting_video.py

You can also watch this tutorial video in YouTube.

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data…

Sign up for Analytics Vidhya News Bytes

By Analytics Vidhya

Latest news from Analytics Vidhya on our Hackathons and some of our best articles! Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Madhav Mishra

Written by

AI Enthusiast

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Madhav Mishra

Written by

AI Enthusiast

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store