Draw Using a Virtual Pen on a Computer Screen using OpenCV in Python
In this project, I use OpenCV in python to draw on the screen using a virtual pen i.e, any marker can be used to draw using the technique of contour detection based on the mask of the desired colored target marker.If you are a beginner and want to impress your friends with a visual demo, this is for you.
Have a look at the final project in action:
If you want to jump straight into the code without explanation then you can scroll down to the bottom of this page, where the complete code is provided or click here for GitHub link.
Steps involved
Step 1:
Find the HSV range of the target Marker/pen and save the values in a .npy file
Step 2:
- First, we will use color masking to get a mask of our colored pen/marker using the above HSV range.
- Next, using contour detection we detect and track the location of that pen,i.e get the x,y coordinates.
- Next, draw a line by joining the x,y coordinates of pen’s previous location (location in the previous frame) with the new x,y points.
- Finally, add another feature to clear the entire Canvas/Screen.
Step 1: Find Color range of target Pen and save it
First and foremost we must find an appropriate color range for our target colored object, this range will be used in cv2.inrange() function to filter out our object. We will also save our range array as a .npy file in our disk so we can access it later.
Since we are trying to go for color detection we will convert our RGB (or BGR in OpenCV) format image to HSV (Hue, Saturation, Value) color format as its much easier to manipulate colors in that model.
This below script will let you use track bars to adjust the hue, saturation, and value channels of the image. Adjust the track bars until only your target object is visible and the rest is black.
Step 2: start drawing
This entire application is built fundamentally on contour detection. It can be thought of as something like closed color curves on compromises that have the same color or intensity, it’s like a blob. In this project we use color masking to get the binary mask of our target color pen, then we use the counter detection to find the location of this pen and the contour to find it.
When we work with it then it is a matter of literally connecting the dots, we just have to draw a line using the x, y dots of the previous position of the pen with new x, y dots and that’s it. , We have a virtual pen.
There are 3 main points in the code
- Apply morphological operations.
- Detect and track the colored object.
- Find the object’s x,y coordinates.
imports
import cv2
import numpy as np
import timeloading HSV value , capturing video and initializing the required parameters
#This variable determines if we want to load color range from #memory or use the ones defined here.
load_from_disk = True# If true then load color range from memory
if load_from_disk:
hsv_value = np.load('hsv_value.npy')cap = cv2.VideoCapture(0)
cap.set(3,1280)
cap.set(4,720)kernel = np.ones((5,5),np.uint8)# Initializing the canvas on which we will draw upon
canvas = None# Initilize x1,y1 points
x1,y1=0,0# Threshold for noise
noiseth = 800
Read video and a Apply morphological operations
while(1):
_, frame = cap.read()
frame = cv2.flip( frame, 1 )
# Initialize the canvas as a black image of the same size as the frame.
if canvas is None:
canvas = np.zeros_like(frame)# Convert BGR to HSV
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
# If you're reading from memory then load the upper and lower ranges
# from there
if load_from_disk:
lower_range = hsv_value[0]
upper_range = hsv_value[1]
# Otherwise define your own custom values for upper and lower range.
else:
lower_range = np.array([134, 20, 204])
upper_range = np.array([179, 255, 255])
mask = cv2.inRange(hsv, lower_range, upper_range)
# Perform morphological operations to get rid of the noise
mask = cv2.erode(mask,kernel,iterations = 1)
mask = cv2.dilate(mask,kernel,iterations = 2)
Find the object’s x,y coordinates and start drawing
# Find Contours
contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# Make sure there is a contour present and also its size is bigger than
# the noise threshold.
if contours and cv2.contourArea(max(contours,
key = cv2.contourArea)) > noiseth:
c = max(contours, key = cv2.contourArea)
x2,y2,w,h = cv2.boundingRect(c)
# If there were no previous points then save the detected x2,y2
# coordinates as x1,y1.
# This is true when we writing for the first time or when writing
# again when the pen had disappeared from view.
if x1 == 0 and y1 == 0:
x1,y1= x2,y2
else:
# Draw the line on the canvas
canvas = cv2.line(canvas, (x1,y1),(x2,y2), [255,0,0], 4)
# After the line is drawn the new points become the previous points.
x1,y1= x2,y2 else:
# If there were no contours detected then make x1,y1 = 0
x1,y1 =0,0
# Merge the canvas and the frame.
frame = cv2.add(frame,canvas)
# Optionally stack both frames and show it.
stacked = np.hstack((canvas,frame))
cv2.imshow('VIRTUAL PEN',cv2.resize(stacked,None,fx=0.6,fy=0.6)) k = cv2.waitKey(1) & 0xFF
if k == 27:
break
# When c is pressed clear the canvas
if k == ord('c'):
canvas = Nonecv2.destroyAllWindows()
cap.release()
Note: Make sure the target maker color is not present in the background to avoid error. press c to clear the canvas.
Full Code
Github link:
https://github.com/GeekyPRAVEE/OpenCV-Projects/tree/master/Virtual%20Pen
other related projects
you can find me elsewhere on
Instagram @programming_fever
Facebook @GeekyPRAVE
Twitter @GeekyPRAVEE
