Harry Potter’s Invisibility Cloak — Using OpenCV, Python — Computer Vision

An awesome project to learn and execute OpenCV, Python

Published in

Analytics Vidhya

5 min readApr 25, 2020

Harry got his cloak of invisibility from his father whilst we can create our own and check how it looks on us, using OpenCV. Yes! no need of going to Hogwarts, all you need to have is interest to learn and a computer.

How are we going to do it?

We all heard of Green screening, all those VFX effects in movies. Here what we do is the opposite to that of green screening(removing background), yes we remove the foreground

Here I used blue color cloth. Can I use any other color! yes red, green will also work

Let’s set the Environment

We need to install the OpenCV (Open Source Computer Vision) library which helps us carry out tasks related to computer vision. We need to do a pip install for the OpenCV library.

pip install opencv-python

Open Spyder(Installation-https://www.youtube.com/watch?v=5mDYijMfSzs)

Import the libraries

import cv2
import numpy as np
import time

Capturing the video

Video_capture = cv2.VideoCapture(0)

The first step towards reading a video file is to create a VideoCapture object. Its argument can be the device index. (0- integrated web came,1- external webcam and so on..)

Let the system camera warm up!

time.sleep(3)
background = 0

Python time sleep function is used to add a delay in the execution of a program. When the statement time.sleep(t) is executed then the next line of code will be executed after t seconds.

for i in range(45):
    ret, background = Video_capture.read()
background = np.flip(background, axis=1)

Here we are capturing the background in the range of 45 and flipping(literally) the background

Color detection and segmentation

while (Video_capture.isOpened()):
    ret, frame = cap.read()
    if not ret:
        break
    frame= np.flip(frame, axis=1)

Here we create a while loop to capture the frame and run the frame. capture.read() returns a bool (True/False). If the frame is read correctly, it will be True. “ret” will obtain return value from getting the camera frame, either true or false. “frame” will get the next frame in the camera (via “capture”).

“if not ret:
break”. This snippet of code breaks the loop if the frame isn’t returned.

#Converting the color space from BGR to HSV
   hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

BGR — HSV is one of the color-space conversion methods available in OpenCV. For color conversion, we use the function cv2.cvtColor(input_image, flag) where flag determines the type of conversion. For BGR -HSV, we use the flag cv2.COLOR_BGR2HSV. “Video_capture.read” which captures the frame in BGR (Blue-Green-Red) colors, while we use HSV format for masking. Hence we flip the color code of the frame.

#Creating masks with ranges of blue color to detect
  lower_blue = np.array([94, 80, 2])
  upper_blue = np.array([126, 255, 255])
  mask_all = cv2.inRange(hsv,lower_blue,upper_blue)

We are creating a range for the color(blue), where we define the lowest part of the highest part of blue color (lowest and highest values of HSV). The “inRange” function returns a binary mask, in which white pixels (255) represent pixels that fall into the upper and lower limit range and black pixels (0) do not. We generated a mask to determine the region in the frame corresponding to the detected blue color

Choose your color and create the invisibility cloak

Just replace the code snippet

#Range of different colors
Red color
low_red = np.array([161, 155, 84]) high_red = np.array([179, 255, 255])
Blue color
low_blue = np.array([94, 80, 2]) high_blue = np.array([126, 255, 255])
Green color
low_green = np.array([25, 52, 72]) high_green = np.array([102, 255, 255])
Every color except white
low = np.array([0, 42, 0]) high = np.array([179, 255, 255])

Now we have to refine this mask and then use it for segmenting out the cloth from the frame.

#Doing morphological operations
mask_all = cv2.morphologyEx(mask_all, cv2.MORPH_OPEN, np.ones((3,3),np.uint8))
mask_all = cv2.morphologyEx(mask_all, cv2.MORPH_DILATE, np.ones((3,3),np.uint8))

Here we perform some Morphological transformations( operations based on the image shape) using cv2.morphologyEx. It needs three inputs, one is our original image, operators and kernel size. “np.ones((3,3),np.uint8)” is the kernel where (3,3) is the size and “np.unit8” is the data type.

The Morphological operators we use here are Opening and Dilation. Dilation increases the white region in the frame or size of foreground object increases. Opening is just another name of erosion followed by dilation. It is useful in removing noise.erosion(just like soil erosion, erodes the boundaries) removes white noises, but it also shrinks our object. So we dilate it. Since noise is gone, they won’t come back, but our object area increases. It also helps in joining broken parts of an object.

#Create an inverted mask to segment out the blue from the #frame(hiding the blue part)
mask1 = cv2.bitwise_not(mask_all)## Segment the blue color part out of the frame using bitwise and #with the inverted mask
streamA = cv2.bitwise_and(frame,frame,mask=mask1)# Creating an image showing static background frame pixels only for #the masked region
streamB = cv2.bitwise_and(background, background, mask = mask_all)

Here we replace the pixel values of the detected blue color region with corresponding pixel values of the static background region which gives us an augmented output creating the effect, converting our cloth into an invisibility cloak. To do this we use bitwise_and, bitwise_not operations.

#Generating the final output
output = cv2.addWeighted(streamA,1,streamB,1,0)
cv2.imshow("Thankyou_Dumbledore",output)
if cv2.waitKey(1) == 27:
     break

Here we use the “addWeighted” object for unsharp masking. The function cv2.imshow() is used to display an image in a window. The first argument is a window name which is a string. The second argument is our image.

“if cv2.waitKey(25) == 27:” waitKey function displays the image for specified milliseconds. Otherwise, it won’t display the image. For example, waitKey(0) will display the window infinitely until any keypress. waitKey(25) will display a frame for 25 ms, Here we are specifying 1ms after which display will be automatically closed. As we have put it in a loop, it will display the video frame-by-frame

27 is the keycode for ‘Esc’. Press “Esc” to stop the video. The break is a keyword in python which is used to bring the program control out of the loop.

Video_capture.release()
cv2.destroyAllWindows()

This releases the webcam, then closes all of the imshow() windows.

Yaas!!

Finally, We have successfully created our own invisible cloak code in Python using the OpenCV library. Try using the methods we have used here in other projects as well.

Cheerioo!!

FULL CODE!

Github repository link — https://github.com/Davinci-Decoder/ML-AI

References: https://www.learnopencv.com/; https://docs.opencv.org/2.4/index.html