Day 33 of 100DaysofML

Face Landmark Detection using Dlib and OpenCV. So I thought of writing a bit about Computer Vision based libraries because of how much they can help us with a number of applications and I was thinking of building a computer vision based application in my coming blogs.

Now using OpenCV we can definitely identify features of a face but if we want to identify specific features of the face, our goto library is Dlib. So Dlib is going to help us identify the essential landmarks of our face such as eyebrows, eyes , jawline etc. It basically consists of a pre-trained model and it helps us with the identification of these features.

Now, I would recommend you to download the model which you can do from the below given link:

davisking/dlib-models

I shall try to give a rough idea of as to how Dlib helps us with the facial feature detection. Have a look at the diagram given below:

Facial Mapping

The different points in the diagram represent the different features of the face and they are used to map out the model or project that I am trying to create. These points basically represent the outline of all the major features of the face such as the lips, eyebrows etc. For the project that I am creating, we are going to be working with the eyes. The model mainly identifies 68 different points.

As a prerequisite, you may need to run the following commands using pip to install some libraries into your environment:

pip install cmake
pip install dlib
pip install cv2

In case you’re wondering, cmake is a library you need to install dlib library which we are using for the model and cv2 is used as the computer vision library. So I added comments to my code in order to make it easy to understand and so that I don't have to explain every single line. I shall add bold lines to make them easier to understand.

#Importing the main libraries
import cv2
import dlib
#Initializing the variable to capture video
cap = cv2.VideoCapture(0)
#Calling the dlib library to read and understand faces
hog_face_detector = dlib.get_frontal_face_detector()
#Importing our pretrained model
dlib_facelandmark = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
#Main Execution code
while True:
    _, frame = cap.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = hog_face_detector(gray)
    for face in faces:
face_landmarks = dlib_facelandmark(gray, face)
for n in range(0, 68):
            x = face_landmarks.part(n).x
            y = face_landmarks.part(n).y
            cv2.circle(frame, (x, y), 1, (0, 255, 255), 1)
cv2.imshow("Face Landmarks", frame)
key = cv2.waitKey(1)
    if key == 27:
        break
cap.release()
cv2.destroyAllWindows()

Some of the key code elements that you should know from the main execution code are:
- cap.read(): used to view the video stream from your webcam
- cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY): This is used to convert the color video stream into grayscale because it tends to provide us with higher accuracy and this is done using OpenCV
- faces = hog_face_detector(gray): using the dlib library to identify the faces in the given frame and we can have multiple faces in the frame which would be part of the loop and it would still compute, just that if you have too many faces, the overall timing comes down.
- dlib_facelandmark: This basically identifies all the landmarks from the face that it has obtained from the list of faces.
- the for loop is for calling all the 68 features that are initialized as part of the model.

The main execution code basically runs an infinite loop and keep checking for faces and keeps pointing to the face and it closes the webcam everytime you use the ‘s’ key.

You can use an image if you don’t want to use your webcam. You just need to switch to a picture and load that using the library called Image.

You can see in the above picture that the outline has been made around my face. I’m going to attach the link to my github from where you can clone the given project and work on it. It is super simple and super cool for starters to learn and work on.

You can download the model and source code from the below given link:

charansoneji/Face-Landmark-detection-OpenCV-and-dlib-

Thanks for reading. Keep Learning.

Cheers.