Chrome Dino Game on Blink using PyAutoGUI and OpenCV

In this article, I will walk through the steps to automate the Chrome Dino game just by blinking your eyes with the help of a highly optimized library OpenCV designed to solve Computer Vision problems and PyAutoGUI that lets your Python scripts control the mouse and keyboard to automate interactions with other applications. Here, you just need to blink the eyes instead of pressing space bar to jump.

About the Game

The Chrome Dino game is a simple endless runner game, which sees you jump over cacti, and dodge underneath obstacles. Chrome has a hidden infinite runner game you can play whenever your computer or phone is in offline mode, but it turns out you can also play it without disconnecting. Just type chrome://dino in your address bar and the “No internet” screen will open — press space to Start the game.

The Complete code of this project is in following GitHub location. Please do ⭐ this repository, if it helped you.

ANJALIAGARWAL-IT/chrome-dino-play-on-eye-blink

Step by Step Implementation

Import these required libraries

1. cv2 — pythons OPENCV version.
2. dlib — This is one of the most important library in this program. From this, we are using frontal face detector and shape_predictor_68_face_landmarks shape predictor.
3. scipy.spatial — The length of a straight line directly between two points through three-dimensional space is spatial distance. Here, we need to import the distance from which we will find out the Euclidean Distance.
4. PyAutoGUI — It is a cross-platform GUI automation Python module for human beings. It will help in controlling the space bar in keyboard.
5. Time — Time module is available in Python which provides functions for working with times.

Eye Aspect Ratio

Now, we need to find out the EAR which stands for Eye Aspect Ratio.

Each eye is represented by 6 (x, y)-coordinates, starting at the left-corner of the eye, and then working clockwise around the remainder of the region. The Formula for Eye Aspect Ratio is down below.

Figure : The eye aspect ratio equation.

The numerator of this equation computes the distance between the vertical eye landmarks while the denominator computes the distance between horizontal eye landmarks, weighting the denominator appropriately since there is only one set of horizontal points but two sets of vertical points.

Figure : The 6 facial landmarks associated with the eye.

JUMP Function

Now, We need to define the Jump function. Using PyAutoGUI library, we need to use the keyDown function to press space and then an equivalent timer function to trigger space. After pressing space, we need to release the space bar using keyUp function.

cap is the variable which will start video from thelaptop webcam and hold all the frames which are capturing by the webcam.

get_frontal_face_detector() is one of the function in dlib, which will detect faces from the images supplied to this function. For more details about this function, please go thorough the link http://dlib.net/python/index.html#dlib.get_frontal_face_detector

shape_predictor is another important librery from dlib, to this funciton, we are passing shape_predictor_68_face_landmarks.dat, so that, this predictor variable will give all the factial points from the supplier image.

Then, we need to convert RGB colour image to grayscale image and then start detecting face using the dlib library.

Drawing Line around Eyes

Figure : Facial Landmarks

Now, the most important part is to calculate EAR as explained above. Syntax : cv2.line(image, start_point, end_point, color, thickness)

Parameters:
image: It is the image i.e. the detected face on which line is to be drawn.
start_point: It is the starting coordinates of line. The coordinates are represented as tuples of two values i.e. (X coordinate value, Y coordinate value).
end_point: It is the ending coordinates of line. The coordinates are represented as tuples of two values i.e. (X coordinate value, Y coordinate value).
color: It is the color of line to be drawn.
thickness: It is the thickness of the line in px.

Return Value: It returns an image.

Here n ranges from 36 to 42 for left eye coordinates and 42 to 48 for right eye coordinates which needs to be detected from the image(frames captured from webcam). We need two points (x,y) coordinates for drawing a line, so we are using (x,y) and (x2,y2) coordinates. At the end, we are storing the EAR value for both left and right eye in left_ear and right_ear respectively.

To detect the blink of our eyes, we need to add both the ratios and find the average ratio of both eyes.

For my eyes, the most suitable value is 0.26. You can do hit and trial method to find your EAR value. imshow function is used to display the real time image captured by the webcam.

To quit the webcam, press Escape key(keycode value is 27) and release the webcam.

CONCLUSION

In this article, I have explained to build the Dino game from scratch. Using OpenCV, you can experiment and build a lot of projects. Pull the code and try it out yourself.

Please click the 👏 button if you liked it and share to help others find it ❤️Stay tuned to check out more interesting blogs related to Machine Learning and Computer Vision.

Follow me on LinkedIn -https://in.linkedin.com/in/anjaliagarwal98. To know more about me, go check out my Website -anjaliagarwal.tech 😄