CodeX
Published in

CodeX

Making Squid Game Cookie Cutter Challenge using OpenCV

Can you beat your own time?

At this point, I think most of the world has gone crazy over the online word game from New York Times called Wordle. While that game is a single-player game, a famous YouTuber JedCal made a P2P version called WordleForFriends and captured his audience by surprise since anyone can send a link to their friends to guess the 5-letter word for a total of 6 tries. Check out his version of Wordle for free here.

I thought that was a good idea — adding a P2P feature to such a simple game could get people talking about it, driving popularity to newest heights. As a game developer, making games is definitely something I find interest in and I thought I could make a similar online game using computer vision techniques. Here comes the idea: remember Squid Game?

Do you know what someone with no money has in common with someone with too much money?

Now, let’s see how it actually works with computer vision. Taking a Dalgona candy image with a triangular symbol, the players need to trace using their index finger the path of the symbol on the candy. Simple and straightforward. The TWO main tools that we may need in coding this game are:

  1. OpenCV
  2. MediaPipe

Clearly, we need some image pre-processing before the target image can be used in any way and OpenCV is the best tool to do that. On the other hand, if we need players to track their fingers, we also need another package called MediaPipe. Then, with some simple calculations and algorithms, we can combine these two tools and make the infamous Squid Game Cookie Cutter Challenge! Let’s see step-by-step on how to do this.

Steps

First of all, we need to prepare several images. The first image is definitely the candy image on which we need to trace. Then, the second image we need is the broken candy image if the player loses. For both of these images, we need to scale and resize them before overlaying them on top of the camera input frame that reads the player’s index finger.

Because the target image is not the same size as the camera frame input, here’s what I did to overlay the target image on top without distorting it. The same is done to the broken target image.

This is enough to overlay the target image on top of the camera input frame with no distortions. Next, we need to preprocess the target image and find the edges within the candy.

Here, because our target image is a fixed input image, we know there are three lines to be detected. Should there be a different target image every time the player plays the game, like in a P2P version, the edge detection method has to be able to adjust accordingly. Consider this the first version of the game using only one single target image.

Notice we append all the detected lines into respective Python lists. This is to obtain the mean of each line and output only 3 single lines on the target image instead of multiple similar detections.

To start the game, we ask the player to place their index finger on a circular starting point, after which the path is traced out. The game begins by checking if the player’s index finger lies on the starting point. Here’s how:

Each point detected from the player’s index finger will be drawn out on the target image, checking if the point is on the detected lines or not.

How to lose

At any one time when the point leaves the lines, the player loses.

How to win

We also want to keep track of the length of the path traced out by the player to determine when the player wins the game. By nature, the length of the path traced out by the player will be longer or similar to the actual length of all three lines. If the length of the path is longer or similar to the actual length of all three lines and the point of index finger is once again within the starting point, the game ends and time is logged.

And that is the entire game! Check out the whole code here if you want to replicate the project.

Living is no fun for them :)

Basically, I have shown you how I made a replica of the Squid Game Cookie Cutter Challenge using OpenCV and some simple image processing techniques that involves hand-tracking. However, the fun doesn’t stop here!

Now, I have gone a step further to include a feature where players can input a path for their friends to trace out. Essentially, the idea involves the player drawing on a canvas on the screen using a mouse and a link can be generated for their friends. Here’s what’s going on at the back (or at least what I think it is):

  1. The player draws on the canvas the path to trace out.
  2. Press on the “Generate Link” button.
  3. The canvas is saved as image to Firebase Storage.
  4. A link is returned, upon pressing which will lead to a new page.
  5. The new page will retrieve the image from Firebase Storage and overlay on top of the camera frame and your friends can start tracing it out!

However, the line detection in this case is not so straightforward. Since the intersection between the player’s index finger and the lines is determined by purely the distance between the circle and the detected lines, having numerous short lines drawn on the detected lines would yield finicky results. This is definitely something I would need time to look into for the best results.

On the flip side, I’m glad the Firebase Storage works as expected, but another issue is that it is currently without authentication and authorization, which is a dangerous problem. Before you get to play the game, there will be some serious cleanup to do to ensure security of the storage. A leaderboard will also be created to keep track of the time scores. When the project is finally complete, it will be time for you to draw out a path for your friends to test our the stability of their hand. Until then, happy coding!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store