Playing the dinosaur game… using mind control???

So you know the moment your internet cuts out and you start pondering what to do for the next 5 minutes until it’s up again, yep it sure does suck. Within the next few seconds, the offline dinosaur game pops up, and all of the negative emotions suddenly leave your mind.

Now I have to admit this game is quite fun but after a while it gets kind of boring, just pressing one key over and over again. I don’t know about you but I am lazy, and I really don’t like using my hands, so I thought: why use hands at all? What if you could control the game with just a thought?

Now, at this point you are probably lost and wondering “what the heck is he talking about??”, let me explain myself. This is all possible using a Brain Computer Interface (for a definition, you can check my other article).

In this article, I will explain exactly what you need to do to recreate this for yourself!!

All you need to complete this project is a laptop and the Muse 2 headset! The headset is an EEG, which stand for electroencephalogram (this basically means, using electrodes, it will read your brain waves and convert them into binary code, and we will code this later on!). The headset is originally meant for meditation so we will have to code a little bit on our own! If you want to learn more about BCI’s check out this article here.

Just a small disclaimer, everything in this guide is optimized for MacOS.

Let’s get started!

To begin, we first need to set up the code!

Step 1

For this we firstly need to download Python, this is the coding language we will be completing this project in. You can download it here.

You are going to click the “Download Python 3.12.2” button. Now this may vary because new versions of Python are always releasing.

Step 2

After downloading python, you are going to open up your computers terminal. You can do this by opening up your “spotlight search” on MacOS. You do this by pressing “⌘ + spacebar”, then typing in “terminal” and then enter. After you that, you should see this menu:

Step 3

After you have this open, you would type in this into the terminal.

python --version

Or if this doesn’t work, then type:

python3 --version

After you have printed this, the terminal should output the version of Python you downloaded, if it doesn’t, you may want to re-download it.

Step 4

After you have assured that you have downloaded Python, we must check if pip is downloaded. Python’s “pip” is a package management system used to install and manage software packages written in Python. It allows users to easily install, upgrade, and remove Python packages from the Python Package Index (PyPI) and other repositories.

To check if its installed, run this command in your terminal (keep in mind if you added “3” in step 3, then add it again here):

pip --version

or

pip3 --version

Step 5

Once you have pip installed, we can then move on to downloading a code editor. The editor we will be using in this project is Visual Studio Code. It is available for MacOS, Windows, and Linux.

To install it just click here.

You want to click the “Download Mac Universal” button, then follow the on screen instructions.

We have now completed most of the code side of this project!

Now we want to actually connect the Muse 2.

Step 1

This would usually just be possible by downloading it from Muse’s actual website, but it has been disabled since 2016.

But I have found a way around that, by using “Petal Metrics” we can connect any EEG device (the headband counts as an EEG) to our laptop. You can download it here.

Once downloaded, open it up.

Step 2

Now that we have Petal Metrics installed, let’s get to connecting our muse to our computer. Once you have it opened, under type, select LSL (Lab Streaming Layer). The name doesn’t matter.

To connect your Muse, power it on and press the big green button called “Stream”

Once it is connected, it will say “streaming Muse#XXXX” once you scroll down. Now our muse is connected to our computer!!

Step 3

You’re now officially streaming data from your headband — congratulations! At this stage, you’re approximately halfway through the entire project. To visualize the data, you need code that translates the signals into a visual representation. Luckily, Alexandre Barachant has developed a fantastic program called Muselsl for this purpose. All you need to do is install Muselsl to both view and work with the Muse data.

It is a Github file, so it may be a little difficult to navigate but let me show you.

We need to clone a Github Repository, let’s see how to do that! Click this link, and you will be led here.

Over here we want to click the big green button that says “Code” and copy the link. After we do that we want to paste it into VS Code.

Click the “Clone Repository” button on the left hand side and paste the link you just copied! Great, now we have the code to connect our computer to the muse!

Step 4

Now we want to open a new terminal in VS code so we can install some things and actually view our brain waves!! Cool isn’t it?

Let’s open the new terminal: to begin, surf your cursor to the top of your screen and click on terminal.

Now we want to click on “New Terminal” and it should open this up on the bottom of your screen.

Step 5

To view our brain waves, we first need to use pip to install muselsl, we can do this by typing in the following code. Once again, the code you paste depends on which version (3 or without the 3) of python your laptop recognizes.

pip install muselsl

or

pip3 install muselsl

Now just let it run and install it!

Step 6

Now we can get to actually viewing and streaming our brainwaves. Firstly, we have to make sure that the muse is connected, open Petal Metrics and connect it!

Then run this code in the terminal:

muselsl view

It should open a screen where you can view your brainwaves! It should look something like this.

We have finally connected our brain to a computer!! Now we have to get it to recognize our blinks!

Playing the Dino game!

Now for the part that you have been waiting for: playing the actual dinosaur game!

If you look closely you can see whenever you blink there are dips, for example:

So we can just code it so that whenever there is a spike or a dip, we need the dino to jump (or press the spacebar, which is how the dinosaur jumps).

Step 1

In VS Code, go to:

exmaple > neurofeedback.py

This is the file that your brainwaves are printed and shown. Once you have opened this file, you want to click run (the side ways triangle on the top right of your screen, once you do this you should run into an error, don’t worry, you have done everything right.

This is the error you should get; you are getting this error because of the imports at the start of the code.

The reason for this error is because your computer does not recognize, or doesn’t have these things installed. You can do so by copying the following code into your terminal (remeber to add/remove the 3):

pip3 install numpy

or

pip install numpy

pip3 install matplotlib

or

pip install matplotlib

pip3 install utils

or

pip install utils

pip3 install sklearn

or

pip install sklearn

Once you copy all of these, you shouldn’t be getting any errors.

Step 2

Your code should be running without throwing an error, but you still dont have an actual EEG stream so the computer has no brain waves to read, we can fix this by connecting the muse and putting it on, once done this you should be seeing the following output:

These numbers represent your alpha waves. Alpha waves are the ones when your brain is in relaxation or it is relaxed, there is not really anything we can do with these because muscle movement does not affect them.

Remember, we want to code so that when we have a dip, we press the spacebar.

One of the waves that will change if we blink (or have any muscle movement) are delta waves. To print these, copy the following code:

deltaWaves = band_powers[Band.Delta]
print("Delta: {}".format(deltaWaves)) 

Copy this code and paste it just below line 124, so it shows like this.

This code creates a variable named “deltaWaves” and sets it to the value of “band_powers[Band.Delta]”. Once you run the code now, it should be printing your delta waves:

Step 3

Delta waves can easily detect muscle movements and eye blinks. We notice that whenever that we blink, the number printed is greater than 1.

We now know that If delta waves > 1 = blink, we just have to turn this into actual code. We can do this using an if statement.

if deltaWaves >= 1: 
      print("Blink!")

Copy this code and paste it just below line 126, like this:

Upon running the program, it only slightly works. It is not as precise as we want it to be, we can see that whenever we blink the delta waves trigger multiple times, making the computer think we blinked multiple times (in other words, the dinosaur would jump multiple times).

We can fix this by making our code more precise.

Step 4

First, we will make a list at the top called “allValues” and store the variable “0” in it. Copy the following code:

allValues = [0]

and paste it in line 95, like this:

Step 5

After setting up the list, it’s time to actually make use of it. Add this code right below where we defined delta waves:

allValues.append(deltaWaves)

This adds the most recent delta wave number to the list we previously generated “allValues”. In order to compare numbers in the following step, this is necessary.

Step 6

Upon including the delta wave in the allValues list, we may modify our if statement to improve its precision. Here’s the code to replace the previous if statement you had.

if deltaWaves >= 1 and allValues[-2] <= 1: 
    print("Blink!")            

This code basically tells the computer that if the delta wave is greater than one and if the previous one in the list is less than one, then there is a blink.

Why does this work? That’s because all we want is for the computer to recognise a blink as the first delta wave larger than one. The remainders of the initial blink will be represented by all subsequent integers larger than one. The code’s allValues[-2] instructs the computer to refer back to the last recorded delta wave value, which is the final value in the allValues list. If the number was less than one, it indicates that a blink is genuinely occurring at this time rather than any residual muscle action.

Step 7

Sequential blinks will not be detected by our code. If we blink quickly after our initial blink, the computer will interpret it as residual muscle action from the first blink because our code only detects blink when the number preceding is smaller than one. Luckily, this can be fixed with a few additional lines of code.

elif deltaWaves >= 1.7 and allValues[-2] >= 1:
    print("Blink!")          

Put this code right under the previous part

If the delta wave in the lines above is larger than 1.7 and the delta wave before it was more than 1, the computer will recognise a blink. How does one go about doing this? The delta waves were supposed to be above one after one blink, however testing revealed that they were never higher than 1.7. But if you blink immediately following the initial blink, the figure will go above 1.7.

With this knowledge, we can determine that there was a previous blink but that the present delta wave is a new blink when the delta wave surpasses 1.7 and the previous delta wave is bigger than 1.

Step 8

We are now done with the blink detection part, now we need to do the “making the dino jump” part, I promise this is easy.

To do this, go to the top of your code and import pyautogui. Pyautogui enables the code to control hardware such as the mouse and keyboard. We will use pyautogui to control the space bar because the dino game demands you to press the space button.

To import it, copy this code and paste it in line 20, like this:

import pyautogui

Don’t forget to install pyautogui by pasting this into the terminal:

pip3 install pyautogui

or

pip install pyautogui

Step 9

Since our goal is to make the dino jump, we can do this by pressing the space bar, so paste this code right here:

pyautogui.press('space')

The final version of your code should resemble this. The word “Blink!” will print each time a blink is detected, and Pyautogui will also push your space key at the same time!

Step 10: HAVE FUN!!!!

Congratulations, you have followed the endless steps above to finally cure your laziness by being able to play the dinosaur game with just your eyes.

Although, you are not restricted. You can also do anything else that requires the space bar such as: watching instagram reals, pausing/playing a video, flappy bird, or the jump key on almost any video game!

The possibilties are endless!

If you would like to see the science and demonstrations of the game, be sure to check out my video here: