Using my Brain to play the Brick Game

Recently, I have been interested in the Brain-Computer Interface (BCI) technology. BCI’s use the brain to create and send responses to a computer or a machine. The computer or machine can then harness and interpret these responses. This is an interesting technology that has many applications. If you want an overview of BCI technology click here.

Around a month ago, I started playing around with this technology. I worked on a game that when the user blinks, with the selected eye, the dinosaur jumps. However, I wanted to see if I could utilize both eyes. For this to work a 2-key game would be needed, so I used the Brick game. Before I explain the code behind this, I would like to explain the hardware behind this.

Hardware

For this project, I used the Muse Model 2, (an EEG headset) as the Muse Model 1 cannot connect with Web Bluetooth.

In the code, the AF7 electrode (see diagram above) was used to look for a certain frequency that would indicate whether I blinked or not.

Web Bluetooth

In order to connect my headset to the computer, I utilized Web Bluetooth on Google Chrome. This is one of the most secure ways for Bluetooth connection between websites and external Bluetooth devices.

Code

First, the code of the original game was copied from Modzilla. Next in order for the game to work the necessary packages need to be installed. Then, a new terminal was created and a new NPM project was initialized.

npm init -y

The Muse-JS was installed which contained the EEG packages which were needed for the game to work. This was comprised of channels, EEG readings, and specific electrodes.

npm install — save muse-js

Next, SystemJS which is a configurable module loader was installed. This installed the necessary libraries. This module will be needed when later working with .map and .filter.

npm install — save systemjs

Index.Html

This is the original code from the Modzilla Brick game html page. This will only tell how big the canvas has to be, so a few changes need to be made.

<canvas id=”myCanvas” width=”480" height=”320"></canvas>

First, the following code will be inserted in the HTML page. The code loads SystemJS and tells the computer where to find the rxjs and muse-js modules. The 18th line tells SystemJs to load the brain.js file. Another system could have been used to handle the modules, however, SystemJS required less configuration.

<script src=”node_modules/systemjs/dist/system.js”></script>
<script>
System.config({
packages: {
‘muse-js’: {
defaultJSExtensions: true,
main: ‘muse.js’,
},
‘rxjs’: {
defaultJSExtensions: true,
}
},
map: {
‘rxjs’: ‘node_modules/rxjs/’,
‘muse-js’: ‘node_modules/muse-js/dist’
}
});
SystemJS.import(‘brain.js’);
</script>

Next at the bottom of the page, the canvas size will be changed and a button for connection will be added.

<canvas id=”myCanvas” width=”960" height=”640"></canvas>
<script type=”text/javascript” src=”Breakout.js”></script>
<div>
<button onclick=”connectToMuse()”> Connect </button>
</div>

Brain.js

A new javascript file will be created, this will tell the computer what happens when a blink is detected and the range that will be looked for.

const { MuseClient, channelNames } = require('muse-js');
require('rxjs/add/operator/filter');
require('rxjs/add/operator/map');

async function connectToMuse() {
const client = new MuseClient();
await client.connect();
await client.start();
const leftChannel = channelNames.indexOf('AF7'); // Left eye electrode
const blinks = client.eegReadings
.filter(r => r.electrode === leftChannel)
.map(r => Math.max(...r.samples.map(n => Math.abs(n))))
.filter(max => max > 500);

Lines 1–8 calls the files we need for the code to work and tells the program to wait until muse is connected. Next, in line 10–11 we create new constants.*

The first const tells the program that we are looking for left eye detection(AF7 electrode). Lines 12–14, the program is instructed which electrode needs to be isolated. Then the program is told that we are looking for values greater than 500, which are the blink looking for.

blinks.subscribe(() => {
console.log('Blinkheard!');
const leftEvent = new Event('keydown');
leftEvent.keyCode = 37; // Space key
document.dispatchEvent(leftEvent);
});

In the code above, line 1 is waiting for a blink event. In the game, you press the left arrow key to move the paddle to the right, so lines 18–20 simulate a dom key event. Now when the user blinks, it’s interpreted as hitting the left arrow key.

At this point, I thought the game would work, with the left eye, but I was wrong. When I blinked the paddle would keep moving in that direction and wouldn't stop. To solve this issue, the following code below was inserted.

const blinksstop = client.eegReadings
.filter(r => r.electrode === leftChannel)
.map(r => Math.max(...r.samples.map(n => Math.abs(n))))
.filter(max => max < 40);
blinksstop.subscribe(() => {
console.log('Blink!');
const leftEvent = new Event('keyup');
leftEvent.keyCode = 37; // Space key
// console.log(leftEvent)
document.dispatchEvent(leftEvent);
});

The code is looking for values that are less than 40. When your eyes open the value is around 20, but 40 was used to allow for some room for error. Now when your eyes are open, a new keyup event is created stopping the left key from being activated.

Right now, the Brain.js file should detect left eye artifacts however we still need to add one for the right. In order to do this, the code above will be copied, but this time the const will be changed to rightEvent, rightChannel, blinks2, blinksstop2, and the key code will be 39.

const rightChannel = channelNames.indexOf(‘AF8’); // Left eye electrode
const blinks2 = client.eegReadings
.filter(r => r.electrode === rightChannel)
.map(r => Math.max(…r.samples.map(n => Math.abs(n))))
.filter(max => max > 500);
blinks2.subscribe(() => {
console.log(‘Blinkright!’);
const rightEvent = new Event(‘keydown’);
rightEvent.keyCode = 39; // Space key
// console.log(rightEvent)
document.dispatchEvent(rightEvent);
});
const blinksstop2 = client.eegReadings
.filter(r => r.electrode === rightChannel)
.map(r => Math.max(…r.samples.map(n => Math.abs(n))))
.filter(max => max < 40);
blinksstop2.subscribe(() => {
console.log(‘Blink2!’);
const rightEvent = new Event(‘keyup’);
rightEvent.keyCode = 39; // Space key
// console.log(rightEvent)
document.dispatchEvent(rightEvent);
});

Finally, for the Brain.Js file, the connectToMuse component needs to be added in a simple line.

window.connectToMuse = connectToMuse;

Breakout.Js

In the Breakout Js file, some of the code was changed to make it easier to work with. First, the ball speed was made slower by changing the dx and dy values. Then the paddle height and width were changed as the canvas, as noted above, was made bigger in the HTML file.

var dx = 0.5;
var dy = -0.5;
var paddleHeight = 20;
var paddleWidth = 150;

Next, the key function was changed to use keyCode for the movement. This means that originally, to move to the left and to the right, the statement was “arrow right” and “arrow left.” However, in order for this to work, we need to use keycode which means now left and right are represented by “37” and “39.” This made it easier to work with as the game components were more consistent.

function keyDownHandler(e) {
if(e.keyCode === 39) {
rightPressed = true;
}
else if(e.keyCode === 37) {
leftPressed = true;
}
}
function keyUpHandler(e) {
if(e.keyCode === 39) {
rightPressed = false;
}
else if(e.keyCode === 37) {
leftPressed = false;
}
}

Demo

In order for the game to run, a web server will be needed to serve the HTML page. I used live-server in order to run the project, however, there are many other options out there.

In the future, I hope to use the power of BCI technology to help people or create technology that people want. For my next project, I plan to create software that can analyze emotions using BCI’s. There is still so much that can be done with BCI technology and I look forward to learning more about it.

• *A constant(const) is different than a variable as it will only have one value. This is useful as we will be calling these const many times in the code.
• *It is similar to an if statement but in this case, it calls back to the “.map” and “.filter” statements from earlier.