Build an AI to play F1-race

The F1-race classic

Really ? an AI to play F1 race game ? Well kind of. This story is more over as a helper guide rather than a tutorial to build the whole system.

I started it out as a project as an experiment and built some groundwork which you can further improve by contributing.

Link to the Github repo

The F1-race game

So, lets first talk about the game.

F1-race gameplay

The F1-race game was published by Nintendo(NES) in 1984.

The gameplay involves a racing car which can be run on 3 tracks sorted by the difficulty levels 1, 2 and 3.

The objective is to complete the track within the time frame of 40 seconds. The tracks have some sharp turns and there are some more cars that makes the game a bit interesting.

There are 4 controls: FORWARD, BACKWARD, LEFT and RIGHT. Combination of two controls like FORWARD-LEFT, FORWARD-RIGHT are also allowed.

The Plan

We’ll convert the problem into a supervised learning task(Images -> Labels).

The pipeline we’ll build is as such:

1. Collect data(game frames and keyboard presses) using emulators.
2. Prepare data(crop images and prepare labels).
3. Train the CNN model.
4. Test the predictions.
5. Make the model to play the game on emulator.

1. Collect Data

I’ll use the OpenEmu emulator for running the F1-race game on Mac. if you are on another OS you can use this website to run the game in the browser. We’ll use this script for grabbing game screenshots and simultaneously the keyboard presses.

Be careful in setting the window area of the game to grab screenshots.

All of the images will be saved in the pics1 directory and a CSV file will be created with the image path and the keys pressed.

Try to collect more and more data as possible. I tried with just 10k images.

2. Prepare Data

Refer to the script here.

Here, the images will be cropped for getting only the relevant portion of the images.

cropped image

After that they’ll be resized into 64x64 or 32x32 or your choice of size. Remember that the size of image will reflect upon the computation power used to train the model.

The labels will be converted into one-hot encoding.

Training the model

Now, we’ll train a CNN model on the data we prepared.

This file contains 4 CNN models to experiment with.

And this script trains the model and plots the loss and accuracy graph.

I trained on 10k images using the CNN Model2 in the models.py file and got 81% accuracy on testing data with RGB images of size 32x32.

Playing the game

To play the game refer to this script.

Remember that OpenEmu doesn’t allow direct keyboard inputs from the script to the window. Therefore, we’ll use the browser emulator here.

Be careful in placing the browser window in appropriate positions for taking screenshots.

Despite the accuracy I got of about 81%, the gameplay of the model was not good. It requires a lot of improvement.

But, I tried to set up a pipeline for building the system and also, this can be used for other games where the game problem can be converted into a supervised learning task.

Feel free to fork the repo and contribute to the project.

After any further improvements in the model, I’ll present to you the next part of this story.

Please click on the 👏 button if you liked the post and hold it for giving more love.

If you wish to connect:

Github LinkedIn Twitter Instagram

This story is published in The Startup, Medium’s largest entrepreneurship publication followed by +431,678 people.

Subscribe to receive our top stories here.