Creating a Gaming-AI with Reinforcement Learning

4 min readFeb 10, 2018

I’m a passionate gamer ever since I was a kid. I started with Super Nintendo (SNES), then I went into computer gaming. For some years now, I’ve been playing on the PlayStation.

So, the gaming industry is very near to my heart.

That’s why the appliance of data science in the gaming industry, especially AI for bots, or AI for computer games in general, really interests me.

Over the past week I’ve been looking into how to get started.

Creating my own AI I can play against.

OpenAI & Elon Musk

It turns out this is actually a big trend. The company OpenAI, co-founded by Elon Musk, is especially big into that.

There’s OpenAI Gym, like gymnasium and OpenAI Universe. Two tools where you can train your AI. On for instance Atari games, or flash games, or browser games, or with Universe any game you want.

It’s very interesting. I’ve been looking to find a niche that interests me a lot and that I can deep dive into. I think this is it.

I always tell people, that they need to find a project that excites them, and just start. Projects help you get practical experience.

So I’m, going to look at selecting a game and then trying to build a AI for that game. The long-term goal for me, is to build in an AI that I can play against.

Reinforcement Learning

How reinforcement learning works is, the algorithm plays one round of the game. After that round he gets a feedback how he performed.

If the performance was good he gets rewarded. If he performed bad, he gets penalized. This then influences the building of the next model.

This feedback lets the AI tune itself over time, so it will hopefully learn and perform better.

Schooled or Apprenticeship Learning

One way to train the algorithm is called schooled or apprenticeship learning. You are teaching the AI by playing the game and letting the AI watch.

The AI then learns and after the schooling phase you let the AI play and learn by itself. The upside of this is, that the algorithm already has some idea how stuff works. It doesn’t start from zero.

Learning Completely From Scratch

The other way is that you start completely fresh. You don’t teach the AI, you don’t let it watch how you play.

You just let it play and let it fail, and fail, and fail. Over time it will learn. Hopefully.

How OpenAI Universe Works

Universe lets you play and learn any game.

How that works is you have two machines: One machine where the actual game and a VNC server is running on and another one where you learn.

The machine where the game runs also has a VNC server. This way the game can be remotely controlled from another computer. What that system also does is, it delivers the performance feedback for the reinforcement learning.

The second system is connected to the VNC server and plays the game. It is also running the actual algorithms that train the AI models. It gets the feedback for the reinforcement learning.

Deploy In The Cloud

Both machines are dockerized, so you can deploy them in the cloud if you want. So you can run the whole process either your PC at home, or you can use cloud instances.

If you like me don’t have a huge gaming system with strong GPUs at home this is the best choice. I for instance only have a MacBook Pro and that is not optimal for for building machine learning models.

Just get two cloud servers and run the game and the actual building of models, the actual learning there. You can deploy for instance on GPU instances to run and train the algorithm. This way you don’t need to use your own computer.

It makes it very easy and cost efficient for beginners, because you don’t need to buy expensive hardware, like Titan X’es or so. You just rent the computing power in the cloud, use it and when you’re finished, switch it off.

Watch the AI Play & Give It a Hand

While, the algorithm plays the game and learns in the cloud, you can also watch how it performs. Over the VNC connection you can connect to the instance where the game is running on.

Watching the algorithm play and checking out how he learns very interesting. you can also influence what is happening by taking over the controls and giving the Ai a hand.

This makes sense if something isn’t working right, or in the case of apprenticeship learning.

Learning Data Science By Doing

The cool thing is that you have an environment that you can easily set up, work and learn with.

You will learn to code in Python, how to use for instance tensorflow, get hands on experience with deep learnng and so on. All that by having fun while createing an AI that is hopefully able to learn and play a wide range of games.

You can start with pong one of the all-time greatest and it goes up until Doom, or I have seen on YouTube guys that create AIs playing Grand Theft Auto 5.

It’s a wide range of games, very, very interesting sector. I read on Wikipedia OpenAI has been collecting about one billion in funding.

It’s a crazy effort to make that technology public, to give that chance and power to people.

Don’t miss my Data Science and Big Data Podcast on SoundCloud

Subscribe to my newsletter: Here