Making a neural network learn to play a game — Part 2 — Interacting with the game

In the last post I kind of gave a short introduction on how the project would work and in this post, we are actually going to start the code.

As I explained before, I coded the game in PyGame but I am not going to talk about how I did that since that is not really the purpose of the series. Rather, it is far better that you just download the game and learn how to use it in your own code and work with it. For the most curious of you, who want to know what is happening and how, I have included a lot of comments and the code itself should be pretty simple to understand.

Do a git pull from my git repository to get the game here.

From this point on, I am assuming that you have downloaded the game and have installed pygame with pip. You can look at the requirements.txt if you are having problems with a certain version of pygame.

Anyway, moving on, let’s start by actually running the game.

To do that, just run -

python game.py

You should get something similar to this -

Now, I will quickly tell the controls. Spacebar is pretty self explanatory. Use ‘W’ and ‘S’ to increase and decrease the overall speed of the game(this actually changes the frame rate). Note that the neural network won’t actually care about the speed of the game, it runs after a certain number of frames not seconds. This means that if it runs after, let’s say, 50 frames then if the game is running at 50 FPS(Frames per second), it would run once every second and if the game is running at 100 FPS, it would run twice every second. The neural network, being a computer program, would adapt itself to work with the game speed. I included W and S key responses only for the visual aspect of seeing the neural network make decisions when you want to by lowering the frame rate and fast forwarding through the learning by increasing the frame rate.

Anyway, let’s move further on how to make a program interact with the game. You might remember this diagram from the previous post -

We have the game part of the diagram, now to make the neural network part that would tell the game what to do. This is a pretty challenging task right now so let’s put our focus on something much simpler, making a program that interacts with the game just as our neural network would but without the actual neural network. Instead of the neural network making the calls, we will let the user make the calls on behalf of the neural network. I know this may sound a bit weird but coding the neural network won’t be much difficult after this.

Let’s create the file “nn.py”. While it does say nn(which means neural network), as I explained, the neural network portion comes later.

We start by understanding a bit to how exactly the neural network would interact with the game.

The story starts with a class. Let’s call it a Wrapper. This wrapper class’s init method really doesn’t do a lot except it calls the function “controlled_run()” in game.py. This is the starting of communication between the neural network and the game. Our neural network has told the game to start. It passes two key pieces of information, one is itself and the other is an integer that just tells the amount of times the game has been run. It has a method called control. The game calls this method control and passes a dictionary containing important information, such as the current score, how far the next enemy is etc. Then, the neural network performs what it needs to do in the control function. and at the end of the function is returns the control back to the game. Let me illustrate it better with a diagram.

I hope you have a rough idea of what is happening. Enough with the explanation.

Let’s start coding!

We will start by writing the skeleton of the class Wrapper.

It should be pretty clear what is happening now. Let’s start by filling the functions with actual pieces of code.

Again, we will start small.

For the time being, forget about the neural network, and let’s just focus on communicating with the game. A function call to controlled run already exists and this would start the game. The game would then call the function control after some time and then we need to make a prediction and return it to the game.

We can tell the game to do either of two things right now, either to jump or to do nothing.

To make it easier to read, I have defined two variables in game.py which we can import. These variables are DO_NOTHING and JUMP. We will return one of these in the end of control function. Once the game is complete, the game would call the function gameover to do cleanup stuff.

Let’s return the JUMP action in control() and run our program.

Try to run the program and if your code is exactly like the listing above, it should run fine.

The player would constantly jump and it might even get a little bit of score. Also, you can see the kind of values you would get in the function control from the game.

This is what that dict looks like when the game started.

{'action': 0, 'closest_enemy': -1, 'score_increased': 0, 'old_closest_enemy': -1}

Here, the action is the action that was performed in the last run of the neural network, the closest enemy is the number of pixels away the nearest enemy is to the player (we will use this for prediction)(it is -1 if there is no enemy), score increased is 0 if the score wasn’t increased and 1 if the score was increased after the last action and finally old_closest_enemy is the number of pixels between the enemy and the player when the neural network sent its action(we will use this for training).

Here is another copy of the values dict when enemies actually existed.

{'action': 1, 'closest_enemy': 667, 'score_increased': 0, 'old_closest_enemy': 976}

There is a lot to digest here so take a while if you need to and make sure you have some idea of what I am trying to say in the last few lines.

Now, that we have some idea of the kind of information we get, let’s ask the user to give input to what the player should do.

I haven’t changed a lot here and the code should make sense. Run the program again and see what you get.

You should now be right in the neural network’s position. Getting the values dict and returning whether the player should jump or not. This is exactly what the neural network should do.

Finally, let’s handle the gameover function so that we can restart the game every time you fail. I am going to declare a couple of global variables, one indicating the total number of games we can play and one counting the games. This is so that we can eventually exit the program.

We have now written the gameover function. One tiny little detail, whenever we call the function controlled_run, we pass in the second parameter as 0. The second parameter should indicate the number of games we have played. 0 is obviously not correct. Luckily we have a global variable games_count recording the number of games we play and we can just that here. With that tiny little change, this is how your code should look -

Now, we can see the number of game we are on in the title of the window!

That is awesome!

All we need to do now is to put a neural network in place of the user to play the game. Let’s do this in the next post. Good job for following till now if you are still here!

All the code is available on my git here

And here is the next part.