Making a neural network learn to play a game — Part 3 — Actually coding the neural network!

So, in the last post we interacted the neural network and the game interacted with each other and the game worked by user input.

I was thinking whether I make this post about visualizing with graphs or implementing the neural network and I am gonna go with the neural network.

There are a few things we need to do :

1. Get the training data
2. Implementing the neural network
3. Making the neural network do random things to train it

Get the training data

What we need to do is create a couple of lists(by lists, I mean numpy lists) x_train and y_train and whenever our neural network makes a correct decision(i.e. gets an increase in score), we append the distance between the player and the enemy into x_train and the decision made into y_train. This forms our training_data.

The first step is to create the x_train and y_train lists above our definition of Wrapper class. Then, whenever a decision is made, check if score was increased, if it was append it to our lists.

Just one more thing before we put all this in code. Maybe most of you already know this but to train a neural network properly, the inputs need to be in the range of 0–1. This is not true for our case. However, all we need to do is a simple division to fix this. The maximum distance between any enemy and the player is around 1000. So, let’s just divide by 1000.

Finally, let’s output the numpy arrays in the gameover function.

This is how the code will look like -

You can now run the game and see if the changes work the way they should.

Let’s move on to adding the neural network.

Implementing the neural network

As most of you may have figured out, the neural network can be really simple. This is a simple linear classification. So, while you can try out any number of layers and neurons in each layer you want and later on, the neural network will need to get more and more complex, right now, we can just work with a neural network of 2 layers. one neuron in its first layer and two output neurons.

Before we create the neural network, we need to figure out when is it going to learn. I was teaching the neural network once every 10 games and I am going to implement that. You can change that as you want. It is a simple if statement in gameover function. When we train the neural network, we will also empty out the x_train and y_train arrays.

Okay, so back to the neural network. This is how we will implement it -

You can now run the code but you will see a little flaw in the way our neural network works. It always jumps. Now, this is totally random behavior in the beginning that every prediction is to jump which means that the neural network never learns to simply do nothing. We need to put some randomness into the code.

Adding randomness to the neural network

You can do it any way you want but my general idea was that that the randomness should be huge at first and then slowly reduce. I began experimenting with some math functions to do that, and I didn’t spend a lot of time on it. I just experimented with a few quadratic and linear functions but the best I found was a very simple linear function.

Desmos graph

y = 50(1-x/50)

Here, y is the probability that the action taken will be the opposite of the neural network’s prediction and x is the run counter. As the run_counter increases, the random rate declines and eventually hits 0 at x=50 or when we get to game number 50.

This is how the code looks like now -

Run the code above and you can see your own neural network learn! (Hit W in the beginning a few times to speed it up).

As usual you can check out the code on my git.

Great job following till this point! In the next post, I will cover drawing graphs and visualizing what the neural network is doing.