Implementing neural networks in matlab 105

Shaun Enslin
Jul 9 · 4 min read

Lets implement a neural network to classify customers according to their key features. Running neural networks in matlab is quite understandable once you understand the equations.

This is part 5 in my series on neural networks. You are welcome to start at part 1. Here are the previous articles explaining the cost function.

  1. Forward propagation
  2. Backward propagation
  3. Calculate cost & gradient
  4. Calculate gradient

Source code used here can be downloaded from github. Spoiler alert, we will only get a 66% accuracy, but, thats the data we have ;-)

Just as a refresher, here is the dataset we downloaded from kaggle, while you can see this article to see why we chose the 3 features below.

Figure 1

The main program

We will have a few utility functions, but those will be covered seperatly. Lets discuss first the main program which will have the following steps:

  1. Load the data
  2. Label our string fields to numeric values
  3. Setup our neural network sizes
  4. Initialise thetas / weights
  5. Cost optimisation to find the best theta
  6. Use our best theta for a prediction to calculate our training set accuracy

Lets discuss these one by one

clear;% open csv file
tbl = readtable(‘test.csv’);
% replace strings fields with labels and create dataset matrix
ds(:,1) = grp2idx(tbl{:,2});
ds(:,2) = grp2idx(tbl{:,3});
ds(:,3) = tbl{:,4};
ds(:,4) = grp2idx(tbl{:,5});
ds(:,5) = grp2idx(tbl{:,6});
ds(:,6) = tbl{:,7};
ds(:,7) = grp2idx(tbl{:,8});
ds(:,8) = tbl{:,9};
[ds(:,9),labels] = grp2idx(tbl{:,10});
% remove rows with NaN in any field values
ds = rmmissing(ds);
[m,n] = size(ds);
% Create X and y matrix
X = ds(:,[2 4 8]);
y = ds(:,n);

After running this step, you should see the following results in X and Y matrix’s.

Figure 2

Then, we can setup the sizes of our neural network, first, below is the neural network we want to put together.

Figure 3

Below initialisations, ensure above network is achieved.

input_layer_size = size(X,2);           % Dimension of features
hidden_layer_size = input_layer_size*2; % # of units in hidden layer
output_layer_size = size(labels,1); % number of labels

Now, we can initialise our theta’s to random small values (remember these are also called weights), you can see the result in figure 4.

% initialise 2 thetas for hidden layer and output layer
initial_Theta1 = initializeWeights(input_layer_size, hidden_layer_size);
initial_Theta2 = initializeWeights(hidden_layer_size, output_layer_size);
% Unroll parametersinitial_nn_params = [initial_Theta1(:) ; initial_Theta2(:)];
Figure 4

Now, we can run our fminunc cost optimisation to find the best theta’s. We will create the myCostFunction later.

% Run cost function to find lowest cost thetas
options = optimset(‘MaxIter’, 50);
lambda = 1;
costFunction = @(p) myCostFunction(p, input_layer_size, hidden_layer_size, output_layer_size, X, y, lambda);
[nn_params, ~] = fminunc(costFunction, initial_nn_params, options);

Now we have our best theta’s, lets reshape back into 2 sets of theta’s so we can check for accuracy.

% Obtain Theta1 and Theta2 back from nn_params
Theta1 = reshape(nn_params(1:hidden_layer_size * (input_layer_size + 1)), hidden_layer_size, (input_layer_size + 1));
Theta2 = reshape(nn_params((1 + (hidden_layer_size * (input_layer_size + 1))):end), output_layer_size, (hidden_layer_size + 1));

Now that we have our theta’s, we can keep them for later predictions, but lets use them, now check for our training set accuracy.

pred = predict(Theta1, Theta2, X);
fprintf(‘\nTraining Set Accuracy: %f\n’, mean(double(pred == y)) * 100);

After running your solution, you should now see below:

myCostFunction

I will paste the code here for reference, but please go through parts 2, 3 & 4 in this series for a detailed explanation forward, backward propagation as well as calculating the cost and gradient. Code also available in github.

Other utility functions

I wont go into detail into the other utility functions as many are covered in the previous articles in this series. Again, their code is in github and they are listed below:

  1. sigmoidGradient
  2. sigmoid
  3. initializeWeights

Conclusion

I hope that this series has given you a good overview of implementing neural networks. I have given you screen shots of the results to visualise each step and hope that helps.

Just a final word, is this course was fantastic. If you want to become an expert, then this is the course for you.

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data…

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Shaun Enslin

Written by

Coding, technology and data are my passions. Oh, and some crypto trading with lots of cycling on the side. https://www.linkedin.com/in/shaun-enslin-4984bb14b/

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com