Building a NEURAL NET from SCRATCH

What is a neural network?

Neural networks are one of the main tools used in machine learning. As neural suggests, they are brain-inspired systems which are intended to replicate the way that we humans learn. NNs consist of input and output layers, as well as a hidden layer consisting of units that transform the input.

Photo by timJ on Unsplash

They are excellent tools for finding patterns which are far too complex or numerous for a human programmer to extract and teach the machine to recognize.

Steps for this mini-tutorial:

  • Define independent and dependent variables.

Step #0: Libraries.

import numpy as np

Step #1: Variables.

# Step #1: Variables.

## trainnig data independent variable (x)
training_set = np.array([[0,1,0], # 3 features, 7 entries
[0,0,1],
[1,0,0],
[1,1,0],
[1,1,1],
[0,1,1],
[0,1,0]])
labels = np.array([[1,
0,
0,
1,
1,
0,
1]])
labels
array([[1, 0, 0, 1, 1, 0, 1]])# reshaping our dependent variable
labels = labels.reshape(7,1)
labels
array([[1],
[0],
[0],
[1],
[1],
[0],
[1]])

Our input set contains seven records. Similarly, we also created a labels set that contains corresponding labels for each record in the input set. The labels are the values that we want our ANN to predict.

Step #2: Hyper-paramaters

Here Random:

  • Seed helps get the same values upon recursive execution
#hyperparameters
np.random.seed(42)
weights = np.random.rand(3,1)
bias = np.random.rand(1)
lr = 0.05

Step #3: Activation function.

The sigmoid function returns 0.5 when the input is 0. It returns a value close to 1 if the input is a large positive number. In the case of negative input, the sigmoid function outputs a value close to zero.

Quick tip: Scroll down on our page to find a dedicated post on ‘Sigmoid Function’ , the pros and cons!

# methods
def sigmoid(x):
return 1/(1+np.exp(-x))
def sigmoid_derivative(x):
return sigmoid(x)*(1-sigmoid(x))

Step #4: Training.

In the context of machine learning, an epoch is one complete pass through the training data. A deep neural network has to be trained for multiple epochs.

#training our model
for epoch in range(30000):
inputs= training_set
XW = np.dot(inputs, weights)+bias
z = sigmoid(XW)
error = z -labels
print(error.sum())
dcost = error
dpred = sigmoid_derivative(z)
z_del = dcost * dpred
inputs = training_set.T
weights = weights - lr*np.dot(inputs, z_del)
for num in z_del:
bias = bias - lr*num

inputs = training_set
1.8806216715619812
1.8525640899325237
1.8243107340899896
1.7958847686438375
1.7673099081827413
1.7386103353186555
1.70981061515949
1.6809356068569574
1.6520103729182125
1.623060087002853
1.5941099409498616
1.5651850517915373
1.5363103695131097
1.5075105863073248
1.4788100480531448
1.4502326687170073
1.4218018483346397
1.3935403951819836
1.3654704526863837
1.3376134315651074
1.3099899476087358
1.282619765453504
...
1.2555217486106485
1.2287138159438014
1.202212904708781
1.1760349401952295
1.1501948119375935
1.1247063563951216
1.0995823459377747
1.074834483918001
1.0504734055578753
1.0265086843374711
1.0029488435338814
0.9798013725310017
0.957072747498054
0.9347684560195374
0.9128930252506302
0.8914500531694373
0.8704422425005396
0.8498714368923037
0.829738658942862
0.8100441496859043
0.790787409166752
0.7719672377610798
0.7535817779124228
0.7356285559897645
0.7181045239925073
0.7010061008564724
0.6843292131409233
0.6680693349025218
0.6522215265873573
0.6367804727964091
0.6217405188029252
0.6070957057219066
0.5928398042522356
0.5789663469307426
0.5654686588547545
0.5523398868453112
0.5395730270373362
0.5271609508956124
0.5150964296665184
0.5033721572851677
0.4919807717659588
0.48091487511165465
0.4701670517820806
0.4597298857684057
0.4495959763229114
0.4397579523971604
0.4302084858437466
0.4209403034383108
0.41194619777942676
0.4032190371243036
0.39475177421811825
0.3865374541742619
0.3785692214618742
0.37084032605586537
0.36334412880318256
0.3560741060574504
0.349023853632331
0.34218709012202697
0.3355576596353842
0.3291295339879512
0.32289681439431384
0.31685373270087624
0.3109946521971938
0.3053140680418689
0.29980660733698206
0.2944670288830382
0.2892902226444559
0.2842712089537457
0.27940513748070583
0.27468728599120895
0.2701130589184907
0.2656779857682292
0.261377719377197
0.25720803404380344
0.2531648235474717
0.24924409907250666
0.24544198705085274
0.2417547269370134

Coding a feed-forward neural network:

#feed forward
XW = np.dot(inputs, weights)+bias
z = sigmoid(XW)

#error
error = z - labels
print(error.sum())

#determining slope
slope = inputs * dcost * dpred

dcost = error
dpred = sigmoid_derivative(z)
z_del = dcost * dpred
inputs = training_set.T
weights = weights-lr*np.dot(inputs, z_del)

for num in z_del:
bias = bias - lr*num
0.238178668927295

Step #5: Outcomes.

In the first case the output (result) is closer to 0, so will be classified as 0. Second one has the value closer to 1 , so will be classified as 1.

# predicting outcomes
single_pt = np.array([1,0,0])
result = sigmoid(np.dot(single_pt, weights) + bias)
print(result)
[0.46808888]single_pt = np.array([0,1,0])
result = sigmoid(np.dot(single_pt, weights) + bias)
print(result)
[0.70869886]

I hope you like it.

No matter what books or blogs or courses or videos one learns from, when it comes to implementation everything can look like “Outside the Curriculum”.

The best way to learn is by doing! The best way to learn is by teaching what you have learned!

Never give up!

See you on Linkedin!

Other references

Master in Data Science. Passionate about learning new skills. Former branch risk analyst. https://www.linkedin.com/in/oscar-rojo-martin/. www.oscarrojo.es

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store