# Building a NEURAL NET from SCRATCH

What is a neural network?

Neural networks are one of the main tools used in machine learning. As neural suggests, they are brain-inspired systems which are intended to replicate the way that we humans learn. NNs consist of input and output layers, as well as a hidden layer consisting of units that transform the input.

They are excellent tools for finding patterns which are far too complex or numerous for a human programmer to extract and teach the machine to recognize.

Steps for this mini-tutorial:

- Define independent and dependent variables.
- Define hyper-parameters.
- Define activation function and the derivative.
- Train the model.
- Make predictions.

# Step #0: Libraries.

`import numpy as np`

# Step #1: Variables.

# Step #1: Variables.

## trainnig data independent variable (x)

training_set = np.array([[0,1,0], # 3 features, 7 entries

[0,0,1],

[1,0,0],

[1,1,0],

[1,1,1],

[0,1,1],

[0,1,0]])

labels = np.array([[1,

0,

0,

1,

1,

0,

1]])

labelsarray([[1, 0, 0, 1, 1, 0, 1]])# reshaping our dependent variable

labels = labels.reshape(7,1)

labelsarray([[1],

[0],

[0],

[1],

[1],

[0],

[1]])

Our input set contains seven records. Similarly, we also created a labels set that contains corresponding labels for each record in the input set. The labels are the values that we want our ANN to predict.

## Step #2: Hyper-paramaters

Here Random:

- Seed helps get the same values upon recursive execution
- Lr is the learning rate; lr is the step size at each iteration while moving towards a minimum of a loss function.

`#hyperparameters`

np.random.seed(42)

weights = np.random.rand(3,1)

bias = np.random.rand(1)

lr = 0.05

## Step #3: Activation function.

The sigmoid function returns 0.5 when the input is 0. It returns a value close to 1 if the input is a large positive number. In the case of negative input, the sigmoid function outputs a value close to zero.

Quick tip: Scroll down on our page to find a dedicated post on ‘Sigmoid Function’ , the pros and cons!

`# methods`

def sigmoid(x):

return 1/(1+np.exp(-x))

def sigmoid_derivative(x):

return sigmoid(x)*(1-sigmoid(x))

## Step #4: Training.

In the context of machine learning, an epoch is one complete pass through the training data. A deep neural network has to be trained for multiple epochs.

#training our model

for epoch in range(30000):

inputs= training_set

XW = np.dot(inputs, weights)+bias

z = sigmoid(XW)

error = z -labels

print(error.sum())

dcost = error

dpred = sigmoid_derivative(z)

z_del = dcost * dpred

inputs = training_set.T

weights = weights - lr*np.dot(inputs, z_del)

for num in z_del:

bias = bias - lr*num

inputs = training_set1.8806216715619812

1.8525640899325237

1.8243107340899896

1.7958847686438375

1.7673099081827413

1.7386103353186555

1.70981061515949

1.6809356068569574

1.6520103729182125

1.623060087002853

1.5941099409498616

1.5651850517915373

1.5363103695131097

1.5075105863073248

1.4788100480531448

1.4502326687170073

1.4218018483346397

1.3935403951819836

1.3654704526863837

1.3376134315651074

1.3099899476087358

1.282619765453504

...

1.2555217486106485

1.2287138159438014

1.202212904708781

1.1760349401952295

1.1501948119375935

1.1247063563951216

1.0995823459377747

1.074834483918001

1.0504734055578753

1.0265086843374711

1.0029488435338814

0.9798013725310017

0.957072747498054

0.9347684560195374

0.9128930252506302

0.8914500531694373

0.8704422425005396

0.8498714368923037

0.829738658942862

0.8100441496859043

0.790787409166752

0.7719672377610798

0.7535817779124228

0.7356285559897645

0.7181045239925073

0.7010061008564724

0.6843292131409233

0.6680693349025218

0.6522215265873573

0.6367804727964091

0.6217405188029252

0.6070957057219066

0.5928398042522356

0.5789663469307426

0.5654686588547545

0.5523398868453112

0.5395730270373362

0.5271609508956124

0.5150964296665184

0.5033721572851677

0.4919807717659588

0.48091487511165465

0.4701670517820806

0.4597298857684057

0.4495959763229114

0.4397579523971604

0.4302084858437466

0.4209403034383108

0.41194619777942676

0.4032190371243036

0.39475177421811825

0.3865374541742619

0.3785692214618742

0.37084032605586537

0.36334412880318256

0.3560741060574504

0.349023853632331

0.34218709012202697

0.3355576596353842

0.3291295339879512

0.32289681439431384

0.31685373270087624

0.3109946521971938

0.3053140680418689

0.29980660733698206

0.2944670288830382

0.2892902226444559

0.2842712089537457

0.27940513748070583

0.27468728599120895

0.2701130589184907

0.2656779857682292

0.261377719377197

0.25720803404380344

0.2531648235474717

0.24924409907250666

0.24544198705085274

0.2417547269370134

## Coding a feed-forward neural network:

#feed forward

XW = np.dot(inputs, weights)+bias

z = sigmoid(XW)

#error

error = z - labels

print(error.sum())

#determining slope

slope = inputs * dcost * dpred

dcost = error

dpred = sigmoid_derivative(z)

z_del = dcost * dpred

inputs = training_set.T

weights = weights-lr*np.dot(inputs, z_del)

for num in z_del:

bias = bias - lr*num0.238178668927295

## Step #5: Outcomes.

In the first case the output (result) is closer to 0, so will be classified as 0. Second one has the value closer to 1 , so will be classified as 1.

# predicting outcomes

single_pt = np.array([1,0,0])

result = sigmoid(np.dot(single_pt, weights) + bias)

print(result)[0.46808888]single_pt = np.array([0,1,0])

result = sigmoid(np.dot(single_pt, weights) + bias)

print(result)[0.70869886]

I hope you like it.

No matter what books or blogs or courses or videos one learns from, when it comes to implementation everything can look like “Outside the Curriculum”.

The best way to learn is by doing! The best way to learn is by teaching what you have learned!

## Never give up!

See you on Linkedin!

# Other references

- Build an Artificial Neural Network From Scratch: Part 1 by Nagesh Singh Chauhan, on KDNuggets.
- What is an artificial neural network? Here’s everything you need to know by Luke Dormehl on DigitalTrends.
- https://www.linkedin.com/posts/mlindia_neural-net-from-scratch-activity-6704638554221395968-w-sQ/