Creating a basic linear regression neural network using Tensorflow
I’m currently interested in utilizing machine learning to create predictive models. So I decided to create a simple 2 variable prediction model from scratch, meaning I only utilized the concepts and documentation of Tensorflow to construct the model. As a result, my model is quite lengthy for something so simple since I didn’t abstract many things.
Step 1: Imports
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import random
I used numpy as the library to create matrices.
In order to visualize the actual line of best fit, I need a library which allows me to graph functions and points. For this purpose, I chose matlplotlib.
I imported random to randomly initialize numbers to fill my training sets to make sure my model can accurately create a line of best fit for all possible data points.
Step 2: Initializing variables
#params
learning_rate = 0.01
training_epochs = 6000
steps = 100
In this block of code, I set how long my variable will train for (6000 epochs) and much we adjust the weights and biases per step (learning rate). The steps variable will be utilized later.
Step 3: Initializing datasets
#training sets, a two arrays with random values
trainX = np.random.rand(9)
trainY = np.random.rand(9)
I created two 1-D arrays that can hold 9 values each and filled them with values between 0–1. TrainX represents the X values of the data points and TrainY represents the Y values. These will be combined into tuples later.
Step 4: Initializing weights and biases
random = random.uniform(0,20)
#weights and biases
W = tf.Variable((random), name = "Weight")
b = tf.Variable((random), name = "Bias")
I initialized my weights and biases using tf.Variable, setting them to a random value between 0–20. These will be subject to change during optimization
Step 5: Create a linear function
#linear model
prediction = tf.add(tf.multiply(X, W), b)
In this snippet, I created the function for the line of best fit. The bias represents the y-intercept and the weight represents the slope. The goal of the model is to set the bias and weight with the least difference between the data points and the line itself.
Step 6: Cost function
#I only have 9 data points in the dataset
n_samples = 9 cost = tf.reduce_sum(tf.pow(pred-Y, 2))/(2*n_samples)
This function is an MSE (Mean squared error) that shows how far off the line of best fit is from the data values that I gave it back in step 3. The m in the picture is the same as n_samples which represent how many different data points are in the set (this was declared as 9 during step 3). The larger the output is, the greater the difference between the predicted Y value from the line of best fit for a TrainX value and the TrainY value for that TrainX value.
Step 7: Set up gradient descent and initialize variables
#minimize cost taking steps of 0.01 down the parabola
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
I set up the gradient descent algorithm by creating an optimizer variable that will minimize the output of the cost function by changing the weights and biases.
Step 8: Run the session
#initialize all variables
init = tf.global_variables_initializer()with tf.Session() as sess:
# Run the initializer
sess.run(init)
Once all variables are initialized and everything is set up, it’s time to begin running the actual session by initializing all the previously declared variables.
Step 9: Begin optimization
# Fit all training data
for epoch in range(training_epochs):
#zip the training values together into tuples
for (x, y) in zip(trainX, trainY):
#feed the optimizer the training values
sess.run(optimizer, feed_dict={X: x, Y: y})
These two lines of code put the same index of trainX and trainY into a tuple and send it into the optimizer which utilizes gradient descent to figure out how the weights need to be adjusted to minimize cost. This process repeats as many times as the set amount of training_epochs (in this case, 6000 times).
Step 10: (OPTIONAL) Print out the cost, weight value and bias value during a certain epoch increment
if (epoch+1) % steps == 0:
c = sess.run(cost, feed_dict={X: trainX, Y: trainY})
print("Epoch:" + str((epoch+1)), "cost=" + str(c), "W=" + str(sess.run(W)), "b=" + str(sess.run(b)))
I wanted to see my MSE, weights and bias values every 100 steps in the optimization process(initialized back in step 2) since it helped me see exactly what was happening during the session. This is purely optional and does not contribute to the model in any way.
Step 11: Plot the new line of best fit along with the training points on a graph
plt.plot(trainX, trainY, 'ro', label='Original data')
plt.plot(trainX, sess.run(W) * trainX + sess.run(b), label='Fitted line')
plt.legend()
plt.show()
First I plot my training points and then I plot my line of best fit using the weights and bias values determined during the session.
That's all there is to the model! Here I only utilized 9 random values, but feel free to go ahead and import some datasets. Pick out two features with some correlation to each other and isolate their data values from the actual dataset using pandas. After that, you could set one as the X variable (Independent) and one as the Y variable (Dependent) and see the trend between both features!