Creating a basic linear regression neural network using Tensorflow

I’m currently interested in utilizing machine learning to create predictive models. So I decided to create a simple 2 variable prediction model from scratch, meaning I only utilized the concepts and documentation of Tensorflow to construct the model. As a result, my model is quite lengthy for something so simple since I didn’t abstract many things.

Step 1: Imports

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import random

I used numpy as the library to create matrices.

In order to visualize the actual line of best fit, I need a library which allows me to graph functions and points. For this purpose, I chose matlplotlib.

I imported random to randomly initialize numbers to fill my training sets to make sure my model can accurately create a line of best fit for all possible data points.

Step 2: Initializing variables

#params
learning_rate = 0.01
training_epochs = 6000
steps = 100

In this block of code, I set how long my variable will train for (6000 epochs) and much we adjust the weights and biases per step (learning rate). The steps variable will be utilized later.

Step 3: Initializing datasets

#training sets, a two arrays with random values 
trainX = np.random.rand(9)
trainY = np.random.rand(9)

I created two 1-D arrays that can hold 9 values each and filled them with values between 0–1. TrainX represents the X values of the data points and TrainY represents the Y values. These will be combined into tuples later.

Step 4: Initializing weights and biases

random = random.uniform(0,20)
#weights and biases
W = tf.Variable((random), name = "Weight")
b = tf.Variable((random), name = "Bias")

I initialized my weights and biases using tf.Variable, setting them to a random value between 0–20. These will be subject to change during optimization

Step 5: Create a linear function

#linear model 
prediction = tf.add(tf.multiply(X, W), b)

My hypothesis function

In this snippet, I created the function for the line of best fit. The bias represents the y-intercept and the weight represents the slope. The goal of the model is to set the bias and weight with the least difference between the data points and the line itself.

Step 6: Cost function

#I only have 9 data points in the dataset
n_samples = 9 
cost = tf.reduce_sum(tf.pow(pred-Y, 2))/(2*n_samples)

The MSE cost function

This function is an MSE (Mean squared error) that shows how far off the line of best fit is from the data values that I gave it back in step 3. The m in the picture is the same as n_samples which represent how many different data points are in the set (this was declared as 9 during step 3). The larger the output is, the greater the difference between the predicted Y value from the line of best fit for a TrainX value and the TrainY value for that TrainX value.

Step 7: Set up gradient descent and initialize variables

#minimize cost taking steps of 0.01 down the parabola 
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)

I set up the gradient descent algorithm by creating an optimizer variable that will minimize the output of the cost function by changing the weights and biases.

Step 8: Run the session

#initialize all variables
init = tf.global_variables_initializer()
with tf.Session() as sess:
# Run the initializer
 sess.run(init)

Once all variables are initialized and everything is set up, it’s time to begin running the actual session by initializing all the previously declared variables.

Step 9: Begin optimization

# Fit all training data
    for epoch in range(training_epochs):
        #zip the training values together into tuples
        
        for (x, y) in zip(trainX, trainY):
            #feed the optimizer the training values
            sess.run(optimizer, feed_dict={X: x, Y: y})

These two lines of code put the same index of trainX and trainY into a tuple and send it into the optimizer which utilizes gradient descent to figure out how the weights need to be adjusted to minimize cost. This process repeats as many times as the set amount of training_epochs (in this case, 6000 times).

Step 10: (OPTIONAL) Print out the cost, weight value and bias value during a certain epoch increment

if (epoch+1) % steps == 0:
            c = sess.run(cost, feed_dict={X: trainX, Y: trainY})
            print("Epoch:" + str((epoch+1)), "cost=" + str(c),  "W=" + str(sess.run(W)), "b=" + str(sess.run(b)))

I wanted to see my MSE, weights and bias values every 100 steps in the optimization process(initialized back in step 2) since it helped me see exactly what was happening during the session. This is purely optional and does not contribute to the model in any way.

The output of the print statement on a randomly generated set of X and Y values

Step 11: Plot the new line of best fit along with the training points on a graph

    plt.plot(trainX, trainY, 'ro', label='Original data')
    plt.plot(trainX, sess.run(W) * trainX + sess.run(b), label='Fitted line')
    plt.legend()
    plt.show()

First I plot my training points and then I plot my line of best fit using the weights and bias values determined during the session.

The plotted line of best fit

That's all there is to the model! Here I only utilized 9 random values, but feel free to go ahead and import some datasets. Pick out two features with some correlation to each other and isolate their data values from the actual dataset using pandas. After that, you could set one as the X variable (Independent) and one as the Y variable (Dependent) and see the trend between both features!