How to improve your Machine Learning model (hands-on example)

Source: Magenta

Every beginner in Machine Learning has a question on what the logic behind improving a model is. YouTube tutorials seem to be a complete waste as they just provide the code without explaining how and why things are done.

In this article, we will cover the following

• Making an entire Machine Learning model using Tensorflow from scratch
• Analyze its flaws and predictions
• Slowly make improvements until we have achieved a desirable result

Before starting, I have assumed that you have a little bit of background in Python and Tensorflow (optional but recommended)

Making The Model

We’ll be creating a linear regression model using a very small sample.

If you want to follow along, I’ll recommend pasting every single line of code into Google Colab in order, or if you want to access the full code, here’s the GitHub link

1. Create a new Google Colab file
2. Import Tensorflow and other required dependencies

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
tf.__version__

I am currently using Tensorflow version 2.15.0.

numpy is the Python library for numbers and matplotlib is used for visualizing data.

3. Create a dataset

X = np.array([-7.0, -4.0, -1.0, 2.0, 5.0, 8.0, 11.0, 14.0])#independent variable
y = np.array([3.0, 6.0, 9.0, 12.0, 15.0, 18.0, 21.0, 24.0])#dependent variable
plt.scatter(X,y)

We created datasets X and y. Do you see any relation between them🤔?

Well, it’s y = X + 10 (eg. -7 + 10 = 3). See the first term of both X and y

Our goal here is to predict the value of y when X is inputted. We can easily do that if we have this equation but we’ll try to make something so that the machine understands.

plt.scatter() plots both X and y.

Screenshot from Google Colab

4. Labelling for Machine Learning

Labelling is a process in which we mark up X and y values and let the machine know what we are going to do (it will make more sense after you write the code)

X = tf.constant(X, dtype=tf.float32)
y = tf.constant(y, dtype=tf.float32)
X = tf.reshape(X, (-1, 1))

First, we need to convert our X and y values into ‘tensors’. Since we are going to use Tensorflow, we need all our data in the form of tensors.

Now let’s get to the main part of making our very own model

5. Finally making the model

tf.random.set_seed(42)

#1. Create the model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1)
])

#2. Compile the model
model.compile(loss = tf.keras.losses.mae,
              optimizer = tf.keras.optimizers.SGD(),
              metrics = ["mae"])

#Fit the model
model.fit(X,y, epochs = 5)

Seems like a lot of complex code, but this is what it means. Before that keep this image in mind that showcases the architecture of a deep learning model.

Source: Analytics Vidhya

We have created a model that has a dense (hidden) layer. In the model.compile code we are using ‘mae’ and ‘SGD’.

mae refers to ‘Mean Absolute Error’. Let’s take an example to understand this. If your model is inputted the value 17 you know it should predict 27 according to our equation. But if our model predicts 30, then our error is

30–27 = 3

We are using a Stochastic Gradient Descent

Stochastic gradient descent is an optimization algorithm often used in machine learning applications to find the model parameters that correspond to the best fit between predicted and actual outputs. It’s an inexact but powerful technique.

Then we fit the model into the data. Epochs refers to the number of times we want our model to learn

6. Analyzing our code

Let’s run the code.

See, our model started with an error of 15.4813 and ended at 14.6734. That’s a good improvement but a loss of ~14 is actually really bad.

y_pred = model.predict([17])
y_pred

Output Screenshot

Let the mode try and predict a value with 17. We should get something approximate to 27, but we got -5.

Improving Our Model

• Create Model: add more layers, increase the number of hidden layers, change the activation
• Compiling model: change the optimizing function or the learning rate
• Fitting the model: here we might fit a model for more epochs or on more data

These are the three most common ways to improve the efficiency of our model. Since we saw that our model was improving each epoch we can go ahead with option 3. Let’s try 100 epochs.

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1)
])

model.compile(loss = tf.keras.losses.mae,
              optimizer = tf.keras.optimizers.SGD(),
              metrics = ["mae"])
model.fit(X, y, epochs = 100)

The only change made was epochs = 100

After running the model, we got close to an error of just 6. That’s fantastic.

Now our model is predicting a value of 29, which is very close to our answer.

We can even try and create more changes. For example, changing the number of dense layers. Here’s an example

model = tf.keras.Sequential([
    tf.keras.layers.Dense(100, activation="relu"),
    tf.keras.layers.Dense(1),

])

model.compile(loss = 'mae',
              optimizer = tf.keras.optimizers.SGD(),
              metrics = ["mae"])
model.fit(X,y, epochs=100)

This is a very simple model, but a good starting point.

Do you want complex models including all the features like train_test_split, etc?

Get the full code on my GitHub

Follow on X (Twitter) for more content