Build a multi-class image classification model with the MNIST dataset.
For beginners|Along with source code
Below are the steps to build a model that can classify handwritten digits with an accuracy of more than 95%. While reading this article I suggest you simultaneously try the code in colab notebook. Follow the steps and observe the outputs.
You can find the complete code on GitHub as well.
1. Prepare the input data
Step-1 Import the required libraries
import numpy as np
import keras
from keras.datasets import mnist
import matplotlib.pyplot as plt- Numpy has many functions that provide support for arrays. An image is nothing but a NumPy array containing pixels of the data points.
- Keras library has many functions that make it really easy to build models.
- Matplotlib helps to plot images and visualize results.
Step-2 Load the MNIST dataset
Dataset- This dataset consists of 60,000 28x28 grayscale images of the 10 digits (0–9), along with a test set of 10,000 images[1]. More info can be found at the MNIST homepage.
(x_train, y_train), (x_test, y_test)=mnist.load_data()Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz 11493376/11490434 [==============================] - 0s 0us/step
Step-3 Reshape the input data
#View the shape of loaded dataset
x_train.shape
x_test.shape
y_train.shape
x_test.shapeThe size of input images is 78x78. But we can pass only flattened arrays in a Deep Neural Network. So let's reshape the images.
#Reshape the datasetx_test=x_test.reshape(-1,784)
x_train=x_train.reshape(-1, 784)
2. Build your model
We will build a Sequential model using Keras layers. In a sequential model output of the previous layer is treated as an input for the next layer.
Step-4 Import the libraries required to build a model
from keras.models import Sequential
from keras.layers import Dense- For building a sequential model we import Sequential class from keras.models.
- We import Dense class to add dense layers in our network. In a dense layer, each neuron of a layer is connected to all the neurons of the next layer. In simple words, a dense layer is a fully connected layer.
Step-5 Add layers to your network
model=Sequential()We created an object of Sequential class called a model. Now we will add the required layers to the model using model.add().
model.add(Dense(units=64, activation=’relu’,input_shape= (784, )))
model.add(Dense(units= 64, activation=’relu’))
model.add(Dense(units= 128, activation=’relu’))
model.add(Dense(units= 64, activation=’relu’))
model.add(Dense(units= 10, activation=’softmax’))In this model, we will add 5 dense layers. For each layer, we have to give values for a few parameters. The two important parameters are units and activation. However, there are many other parameters like kernal_initializer, bias_initializer, etc, which will be initialized with their default values.
- Units: refers to the number of neurons in a layer. We generally increase the number of neurons preferably in the form of 2^n. In multiclass classification number of units in the last layer is equal to a number of different classes.
- Activation: The activation function to be used in each layer. In our model, we have used Relu(Rectified Linear Unit function) in hidden layers and Softmax function in the output layer.
- input_shape: This argument is passed only for the first layer because the first layer does not know what kind of input will be given by the user. Hence we have to give it explicitly. The other layers will get the input of the same shape as the shape of the output of the previous layer.
The output is given as activation(dot(input, kernal)+bias) where activation is the element-wise activation function passed as the activation argument, kernal is a weights matrix created by the layer, and bias is a bias vector created by the layer.
View the model summary if you want
model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense (Dense) (None, 64) 50240 _________________________________________________________________ dense_1 (Dense) (None, 64) 4160 _________________________________________________________________ dense_2 (Dense) (None, 128) 8320 _________________________________________________________________ dense_3 (Dense) (None, 64) 8256 _________________________________________________________________ dense_4 (Dense) (None, 10) 650 ================================================================= Total params: 71,626 Trainable params: 71,626 Non-trainable params: 0 _________________________________________________________________
Step-6 Compile the model
Now we will compile the model and give the following parameters :
- Optimizer: It optimizes the loss function. Here we will use adam optimizer.
- loss: This model is performing multiclass classification so we will use categorical_crossentropy loss.
- metrics: It defines on what basis we have to evaluate our model. Here, we will use accuracy as the metrics.
model.compile(optimizer=”adam”, loss=’categorical_crossentropy’,metrics=[‘accuracy’] )Step-7 Convert the output vector to one hot vector
To know what is one-hot encoding and why it is necessary read this.
from keras.utils import to_categorical
y_train=to_categorical(y_train)3. Train your model
Step-8 Fit the model on your training dataset
Now we will fit the model on training data and save the model as ‘hist’. We have to give the values of the following parameters:
- x: the input vector of training data.
- y: output vector of training data, which was one-hot encoded.
- batch_size: it is significant for large datasets. We chose 32 as the batch size which means in every iteration 32 examples will be processed.
- epochs: the number of epochs
- validation_split: the ratio of total data to be used for validation.
There are many other parameters but they will be initialized with their default values.
hist=model.fit(x=x_train, y=y_train, batch_size=32, epochs=10, validation_split=0.2, shuffle=True)
Epoch 1/10 1500/1500 [==============================] - 3s 2ms/step - loss: 0.8803 - accuracy: 0.8268 - val_loss: 0.3275 - val_accuracy: 0.9077
Epoch 2/10 1500/1500 [==============================] - 3s 2ms/step - loss: 0.2615 - accuracy: 0.9238 - val_loss: 0.2284 - val_accuracy: 0.9383
Epoch 3/10 1500/1500 [==============================] - 3s 2ms/step - loss: 0.2054 - accuracy: 0.9406 - val_loss: 0.1965 - val_accuracy: 0.9447
Epoch 4/10 1500/1500 [==============================] - 3s 2ms/step - loss: 0.1703 - accuracy: 0.9504 - val_loss: 0.1894 - val_accuracy: 0.9503
Epoch 5/10 1500/1500 [==============================] - 3s 2ms/step - loss: 0.1489 - accuracy: 0.9569 - val_loss: 0.2132 - val_accuracy: 0.9452
Epoch 6/10 1500/1500 [==============================] - 3s 2ms/step - loss: 0.1319 - accuracy: 0.9620 - val_loss: 0.1746 - val_accuracy: 0.9558
Epoch 7/10 1500/1500 [==============================] - 3s 2ms/step - loss: 0.1182 - accuracy: 0.9655 - val_loss: 0.1546 - val_accuracy: 0.9583
Epoch 8/10 1500/1500 [==============================] - 3s 2ms/step - loss: 0.1066 - accuracy: 0.9695 - val_loss: 0.1533 - val_accuracy: 0.9605
Epoch 9/10 1500/1500 [==============================] - 4s 2ms/step - loss: 0.0952 - accuracy: 0.9722 - val_loss: 0.1680 - val_accuracy: 0.9617
Epoch 10/10 1500/1500 [==============================] - 4s 3ms/step - loss: 0.0868 - accuracy: 0.9747 - val_loss: 0.1775 - val_accuracy: 0.9574
4.Test your model
Step-9 Predict outcome for any random datapoint
Now lets test one random example from out dataset, say example no 999. To make predictions we use model.predict() function.
model.predict_classes(x_test[999].reshape(-1,784))The output given is 9 in my code. To check if the prediction made by your model is correct plot the data using plt.imshow().
plt.imshow(x_test[999].reshape(28,28), cmap=’gray’)
Our model predicted the input digit correctly.
References
[1] Cohen, G., Afshar, S., Tapson, J., & Van Schaik, A. (2017, May). EMNIST: Extending MNIST to handwritten letters. In 2017 International Joint Conference on Neural Networks (IJCNN) (pp. 2921–2926). IEEE.
