Carnivores Image Classification using Google Colab
Convolutional Neural Network with the power of data augmentation and transfer learning!
General Information
CNN
A Convolutional Neural Network (CNN) is the foundation of most computer vision technologies. Unlike traditional multilayer perceptron architectures, it uses two operations called ‘convolution’ and pooling’ to reduce an image into its essential features, and uses those features to understand and classify the image.
Data Augmentation
It is a technique that can be used to artificially expand the size of a training dataset by creating modified versions of images in the dataset. Training deep learning neural network models on more data can create variations of the images that can improve the ability of the fit models to generalize what they have learned to new images.
Transfer learning
It is the idea of overcoming the isolated learning paradigm and utilizing knowledge acquired for one task to solve related ones. We freeze the early convolutional layers of the network and only train the last few layers which make a prediction. The idea is the convolutional layers extract general, low-level features that are applicable across images and the later layers identify specific features.
Why Google Colab?
Training a deep learning model on your regular laptop uses a lot of computational power and often runs endlessly. This can dissuade beginners from personally exploring the world of deep learning.
With Colab you can execute code on Google’s cloud servers by leveraging the power of Google hardware, including GPUs and TPUs, regardless of the power of your machine.
Creating a new notebook
To create a new Notebook on Colab, open https://colab.research.google.com/, and it will automatically show your previous notebooks and give an option to create a new notebook.
Here you can click on NEW NOTEBOOK to start a new notebook and start running your code in it. By default, it is a Python 3 notebook.
All the notebooks you create in Google Colab, are by default stored in your Google Drive. There is a folder in your drive named “Colab Notebooks” where you can find all your notebooks created from Google Colab.
To open a notebook from Drive in Colab, right-click on the desired notebook and “Open With > Google Colaboratory”.
Load Data from Drive
You can easily load data from Google Drive by mounting it to the notebook. To do this, type the following code in your notebook.
from google.colab import drive
drive.mount('/content/drive')It will give you a link to open,
- Go to the link
- Login to your Google Account
- Copy the code
- Paste it in notebook
Now if you see in your “Files” section, you will find your ‘drive’.
Let’s say you have uploaded images under the carnivores folder.
You can get the images by copying their path and pasting them to path variables.
train_path='/content/drive/My Drive/carnivores/train'
test_path='/content/drive/My Drive/carnivores/test'Carnivores Image Classification
The data is comprised of images for 4 carnivores from 3 continents → Cheetah and Hyena (Africa), Jaguar (South America) and Tiger (Asia). The training set consists of 3600 images (900 for each carnivore) and the test set has 400 images (100 for each carnivore). The data has very few examples to learn from and thus this is a challenging machine learning problem, but it is also a realistic one: in a lot of real-world use cases, even small-scale data collection can be extremely expensive or sometimes near-impossible.
For downloading data, visit the following link:
We will use Keras deep learning neural network library which provides the capability to fit models using image data augmentation via the ImageDataGenerator class. We will also use a pre-trained InceptionV3 model for transfer learning.
Importing Packages
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import sklearn
import mathimport warnings
warnings.filterwarnings("ignore")Pre-trained CNN Model as a Feature Extractor
Let’s leverage Keras, load up the InceptionV3 model, and freeze the convolution blocks so that we can use it as just an image feature extractor.
import keras
from keras.applications.inception_v3 import InceptionV3
from keras.models import Model,load_modelconv_base = InceptionV3(weights='imagenet',include_top=False,
input_shape=(300, 300, 3))output = conv_base.layers[-1].output
output = keras.layers.Flatten()(output)
model_tl = Model(conv_base.input, output)model_tl.trainable = Falsefor layer in model_tl.layers:
layer.trainable = Falselayers = [(layer, layer.name, layer.trainable) for layer in
model_tl.layers]
model_layers=pd.DataFrame(layers, columns=['Layer Type', 'Layer
Name', 'Layer Trainable'])print(model_layers)
All the layers of the InceptionV3 model are now frozen (Layer Trainable is False for all layers) because we don’t want their weights to change during model training. The last activation feature map in the InceptionV3 model gives us the bottleneck features, which can then be flattened and fed to a fully connected deep neural network classifier.
Data Pre-processing and Data Augmentation
In order to make the most of our few training examples, we will “augment” them via a number of random transformations, so that our model would never see twice the exact same picture. This helps prevent overfitting and helps the model generalize better.
In Keras this can be done via the ImageDataGenerator class. This class allows you to:
- configure random transformations and normalization operations to be done on your image data during training
- instantiate generators of augmented image batches (and their labels) via
.flow_from_directory(directory). These generators can then be used with the Keras model methods that accept data generators as inputs.
from keras.models import Sequential
from keras.layers import Dense, Dropout, Conv2D, MaxPool2D, Flatten
from keras.preprocessing.image import ImageDataGenerator
from keras import optimizerstest_size=400
batch_size=32
epochs=25train_path='/content/drive/My Drive/carnivores/train'
test_path='/content/drive/My Drive/carnivores/validation'target_size=(300,300) #resize all images to 300x300train_datagen = ImageDataGenerator(rescale=1./255, zoom_range=0.3,
rotation_range=50,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
horizontal_flip=True,
brightness_range = [0.8, 1.2],
fill_mode='nearest',
validation_split=0.2)test_datagen = ImageDataGenerator(rescale=1./255)# The list of classes will be automatically inferred from the subdirectory names/structure under train_dirtrain_generator = train_datagen.flow_from_directory(
train_path,
target_size=target_size,#
batch_size=batch_size,
class_mode='categorical',
subset='training')validation_generator = train_datagen.flow_from_directory(
train_path,
target_size=target_size,
batch_size=batch_size,
class_mode='categorical',
subset='validation')
Keras has now added train / validation split from a single directory using ImageDataGenerator. Using the validation_split in ImageDataGenerator, the training data is split into 2 subsets→ train and validation.
Building Model Architecture
Let’s build the architecture of our deep neural network classifier now, which will take the above mentioned flattened bottleneck features as input.
from keras.models import Sequential
from keras.layers import Dense, Dropout, Conv2D, MaxPool2D, Flatten
from keras import optimizers# building a linear stack of layers with the sequential model
model =Sequential()
model.add(model_tl)# hidden layer
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.2))# output layer
model.add(Dense(4, activation='softmax'))# compiling the sequential model
model.compile(loss='categorical_crossentropy',
optimizer=optimizers.RMSprop(lr=1e-4),
metrics=['acc'])print(model.summary())
Saving Model Checkpoints
Google Colab provides maximum gpu runtime of 12 hours “ideally” at a time, it may disconnect earlier than this if they detect inactivity (for 90 minutes) or when there is a heavy load. We will therefore save checkpoints using callbacks in Keras before fitting the model so that our progress is not lost.
from keras.callbacks import *filepath="/content/drive/My Drive/MyCNN/epochs:{epoch:03d}-val_acc:
{val_acc:.3f}.hdf5"checkpoint = ModelCheckpoint(filepath,
monitor='val_acc',
verbose=1,
save_best_only=False,
save_freq='epoch',
mode='max')callbacks_list = [checkpoint]
- filepath : is for folder called MyCNN in drive (create this new folder in your drive) and each file will be stored with epoch number and validation accuracy, these files contain weights of your neural network.
- ModelCheckpoint : for the arguments passed in the above code it is monitoring validation accuracy and stores validation accuracy after every epoch.
- callbacks_list : make it a list so that you can append any other callbacks to this list and pass it to fit function while training.
Resuming training once disconnected
- Mount the drive in the new runtime
- Use the same architecture and create the model
model.load_weights('/content/drive/My Drive/MyCNN/epochs:020-val_acc:0.985.hdf5')loading weights from a checkpoint where at 20th epoch it had reached a 98.5% validation accuracy.- Then compile and fit the model (in model.fit() add an extra parameter → initial_epoch=20; the last epoch till your model was saved), resuming from 21st epoch and so on.
Training model
We will use the architecture with checkpoints and fit the model to the training data.
history = model.fit(
train_generator,
steps_per_epoch=train_generator.samples//batch_size,
validation_data=validation_generator,
validation_steps=validation_generator.samples//batch_size,
epochs=epochs,
verbose=1,
shuffle=True,
callbacks=callbacks_list)
You can clearly see that it starts saving the model after every epoch
Model Performance
To evaluate and visualize model performance we have created a function:
- LearningCurve: Plots model accuracy and losses after each epoch to understand if the model is a goof fit or not.
# Model evaluationscores_train = model.evaluate(train_generator,verbose=1)
scores_validation = model.evaluate(validation_generator,verbose=1)
print("Train Accuracy: %.2f%%" % (scores_train[1]*100))
print("Validation Accuracy: %.2f%%" % (scores_validation[1]*100))#For plotting Accuracy and Lossdef LearningCurve(history):
# summarize history for accuracy
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'validation'], loc='upper left')
plt.show()# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'validation'], loc='upper left')
plt.show()LearningCurve(history)
#Save the trained model to a file model_weight_file='/content/drive/MyDrive/MyCNN/
carnivores_tlearn_img_aug_cnn.h5'
model.save(model_weight_file)
We can see that our model has a training accuracy of 99.51% and validation accuracy of 99.03%. The train and validation accuracy are quite close to each other, indicating that the model is a good fit.
Making Predictions
We will use the test data to make predictions and see how our model performs on unseen data.
# We take the ceiling because we do not drop the remainder of the batchcompute_steps_per_epoch = lambda x: int(math.ceil(1. * x / batch_size))test_steps = compute_steps_per_epoch(test_size)test_generator = test_datagen.flow_from_directory(
test_path,
target_size=target_size,
batch_size=batch_size,
class_mode=None,
shuffle=False)test_generator.reset()#Calling the saved model for making predictions
tl_img_aug_cnn = load_model(model_weight_file)pred=tl_img_aug_cnn.predict(test_generator,
verbose=1,
steps=test_steps)predicted_class_indices=np.argmax(pred,axis=1)
labels = (test_generator.class_indices)
labels = dict((v,k) for k,v in labels.items())
predictions = [labels[k] for k in predicted_class_indices]filenames=test_generator.filenames
results=pd.DataFrame({"Filename":filenames,
"Predictions":predictions})
#create a function for visualizing model performanceimport seaborn as snsdef PerformanceReports(conf_matrix,class_report,labels):
ax= plt.subplot()
sns.heatmap(conf_matrix, annot=True,ax=ax)
#labels, title and ticks
ax.set_xlabel('Predicted labels')
ax.set_ylabel('True labels')
ax.set_title('Confusion Matrix')
ax.xaxis.set_ticklabels(labels)
ax.yaxis.set_ticklabels(labels)
plt.show()
ax= plt.subplot()
sns.heatmap(pd.DataFrame(class_report).iloc[:-1, :].T,
annot=True,ax=ax)
ax.set_title('Classification Report')
plt.show()from sklearn.metrics import confusion_matrix,classification_report,accuracy_scorelabels=['cheetah','hyena','jaguar','tiger']
test_labels = [fn.split('/')[0] for fn in filenames]cm=confusion_matrix(test_labels,predictions)
print(cm)cr=classification_report(test_labels, predictions)
class_report=classification_report(test_labels, predictions,
target_names=labels,
output_dict=True)
print(cr)PerformanceReports(cm,class_report,labels)
Using the .predict() function will give output in probabilities so we need to convert them to class number. In this case it was 4 classes, so class numbers were 0,1,2 and 3.
predicted_class_indices=np.argmax(pred,axis=1)
Next step is we want the name of the classes:
labels = (train_generator.class_indices)
labels = dict((v,k) for k,v in labels.items())
predictions = [labels[k] for k in predicted_class_indices]The class numbers will be replaced by the class names. Finally, arrange it in a dataframe with the image names appended with the class predicted.
filenames=test_generator.filenames
results=pd.DataFrame({"Filename":filenames,
"Predictions":predictions})
Before You Go
Thanks for reading! Feel free to apply this methodology to your image classification problems. If you have any difficulty or any doubts kindly comment below. Your support is always highly appreciated. If you want to get in touch with me, reach me on jatin.kataria94@gmail.com.
