ISIS Vs. Al-Qaeda — Building an Image Recognition Algorithm Based on Only 2 Images and Using Data Augmentation
My goal over this article will be to build and train a CNN that can accurately identify images of Al-Qaeda flags and ISIS flags.
1. Business Understanding
ISIS (Islamic State of Iraq and Syria), which it’s official name since June 2014 is the Islamic State and referred to by its Arabic-language acronym Daesh (داعش), is an Islamist militant jihadist group and former unrecognized quasi-state that follows a Salafi jihadist doctrine based on the Sunni branch of Islam. It was founded by Abu Musab al-Zarqawi in 1999 and gained global prominence in 2014, when it drove Iraqi security forces out of key cities during the Anbar campaign, which was followed by its capture of Mosul and the Sinjar massacre.
Al-Qaeda (القاعدة, translation: “the Headquarters” alternatively spelled al-Qaida and al-Qa’ida) is a multinational militant Sunni Islamic extremist network composed of Salafist jihadists. It was founded in 1988 by Osama bin Laden, Abdullah Azzam, and several other Arab volunteers during the Soviet–Afghan War.
2. Data Understanding
The first thing I need to do is get the data for which I’ll be training the CNN.
Since there is no public data set of ISIS Versus Al-Qaeda flags, that is, there is no Kaggle ISIS Versus Al-Qaeda competition, then in fact I need to collect images of ISIS and Al-Qaeda flags by myself.
But in my case, there are two problems related to collecting images of ISIS and Al-Qaeda flags from the web:
- Since I live in Israel and in the last month Israel has experienced 2 attacks carried out by ISIS terrorists — then my search for pictures of ISIS flags on the web will bounce my profile to the Israeli Security Agency and may cause me a little bit of a problem (which can be easily solved since I’m an Israeli Jewish patriot — but I really prefer to avoid this kind of unpleasantness)
- Since I am interviewing at the moment for a data scientist at Israeli start-ups that deal in the cyber field — I assume that my computer and laptop have already been infected with some kind of spyware, so I can imagine that surfing to ISIS websites might brand me as someone who should not be employed.
These two reasons have led me to think of a different way of obtaining images of Al-Qaeda flags and ISIS flags. I decided to go to Wikipedia, download one ISIS flag image and one Al-Qaeda flag image from it and use data augmentation in order to create hundreds of new images of ISIS and Al-Qaeda flags based on modifications of those 2 images from Wikipedia.
In this section, I’ll demonstrate how to use data augmentation on images using TensorFlow’s Keras API.
The dataset can be downloaded from here.
Data augmentation occurs when new data is created based on modifications of existing data.
In my case, the data I’ll work with will be images. For image data specifically, data augmentation could consist of things like flipping the image horizontally or vertically, rotating the image, zooming in or out, cropping, or varying the color.
For starters, it will help us obtain more data for training. Maybe I have a small training set, or maybe I just want to make my training set larger. I can do that by augmenting my existing data and then adding that data to the training set.
Another reason to use data augmentation is to reduce overfitting.
Let’s now see how I can perform data augmentation using Keras.
First, I import all the libraries I’ll be using.
import matplotlib.pyplot as plt
from scipy import ndimage
import numpy as np
import pandas as pd
import os
import shutil
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.preprocessing.image import ImageDataGenerator
import itertools
import shutil
from random import randint
import random
import glob
import matplotlib.pyplot as plt
import warnings
warnings.simplefilter(action=’ignore’, category=FutureWarning)
%matplotlib inlineI’ll first define this variable called gen as an ImageDataGenerator. All the parameters being passed are the different ways I'm telling Keras to augment the image.
gen = ImageDataGenerator(rotation_range=90,
width_shift_range=0.1,
height_shift_range=0.1,
shear_range=0.15,
zoom_range=0.1,
horizontal_flip=True)Check out the documentation to understand the units used for each augmentation technique I’ve specified. For example, rotation_range is measured in degrees, while width_shift_range is measured as a fraction of the width of the image.
Next, I will create a variable called image_path and set that to the relative location on disk of the chosen image.
image_path = 'C:\\Users\\רועי\\images\\isis.png'Note, to follow along, you will need to point to a valid location and image file on your machine.
Next, I’ll obtain the image by reading the image from disk by using plt.imread() and passing in the image_path. I also, expand the dimensions so that the image is compatible for how I'll use it later.
image = np.expand_dims(plt.imread(image_path), 0)Now, I'll plot the image just to see what the original image looks like.
plt.imshow(image[0])
Next, I will specify a parameter save_here and set it equal to an empty directory called “isis”.
save_here = ‘C:\\Users\\רועי\\images\\isis’Next, I’ll fit the original image.
gen.fit(image)Next, I’ll generate batches of augmented images from the original image. The flow() function takes numpy data and generates batches of augmented data.
Now I’ll get 700 samples of the augmented images.
for x, val in zip(gen.flow(image,\
save_to_dir=save_here,\
save_prefix=’isis’,\
save_format=’png’),range(700)) :
passThese are 700 images that have been augmented from the original image according to the parameters I passed to the ImageDataGenerator earlier.
Some of the images have been flipped horizontally, some have slight color variation, some are tilted slightly to the left or right, and some are shifted down or up slightly.
I’ll do the same for Al-Qaeda.
image_path = ‘C:\\Users\\רועי\\images\\alqaeda.png’
image = np.expand_dims(plt.imread(image_path), 0)
plt.imshow(image[0])
save_here = ‘C:\\Users\\רועי\\images\\alqaeda’
gen.fit(image)
for x, val in zip(gen.flow(image,\
save_to_dir=save_here,\
save_prefix=’alqaeda’,\
save_format=’png’),range(700)) :
pass3. Data Preparation
In this section, I’ll go through all the necessary image preparation and processing steps to get set up to train my first convolutional neural network (CNN).
3.1. Organize The Data
Once I have two directories isis directory and alqaeda with 700 images each, I’ll move those two two directories to a parent directory called data.
That’s it for the manual labor! At this point, I have 1,400 labeled images of ISIS and Al-Qaeda in my data directory. Note, the labels are included in the file names. Now, I'll do the remainder of the data organization programmatically.
First let’s import all the libraries I’ll be making use of over the article.
import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Activation, Dense, Flatten, BatchNormalization, Conv2D, MaxPool2D
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.metrics import categorical_crossentropy
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from sklearn.metrics import confusion_matrix
import itertools
import os
import shutil
import random
import glob
import matplotlib.pyplot as plt
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
%matplotlib inlineUsing the script below, I’ll organize my ISIS data into train, validation, and test sets. I’ll do this by moving subsets of the data into sub-directories for each separate data set.
# Organize ISIS data into train, valid, test dirs
os.chdir(‘C:\\Users\\רועי\\data\\isis’)
if os.path.isdir(‘C:\\Users\\רועי\\data\\train\\isis’) is False:
os.makedirs(‘C:\\Users\\רועי\\data\\train\\isis’)
os.makedirs(‘C:\\Users\\רועי\\data\\valid\\isis’)
os.makedirs(‘C:\\Users\\רועי\\data\\test\\isis’)for i in random.sample(glob.glob(‘isis*’), 500):
shutil.move(i, ‘C:\\Users\\רועי\\data\\train\\isis’)
for i in random.sample(glob.glob(‘isis*’), 100):
shutil.move(i, ‘C:\\Users\\רועי\\data\\valid\\isis’)
for i in random.sample(glob.glob(‘isis*’), 50):
shutil.move(i, ‘C:\\Users\\רועי\\data\\test\\isis’)os.chdir(‘../../’)
I’ll do the same for my Al-Qaeda data.
# Organize Al-Qaeda data into train, valid, test dirs
os.chdir(‘C:\\Users\\רועי\\data\\alqaeda’)
if os.path.isdir(‘C:\\Users\\רועי\\data\\train\\alqaeda’) is False:
os.makedirs(‘C:\\Users\\רועי\\data\\train\\alqaeda’)
os.makedirs(‘C:\\Users\\רועי\\data\\valid\\alqaeda’)
os.makedirs(‘C:\\Users\\רועי\\data\\test\\alqaeda’)for i in random.sample(glob.glob(‘alqaeda*’), 500):
shutil.move(i, ‘C:\\Users\\רועי\\data\\train\\alqaeda’)
for i in random.sample(glob.glob(‘alqaeda*’), 100):
shutil.move(i, ‘C:\\Users\\רועי\\data\\valid\\alqaeda’)
for i in random.sample(glob.glob(‘alqaeda*’), 50):
shutil.move(i, ‘C:\\Users\\רועי\\data\\test\\alqaeda’)os.chdir(‘../../’)
I first check to see if the directory structure is already in place. If it’s not, I proceed with the script. The script first makes train, valid, and test directories, which each contain sub-directories called isis and alqaeda.
The full data set contains 1,400 images, half of which are alqaeda flags, and half are isis flags. The remainder of the script moves 1000 samples into the training set, 200 samples into the validation set, and 100 samples into the test set. Each set has have an equal amount of alqaeda and isis flags.
This is it for all the data organization on disk. Now let’s check out the data processing that needs to be done before I can pass this data to the network.
3.2. Process The Data
I’ve already imported all the TensorFlow and Keras modules above. If you’re using a GPU (not required), then I can check to be sure that TensorFlow is able to identify the GPU using the code below.
physical_devices = tf.config.experimental.list_physical_devices(‘GPU’)
print(“Num GPUs Available: “, len(physical_devices))Num GPUs Available: 0
I then create variables for which the the paths to the train, valid, and test data directories are assigned.
train_path = ‘C:\\Users\\רועי\\data\\train’
valid_path = ‘C:\\Users\\רועי\\data\\valid’
test_path = ‘C:\\Users\\רועי\\data\\test’I can either specify an absolute path or a relative path relative to where my Jupyter notebook resides. I’m specifying the relative path.
Now, I use Keras’ ImageDataGenerator class to create batches of data from the train, valid, and test directories.
train_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input) \
.flow_from_directory(directory=train_path, target_size=(224,224), classes=[‘alqaeda’, ‘isis’], batch_size=10)valid_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input) \
.flow_from_directory(directory=valid_path, target_size=(224,224), classes=[‘alqaeda’, ‘isis’], batch_size=10)test_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input) \
.flow_from_directory(directory=test_path, target_size=(224,224), classes=[‘alqaeda’, ‘isis’], batch_size=10, shuffle=False)
Found 1000 images belonging to 2 classes.
Found 200 images belonging to 2 classes.
Found 100 images belonging to 2 classes.
ImageDataGenerator.flow_from_directory() creates a DirectoryIterator, which generates batches of normalized tensor image data from the respective data directories.
Notice, to ImageDataGenerator for each of the data sets, I specify preprocessing_function=tf.keras.applications.vgg16.preprocess_input. For now, just understand this does an additional processing step on the images.
To flow_from_directory(), I first specify the path for the data. I then specify the target_size of the images, which will resize all images to the specified size. The size I specify here is determined by the input size that the neural network expects.
The classes parameter expects a list that contains the underlying class names, and lastly, I specify the batch_size.
I also specify shuffle=False only for test_batches. That's because, later when I plot the evaluation results from the model to a confusion matrix, I'll need to able to access the unshuffled labels for the test set. By default, the data sets are shuffled.
3.3. Visualize The Data
I now call next(train_batches) to generate a batch of images and labels from the training set. Note that the size of this batch is determined by the batch_size I set when I created train_batches.
imgs, labels = next(train_batches)I then use this plotting function obtained from TensorFlow’s documentation to plot the processed images within my Jupyter notebook.
def plotImages(images_arr):
fig, axes = plt.subplots(1, 10, figsize=(20,20))
axes = axes.flatten()
for img, ax in zip( images_arr, axes):
ax.imshow(img)
ax.axis(‘off’)
plt.tight_layout()
plt.show()plotImages(imgs)
print(labels)
This is what the first processed random batch from the training set looks like. Notice that the color appears to be distorted. This has to do with the VGG16 processing I applied to the data sets. Don’t worry about it, just know that the RGB pixel data has been processed in such a way that the image data now looks like this before being passed to the network.
Note that isis flags are represented with the one-hot encoding of [0,1], and alqaeda flags are represented by [1,0].
I’m now all set up to work with this data! In the upcoming sections, I’ll use this data to train a convolutional neural network.
3.4. Specifying The Train and Valid data
X_train = train_batches
y_train = train_batches.classesX_valid = valid_batches
y_valid = valid_batches.classes
4. Modeling
In this section, I’ll demonstrate how to build a simple convolutional neural network (CNN) and train it on images of alqaeda flags and isis flags using TensorFlow’s Keras API.
I’ll be working with the image data I prepared in the last section. Be sure that you have gone through that section first to get and prepare the data, and also ensure that you still have all of the imports I brought in last time, as I’ll be continuing to make use of them here.
4.1. Build A Simple CNN
To build the CNN, I’ll use a Keras Sequential model. Recall, I first introduced a Sequential model.
model = Sequential([
Conv2D(filters=32, kernel_size=(3, 3), activation=’relu’, padding = ‘same’, input_shape=(224,224,3)),
MaxPool2D(pool_size=(2, 2), strides=2),
Conv2D(filters=64, kernel_size=(3, 3), activation=’relu’, padding = ‘same’),
MaxPool2D(pool_size=(2, 2), strides=2),
Flatten(),
Dense(units=2, activation=’softmax’)
])The first layer in the model is a 2-dimensional convolutional layer. This layer will have 32 output filters each with a kernel size of 3x3, and I'll use the relu activation function.
Note that the choice for the number of output filters specified is arbitrary, and the chosen kernel size of 3x3 is generally a very common size to use. You can experiment by choosing different values for these parameters.
I enable zero-padding by specifying padding = 'same'.
On the first layer only, I also specify the input_shape, which is the shape of my data. My images are 224 pixels high and 224 pixels wide and have 3 color channels: RGB. This gives us an input_shape of (224,224,3).
I then add a max pooling layer to pool and reduce the dimensionality of the data.
I follow this by adding another convolutional layer with the exact specs as the earlier one, except for this second Conv2D layer has 64 filters. The choice of 64 here is again arbitrary, but the general choice of having more filters in later layers than in earlier ones is common. This layer is again followed by the same type of MaxPool2D layer.
I then Flatten the output from the convolutional layer and pass it to a Dense layer. This Dense layer is the output layer of the network, and so it has 2 nodes, one for alqaeda flag and one for isis flag. I'll use the softmax activation function on my output so that the output for each sample is a probability distribution over the outputs of alqaeda flag and isis flag.
I can check out a summary of the model by calling model.summary().
model.summary()
Now that the model is built, I compile the model using the Adam optimizer with a learning rate of 0.0001, a loss of categorical_cross_entropy, and I'll look at accuracy as my performance metric.
model.compile(optimizer=Adam(learning_rate=0.0001), loss=’categorical_crossentropy’, metrics=[‘accuracy’])Note that when I have only two classes, I could instead configure my output layer to have only one output, rather than two, and use binary_crossentropy as my loss, rather than categorical_crossentropy. Both options work equally well and achieve the exact same result.
With binary_crossentropy, however, the last layer would need to use sigmoid, rather than softmax, as its activation function.
4.2. Train A Simple CNN
Now it’s time to train the model.
I'll be using the model.fit() to train a model
model.fit(x=X_train,
steps_per_epoch=len(X_train),
validation_data=X_valid,
validation_steps=len(X_valid),
epochs=10,
verbose=2
)I need to specify steps_per_epoch to indicate how many batches of samples from my training set should be passed to the model before declaring one epoch complete. Since I have 1000 samples in my training set, and my batch size is 10, then I set steps_per_epoch to be 100, since 100 batches of 10 samples each will encompass my entire training set.
I’m able to use len(X_train) as a more general way to specify this value, as the length of X_train is equal to 100 since it is made up of 100 batches of 10 samples. Similarly, I specify validation_steps in the same fashion but with using X_valid.
I’m specifying 10 as the number of epochs I'd like to run, and setting the verbose parameter to 2, which just specifies the verbosity of the log output printed to the console during training.
When I run this line of code, I can see the output of the model over 10 epochs.
From this output, I can see the performance of this simple model on the training set is great, with accuracy reaching 100% and loss nearing 0, however, by comparing these results to the validation metrics, I can see that my model is well generalizing to the validation set.
5. Evaluation
In this section, I’ll demonstrate how to use a convolutional neural network (CNN) for inference to predict on images of alqaeda flags and isis flags using TensorFlow’s Keras API.
5.1. Preparing The Test Data
First, ensure that you still have the code in place from last time when I built the model, as well as the earlier section when I created my test set.
I’ll now run next(test_batches) to extract a batch of images and their corresponding labels from the test set.
test_imgs, test_labels = next(test_batches)Recall, test_batches was created in a previous section and was created using Keras ImageDataGenerator.flow_from_directory() to generate batches of data from the test set that resides on disk.
Using the plotImages() function I previously introduced, I can see what this batch of test data looks like.
plotImages(test_imgs)
print(test_labels)
Just as I saw before, alqaeda are labeled with a one-hot encoding of [1,0], and isis are labeled as [0,1].
Note, because I chose to not shuffle my test set when I originally created it, the first half of the test data is all alqaeda, and the second half is all isis. Also, recall that the color data appears skewed due to the VGG16 preprocessing I specified when I created the data sets.
5.2. Specifying The Test data
X_test = test_batches
y_test = test_batches.classes5.3. Predicting On The Test Data
Now I’ll use my previously built model and call model.predict() to have the model predict on the test set.
predictions = model.predict(x=X_test, verbose=0)I pass in the test set, X_test, and set steps to be then length of X_test. Similar to steps_per_epoch that was introduced in the last episode, steps specifies how many batches to yield from the test set before declaring one prediction round complete.
I also specify verbose=0 to see no output during the evaluation process.
After running the predictions, I can print the rounded predictions to see what they look like.
np.round(predictions)
These are the labels that the model is predicting for my images.
Note, I can access the unshuffled true labels for the test set by calling y_test.
y_test
I transform the one-hot encoded predicted labels to be in the same format as the true labels by only selecting the element with the highest value for each prediction using np.argmax(predictions, axis=-1).
y_pred = np.argmax(np.round(predictions), axis=-1)5.4. Plotting Predictions With A Confusion Matrix
I’ll create the confusion matrix using scikit-learn.
I then define the plot_confusion_matrix() function that is copied directly from scikit-learn.
def plot_confusion_matrix(cm, classes,
normalize=False,
title=’Confusion matrix’,
cmap=plt.cm.Blues):
“””
This function prints and plots the confusion matrix.
Normalization can be applied by setting `normalize=True`.
“””
plt.imshow(cm, interpolation=’nearest’, cmap=cmap)
plt.title(title)
plt.colorbar()
tick_marks = np.arange(len(classes))
plt.xticks(tick_marks, classes, rotation=45)
plt.yticks(tick_marks, classes)if normalize:
cm = cm.astype(‘float’) / cm.sum(axis=1)[:, np.newaxis]
print(“Normalized confusion matrix”)
else:
print(‘Confusion matrix, without normalization’)print(cm)thresh = cm.max() / 2.
for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
plt.text(j, i, cm[i, j],
horizontalalignment=”center”,
color=”white” if cm[i, j] > thresh else “black”)plt.tight_layout()
plt.ylabel(‘True label’)
plt.xlabel(‘Predicted label’)
I can then inspect the class_indices for the labels so that I know in which order to pass them to my confusion matrix.
X_test.class_indices{'alqaeda': 0, 'isis': 1}
Finally, I plot the confusion matrix.
cm_plot_labels = [‘alqaeda’,’isis’]
plot_confusion_matrix(cm=cm, classes=cm_plot_labels, title=’Confusion Matrix’)
I can see that the model correctly predicted that an image was an alqaeda flag 50 times when it actually was an alqaeda flag, and it correctly predicted that an image was a isis flag50 times.
Given what I saw last time from the validation metrics, these results are not surprising.
5.5. Plotting Predictions With A Classification Report
To quote from Scikit Learn:
The precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier to not label a sample as positive if it is negative.
The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples.
The F-beta score can be interpreted as a weighted harmonic mean of the precision and recall, where an F-beta score reaches its best value at 1 and worst score at 0.
The F-beta score weights the recall more than the precision by a factor of beta. beta = 1.0 means recall and precision are equally important.
The support is the number of occurrences of each class in y_test.
from sklearn.metrics import classification_report
print(classification_report(y_test, y_pred))
5.6. Plotting Predictions With A ROC Curve
The receiver operating characteristic (ROC) curve is another common tool used with binary classifiers. The dotted line represents the ROC curve of a purely random classifier; a good classifier stays as far away from that line as possible (toward the top-left corner).
from sklearn.metrics import roc_auc_score
from sklearn.metrics import roc_curve
cnn_roc_auc = roc_auc_score(y_test, model.predict(X_test, batch_size = None)[:,1])
fpr, tpr, thresholds = roc_curve(y_test, model.predict_proba(X_test, batch_size = None)[:,1])
plt.figure()
plt.plot(fpr, tpr, label=’Convolutional Neural Network (area = %0.4f)’ % cnn_roc_auc)
plt.plot([0, 1], [0, 1],’r — ‘)
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel(‘False Positive Rate’)
plt.ylabel(‘True Positive Rate’)
plt.title(‘Receiver operating characteristic’)
plt.legend(loc=”lower right”)
plt.savefig(‘CNN_ROC’)
plt.show()
6. Deployment
Since I built, trained and tested my model on various modifications of only 2 images: one of an Al-Qaeda flag and the other of an ISIS flag, now I have to check whether or not my model can tell what terrorist organization is it from a picture of a terrorist holding the flag of one of those notorious terrorist organizations.
To do this I downloaded only 2 more pictures. The first picture is an image of a ISIS terrorist holding a ISIS flag from the BBC website .
The second picture is an image of an Al-Qaeda terrorist holding an Al-Qaeda flag from the VOIC OF AMERICA website.
I’ll create a new directory and called it “real_time”. I will save those two new pictures to this directory.
Next, I’ll randomly draw a picture from the “real_time” and pass it to a new directory called “pred”.
# Organize data into train directories
os.chdir(‘C:\\Users\\רועי\\data\\real_time’)
if os.path.isdir(‘C:\\Users\\רועי\\data\\pred’) is False:
os.makedirs(‘C:\\Users\\רועי\\data\\pred’)
for i in random.sample(glob.glob(‘*’), 1):
shutil.move(i, ‘C:\\Users\\רועי\\data\\pred’)Let’s see the picture I drew
from IPython.display import Image
os.chdir(‘C:\\Users\\רועי\\data\\pred’)
for i in random.sample(glob.glob(‘*’), 1):
Image(filename=i)Image(filename=i)
Obviously, this is an Al-Qaeda flag, but let’s see what my model think of that picture:
import pandas as pd
from sklearn import metricspred_filenames = os.listdir(“C:\\Users\רועי\\data\\pred”)
pred_df = pd.DataFrame({‘filename’: pred_filenames})
nb_samples = pred_df.shape[0]
batch_size = 10pred_gen = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input)
pred_generator = pred_gen.flow_from_dataframe(pred_df, “C:\\Users\רועי\\data\\pred”,
x_col=’filename’,
y_col=None,
class_mode=None,
batch_size=batch_size,
target_size=(224, 224),
shuffle=False)accuracy = “{:.2%}”.format(metrics.accuracy_score(y_test, y_pred))X_pred = pred_generatorpred = model.predict(X_pred, verbose=0)rounded_pred = np.argmax(np.round(pred), axis=-1) if rounded_pred[0] == 0:
print(“\033[1m The algorithm predicts, with accuracy of”, accuracy, “that this is an image of Al-Qaeda flag”)
else:
print(“\033[1m The algorithm predicts, with accuracy of”, accuracy, “that this is an image of ISIS flag”)
Found 1 validated image filenames.
The algorithm predicts, with accuracy of 100.00% that this is an image of Al-Qaeda flag
Awesome!!! my model was right.
Let’s delete that Al-Qaeda image from the “pred” directory and pass the last picture in the “real_time” directory from that directory to the “pred” directory and see if my model can score again.
# Organize data into train directories
os.chdir(‘C:\\Users\\רועי\\data\\real_time’)
for i in random.sample(glob.glob(‘*’), 1):
shutil.move(i, ‘C:\\Users\\רועי\\data\\pred’)Let’s see the picture I drew
Obviously, this is a ISIS flag, but let’s see what my model think of that picture:
import pandas as pd
from sklearn import metrics
pred_filenames = os.listdir(“C:\\Users\רועי\\data\\pred”)
pred_df = pd.DataFrame({‘filename’: pred_filenames})
nb_samples = pred_df.shape[0]
batch_size = 10
pred_gen = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input)
pred_generator = pred_gen.flow_from_dataframe(pred_df, “C:\\Users\רועי\\data\\pred”,
x_col=’filename’,
y_col=None,
class_mode=None,
batch_size=batch_size,
target_size=(224, 224),
shuffle=False)
accuracy = “{:.2%}”.format(metrics.accuracy_score(y_test, y_pred))
X_pred = pred_generator
pred = model.predict(X_pred, verbose=0)
rounded_pred = np.argmax(np.round(pred), axis=-1)
if rounded_pred[0] == 0:
print(“\033[1m The algorithm predicts, with accuracy of”, accuracy, “that this is an image of Al-Qaeda flag”)
else:
print(“\033[1m The algorithm predicts, with accuracy of”, accuracy, “that this is an image of ISIS flag”)Found 1 validated image filenames.
The algorithm predicts, with accuracy of 100.00% that this is an image of ISIS flag
Amazing!!! my model was right again.
Summary
In this article, I’ve demonstrated how to use data augmentation on images using TensorFlow’s Keras API.
Data augmentation occurs when new data is created based on modifications of existing data.
In my case, the data I’ve worked with was images (only 2 images). For image data specifically, data augmentation could consist of things like flipping the image horizontally or vertically, rotating the image, zooming in or out, cropping, or varying the color.
I’ve built, trained and tested my model on images of ISIS flags and Al-Qaeda flags from Wikipedia and validated it on 2 real images of terrorist holding flags from Google pictures.
My model is good and ready to distinguish between ISIS flags and Al-Qaeda flags.
The Jupyter notebook used to make this article is available here. I would be pleased to receive feedback or questions on any of the above.
About the Author
Roi Polanitzer, PDS, ADL, MLS, PDA, CPD, F.IL.A.V.F.A., FRM, is a data scientist with an extensive experience in solving machine learning problems, such as: regression, classification, clustering, recommender systems, anomaly detection, text analytics & NLP, and image processing. Mr. Polanitzer is is the Owner and Chief Data Scientist of Prediction Consultants — Advanced Analysis and Model Development, a data science firm headquartered in Rishon LeZion, Israel. He is also the Owner and Chief Appraiser of Intrinsic Value — Independent Business Appraisers, a business valuation firm that specializes in corporates, intangible assets and complex financial instruments valuation.
Over more than 16 years, he has performed data science projects such as: regression (e.g., house prices, CLV- customer lifetime value, and time-to-failure), classification (e.g., market targeting, customer churn), probability (e.g., spam filters, employee churn, fraud detection, loan default, and disease diagnostics), clustering (e.g., customer segmentation, and topic modeling), dimensionality reduction (e.g., p-values, itertools Combinations, principal components analysis, and autoencoders), recommender systems (e.g., products for a customer, and advertisements for a surfer), anomaly detection (e.g., supermarkets’ revenue and profits), text analytics (e.g., identifying market trends, web searches), NLP (e.g., sentiment analysis, cosine similarity, and text classification), image processing (e.g., image binary classification of dogs vs. cats, and image multiclass classification of digits in sign language), and signal processing (e.g., audio binary classification of males vs. females, and audio multiclass classification of urban sounds).
Mr. Polanitzer holds various professional designations, such as a global designation called “Financial Risk Manager” (FRM, which indicates that its holder is proficient in developing, implementing and validating statistical models and mathematical algorithms such as K-Means, SVM and KNN for credit risk measurement and management) from the Global Association of Risk Professionals (GARP), a designation called “Fellow Actuary” (F.IL.A.V.F.A., which indicates that its holder is proficient in developing, implementing and validating statistical models and mathematical algorithms such as GLM, RF and NN for determining premiums in general insurance) from the Israel Association of Valuators and Financial Actuaries (IAVFA), and a designation called “Certified Risk Manager” (CRM, which indicates that its holder is proficient in developing, implementing and validating statistical models and mathematical algorithms such as DT, NB and PCA for operational risk management) from the Israeli Association of Risk Managers (IARM).
Mr. Polanitzer had studied actuarial science (i.e., implementation of statistical and data mining techniques for solving time-series analysis, dimensionality reduction, optimization and simulation problems) at the prestigious 250-hours training program of the University of Haifa, financial risk management (i.e., building statistical predictive and probabilistic models for solving regression, classification, clustering and anomaly detection) at the prestigious 250-hours training program of the program of the Ariel University, and machine learning and deep learning (i.e., building recommender systems and training neural networks for image processing and NLP) at the prestigious 500-hours training program of the John Bryce College.
He had graduated various professional trainings at the John Bryce College, such as: “Introduction to Machine Learning, AI & Data Visualization for Managers and Architects”, “Professional training in Practical Machine Learning, AI & Deep Learning with Python for Algorithm Developers & Data Scientists”, “Azure Data Fundamentals: Relational Data, Non-Relational Data and Modern Data Warehouse Analytics in Azure”, and “Azure AI Fundamentals: Azure Tools for ML, Automated ML & Visual Tools for ML and Deep Learning”.
Mr. Polanitzer had also graduated various professional trainings at the Professional Data Scientists’ Israel Association, such as: “Neural Networks and Deep Learning”, “Big Data and Cloud Services”, “Natural Language Processing and Text Mining”.