Computer Vision and Pneumonia Detection Part 1: Technical
Recently, I tasked myself with analyzing chest x-ray images to determine whether or not a patient has pneumonia. The x-rays were provided by the Guangzhou Women and Children’s Medical Center in Guangzhou, China, on patients ranging from ages 1–5 years old. The image set contains images of “Normal”, “Viral” & “Bacterial” Pneumonia, classified into 2 categories: “Normal”and “Pneumonia”.
The dataset was provided in ideal conditions, free of poor quality scans. Before building the model, I checked the training data for class imbalance. There were 1341 Normal scans and 3875 Pneumonia scans. Killing 2 birds with one stone, I added 2534 additional images, and augmented them to reflect real world scenarios- blurry, distorted images, shifted/rotated images, and images that are zoomed in and out of focus.
# Set criteria for image augmentation
datagen = ImageDataGenerator(
rotation_range = 40,
width_shift_range = 0.2,
height_shift_range = 0.2,
rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True,
fill_mode = 'nearest')# Select an image from the dataset
img = load_img('chest_xray/train/NORMAL/IM-0757-0001.jpeg')# Convert image to an array
img_array = img_to_array(img)# Reshape the array to a (1 X n X m X 3) array
img_array = img_array.reshape((1,) + img_array.shape)# Set the path where the updated images should be saved
norm_dir = 'chest_xray/train/NORMAL/'# Add 2534 photos to the directory in batches of 150
count = 0
for batch in datagen.flow(img_array, batch_size=150, save_to_dir=norm_dir, save_prefix='IM', save_format='jpeg'):
count +=1
if count == 2534:
break
print('2534 images have been generated at', norm_dir)
Once the images were added, I reduced the size of the images to 64 x 64, so that the sizes would be uniform and less computationally expensive to input into the model, and save the updated images to the original file directory. Using the next() function, I iterated through the image files and returned the images and labels that have been stored in the variables “__images” and “__labels”, then I normalized the images by dividing the pixels by 255.
# Train set
train_generator = ImageDataGenerator(rescale=1./255).flow_from_directory(train_dir,
target_size = (64, 64), batch_size= 7685)# Validation Set
val_generator = ImageDataGenerator(rescale=1./255).flow_from_directory(val_dir,
target_size=(64, 64), batch_size=16)# Test Set
test_generator = ImageDataGenerator(rescale=1./255).flow_from_directory(test_dir,
target_size = (64, 64), batch_size=624)# Create the image-label datasets
train_images, train_labels = next(train_generator)
val_images, val_labels = next(val_generator)
test_images, test_labels = next(test_generator)
The final preprocessing step is to resize the labels to an (l x 1) array, then preview an image and label to ensure that the correct labels are corresponding with the respective images.
train_labels_final = np.reshape(train_labels[:,0], (7685,1))
val_labels_final = np.reshape(val_labels[:,0], (16,1))
test_labels_final = np.reshape(test_labels[:,0], (624,1))# Sanity check on image 4000
array_to_img(train_images[4000])
train_labels_final[4000, :]
Now on to building the model. The initial model was run with the default settings, then expounded upon, after each iteration to see which parameters helped the model to achieve optimal performance. Below is a code snippet that iterates through different optimizers while fitting to the training and validation data. After the code block is run, you can visually inspect the performance of the optimizers via a line plot.
# Create a dictionary of optimizers
optimizers = {"RMSprop": {"optimizer": RMSprop(), "color":"blue"},
"adam_01": {"optimizer": Adam(lr=0.01),"color":"red"},
"sgd": {"optimizer": SGD(), "color":"purple"},
"adadelta": {"optimizer": Adadelta(), "color":"pink"},
"adagrad": {"optimizer": Adagrad(), "color":"yellow"}}for optimizer, d in optimizers.items():
print(f'Testing {optimizer}')
# Build the CNN
model = Sequential() model.add(layers.Conv2D(32, (3, 3), activation='relu',
input_shape=(64, 64, 3))) model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(32, (4, 4), activation='relu'))
model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2,2))) # Flattening
model.add(layers.Flatten())
model.add(layers.Dense(64, activation = 'relu'))
model.add(layers.Dense(1, activation='softmax'))
model.compile(optimizer=d['optimizer'],loss='binary_crossentropy',
metrics=['accuracy']) results = model.fit(train_images, train_labels_final,
epochs=15, batch_size=500, validation_data=(val_images,
val_labels_final)) print("Stored Results")
d['loss'] = history.history['loss']
print('='*125) # Add a partition in between optimizer results
Next, evaluate the final loss and accuracy of the model, then make predictions on the validation images. Access the total performance of the model using a confusion matrix in conjunction with a Classification Report. My final model resulted in having zero False Negatives and eight False Positives, which demonstrates a best case scenario because it’s better to flag a patient for additional testing, than to release a patient carrying pneumonia and their symptoms worsen or they spread viral pneumonia to a loved one.
# Evaluate the model on the metrics accuracy and loss
model.evaluate(train_images, train_labels_final)
model.evaluate(val_images, val_labels_final)# Make predictions on the validation set
preds = model.predict(val_images)
plot_confusion_matrix(val_labels_final, preds)# Print the Classification Report, which returns Recall, Precision, and F1-Score
classification_report =classification_report(val_labels_final, preds)
Continue to tune your hyper parameters, and train your best performing model on your testing data, then repeat. Compare your results to a pre-trained model. Below I used VGG16.
# Import Pretrained Model
from keras.applications.vgg16 import preprocess_input
from keras.applications.vgg16 import VGG16# Edit the default image size to (64, 64, 3)
pretrain_mod = VGG16(include_top= False, input_shape=(64,64,3))# Initialize a model
model = Sequential()
model.add(pretrain_mod)
model.add(layers.Flatten())
model.add(layers.Dense(132, activation='tanh'))
model.add(layers.Dense(1, activation='softmax'))# Freeze layers (do this to speed up
pretrain_mod.trainable = False
for layer in model.layers:
print(layer.name, layer.trainable)
print(len(model.trainable_weights))model.compile(loss='binary_crossentropy', optimizer=Adam(lr=0.01), metrics=['accuracy'])results = model.fit(train_images, train_labels_final, epochs=11, batch_size=750, validation_data=(test_images, test_labels_final))
Don’t forget to save your best model!
print('saving model to disk \n')
mod = './/Models/pretrained_mod'
model.save(mod)
