How I Detected Cancer Metastasis Using AI
My Process Using the PatchCamelyon Dataset With TensorFlow
For context and some background information about Artificial Intelligence in diagnostics, see the first article in this series, Using AI in Diagnostic Imaging. To see my first AI project, see my second article in this series, How I Created a Model that Identifies Cardiac Conditions. For the link to my GitHub repository for this project, click here.
Looking at the sample images above, are you able to discern the ones with cancerous cells and those without? Although I could recognize some loose patterns for the two classifications (contains metastatic tissue and does not contain metastatic tissue), I most certainly would not be confident enough to bet money on the label of any similar images — and this is to be expected. While I am immersed in the veterinary field, I am not a trained professional who is responsible for being able to identify cancerous and non-cancerous cells, and neither are the vast majority of people. This is one of the reasons why veterinarians and doctors are so valuable, but also one of the reasons why diagnostics can be so costly. Sometimes, diagnostic images similar to the ones above will be outsourced to specialty doctors who are specialized in performing different diagnostic imaging analyses. This often causes a further increase in price for diagnostics, with little to no additional profit on the side of your original doctor or veterinarian. In fact, if fewer patients or clients are opting to use diagnostic services, it can be detrimental to the profits of clinics and hospitals.
Artificial Intelligence, or AI, however, might soon overcome this issue through image recognition. Diagnostic images can be interpreted by machine learning models to predict a diagnosis, opening opportunities to reduce costs of outsourcing to clients. To develop my skills further in AI from my last project, I decided to start a new project that would be slightly more challenging. I chose to make an image recognition model to detect cancer metastasis from the PatchCamelyon dataset with TensorFlow. Again, I wrote my code in Python on Google Colab.
Steps for Creating the Model
Imports and Setup
First, I performed several imports to be able to use the libraries and modules necessary for my project. I imported TensorFlow, Keras (from TensorFlow), the tensorflow_datasets module, OS, and InceptionV3, as well as Input, Dense, GlobalAveragePooling2D, Model, and Sequential from Keras:
import tensorflow as tf
from tensorflow import keras
import tensorflow_datasets as tfds
import os
from tensorflow.keras.applications.inception_v3 import InceptionV3
from tensorflow.keras.layers import Input, Dense, GlobalAveragePooling2D, BatchNormalization
from tensorflow.keras.models import Model
from tensorflow.keras import SequentialNext, as the PatchCamelyon dataset was one of the datasets already in the TensorFlow library, I was able to upload it directly, split it into training, validation, and testing sets:
(ds_train, ds_validation, ds_test), ds_info = tfds.load(
"patch_camelyon",
split = ["train", "validation", "test"],
shuffle_files = True,
as_supervised = True,
with_info = True)Dataset Information
Once the imports were completed, I printed the information of the dataset from the TensorFlow library:
print(ds_info)Which gave:
To better visualize the image data, my next step was to use tfds.show_examples to see a few of the images and their labels:
fig = tfds.show_examples(ds_train, ds_info, rows = 4, cols = 4)Which shows:
Preparing the Training, Validation, and Testing Sets
Before being able to set up a model, I next had to prepare the images in the training, validation, and testing sets and the sets themselves.
My first step in this process was to normalize the images through resizing them and normalizing their shape:
def normalize_img(image, label): return tf.cast(image, tf.float16)/255.0, label
Next, I defined AUTOTUNE, which allows TensorFlow to optimize the productivity of actions when applied, and set the BATCH_SIZE to 16:
AUTOTUNE = tf.data.experimental.AUTOTUNEBATCH_SIZE = 16
To increase the speed of training in the second-to-last step, I used “Counter” and applied it to ds_train:
counter = tf.data.experimental.Counter()ds_train = tf.data.Dataset.zip((ds_train, (counter, counter)))
I then applied some functions to ds_train:
ds_train = ds_train.map(normalize_img, num_parallel_calls = AUTOTUNE)ds_train = ds_train.cache()ds_train = ds_train.shuffle(ds_info.splits["train"].num_examples)ds_train = ds_train.batch(BATCH_SIZE)ds_train = ds_train.prefetch(AUTOTUNE)
The first line applies the normalize_img function that we defined earlier to the training set and sets the number of parallel calls in the dataset that can occur to AUTOTUNE. The second allows some of the data to be kept in memory so it will be loaded more quickly in subsequent runs. The third line shuffles the data randomly, and the fourth line applies the batch size of the training set to the variable BATCH_SIZE that we defined earlier. Lastly, the fifth line applies “prefetch” and sets it to AUTOTUNE, which increases efficiency by allowing images to be prepared in advance while the GPU is running.
Next, I applied some of these functions to the validation and test sets:
ds_validation = ds_validation.map(normalize_img, num_parallel_calls = AUTOTUNE)ds_validation = ds_validation.batch(32)ds_validation = ds_validation.prefetch(AUTOTUNE)ds_test = ds_test.map(normalize_img, num_parallel_calls = AUTOTUNE)ds_test = ds_test.batch(32)ds_test = ds_test.prefetch(AUTOTUNE)
Preparing the Model
Once the datasets were prepared, the model for learning had to be built. To set up the actual architecture, I created a function named “create_model” with the parameter of image size (96 for the images in this dataset). I then defined the variable “inception”, which represented the pre-trained model InceptionV3 and defined some parameters within the model. In the “create_model” function, I also froze the InceptionV3 layers with “layer. trainable = False”. Next, I built the structure, starting with the Sequential model. Before coding the additional layers in the model, I used “model.add” to include InceptionV3 and GlobalAveragePooling2D, and set the expected shape of the input to the shape of the data. I then listed the dense, batch normalization, and drop layers, connecting each layer to the previous one. Lastly, I defined the output as a final dense layer, set the model inputs and outputs, and returned the model:
def create_model(image_size = 96): inception = InceptionV3(weights = 'imagenet', include_top = False, input_shape = (image_size, image_size, 3)) for layer in inception.layers: layer.trainable = False model = Sequential() model.add(inception) model.add(GlobalAveragePooling2D()) input = Input(shape=(image_size, image_size, 3)) encoded = model(input) dense1 = Dense(256, activation='relu')(encoded) batch_normalization1 = BatchNormalization()(dense1) drop_layer = tf.keras.layers.Dropout(0.7)(batch_normalization1) dense2 = Dense(128, activation='relu')(drop_layer) batch_normalization2 = BatchNormalization()(dense2) drop_layer2 = tf.keras.layers.Dropout(0.7)(batch_normalization2) output = Dense(1, activation='sigmoid')(drop_layer2) model = Model(inputs=[input], outputs=output) return model
To view the architecture of the model before running it, I created a code block to print its summary:
model = create_model()model.summary()
Which printed:
The last steps before being able to train the model were to compile the model and set its parameters, and define the variable “callbacks” to enable the model to stop running early if the loss on the validation set does not improve after 5 consecutive epochs:
model.compile(optimizer=tf.keras.optimizers.SGD(learning_rate=0.001), loss='binary_crossentropy', metrics=['accuracy'])
callbacks = [tf.keras.callbacks.EarlyStopping(monitor="val_loss" , patience=5 , verbose=1)]
Training the Model
As the last step before evaluating the effectiveness of the model, I had to train it on the training data and run it on the validation data to estimate its accuracy on the testing data and detect possible overfitting or underfitting:
model.fit(ds_train, batch_size = 16, epochs = 20, verbose=1, validation_data = (ds_validation), callbacks = callbacks)
These were the results of the training:
Evaluating the Model
Lastly, to evaluate the accuracy of the model on the testing data, I ran the following code:
model.evaluate(ds_test)Which gave:
Conclusion
In the end, I achieved an accuracy of approximately 80% on this model. This is comparable with the results of other developers on this dataset, such as Geert Litjens, who achieved an accuracy of 85.63% on his model for the PatchCamelyon dataset. For this project, I experimented for several hours testing out different optimizers, learning rates, and adding and removing dense, and dropout layers, particularly within the “create_model” function and the model compilation. I found this very valuable, as I began to realize that while it may be portrayed more simplistically and be more standardized in the mainstream media and ML guides, different optimizers and parameters work better for certain projects. I believe that because of the sometimes seemingly insignificant differences between the images where cancer metastasis is detected and images where it is not, this was especially true for my model. It was definitely helpful to learn this through this project, as my next project will not be as guided or inspired by previous projects, and it will be important to be more creative when resolving issues such as low accuracy.
