Detecting A Deadly Skin Cancer With Images and AI

14 min readJan 1, 2024

A walkthrough of how I used a CNN to classify tumours (moles) with a single image output in seconds which could save millions of lives

The trajectory of your life can take a turn for the worst all because of a small spec on your skin.

Don’t you just love soaking in the sun? ☀️ I sure do. When I’m on the beach, I find a lot of relaxation and internal peace when I lie down on a sunbed, close my eyes, and feel the warm yet soft rays of sun blanket my skin. It feels great since for more than half of the year it’s so cold in Canada 🥶. On the beach, hearing the sound of the waves crashing, children playing in the sand, and families bonding, is a surreal experience. The world goes by and I feel like I’m paused in the space and time. 🧘🏾

Although it sounds and feels amazing, it really isn’t for your skin.

Thousands if not millions of people living in countries with warm climates have this same mindset and craving for sun tanning. Take Lisa Pace for example. In her teens, she would crave sun tanning to the point where it would become a daily habit. The feeling of the sun on her skin and the colour her skin would become made her feel really good; it helped her “fit in.” But in her early 20s, her life completely changed. She tanned so much that her skin went trough irreversible changes and ended up with 86 skin tumours. 😬

The culprit is no other than melanoma: a deadly skin cancer. And it’s a huge problem right now.

For some context, melanoma, often referred to as the “king of skin cancers,” is a type of cancer that originates in the pigment-producing cells of the skin called melanocytes. While it accounts for only a small percentage of cancer cases, melanoma is notorious for its aggressive nature and the potential to spread to other parts of the body.

Melanoma appears when melanocytes, cells responsible for producing melanin — the pigment giving colour to our skin, hair, and eyes undergo genetic mutations. This leads to uncontrolled growth and the formation of malignant tumours — what we commonly refer to as moles. Exposure to UV radiation from the sun or tanning beds is a significant risk factor for these mutations, alongside genetic predisposition and a history of severe sunburns. 🥵

Melanoma is the most common form of any cancer; approximately 5.4 million new cases are diagnosed in the USA alone every year. While the five-year survival rate for melanoma detected in its earliest states is around 97 percent, that drops to approximately 14 percent if it’s detected in its latest stages. This makes melanoma the most fatal and invasive skin cancer.

I know what you’re thinking. “How is melanoma so prevalent and deadly if it appears on our skin? We can see other cancers in our body. Shouldn’t we be able to detect it right away?” 🧐

The Problem: Detecting Skin Cancer is a “Multifaceted Problem”

The first signs of melanoma usually appear with a mole that looks irregular on you skin.

The ABCDE rule serves as a simple guide for identifying potential warning signs of melanoma:

A — Asymmetry: One half of the mole or spot doesn’t match the other half.
B — Border irregularity: The edges are not well-defined and may appear jagged or notched.
C — Colour: The colour is uneven, with shades of brown, black, or other colours.
D — Diameter: Melanomas are often larger than the size of a pencil eraser.
E — Evolving: Any change in size, shape, colour, or elevation warrants attention.

However, many individuals may not be fully aware of the risk factors and warning signs of melanoma. Consequently, they may not actively monitor their skin or seek medical attention for changes that could indicate the early stages of the disease. Or, if you are like me when I was younger, you may be tempted to rip them off of your skin… especially if they made you look “ugly.” Now, you can’t determine if cancer could be growing on your skin! 🤦‍♂️

Melanomas can also resemble normal moles, especially when they first develop. This similarity makes it tricky for people to distinguish between harmless moles and potentially dangerous melanomas.

The changes associated with early-stage melanoma can also be subtle and gradual, making them easy to overlook. People might dismiss alterations in size, shape, or colour as normal variations (I know a lot of people fall vicitm to this).

Plus, melanomas can exhibit diverse appearances. While some may follow the typical ABCDE rule (asymmetry, irregular borders, varied colours, larger diameter, and evolving features), others may not display these characteristics. Atypical or amelanotic melanomas may lack pigment and appear pink or skin-coloured, making them even more challenging to identify.

We also need to consider the fact that a lot of people in rural areas don’t have much access to healthcare facilities and dermatologists to monitor their melanoma.

Try looking at this set of images. Can you classify which of the following moles are cancerous? I bet it’s harder than you previously thought.

Random set of malignant and benign tumours (moles)

The Solution Lies in Artificial Intelligence 🤖

Melanoma is nothing new to the medical field… we’ve been trying to treat and detect it since 1804. Apart from ABCDE’s, medical screening has been the only proven method of detecting melanoma. So what’s the solution?

The use of Artificial Intelligence for early detection of cancers like melanoma have proven be an effective method, if not one of the most effective methods we have. The logic is simple: If we can detect the cancer in its earlier stages, the patient’s survival rate increase exponentially.

Graph of survival rate per melanoma stage (percentage)

As seen in the graph, in the early stages of the cancer, the survival rate is extrodinarly high. But, in the later and latest stage of the disease, the survival rate drops drastically. So, the key to reducing death rates melanoma causes lies in detecting the disease early with the use of AI.

So what methods or companies working of detection methods are available right now?

Stanford University: Computer scientists at Stanford have trained an algorithm to visually diagnose potential skin cancer using a database of nearly 130,000 skin disease images. The deep learning algorithm performs as well as dermatologists in identifying skin cancer.
Skin Analytics: Operates a skin cancer screening platform, offering artificial intelligence (AI) supported teledermatology services to assist clinicians in assessing skin lesions. The company’s AI medical device, DERM (Deep Ensemble for the Recognition of Malignancy), utilizes advanced imaging algorithms to accurately detect skin cancer, including melanoma and squamous cell carcinoma.
Proscia: The company’s AI technology, including DermAI, analyzes dermoscopic images to identify melanoma accurately and has been validated through collaborative studies with institutions like Thomas Jefferson University and the University of Florida. Proscia’s advancements in deep learning aim to lower misdiagnosis rates, enhance diagnostic accuracy, and integrate AI into routine workflows.

I believe that the model created by researchers at Stanford University Artificial Intelligence Laboratory have developed the most capable and practical model. I discovered their model is trained from over 130 000 images provided by the University of Edinburgh and the International Skin Imaging Collaboration Project which represented over 2 000 different diseases. This is the largest dataset I’ve seen any model in this space been trained on. In addition, their model runs on smartphones, allowing anyone anywhere with the device to check their skin for melanoma, making it very practical and accessible. Quite honestly, if this model gets mainstream, it has huge potential to disrupt the early cancer detection space.

How do these models even work? 🧐

Most if not all models for detecting melanoma with deep learning use a Convolutional Neural Network (CNN). They are the backbone of all image classification deep learning models. They work by using specialized filters (gradient, texture, edge filters) to scan and learn features from input images in a hierarchical manner. These learned features are then used to make predictions, enabling CNNs to effectively recognize patterns and objects in images, making them widely used in tasks such as image classification and perfect for our model.

For a high-level overview on how CNNs work check out an article I wrote here.

Anyways, I decided to implement the CNN for this project while following the original paper which can be found here. I created my code with the help of a walkthrough found here. Check out the video below where I break down the code and show off the completed model:

YouTube Video I created for this article

Diving straight into the code 🧑🏽‍💻

Quick note: If you want to code out this model, I highly suggest referring to the GitHub repo I have set up with all of the code that went into this model.

The general process of creating our model

For our model, I used an image dataset consisting of 10 000 labeled images of melanoma cases on kaggle (access it here). This dataset is what our model will be trained and tested on.

Note ⚠️: Since I’m not part of a research lab and don’t have access to creating my own dataset, the one I’m using, which has a lot of different melanoma cases, will be sufficient for a model to achieve a pretty good accuracy and for a project that won’t be in a clinical setting (just yet 😉).

I split the datasets into training and testing with an 90–10 split. So we’ll have 4500 “benign” and 4500 “malignant” training images and 1000 testing images.

Data Labeling and One-Hot Encoding

Since we’re only dealing with two classification classes: benign and malignant (moles are tumours), I used one-hot encoding to represent these classes. This technique simplifies classification by converting class labels into binary vectors. For our model, this binary representation helps the model discern between benign ([1, 0]) and malignant ([0, 1]) lesions.

# one-hot vectors
# [1,0] = benign
# [0,1] = melanoma

# Load training and testing data
ben_training_data = load_and_process_images(ben_training_folder, [1, 0])
mal_training_data = load_and_process_images(mal_training_folder, [0, 1])
ben_testing_data = load_and_process_images(ben_testing_folder, [1, 0])
mal_testing_data = load_and_process_images(mal_testing_folder, [0, 1])

Image Processing and Resizing

Some of the images in the dataset weren’t the right size or not available to use. To solve this, I used OpenCV for image processing which ensures that the data is in a consistent and manageable format. I set all images to a standardized size of 50x50 pixels.

I also define a function load_and_process_images, which loads and resizes the images while assigning the corresponding labels in one-hot vector format, creating a structured dataset.

# 50x50 pixels
img_size = 50 

# Load and process images function
def load_and_process_images(folder, label):
    data = [[np.array(cv2.resize(cv2.imread(os.path.join(folder, filename), cv2.IMREAD_GRAYSCALE), (img_size, img_size))), np.array(label)] for filename in os.listdir(folder)]
    return data

Dataset Organization and Information

I then organize the dataset by combining images and labels, converting them to NumPy arrays, and shuffling the data to enhance the learning process. The resulting training and testing datasets are saved in NumPy format, facilitating easy retrieval and reuse.

I also print counts of images in each category, to measure the distribution of the dataset.

# Convert lists to NumPy arrays
ben_training_data = np.array(ben_training_data, dtype=object)
mal_training_data = np.array(mal_training_data, dtype=object)
ben_testing_data = np.array(ben_testing_data, dtype=object)
mal_testing_data = np.array(mal_testing_data, dtype=object)

# Combine and shuffle training and testing data
training_data = np.concatenate((ben_training_data, mal_training_data), axis=0)
np.random.shuffle(training_data)

testing_data = np.concatenate((ben_testing_data, mal_testing_data), axis=0)
np.random.shuffle(testing_data)

# Save the processed data
np.save("melanoma_training_data.npy", training_data)
np.save("melanoma_testing_data.npy", testing_data)

Coding the Brain of Our Model (CNN) 🧠

Understanding how a CNN works

What makes a CNN different from other Artificial Neural Networks (ANNs) is 3 layers:

Convolutional Layer(s): The Convolutional Layer is the foundational building block of a CNN. Its purpose is to detect features like edges, textures, or more complex patterns within the input image. This layer applies filters (also known as kernels) to small regions of the input image, moving these filters across the entire image. This operation is called convolution. Each filter learns to recognize specific features, and the output is a set of feature maps.

2. Pooling Layer: The Pooling Layer follows the Convolutional Layer. Its primary function is to reduce the spatial dimensions of the input volume while retaining important information. Pooling is achieved by taking the maximum or average value within a small region of the feature map. This process helps to make the detection of features invariant to scale and orientation changes.

3. Fully Connected Layer: The Fully Connected Layer is the final layer in a CNN. It connects every neuron from the previous layer to each neuron in this layer. This layer combines the features learned in the previous layers to make predictions about the input. It’s essentially a classic neural network layer where each neuron’s output is a weighted sum of the inputs followed by an activation function.

Here’s what the full CNN looks like:

Programming the Neural Network (With PyTorch)

class Net(nn.Module): 

    def __init__(self):
        super().__init__() 
        self.conv1 = nn.Conv2d(1, 32, kernel_size=5)
        self.conv2 = nn.Conv2d(32, 64, kernel_size=5)
        self.conv3 = nn.Conv2d(64, 128, kernel_size=5)
        self.fc1 = nn.Linear(128 * 2 * 2, 512)
        self.fc2 = nn.Linear(512, 2)

    def forward(self, x):
        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
        x = F.max_pool2d(F.relu(self.conv2(x)), (2, 2))
        x = F.max_pool2d(F.relu(self.conv3(x)), (2, 2))
        x = x.view(-1, 128 * 2 * 2)
        x = F.relu(self.fc1(x)) 
        x = self.fc2(x)
        x = F.softmax(x, dim=1)  # Specify dim=1 to apply softmax along the correct dimension
        return x

Convolutional Layer: In the convolutional layers, the network learns to detect hierarchical features within the input images. The first convolutional layer processes the raw pixel values of a grayscale image (50x50 pixels) through 32 filters of size 5x5, extracting low-level features. Subsequent convolutional layers (64 filters in the second and 128 in the third) capture increasingly complex patterns. Each convolutional layer is followed by a rectified linear unit (ReLU) activation function, introducing non-linearity to the model.
Pooling Layer: Pooling layers reduce the spatial dimensions of the feature maps, focusing on the most salient information. The max-pooling operation is applied after each convolutional layer, downsampling the feature maps by taking the maximum value in each 2x2 region. This step helps maintain important features while reducing computational complexity and preventing overfitting.
Fully Connected Layer: The fully connected layers integrate the extracted features and make final predictions. The flattened output from the last convolutional layer is passed through two fully connected layers. The first fully connected layer has 128 * 2 * 2 input features, derived from the dimensions of the previous layer’s output. It introduces another layer of non-linearity through the ReLU activation function. The final fully connected layer has two output neurons, representing the number of classes for classification.

In the forward pass, the neural network processes input data through convolutional and fully connected layers, employing activation functions and max-pooling operations. The final layer utilizes a softmax activation, producing class probabilities; when trained on labeled image data, the CNN learns to make accurate predictions for new images by leveraging the features it has extracted.

Training 🏋🏽

We can now train the model as we have all the necessary components for our model.

For training, I implemented the Adam optimizer that learns at a rate of 0.001. This optimizer facilitates the adjustment of model parameters during training to minimize the chosen loss function. I’ve also set a batch size to 100 images.

I also define a function that will run our CNN, optimizers, loss, and backpropogation in. I’ve set the function to run at 15 epochs. Here is an overview of what goes on within this function for each epoch:

The function processes the training data in batches
Input data undergoes a forward pass through the neural network, generating predictions. Since we’re using one-hot vectors a prediction may look like [0.99, 0.01]
The loss between predictions and actual labels is computed using the selected loss function. For example, let’s say our prediction was [0.99, 0.01], the loss will be calculated from the actual value of [1, 0].
Backpropagation calculates gradients of the loss with respect to model parameters

This process is repeated per epoch or 15 times.

After this, you can test the model with the testing data we’ve left aside and should get an accuracy of anywhere between 87–90% (depending on how much you play around with the CNN parameters and epochs). Considering this model had a relatively small dataset and was created in a few days, this accuracy is amazing.

And just like that you’ve successfully created a deep learning model that can detect melanoma 👏

Although a small project, this classification model has a lot of impact in transforming the cancer detection space and making it accessible.

Web App for Remote Areas 💻: I plan to create a web-application that will add on to this project. It will be an accessible interface where anyone could upload a picture of their mole of concern and get a result in less than 5 seconds. This project could reach individuals in remote areas with limited access to a dermatologist. Anyone, anywhere, at any time could get a result in seconds — that’s extremely powerful.
Integration with Clinical Workflow 🩺: The model can be integrated into existing clinical workflows, providing dermatologists with additional information to support their decision-making process. This integration can enhance the overall efficiency of the diagnostic pipeline. Plus, dermatologists and clinicians will be able to spend less time on screening and focus on treating patients with melanoma or more important cases.
Real-Time Monitoring 📈: My model could also have real-time monitoring. With a bigger dataset and more advanced techniques, this model could classify tumours of concern and predict when or if the tumour will become malignant. Patients would no longer have to attend check-ups and be able to monitor their skin health quickly.

In the future, a model like this will become as common as checking glucose levels for diabetics (with a finger-prick or Dexcom). It will lead to countless number of lives will be able to be saved and result in less time and money spent on melanoma screening. All this being done with a single image 🤯.

The use of AI in the tumour detection space doesn’t end at melanoma. Soon, we’ll be able to detect more advanced and deadly cancers like pancreatic cancers or liver cancers all with a single image. Also, there have already been projects in development that use AI for tumour classification and cancer treatments. The future of cancer detection and treatment is looking very bright and yet there’s so much more to come from this space. Who knows, you may just see me at the forefront of these developments…

All in all, AI is 110% at the forefront of impacting the cancer. I can’t wait to begin my next project and solve the biggest problems in medicine 🦾.

Hey I’m Arjun, a 16 year-old Canadian passionate about using AI to solve the biggest issues in cancer. Want to see more of my work and who I am? Follow me or check out my LinkedIn or Twitter. Want to be updated on all of my projects and advancements? subscribe to my monthly newsletter. Thank you for reading and I hope you learnt something new!