How did we use computer vision to help medical experts diagnose Follicular Lymphoma?

TL;DR

Introduction

What is Follicular Lymphoma? What are the challenges in its diagnosis?

How could deep learning help in its detection?

Why did we use a patch-based classification?

  • the patches must be big enough so that the follicles remain visible in them
  • the patches should be small enough so that training a model can be done in a reasonable amount of time

Training Set

Validation Set

Testing Set

Modelling

The images are first divided into patches, then normalised before they are fed to the model for training.
At inference time, new whole-slide are divided into patches before the model predicts a class for each one of them. Parts of images responsible for predicting FL class are highlighted to help monitoring the results.

Data preparation and processing

1 — Tiling

2 — Stain normalisation

Results of three different stain normalisation : a target image colouring is normalised to a base image colour distribution.

Training a Resnet-18 classifier

Testing

  • A simple resnet-18 as baseline
  • A resnet-18 + stain normalization on the dataset
  • A resnet-18 + stain normalization on the dataset + mixup as data augmentation
The results of 3 different models on the 16 selected slides of Follicular Lymphoma. We can see the effect of stain normalisation and mixup on performance.

Interpreting the results of a computer vision classifier

Parts of the image that has most contributed to the prediction of the class Follicular Lymphoma are highlighted on the right sided image (12 patches)

Conclusion and Key learnings

  • The high importance of color normalisation when training a model with this type of dataset
  • Usage of advanced data augmentation technique such as mixup can help increase performances
  • The tight collaboration with medical-expert to challenge models at each iteration