Deep look into classical image processing methods for segmentation
In publications and tools for image data, we hear more and more about the usefulness of deep neural network models. Day by day, research teams are outdoing themselves in presenting new architectures that segment or classify images in an even better, more precise way. At the xLungs — Responsible Artificial Intelligence for Lung Diseases project, these methods are no stranger to us, and we apply advanced algorithms to improve existing solutions. But surely these complex models always work better? Let’s find out!
Classical image processing
Have you ever heard about image segmentation before the era of CNNs? Or image classification? It seems like a prehistory but actually not so long ago Classical Image Processing methods were used for these tasks. Their main focus lies in analyzing image intensities and edges in order to segment specific image regions. Classical algorithms range from very simple, like thresholding pixel values, to very complex like morphological snakes (which requires solving complex energy equations).
Hounsfield units
Classical image processing methods are still very popular in medical image analysis especially Computed Tomography. There each voxel has value in Hounsfield scale. Values in Hounsfield units (Hu) are based on tissues linear attenuation coefficient μ which is transformed using radiodensity of distilled water (which corresponds to 0Hu) and air (-1000Hu), therefore each pixel value represents actual tissue density, for example: lungs tissue is in range (-700Hu, -600Hu). Thanks to this property even the simplest methods, like thresholding, are still applied by radiologists in their daily routine.
Example methods
Here I would like to introduce to you 2 algorithms that are very efficient despite their relative simplicity. These are: flood fill and watershed algorithms. They are both implemented in major image processing libraries like scikit-image, opencv or SimpleITK. It is worth noting that these methods are intensity based, so you have to apply denoising on image in order to get good quality segmentation.
Floodfill algorithm
It is also known as “bucket” fill. Idea behind it is similar to graph connectivity. Firstly we find a seed pixel and then calculate the difference of intensity values between its neighbors. If it is lower than specified value (called tolerance) then they become part of the mask. We repeat the process for every “positive” pixel. In the end we receive a set of connected pixels with similar intensity values.
In case of CT scans it is very useful for segmenting whole tissue groups like lungs and trachea, bones and abdomen. However you have to pay attention to a seed point (so is in the desired area) and the tolerance value, because both of them have a tremendous effect on the algorithm outcome.
Watershed algorithm
Watershed algorithm is also intensity based algorithm. To illustrate how it works let’s imagine an image as a landscape where low pixel intensities are planes and high ones — mountain peaks. Probably you can imagine valleys that are created from all intermediate values. The algorithm looks like there is water rising from specific lower areas up the hills. As a result we can select specific valleys that are ‘under water’, thus obtaining region segmentation.
Furthermore if 2 regions meet during ‘pouring the water’ then the algorithm sets a border between them. It allows for even more detailed region-based segmentation (like lobes of lungs or specific abdominal organs).
Even though the algorithm seems simple it requires very specific preprocessing: regions definitions. We need three of these: one that contains pixels that are certainly not included in segmentation (like air around patients or bones). Second is the area that is definitely within the mask (specific clusters of points) on segmented organs where ‘water starts pouring’. Usually it is obtained from distance transform of thresholded region. The third one is the area that is to be searched (usually everything that is in neither of the previous regions).
The main difference between watershed and flood fill is that it keeps region splits based on the lowest points of the mask. Also watershed is very dependent on initial regions and has tendency to include area around the tissue in this particular case.
Comparison to Deep Learning
After the AlexNet revolution in 2012 deep learning algorithms (especially CNNs) entered the world of medical images processing. They have far better performance when it comes to segmentation and also can classify images. However they have several tremendous flaws. They require a ton of well labeled data (which in healthcare is extremely time-consuming and costly). Also they often function as black boxes: radiologists don’t know which features were important for classification or segmentation.
Classical image processing algorithms solve these problems. Their decision making process is clear and relatively easy to track (even for complex algorithms) and usually they don’t require thousands of segmented images. That is why they are still used in new tasks that don’t have labeled data. But keep in mind: they have relatively poor performance on scale. So even though they are said to be ‘automatic’ radiologists still have to tweak their parameters for specific images. Nonetheless classical algorithms still greatly help their users.
If you are interested in other posts about explainable, fair, and responsible ML, follow #ResponsibleML on Medium.
