Installing The MONAI Framework To Segment Medical Images

Learn about the MONAI framework for machine learning with medical imaging data.

The MONAI framework

This is the third part of a series of blog posts describing 3D Slicer biomedical image viewer and its applications, progressing from installation and basic use to using automated segmentation extensions. The first and second parts are available, and the fourth and fifth parts of the series discuss using the MONAI framework in 3D Slicer.

In this article we will install the MONAILabel 3d Slicer plugin for automatic segmentation of biomedical images. As a primer for the article, watch this quick 4 minute video announcing the release of MONAILabel[1].

This article assumes that you are familiar with the 3D Slicer software. If you are not. it is a good idea to review the previous articles in the series.

Introduction

In a previous article I briefly introduced Convolutional Neural Networks (CNN). CNN’s are a form of a deep neural network that are the power behind the deep learning image segmentation tools that are available for 3D Slicer. While we will not revisit the details of CNN’s in this article, the MONAILabel plugin for 3D Slicer allows for easy implementation of a number of different CNN network structures to segment biomedical images.

We will not be changing any parameters or features of the networks that we use in this article. Instead, we will be using these models as “plug-and-play” models that interested clinicians can use as desired in their workflow to save significant time when wanting to to segment images. However, I will also outline the process of training a custom model.

What is MONAI?

The Medical Open Network for Artificial Intelligence (MONAI) framework is a collaborative open-source initiative built by academic and industry leaders for deep learning in healthcare imaging. It has broad applications in medical imaging, and is aimed at both researchers and clinicians. Clinicians can use MONAILabel to reduce the time and effort of annotating new datasets. Monai is open source, standardized, user friendly, reproducible and can be easily integrated into clinical or personal workflows, which makes it a powerful tool for segmentation of biomedical images.

The MONAILabel extension for 3D Slicer allows users to segment images automatically using pre trained models, as well as the functionality to easily train new models or use transfer learning to quickly train a model on a new dataset on an AI model.

MONAI is built on top of PyTorch, which is a deep learning library for the Python programming language. When you enter commands into the Anaconda Prompt, you are using the MONAI Application Programming Interface (API). You can use MONAI through the API or through 3D Slicer, which is how we will use it. Do not worry! The aim of this article is for the clinician to learn to quickly import datasets into 3D Slicer and segment them and save time. You will not have to code in Python or any other scripting language to benefit from the MONAI framework. I will provide and explain any commands that you are required to execute to allow us to successfully use the 3D Slicer plugin to save significant time annotating or segmenting biomedical images.

MONAI supports image management solutions such as DICOMweb or datasets stored in a local directories. MONAI comes with default pretrained segmentation models which we will discuss and use in the next article in the series. MONAI also provides a Model Zoo offering many more pretrained models built by the MONAI user community, all of which are implemented easily using the MONAI Bundle tool that we will discuss in the final article of this series. Both MONAILabel models and MONAI Model zoo models models are quickly launched and easy to use on custom datasets — this is the power of the MONAI framework.

In this article we will install MONAILabel and it’s requirements. In the next article, we will use the 3D Slicer MONAILabel plugin to import data and automatically segment structures using the segmentation models, and in the final article of this series we will download and use models from the Monai Model Zoo.

Anaconda Distribution Manager and Python

In order to be able to install and use MONAILabel, we first need to install Python and Anaconda. Python is the programming language that MONAILabel is written in and Anaconda is an environment manager that we will use to install Python.

All that you need to know about Anaconda Distribution Manager for the purpose of this article is that it will install Python to your workstation and also will control conda environments. A conda environment is essentially a folder into which you can install software, keeping each environment in separate folders from the others. This is a useful way for individuals to keep their software installations separate and reduce interactions between them.

We will install the Anaconda Distribution Manager, which will also install Python and it’s important data science libraries to your computer.Once this installation is done you will not have to return to this process and you will now have the power of the Monai framework and the MONAILabel extension at your fingertips.

Install Anaconda Distribution Manager

In order to install Anaconda, you simply have to navigate to the install page of the Anaconda website. Download the installer and run it. Follow the prompts and install Anaconda. Do not add Anaconda to your path environment variable.

Simply follow the prompts to install Anaconda on your system

Do not add Anaconda to your PATH environment variable

You will find new applications on your computer called Anaconda Navigator, Anaconda Prompt, and Anaconda Powershell Prompt. Do not worry about these programs for the purpose of this tutorial; however, we will be using Anaconda Prompt to execute commands shortly.

Create A Conda Environment and Install MONAILabel¹

Virtual environments are a feature that exist so as not to cause issues between Python libraries. While you have likely just installed Python for the first time, it is good practice to manage your Python installations.

The conda package manager allows us to easily manage our package installations with conda environments. A conda environment is simply a directory that contains a specific collection of conda packages that you have installed (source). You will use Anaconda to create a conda environment called “Monai”, into which you will install the Monai library. We will include a command to create the environment with Python v3.9 (not all Python versions are compatible with Monai).

To create the environment, open Anaconda Prompt and type:

conda create --name Monai python=3.9

Press y to proceed.

Check to see if the environment was created correctly:

conda env list

You will see a list with the environment “Monai”. Activate the environment by executing the following code:

conda activate monai

We will install some dependencies into this environment from a text file by running this code provided by Monai:

pip install -r https://raw.githubusercontent.com/Project-MONAI/MONAI/dev/requirements-dev.txt

Downloading the Python dependencies required to use Monai

Now that we have all the behind the scenes python dependencies taken care of we can finally install MONAILabel to our environment:

pip install monailabel==0.7.0rc5

Finally! We have installed the MONAILabel python library in a containerized environment and we can continue our segmentation efforts worry free of any software issues. Leave the Anaconda Prompt window for now.

You can verify the installation of all your new software packages by executing this code and reviewing the list:

pip list

When you are done working, you can deactivate the environment and close the prompt:

conda deactivate

This environment will have to be activated each time you want to run the MONAILabel extension.

Up next we will install software to speed up the processing time for segmentations.

Install CUDA for faster segmentation processing

Deep learning networks such as those behind the MONAILabel package process many mathematical calculations to segment structures in medical images. These calculations take a lot of processing power and consequently, can take a lot of time.

NVIDIA CUDA will allow you to use our GPU to decrease processing time of image segmentations if you have a capable GPU in your workstation. We installed CUDA in a previous article.

If you do not have a CUDA capable GPU (check here) you will just use your CPU to process your segmentations. The downside to this is that the process will take much longer.

If you do have a capable GPU, check the previous article, or follow the installation instructions provided on the website. If you are not sure if you already have CUDA installed you can check by opening Anaconda Prompt and typing:

# Check if cuda enabled
python -c "import torch; print(torch.cuda.is_available())"

If you do not have a GPU or CUDA is not installed, this will return False. If CUDA is not installed, follow the installation instructions here and run the code again and you it should return True. This means that CUDA is installed on your computer and can be used with monailabel.

Great! We now have everything we need to begin using the 3D Slicer. We have Python and it’s important data science libraries installed, a tool to speed up our processing time, and the monailabel Python package installed as well.

Now, we need to install the preview version of 3D Slicer.

Install the 3D Slicer Preview Version and MONAILabel

We need to install the preview version of 3D Slicer in order to have full functionality of the MONAILabel extension at the current time. Head to the download page and download and install the preview version of Slicer.

credit

Once downloaded, open the preview 3D Slicer on your workstation and install the MONAi Label plugin.

• Go to View -> Extension Manager -> Active Learning -> MONAI Label
• Install MONAI Label plugin
• Restart 3D Slicer

You should enable Developer Mode as well. To do so,

• Go to Edit -> Application Settings

Application Settings

In the Application Settings window you will have to enable developer mode twice. Above you can see the MONAI Label tab. You must check Developer Mode here. Following that, go to the Developer tab highlighted in red and check Enable Developer Mode. Restart 3D Slicer again.

Finally! You are completely ready to begin using the MONAILabel 3D slicer extension. Let’s learn about the extension before we begin.

MONAILabel³

Installing the 3D Slicer plugin will give us the ability to download and use pre trained segmentation models through the MONAILabel radiology application. Watch this 5 minute video to get an overview of the tools available in the MONAILabel 3D Slicer plugin.

The MONAILAbel Radiology application has sample models for auto segmentation with the latest deep learning models (e.g., UNet, UNETR) for multiple abdominal organs, as well as interactive tools DeepEdit and Deepgrow for actively improving trained models and segmentations.

Models available in the radiology application

Description of available models — Credit

You use MONAI through the Python API. We will be entering commands in Anaconda Prompt. Here are a list of commands that you give in the following format² :

monailabel start_server --app ChooseApp --studies path/dataset --conf model ChooseModel

# skip this if you have already downloaded the app or using github repository (dev mode)
monailabel apps --download --name radiology --output workspace

# Pick DeepEdit model
monailabel start_server --app workspace/radiology --studies workspace/images --conf models deepedit

# Pick Deepgrow And Segmentation model (multiple models)
monailabel start_server --app workspace/radiology --studies workspace/images --conf models "deepgrow_2d,deepgrow_3d,segmentation"

# Pick all stages for vertebra segmentation
monailabel start_server --app workspace/radiology --studies workspace/images --conf models "localization_spine,localization_vertebra,segmentation_vertebra"

For example, if your images have been downloaded to a folder called “datasets/Task09_Spleen/imagesTr” , to start the DeepEdit model you would type:

monailabel start_server --app radiology --studies datasets/Task09_Spleen/imagesTr --conf models deepedit

DeepEdit

This model works for single and multiple label segmentation tasks. Using this model you can choose to either segment images using the pre-trained model or you can choose to disable the pre trained model, change the labels, and train your own model. That is outside the scope of this article; however, it is the same process as the tutorial that we will be using on the spleen dataset.

monailabel start_server --app workspace/radiology --studies workspace/images --conf models deepedit

DeepGrow

This model based on deep grow: an algorithm that combines the capabilities of multiple models into one, allowing interactive segmentation based on foreground/background clicks. We will not be using DeepGrow.

monailabel start_server --app workspace/radiology --studies workspace/images --conf models deepgrow_2d,deepgrow_3d

Segmentation

This model based on UNet for automated segmentation. This model works for single and multiple label segmentation tasks. Using this model, we can import datasets and quickly segment 104 organs.

monailabel start_server --app workspace/radiology --studies workspace/images --conf models segmentation

Multistage Vertebrae Segmentation

This is an example of a multistage approach for segmenting several structures on a CT image. The model has three stages that can be use together or independently that first localizes and than segments all vertebrae of the spine.

monailabel start_server --app workspace/radiology --studies workspace/images 
--conf models localization_spine,localization_vertebra,segmentation_vertebra

Download the radiology App

We need to run one of the commands above to download the radiology app. We will download it into a folder located at C:\Users\user\apps. We do this by passing the command — output apps. We will repeat this step with the monaibundle app in the next article.

monailabel apps --download --name radiology --output apps

MONAILabel file structure

MONAILabel works with DICOM, NRRD, and NifTI images. However, there is a set folder structure that MONAI Label uses.

Place your image data in a folder and, if you have any segmentation files, create and place them in a subfolder called labels/final. You can see an example below:

In the above image, green files are the volumes. Red files located in labels/final folder are the annotations of green files (volumes).

All of the slices for a single scan have to be in a single file. In the below example, spleen_10 is a single CT scan, as is spleen_11, and so on. You can see this sample dataset for an example of how the files can be structured. Each of the 20 files in that dataset is a seperate CT scan of a patient. We will see the structure of the dataset that we use in the next article.

dataset
│-- spleen_10.nii.gz
│-- spleen_11.nii.gz
│   ...
└───labels
    └─── final
        │-- spleen_10.nii.gz
        │-- spleen_11.nii.gz
        │   ...

Conclusion

That was a lot of work and a lot of information. I think that before we begin segmenting structures, we should take a break. Close out 3D Slicer, run this command in Anaconda prompt:

conda deactivate

Finally, you can close Anaconda Prompt.

That wraps up this article. Let’s review we have covered:

1. Introduced to the MONAILabel extension for 3D Slicer
2. Installed the weekly preview version of 3D Slicer
3. Installed Anaconda Distribution Manager and Python
4. Set up a conda environment for MONAILabel as well as its dependencies
5. Installed the monailabel Python package and 3D Slicer Plugin Extension
6. Detailed the pre-trained models provided in MONAIALabel 3D Slicer extension

Up Next: Using MONAILabel

Next, meet me in part 4 of this series where we will import a dataset and immediately segment structures using MONAIlabel. Then, in the fifth and final part of this series, we will use available custom models available from the Monai Model Zoo.

Monai Resources³

• Website: https://monai.io/
• API documentation: https://docs.monai.io/projects/label
• Code: https://github.com/Project-MONAI/MONAILabel
• Project tracker: https://github.com/Project-MONAI/MONAILabel/projects

References

[1] Installation¶. Installation — MONAI Label 0.7.0rc6 Documentation. (n.d.). Retrieved April 28, 2023, from https://docs.monai.io/projects/label/en/latest/installation.html#windows

[2] Sofisappia. (n.d.). Sofisappia/monai_radiology_app. GitHub. Retrieved April 28, 2023, from https://github.com/sofisappia/monai_radiology_app/

[3] Project-MONAI. (n.d.). Project-Monai/Monailabel: Monai label is an intelligent open source image labeling and learning tool. GitHub. Retrieved April 28, 2023, from https://github.com/Project-MONAI/MONAILabel