Using Monai Bundle and The Monai Model Zoo To Segment Medical Imaging Data

Automatic segmentation using open-source pre-trained models developed by the MONAI community

8 min readApr 28, 2023

This is the fifth and final part of a series of blog posts describing 3D Slicer biomedical image viewer and its applications, progressing from installation and basic use to using automated segmentation extensions. The first and second part of the series can be found here, and the third and fourth parts of the series discuss using the MONAI framework in 3D Slicer.

Monai Bundle and the Model Zoo

The MONAI Model Zoo hosts a collection of community developed medical imaging models in the Monai Bundle format. Monai Bundle allows you to easily pull any model that is available from the MONAI Model Zoo and import it into MONAILabel.

Model Zoo

The MONAI Model Zoo hosts a collection of medical imaging models in the MONAI Bundle format.

Monai Bundle

The MONAIBundle App is how you will gain access to these models for use within MonaiLabel in 3D Slicer. Monai Bundle currently supports most labeling models in the Model Zoo.

Available Bundles are also listed on the Model Zoo Github Page

Monai Bundle Commands¹

Monai Bundle commands are very similar to Monai Label commands. For example:

# Pick spleen_ct_segmentation_v0.1.0 model
monailabel start_server --app workspace/monaibundle --studies workspace/images --conf models model

That’s all you need to know to use the models as we will in this article. Let’s get started.

Activate Monai Environment

Before we begin, we want to return to the Monai environment that we created in the previous article. Open Anaconda Prompt and type:

conda env list

Activate the Monai environment that you created in the previous article

conda activate Monai

Download Monai Bundle

Now lets download and install the Monai Bundle application to the same folder we downloaded the radiology application to, C:/Users/User_/apps:

monailabel apps --name monaibundle --download --output apps

Now we can use any of the available models from the Model Zoo :

Let’s try the wholeBody_ct_segmentation model.

Whole Body CT Segmentation

The first model we will try from the Model Zoo is the Whole Body CT Segmentation model. This model will segment 104 structures. This model is basically the same thing as the TotalSegmentator extension, in fact, it is trained on the TotalSegmentator dataset (the networks behind the two models may be different).

Let’s use a sample 3D Slicer dataset to see how quickly we can launch and use this model from the model zoo. First, open your preview version of 3D Slicer and download the sample abdomen dataset.

Now, all we have to do is run this command, ignoring the studies method as we have already loaded our volume into Slicer.

monailabel start_server --app apps/monaibundle --studies PATH --conf models wholeBody_ct_segmentation

Once the model downloads it will automatically start. In Slicer, head to the MONAILabel module. Click the Green Arrow to connect to the MONAILabel server and load the model into 3D Slicer. Click the Run button and let your computer do it’s thing. This took about 5 minutes to run on my CPU.

Run the model to generate the segmentations

Allow the model to run. Once it has completed, within the Segment Editor module, click Show 3D, and center the volume within the 3D view.

There you go, automatic segmentation very similar to the TotalSegmentator dataset. You can compare the segmentations made by this model zoo model with the same 104 labels by running the TotalSegmentator extension on the same dataset.

Let’s use another segmentation model from the model zoo. In Anaconda Prompt, hit Ctrl+C to end the server, and confirm with Y.

Whole Brain Segmentation

Let’s download and use the Whole Brain Segmentation model. This model is for whole brain segmentation with 133 structures from a T1W MRI image. You can find more information about the model at the Model Zoo.

We will download a sample dataset from the open source Age-ility dataset. Navigate to this page and select any volume and download the volume. Once downloaded, unzip the file to folder you will create titled /datasets/Age-ility. Navigate to the folder and the T1 subfolder. Move the .nii file in this folder to the Age-ility folder. Repeat this process for any more volumes you download from the dataset.

Ignore the other files in the folder — in fact I would delete them after retrieving the image volume.

Run the following command. This will allow us to use the Next Sample button to load however many volumes you saved into the Age-ility folder. The model will have to download and then the server will start:

monailabel start_server --app apps/monaibundle --studies datasets/Age-ility --conf models wholeBrainSeg_Large_UNEST_segmentation

Once the server is started, Navigate to 3D Slicer and connect to the server with the green arrow. Use the Next Sample button to retrieve a random volume from the dataset folder, hit Run, and marvel at the segmentation! Since I do not have a GPU, the process took about 10 -15 minutes on my computer with CPU processing the segmentations.

Select **Run** to create the segmentation labels

The segmentation is quite impressive. However, I want to remove the incorrect labels created around the bottom of the volume. I navigate to the Segment Editor tool within the MONAILabel module and check Apply to visible segments. I can then select the Scissors tool.

We have some work to do with the automatic segmentations

I can now outline the area on the 3D volume and remove the area inside as I have selected (or choose a different way) to remove the portions I don’t want. After some editing, I have a much cleaner segmentation:

Alternatively, I can use Erase Outside and encircle the brain completely.

However I decide to do it, after some clean up I can end up with a better looking segmentation.

A cleaner segmentation in just a minute with Slicer segmentation tools

Although I am no specialist, the segmentation of the lobes does not look perfect, especially on the right side of the brain. The models are not perfect, but are certainly a great starting point and can save a lot of time, even if some editing is necessary at the end.

This could be due to the image features (dimensions, etc.), a fact we have not discussed in any regard. These models do work best with images that are similar to the training images. This is more for researchers who will be training models and thus we have ignored this complication. I do know that the MONAI team is working towards large training datasets and better generalization for their models in the future.

Now, Let’s use one more model from the model zoo. Les the long nodule detection model on a custom dataset to not segment, but localize possible lung lesions.

In Anaconda Prompt, hit Ctrl+C to end the server, and confirm with Y.

Lung Nodule Detection

This model is not a segmentation model but instead a lung detection model. According to the description on the Model Zoo, the goal of the model is to identify locations of possible nodules,and to assign a probability for being a nodule to each location from raw CT scans.

First, let’s find a suitable dataset to use it on. We are going to download a dataset and import the entire dataset in the 3D Slicer DICOM module.

We will use the open-source Rider Lung CT dataset available from The Cancer Imaging Archives (TCIA). You have to download and install the NBIA Data Retriever. When this is installed, you can use it to open the file from the Rider Lung CT dataset and download files.

Downloading the RIder Lung CT dataset from the TCIA

As you can see, I downloaded the dataset into a folder on my desktop. Then, in 3D Slicer, I can head to the DICOM module and import the entire folder.

Head to the DICOM module and import your dataset folder

Once the folder is imported, select and load a volume from the DICOM database.Run the following command to start the server. We have to provide a location to the “studies” method to satisfy the command even though we will not retrieve any images this way.

monailabel start_server --app apps/monaibundle --studies Desktop --conf models lung_nodule_ct_detection

The pre trained lung_nodule_ct_detection model will download and the server will begin. Once the server is started, navigate to 3D Slicer, head to MONAILabel and hit the Run button:

You can see the model output above. Rather than segmentations, it marked possible nodules. This model exemplifies that 3D Slicer can do more than just segmentation.

Conclusion

In this article we have discussed and used community developed models that are publically available in the Monai Model Zoo. Speficially, we:

Discussed and install the Monai Bundle application
Segmented 104 organs of a sample dataset with the whole body CT segmentation model
Segmented a sample brain dataset using a brain segmentation model
Localized possible lung lesions on a custom dataset using the lung nodule detection model

As you can see, the Monai framework and the Monai Model Zoo are a valuable resource for the biomedical imaging community. Using just a simple command, you can download any of the available models in the model zoo for a variety of uses, including image segmentation or localization.

The MONAI framework is a fairly new tool, having been released in 2020. Thus, the functionality of both MONAILabel and Monai Bundle, as well as the collection of models available in the model zoo, will only continue to grow as time passes.

That concludes this series on automatic segmentation in 3D Slicer. You should now have a wide variety of tools at your hands to jump-start any segmentation efforts you want to undertake. You can check the Monai Bundle Github for an updated list of models in the model zoo and use them at any time.

Happy segmenting!

Monai Bundle Resources

There are many resources and tutorials available for Monai Bundle and the Model Zoo on the MONAI Github page.

References

[1] Project-MONAI. (n.d.). Monailabel/sample-apps/Monaibundle at main · Project-MONAI/Monailabel. GitHub. Retrieved April 28, 2023, from https://github.com/Project-MONAI/MONAILabel/tree/main/sample-apps/monaibundle