Inference through TOMAAT client
A few days ago we have published an article about the motivations behind TOMAAT, our open source project, aiming to make deployment of medical image processing algorithms over the cloud simpler.
Since then we have developed a new version of our website and a new video tutorial on how to use TOMAAT client through 3D Slicer to process medical images in the cloud.
We have decided to follow up with a step by step guide on how to use the TOMAAT 3D Slicer integration.
Step 1: Install 3D Slicer
3D Slicer is a popular open source medical image visualization tool that can be used on windows, linux and mac platform. It is built on top of ITK/VTK and can be extended easily. It has a built-in python interpreter that makes extensions pretty easy to be developed.
To install Slicer you need to visit their website and download the install package that is most appropriate for your needs and platform. If you are adventurous, you can opt for the nightly built version, while if you don’t like surprises you can go ahead and dowload the stable version. TOMAAT client extension is available for both versions.
Open the dowloaded file and run 3D slicer for the first time! Some operating systems will ask you to allow 3D slicer to run, prompting a security alert. On Mac it is necessary to go in the ”System Preferences” panel > “Security and Privacy” and manually allow 3D slicer to run. In case of problems briefly search on Google.
Step 2: Install TOMAAT extension
Once you open slicer you will be presented with this interface:
Using the “Extensions manager” button, which is located in the right side of the toolbar, you can install TOMAAT.
The main window of the “Extensions manager” will pop up:
We need to search for TOMAAT among the multitude of modules that have been developed and published for 3D-Slicer. We can make use of the convenient search box present in the top right part of the “Extensions manager” window. Once we find TOMAAT we can click on “install” and finally on “restart” to restart Slicer. This is shown in picture (action 1: search; action 2: install; action 3: restart slicer):
Step 3: Navigate to TOMAAT
Once we restart Slicer we can access the TOMAAT module by navigating the slicer menu as shown in picture. Important note: users who install tomaat in privileged access locations (administrator access) will need to briefly run Slicer with privileges. On windows, for example, many users found necessary to run Slicer with privileges the first time after installing TOMAAT. This is due to the fact that TOMAAT extension needs additional packages installed in the internal python interpreter of Slicer. As of today these two packages are ‘requests’ and ‘requests_toolbelt’ which are installed the first time that TOMAAT runs and are needed in order to enable the client to make HTTP requests. You can find this expressed in code here (line 5, line 13).
Click on TOMAAT to load the extension and start working with it. You will see this UI:
Step 4: Load medical data into Slicer
You can load medical images having any of the most widespread medical format out there. Slicer is compatible with most formats including DICOM, Nifty and other ITK formats. You simply need to drag&drop files into the Slicer window to see them loaded and shown in the UI.
Step 5: Process medical images
In this example we will restrict ourselves to the case where we want to use a publicly available model, such as those listed here, to do inference on our newly loaded medical image.
As a first step we click on the “discover services” button which will query the public server list hosted on http://tomaat.cloud and obtain the specifics of all the services that are currently available and registered on our platform. Anyone having a working model hosted in any machine with an internet connection can add their model to this list for free, after getting an API key from us. It is also possible to run independent (unofficial) “public server lists” inside private organizations or as an alternative to the current centralized list hosted on our servers.
After pressing the “Discover services” button we obtain the list of available predictions endpoint sorted by modality, anatomy, and task.
Since we have loaded a prostate MRI scan, in this particular example, we go ahead and select the appropriate service. Other services, in fact, won’t really work with this data as they are trained for other kinds of tasks.
Once we click on the service highlighted in this picture, the TOMAAT slicer extension will build a GUI according to the interface of this specific server. Every server needs its own set of parameters to run, TOMAAT can accommodate this by implementing a mechanism to vary the UI presented to the users in order to allow them to interact with any server.
In this case, the UI highlighted in red has been built. Essential action by the user here is to select the input volume (in this case it’s called “example” and is already selected) and click the button “Process”.
At this point a warning will appear, if the user is ok with what stated in the warning the “Submit” button should be pressed in order to upload the data to the remote prediction endpoint and obtain a response back.
After a few moments the final result is obtained. It is rendered in the 3D view offered by Slicer and also overlaid on top of the image in the axial, coronal and sagittal plane views.
This concludes our tutorial which is also available in a video here. Don’t forget to visit our website http://tomaat.cloud