We want to make research accessible

Computer vision and medical image analysis have been transformed by the advent of modern machine learning and in particular of deep learning. Published research has become more and more applicable in practical scenarios and, as newer and more sophisticated approaches are proposed, it becomes necessary to find a way to make them accessible by the community. With TOMAAT we hope to be able to pave the road towards a standard way to deploy medical image analysis algorithms. We want to provide flexibility, ease of use, speed, accessibility and scalability. We decided to open all of our source code, such that anyone who is sufficiently interested can contribute and improve or extend this work.

Being able to benchmark and compare different algorithms on a specific dataset is essential during the development of novel methods. This is useful when we want to see how other approaches deal with our own challenges and when we need to draw conclusions about their advantages and shortcomings. Right now, although many authors release their implementations and pre-trained models on the internet, the process of accessing the methods to make comparisons is tedious and time consuming. Handling specific data types might be problematic, for example, or there might be other unforeseen issues at runtime. There is no standard way to access pre-trained models, such that data loading, pre-processing, inference and post-processing (let alone communication over the network) can be run seamlessly with a couple of clicks. One of the main purposes of TOMAAT is to solve this situation by having public prediction endpoints in the cloud that can be accessed through a convenient client (Eg. from 3D slicer).

Another important issue arises when we are developing a demo or a prototype that necessitates of some image analysis algorithm. It may happen that the expertise to install, run, refactor and re-package the algorithm is not immediately available. With TOMAAT it is possible to realize local deployments that expose a standardized yet flexible interface which can be accessed programmatically using our python implementation available on GitHub. In this way third party algorithms can be embedded into own workflows without the need to install code or, when using the cloud, fulfil hardware requirements.

In a research setting, where novel algorithms are tested in a clinical setting, TOMAAT can help deploy algorithms in the local network such that they can be available from offices and operating rooms without any need to bring bulky hardware onsite, and with minimal latency. All it’s needed is a simple network connection.

Our goal is, ultimately, to serve the research community such that new methods can be built upon older ones which are available and ready for benchmark in the cloud. At the same time, state of the art approaches can be made public and become popular, ultimately getting the recognition they deserve. We want to empower everyone to be finally able to compare, validate, embed and use state-of-the-art approaches easily, with a few of clicks or a couple of API calls and with the maximum flexibility.

This is a sketch of our current architecture:

As you can see TOMAAT includes both server and client software as well as an “announcement service” which keeps a list of public services available worldwide.

We provide a convenient API to package apps (an app packages pre-processing, inference and post-processing) and turn them into services (exposing an interface to the outside world). Services are available through the network. It is possible to specify in a very straightforward manner the interface that each service needs in order to feed the methods with the data required to run. That is, each service can require a variable number and type of parameters to run, and this can be accommodated seamlessly by TOMAAT. You just need to define what you need and TOMAAT will take care of the rest.

Once services are running it is possible to announce them to the “announcement service” which keeps a centralized list of available services. Of course, not all services need to be made public, for example it would be possible for an organization to have services that are available only locally.

Clients can at that point connect either directly to any running service whose hostname is known or by retrieving the list of available services from the announcement system. The client obtains the service interface at this point and discovers what kind of data the algorithm needs in order to run. According to this “blueprint” it builds the user interface. At that point the prediction request can be made. An example of TOMAAT client can be found here and is available in the 3D-Slicer extension manager.

We believe this provides flexibility to accommodate diverse workflows.

We aim to also provide docker containers of public services through dockerhub in order to enable people to run their own endpoints locally or to create mirrors of our services.

When algorithms are turned into services via TOMAAT, they expose a standard but flexible interface which is something we believe was missing in the panorama of current practices in ML research (especially applied to healthcare).

We believe that TOMAAT is still in an embryonal phase and we want the community to shape it. Our final goal is to make research even faster than it is today in a way that possibly goes even beyond deep learning or medical image analysis. We need ease of use and extreme flexibility to allow diverse use of our package.

Please learn more by reading our paper which will be presented in a short format at MIDL 2018.