Google Summer of Code 2021
Project title: Input/Out Workflows for Active Segmentation Platform
Developers:
Student: Piyumal Anthony Demotte
Mentors: Dimiter Prodanov, Sumit Kumar Vohra
Organization:
International Neuroinformatics Coordinating Facility
About Active Segmentation: https://sumit3203.github.io./
My contribution: https://github.com/piyumalanthony/ACTIVESEGMENTATION/commits/workflow_seg
Project description: ImageJ is extensively used in major areas of biological and material sciences. Previously developed active segmentation platform as a plugin for ImageJ incorporate Weka toolbox-based statistical machine learning algorithms as well as deep learning techniques for trainable image segmentation. The end goal of the active segmentation platform for ImageJ is to provide researchers an extensible toolbox enabling them to select custom filters and machine learning algorithms for their research. Especially previously developed semantic segmentation-based techniques are currently in the process of integrating with ImageJ under the Active Segmentation plugin. These techniques are especially based on deep learning algorithms which are state-of-the-art performing procedures for image segmentation.
Under the existing implementation of the active segmentation platform, it only supports users with a limited way to load the ground truth for learning (at the moment only as of the region on interest format (ROI)). Thus, this reduced the usability of the tool and urges the users to convert the ground truth to the specific format which is designed to be used within the application. Thus this design is not useful for multiple applications. Therefore the main contributions under this project will be to incorporate TIFF-based ground-truth format as an image-based format in which each pixel uniquely belongs to a particular class and, partial ground truth format in which instead of the whole image and several partial boxes in an image or stacks are labeled.
Therefore, this project includes mainly incorporating ground truth labels-based segmentation implementation for weka classifiers and enhancing deep learing based training. The workflow for ground truth labels-based segmentation is completed and the capsule network-based deep learning procedure is yet to be completed. The rest of the project will be continued after the GSOC period.
Development details: The integration of ground-truth-based input/output workflow with the active segmentation platform has reused the existing project creation and project opening procedure. Here users could create a segmentation project and load training images as indicated in figure 1 and figure 2.
The active segmentation panel was revised and added the functionality to load ground truth labels from TIFF-based image format and train classifiers based on user preferences. The bit-depth of images is expected to be 8 where the classification could accommodate 256 classes for classification.
As indicated in figure 4, the users have the ability to load the ground truth labels and visualized the ground truth labels as an overlay. Figures 5, 6 and 7 display the procedure to load the ground truth labels and the overlay image after loading the ground truth label images.
The lookup table was designed to change the meta information of the segmentation classes that resides in the ground-truth labels. This meta-information includes color and class label of ground truth classes. Figures 7, 8, and 9 illustrate the usage of the lookup table to change the meta information of classes.
Users can set the number of instances for training out of the training instances loaded. The rest of the loaded images would be utilized for testing purposes only. The segmented images could be illustrated as an overlay and a mask as in Figures 12 and 13.
Further proposed functionalities: Capsule network implementation for image segmentation
Active segmentation platform at the moment includes u-net and segnet deep learing architectures for active segmentation. Furthermore, these deep learing architectures do not perform well with less data availability. Capsule networks have proven to be effective for most medical imaging tasks with less labeled data. Thus under this project as a customized model capsule network-based procedure is proposed to be implemented. Users will be able to customize the model based on the number of capsules as well as the model parameter with regard to the existing model. deeplearning4j will be used for the implementation of capsules. This will be a good opportunity since no other platform still provides capsules based image segmentation which is state-of-the-art for medical imaging at the moment. The following paper will be used for the implementation.
