Machine learning workflows for large-scale reconstruction projects
Reconstructing biological objects (e.g. neurons) from tera- or peta-scale image datasets is a challenging task. webKnossos has some helpful features to assist with large-scale collaborative reconstruction projects.
webKnossos is a web-based platform for visualizing, annotating and sharing large 3D image datasets. One of the most popular use cases for webKnossos is reconstruction of Connectomes from 3D electron microscopy (EM) datasets. For that, neurons need to be segmented and the synapses between pairs of neurons need to be detected. The datasets typically range from several terabytes up to petabytes in size. Due to the large volume of data, automated reconstruction is required. webKnossos supports these machine learning based workflows through several features. Although, webKnossos works great for Connectomics, it also supports many other image reconstruction use cases in life sciences.
Generate training data
webKnossos has several annotation tools that can be used to generate ground truth to serve as training data for machine learning models. Use the brush or lasso tools to annotate segmentations. There is a task system that allows to distribute the work among multiple annotators. With a project dashboard, you can easily follow the progress. webKnossos supports redundant tasks in order to implement a quality control scheme. Once the project is complete, all annotations can be collected into one place for further processing.
With this training data, you can train a model in your favorite deep learning toolbox. Check out Voxelytics, which is our specialized toolbox for large-scale reconstructions.
Visualize predictions
Once you have a model that works well, you can run it on your dataset and load the resulting probability maps. webKnossos supports visualization of multiple layers as overlays on the raw data. The per-layer histograms are useful to understand the range of values. Use the sliders below the histograms to select good probability thresholds for turning your prediction into a segmentation. With the sliders this can be done in an interactive way.
When you notice some issues with your prediction, you can easily send a link to a collaborator and they will be able to see exactly the same data at the same location as you are. Additionally, you can annotate error locations with the annotation tools. This is very handy for collaborative debugging.
Visualize and proof-read segmentations
webKnossos has great support for working with instance segmentations. Again, you can load them as an overlay to the raw data. Each segment will be shown in a different color with an additional random pattern to help visual distinction.
Many large-scale segmentation approaches are based on over-segmentations that have additional agglomeration steps, where segments are automatically merged together. webKnossos has ID mapping built in, which makes it easy to switch between different agglomeration strategies without rewriting the data on-disk.
Manually fixing over-segmentations can be easily done with the “merge mode”, where an agglomeration graph is interactively created with the skeleton annotation tools. Existing agglomeration graphs can also be loaded and edited from webKnossos. Use the brush tools to correct any contours that are off. The task system allows to distribute a large amount of proof-reading tasks among annotators.
Render and export meshes
Once you are happy with the state of your segmentation, you can visualize meshes of your reconstructed objects in webKnossos. These meshes can either be precomputed by an external tool or generated on-demand. Take pictures directly from webKnossos or download them as 3D files (STL) for using them in your favorite rendering tool (e.g. Blender, Amira etc.).
Publish your data
When you are ready, you can use webKnossos to publish your data alongside your paper. Link directly from your paper or a figure caption to a location in webKnossos. With that, your readers will be able to explore your data and your story interactively. Of course, you can also publish any annotations with the dataset.
Check out the publication gallery on webknossos.org to see a selection of datasets from the scientific community. We are happy to add your dataset to the gallery.
Try out webKnossos with any of the published data or with your own (private) datasets on webknossos.org. Get a free account today.
Image credits
- Predictions, segmentations and meshes by scalable minds
- Raw EM data, Mouse Cortex by Motta et al., Science, 2019
