Introducing webKnossos’ nuclei classifier for Volume EM

When working in nanoscale neuroscience, you are almost always confronted with the difficulty of biological image analysis for large-scale datasets. In order to make scientific discoveries, most of the time you will need to annotate your data either manually or with the help of a Machine Learning system. webKnossos is the perfect tool to annotate large-scale datasets collaboratively. We decided to go one step further and implement the first automated segmentation feature directly into webKnossos: a pre-trained nuclei classifier.

Moving through the slices of EM-data on which the nuclei were automatically detected by the nuclei classifier in webKnossos. Have a look at the dataset here.

In this blog post, I will explain how to activate and use the nuclei classifier in webKnossos, give you some insights on how we built it and share some possible use cases for nuclei detection.

How to start an automated nuclei detection in webKnossos

Click on the “Start Nuclei Inferral” button to trigger a nuclei segmentation of your dataset.

To generate a nuclei detection, upload your raw aligned EM-data in webKnossos.

View your dataset and look in the right side bar in the “Info” tab for “Process Dataset”. Hover over it and click on “Start Nuclei Inferral”.

This automatically creates a copy of your dataset, to make sure your data and existing segmentations won’t be affected. The nuclei detection job then starts on this copy.

It might take some time until the job is completed. A small dataset (100³ vx) will need approximately 30 seconds, whereas a larger one (500³ vx) will need about 5–10 minutes.

You can view the progress of your job in the Administration tab under “Processing Jobs”.

The nuclei prediction layer.

Once the job is completed, you can open your dataset and look at the results! The nuclei prediction appears as a segmentation layer on the left sidebar. Just as other segmentations, you can work with the ID mapping, the color and the opacity of the patterns.

You can also compute and load the nuclei’s meshes, and export or visualise them as usual.

Make sure to deactivate “Render Missing Data Black” in the Settings tab to see the nuclei segmentation in all zoom steps.

Complete workflow of nuclei detection in webKnossos

Watch a complete workflow of a nuclei prediction in webKnossos on these animated images:

Step 01: Start the nuclei inferral from the main webKnossos interface. Navigate to the “Processing Jobs” page to monitor the progress.

Step 02: When the job is completed, open your dataset with the nuclei segmentation, zoom out and enjoy the results.

Step 03: Load meshes and visualise the nuclei in 3D space.

Some insights on the building of the nuclei classifier

The easily recognisable characteristics of nuclei in EM data made the implementation of a nuclei classifier a good first step towards more automated segmentation features in webKnossos.

In order to build this classifier, we trained a U-Net on multiple publicly available Volume EM datasets, including the SBEM data of mouse cortex from layer 4 by Motta et al.

To achieve an accurate segmentation, we decided to use a distance regression instead of a binary classification. Thus, the model was trained to predict the distance to the nuclei’s borders.

When visualised as a grey-scale image, voxels closer to the core of a nucleus appear in white and gradually become darker the further away they get from the nucleus’ center.

Distance prediction: The voxels are white when they are 3​​μm or more away from the nuclei’s borders, towards its center. The voxels on the nuclei’s borders take a middle grey tone (50% lightness). When the distance to the borders exceeds 3​​μm, the voxels become black.

Once we had this prediction (a.), we used 1μm as a threshold to generate watershed seeds (b.). Then, a watershed mask was created at threshold 0 (the nuclei’s borders)(c.).

Four steps of the nuclei detection: a) prediction, b) watershed seeds with 1μm as a threshold, c) watershed mask at threshold 0, d) nuclei segmentation.

This allowed us to generate a correct segmentation (d.) and avoid errors, such as having very small or long shaped objects recognised as nuclei, or having two close-by nuclei recognised as only one.

Nuclei detection only works at a specific resolution. To compute detection with your dataset, webKnossos will pick the adapted resolution itself.

Some use cases of nuclei detection

Histology

A primordial aspect of research in life sciences is to study the microscopic anatomy of biological tissues: histology. The first functional units you will be observing in a tissue sample imaged by SEM are the cells. Mapping the location and the shapes of the cells’ nuclei will tell you what the tissue is about.

Typically, in neuroscience, the concept of layers in the cerebral cortex is based on the distribution of the nuclei: In the top layer of the cortex (layer 1), there should not be any nuclei.

Isotropic distribution of detected nuclei in a sample of layer 4 of a mouse’s cortex.

However, an isotropic distribution of the nuclei (nuclei randomly spread over the whole sample) means you might be looking at a dataset fully inside one histological layer such as layer 4. On another hand, if the nuclei are for instance distributed across a plane, you might be looking at a cell monolayer — such as the mitral cell layer in the olfactory bulb.

Seeds for tracing

Carles Bosch, a neuroscientist working at the Francis Crick Institute (sensory circuits and neurotechnology lab), talked to us about some of his workflows. He and his team are using correlative imaging to map neural circuits. They are working with huge datasets, sometimes containing over 1,000 nuclei in only one dataset (or up to 300 nuclei in only one 20μm-thick layer).

Skeleton traced manually in webKnossos starting from the center of a cell’s nucleus.

For his research purposes, Carles needs all the cells’ dendrites traced manually. This manual annotation process can be very time consuming and expensive (read our article about the importance of manual annotations in nano-scale neuroscience). The distribution of the tracing tasks over the different annotators, datasets and redundancy steps is complex.

Having a complete set of starting points (seeds) for all the tasks, like all nuclei’s centers, is very helpful. The nuclei detection in webKnossos is a great way to automatically generate seeds for tracing.

In correlative imaging

In research using correlative imaging, linking two datasets obtained with different microscopy techniques (for instance SEM and Fluorescent Light Microscopy) is a challenge. You have to find recognisable landmarks (for instance, a blood vessel branching in a particular way) in each of the datasets, despite their different scales and looks. Thanks to the landmarks’ coordinates, it is possible to map one dataset on the other.

Here is when having all nuclei segmented is very useful: it will provide the complete census of cells in the tissue, and by correlating the two datasets it will provide the identity in the EM dataset of the cells that were imaged in vivo.

Whether you will continue your analysis with a dense segmentation of all neighbouring cells and their organelles or not, the nuclei detection will be an important first step. Not needing to annotate them manually will save you a great amount of time and money.

Connectomics: 3D Meshes of four fully reconstructed neurons.

The feature is available on webknossos.org for testing. We are still iterating on the quality of the classifier over time. Try it out today and send us your feedback!

Image credits

• Predictions, segmentations and meshes by scalable minds
• Raw EM data, Mouse Cortex by Motta et al., Science, 2019

Thanks to Carles Bosch for providing input to this blog post!