New annotation and usability features in webKnossos

Today I would like to give you an overview of the many new features and UX improvements that we added to webKnossos over the last half-year.

If you want to try the new features yourself, head over to https://webknossos.org and sign up for a free account.

At scalable minds, webKnossos is our primary tool for viewing the results of our automated neuron reconstructions, collaborate on training data annotation, and share the results with colleagues. Many of the new features are therefore inspired by our work in Connectomics and biomedical image reconstruction.

A dense, volume neuron reconstruction in webKnossos. The segmentation is the result of scalable minds’ automated machine learning pipeline. EM data: Mouse Cortex by Motta et al., Science 2018

Usability updates to the look and feel

We want to make webKnossos as easy to work with as possible, especially for new users. To that effect, we made many of the annotation modes and tools available through buttons and menus instead of relying solely on keyboard shortcuts. (No worries, power users can still use those.)

A good example of this is the new annotation toolbar at the top of the screen when opening a dataset. By default, webKnossos is launched with the move tool being active. (Left mouse dragging moves the data within the viewports) From the toolbar, it is easy to switch to the different volume annotation tools (brush, lasso & bucket icons for volume labeling, volume outlining, and area fill) or create a new volume segment altogether.

The new toolbar at the top of the screen has quick access to many typical annotation operations, e.g. brushing in volume annotations, flood fill operations, navigating in the dataset around, and others.

For those of you who frequently work with line-segment annotations (“skeleton annotations”), we also made many common interactions for those more discoverable. Shift + right-clicking on a node will open a context-sensitive menu with options for that node, e.g. selecting it as a new active node, deleting it, creating a new node, etc. I especially like how this makes it super easy to do operations between two skeletons: When shift-right clicking on a node from a different skeleton I have the option to place an edge and merge both skeletons.

Another little gem hidden in this menu is the ability to measure the distance between two nodes. When shift + right-clicking on any second node, webKnossos will tell you the distance to the currently active node in nanometers.

A context-sensitive right-click menu make many skeleton annotation operations directly available.

If you are interested in learning more about the webKnossos interface and basic interactions I recommend you watch our “Beginner’s Guide” on YouTube:

Improved Annotation Capabilities

For many research questions, it is sufficient to have sparse line-segment annotations (“skeletons”), e.g for neuron reconstructions. In other cases, your projects might require dense volume annotations, e.g. as training data for machine learning models in synapse detection. webKnossos has always supported both sparse and dense annotation modes, but they could only be used separately from each other. Going forward, all new annotations support all skeleton and volume tools in one combined annotation at the same time for convenience.

At the same time, we are currently working to improve working with volume annotations. There are small things like volume annotation layers now featuring a small pattern/texture to make them more distinguishable from one another which is especially useful when looking at large volume annotations with thousands of segments as produces by automated machine learning pipelines.

Pattern textures make it easier to distinguish neighboring annotation segments.

Another feature that I use all the time is the histogram slider. Use it to fine-tune the contrast and brightness for both your underlying microscope data or for any of the segmentation and volume layers. (ProTip: Our engineers also like to use the histogram to threshold prediction maps produced by an ML model.)

(left) The thresholding for each data/annotation layer can be used to improve the contrast and brightness of your data. (right) Binary prediction maps of a ML classifier can be visualized as annotation layers and thresholded for exploration.

While we are on the topic of volume annotations: You can now use the volume annotation tools (brush, outline tracer) at any zoom level. Where previously volume annotations were restricted to the default zoom magnification, you can now zoom out/in and webKnossos will up/downscale the volume annotation for you. This is great for getting an overview in an existing annotation layer or creating a very quick, rough annotation from far out.

Volume annotations can now be viewed and created at any zoom level.

And in cases where you mess up, you can now undo volume annotations to make life easier.

Undo works for volume annotations too.

(And for those of you who don’t want to annotate their datasets themselves, we now offer an annotation service where we do the work for you: https://webknossos.org/services/annotations)

What’s on our road map?

We are always looking to make webKnossos smarter. So far, we only offer EM dataset alignment and automated neuron reconstruction as services but we are looking into ways to include these into webKnossos in the mid-term. We are experimenting with ways to integrate machine learning models directly into webKnossos.

In short-term, expect more usability improvements. The context-sensitive right-click menu will be expanded and even more interactions will be made available directly through the UI.

I am also excited about proper 3D mesh support in webKnossos. Stay tuned for automatically turning volume segmentations into 3D objects and viewing them in webKnossos.

3D mesh rendering of a neuron (dendrites with spine head attachments visible) based on an automated ML reconstruction by scalable minds

What are you interested to see in the future? Let us know at hello@webknossos.org

PS. Check out the re-designed webKnossos website: https://webknossos.org