Fast Skeleton Annotation in webKnossos
webKnossos has very efficient modes of manual annotation for neurons. Flight mode is a unique interaction mode that enables users to create skeleton traces very fast.
This is the second post in a series of webKnossos-related posts. Read this introductory post about webKnossos. The next post will be about webKnossos’ volume annotation features.
Dendrites and especially axons are long and tube-like processes in the brain. In the Connectomics community it has become popular to reconstruct these neurons by an approximation of their graph-like structure. We call these approximations skeletons. webKnossos has very efficient modes of manual annotation for skeletons. They are enabled by fast data delivery mechanisms and unique annotation tools.
Fast delivery of 3D image data
In order to enable fast annotation speeds, the data needs to be streamed to the browser in an efficient way. The first novel optimization in webKnossos was to transmit cube-based data instead of section images. Since the data is interacted with in a three dimensional way, it is a good design choice to also deliver 3D containers of data. After experimenting with several data cube sizes, we decided on 32×32×32 voxel cubes. That gave us the best results considering the bandwidth implications of over-fetching and latency constraints of many small requests. When navigating through the data, webKnossos prefetches cubes in the background. For that, a preview volume based on the direction of the user movement is computed. Other optimizations such as GZIP-encoding, HTTP/2 and custom on-disk file formats also improved the overall performance of the data transmission.
In the webKnossos paper, we report a benchmark of the achieved delivery speeds. The data transmission is fast enough to allow for annotation speeds that exceed the human cognitive limits. Even on lower bandwidth (e.g., mobile 3G) and high latency connections (e.g., server in Europe and client in South America), the speed is acceptable for productive annotation.
When the data reaches the browser, it is piped onto the graphics card and rendered with WebGL. We implemented a lot of optimizations to make the rendering of and interaction with the data as fluent as possible.
Flight mode: Projecting data on a sphere
Once we had fast data delivery mechanisms, we set out to improve the user interaction modes as well. Previously, users would annotate skeletons in orthogonal views. However, we found that tracing neurons in orthogonal views is suboptimal. Neurons are long tube-like processes that have curves in them. When following these curves, the users need to switch viewports frequently to get a good angle. Our idea was to use arbitrary reslices of the data that are orthogonal to the curvature of the neuron.
Additionally, we decided to project the data onto a sphere which allows to rotate the viewport in a continuous manner. With straight slices, the data on the periphery would be changing when rotating. We found straight slicing to be very confusing, because the context of the data is frequently lost when following the curve of the neuron. With the spherical projection, this kind of navigation feels more natural.
Combining this spherical projection and the fast data delivery, we published a novel annotation mode that we call “flight mode”. The name was coined, because the user has an ego-centric perspective of the data and moves with typical flight simulator controls.
Flight mode is fast and accurate
Flight mode enables very fast neuron tracing. We measured annotation speeds of 1.5 to 2.1 mm/h in mouse cortex which demonstrates a 4- to 13-fold improvement over existing methods. We also quantified the error rates in flight mode compared to orthogonal mode and did not find significant changes in accuracy. In practice, we also find that flight mode takes less training for new annotators than orthogonal mode.
Today, flight mode is frequently used for long axon tracings in order to establish ground truth for automated approaches. It is also used for short proof-reading tasks. When dealing with over-segmented datasets, flight mode is an efficient way to determine the connectivity of two neighboring segments. Either annotators end up in the queried segment or not. Motta et al. describe this method in their recent publication (preprint). webKnossos has task hot-swapping capabilities that make it possible to work on very short tasks and quickly switch to the next.
Try it out now
You can start flying through a published dataset right away. By signing up for an account, you will be able to save your annotations and start collaborating with others. How fast will you go?
The next post will be about the advanced volume annotation features of webKnossos. The previous post covered the origin story of webKnossos in the field of Connectomics.
