BEHIND THE SCENES OF WEBKNOSSOS
INTERVIEW WITH PHILIPP OTTO
Today, we chat with Philipp Otto. Philipp has been one of the principal developers of WEBKNOSSOS for a decade and will share his story and insights from behind the scenes of developing such a product, including the challenges and the enjoyment it brings.
What does a typical working day at scalable minds look like?
My typical working day involves a lot of programming, naturally, since I am employed as a software developer. This includes coding new features, as well as improving performance and investigating bug reports from users. However, there’s also a project management aspect to my role, where I write and prioritize issues or delegate them to ensure we continuously improve the product. Collaboration with my colleagues is a significant part of my day, as well. We discuss problems and solutions across various areas.
Do you work only on WEBKNOSSOS, or do you have other projects?
I also work on Voxelytics, which is our machine learning pipeline for automated alignment and segmentation of datasets. My primary focus within that project is on the alignment service.
What would you say is your favorite part of your job?
My favorite part is the small joys that come with a “breakthrough”. Often, you work on something for a while without success, and then suddenly, everything falls into place, and it works — that’s always exciting. Whether it’s fixing a difficult bug and discovering its root cause or deploying a major feature after extensive development and releasing it to our users, those moments are incredibly gratifying. Additionally, seeing our software used globally in meaningful ways is very rewarding. It affirms the value and impact of our work.
What are typical challenges you face in your work?
Dealing with large datasets, sometimes in the realm of petabytes, is a constant challenge. We must ensure our methods scale effectively with these massive volumes of data, a task that confronts me almost daily. Moreover, maintaining backward compatibility is crucial since many users rely on our platform; we cannot afford to disrupt their work by introducing breaking changes. Investigating and resolving bugs also represents a recurring challenge in ensuring the reliability and performance of our software.
Do you have examples?
Yes, one example comes to mind. WEBKNOSSOS uses WebGL to render data, and since this runs in the user’s browser, our software depends on various factors like the browser version, the computer’s graphics card, and the installed drivers. This dependency can lead to problems due to the vast array of hardware, drivers, and their imperfect implementations, over which we have little control. We encountered an issue where a user reported that one of the WEBKNOSSOS viewports was displaying entirely red without showing any data. This issue was specific to a particular dataset, but it was challenging because it could not be reproduced by our team and no logs indicated details about this bug. Fortunately, we had the opportunity to meet this user at a conference and could use their computer to debug the shader code running on the GPU. Ultimately, we discovered it was a driver bug, not a fault in our software. Nevertheless, we had to find a workaround. By making a minor adjustment to the shader code — a two-line change that theoretically should not have made any difference — we managed to fix the rendering issue on this specific computer.
Do you have another challenge in mind?
Another challenge is the scalability of our software with large datasets. During a hackathon, our team experimented with a split feature for segmentation data, which is essential for separating two cells from each other that were annotated as a single segment before. Our initial approach worked for small datasets by performing a “min cut” on the voxel level, and with additional adjustments, it could somewhat handle medium-sized datasets. However, it became clear that this method was not scalable. In retrospect, the limitations should have been apparent, but the excitement of the hackathon carried us away. We had to go back to the drawing board and develop a new approach. The split feature in WEBKNOSSOS now functions as part of a proofreading tool, using a special data structure called an supervoxel graph that must be pre-computed. With this setup, splitting and merging segments is feasible even for our largest datasets. The hackathon served as a proof of concept for this new approach, which we later refined to address scalability.
How do you decide which features to develop?
We try to put ourselves in the shoes of our users to understand their needs and workflows. This often involves dog-fooding our products, meaning we use them ourselves. For instance, Voxelytics computes alignments and segmentations, and the results are displayed in WEBKNOSSOS. Similarly, when annotating new training data for machine learning projects, we use WEBKNOSSOS — getting first hand feedback from our team. We also conduct user studies to gather insights more systematically, ranging from broad surveys on user satisfaction and feature requests to targeted one-on-one interviews on specific topics. Additionally, users can directly communicate their needs or report issues via email or by opening an issue in our system. This helps us prioritize development efforts.
Last but not least, we regularly have hackathons, where we let our creativity run wild. Following these events, we decide whether the feature ideas align with our users’ actual needs before developing them further. A good example of this is the quick-select feature, which started during a hackathon and is now a widely used feature in WEBKNOSSOS.
Interesting! How many projects from hackathons would you say are successful?
It depends on how you define success, but I would estimate about 20–30% have an impact on our product. Our company views hackathons as opportunities to explore new features or projects without the expectation of immediate practical outcomes. They allow us to tackle ideas that wouldn’t typically be prioritized during regular office routines, which I find very valuable.
What motivates you to keep innovating and pushing the boundaries with WEBKNOSSOS?
The real drive comes from seeing our work used in meaningful research. Knowing that WEBKNOSSOS and our other tools help scientists achieve their goals is incredibly rewarding. It’s all about making a difference where we can and learning as we go.
