On Doing Deals Remotely, Backing PhD Dropouts and Building Infrastructure for the Next Generation of Software

Our investment in SuperAnnotate

As a SaaS focused investor, we at P9 have been spending quite some time looking at companies in the DataOps space. As companies start using more and more data to make and automate decisions, there’s a whole new generation of infrastructure companies that are currently emerging such as data pipeline or ML model monitoring businesses. I’ll keep it short, but if you’re interested in the space, I’d recommend reading this post from Astasia Myers at Redpoint. We’ve actually done two investments over the past 6 months in the broader DataOps space and here’s the first one we’d like to announce: SuperAnnotate.

A quick (and somewhat scary) primer on computer vision and the data annotation market

As most ML teams will tell you, 80% of their time and resources are actually spent on preparing and cleaning data rather than building models. In the field of Computer Vision, this means annotating data in order to teach machines how to recognize objects and/or movement in images. To make it very simple, if you want machines to recognize cars, you need to feed algorithms with thousands to millions of images of cars. In 2009, Fei Fei Li, who was at the university of Princeton at the time, presented ImageNet. ImageNet was one of the first open dataset of labelled data with 14 million images of 20 thousand different classes (or types of objects). Since ImageNet, there’s been a rush to collect more data in order to build new computer vision powered applications. This goes from autonomous vehicles (cars, trucks) to aerial images, drone or medical applications. Looking at it at a higher level, considering that it’s about adding perception capabilities to software, there are (and will likely) be endless possibilities across industries to build computer vision applications.

While the field is relatively new, it’s growing fast and companies are predicted to spend >$2.5bn a year just on annotation tools. Beyond tools, a whole new market of annotation services has emerged, letting companies outsource the very tedious task of annotating data. There are currently >70,000 people in the world today working full time, detouring objects and adding labels to these so that machines can learn from them. While this does look scary as humans are now serving machines (rather than the opposite) and some work in terrible conditions (more in this NYT article), it remains a necessary step towards building computer vision software. In addition to this, when it’s about detouring tumours to build tumour detection algorithms, the labelling work actually requires some expertise. Such that now, hundreds of small annotation shops have emerged in various countries around the world, annotating images in various fields in which they specialize. Last but not least, if it’s about teaching machines how to best recognize objects, the more precise is the data that’s provided, the faster will algorithms learn how to best recognize them. Simply because this reduces the noise added to the image.

Meet SuperAnnotate

SuperAnnotate’s initial product is built based on Vahan’s, the company’s CTO, PhD thesis at KTH. While there, he created an algorithm that allows humans to annotate images and videos 10x faster than a human would do on his/her own.

The video below will show you how:

This first algorithm (called superpixel annotation) is actually just the first one out of several other ML algorithms that SuperAnnotate has built as part of its platform to automate as much as possible of the annotation work. The platform now includes iterative model training (i.e. retraining the model after each annotation), object predictions (to predict the next instance of an object once several ones have been labelled), and active learning (to learn on smaller datasets).

On top of SuperAnnotate’s sets of algorithms, the platform provides useful workflow tools to help teams (e.g. users management between various roles: annotators, QA teams, engineers) collaborate while ensuring the highest level of annotation quality.

Last, the platform adds a marketplace component, where SuperAnnotate’s clients can share small pilot projects with SuperAnnotate’s partnering annotation services in just one click. Based on the pilot results, companies can then choose the right partner for their project in line with on their requirements in terms of price/quality/timing in just a few days. In short, SuperAnnotate is another SaaS-enabled marketplace, which we have had a particular love for, for quite some time now (remember that post from 2016? :)).

Investment thesis

We articulated our investment thesis around 4 points:

1. Deep ML and industry insights

First, Vahan and Tigran are two ML PhD dropouts and they have built a great team of ML experts around them in Armenia. Their initial work actually led them to attract star Computer Vision professors in the US as advisors such as Pieter Abbeel, Trevor Darrel or Fischer Yu.

But building great algorithms was not enough to apprehend the complexity of building annotation software, so the two brothers initially started by building their own annotation service in Armenia. By doing so (and scaling their team to 50+ annotators), they learned the pain points of annotation teams and their supervisors first-hand (a.k.a. they ate their own dog food) while selling to computer vision companies in the US to learn their pain points when picking an annotation service. That’s how it became clearer that they would need to build i) a faster annotation software end ii) a marketplace on top of their automated annotation platform.

2. A small but fast-growing market

We tend to like small, but new markets that are growing fast. While some annotation companies like Mighty.ai (acq. by Uber) started over 5 years ago, we believe that the timing in this market is right now. Primarily because data and open-source algorithms are now widely available so that it’s become easier than ever for anyone to start building computer vision software — not only sophisticated tech companies. As computer vision started spreading in various industries, the market for tooling also grew to hundreds of millions and market reports now predict double-digit growth within the next 10 years.

3. Long product roadmap towards owning the full computer vision pipeline

Data annotation is only the first step in a much longer engineering pipeline that goes up until deploying and monitoring models in production. In certain cases where the accuracy of the model is critical, SuperAnnotate could also add a “human-in-the-loop” component. Assuming that the model accuracy goes below a certain threshold, the image would then be sent to a human reviewer, as a step in the whole production process.

An example would be an insurance company using computer vision to predict the severity of a car crash automatically — like Tractable in the UK. Instead of automating a potentially bad decision in case the accuracy was too low, SuperAnnotate would send the image to a human expert that would guarantee the decision quality while teaching the machine how to make a better prediction at the next iteration. In the end, this is just one example of how SuperAnnotate could become a core piece of infrastructure for the next generation of software like Algolia or Contentful in our portfolio have now become for the current generation!

4. Competition

Like any new and fast-growing market, the market of computer vision infrastructure attracts competition. As briefly mentioned earlier, the first generation of labelling companies emerged 5–10 years ago and while some of them have raised significant amounts of money, none of them has become truly large so far. The first ones that seem to be getting to escape velocity emerged more recently. Scale, which raised >$100M from Index, Accel, Thrive and Founders Fund, is an ML-powered annotation service that focuses primarily on providing services to autonomous driving companies. Labelbox, which just raised a $26M Series B from a16z, is probably the most direct competitor of SuperAnnotate as they’re also building a workflow software for data annotation. While Labelbox’ workflow tool was slightly more comprehensive when we invested (and it’s probably not anymore ;)), SuperAnnotate offers significantly more automation capabilities in the annotation task, which we believe is the most important component to differentiate and compete on (because annotation is a primary driver of costs for computer vision companies).

Betting on companies that compete on what we believe to be the key cost/ROI driver of their market has been at the center of several of our investment theses in the past. Typeform has the best UX/UI in the survey market, which drives a simpler creation experience and a higher completion rate of surveys. PlayPlay emerged 10+ years after the first video production software but it also competes on the simplicity of usage and the quality of the output. In this case, we believe that the core to focus on is the automation speed while guaranteeing the highest level of quality of annotation. This is SuperAnnotate’s highest priority.

Our first deal in Armenia

Last but not least, it’s our first investment in a company in Armenia. While we at P9 are based between Berlin, Paris and London, we have been operating as a remote VC since day 1 and have been backing companies globally since day 1. Actually, since Pawel and Christoph backed Vend in New Zealand and Clio in Canada in 2011!

We believe that SaaS markets are global and that winners can come from anywhere. So we look for the team we feel most strongly about, wherever they start from. Since the COVID-19 outbreak, doing deals remotely has become the new normal but it was not completely new to us. So, we’re grateful to have met Tigran and Vahan, although none of us got the chance to visit Yerevan. We hope to be able to do it soon!

💥 Welcome to the P9family, Vahan, Tigran and the rest of the SuperAnnotate team.

👊 We’re pumped to be a small part of your hopefully very long journey, alongside our long-standing friends at Runa (who we owe the intro to) and Fathom (who we always enjoy working with)!