Getting to 0 users
In January of 2019, things were going pretty well for Meeshkan, a company I founded in 2017. We had just won the Slush 100 pitch competition. We launched a project called unmock and were seeing a healthy increase in week-over-week downloads. Our at-the-time flagship product, the Meeshkan Slack Bot, had just signed a large deal with a cutting-edge med-tech AI firm.
Fast-forward to August 2019 and the Meeshkan Slack Bot is dead, the 0.0.X series of unmock went from 1000s of users to 0 users, and the company has never been doing better. In this article, I’d like to talk about how we took the unmock pilot from 1000 to 0 users and what we learned by doing it. But first, it’s worth it to discuss how the Meeshkan Slack Bot fizzled out, as it turned out to be a harbinger of things to come with unmock.
Onlookers versus users
The apogee of interest in the Meeshkan Slack Bot came on the heels of the Slush 100 pitch competition, when we got more traffic to our website in two days than the previous twelve months and netted over fifty sign-ups for our product. Fifty may not sound like a lot, but in the Machine Learning space for a relatively niche tool, we thought it was pretty darn cool.
The problem with this mini-surge, as well as the Slush event in general, was that there was far more interest in our vision around the product than the product itself. Those that were “supposed” to be interested in the Slack Bot simply weren’t jumping on board. Data scientists would give up after halfhearted attempts to use it and go back to their arsenal of comfortable, hand-rolled devops hacks. Deals never felt organic — when we pitched to C.T.O.s or heads of ML, we were painfully aware that their team was not yet using the free version of Meeshkan. This is the IT equivalent of a death sentence, as the only products that ever succeed are ones with strong organic usage before paid buy-in.
Ultimately, the Meeshkan Slack Bot was ahead of and behind its time. It was ahead of its time because its lofty goal — to coordinate the training of machine learning models on Slack — would have required teams to already have organized, sanitized piles of data waiting to be machine-learned. These teams do exist, but they tend to have dedicated professionals working with them on ML infrastructure who would often rather build something themselves then trust a third-party vendor. On the other hand, the Slack Bot was behind its time because no one in the world pays for products like that anymore. Netflix, for example, recently open-sourced Metaflow, which has 70% shared DNA with our Slack Bot and is 100% free. Ultimately, unless lawmakers in America blow tech giants to smithereens with new anti-trust legislation, massive companies will be solving massive problems with state-of-the-art tools that benefit from being reviewed and maintained by an open-source community. While there is money to be made by hosting, using, and teaching those tools, producing them is no longer a viable business model.
The Meeshkan Slack Bot was like a Janus with the heads rotated 180 degrees— the future was staring down the past in a tight loop that sealed its doom. So why all the hype around it? Ultimately, Meeshkan had 10,000 onlookers for every user. Its avant-garde use of Slack coupled with its status as an ML tool was enough to titillate even the mildest of buzzwordophiles. But even more so, we put on a good show. Watching our slick presentation of the tool activated the same neural pathways as attending a good production of Parsifal at the Met or seeing a Parliament-Funkadelic reunion. This is what I have informally dubbed the “IT Spectacle Economy” — an international network of meetups and conferences where pizza and beer flow freely and people sometimes pay serious cash to be wowed by professionals talking about the code they write and the tools they use. I unwittingly became a participant in and purveyor of this circuit because (a) it’s fun; (b) I confused onlooker enthusiasm with user engagement; and (c) my background is in the performing arts and I like playing to a receptive and engaged crowd.
This intractable gap between onlookers and users is ultimately why we shut the Meeshkan Slack Bot down. But we were about to repeat the same mistake, in an even more colossal fashion, with unmock. The rest of this story is about that process.
A fake version of the internet
When we launched unmock, we did so with an even loftier and more ambitious goal than the Meeshkan Slack Bot — to make a fake version of the internet that you can mess around with to your heart’s desire before messing around with the real internet. That is still the goal of the project, but in January, rather than starting off small, we (I) started building the final version of unmock. We were creating hand-crafted mocks of certain popular services, heating our Helsinki office with GPUs doing NLP on various corners of the internet. We had a killer demo where you could basically play with a fake version of Behance that felt like the real one, which allowed people to see how unmock effortlessly generated oodles of real-looking users, profiles, images, comments etc.
If onlookers liked the Meeshkan Slack Bot show, they went gaga over watching us mess around with a fake internet on stage. We got programmed at conferences and meetups in San Francisco, Seattle, London, Dublin, Berlin, Tel Aviv, and Stockholm. Our usage numbers were even ostensibly better, as I always left people with specific instructions for how to try out unmock at home and they usually dutifully followed them.
The problem, as you may have guessed, is that while people liked the idea of a fake version of the internet, no one actually knew what to do with it. So, after taking unmock for a spin, usage puttered out. We had more churn than a nineteenth-century butter maker. Our numbers looked like the skyline of the alps with peaks the day after meetups and troughs on all other days. For a while, my solution was simply to do more meetups to make the mountain range more dense. But of course, that doesn’t fix the underlying problem.
The craziest part of all this was that we had ten to twenty genuine signups a day for a tool with zero marketing and almost zero usage. So who the heck were these people? I reached out to a lot of them, and the responses were consistently the same — they thought what we were building was interesting and, while they couldn’t figure out what to do with it, wanted to be part of the journey somehow. While this is, of course, better than nothing, it is a long way from a viable business or even project.
In June, we had a week-long powwow to figure out how to handle all this. We could only think of two options, neither of which was pretty: double down on the companies that had signalled interest and develop an enterprise grade-product to suit their needs, or whittle down the codebase to only the most unbroken, useful and sustainable features and open-source the whole thing. The status quo was not an option: we were repeating our onlooker mistake, but at scale and more expensively, and even worse, we were actually calling these people “users” because they signed up for the thing and maybe even used it a few times.
We decided on the second option, namely, open-sourcing a useable version of our codebase. This required asking several questions, none of whose answers were obvious at the time. What was useable? What was useful? How would this sync up with the broader vision of the project and company? How could any of this be commercially viable? To start answering these questions, we proceeded by subtraction rather than addition. We stopped, for example, disseminating a clear-cut tutorial on how to use unmock, which resulted in a pretty drastic fall in usage, but those who remained gave us valuable insights into what people actually did with the tool. We conducted exit interviews with dozens of fleeing (non-)users, all of whom told us the various sources of their indifference. For example, it was from this that we learned that unmock needed to be fully offline and run locally. Finally, and most importantly, we cut off new signups to our service. Our onboarding process was so slick compared to the rest of the machine that it was getting in our way of thinking about that onto which people were boarding. This decision was inspired by a legend about the sculptor Rodin, which I recount here with slight embellishments that come from years of telling it and misremembering it.
One day, Rodin unveiled a new work to his students, and they all reacted with predictable awe to the creation of their talented master. One student effused “My, what agony, and what hands. From them pours a thousand years of inexpressible grief.” A second student opined “Master, what awful tragedy is masked behind those terrible, grotesque, hideous, and singularly beautiful hands?” Irritated, Rodin found a third student that, upon inspection, declared triumphantly “You have created the hands to which all future hands, real or unreal, will be compared.” Rodin, furious, grabbed an axe (in my version he happens to have an axe handy) and cut off the hands. His students asked in disbelief why he had mutilated his prodigal handiwork. His response: “No element of a work should ever be so great that it distracts from the greatness of the whole.”
Granted, unmock 0.0.0 was not great — it was at best bad — but its onboarding process was only bad whereas everything else was awful, so we cut it off. This allowed us to focus purely on signal — people reaching out to us and asking about unmock.
Fast forward a few months and unmock 0.1.5 is alive and kicking! It has seen slower, but steadier, week-over-week growth in usage and, most importantly, we can focus on how people are using it instead of how they would like to use it. Because it is fully open-source and most usage happens in the open, we have a great view of users’ processes and what issues they are confronting. And because there is a crystal clear separation of concerns — unmock is an open-source project whereas Meeshkan is a commercial entity that helps companies build cool stuff with unmock — there is no confusion about what is free and what’s not.
Getting to 0 users of unmock 0.0.0 was embarrassingly easy because it wasn’t a product — it was a vision that got a lot of intellectual buy-in but was less stable than CryptoKitties. Accruing a user base for unmock 0.1.X has been more difficult but more rewarding, and if I had to do it all over again, I’d start with the 0.1.X strategy and leave 0.0.0 for later. We will build our fake version of the internet, but we will do it one (real) community member at a time.
