There is such a thing as too much money
Docker was (and still is) a fantastic developer productivity tool, probably the most momentous change after the introduction of Linux. Then VCs came and gave them $300M: it truly was the beginning of the end.
Yesterday, Protocol published an interview with Docker’s “fourth CEO in less than three years,” which provides a reasonably accurate, if not entirely “candid” account of all that went wrong at Docker.
Let’s be clear: even though
Linux cgroups (aka, “containers”) have been around for decades, without the undeniable genius of Solomon Hykes and the early-stage Docker employees, they would have probably remained one of the many more or less ingenious, but hard-to-use and obscure Linux features.
Personally, I still remember attending one of the first Docker meetups, in a (pretty dingy, to be honest) basement of their tiny office: I think it was Palo Alto, but really the only memory I have of the day was how small the office was, and how amazed I was at learning about this new-fangled thing called “Docker containers.”
As a longtime developer, and someone who was starting to appreciate the storm that was about to sweep Enterprise software, aka Public Cloud (but, back then, it was just “AWS”) the attraction of having this kind of “fully-formed, throwaway, always in a known state” environment, was truly an “aha” moment.
What most people tend to forget is that, until then, the best you could hope for was that your VM’s snapshot was still in a good state, that it would boot up, and that no one had messed with it by installing something or other: having a declarative way of describing your environment, being able to keep it under source control (which, by then, meant only one thing:
git) and being able to compose them via the simplicity of shell scripts (and eventually via
docker compose templates) was going to completely upend the way we would build, test and deploy our code.
The truly momentous innovation was the
Dockerfile: it revolutionized the way we think about building and deploying our services.
Further, it would terminally smash down the wall that until then had existed between Developers and Operations: with the former throwing their fully-formed (but usually untested, outside of the “it works on my machine” attitude) code over the wall; and the latter having no clue (usually) about what they were deploying in Prod, and coming up with fantastically complicated schemes to make this foreign object sort of work on a good day.
With Containers, we developers all of a sudden could define a runtime environment, test it fully locally or remotely (remember, by then AWS was only a credit card swipe away) and then pass it on to our buddies who would run exactly the same thing, just bigger and badder in Production.
Not only that, but in doing so we would soon have to (and, most of us, certainly me, want to) learn a lot more about networking, routing, name resolution and all that magic that made our services hum; not to mention, it was all of a sudden very easy to simulate failure modes and test all sort of doomsday scenarios on our laptops, and make our services resilient to the most bizarre failure modes.
It was the beginning of DevOps, and opened up the doors to the subsequent “abstraction” layer in the form of Kubernetes (as well as the half-baked, and horribly managed, Apache Mesos) whose long-term effects are still fully developing before our eyes today.
So, what everyone wants to know (and the interview tries to, unsatisfactorily, answer) is: why isn’t Docker (the company) out there, wildly successful, basking in their success at “changing the world,” and is instead still trying to figure out what it wants to do as a grown up?
Well, VCs happened, this is what.
Over several rounds, more than $300M were poured onto a company who could have probably, legitimately spent a tenth of that, and even then it would have been hard.
Remember, the whole premise of Containers (and DevOps, and declarative software, and automation) is that you can do a lot, with very little — you don’t need an army of developers, and a swarm (ah! a pun!) of sysadmins, where probably 4–5 smart DevOps can run an entire suite of microservices, running over tens, if not hundreds of EC2 instances (I know, I did it).
Further, the market for Containers (and
dockerd the server) and associated services is by its very nature pretty tiny (it is after all, the “tools for developers” one) given that the runtime (which is what powers tens of millions of public and private cloud application servers) is open-source, so there are no licensing or royalties revenues to be had (remember, the underlying technology is, after all, native to Linux
However, when one collects $300M from VCs, then one has made a pact with the devil: they must deliver an IPO (or a mega-acquisition) with a valuation of several $’Bs, lest they be considered a failure.
The only way to do so (which calls usually for an ARR — annual revenue run-rate — of at least $100M and probably more) is to go chase the Enterprise market, selling a product whose price tag must be in the $’M annual range and justify the cost in the face of pretty tough and demanding procurement officers (not to mention, one suddenly faces lead times of months if not years, with lumpy revenues and a lot of uncertainty around repeatability).
And, again, all this in the face of a perfectly working open source offering which, with a little bit of ingenuity and coding, can be used to scale at Enterprise level (well, sometimes a lot of coding: took me around 10,000 lines of Python code to orchestrate and deploy around 1,000+ containerized services, across several thousands of EC2 instances, via Amazon CloudFront and ECS)
Thus, here comes Docker Swarm (don’t ask me, I never really got into it) and a number of other enterprise offerings, which however run into the biggest, baddest, most terrifying competitor of all times in the Cloud Enterprise orchestration market space: Google Kubernetes.
It was only a matter of time, but eventually late last year, Docker threw in the towel and decided to go back to their roots in the developer tooling space (at least, that’s what they say); but one cannot but regret the years wasted chasing the rainbow, and wondering what wonderful stuff Hykes and co. would have been able to deliver to the development community had they not been distracted by “the wall of money” that came upon them.
I for one, cannot wait to see what Docker will deliver next: I remain a great fan, and Docker containers are definitely the one tool I reach for when running/testing applications composed of several microservices (on Kubernetes, obviously!).
The lesson to learn, however, is that sometimes there is such a thing as “too much of a good thing” and, before accepting VCs’ sackfuls of $$$’s, one has better be clear which market they’re aiming for, and it is truly one that can be conquered.
And, in the process, a good dose of self-introspection about the product’s true potential wouldn’t hurt either.