Nov 29, 2018
The Paradox of Scale
Netflix couldn’t build Netflix
There are a bunch of heroes in the digital world that are held up as aspirational examples in conversations about digital business, technology and ways of working. Netflix, Spotify, Uber, Amazon and a host of our favourite characters, but these myths can become damaging fictions.
Admired and respected as towering giants of our digital world, our hero companies emanate an almost mythical quality. The scale, power and inspiration they command are the stuff of legend. Glib statements about “business” distort their stories into gaudy two-dimensional caricatures whilst organisations seeking Digital Transformation aspire to emulate what they see in this theatre. Paradoxically our heroes would be the first to point out they wouldn’t be able to build themselves as they stand today.
Gall’s law has been on my mind lately. So much so that my partner has quoted it back to me — and she doesn’t work in tech, she works in the woods. Hopefully it’s that I communicated it well, but more likely it’s that she’s a very smart woman. For me it’s a statement that goes to the heart of why so many institutional IT projects cost a fortune and deliver little, with alarming regularity:
A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. — John Gall (1975, p.71)
If you work with technology and your role includes the word “architect” (as mine sometimes does) your ears may well be burning.
The paradox of scale
I spend plenty of my time in conversations about microservices. API design, infrastructure, operations, architecture. If you’ve been there, you’ll have heard phrases like scheduling, circuit breakers, service discovery, distributed tracing, health checks, cenralised logging, mutual authentication, rolling deployments, traffic splitting, the list goes on.
When you consider that each of those can easily balloon into a couple of months of work, especially when they come “for free out-of-the-box”, a spot of arithmetic will tell you you could be looking at two years of peripheral work before you even think about the thing you’re actually trying to build.
Do you have a million users and a billion transactions? And do you have them today? Or are you just starting out with a new product? It’s easy to assume this stuff is critical to running production-quality systems and, you know what, it might be, but more likely it isn’t right now.
The question is not “whether”, but “when” these things are useful
Paradoxically, trying to build it all — to emulate our heroes on day one — is more likely to be disastrous. In Malcolm Gladwell’s re-examining of the story of David and Goliath, David refuses to put on the heavy battle armour of professional soldiers, preferring to fight with only a shepherd’s sling. David knew he couldn’t bear the weight of the armour and wouldn’t be at his best, potentially putting himself in even greater danger. As Malcolm Gladwell paraphrases it, "I've never worn armor before. You've got to be crazy."
The rise of the titans
Google is 20 years old, yet Google Cloud is only just starting to challenge AWS. Google boasts impressive infrastructure, tens of billions invested, including undersea cables and a global network estimated to carry 25% of the world’s Internet traffic. Only now are they offering this through Google Cloud. It’s been a long road. This is a far cry from the humble beginnings of 1998, when Google was an underdog search-engine beloved of 90s hipster-equivalents.
One does not simply build Google
20 years is a long time. Long enough to accumulate extraordinary experience, infrastructure and legacy. A long road with plenty of twists and turns along the way. I don’t know specifics, but I do know what life is like. Success is a messy business, exploratory, trying, failing, scratching your head, learning something new, trying to think different.
And so it’s been, I’ll bet, for all our heroes. They thought big, acted small, found a foothold and started journeying. For better for worse, for richer for poorer, in sunshine and in rain, wax on, wax off, they fixed the plumbing and built the roof, they put one foot in front of the other until today they stand towering in the world they helped to create. Where they stand now is (and continues to be) their journey, rather than their first port of call.
On becoming a hero
The thing no one ever tells us about standing on the shoulders of giants is that we first have to get up onto those shoulders. Looking at what our heroes do today and trying to copy it is a highly effective way to fail. First because it’s going to be expensive and take years, second because they are on their journeys and will keep moving in their own directions.
By the time we ever get there, they’ll be long gone
But if we shift our perspective and look, not at where they are today, but at how they got started and the way that they move, there’s a beautiful release. Now we don’t need to be dazzled by their scale, wondering how we could possibly be like that. Instead we can look at where we’re starting from, face the direction we want to go (which will be unique to each of us) and start putting one foot in front of the other.
It’s disheartening to look up and feel small. Our efforts feel tiny, embarrassing even, if we compare ourselves to where our heroes stand today. If we can remember that they too came from humble beginnings, focus our attention on comparing ourselves only to where we stood yesterday and keep putting one foot in front of the other, pretty soon we’ll be able to look back and stand taller.
