The Paradox of Scale

Netflix couldn’t build Netflix

David Carboni
Nov 29, 2018 · 4 min read

There are a bunch of heroes in the digital world that are held up as aspirational examples in conversations about digital business, technology and ways of working. Netflix, Spotify, Uber, Amazon and a host of our favourite characters, but these myths can become damaging fictions.

Admired and respected as towering giants of our digital world, our hero companies emanate an almost mythical quality. The scale, power and inspiration they command are the stuff of legend. Glib statements about “business” distort their stories into gaudy two-dimensional caricatures whilst organisations seeking Digital Transformation aspire to emulate what they see in this theatre. Paradoxically our heroes would be the first to point out they wouldn’t be able to build themselves as they stand today.

Photo by YIFEI CHEN

Gall’s law has been on my mind lately. So much so that my partner has quoted it back to me — and she doesn’t work in tech, she works in the woods. Hopefully it’s that I communicated it well, but more likely it’s that she’s a very smart woman. For me it’s a statement that goes to the heart of why so many institutional IT projects cost a fortune and deliver little, with alarming regularity:

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. — John Gall (1975, p.71)

If you work with technology and your role includes the word “architect” (as mine sometimes does) your ears may well be burning.

The paradox of scale

I spend plenty of my time in conversations about microservices. API design, infrastructure, operations, architecture. If you’ve been there, you’ll have heard phrases like scheduling, circuit breakers, service discovery, distributed tracing, health checks, cenralised logging, mutual authentication, rolling deployments, traffic splitting, the list goes on.

When you consider that each of those can easily balloon into a couple of months of work, especially when they come “for free out-of-the-box”, a spot of arithmetic will tell you you could be looking at two years of peripheral work before you even think about the thing you’re actually trying to build.

Do you have a million users and a billion transactions? And do you have them today? Or are you just starting out with a new product? It’s easy to assume this stuff is critical to running production-quality systems and, you know what, it might be, but more likely it isn’t right now.

The question is not “whether”, but “when” these things are useful

The rise of the titans

Google is 20 years old, yet Google Cloud is only just starting to challenge AWS. Google boasts impressive infrastructure, tens of billions invested, including undersea cables and a global network estimated to carry 25% of the world’s Internet traffic. Only now are they offering this through Google Cloud. It’s been a long road. This is a far cry from the humble beginnings of 1998, when Google was an underdog search-engine beloved of 90s hipster-equivalents.

One does not simply build Google

And so it’s been, I’ll bet, for all our heroes. They thought big, acted small, found a foothold and started journeying. For better for worse, for richer for poorer, in sunshine and in rain, wax on, wax off, they fixed the plumbing and built the roof, they put one foot in front of the other until today they stand towering in the world they helped to create. Where they stand now is (and continues to be) their journey, rather than their first port of call.

On becoming a hero

The thing no one ever tells us about standing on the shoulders of giants is that we first have to get up onto those shoulders. Looking at what our heroes do today and trying to copy it is a highly effective way to fail. First because it’s going to be expensive and take years, second because they are on their journeys and will keep moving in their own directions.

By the time we ever get there, they’ll be long gone

It’s disheartening to look up and feel small. Our efforts feel tiny, embarrassing even, if we compare ourselves to where our heroes stand today. If we can remember that they too came from humble beginnings, focus our attention on comparing ourselves only to where we stood yesterday and keep putting one foot in front of the other, pretty soon we’ll be able to look back and stand taller.

Notbinary

Sharing our thinking on digital transformation through data, software and automation

David Carboni

Written by

Hands-on technology and organisation culture transformation. Work hard be kind. Technology Director and head geek at Notbinary (https://notbinary.co.uk)

Notbinary

Notbinary

Sharing our thinking on digital transformation through data, software and automation

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade