I was a 10x engineer for 7 months and then I was a 0x engineer for a year and a half. You burn the candle at both ends. You end up with alcoholism and depression. You’re talking about a very small subset of people. And they might end up in divorce and financial ruin. When people think you’re a 10x engineer, they think you have skills that you don’t. You invariably let people down.
— Anonymous
The Mythology Begins
Any endeavor to track down the origin of “10x engineer” will end up at Exploratory Experimental Studies Comparing Online and Offline Programming Performance by Sackman, Erickson and Grant. It was published in 1968. Computers were expensive and slow. Often, programs were written “offline,” without the programmer ever touching the computer. Instead, programs were scheduled and run by an operations team to ensure effective resource utilization. Allowing direct interaction of the programmer with the computer was far too expensive. Still, another approach blossomed:
Time-sharing developed out of the realization that while any single user was inefficient, a large group of users together was not. This was due to the pattern of interaction: Typically an individual user entered bursts of information followed by long pauses; but a group of users working at the same time would mean that the pauses of one user would be filled by the activity of the others.- Wikipedia
At the time of this foundational study, a debate raged. Time-sharing allowed programmers more direct access to the computer, but required “expanded hardware and more extensive software.” Exploratory Experimental Studies sought to explore the productivity benefits of such an approach despite the increased cost.
The research was conducted with a mere 21 individuals across two studies. Programmer performance in writing two types of program was measured — one, a program to find a route through a 20x20 cell maze; another, to interpret teletype-inserted, algebraic equations. In addition to testing productivity across “online” (direct access to the computer) vs “offline” conditions, the study found large individual variances in the performance of the tasks — “typically by an order of magnitude.” As the authors state:
… one poor performer can consume as much time or cost as 5, 10 or 20 good ones. Validated techniques to detect and weed out these poor performers could result in vast savings in time, effort and cost.
And so the mythology of the 10x engineer was born.
The Mythology Grows
The research conducted in Exploratory Experimental Studies Comparing Online and Offline Programming Performance was deeply flawed, as the authors themselves acknowledge in the original paper:
Before drawing any conclusions from the results, consider the scope of the two studies. Each dealt with a small number of subjects — performance measures were marked by large error variance and wide-ranging individual differences, which made statistical inference difficult and risky. The subject skill range was considerable, from programmer trainees in one study to highly experienced research and development programmers in the other. The programming languages included one machine language and two subsets of JOVIAL, a higher order language…. The representativeness of [the problems tested] for programming tasks is unknown.
However, this didn’t stop anyone from wildly extrapolating its results, selectively quoting from the paper and carelessly amplifying its tenuous “order of magnitude” message across the tech industry, even years later.
The Mythical Man Month was published in 1975 by Fred Brooks. In many ways, it is a memoir. The memoir of the development OS/360, a batch processing operating system developed by IBM. The book’s theme, as many of you will recognize, is that adding manpower to a late software project makes it later.
Here is the epigraph of Chapter 3:
These studies revealed large individual differences between high and low performers, often by an order of magnitude.
Sackman, Erickson and Grant
The author expounds:
Programming managers have long recognized wide productivity variations between good programmers and poor ones. But the actual measured magnitudes have astounded all of us. In one of their studies, Sackman, Erikson, and Grant were measuring performances of a group of experienced programmers. Within just this group the ratios between best and worst performances averaged about 10:1 on productivity measurements and an amazing 5:1 on program speed and space measurements! In short the $20,000/year programmer may well be 10 times as productive as the $10,000/year one.
The myth had now exited an uncertain adolescence, destined for greater things.
The Mythology Becomes Entrenched
In the decades since the original paper, additional studies have been added to the “10x engineer” canon. They, too, are rife with problems and, like the original study, have been heavily criticized. For example:
- Many studies look at highly problematic metrics, such as lines of code or time to find and correct specific bugs in a Fortran program. These metrics aren’t neccessarily indicative of true value production or productivity.
- Much of the cited research was conducted in a very different time, in a very different context, leveraging very different technologies and facing very different problems than programmers of 2013.
- The research tends to focus on very specific programming tasks, but is generalized by the culture to overall competency and contribution.
- Many referenced studies were conducted across a relatively small sample size or with poorly controlled conditions that bring into question the accuracy of the results.
Unfortunately, these studies nonetheless get cited extensively in other texts, which themselves get cited in other work, and so and so on. You end up in a self-referential, naval-gazing miasma of bad research, faulty conclusions and acculturated mythologies. For example, let us look at Steve McConnell, who has written extensively on the 10x engineer. Despite leaning heavily on the original 1968 research and other problematic and highly suspect research, McConnell has been quoted everywhere from the vastly popular Coding Horror blog to recent, widely-circulated presentations on scaling engineering organizations.
While an exhaustive critique of all the research in this area is outside the scope of this post…
There is no conclusive body of scientific research to suggest that the 10x engineer is in fact a real phenomenon, much less one that provides us with a deep and actionable understanding of the factors, conditions and stipulations of their existence. Yet we continue to regurgitate the mythology, over and over again.
Why?
The Power of Memes
Perhaps we use “10x” engineer and other variations on the concept — “superlatives”, “volatiles,” “rockstars”, “ninjas” — because they give us a pithy shorthand to describe the very real differences in contribution, value created, and competency in people we’ve worked with:
The Mythology, Poisoned
The problem is that, just the way the “culture fit” meme covers up many more complex and interrelated factors, so does the “10x” meme erase a number of factors related to situation, culture, experience level, motivation, opportunity for growth, and more. Here’s what happens when we use a gross simplification of these complex factors instead of a critical, conscious approach:
- It keeps us from having to specifically define what we actually value in programmers, preventing us from critical reflection on what we need in our teams, companies and community.
2. It over-focuses on the role of the individual and individual contribution in success, reinforcing Silicon Valley’s tendency towards hero worship, elitism and destructive individualism while ignoring the context of situation and privilege.
3. It can provide a cover for destructive and abusive behavior by the “10x engineer”.
4. It erases many of the essential team and cultural dynamics involved in true productivity.
5. It erases discussion of the individual’s journey towards higher levels of contribution and the role of mentorship, experience and personal growth in that journey.
6. In the individual’s desire to live up to the “10x” mythology, or the culture’s expectations that individuals be extreme performers, workers may succumb to a variety of destructive tendencies including drug use, workaholism, heroism and alcoholism.
Death of a Mythology
The technology industry is full of memes, mythologies and metaphors that give us tools to communicate, to structure the world, to enforce our values, to shape our work and construct the way we see ourselves and our industry. “10x engineer” is one of those mythologies.
By unpacking it, we both see our culture more clearly and are more equipped to intervene in it — asking questions about what we truly value in engineers, what we need in teams and companies to make them succeed, and how we can stop contributing to harmful team and individual dynamics.
For more perspectives on 10x engineers, please see the Twitter conversation that inspired this post. Thanks to everyone who contributed their thoughts and ideas to helping me write this!!
