See Beyond the Average
How the Air Force inadvertently caused pilots to crash by doing the obvious — designing a cockpit based on average pilot dimensions. Extended to the web.
Have you ever heard the story of how the United States Air Force originally botched the cockpit in their planes?
In 1950 the U.S. Air Force took physical measurements of 4,000 pilots across 140 dimensions of size in order to inform airplane cockpit specs. That is, determine the average torso length or arm length and use that average to inform the placement of the cockpit seat, the yoke (i.e. the steering wheel), etc.
Seemed like a reasonable approach.
Only it turns out that the data-driven designed cockpit was a disaster. Good pilots were having uncontrolled crashes. Importantly, no one, including the pilots, quite understood what was amiss.
Is there such a thing as average pilot?
Thankfully, one 23-year-old analyst — Lt. Gilbert S. Daniels — had some experience in measuring the human body and had a theory: perhaps there was no average pilot size.
To test his hypothesis, Daniels looked at the 10 most relevant physical dimensions of pilots from the study and created an “average pilot” based on the middle 30% range of the dimensions. Per Daniels’ analysis, the “average pilot” ranged in height from 5'7" to 5'11". Daniels then looked at the entire pool of 4,063 pilots and tried to match individual pilots to his “average pilot.”
Daniels was right. Not a single pilot fit the average. As the article so succinctly put it, “If you’ve designed a cockpit to fit the average pilot, you’ve actually designed it to fit no one.”
Armed with this insight the Air Force radically changed their design philosophy:
By discarding the average as their reference standard, the air force initiated a quantum leap in its design philosophy, centered on a new guiding principle: individual fit. Rather than fitting the individual to the system, the military began fitting the system to the individual.
The Air Force had engineers immediately fix the problem. In short order components were made adjustable — the cockpit debacle is what ultimately led to adjustable car seats.
There is no ‘average man’
Why doesn’t the ‘average man’ work? Daniels summed it up:
“The tendency to think in terms of the ‘average man’ is a pitfall into which many persons blunder … It is virtually impossible to find an average airman not because of any unique traits in this group but because of the great variability of bodily dimensions which is characteristic of all men.”
Daniels’ insight — that there is no average human — could be characterized as a profound, yet intuitive truth: people are (and life is) complex and that complexity doesn’t do well with one-size-fits-all solutions.
Minding the average when we build tools
Web analytics are prone to the same “flaw of averages” Daniels saw in the Air Force. Just as with human physical dimensions, there is great variability in how individuals access and interact with the Internet. Consider:
- Device variability (laptop, desktop, tablet, smartphone),
- Intra-device variability (iOs or Android; Windows, OS X, Linux),
- Intra-OS hardware variability (Over 24,000 different Android devices estimated),
- Browser variability (Chrome, Firefox, Safari, others),
- Screen resolution differences,
- Window sizing preferences,
- more …
Iterate on the above combinations (some of which could change for the same user within the same day or same website session!) and you get an unfathomably differentiated population of use-cases.
What’s more, easily measured hardware and software variations say nothing about user variations with regard to intent — what a given user is actually trying to do on your site. Aggregates at the qualitative level are nearly impossible.
A path to seeing beyond the average
We believe the best tools will adapt to and account for human variation. In building a web analytics tool, we can design it so that the data connects from the highest-level, aggregated metrics all the way down to the individual users, seeing them uniquely as they are.
