FizzBuzz and Folklore

If you’re a working programmer, you may have heard that “most people applying for developer jobs can’t even solve FizzBuzz”:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

Most good programmers should be able to write out on paper a program which does this in a under a couple of minutes. Want to know something scary? The majority of comp sci graduates can’t. I’ve also seen self-proclaimed senior programmers take more than 10–15 minutes to write a solution.
Imran Ghory—Using FizzBuzz to Find Developers who Grok Coding

This was originally posted in early 2007, and it took off a month later after it showed up in Coding Horror:

I’m more than willing to cut freshly minted software developers slack at the beginning of their career. Everybody has to start somewhere. But I am disturbed and appalled that any so-called programmer would apply for a job without being able to write the simplest of programs. That’s a slap in the face to anyone who writes software for a living.
Jeff Atwood—Why Can’t Programmers… Program?

FizzBuzz has since become a staple of programmer interview screening questions, and yet is still successfully screening out candidates. In 2012:

The candidates sat at my desk and wrote out their code while I kept an eye on the time, with the intention of invoking the Mercy Rule at the ten-minute mark. They were free to ask any questions during the session; the most common was “Is there some sort of trick in the wording of the assignment?” (there isn’t), and the most unusual was “Do you think I should use a loop?”, which nearly made me end the interview right then and there. Had any of them asked to see what some sample output would look like, I would have shown them a text file containing the first 35 lines of output from a working FizzBuzz program.

The interviews concluded last week, after which we tallied the results: 40% of the candidates passed the FizzBuzz test. Worse still, I had to invoke the Mercy Rule for one candidate who hit the ten-minute mark without a solution.
Joey DeVilla—FizzBuzz Still Works

And the Recurse Center includes a variation of it (to prevent Googling, one presumes) in their application:

“Most people interviewing for programming jobs can’t complete FizzBuzz” has become part of our conventional wisdom, like “Good developers are 10x as effective as mediocre ones” and “bugs cost 100x more to fix in production than planning.” But much like those bits of conventional wisdom, I suspect that the FizzBuzz phenomenon is more folklore than fact.

FizzBuzz: Fact or Fiction?

I am by no means a seasoned interviewer, but even when interviewing for junior developer roles, I have rarely encountered people who cannot program at all; the 40% statistic cited by DeVilla seems implausibly high. Others have similar experiences: while most have encountered some people who cannot complete FizzBuzz, none of them see it as frequently as Ghory or DeVilla did:

There are a number of possible reasons for this:

• A greater proportion of new programmers go through vocational programs (e.g. bootcamps) where either a) they are coached on interview questions or b) they are more prepared for “real work” than CS graduates
• The size and behaviors of the industry has changed such that there are more qualified people looking for work at any given time; the applicant pool is no longer dominated by people who apply to jobs they are unqualified for
• The preferred programming languages have changed since 2007, as more jobs shift from Java / .NET to scripting languages; perhaps web developers are better suited to FizzBuzz tests because it’s easier to write “small programs” in Ruby / Python

Of course, anecdotes are not data: neither the stories supporting nor refuting the FizzBuzz phenomenon are statistically significant. Really, there is no conclusive evidence that any specific percentage of job applicants can solve FizzBuzz. So why does this meme persist? Why do people believe in the FizzBuzz phenomenon?

Fallacies of FizzBuzz

I noticed that many of the posts that introduced the FizzBuzz phenomenon argue not from experience, but via thought experiments. The FizzBuzz phenomenon took off not necessarily because it was widely experienced, but because it “makes sense” to programmers. But “makes sense to me” is just another way of saying “supports my preexisting biases.” The ideas that support the FizzBuzz phenomenon (in lieu of actual evidence) are built on logical fallacies, invalid analogies, and junk science. For example:

Many interviewers have encountered people who can’t solve FizzBuzz, but the applicants who fail the FizzBuzz problem are memorable precisely because they’re unusual; their memorability is an example of the Availability Heuristic gone wrong

The existence of a large pool of unqualified applicants is analogous to how dating sites are full of “losers”:

After a few months of “dating,” people can become very cynical. “Where are all the good men/women/womyn?” They plaintively ask. The answer, we all know, is that good people are rarely looking. They are already in relationships. If they become single, after a reasonable period to recover from their experience, they date very briefly and then get into another relationship.

People with “issues,” on the other hand, spend a lot of time dating and searching for mates but relatively less time in relationships.

The net result is that on a dating site, the majority of the people looking for mates are not the people you want to meet.
Reginald Braithwaite —Thank you for writing such a heartfelt critique of the Fizzbuzz screening question

There are two major problems with this analogy. First, romantic suitability is not an absolute, objective measure — one person’s ideal mate can be totally wrong for someone else. Likewise, a programmer’s personality and skillset could match well with one company’s culture and be all wrong for another.

Secondly: dating is not all that similar to hiring! Many people spend years at a time being single; few can spend years being unemployed. Employers can have thousands of employees; relatively few individuals can simultaneously date that many people.

Programmers are unusual and a distinct group from non-programmers: Around the time the FizzBuzz phenomenon was first proposed, a paper called The Camel has Two Humps suggested that some people are simply unable to learn how to program:

Despite the enormous changes which have taken place since electronic computing was invented in the 1950s, some things remain stubbornly the same. In particular, most people can’t learn to program: between 30% and 60% of every university computer science department’s intake fail the first programming course. Experienced teachers are weary but never oblivious of this fact; brighteyed beginners who believe that the old ones must have been doing it wrong learn the truth from bitter experience; and so it has been for almost two generations, ever since the subject began in the 1960s.

However, co-author Richard Bornat retracted this paper in 2014:

I need to make an explicit retraction of what it claimed. Dehnadi didn’t discover a programming aptitude test. He didn’t find a way of dividing programming sheep from non-programming goats. We hadn’t shown that nature trumps nurture. Just a phenomenon and a prediction.

I’m fairly sure that I believed, at the time, that there were people who couldn’t learn to program and that Dehnadi had proved it. Perhaps I wanted to believe it because it would explain why I’d so often failed to teach them. The paper doesn’t exactly make that claim, but it comes pretty close. It was an absurd claim because I didn’t have the extraordinary evidence needed to support it. I no longer believe it’s true.

Folklore

I think that the most useful lens to view the FizzBuzz phenomenon is as folklore — it’s a response to an observed natural phenomenon, but its purpose is to support a cultural practice. And like most folklore, it is based in incomplete, outdated or fallacious reasoning.

Consider the famous “10x developer” phenomenon: it’s not completely false, but it’s rooted in the era of batch programming with punchcards; the practice of programming has changed such that however real the 10x phenomenon may have been in the 1960s, it is irrelevant today. I suspect the myth of the 10x developer persists today because it supports the ideology of tech startups—that a tiny group of elite developers can “out-program” an army of mediocre corporate drones.

Likewise, I suspect that the FizzBuzz phenomenon persists because it explains why hiring in tech is difficult, but in a way that is flattering to people already in the industry. “Half of the people we interview don’t even know how to program” is a lot more appealing than “we don’t know how to hire people.”

Thanks to Milan Loveless, Vijith Assar, and Matthew McVickar for review.

For more folklore and mythology in software, see Laurent Bossavit’s The Leprechauns of Software Engineering.