Lies, Damned Lies, and Statistics: Coding Bootcamps and the Authenticity of Placement Rates

AUTHENTICITY

We’ve never formalized our core values at Bloc, but if you surveyed our employees you would probably see authenticity in the top three most cited responses — followed closely by swag and batman. We’ll focus on authenticity today.

Authenticity is a word that we use very specifically, and we don’t use it to mean the same thing as honesty or transparency. Richard Feynman describes it as “scientific integrity”:

Last night I heard that Wesson Oil doesn’t soak through food. Well, that’s true. It’s not dishonest; but the thing I’m talking about is not just a matter of not being dishonest, it’s a matter of scientific integrity, which is another level. The fact that should be added to that advertising statement is that no oils soak through food, if operated at a certain temperature. If operated at another temperature, they all will — including Wesson Oil. So it’s the implication which has been conveyed, not the fact, which is true, and the difference is what we have to deal with.

From: Cargo Cult Science

The easiest way I’ve found to articulate the difference is to explain it in the context of someone asking a question:

Honesty is truthfully answering the question someone asked.

Authenticity is truthfully answering the question someone intended to ask.

Transparency is a bulk CSV export of your data.

Here’s an example: when we raised our Series A investment last year, a few of my friends asked me if I was now a millionaire.

An honest answer would be yes. On paper, if we had hypothetically raised a round with a post-money valuation over $5M and I owned at least 20% of the company I would have 20% x $5M = $1M ownership in a privately valued company and could technically be considered a millionaire.

The authentic answer would be no, not even close. The question my friends intended to ask was “do you have a million dollars of liquid cash that you can spend to buy me a Tesla?” And the answer to that question is decidedly “no”, unless Elon would accept Bloc equity as cash.

… AND STATISTICS

The developer bootcamp industry has an obsession with something called “the placement rate number”. It’s meant to measure a program’s efficacy by quantifying the percentage of graduates who successfully start careers as developers.

Bloc is one of few programs that has never advertised a placement rate, and many bootcamps have removed their rates from their sites as well. Prospective students are eager to ask us for this statistic, and I don’t necessarily blame them given how appealing it is to use a simple benchmark to compare programs. We don’t publish a placement rate though as we believe it would potentially conflict with our commitment to authenticity, not because we lack confidence in the efficacy of our program.

When a prospective student asks us “what is your placement rate?”, we could honestly say anywhere between 0–100% depending on how we qualify our answer. We could, today, say that 99% of our students find jobs after they graduate from Bloc in a way that is both technically honest and legally defensible, but not authentic or ethical. It’s not very difficult to game that statistic.

The truly authentic answer has nothing to do with statistics though. The question our students intend to ask is closer to “Does your program work?”, or more specifically “Will your program work for me?” We’ve found a better way to answer that question: our Software Engineering Track comes with a tuition reimbursement policy for students who are unable to find new careers in software development after graduating, and now our students don’t have to worry about landing on the wrong side of a program’s 90% placement rate.

When there are programs with less than 20 grads touting a 100% placement rate and dozens of hidden qualifications, that number devolves from a transparent industry benchmark to a disingenuous marketing prop. While we look for authentic and quantifiable ways to evaluate program quality, I’ll encourage students to dig deeper: ask about the curriculum, background and experience of instructors, tuition and opportunity costs, and the hidden qualifications of these placement rate numbers.