My Joel Test

A few years ago, my friend, Richard, introduced me to the Joel Test. Written in 2000, the Joel Test measures the “quality” of a development team in twelve questions. Richard described it as a list of questions to ask a team during the interview process. A team doesn’t need to nail the test, but it definitely helps you decide whether the team is one you want to join.

Recently, I found myself re-reading the Joel Test, and it didn’t feel right anymore. While the original points are still as relevant as ever, I don’t think they cover a lot of the problems I face doing software development today. Nowadays, problems within teams, like establishing continuous deployment, don’t really worry me. Instead, I find myself worrying we spend months (or years!) building a product, ignore our users, and it turns out shit. These are the kind of problems which you find yourself banging your head against, and constantly trying to improve, before you finally throw in the towel, and move on. These are the problems I want to know about before I join a team, but they’re never really discussed during an interview process.

Because of this, I have produced my own extensions to the Joel Test:

1. Do you know how people are using your product?

2. Do you validate features with real users?

3. Do you delete unused features?

4. Are you responsible for your architecture?

5. Do you actively maintain a diverse team?

6. Do you regularly challenge your own processes?

1. Do you know how people are using your product?

Without a decent feedback loop, your product strategy may as well be tossing a coin. The feedback loop can come from user research or analytics, but you should have both. Fitting analytics into your product is never too difficult, and no product is too special. There’s Google Analytics or Piwik, and you can even get away with just rolling your own event logger.

2. Do you validate features with real users?

One of the most annoying feelings is to put your heart into developing something, only to see it gather dust. In the same way that bugs cost more the closer they are to production: bad ideas cost more time, and cause more frustration, the closer they are to production.

In an ideal world, user research should be driving most* of your product roadmap. You should, at least, be constantly iterating on new ideas, and validating them with users.

* Sometimes it’s better to have a vision than to work on features that users are asking for, but only if it’s what they really need. Think cars over faster horses!

3. Do you delete unused features?

When talking about features, and bugs, the 80/20 rule is more prevalent than ever. New features add complexity, and complexity breeds scary sections of your code.

The distinguishing factor between a good product, and a bad one, is what happens to these features. Bad products quickly become swiss army-knives, except with Twitter, Facebook, and LinkedIn integration. Good products, on the other hand, are like well-maintained code. They shed unused features, and aren’t afraid to re-organise themselves as requirements change.

4. Are you responsible for your architecture?

Like requirements, nowadays the best architectures emerge, rather than be prepared up front. Teams have more context than anyone, and have to suffer with the consequences of a bad architecture, so they should have a large say in what that architecture is. There is room for people to sit at a context beyond a single team, and make sure they aren’t doing the same work. But, nothing sabotages teams quite like ivory tower architects or non-technical stakeholders defining the architecture. It’s also a huge drain being ham-strung by an architecture you know won’t work.

5. Do you actively maintain a diverse team?

It’s draining to see startups and big companies fail to attract diverse development teams. Most treat it as a corporate social responsibility, rather than the advantage it actually is. I don’t want to work in a team comprised entirely of rich, white guys.

6. Do you regularly challenge your own process?

People like to classify teams as Scrum, or Agile, or Lean, or some awkward, edgy, portmanteau. It can be useful. It’s helpful when speaking to a potential new hire, since it can compress a lot of information about a team (e.g. do you have a board; do you do stand-ups?) into few words. It’s less helpful when it’s enforced, and can quickly strangle effective teams into failure.

If I had one required part of any process, it’s the retrospective. Teams should experiment with their process. In retrospectives, they should analyse what’s going well, and what isn’t, and be honest about it. Exactly like their product.