The Bear Test v0.1

Almost two decades ago, Joel Spolsky wrote the venerable Joel Test. The test is a series of twelve yes/no questions that signal the quality of the software produced by the team taking it.

Parts of the test are specific to the kind of software Joel and his team at Fog Creek were making. At our company, the central problem is not the quality of the software we deliver. A more far-reaching and harder problem for us to solve is that of retaining good developers.

Creating a company that developers will want to stay at for longer than a few years is difficult. Each developer on your team will have different priorities about what they want out of their careers. To maximise the likelihood that they stick around, you’ll need to tailor their employment experience to their specific needs.

At the same time, there are some generalisations we can make that, all other factors considered, would be preferable for experienced developers in the long term. These include process and practices that help keep the team motivated and productive.

1 • Do you plan at least one week of development work in advance? Answer no if your developers spend more of their time fighting emergent issues than they do on feature development. Answer yes if, for example, you have a backlog of work out of which your development completes tasks in timeboxes of at least one week.

2 • Do you allow for elective code reviews? Answer no if your development team is not in the habit of doing code reviews as a regular part of development. Answer no if your developers are forced to do code reviews for all changes. Answer yes if developers can choose to have their code reviewed if they feel a review is worth it in terms of potential increase to the quality of software.

3 • Is your company equipped to work remotely? Answer no if your entire company is 100% on-site. Answer no if you have some staff working remotely but all the important communication happens in-person. Answer yes if you’re comfortable and productive with a significant proportion of your staff working remotely almost all of the time.

4 • Do employees choose their own working hours? Answer no if you have a daily morning stand up meeting that you expect everyone to attend (virtually or otherwise). Answer no if you expect everyone to show signs of doing work at certain hours. Answer yes if employees can choose their own working hours on a day-to-day basis if they want to. You can answer yes if you specify e.g. four “core” hours. You can answer yes if employees choose their hours to coincide with meetings at mutually agreed times.

5 • Is there an explicitly specified maximum number of working hours per day? Answer no if you regularly expect employees to work over-time without pay. Answer no if you ever expect employees to work on weekends without time off in lieu. Answer yes if you have documentation that you encourage employees to read that explicitly specifies a maximum number of daily working hours.

6 • Do all meetings have an agenda? Answer no if any of your last three meetings were open-ended. Answer yes if you had a clear agenda that someone attending enforced.

7 • Is there a budget for books, conferences, workshops and other training? Answer no if there is a long approval process for expensing training costs. Answer yes if developers can easily expense training costs that are conceivably beneficial to their day-job.

8 • Are developers consulted while planning features? Answer no if a non-developer plans/requests work, drops it in your developers’ laps and expects them to implement. Answer yes if there’s a consultative process that allows developer input before work is prioritised.

9 • Can developers deploy in one step? Answer no if deploying a simple change to end-users requires multiple tasks and/or more than one developer to complete successfully. Answer yes if deploying a simple change is a discrete task that can be carried out by anyone with the required access rights.

10 • Is evaluating development candidates centred around a work sample test? Answer no if “culture fit”, coding at a white board or logic puzzles designed to “see how candidates think” feature in your hiring process. Answer yes if you employ a work sample test that includes tasks that are representative of what the candidate will have to do in their job and the result of this test is the most significant factor in your decision to extend an offer.

11 • Do all employees have regular one-to-one meetings with their managers? Answer yes only if all employees regularly attend one-to-one meetings with their manager that include time for open-ended discussion.

12 • Does your company have an established business model? Answer no if you’re a startup that has just received funding. You should probably answer no if you’re not profitable. Answer yes if you have an established source of revenue and can depend on that revenue being there for the next year.

Reasoning

The goal of this test is to evaluate how likely you are to retain developers over the long term. It doesn’t attempt to address the quality of the software you will produce, how productive you will be or how likely you are to attract developers in the first place.

The test tries to identify how well the company maximises the following:

• Autonomy. How much say do developers have in the content, methods and logistics of the work they do? The more control they have, the more invested they will be in the outcomes.
• Technical skill development. A developer’s technical skill is their primary asset. They are leasing that asset to you for their monthly salary. It is in both yours and their interest to increase the value of that asset. Will the value of their skills improve or stagnate while employed with you?
• Quality of life. What might be termed “work-life balance”. How does their employment with you positively or negatively effect the rest of their lived day-to-day experience?

1. Do you plan development work in advance?

Planning work in advance, preferably at least a weak ahead of current development, means that developers have an idea of what’s expected of them in the next time box of work. For some teams that’s a week, for others it’s two, and for others it might a month or longer.

By giving your developers a set amount of work to be actioned over a set amount of time, you give them the ability to plan the timebox and have some agency in the day-to-day work they do. By doing the planning up front, someone will have made a decision on the priority of the tasks, so the developer can be assured that what they’re working on is in fact valuable to the business.

The alternative of responding to an unfiltered stream of bug reports and feature requests can be highly stressful for a developer. Without a clear “win condition,” it’s difficult to show to yourself and to others that you’re making a productive contribution to the business.

2. Do you allow for elective code reviews?

Code reviews are one of the best ways for your technical team to share knowledge and establish norms and values. By allowing the option for them in your process, you give your team space to improve their skills together and increase the quality of your delivered software.

The elective part is important. Forced code reviews (or forced any non-critical process) indicates a lack of autonomy, so you shouldn’t get any points if you make them mandatory.

3. Is your company equipped to work remotely?

“Remote” is another word for location-autonomous. Some developers like to work in open-plan work spaces, others in cafes, and others in an office with a door that shuts. You get the most value from developers when they’re given the choice to work where they’re most productive.

Working remotely increases your candidate-pool, allowing you to be more selective with your hiring. By choosing better qualified candidates, you end up with more highly skilled developers than otherwise and this higher skill level raises the bar for everyone on your team.

Allowing your employees to work from wherever they happen to find themselves has a huge impact on their quality of life. You save hours of their time commuting. They can spend this time doing things they care about, thanks to your commitment to allowing for remote work.

4. Do employees choose their working hours?

People are productive at different hours of the day. Especially for development work, some programmers work better at odd times (like late at night or early morning) when they’re less likely to be interrupted. By letting staff choose their own working hours, you allow them to give you the best working hours of the day, rather than the arbitrary nine-to-five.

Choosing your own working hours allows you to fit your job around the rest of your life. That means you can drop your children off at school every day, have lunch with people you care about, go to the gym, take a class, read a book, or whatever you want outside of working hours, while still doing a good days work for the company. It means not having to squeeze the rest of your life into the last two or three hours of the day.

5. Is there a maximum number of working hours per day?

If your goal is to retain developers over the long term, then one of your primary concerns should be avoiding programmer burn out. Given the opportunity, many programmers will work themselves into the ground to get a feature launched in time for a deadline.

This results in bad quality software and exhausted developers that are neglecting other parts of their lives. Over-worked developers have very little incentive to stick around for more punishment.

You can avoid developers over-working by setting an explicit limit on the number of hours they’re allowed to do work for you.

6. Do all meetings have a clear agenda?

Having agendas for meetings shows that you value your team’s time. Instead of having open-ended discussions without direction, setting a clear agenda makes sure that the time spent in the meeting isn’t wasted.

This is especially important for developers, who are typically held accountable for how long software takes to build. If you fill up their day with open-ended meetings that don’t achieve much, you’re taking away time they could be using to deliver software.

7. Is there an easily accessible training budget?

This speaks directly to your team’s ability to improve their technical skills. By having a budget earmarked for training, you do two things.

You first make it possible for developers to access training material they otherwise wouldn’t be able to. High-quality training is expensive.

You also send a clear message that an investment in your team’s skill is an investment in the company. Sending developers to conferences, giving them time to study and providing them with books/courses as required makes it clear that you want their skills to improve while working with you.

8. Are developers consulted while planning features?

A central problem in software development is the gap between what you want built and what a developer builds for you. The wider this gap, the more risk there is that a project will fall behind schedule and go over budget. In almost all cases, the development team will get the blame, no matter how badly thought out the requirements are.

If you consult with developers earlier in the planning progress you give them an opportunity to better align themselves with the project plan. You’re better able to manage risk and more quickly deliver working software. You’re helping to set developers up for success rather than failure.

9. Can developers deploy in one step?

A developer being able to deploy in one step is signal for how much time developers are enabled to invest in improving the quality of your software.

If an operations process as common as deployment is a complex, multi-stage process that you’ve invested no time in optimising, you are severely underestimating the cost of developer time.

If you give developers time to make your infrastructure less error-prone and more efficient to operate, you’re a much better technical manager than otherwise. The experience of working on your systems will be concretely better and developers will spend less cognitive effort fighting fires.

10. Is evaluating development candidates centred around a work sample test?

Orchestras use blind auditions as part of their selection process. They do this because the best indicator of performance is a candidates ability to carry out the kinds of tasks that they will be expected to do on the job.

This is definitionally true. If the job you were applying to required you to jump three feet off the ground every day, you would expect to demonstrate your jumping ability in the interview.

You wouldn’t need to have a conversation about trends in the jumping community. You wouldn’t talk about what jumping blogs you read or jumping conferences you’ve attended. You wouldn’t sketch solutions to jumping problems on a whiteboard. All you would have to do to demonstrate your jumping ability is jump.

The same should be true when selecting programmers. The best indicator of a potential new hire’s ability is their performance on a representative work sample test.

If your job requires that developers help with requirements gathering, there should be a section on the test that evaluates requirements gathering ability. If your job requires that developers refactor code, fix bugs or manage incidental code complexity, those things should be on the test too.

The test should allow you to make an apples to apples comparison between one candidate and another. Any two people at your company evaluating a candidate should come up with the same results when evaluating a candidate’s submission.

Every new hire either raises or lowers the overall engineering ability of the team. When you select for ability at doing the job (rather than ability at interviewing) you’re more likely to hire competent people.

Competent developers want to work with other competent developers. They are more likely to leave if less competent developers join the team, especially at the same pay-grade. Avoid this by having a hiring process that selects for ability to do the job.

11. Does everyone do one-on-one meetings?

A regularly scheduled one-on-one meeting between managers and developers is the minimum required to find and resolve problems developing in your organisation.

These meetings can take many formats, but typically serve a few purposes:

• Build rapport between the two attendees.
• Allow the developers to surface any workplace issues that they would otherwise not have mentioned.
• Allow the manager to give feedback to the developer about their performance.
• Deciding on a plan of action to address any identified issues.

These conversations may happen without a formal one-on-one process, but instigating them regularly signals an awareness that personal relationships within the company are important to senior management.

A team that has honest conversations about problems and works together to resolve them is more likely to retain their staff than otherwise.

12. Does your company have an established business model?

Developers want to stay productive and deliver software. It’s what you’re hiring them to do, it’s what they’ve chosen as their career and they’re happy to do it.

A big distraction that gets in the way of this is worrying about job security. A business that’s constantly changing tack is one that may not require your services six months from now.

In this environment you can never fully commit to the company. You might take a lot of freelance work on the side and keep your options open with companies you’ve interviewed with in the past. The company doesn’t provide the stability required for its team to fully engage.

Developers with more experience are older. As we get older, the number of commitments and responsibilities we have increases. We can’t afford to take risks with our income. Developers can have their pick of jobs just now, so can find a stable job with an established company almost overnight if they want.

Developers will still leave

There are other factors that could be just as important to certain developers leaving. The test is meant to be diagnostic rather than prescriptive. You won’t necessarily build an organisation that developers will want to stay at just by making your company pass the test.

No matter what you do, developers will still leave. The market for technical talent, the particular priorities of each developer and our assumption that the grass is perpetually greener elsewhere guarantee it.

What we can do however is regularly assess the type of work environment we make for our teams and incrementally improve it. Over time this will result in a workplace that will be progressively more difficult to leave, as the alternatives will be so much worse in comparison.

If you’ve read this far, you should probably sign up for the Happy Bear Software mailing list. We’ll send you articles about running successful software projects, never spam you and you can unsubscribe at any time.