Measuring code quality

Anyone who has worked in an organisation with more than a couple of dozen of developers has probably been asked to report on ‘code quality’.

In order to make this measurable, this often takes the form of percentage code coverage, static code analysis (SCA) failures, percentage of methods with comments, cyclomatic complexity analysis etc. Tools like Sonar provide really powerful analytics for these kinds of stats and let you track changes over time and allow you to dive into problem spots and help you determine which parts of your code base need some TLC. As they easily integrate with any CI tool, it’s easy to keep these stats up to date.

I've found these tools really helpful for analysing applications I know well. They help me keep an eye on how disciplined I/my team is being with writing tests, adding comments and keeping things simple.

Can this approach really help with measuring the quality of a whole code base across an organisation?

I don’t think I’m in the minority if I tell you my answer is an emphatic “No”.

It’s all well and good having company wide code coverage targets, but this can only help if the quality and usefulness of those tests is consistent across all the applications and all teams. A colleague once told me anecdotally of a team (in a company I've never worked for!) that simply asserted the equals and hashCode method on every class they added in order to reach the target.

As soon as code ‘quality’ is reduced to a percentage its opened to being gamed. Code coverage becomes something to be manipulated rather than a useful tool for teams to reflect on their performance. Teams start focusing on removing SCA failures rather than fixing genuine technical debt.

We can’t fault organisations that want to measure code quality. High quality code is a huge business advantage that means the organisation is likely to be able to react quickly to changes in the market and get useful, profitable software out to customers quickly.

So how can we gain an accurate view of whether the code we’re writing is any good?

For me, Jon Sonmez has provided the most succinct answer:

Yes it’s satire and yes it’s offensive, but it’s also true. So let’s look at how we can really measure code quality.

1. Code quality should be an impassioned debate

Given the opportunity, most developers care about code quality. They want to write good code — they want to produce work they’re proud of. They probably care more about code quality than most people who want to set targets for it.

Try it out. Take any group of developers and ask them whether the code in the applications they work on is of high quality and you will be there for hours.

At least you should be. If you’re not, you need to ask why. Do they not care? (Really worrying!) Do they feel they can’t speak the truth? (Change this, fast.)

We need to create a culture where our developers, at whatever level, can talk about a topic they care deeply about — writing good code.

2. It should be defined by those responsible for writing and maintaining the code

Too often code quality ‘targets’ are set ‘from above’. These rarely get the results desired and I doubt they ever inspire teams to write better code.

A much better approach is to get the teams themselves to define what high quality code looks like. This can be done at any level — organisation, application, team, individuals.

We’re all used to running retrospectives at the end of a sprint — why not do the same for code quality? Get a group of developers together and ask them to complete the sentence, “High quality code is …”.

The results will almost certainly not be as measurable as ‘achieving 75% unit test coverage’ but it will be realistic, it will be unique to your organisation / department / application and the developers who came up with the definition will more likely live by it because they wrote it - so they own it.

And that’s the goal — getting developers to own code quality. After all, they own the writing, testing, deploying and maintaining of the code — they should be responsible for making it high quality.

3. Time should be spent on it

Writing quality code is an investment — it takes longer initially but the return on investment is virtually guaranteed in terms of fewer bugs, easier to add new features, easier to onboard new team members etc.

If we define a set of code quality ‘hallmarks’ it will take time to bed them in. It will take time to coach others and it will take time to make them part of the ‘developer DNA’.

We need to ensure that our developers have time to build quality in from the outset. Practices like pair-programming can really help with this as you get two pairs of eyes on the code rather than relying on after-the-fact peer reviews which can happen after the damage has been done.

This isn’t to say peer reviews are bad (I’m a huge fan of using Git’s pull requests as they let me take the time I want to review code) but they’re best done in small chunks.

We need to make sure our developers understand that we (the pointy haired managers) know an upfront investment in our code is really worth it in the long term.

When walking around our development teams, we should see developers huddled around screens / whiteboards discussing stuff. A lot less code gets written this way — but I’d argue the overall value added to a business is higher than if everyone has their headphones on and is developing at a thousand lines of code a minute.

4. Honesty is key

If we make being passionate about code quality part of our culture then we should be prepared for some heated discussions (WTFs/minute).

“Why on earth have you extended this class? Haven’t you heard of composition? You couldn’t extract a meaningful interface from this class if your life depended on it…” etc.

This is good and to be encouraged. It can create a peer pressure that raises the bar of every developer and turns every review into a coaching opportunity.

It can also hurt people and discourage those who are lacking in confidence. So we need some rules:

1. Always assume the other person acted from the best intentions
2. Always assume the other person wants to learn and improve
3. Believe the other person has something to teach you
4. Genuinely believe that the code you are reviewing is as much your responsibility as it is the person who added the latest commit.

Culture can’t be measured, but it can be tested

So let’s assume we’ve invited our developers to create a definition of code quality and they came up with the following:

High quality code is:

1. Easy to add new functionality to
2. Can be deployed to production frequently with little/no manual regression testing
3. Self-documented by its tests
4. Easily understood by new developers
5. Performant as defined by mutually agreed NFRs

We’ve told them to take responsibility for building this into the code from the start and we’ve noticed a marked increase in the consumption of white board markers and a greater volume on the office floor.

How do we know if it’s worked?

The problem is that it’s hard to measure a culture (which is what code quality is really about). But that doesn’t mean we can’t test it.

We can say to our teams, “I’m fully on board with the definition of code quality you have come up with. I will give you whatever support you need to make this part of our culture. In our 1-to-1s I will be asking for evidence that we’re making progress.”

In the weeks and months to follow we can start to probe a bit:

“I know Bob joined your team last week, what have you got him working on now? How quickly has he got up to speed?”

“Can I see some of the documentation produced by your tests?”

“What are the NFRs for application X? How often do you verify them?”

Moving away from code quality metrics to code quality definitions doesn’t mean we can’t evaluate how we’re doing. In fact, we can actually delve much deeper into what our code is really like and ask the questions that really matter.

At the same time we’re empowering our developers at all levels and letting them take responsibility for the quality of the code used by our customers.

Conclusions

Code quality is about the culture of our development teams way more than it is about measuring a set of metrics.

This isn’t to say these metrics (code coverage, SCA failures etc) are not valuable — they can be very helpful to people who have an in-depth knowledge of the code bases they’re analysing.

We can foster high quality code cultures by:

1. Get people talking about it in formal and informal settings
2. Make the teams/people who write the code responsible for defining what high quality looks like
3. Empower these same people to ensure these practices are lived out
4. Regularly test whether they are practicing what they preach