What should you really measure Code Quality against?

Answer: Test coverage.

But how do you really measure test coverage?

Popular answer: number of lines of code executed by test cases.

NO!

Why? Because ensuring 100% coverage in terms of lines of code executed doesn’t really get you a quality product.

Then what does?

Good Test Cases and Adequate test coverage.

To achieve both, you need to look critically at your points of reference.

Improving the ratio of Good Test Cases in your Test repository
A good test case traces a requirement (positive and negative flows included) to fulfilment. All you have to do to ensure you mainly have good test cases, is to establish Requirements Traceability. In Agile projects, given you’re supposed to only focus on requirements intended for the next immediate release, achieving 100% test coverage by Requirements should be fairly straightforward if you use an elementary Traceability Matrix.
If you are only testing to requirements, you should only have good test cases in your repository.

Adequate test coverage
If you only write test cases that can trace back to a requirement, you’re effectively eliminating any redundant, unnecessary test cases. Now, if you turn that around, what you’d get is this: if you ensure 100% of your requirements are covered by test cases, then you have all the test cases you need. Then again, how do you ensure you have 100% accurate requirements? What if some requirements are incorrect, or missed? Well, as they say, “A high quality product built on bad requirements, is a poor quality product.” The focus then shifts to ensuring you have high quality user stories that cover all functional and non-functional requirements.

With Test-Driven Development, this is truly achievable. And if you are able to demo a working, almost-bug-free product every Sprint, and follow this up with mop-up Spikes to fix the remaining unseemly bugs, my experience suggests you’re in a good place to go to Release.

So, What Should Code Quality be really measured against?

Both requirements coverage and number of lines of code executed. I believe ensuring 100% requirements coverage is adequate. After all, with Agile, you’re supposed to only work on the high priority requirements for a release all the time, thereby reducing waste — of effort, time, money. Then again, humans aren’t perfect. Bad requirements could sneak into your scope. Good requirements could be missed. You may not discover this until late into your project. We can’t always guarantee 100% accuracy with requirements. It’s quite common to encounter projects where developers have written so many more lines of code than is necessary. Coding effort veers away from the requirements, and developers spend precious resource writing unnecessary lines of code. The golden rule is only write code necessary to deliver a requirement. Even then, there may be scenarios where 100% coverage may not be achievable or necessary. The goal, therefore shouldn’t ever be to achieve 100% code coverage. Although when combined with 100% requirements coverage, you should be able to find that code coverage by default hovers in the high 80s and even low to high 90s.