Coverage != Quality

Ramin A
3 min readSep 5, 2018

--

In software engineering the presence of unit tests that ensure your code behaves the way you intended it to is a crucially important practice. This is why development practices such as TDD (test driven development) and BDD (behavior driven development) were developed to make sure the code you deliver is well tested (covered).

DISCLAIMER: Before I go on, I want to make sure you all understand that I’m a *HUGE* advocate of code coverage. I fully believe that an app (or service) must have a good battery of unit and integration tests, combined with a much smaller number of end-to-end tests (at times called smoke tests) to make sure the code base is protected against behavior regressions (bugs) during the many code changes that are part of the standard SDLC (software development life-cycle).

Having said that, there seems to be a tendency in the software world to equate the mere presence of tests with code quality. I’ve now seen this attitude over and over at several companies at which I’ve worked. I believe this not at all to be true and here’s why…

They say a picture is worth a thousand words. Well, here are 2 to convey my point that code coverage has *ZERO* to do with code quality:

I love analogies, so consider the case of the following 2 cars…

The one on the right is a brand new 2019 Volvo S60 (just might be my next car! :-), and the one on the left… well, frankly I’m not sure but if I had to guess I’d say an American 1970s era gas guzzler of some type. For the purposes of making my point it doesn’t really matter.

Here’s what’s true about BOTH of these cars. They will…

— — Pass 100% of their unit tests:

1. Doors will open

2. Wheels will turn

3. Tires make contact with the road

4. Windows will roll up & down

5. Headlights will turn on when power applied

6. Etc, etc, etc… you get my point.

— — Pass 100% of their integration tests:

1. Engine will start when start switch is activated

2. Accelerate when in gear & gas pedal is pressed

3. Change direction of travel when steering wheel turned

4. Decelerate when brakes applied

5. Etc, etc, etc… you get my point.

Now I ask you… which one of these cars would you want to…

· Drive on a regular basis?

· Show off to friends & family?

· Proudly present to the market? (open source community!)

· Put effort/money into maintaining?

· Hinge your business on?

Which one of these cars will mechanics (software developers) actually want to work on? And no, I’m not talking about the specialized hobbyist who don’t mind spending 100s of 1000s of $$$ restoring a vintage automobile. No, I’m talking about your average mechanic at an auto shop.

Conclusion: Quality of anything (not just code) has to do with the quality of the materials, design, and most important of all… skill and craftsmanship that went into creating it. Not whether or not the product functions as intended (passes its unit & integration tests). Code is no different. The skill set of the developer, the craftsmanship that he/she applies to it, driven by their pursuit of good SW engineering practices, are all what determine the quality of the code they ultimately produce.

Agree? Disagree? Let me know! :-)

--

--

Ramin A

DevOps/GitOps, CD, CloudNative tinkerer. Developer & clean code advocate. IoT junkie. Teacher/mentor.