Pro Tip #2: Leave it better than you found it — Code coverage done right

My second pro tip is about code coverage. And not the way you think about it if you’re a “coverage skeptic”. I think this method provides value to any project, at any stage and regardless of previous code coverage.

The two problems with code coverage are:

• A well tested project that grows in size will eventually let code slip through the coverage checks (unless you’re actually doing 100% coverage).
• An old project with low to nonexistent coverage is not fun to start checking coverage on. It just provides you with a feeling of hopelessness!

The idea

Let’s solve this once and for all. Instead of checking the coverage of the entire codebase or files — let’s check the coverage of our actual changes.

This way we can ensure that what we’re commiting is tested. It works great both for that huge codebase that is pretty well tested where something still might slip though the cracks-and it’s equally good for a project that starts out with 0% coverage.

What we need

To make this work we basically have to do three things:

• Use a tool that’s able to check the coverage on your diff. We use a python package called diffcover.
• Generate reports that this tool is able to use. For diffcover this would be the cobertura format. Usually this is something you can just enable with a simple config change in your test setup.
• Run this check as a pre-push script. This is something I covered in my last Pro Tip.

How to

I’m doing fullstack JavaScript so I’ll show you how to do it in a Node/Mac environment. The steps are similar for other frameworks — so you should be able to use this principle however your setup looks.

1. Set up a pre-push script (if you haven’t done it — here you go)
2. Install python and pip if you haven’t already (Python’s installed by default on Macs)
3. Install diffcover:
   pip install diff_cover
4. a, Make sure your tests outputs a cobertura report. For jest you’d just add it to the coverageReporters array:
   “jest”: {
   …,
   “coverageReporters”: [
   …,
   “cobertura”
   ],
   …
   },
   b, For istanbul you’d add it to .istanbul.yml like this:
   reporting:
   …
   reports:
   -cobertura
   …
5. Run your tests to generate coverage and add this script to your package.json with your cobertura path:
   “cover-check-diff”: “diff-cover my/cobertura/file.xml — fail-under=90”
6. Make sure this script is being run in your pre-push sequence.

Wrapping it up

What we did here is get rid of most of the pains of code coverage — and at the same time we made it valuable in any project regardless of previous coverage.