Angular Unit Testing Code-Coverage Lies
Hopefully, you’re a fan of unit testing. When done correctly, unit tests can give you high confidence in your code doing exactly what you expect it to. But, if you are trusting your code-coverage reports to give you the warm fuzzies, your confidence is terribly misplaced. Let’s see a few examples.
First, clone the near-earth-asteroids repo from Github:
This is a repo I use for doing unit testing training and explaining some of the gotchas in unit testing. We will just be focusing on one small part of the app to illustrate how code-coverage metrics lie to us.
Do a quick npm install and then run ng test --code-coverage to generate a code coverage report. While that is running, open up src/app/features/coverage-metrics-lie/coverage-metrics-lie.component.ts.
In your project folder you will have a new coverage folder. Open the index.html from that folder. Click on the coverage-metrics-lie link to drill down into that folder’s code-coverage results.
Looks good! We are only missing a quick test for the delete function and an even-numbered scenario in the getSomeData function. Not bad!
But is that really true? Open the spec file for this component at src/app/features/coverage-metrics-lie/coverage-metrics-lie.component.spec.ts :
Wait a second! We haven’t written any tests for this component yet?! This is just CLI boilerplate for a component unit test. How can we have 87.5% coverage on a component with this much code without any unit tests?
Lies!
The truth is that this component has 0% unit test coverage.
Here we start to see how we can misinterpret the meaning of a code-coverage report. The report only shows us what lines were executed, not what lines were tested. In Angular, when you compile a component in a unit test ngOnInit and everything it calls will get “covered” immediately.
Coverage = Executed
Coverage != Tested
Let’s compound the issue. How much of the template is getting tested? The coverage report doesn’t even report on the template files! Is there code in your templates? Yes. Here is our template:
There are at least four tests needed to cover this template.
If you have bindings: [someProperty]="someValue" or (click)="onClick($event) or {{ something }} then you have code in your template that you need to write tests for. You need to know that you bind the right data into child components, render the right data to the DOM, and that you handle events from child components correctly.
If you have *ngFor or *ngIf in your template then you have code in your template. You need to test that those conditions update correctly based upon the component state.
In addition, if you have @HostBinding's in a directive, these do not generate any executable code. But you may should write tests that verify that a component ends up with the right styling and behavior when that directive is attached to it.
I have not found any tooling that supports analyzing the template code or dealing with @HostBinding in any way. Or that can tell the difference between tested and executed.
So, what should we do to ensure that our unit tests accomplish their purpose?
First, when you are writing your unit tests (assuming you’re not doing TDD or BDD from the outset) start with the template. Why the template? To begin with, it’s easy to forget. In addition, there are many times where you will find that testing the template first accomplishes testing most, if not all, of the code in the component file as well. Remember that the template represents most of the behavior of the component. And testing the behavior is really what our goal should be. Does this component do what we expect? (Later, I’ll post a follow-on article on what makes a good unit test.)
Second, be sure to do quality code reviews. A code review is the opportunity to compare the unit tests with the template and component code files and ensure that the behavior is tested.
So, is there any value in a code-coverage report? For unit testing in Angular, no. It’s misleading. So, if your manager is asking you for 80% code coverage based upon Sonar, or hosting the report in TeamCity, or CI reporting of your choice, share this article with them and help them understand what it means. Help them put the emphasis on good user stories with good acceptance criteria and definition of behavior (i.e. Gherkin syntax), and quality code reviews.
Then you can sleep well at night.
EnterpriseNG is coming November 4th & 5th, 2021.
Come hear top community speakers, experts, leaders, and the Angular team present for 2 stacked days on everything you need to make the most of Angular in your enterprise applications.
Topics will be focused on the following four areas:
• Monorepos
• Micro frontends
• Performance & Scalability
• Maintainability & Quality
Learn more here >> https://enterprise.ng-conf.org/
