Combined code coverage for Flutter and Dart
As your Flutter app evolves, it is often a good strategy to break-out Flutter and pure Dart code into their own packages as local dependencies.
Extracting local package dependencies encourages modularity and code reuse and facilitates testing.
As an added bonus, your business logic and model layer, etc… can be extracted into pure Dart packages and re-used in Angular Dart or other web framework. So developing a web version of your app suddenly becomes much easier.
However generating project-wide code coverage reports when using local package dependencies requires some extra steps.
What is combining?
Combining for code coverage is a method of gathering the code coverage data generated by each local package and combining it into one project-wide data file. A project-wide code coverage report can then be generated giving you visibility into the coverage of your tests.
What’s the advantage of a project-wide code coverage report?
As a project grows, untested code can creep-in. This increases the opportunity for bugs, and undetected regressions, making it into the released app.
A project-wide coverage report
- highlights important functionality in the code that has not been tested
- identifies code that is not testable (not an uncommon phenomenon), which indicates the code should be refactored
- finds code that is no longer being used… or never was (we’ve all been there)
- useful when code is being committed and during code review (if you do code reviews)
- monitors the coverage trend over time
Overall, a project-wide coverage report facilitates catching bugs, and risks of regression, as early as possible. That’s a good thing.
Here’s an interactive code coverage report that shows what I mean. Clicking on the outer circles will drill-down and then take you to individual files for a closer look on what your tests are covering and not covering (report is towards the bottom of the linked page):
How to combine
Well, there are a few tricks required to get this to work when using local package dependencies… here’s why…
- Flutter can only generate code coverage for a single Flutter package.
- Dart is generally the same, in that it too generates code coverage for a single pure Dart package.
So how can a, guaranteed up-to-date, project-wide, coverage report be generated?
Here’s the trick… it’s actually pretty simple:
- Scan directories from the root of the repo for packages
- Look for tests in each package
- Find out if a package is a Flutter package or a pure Dart package
- Run the tests in each package with coverage enabled and gather the coverage data
(Flutter and Dart tests have different invocations) - Append the coverage data into a common data file
The following utility will do all of these things… it runs the tests of each package (Flutter or pure Dart), with code coverage enabled, and combines the coverage data of each package into a single coverage data file. You can then run a project-wide code coverage report.
A project-wide code coverage report can be generated from the combined data file either locally as you develop, or automatically in a CI environment (like travis or cirrus, or in-house). CI (Continuous Integration) can also automatically include this report in Pull Requests where code reviews occur (if you do code reviews).
Show me some reports!
A report, useful during development, that you can run on your own project right now, looks like:
The beautiful reports from CodeCov, generated automatically during CI, look like:
A report from Coveralls, generated automatically during CI, looks like:
Let’s see it in action!
It is simple to use this utility in a CI environment (such as one set-up in Travis-CI or Cirrus-CI). Just call the utility during the CI build to generate the coverage data file and pass the data file to the code coverage service (such as CodeCov or CoverAlls) to generate the report.
You can see an example in this .travis.yml file.
(In GitHub you need to grant permission, to the CI service and the code coverage service, to run the CI build and access the code coverage data file.)
To see an example of the result of generating project-wide code coverage in a CI environment, using this utility, see:
For an example of a complete CodeCov report, using a code coverage data file automatically generated by this utility during CI, see:
Conclusion
Project-wide code coverage is useful to show that new code is being tested. As the project evolves over time, it also shows coverage trends in the code base…. not to mention you get handy reports that can help while writing those tests.
If you are not currently writing tests I highly recommend you do, especially for your business logic. It dramatically increases the quality of your code. It also makes it easier to add progressively more and more advanced business logic and more advanced features, etc… Testing can have a positive impact on productivity over time.
In a follow-up article I will show how to use this coverage reporting as part of a CICD (Continuous Integration/Continuous Deployment) solution for Flutter on Travis-CI to deliver your app automatically to the Apple and Play stores.
Finally…
Let me know if you have any questions or comments about this method of generating a project-wide coverage report for Flutter and Dart, below or on GitHub or Twitter.
If you think this article is useful, it helps if you clap, like, subscribe and share with your friends for more Flutter developer articles.
