Flutter test coverage in Gitlab CI

Showing test coverage and whether it is going up or down on a pull request can change whether or not coverage gets added by your team.

Visualizing and giving a feedback signal when test coverage changes encourage adding more tests.

In Gitlab, you can get the test coverage added like this:

Example of a test coverage report in Gitlab

Not only is the current test coverage shown, but you also get a red or green diff in percentage from the code being changed.

Add one line

Adding the test coverage to your pipeline job is only one line.

job_name:
  image: cirrusci/flutter
  stage: test
  script:
    - ...
  # add this line
  coverage: '/\s*lines\.*:\s*([\d\.]+%)/'

That is if you already have tests running and outputting coverage. If not, read on.

The regular expression here will be applied to each line of the output and match the coverage percentage and add it to the merge request. This regex specifically will match output from lcov which you can use to parse the flutter coverage report.

Overall coverage rate:
  lines......: 65.0% (279 of 429 lines)

Adding test coverage to Flutter

The flutter test library comes with test coverage out of the box. Add the coverage parameter to your test run command:

flutter test --coverage

You can also generate reports for the test coverage as to see where the missing coverage is. This is possible with lcov and genhtml.

I use a custom container that has lcov added to it, but you can just as easily install lcov on top of the more common containers by adding another line to your script. But very likely, your container already has lcov available.

build:
  script:
    - apt-get update && apt-get install -y lcov
    - ...

A complete Gitlab CI job can look like this:

build:
  # a custom docker container I use
  image: greycastle/flutter:3.0.1
  script:
    - flutter test --coverage
    - lcov --list coverage/lcov.info
    - genhtml coverage/lcov.info --output=coverage
  coverage: '/\s*lines\.*:\s*([\d\.]+%)/'
  artifacts:
    paths:
      - coverage

Why not GitHub

A few years back, GitHub had yet to add GitHub actions and Gitlab in comparison, came with a free GitHub CI tier to run your CI/CD.

Comparing it to GitHub actions now, the two are very like each other, and GitHub definitely makes for a good alternative to Gitlab.

Personally, I have begun moving some of my projects over to GitHub instead and for a full reference on how to do CI/CD for Flutter on GitHub actions, check out the TravelRates app repository.

GitHub has a similar way of displaying code coverage on pull requests:

Code coverage report on a GitHub pull request

Tests where worthwhile

Having tests can give you peace of mind that you haven’t broken anything unintentionally.

However, writing and maintaining tests can be difficult, so you should consider when it is worth it.

My flutter projects have coverage of about 65%.

I try to test most widgets in isolation to see if they can render, and then I use TDD to write out functionality for logical parts where the code transforms or calculates something. Where I can, I add integration tests to check that API parsing works fine as well.

Regardless, having that coverage percentage keeps me accountable and helps motivate me to keep that coverage from dropping too low.

Hope this helps.