Azure DevOps YAML Pipeline with .NET Core and Coverage Report

I played with the new feature of Azure Pipeline. For the initial setup, you can see this video on the announcement. As my personal project, I developing a tool which use Go lang as CLI and Azure Function V2(C#) as backend. I’d like to share how we can create Continuous Integration pipeline with .NET Core with Coverage Report.

Azure DevOps with Code Coverage using .NET Core

Motivation

We have a good instruction from the Azure DevOps official documentation. However, it tightly coupled with Visual Studio. My project is much Open Source Project, so I don’t want to limit the contributor someone who has Visual Studio.

Other problem is, this pipeline never show the Coverage Report on the website. We need to download and click the detail report on the Visual Studio.

TL;DR

For someone who want to know the pipeline I’d like to share the yaml file, first.

This is the target project for the CI.

Strategy

I use coverlet for collecting coverage. We can use it not only for Windows but also Linux and Mac.

For generating HTML report, I use ReportGenerator.

Generate Coverage Report

To Generate the coverage report, you can write like this. You need to add nuget package `coverlet.msbuild` on your test project. Then you can use /p:CollectCoverage=true parameter. Also you can choose the format of the coverage like cobertura , opencover , etc. However, for the Azure DevOps, I recommend to use cobertura since the following task request cobertura or jacoco as a format to upload coverage. StrikesLibrary.Test is the target test project. Unfortunately, We need to specify one by one. By default the report is written under the target test project directory called coverage.cobertura.xml

dotnet test --logger trx /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura  StrikesLibrary.Test

Merge the result

Since we need to specify several test projects, we need to merge the result.

dotnet test --logger trx /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:MergeWtih=$(Build.SourcesDirectory)\StrikesLibrary.Test\coverage.cobertura.xml StrikesRepository.Test

This script test the second test project called StrikesRepository.Test You need to specify the report which is specified the last one. /p:MergeWith=....\coverage.cobertura.xml will merge the report and write it under the target test directory as coverage.cobertura.xml as well

Generating Report

For generating, the report, you need to install report generator.

dotnet tool install dotnet-reportgenerator-globaltool --tool-path . --version 4.0.0-rc4

When you use this on your client machine, you don’t need --tool-path however, on the CI environment, I recommend to specify it for find the tool. Because the shell should be reloaded.

Generate HTML report

Execute the command with specifying the report and target directory. In this case, I output the HTML report on the .\results directory.

.\reportgenerator "-reports:$(Build.SourcesDirectory)\StrikesRepository.Test\coverage.cobertura.xml" "-targetdir:results" "-reporttypes:HTMLInline;HTMLChart"

The point is the reporttypes: Use HTMLInLine for enabling the output on the Azure DevOps page. Azure DevOps Coverage page show index.html on the web. However, the CSS and Javascript should be included. For this purpose we can use HTMLInline to include CSS and Javascript on the index.html.

Conclusion

Using YAML, we can use full functionality of the VSTS tasks. Now you can see the Unit testing and coverage report on the web. IMO. The important thing is getting feedback quickly and notice an issue. That is why I want make it available on the web. This YAML feature which is the pipeline as code works very well and we can include it on our repository. I love this new feature.

Resources

Some useful links.

YAML definition

Official page for .NET Core pipeline

VSTS task detail spec

You can see both YAML and GUI configration

If you can’t find the spec, you can see the task.json on this repo. This link is an example of the script.