How to setup your dotnet project with test coverage reporting

7 min readDec 21, 2022

You all probably use Visual Studio Test coverage analysis, ReSharper dotCover or JetBrains Rider and quite happy with it. But there is an another option, which could become quite appealing when working on CI pipelines, or if you want to do it yourself.

I want to talk about Coverlet and ReportGenerator, and maybe show you some interesting MSBuild tricks along the way.

Coverlet

Coverlet is a cross platform code coverage framework for .NET, with support for line, branch and method coverage. It works with .NET Framework on Windows and .NET Core on all supported platforms.

It ticks some of my personal boxes for being an interesting project:

Part of the .NET Foundation
free, opensource, cross-platform
174+ Million downloads on nuget
2.6K stars on Github
has 90+ contributors
awesome documentation
built-in support with MSBuild

So what does it do? Gathers coverage metrics of your tests and exports them to one of many supported formats. quite simple. It’s a gross simplification of the tool but it will suffice.

Let’s wire it up

Quick note on compatibility: Coverlet supports .NET and .NET Framework. However, it strictly requires SDK-style projects.

Add the following package references to the following packages to your test project:

<Project>
  <ItemGroup>
    <PackageReference Include="coverlet.collector" Version="3.2.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>
    <PackageReference Include="coverlet.msbuild" Version="3.2.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>
  </ItemGroup>
</Project>

Notice that we have to include also the assets. they are build time assets, so no worries. use the terminal of your choice to run it

dotnet test --collect:"XPlat Code Coverage"

or… since we have MSBuild integration lets have some fun 😄
lets edit the test csproj file and add:

<Project>
<!-- added -->
 <PropertyGroup>
  <CollectCoverage>true</CollectCoverage>
  <CoverletOutput>../coverage/projects/$(MSBuildProjectName)/</CoverletOutput>
  <CoverletOutputFormat>opencover</CoverletOutputFormat>
 </PropertyGroup>
<!-- end   -->
 <ItemGroup>
  <PackageReference Include="coverlet.collector" Version="3.2.0">
   <PrivateAssets>all</PrivateAssets>
   <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
  </PackageReference>
  <PackageReference Include="coverlet.msbuild" Version="3.2.0">
   <PrivateAssets>all</PrivateAssets>
   <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
  </PackageReference>
 </ItemGroup>
</Project>

Now, coverlet will be used every time to collect test coverage metrics when you run this test project. In this example, the results will be created in opencover format and stored at ../coverage/projects/$(MSBuildProjectName)/ path.

Kinda nice… but we ain’t done yet.

Since you probably have more than a single test project (unit tests, functional tests, integration tests), you should probably avoid putting this code in each an every test project csproj, and instead prefer using the Directory.Build.props

In short, the Directory.Build.props is a file that MSBuild uses to amend your csproj files. When building a csproj file, MSBuild will actively look for a Directory.Build.props file in the current directory, recursively, until the first one is found.

<Project>
  <Import Project="$([MSBuild]::GetPathOfFileAbove('Directory.Build.props', '$(MSBuildThisFileDirectory)../'))" />
  <PropertyGroup>
    <CollectCoverage>true</CollectCoverage>
    <CoverletOutput>../coverage/projects/$(MSBuildProjectName)/</CoverletOutput>
    <CoverletOutputFormat>opencover</CoverletOutputFormat>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="coverlet.collector">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>
    <PackageReference Include="coverlet.msbuild">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>
    <PackageReference Include="ReportGenerator" />
  </ItemGroup>
</Project>

adding this file to the ./tests directory (assuming you have one to contain all the test projects) will apply those properties and packages to all projects located beneath it.

The first line ensures that if you have a Directory.Build.props file above this one, it will be included as well. For more information about how the props and targets files work please visit the Microsoft MSBuild documentation. After running the tests we see a nice console print out:

Passed!  - Failed:     0, Passed:    23, Skipped:     0, Total:    23, Duration: 131 ms - DemoSolution.UnitTests.dll (net6.0)


Calculating coverage result...
  Generating report '..\coverage\projects\DemoSolution.UnitTests\coverage.net6.0.opencover.xml'


+-------------------------+--------+--------+--------+
| Module                  | Line   | Branch | Method |
+-------------------------+--------+--------+--------+
| DemoSolution.Core       | 62.31% | 58.33% | 56.41% |
+-------------------------+--------+--------+--------+


+---------+--------+--------+--------+
|         | Line   | Branch | Method |
+---------+--------+--------+--------+
| Total   | 62.31% | 58.33% | 56.41% |
+---------+--------+--------+--------+
| Average | 62.31% | 58.33% | 56.41% |
+---------+--------+--------+--------+

but we also have some reports in our report folders.

Code exclusion from coverage is achieved by using the standard ExcludeCodeFromCoverage attribute. Please see documentation.

Those reports are a bit hard to read, they aren’t targeted for human readers . Instead they target CI tools (like Jenkins or TeamCity) and code quality analyzers (Codacy, CodeCov, SonarQube).

A note on integration: Coverlet doesn’t provide out-of-the-box Visual Studio integration, which makes it a bit of a bummer to use if you want to know which parts of the code are not covered. There are, however, various Visual Studio plugins to achieve such functionality.

ReportGenerator

ReportGenerator converts coverage reports generated by coverlet, OpenCover, dotCover, Visual Studio, NCover, Cobertura, JaCoCo, Clover, gcov or lcov into human readable reports in various formats.

ReportGenerator could be used to aggregate the separate reports created by Coverlet. The created report has some awesome features, I will leave you to discover them on your own in their Github repo, but I will talk about a few.

ReportGenerator could be used as a dotnet tool simply by installing it globally

dotnet tool install --global dotnet-reportgenerator-globaltool

It could also be easily imported as part of the project (i.e. to ensure it exists on the developer machine).

<PackageReference Include="ReportGenerator" Version="5.1.13" />

Note: use the most up to date version at the time of reading, not specifically this one.

In this example I’ll assume you have it as a global tool.

Again, using your trusty terminal lets run the tool (the ` is powershell newline escape — for bash use backslash \ character instead)

reportgenerator `
  -reports:./tests/reports/project/*/*.xml `
  -targetdir:./tests/coverage/report/ `
  -reporttypes:Html_Dark `
  -sourcedirs:./src/ `
  -historydir:./tests/coverage/history

we’ll get something like this printed out on our console screen

Arguments
 -reports:./tests/coverage/projects/*/*.xml
 -targetdir:./tests/coverage/report/
 -reporttypes:Html_Dark
 -sourcedirs:./src/
 -historydir:./tests/coverage/history
Reading historic reports
Parsing historic file './tests/coverage/history\2022-12-18_17-06-46_CoverageHistory.xml'
Parsing historic file './tests/coverage/history\2022-12-18_17-09-57_CoverageHistory.xml'
Parsing historic file './tests/coverage/history\2022-12-19_09-11-27_CoverageHistory.xml'
Parsing historic file './tests/coverage/history\2022-12-19_09-15-22_CoverageHistory.xml'
Parsing historic file './tests/coverage/history\2022-12-19_11-36-25_CoverageHistory.xml'
Parsing historic file './tests/coverage/history\2022-12-19_11-40-53_CoverageHistory.xml'
Writing report file './tests/coverage/report/index.html'
Creating history report
Report generation took 0.2 seconds

Now go to your ./tests/coverage/report/ folder and open the index.html file… that’s nice, ain’t it?

Now for the final trickery step. Even though the ReportGenerator tool has MSBuild integration it is a bit flawed. Targets in MSBuild are jobs, ran by MSBuild to do the build process. Every piece of logic running in the context of a build is a target. The thing is, targets are ran in the scope of a csproj that is why when testing. Every csproj is tested separately. So when running ReportGenerator as a MSBuild target in a solution with 3 test projects, it will run 3 times. Which could be problematic:

If we run the dotnet test with an /m:1 flag that will restrict MSBuild to work on a single process at once, the ReportGenerator tool will not fail. However, it will create 3 reports, meaning 2 history items, each after a single test run. This solution is fine if you have a single test project, it is a bit less practical for larger projects.
If we are not opting for the limiter flag — we will have fun with multiple processes accessing the same files concurrently, which will certainly fail the whole process.

How do we run it only once per solution?!

So, now it is time for some magic. The build process in MSBuild is actually creating a meta project that contains all the targets to build/test/etc the project. You could also drop a binlog of the build process

dotnet build -bl:binarylogfilename.binlog

and use Bin Log Reader (msbuildlog.com) to see this meta-project.

I’ll take a trick from their book. We can tie into the process by creating a file called after.<solution name>.sln.targets this file will be included in the end of the “virtual csproj” file, meaning it will happen only once. More information on that magic could be found in the MSBuild documentation.

So lets create such a file after.DemoSolution.sln.targets and fill it with some joy:

<Project>
 <Target Name="GenerateHtmlCoverageReport" AfterTargets="VSTest">
  <Exec Command="ReportGenerator -reports:./tests/coverage/projects/*/*.xml -targetdir:./tests/coverage/report/ -reporttypes:Html_Dark -sourcedirs:./src/ -historydir:./tests/coverage/history" />
 </Target>
</Project>

We are creating a new Target called GenerateHtmlCoverageReport (think of it as a function) that will run after the VSTest target. This target will use the Exec task to call the ReportGenerator global tool.

and Voilà! We have an automated process to augment your testing experience, with metrics, historic metrics, trends, hot-points for testing, code coverage drill-down. All cross-platform, all free and very fast.

Alternatively, if your projects already use Nuke , Cake or Fake, you’d be happy to know that ReportGenerator has integrations into those build systems.

Of course not everybody feels at home with such customizations to the build process. It’s ok to keep it simple and just create a small script to run the tests using dotnet test and then run the ReportGenerator tool… it’s probably better this wayany 😄

Summary

We’ve seen how to wire-up your solution with code coverage tooling (Coverlet) and reporting (ReportGenerator).

The next step is integrate your Github Actions with those deliveries and push them to CodeCov to get deep analysis of your coverage and code quality.