Embracing Native V8 Coverage Reports in 2024

Background and Current Situation

As early as 2017, V8 started offering native support for coverage data, as seen here However, we have essentially only been able to see coverage reports from Istanbul, and this is because most testing tools convert native V8 coverage data into the format of Istanbul, such as Jest, Vitest. This has led to several issues:

• Native Byte coverage statistics are not available.
• Coverage support for runtime code, especially minified code, is not available.
• CSS coverage reports are not available.
• The commonly used conversion tool is v8-to-istanbul, but it has a few problems here

Introducing Native V8 Coverage Reports

Nowadays, native V8 coverage reports are possible, and you can directly use V8 coverage data to generate V8 coverage reports. This can be achieved using the tool monocart-coverage-reports. More details can be found at the GitHub repository

A Brand-New V8 Coverage Report User Interface

• It provides a superior user interaction experience.
• It offers file tree grouping and supports search and filter functions.
• Supports an interface that lets you view and quickly locate uncovered code blocks.
• It provides the positioning of the code with the highest execution number, making it easier for you to identify the most frequently executed code (which may be crucial areas affecting performance).
• It provides markers for blocks of code run multiple times, as well as additional statistics for blank lines and comment lines.

Support for Native Byte Statistics

Byte coverage is natively supported by V8, displaying how many bytes of a file are covered. This is an excellent measure of the completeness of coverage.

Coverage for Any Runtime Code

Since the runtime code is most likely to be compressed and minified, ‘monocart-coverage-reports’ provides a formatter function, allowing easy access to coverage of formatted code. An asynchronous worker provides support for large volume files, which also includes syntax highlighting.

CSS Coverage Support

In today’s front-end projects, CSS code is crucial, making checking CSS coverage quite important. This makes it easier to find and remove useless CSS code.

Attempting to Solve the Conversion Problems from v8-to-istanbul

More details can be found here

How to Collect Native V8 Coverage Data

• On the browser side (Chromium only) Use the Chromium Coverage API to collect coverage data. For example, Playwright Coverage Class

await Promise.all([
    page.coverage.startJSCoverage({
        resetOnNavigation: false
    }),
    page.coverage.startCSSCoverage({
        resetOnNavigation: false
    })
]);
await page.goto(url);
const [jsCoverage, cssCoverage] = await Promise.all([
    page.coverage.stopJSCoverage(),
    page.coverage.stopCSSCoverage()
]);

const coverageData = [... jsCoverage, ... cssCoverage];

• Node.js The following methods can be used:

1. Using env NODE_V8_COVERAGE=dir, see tool c8
2. Using env NODE_V8_COVERAGE=dir and manually saving v8 with interface v8.takeCoverage() on demand, see example
3. Using Inspector API, see tool collect-v8-coverage
4. Using CDP Coverage API, see example

For more detailed introductions please visit this link

How to Generate Native V8 Coverage Reports

First, create a new coverage report coverageReport. Next, add the collected V8 coverage data such as coverageData1 and coverageData2 to coverageReport. Finally, execute generate() to generate the report. The coverage data of the same file will automatically combine together.

const MCR = require('monocart-coverage-reports');
const options = {
    outputDir: './coverage-reports',
    reports: "v8"
}
const coverageReport = MCR(options);
await coverageReport.add(coverageData1);
await coverageReport.add(coverageData2);
const coverageResults = await coverageReport.generate();
console.log(coverageResults.summary);

Other

• In fact, monocart-coverage-reports also fully supports all Istanbul coverage reports and integrates it with V8 reports. The report formats include:v8, v8-json,console-summary, clover,cobertura,html,html-spa,json,json-summary,lcov,lcovonly,none,teamcity,text,text-lcov,text-summary. see Available reports
• An onEnd hook is also provided to enable users for custom programming, like checking whether the coverage has reached the required thresholds, see example
• If you use @playwright/test as your automation test tool, you can directly use the monocart-reporter custom report as it has already integrated native V8 coverage report features. More details can be found here.

Originally published at http://github.com.