Generate a Beautiful Test Report from running Tests on Cypress.io

Hello everyone. Today I’m gonna show you on how to generate a beautiful test report from running tests on Cypress.io. As you may know Cypress is built on top of Mocha, so most of Mocha’s features can be used with Cypress i.e. Mocha’s bdd syntax, hooks or even reporting module.

By default, Mocha provides a lot of Reporters (more than 9 reporters) that you can choose one (or more) of them as your own test report. Configuring Mocha’s reporter on Cypress.io is so simple, just set the value of the reporter parameter in cypress.json file.

First of all, let me show you some built-in Mocha’s reporters and how to use it with Cypress.io

1. Spec

Spec reporter is a default reporter on Mocha. So if you don’t configure any reporter on your project, then you’ll see this one as your test report. The spec reporter displays test results on TTY with ANSI-color like this

Spec Reporter

To configure the Spec reporter on Cypress.io, configure the reporter parameter as spec

2. Dot Matrix

If you wanna see some progress while your tests are running, you can choose this one. Dot Matrix reporter is a series of Dot . characters which represent the progress of running tests. Failure of tests are highlighted in red exclamation marks ! , pending or skipped tests are highlighted in blue comma ,

To configure Dot Matrix reporter on Cypress, specify reporter parameter as dot

3. Nyan

If you think the two reporters above are too boring, what about this one? Nyan reporter might be the one that you’re waiting for. It’s a cute cat running on your TTY while your tests are running (=‐ω‐=)

Nyan Reporter

There are more built-in Mocha’s reporters that you can choose. Please find more information on this page https://mochajs.org/#reporters

Let’s create a Beautiful Mocha Reporter with Mochawesome

Mochawesome is a custom mocha reporter which generates an offline HTML/CSS report to visualize your test results. It displays your test case step-by-step and stack trace for failed tests. Moreover, it supports for adding your own custom context to the report. That means you can add screenshots to the failed tests for an example. It’s awesome like its name.

Step 1: Install Mochawesome

sh$ yarn add mocha@5.2.0 mochawesome mochawesome-merge

• mocha is used as a dependency library to create a custom reporter.
• mochawesome generates a beautiful report in HTML format.
• mochawesome-merge Since Cypress v3.0.0, it runs every spec separately, which leads to generating multiple mochawesome reports (one for each spec file) So we have to use this library to merge all of reports and then generate it as one HTML file.

Step 2: Configure Mochawesome as a Cypress Reporter

Configure reporter as mochawesome and you can also add some options for the reporter. The reporterOptions parameters are depends on the reporter that you’ve specified. Here are some options for the Mochawesome reporter.

• reportDir: To specify the output directory for your HTML report. You can place it anywhere as you want.
• overwrite: To overwrite existing report files. In this case since Cypress runs each test spec file separately, so I have to set this parameter to false to avoid replacing existing report files.
• html: To generate html report after the test (spec) runs. I plan to generate a single HTML report after ALL of my spec files finish. So I set this parameter to false
• json: To generate JSON report after the test (spec) runs. Yess I want this because I want these files (one per each spec) to merge all of my tests run into a single HTML report. So I set this parameter to true
• timestamp: To append timestamp in specified format to the report filename. I prefer this option rather than the default one (The default one is just a running number like 001, 002)

You can find more options for the mochawesome reporter here

3. Create your own Cypress Custom Test Runner

Normally, you can run tests on Cypress with a command cypress run for any simple use case. But in this case, I want to do something before and after all of my test runs. So I have to create my own Custom Test Runner using Cypress Module API

Install Dependencies for Custom Test Runner

sh$ yarn add yargs ls rimraf

• yargs: To parse the argument from the command line options. This is an optional.
• ls: To list all of existing report files.
• rimraf: To delete all of existing report files before test runs.

So the report is generated after ALL of the tests run via the generateReport function.

sh$ node cypress_runner -b chrome

Gotcha! This is the Mochawesome HTML report. You can see the detail of your test cases e.g. Test Code, Test Execution Time (for each test and for all tests) and Test Filtering menu on the left hand side. It’s pretty cool.

4. Attaching Screenshots to the Report for Failed Tests

Like I mentioned earlier, you can add your own custom context to the mochawesome reporter. So I’m gonna attach screenshots for the failed tests to my report.

Actually, Cypress automatically capture screenshot when a failure happens during test runs. So I just add screenshot files to the report and that’s it. I listen to the test:after:run event to attach screenshots to the report when failure happens via Cypress Support File

addContext is an Object provided by Mochawesome which allows you to add any custom context to the report. Not only a screenshot file but also anything you want like String, Object, etc. You can read more detail about this API here.

Then I update screenshotFolder parameter in cypress.json to place my screenshots to the Mochawesome assets directory. I want to make sure that my test report can reference to the screenshot files properly.

I create a new test case which always fail at run time. After that, I run the test again.

Finally, the test report is generated look like this.

The screenshot is attached to the report beautifully. Nice!

Woohoo. The screenshot is attached to the report! You can see my full test report here.

Lastly, you can apply this technique to your Continuous Integration system. It helps you to visualize test results in a beautiful format. Your team members can read and understand the test report clearly. I hope this article will be useful for your work. Happy Testing!

nottyo/cypress-e2e-code-coverage

Thanks to these cool lib: https://github.com/adamgruber/mochawesome, https://github.com/Antontelesh/mochawesome-merge