Generate a Beautiful Test Report from running Tests on Cypress.io
Hello everyone. Today I’m gonna show you on how to generate a beautiful test report from running tests on Cypress.io. As you may know Cypress is built on top of Mocha, so most of Mocha’s features can be used with Cypress i.e. Mocha’s bdd syntax, hooks or even reporting module.
By default, Mocha provides a lot of Reporters (more than 9 reporters) that you can choose one (or more) of them as your own test report. Configuring Mocha’s reporter on Cypress.io is so simple, just set the value of the reporter
parameter in cypress.json
file.
First of all, let me show you some built-in Mocha’s reporters and how to use it with Cypress.io
1. Spec
Spec reporter is a default reporter on Mocha. So if you don’t configure any reporter on your project, then you’ll see this one as your test report. The spec reporter displays test results on TTY with ANSI-color like this
To configure the Spec
reporter on Cypress.io, configure the reporter
parameter as spec
2. Dot Matrix
If you wanna see some progress while your tests are running, you can choose this one. Dot Matrix
reporter is a series of Dot .
characters which represent the progress of running tests. Failure of tests are highlighted in red exclamation marks !
, pending or skipped tests are highlighted in blue comma ,
To configure Dot Matrix reporter on Cypress, specify reporter
parameter as dot
3. Nyan
If you think the two reporters above are too boring, what about this one? Nyan
reporter might be the one that you’re waiting for. It’s a cute cat running on your TTY while your tests are running (=‐ω‐=)
There are more built-in Mocha’s reporters that you can choose. Please find more information on this page https://mochajs.org/#reporters
Let’s create a Beautiful Mocha Reporter with Mochawesome
Mochawesome is a custom mocha reporter which generates an offline HTML/CSS report to visualize your test results. It displays your test case step-by-step and stack trace for failed tests. Moreover, it supports for adding your own custom context
to the report. That means you can add screenshots to the failed tests for an example. It’s awesome like its name.
Step 1: Install Mochawesome
sh$ yarn add mocha@5.2.0 mochawesome mochawesome-merge
mocha
is used as a dependency library to create a custom reporter.mochawesome
generates a beautiful report in HTML format.mochawesome-merge
Since Cypressv3.0.0
, it runs every spec separately, which leads to generating multiple mochawesome reports (one for each spec file) So we have to use this library to merge all of reports and then generate it as one HTML file.
Step 2: Configure Mochawesome as a Cypress Reporter
Configure reporter
as mochawesome
and you can also add some options for the reporter. The reporterOptions
parameters are depends on the reporter that you’ve specified. Here are some options for the Mochawesome reporter.
reportDir
: To specify the output directory for your HTML report. You can place it anywhere as you want.overwrite
: To overwrite existing report files. In this case since Cypress runs each test spec file separately, so I have to set this parameter tofalse
to avoid replacing existing report files.html
: To generate html report after the test (spec) runs. I plan to generate a single HTML report after ALL of my spec files finish. So I set this parameter tofalse
json
: To generate JSON report after the test (spec) runs. Yess I want this because I want these files (one per each spec) to merge all of my tests run into a single HTML report. So I set this parameter totrue
timestamp
: To append timestamp in specified format to the report filename. I prefer this option rather than the default one (The default one is just a running number like001
,002
)
You can find more options for the mochawesome reporter here
3. Create your own Cypress Custom Test Runner
Normally, you can run tests on Cypress with a command cypress run
for any simple use case. But in this case, I want to do something before and after all of my test runs. So I have to create my own Custom Test Runner using Cypress Module API
Install Dependencies for Custom Test Runner
sh$ yarn add yargs ls rimraf
yargs
: To parse the argument from the command line options. This is an optional.ls
: To list all of existing report files.rimraf
: To delete all of existing report files before test runs.
So the report is generated after ALL of the tests run via the generateReport
function.
sh$ node cypress_runner -b chrome
Gotcha! This is the Mochawesome HTML report. You can see the detail of your test cases e.g. Test Code, Test Execution Time (for each test and for all tests) and Test Filtering menu on the left hand side. It’s pretty cool.
4. Attaching Screenshots to the Report for Failed Tests
Like I mentioned earlier, you can add your own custom context
to the mochawesome reporter. So I’m gonna attach screenshots for the failed tests to my report.
Actually, Cypress automatically capture screenshot when a failure happens during test runs. So I just add screenshot files to the report and that’s it. I listen to the test:after:run
event to attach screenshots to the report when failure happens via Cypress Support File
addContext
is an Object provided by Mochawesome which allows you to add any custom context to the report. Not only a screenshot file but also anything you want like String, Object, etc. You can read more detail about this API here.
Then I update screenshotFolder
parameter in cypress.json
to place my screenshots to the Mochawesome assets
directory. I want to make sure that my test report can reference to the screenshot files properly.
I create a new test case which always fail at run time. After that, I run the test again.
Finally, the test report is generated look like this.
Woohoo. The screenshot is attached to the report! You can see my full test report here.
Lastly, you can apply this technique to your Continuous Integration system. It helps you to visualize test results in a beautiful format. Your team members can read and understand the test report clearly. I hope this article will be useful for your work. Happy Testing!
Thanks to these cool lib: https://github.com/adamgruber/mochawesome, https://github.com/Antontelesh/mochawesome-merge