ReportPortal integration with Playwright
If the Playwright used on your project to write and run tests, the results can reach the ReportPortal via the agent-js-playwright. The agent is a custom Playwright reporter that sends test execution results to ReportPortal.
Let’s walk through the steps to set up integration.
You can also watch our video guide about ReportPortal integration with Playwright.
Installing the Agent
The first step involves installing the agent into the project. There is a small project with a few Playwright tests.
The tests are simple.
The agent package is available on NPM and GitHub registries.
Below, you can find the commands to install it using npm, yarn or pnpm.
After installation, it will appear in the list of dependencies. At the time of writing this article, the latest version is 5.1.8.
After the installation, it is required to provide some ReportPortal configuration data.
Configuration of the Agent
The next step involves configuring the agent and providing some basic options that will connect ReportPortal with the project.
First, create a Playwright config file if you don’t have one.
The basic configuration is presented below.
The ‘dotenv/config’ used to get access to the ENVIRONMENT variables specified in ‘.env’ file. It will be used to store the API key value.
To obtain the configuration options, the ReportPortal profile page can be used.
1. Open the ReportPortal instance (if you don’t have one the demonstration instance can be used) and log in. For a demo instance the GitHub login can be used.
2. Then navigate to the profile page. There, we can find a configuration example for Node.js.
3. Simply copy the JSON object and paste it into your project file assigning it to a variable, f.e. ‘rpConfig’.
Let’s name the launch as f.e. ‘Custоm regressiоn with Playwright’ right after that.
4. Provide an API key which can be generated on the API keys page in your profile.
Open the API Keys page and click on ‘Generate API Key’ button.
Specify the API Key name and click on ‘Generate’.
Detailed API Keys description can be found in the documentation.
After generation the API Key can be copied to use it in the code.
Paste it as a value for REPORTPORTAL_API_KEY in ‘.env’ file and update ‘rpConfig’.
5. Instruct Playwright to use the agent as a reporter.
NOTE:
The full list of options with their description can be found in the agent’s repository.
Running the Tests
After completing the two steps above, run tests as usual.
1. Run “npm run test” command.
2. Monitor the test’s progress in ReportPortal. The state can be refreshed to obtain the latest results.
Let’s check the embedded reporter of Playwright. The statistics here match exactly.
Update tests with additional data
You can enrich your Playwright test report by adding additional data. This includes description, attributes, logs, attachments, TestCaseID and status.
Description and attributes can be used to specify any extra information about the test and can also be used for filtering and building some widgets, like “Component Health Check”. Extra logs can also be provided together with attachments. The TestCaseID can be used as a history ID across different launches. There’s also the option to change the status in ReportPortal for a particular test result.
How to add additional data to the tests?
1. To provide attributes and a description, import the “ReportingApi” from the agent package and use it within tests.
After running the tests again, the specified data will be visible in ReportPortal in the newly appeared launch. We just need to drill down into it to see the results.
2. To add logs with specific log levels, use the Reporting API methods or report as is if you’re writing logs to the standard output.
Let’s run our tests again and check the new launch in the system. To see the logs for particular test result, we can drill down to its latest level.
The error logs thrown by the failed tests are reported by default on the test end.
3. To insert attachments, call the “attach” method on the “testInfo” object provided by Playwright.
After launch appearing in ReportPortal, the attachments can be found on the same place as logs.
4. The TestCaseID is a unique test case identifier across all launches in ReportPortal. It is generated automatically based on code reference (physical location of the test within the project) and parameters (if exists). But the real test case ID from your test case management system can be used instead. To provide TestCaseID explicitly, call the “setTestCaseId” method of “ ReportingApi” and provide a value.
Applied TestCaseID can be checked via Test item details modal window.
NOTE:
Once the TestCaseID changed for the test result, it will be unlinked from its previous executions from historical data perspective.
History of this test result within previous launches, where the TestCaseID was persistent between runs.
5. In case for some reason necessary to set the specific status for test result in ReportPortal only, the “ReportingApi” “setStatus” methods can be used.
Now in ReportPortal we can check that test that actually failed marked as ‘passed’ in the system.
NOTE:
Detailed information about the additional data and methods for adding it can be found in the agent’s repository within the Readme instructions.
By this step-by-step process, you can fully integrate your Playwright automation with ReportPortal. Once set up, you can view test results in real-time, which is particularly useful for lengthy and extensive test suites. Moreover, ReportPortal can analyze your data using Machine Learning algorithms. If you’re looking to expedite the process, you can delegate it to the ReportPortal Analyzer to quickly identify the root causes of defects.
ReportPortal also provides a comprehensive visualization of your automation process, allowing you to generate charts, track test automation metrics, and keep an eye on your project’s stability with our test reporting dashboard.
Bring your Playwright test reporting to the next level!
Useful links:
· Other available integrations
Learn more about ReportPortal:
