How to generate automated test reports using Postman
Postman is a tool for API development. Postman’s features simplify each step of building an API and testing it. It offers possibility to automate manual tests and integrate them into your CI/CD pipeline to ensure that any code changes won’t introduce regressions in Production.
Requests in Postman are organised in collections and folders. Postman contains a collection runner that is useful when you want to automate API testing. When you run a collection, you send all requests in your collection one after another.
It’s a powerful tool when developing automated tests, it helps visualize details of each iteration and test results. Still, it presents limitations such as the inability to share test results, or occasional crashes linked to memory issues.
To remediate those issues, we are going to use the CLI tool of Postman called newman and a custom reporter that generates detailed reports.
Test execution report overview
There is a powerful tool called htmlextrareport that generates beautiful reports. This makes it possible to have an overview of all test runs, and share Postman API test reports that the stakeholders can understand through the description feature available for collections, folders and requests. This gives also the possibility to fill in gaps with new test cases after business review to have a complete functional coverage.
How to wire things up
Install node
Download and install the LTS version of Node.js.
Install newman
Newman is a command line Collection Runner for Postman. It allows you to run and test a Postman collection directly from the command line. It is built with extensibility in mind so that you can easily integrate it with your continuous integration servers and build systems. You can find more details in Postman’s official documentation.
Once Node.js is installed, the node package manager (NPM) is installed as well and you can run the following command to install newman globally so you can run it from anywhere. Now open the terminal and run:
npm install -g newmanInstall custom reporter
Newman supports plenty of reporters by default, but none of them gives the level of details of htmlextrareport. Install it globally using the command:
npm install -g newman-reporter-htmlextraBuild Postman collection
For the sake of simplicity of this how-to, we will use the Postman Echo collection, created for test purposes by the Postman team. It features a full list of functionalities supported by Postman. Among them, different types of requests, API documentation and many tests with different assertion syntaxes. If it is your first time playing with Postman collection, it is a good entry point.
Click on the “Run in Postman” button to import the collection locally.
Run the Postman collection in collection runner:
Prepare collection to run from CLI
To run the collection from the command line, you need to make it accessible. There are two methods to do that:
To avoid exporting collections every time you change them, or in automated test development phases, I recommend using the shareable link and updating it each time the collection is modified. Use the “Update Link” button for that.
Run collection in CLI
newman run https://www.getpostman.com/collections/161a23965791d263a3e6 -r htmlextra --reporter-htmlextra-title "Automated test reporting - Postman echo" Running this command creates a folder in the working directory with the name “newman”, where you’ll find the generated report.
Report in html
In the generated report, you can find the description (with Markdown support) of the collection and of each request. It also contains console logs, and the request and response payloads of each test.
There is also global information about the request, response and test pass information.
As well as the headers and payloads for the requests and the responses.
It’s much more advantageous when payloads are overloaded dynamically from a data source, as you avoid redundancy and keep things easy to maintain. A post on this matter will be coming soon. Stay tuned.
For more details about this custom reporter, refer to its official repository. Especially its parameters list that offers customized naming, title, theme, logging and other useful parameters.
In conclusion, this provides a better way to implicate business on enhancing the testing process and simplify the process of adding automatic testing into CI/CD pipeline.
Special thanks to Thomas Goldstein for his time to help me finetune this blog post.
