How to generate automated test reports using Postman

Alee
Alee
Jan 31 · 4 min read

Postman is a tool for API development. Postman’s features simplify each step of building an API and testing it. It offers possibility to automate manual tests and integrate them into your CI/CD pipeline to ensure that any code changes won’t introduce regressions in Production.

Super Postman to the rescue!

Requests in Postman are organised in collections and folders. Postman contains a collection runner that is useful when you want to automate API testing. When you run a collection, you send all requests in your collection one after another.

It’s a powerful tool when developing automated tests, it helps visualize details of each iteration and test results. Still, it presents limitations such as the inability to share test results, or occasional crashes linked to memory issues.

To remediate those issues, we are going to use the CLI tool of Postman called newman and a custom reporter that generates detailed reports.

Test execution report overview

There is a powerful tool called htmlextrareport that generates beautiful reports. This makes it possible to have an overview of all test runs, and share Postman API test reports that the stakeholders can understand through the description feature available for collections, folders and requests. This gives also the possibility to fill in gaps with new test cases after business review to have a complete functional coverage.

htmlextrareport example
htmlextrareport example
HTMLExtraReport example

How to wire things up

Install node

Download and install the LTS version of Node.js.

Install newman

Newman is a command line Collection Runner for Postman. It allows you to run and test a Postman collection directly from the command line. It is built with extensibility in mind so that you can easily integrate it with your continuous integration servers and build systems. You can find more details in Postman’s official documentation.

Once Node.js is installed, the node package manager (NPM) is installed as well and you can run the following command to install newman globally so you can run it from anywhere. Now open the terminal and run:

npm install -g newman

Install custom reporter

Newman supports plenty of reporters by default, but none of them gives the level of details of htmlextrareport. Install it globally using the command:

npm install -g newman-reporter-htmlextra

Build Postman collection

For the sake of simplicity of this how-to, we will use the Postman Echo collection, created for test purposes by the Postman team. It features a full list of functionalities supported by Postman. Among them, different types of requests, API documentation and many tests with different assertion syntaxes. If it is your first time playing with Postman collection, it is a good entry point.

Click on the “Run in Postman” button to import the collection locally.

Run the Postman collection in collection runner:

Run collection in the Collection Runner

Prepare collection to run from CLI

To run the collection from the command line, you need to make it accessible. There are two methods to do that:

  1. Export the collection as a JSON file
  2. Generate a shareable public link of the collection

To avoid exporting collections every time you change them, or in automated test development phases, I recommend using the shareable link and updating it each time the collection is modified. Use the “Update Link” button for that.

Run collection in CLI

newman run https://www.getpostman.com/collections/161a23965791d263a3e6 -r htmlextra --reporter-htmlextra-title "Automated test reporting - Postman echo" 

Running this command creates a folder in the working directory with the name “newman”, where you’ll find the generated report.

Report in html

In the generated report, you can find the description (with Markdown support) of the collection and of each request. It also contains console logs, and the request and response payloads of each test.

Description section of the request

There is also global information about the request, response and test pass information.

Global overview information about the request

As well as the headers and payloads for the requests and the responses.

Example of request payload
Example of response payload

It’s much more advantageous when payloads are overloaded dynamically from a data source, as you avoid redundancy and keep things easy to maintain. A post on this matter will be coming soon. Stay tuned.

For more details about this custom reporter, refer to its official repository. Especially its parameters list that offers customized naming, title, theme, logging and other useful parameters.

In conclusion, this provides a better way to implicate business on enhancing the testing process and simplify the process of adding automatic testing into CI/CD pipeline.

Special thanks to Thomas Goldstein for his time to help me finetune this blog post.

YounitedTech

Le blog Tech de Younited, où l’on parle de développement, d’architecture, de microservices, de cloud, de data… Et de comment on s’organise pour faire tout ça. Ah, et on recrute aussi, on vous a dit ?

Alee

Written by

Alee

QA @ Younited Credit

YounitedTech

Le blog Tech de Younited, où l’on parle de développement, d’architecture, de microservices, de cloud, de data… Et de comment on s’organise pour faire tout ça. Ah, et on recrute aussi, on vous a dit ?

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade