Integrate automated test in Azure DevOps using the Postman API

Alee
Alee
Mar 3 · 5 min read

Following my previous post about generating reports using Postman. Here I am again, writing about Postman and its amazing tips to help deliver working software more efficiently. Welcome to this step-by-step guide to demonstrate integration of Postman automated tests in a Microsoft Azure Pipeline using the Postman API to #MakeItSimple

Image for post
Image for post
Continuous Integration flow

The biggest advantage of using the Postman API is the automatic synchronization of the Collections edited in the Postman GUI with the Postman Cloud (which is completely free). This avoids using Source Code Management tools (Git, SVN, etc.) to host your collections. Nevertheless, this is not the only option. You can also use the public shareable link of your Postman collection, but this may present a security issue in your organization.

Beside that, exporting a collection in JSON, pushing it to a repo and then retrieving it in your CI pipeline is a fastidious way of automating tests. Especially for a lazy QA engineer like me. Furthermore, at Younited Credit we believe that #FasterIsBetter

Retrieve your collection using the Postman API

In order to be more efficient and benefit from the automatic synchronisation of the Postman collections, we are going to use the Postman API. First of all, go to the Postman API and click the “Run in Postman” button:

Image for post
Image for post

Follow the instructions to import the Postman API collection in your Postman desktop client, then click on your profile to generate a Postman API key.

Image for post
Image for post
Setting menu in Postman desktop GUI

Then in the Postman Cloud GUI, click on the “Account Settings” button:

Image for post
Image for post

And generate an API key:

Image for post
Image for post
Steps to generate Postman API key

Go back to Postman and add the generated key value in the header of the All collections request (1), then find the collection you want to integrate in your pipeline and copy its uid property (2).

🔴 Note that with the same Api-Key you can get any of your environments (Staging, Pre-production…). For that, use the All environments request.

Image for post
Image for post
Steps to retrieve UID of the collection

This helps you get your collection/environment using a cURL command that looks like this:

This command downloads your collection to your local machine and puts it in a file named myCollection.json.

🔴 cURL is a command line interface to transfer data. It is natively supported in all Mac OS X and Linux systems, and lately Microsoft added it to Windows 10. You will need to install it manually if you are using Windows 7 or earlier versions.

Run your collection in the Azure Pipeline

In what follows, I assume that you already have access to Azure Devops and have the appropriate access rights to create and run a Pipeline in your organization. If not, it might be the time to create a DevOps ticket for that :)

Running a Postman collection is pretty simple, you will need newman. My previous post may help you find the references to perform this locally. But in Azure Devops, the steps are as follows:

1- Create an npm task to install newman

Example of npm task configuration
Example of npm task configuration
Configuration example to install newman in npm task

2. Create an npm task to install newman-reporter-junitfull

Create an npm task as in the previous step, except with a different npm package name. Instead of “newman”, install “newman-reporter-junitfull”.

This package helps your newman generate correct JUnit reports when you have multiple iterations in your tests, which is generally the case. From what I understood, the default JUnit reporter is not yet mature enough to handle this issue.

The JUnit reports are useful to publish results in Azure DevOps. We’ll see that later, be patient. 🤗

3. Download the Postman collection and run it

Add a command line task. In the script section, add the cURL command you built in the first section of this post.

Image for post
Image for post
Command line task example to download and run the Postman collection

4. Publish test results in Azure DevOps

Add a Publish Test Results task and configure it as follows:

Image for post
Image for post
Configuration example of the Publish test results task

5. Save and create a release in your pipeline

Now that you’ve configured all the steps, save your configuration and create a release in your pipeline. Once finished, click the “Tests” tab to visualize the test results in Azure DevOps style.

Image for post
Image for post
Test summary in Azure DevOps

Note that Azure DevOps has a limitation to link an automated test to a written test in the Test Plans just like TestLink, HP ALM and other test management tools. I’ve opened a feature suggestion here 🆙. Feel free to upvote it! If Microsoft ever solves this issue, I will blog about it.

Regardless of the CI tool you are using (Azure pipeline, Jenkins, Travis CI, CircleCI, Hudson or any other one), the steps above remain the same. The only thing that changes is the configuration of the tool before creating and running the job.

Thanks to Thomas Goldstein for helping me finetune this blog post.

YounitedTech

Le blog Tech de Younited, où l’on parle de développement…

Alee

Written by

Alee

QA Engineer @ Younited Credit

YounitedTech

Le blog Tech de Younited, où l’on parle de développement, d’architecture, de microservices, de cloud, de data… Et de comment on s’organise pour faire tout ça. Ah, et on recrute aussi, on vous a dit ?

Alee

Written by

Alee

QA Engineer @ Younited Credit

YounitedTech

Le blog Tech de Younited, où l’on parle de développement, d’architecture, de microservices, de cloud, de data… Et de comment on s’organise pour faire tout ça. Ah, et on recrute aussi, on vous a dit ?

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store