Load Testing with Postman and Grafana K6

Leveraging Postman and Grafana K6 for quick and effective load testing and a step-by-step tutorial on getting started.

Introduction

Ensuring the performance and reliability of our software applications under load, expected or unexpected, is crucial; that’s where Postman and Grafana K6 play a vital role. With Postman, we can effortlessly leverage existing test collections to create comprehensive testing scenarios. And K6, an immensely powerful load testing tool, which can be used to simulate hundreds of user requests concurrently whilst providing data visualisation and monitoring capabilities. Together, Postman and K6 provide efficient testing, robust load simulation, and extensive performance monitoring features, all of which contribute to addressing software performance and reliability.

Postman (Newman)

Newman is a command-line tool for running and automating API requests created in Postman.

Its user-friendly interface simplifies integration into CI/CD pipelines, making it straightforward to incorporate testing into any development workflow with ease.

By utilising Newman’s intuitive CLI, we can easily define requests, configure parameters, and obtain valuable insights into the performance of APIs or web services.

An example of a simple API GET, with verbose output and metrics for each request.

Postman and Newman’s simplicity and ease of use make it an invaluable addition to any testing arsenal, streamlining development processes and assisting in ensuring the reliability of applications.

Grafana K6

With its strong data visualisation capabilities, K6 thoroughly analyses data collected through its load tests and can even generate real-time dashboards and reports in tools such as Grafana Dashboards and Datadog.

At the core of the open-source framework lies its impressive command-line interface (CLI), which enables us to effortlessly define test scenarios, configure and execute tests with different load patterns and collect crucial performance metrics.

An example of using Postman and K6 combined for executing and simulating a load of 3 users.

With the CLI, we can easily tailor our load tests to specific use cases, ensuring accurate and comprehensive performance analysis. Moreover, like Postman, K6’s CLI can integrate into existing workflows, allowing for efficient test execution and streamlined result analysis.

The Dynamic Duo

K6 only enhances and synergises with Postman, and when it comes to load testing the two make a powerful duo. By combining them, we can leverage Postman to define API tests, while K6 provides the ability to execute scenarios with multiple virtual users and differing load patterns to collect performance metrics.

This seamless collaboration between Postman and K6 allows us to perform efficient and effective load testing to identify and address performance bottlenecks in applications, and optimise our applications to help deliver smooth user experiences.

Load Testing with Postman

This short guide aims to get you started on defining tests in Postman and running load tests with the Newman collection runner.

For the sake of simplicity, requests to endpoints or web pages in this guide will be limited to GET requests and not involve authentication.

Prerequisites

• Postman GUI
• Newman

This guide will also require an API with endpoints; we will be using K6’s test API located at test-api.k6.io.

Define Your Collections

With Postman installed and open, create a new collection for your tests to reside:

Let’s set a BASEURL for our collection under ‘Variables’ in the newly created collection. We can set this to the base API path, https://test-api.k6.io:

Define your Requests

Our collection is now ready for requests to be defined.

Let’s start with a simple GET request to fetch all public crocodiles from the K6 API. The following endpoint fetches all crocodiles:

GET {{BASEURL}}/public/crocodiles

The response should contain a list of crocodiles and their properties.

Postman Test Scripts

Postman enables fine-grained control over test execution flow.

Suppose we want the test to request another endpoint once it has received a 200 response from Get All Crocs. We can do that under the ‘Tests’ scripts tab:

Here’s how we can configure a subsequent request to GET a specific crocodile after successfully receiving all crocodiles:

if (pm.response.code == 200) {
    postman.setNextRequest("Get Specific Croc");
}
else {
    postman.setNextRequest(null);
}

We can then set up another request with the same name as the one in the above condition, Get Specific Croc , in our collection:

GET {{BASEURL}}/public/crocodiles/1

Our collection is now complete, with two GET requests and test scripts ready to be used with Newman.

Using the Newman Collection Runner

With the collection now set up, we can start using the Newman CLI to run our tests.

We first need to export the collection to JSON. You can do this by exporting your collection through Postman:

You should now have a JSON collection file, defining the collection and its tests.

To run this collection with Newman, you can use the following command:

$ newman run <yourPostmanCollection.json> -n 2

The -n flag indicates how many iterations should be executed.

Be sure to replace <yourPostmanCollection.json> with your JSON collection.

The results from the test comprises of useful information such as:

• Number of successful/unsuccessful iteration executions
• Average response times
• Amount of data received

Example Newman test run output with 2 iterations.

For verbose logging, use the --verbose flag for per-iteration statistics and metrics.

The Key Limitation

Whilst Postman with Newman is a quick and simple way of getting started with load testing, it is essential to understand that the tests are executed sequentially and not in parallel.

A real-world scenario differs from this, with many users accessing web pages and requesting API endpoints at a time, a more realistic load testing approach can be carried out with K6.

Load Testing with Grafana K6

Luckily, we can convert our exported Postman collections to K6-compatible tests. The tests can then be executed using the K6 CLI with multiple virtual users (VUs) in parallel, more accurately simulating real-world web and API traffic scenarios.

Prerequisites

• K6
• postman-to-K6

Convert Postman Collection to K6 Script

A simple command using the postman-to-k6 utility can convert your Postman collection to a K6 script.

$ postman-to-k6 <yourPostmanCollection.json> -o k6-script.js

This will generate a k6-script.js file in the root of your project.

Note: If postman-to-K6 is not installed globally, the converter will need to be run with npx @apideck/postman-to-k6 yourPostmanCollection.json or, if you are using an older versions of npm, ./node_modules/.bin/postman-to-k6 yourPostmanCollection.json.

Adjusting k6-script.js

Before running with the K6 CLI, a slight adjustment is required in the newly generated file.

Since K6 does not support postman.setNextRequest(), the subsequent request function after the initial request must be manually added to the condition.

The main script function would go from looking like this:

export default function() {
  postman[Request]({
    name: "Get All Crocs",
    id: "0e599923-2cad-423d-bbd3-d4b2fd9d3dcb",
    method: "GET",
    address: "{{BASEURL}}/public/crocodiles",
    post(response) {
      if (pm.response.code == 200) {
        postman.setNextRequest("Get Specific Croc"); // Remove
      } else {
        postman.setNextRequest(null); // Remove
      }
    },
  });

  // Move this request into above condition
  postman[Request]({
    name: "Get Specific Croc",
    id: "640cb716-34e7-4999-bc91-d921d0419f7e",
    method: "GET",
    address: "{{BASEURL}}/public/crocodiles/1",
  });
  // 
}

To this:

export default function() {
  postman[Request]({
    name: "Get All Crocs",
    id: "0e599923-2cad-423d-bbd3-d4b2fd9d3dcb",
    method: "GET",
    address: "{{BASEURL}}/public/crocodiles",
    post(response) {
      if (pm.response.code == 200) {
        postman[Request]({
          name: "Get Specific Croc",
          id: "640cb716-34e7-4999-bc91-d921d0419f7e",
          method: "GET",
          address: "{{BASEURL}}/public/crocodiles/1",
        });
      } else {
        return;
      }
    },
  });
}

Running in the K6 CLI

The tests are now ready to be executed in the K6 CLI. The following command runs the tests with 5 virtual users (-vus 5) and for a duration of 5 seconds (--duration 5s).

$ k6 run k6-script.js --vus 5 --duration 5s

K6 provides extremely useful metrics upon test completion. Some of these include:

• Data sent/received
• Detailed http request/response metrics (duration, receiving, sending, etc)
• Iterations metrics

An example output is shown below:

          /\      |‾‾| /‾‾/   /‾‾/   
     /\  /  \     |  |/  /   /  /    
    /  \/    \    |     (   /   ‾‾\  
   /          \   |  |\  \ |  (‾)  | 
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: k6-script.js
     output: -

  scenarios: (100.00%) 1 scenario, 5 max VUs, 35s max duration (incl. graceful stop):
           * default: 5 looping VUs for 5s (gracefulStop: 30s)


     data_received..................: 116 kB 22 kB/s
     data_sent......................: 45 kB  8.6 kB/s
     http_req_blocked...............: avg=5.94ms   min=0s       med=4µs      max=263.02ms p(90)=12.1µs   p(95)=16.04µs 
     http_req_connecting............: avg=2.27ms   min=0s       med=0s       max=101.18ms p(90)=0s       p(95)=0s      
     http_req_duration..............: avg=109.69ms min=97.02ms  med=108.92ms max=356.73ms p(90)=117.94ms p(95)=119.68ms
       { expected_response:true }...: avg=109.69ms min=97.02ms  med=108.92ms max=356.73ms p(90)=117.94ms p(95)=119.68ms
     http_req_failed................: 0.00%  ✓ 0         ✗ 220
     http_req_receiving.............: avg=81.26µs  min=14µs     med=65.5µs   max=316µs    p(90)=177µs    p(95)=227.19µs
     http_req_sending...............: avg=30.9µs   min=4µs      med=22µs     max=253µs    p(90)=63µs     p(95)=76.05µs 
     http_req_tls_handshaking.......: avg=2.77ms   min=0s       med=0s       max=125.36ms p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=109.58ms min=96.88ms  med=108.85ms max=356.45ms p(90)=117.86ms p(95)=119.63ms
     http_reqs......................: 220    42.15035/s
     iteration_duration.............: avg=464.08ms min=415.76ms med=438.15ms max=701.24ms p(90)=588.11ms p(95)=699.11ms
     iterations.....................: 55     10.537587/s
     vus............................: 5      min=5       max=5
     vus_max........................: 5      min=5       max=5


running (05.2s), 0/5 VUs, 55 complete and 0 interrupted iterations
default ✓ [======================================] 5 VUs  5s

Verbose logging with --verbose is also an option that is available for detailed metrics and logs.

It is important to note that virtual users have the goal of executing test script code as fast as possible and may not accurately mimic real user behaviour, who would take time absorbing page information between each request. Therefore it is recommended that sleep statements are used to space out requests made by each individual virtual user.

Conclusion

In this article we explored how we can use Postman, Newman and K6 to get started with load testing API endpoints or web pages in just a few minutes.

Both of these tools can be seamlessly integrated into existing CI/CD pipelines quickly enabling performance testing and monitoring as part of development workflows. The metrics and results from tests can be used to create application insights dashboards using tools such as Grafana Dashboards or Datadog.

Both Postman (Newman) and K6 test scripts are highly customisable; I thoroughly recommend diving deeper and understanding the more complex and sophisticated capabilities of both these tools. You can find out more about Postman here, and K6 here.