Introduction to K6: Load Testing Your APIs

I am a software developer for quite a while now and I found that one aspect that is often neglected is load testing. As developers, we are usually first and foremost interested to “make it work”. Sure, we will come up with all the well-known design patterns that have proved themselves efficient countless times. We think that everything will work just fine and most often than not, it does. However, you should always know how your application will behave under stress. And that is why you need Load and/or stress testing. I witnessed this situation where a performance expert comes into play and run some one-time Jmeter scripts on your application to evaluate whether or not it is performing well.

Well, In a cloud world, let me tell you that shouldn’t fly anymore. You need self-sufficient teams that can take care of every aspect of the application. Needless to say that the performance aspect should be considered from the beginning. Anyone wants to use applications that are fastly responding, no matter the load it is under. And who better than the development team to take care of this. They know the application best, it’s just so hard to find an attractive enough tool to get a developer engaged. That’s where K6 revealed himself as a great option:

• Open-source
• Runs on a local environment
• Everything as code
• Automation
• Results visualization
• CLI tool

Scenario

To illustrate a load testing scenario, I will reuse the chatbot project from my previous post. It is the most simple chatbot you can make with DialogFlow and we will enhance it just a bit to come up with an example that will cover a conversation with the chatbot.

Go to your chatbot and add the following 2 intents:

• ask-joke

• reaction

With these 2 new intents, we can now build our scenario that will consist of a conversation when the users say:

• “Hi”
• “Tell me a joke”
• “Good one”

Each time, the bot should answer the corresponding intent. We are going to load test this scenario as if 60 users were running the same scenario at the same time.

Build up the scenario in Postman

K6 is offering a tool to generate a script from a Postman collection. This is how I decided to start even if you could build your script yourself from scratch.

In Postman, you need to create each call individually. Remember to use the “COPY CURL” from the trying area to get the necessary information:

• URL
• body request
• bearer token

You should also add tests to verify the response you are receiving (Beware that the second test differs according to the intent you are expecting):

pm.test("Status code is 200", function () {
pm.response.to.have.status(200);
});
pm.test("intent is Default Welcome Intent", function () {
JSON.parse(responseBody).queryResult.intent.displayName === "Default Welcome Intent";
});

Once you have your 3 calls ready, you can export your collection. We will use the JSON as input later on.

Start with K6

First of all, you should install K6. Then, because we want to generate the script from our previously built collection, you will need postman-to-k6.

Clone the repository and from there, run the npm or yarn command indicated in the documentation.

You will then be able to run the conversion script by giving the exported JSON from Postman and specifying an output script file:

postman-to-k6 ../K6.postman_collection.json  -o ./k6-DF-script.js

You should now be able to run the generated script (if you encounter an error at this step, it might be that your token expired. You can replace it in the script):

k6 run k6-DF-script.js

This is a fine unit test that ran with one virtual user. But we need to simulate 60 users running the same scenario at the same time. While you can pass the arguments to the command line, I prefer to set them in the script itself. Go ahead and open the k6-DF-script.js file and update the options as follow:

export let options = {
maxRedirects: 4,
duration: "1m",
vus: 60,
iterations: 60
};

By doing this, we are telling the test that we want 60 different virtual users. The test will run for a maximum of one minute. The “iterations” option is set as 60 as well because we want one scenario iteration per virtual user. Otherwise, it will run multiple iterations of the same scenario.

For more consistency with a real-world scenario, we also want to pause between each call as the user is taking the time to read the response from the chatbot. So we will simulate this by adding a sleeping time in between the calls.

import { sleep } from 'k6';
const SLEEP_DURATION = 1;
...
export default function() {
postman[Request]({
...
});
sleep(SLEEP_DURATION);
postman[Request]({
...
});
sleep(SLEEP_DURATION);
postman[Request]({
...
});
sleep(SLEEP_DURATION);
}

The generated script is a great starting point but you should know that you can make it better by using the collection and environment variables in order to avoid URL or token repetition for example. There is also a lot more available options that you can leverage for your test. You can also look at the main source I used to start my investigations.

Now that we have updated the options and improved a bit the script to be closer to reality, we can run the test again:

Everything had passed and you can get some information about how much time on average an HTTP request takes (120.81ms) or the maximum time the full scenario is taking to get processed (3.82s) as well as the confirmation of how many virtual users have been used, the total number of iterations and the total number of HTTP requests.

But now, let’s see what happens when the system stops responding. DialogFlow, with the free version, allows up to 180 requests per minute. That why we have been using 60 virtual users. Each of them executing 3 requests. Let’s go ahead and increase the number of virtual users to 70 in the script so that we end up exceeding the quota.

As we can see, we don’t have 100% success in our test but only 97.14%. According to your tolerance to errors in your application, you should take action in order to fix such a gap.

Next steps

In any project where you want to continuously track the performance of your APIs, you would need to integrate K6 script execution to your pipeline. Since K6 is everything about code, it is very flexible and allows you to do so.

You can also visualize the results in an external application. Whether you choose DataDog, Grafana, Kafka, or simply JSON.

Extend your tests. K6 allows you to run various types of tests:

• unit tests
• integration tests
• Smoke tests
• Load tests
• Stress tests
• Soak tests

They might not all be relevant for every use case but they will all give you relevant information about your application behavior under certain circumstances.

Summary

K6 is pretty recent (2017), written in Go, and offers a lot of possibilities. This is a first time experience for me and I am really happy with the outcome you can get pretty fast. The possibilities on the automation side are also worth investigating in my opinion. A fully integrated performance testing solution is something of great value that again, I didn’t see in place in any of the projects I worked on. Will definitely keep this tool in mind and look forward to using it and to contribute if necessary.