A Quick Start to Load Test with k6
Note: To access the files of this post please visit https://github.com/exaco/k6-loadtest
Introduction
After developing an overwhelming, ready-to-sell product, every organization will face a question titled: Is it ready for production yet or not? Besides the business-market fit, UI/UX acceptance, on the technical side, we are worried about the product under load conditions. It is working and fully functional with one user. What about 10, 100, or 1000 users working simultaneously? Would it crash or not?
The above questions are typically answered by the following QA/QC and performance test concepts defined here:
Smoke Test: Targets the system’s functionality under the slightest load: Is it working with only one user?
Load Test: Targets the system under normal usage by the users. You should ask: “How many users are using the system simultaneously typically, and how long will their sessions last?:
Stress Test: What is the maximum capacity of the system? How many users with what kind of behavior should use the site to disrate its quality due to the SLO: i.e., availability, request latency, throughput, etc.
Soak Test: Would the system sustain for a long time(generally from hours to days) under normal conditions? It’s working for 15min under typical conditions in the load test but is it feasible for a much longer time?
Answering the above questions helps you have a better perspective of the capability of your product under different conditions while redirecting the company’s business team to schedule marketing and release plans better according to the system capacity.
Now that we know the load tests’ importance, the next question comes to mind titling: “What tools can we utilize?”
Load Test Tools
There are various load test tools; among them, some popular are:
- k6.io: Created by Load Impact; written in GO; scriptable in Javascript
- Locust: Created by Jonathan Heyman; written in Python; scriptable in Python
- Jmeter: Created by Apache foundation; written in Java; scriptable(Limited) in XML.
We’ll investigate k6 here. It’s an Open source load testing tool and SaaS for engineering teams. Some of its pros are as follows:
- Easy to write test scenarios, especially with a screen recorder
- Capable of handling 30.000–40.000 generating up to 300,000 Requests per Second(RPS) with only one instance.
- Recently acquired by GrafanaLabs, demonstrating its reputation.
- Easily exportable to influx DB and visualizable in Grafana(lovely by SRE teams)
What tool should I use?
If you want to test your APIs, server backend services, k6 can be a reasonable choice. It’s easy to use and capable. But remember, it can only test the backend services, not the frontend components! If you want to test those components(client-side), too, Selenium is the better choice. Keep in mind that Selenium can only perform the functional(Smoke) test and not the load or stress tests!
Getting started with k6
To use k6 and acquire the results, follow these steps:
1. Install the requirements
To install the above tools, you can use the below docker-compose file:
version: '3'
services:
influxdb:
image: influxdb:1.8
volumes:
# Mount for influxdb data directory and configuration
- ./influxdb_data:/var/lib/influxdb
ports:
- "8086:8086"
grafana:
image: grafana/grafana:8.1.4
ports:
- "3000:3000"- AUT: The application under test; As an example, we used a Flask app in conjunction with VueJs provided by https://github.com/testdrivenio/flask-vue-crud. The installation guide is in the repository readme.
- k6: Run the below commands:
$ sudo apt-get update && sudo apt-get install ca-certificates gnupg2 -y$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
$ echo "deb https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
$ sudo apt-get update
$ sudo apt-get install k6
2. Write the test scenario
k6 uses a javascript file as the tests scenario. Don’t freak out if you’re not familiar or dominated with js. k6 has provided a beautiful and friendly Chrome extension. If you’re testing a web application, install the extension here.
You can learn the usage of the extension by the below gif:
Eventually, the recorded extension will redirect you to the below page.
You may choose the k6 native test builder, which provides a graphical request editor capable of launching testing online. But if you want to run k6 on your own, choose the script editor. Copy the js file into another file called test.js.
Remember that the current file is not perfect for the test, and you’ve to check all of the requests to make sure of the being normal and generalization of the scenario. Furthermore, the k6 screen recorder only catches the requests from the browser and cannot fetch your application frontend logic. For example, you may use xsrf-tokens or pass a last request parameter to the next one. Like our AUT example, which uses book ids from the Get request, we’ve to edit the put and delete requests to support the Ids. So, always check and edit the test files according to your requirements.
3. Run the test
After editing the Test file, you can run it via:
k6 run --vus=2 --duration=3m test.jsvus is the number of simultaneous virtual users, and duration is the test duration.
Otherwise, you can determine it in the test file with the option variable:
export let options = {
stages: [
{ duration: '2m', target: 30 }, // simulate ramp-up of traffic from 1 to 30 users over 2 minutes.
{ duration: '2m', target: 30 }, // stay at 30 users for 3 minutes
{ duration: '2m', target: 0 }, // ramp-down to 0 users
];
};4. Test Results
k6 will give you a summary of the results. This summary includes check statuses, number iterations, and http_req_duration stats which is the most important because it shows TTFB(Time to First Byte).
k6 lets you store the results with more details in the Influxdb and visualize preferable insights via Grafana. To do that, add the — outoption with Influxdb address just like below:
k6 run --out influxdb=http://localhost:8086/myk6db test.jsNow you can jump into Grafana and see the results. But before that, you need a pre-created dashboard. You can import the Grafana dashboard provided here.
Run a smoke test
In this smoke test, only one user is using the system for one minute. To launch it, add test_mode=smoke environment variable.
k6 run -e test_mode=smoke --out influxdb=http://localhost:8086/myk6db test.js
It’s evident that all of the requests are executed under 10ms and passed 100%.
Run a stress test
Now we run a stress test on the system to see its capability to handle 200 users simultaneously.
The system is facing severe issues. The latencies have increased from milliseconds to 9.3s. Some endpoints were unable to finish the request. This system may be considered a failure under some SLOs. Of course, it’s not a surprise! We’ve used its Flask in development mode(not production with several workers), increasing its capacity much higher.
