k6 with docker-compose + influxdb + Grafana

In this quick article, we will explore how we can run some load tests using k6 and use docker-compose to set up influxdb and Grafana to visualize our load test run results in near real-time using k6 dashboards available in Grafana.

Before that, I will add a brief description of these various tools as taken from their official documentation.

What is k6?

Grafana k6 is an open-source load testing tool that makes performance testing easy and productive for engineering teams. k6 is free, developer-centric, and extensible. Using k6, you can test the reliability and performance of your systems and catch performance regressions and problems earlier. k6 will help you to build resilient and performant applications that scale.

For more information visit this link.

What is InfluxDb?

InfluxDB is an open-source time-series database. This includes APIs for storing and querying data, processing it in the background for ETL or monitoring and alerting purposes, user dashboards, visualizing and exploring the data, and more.

For more information, visit this link

What is Grafana?

Query, visualize, alert on, and understand your data no matter where it’s stored. With Grafana you can create, explore, and share all of your data through beautiful, flexible dashboards.

For more information, visit this link

Step 1: Create an Ubuntu instance in AWS

We will go ahead and create an Ubuntu VM instance, navigate to your EC2 dashboard and then click on the Launch Instance button, you will be navigated to the below page.

Enter the below details and set up the Ubuntu instance.

Name: test-instance
OS: Ubuntu Server 20.04 LTS (64 bit x86)
Instance Type: t2-small

Under security groups, allow the following TCP ports: 8086 for influxdb and 3000 for the Grafana server, and 22 for SSH.

I will increase the storage from 8 to 10 GiB and launch the Ubuntu instance by creating a new key pair. Connect to the Ubuntu instance using your key pair details and let’s get started with the setup process.

First, I will run the below command to switch as the root user and then run the update command using apt update.

sudo su -
apt update -y

Step 2: Install docker & docker-compose in the Ubuntu instance

We will install docker and docker-compose,run the below set of commands.

apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release -y

 mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Post-running the above set of commands, run an apt update command.

apt-get update

Install all relevant components required for docker, docker-compose

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-compose -y

Next, we will install Git using the below command

apt install git -y

And we are done, with the setup part for docker, docker-compose, and git.

Step 3: Setup influxdb and Grafana using docker-compose

Run the below command to clone a sample GitHub repository provided by k6 and Grafana which contains quite a few boilerplate scripts and the docker-compose file which can be used to easily set up influxdb and Grafana using a few commands.

git clone https://github.com/grafana/k6 && cd k6

If you run an “ls” command, you will see the docker-compose.yml file which is available in the cloned git repository under the k6 folder.

version: '3.4'

networks:
  k6:
  grafana:

services:
  influxdb:
    image: influxdb:1.8
    networks:
      - k6
      - grafana
    ports:
      - "8086:8086"
    environment:
      - INFLUXDB_DB=k6

  grafana:
    image: grafana/grafana:latest
    networks:
      - grafana
    ports:
      - "3000:3000"
    environment:
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_BASIC_ENABLED=false
    volumes:
      - ./grafana:/etc/grafana/provisioning/

  k6:
    image: grafana/k6:latest
    networks:
      - k6
    ports:
      - "6565:6565"
    environment:
      - K6_OUT=influxdb=http://influxdb:8086/k6
    volumes:
      - ./samples:/scripts

Now, we will use the above docker-compose.yml to set up influxdb and Grafana in our instance using docker-compose, as seen in the YAML file we are using influxdb version 1.8 and have mapped container port 8086 with the host port for influxdb and our database name is k6. Similarly, for Grafana we will be running on port 3000.

Run the below command which will pull in all the applicable docker images and set up our containers using docker-compose.

docker-compose up -d influxdb grafana

Step 4: Access the Grafana dashboard from the browser

Next, we will use our EC2 instance public IP address and connect to our Grafana dashboard using port 3000. In my case, it will be as below.

http://13.116.150.136:3000/login

Use the below default login credentials for Grafana.
Username: admin
Password: admin

Post login with admin user you might see an option to change the default password: “admin”, if you want to change the default password you can do so or click on the Skip button to proceed and you will be navigated to the landing page for Grafana.

Here, we can check the default data source that will be added and configured in Grafana for influxdb done during the docker-compose setup.

Navigate to the URL: http://13.116.150.136:3000/datasources

When you open the data source you can view the details, here we can verify the URL and database name which was configured in the YAML file.

Next, what we can do is clone the same k6 git repository in our local machine folder and run the below command using k6 which will trigger the sample “stage.js” script and dump the results to influxdb which is running on port 8086 in AWS setup that we have configured (Note: You will need to install git and k6 in your local machine first before running this)

git clone https://github.com/grafana/k6 
cd k6/samples
k6 run --out influxdb=http://13.116.150.136:8086 stages.js

Post running this k6 run command, our first set of results will be pushed to the influxdb database called k6, now in order to visualize our run results we can configure a Grafana dashboard using the below steps.

Navigate to your Grafana home page and go to the Dashboard section

Under New, select the import option

In the Import page, we will use ID: 2587 which is a pre-configured dashboard available for k6 load tests in Grafana.

For more information, you can refer to this link.

In below page, we can add 2587 in the textbox and click Load button.

Finally, select the source as “myinfluxdb (Default)” option and click on the Import button which will load up our dashboard as seen below.

Step 5: Run k6 load tests and visualize in Grafana

Now, you can use sample k6 scripts and use the “k6 run” command with the out parameter to push to influxdb which can be visualized in Grafana.

My k6 sample load test script: stage.js

import http from "k6/http";
import { check } from "k6";

export let options = {
    stages: [
        // Ramp-up from 1 to 30 VUs in 30s
        { duration: "30s", target: 30 },

        // Stay on 30 VUs for 60s
        { duration: "60s", target: 30 },

        // Ramp-down from 30 to 0 VUs in 10s
        { duration: "10s", target: 0 }
    ]
};

export default function() {
    let res = http.get("http://httpbin.org/");
    check(res, { "status is 200": (r) => r.status === 200 });
}

k6 command to start load test execution is as below, out to influxdb url

k6 run --out influxdb=http://13.116.150.136:8086 stages.js

Post running the above command, you can view the real-time load test run results in Grafana along with relevant details as seen below.

Finally, use the below docker-compose command to tear down infra.

docker-compose down

I hope you found this article to be helpful, thanks for reading!!!