A Docker solution to JMeter + InfluxDB + Grafana performance testing

Ellen Huang
10 min readMay 14, 2019

As more and more software migrates to the web, it is important to conduct performance testing on web apps in order to ensure good user experience and economize resources. If a web app doesn’t have sufficient resources to handle load, it might become slow and unstable, and negatively impact user experience. However if too many resources are allocated to the web app, this becomes costly for the supplier. Performance testing reveals these problems and can drive development of solutions.

A popular open source performance testing solution is to use the JMeter + InfluxDB + Grafana combo, with JMeter executing the performance tests, InfluxDB storing the test results and Grafana visualizing the results.

Problem

While JMeter + InfluxDB + Grafana is already a widely-used performance testing suite, preparing the test setup is a complicated and time-consuming process. In order to load test a webpage and see the results displayed in Grafana, we need to:

  • install JMeter, Grafana, InfluxDB and run all three services
  • write a JMeter test plan to be executed during testing
  • add a JMeter backend listener to send real-time test results to InfluxDB
  • configure JMeter with test properties including the test duration, ramp up time, number of threads, as well as the url of the webpage to be tested
  • create a database in InfluxDB to store JMeter results
  • add an InfluxDB datasource to Grafana and specify the host and port of the running InfluxDB service
  • create a dashboard in Grafana and query the InfluxDB data to visualize the JMeter test results

In order to skip this tedious and error-prone process, I implemented a solution that automates the entire setup, so that users who want to use the JMeter + InfluxDB + Grafana setup to conduct performance testing can do so with a single command.

Solution

My solution is to run the entire test setup as a multi-container Docker application, managed by Docker Compose, with three separate containers running JMeter, Grafana and InfluxDB respectively.

Some constraints that I considered while designing and implementing this solution are:

  • the solution must allow users to specify the url of the webpage they want to put load on
  • the solution should be lightweight and resource-efficient
  • the solution should be intuitive and user-friendly

Technologies

This section gives a brief overview of the technologies used in this solution. In a nutshell, JMeter runs the load tests, InfluxDB stores the test results, Grafana displays the test results, and all of these services are run in Docker.

JMeter

Apache JMeter is an open-source performance testing software application. JMeter conducts its testing by simulating a group of users sending requests to a target server (web application, database, etc.) and generates reports of the server response [1]. This is the software we use to put load on our web app.

InfluxDB

InfluxDB is a time series database that is mainly used to store monitoring data, metrics, real-time analytics and other data that is co-dependent on time [2]. InfluxDB provides the connection between JMeter and Grafana; InfluxDB listens to JMeter and stores the test results in a database, and Grafana reads the data from the InfluxDB database to display in its graphs.

Grafana

Grafana is an open-source real-time data visualization tool [3]. It provides a webpage where users can create, view and interact with graphs and dashboards displaying the data from its datasource (in this case InfluxDB). This is the software we use to gain visual feedback on the test results.

Docker

Docker is an open-source software that runs software packages called containers, which gather all the software and dependencies together [4]. The basic Docker workflow is:

  • we write a Dockerfile in which we define our environment and install any dependencies
  • we build a Docker image from the Dockerfile
  • we run the Docker image to start a Docker container¹, which starts the service

In our case, JMeter, Grafana and InfluxDB all run in separate Docker containers which talk to each other.

Implementation

This section delves into implementation details and explains some of the engineering decisions I made. If you are just interested in using the solution, feel free to skip ahead to the Product section!

Why use Docker?

Reproducibility: running a set of Docker containers is guaranteed to produce the same results every time with no possibility of random error. This is very important in testing, since tests are meant to be run repeatedly with controlled variability.

Installed dependencies: this performance testing solution requires many things to be installed, chief among them the JMeter, InfluxDB and Grafana software. Docker containers, which package all the software and dependencies together, seem to be a good fit.

Why a multi-container Docker solution?

Splitting out the application’s functionality into individual containers is the recommended approach, as opposed to running multiple services in a single Docker container, since the multi-container approach makes it much easier to update/scale individual components.

Scalability: the multi-container approach allows us to define a different number of containers for each service, something that cannot be easily accomplished if the services are all running inside a single container.

Updates: the multi-container approach allows us to only restart the container running the updated service, as opposed to having to restart the entire application just to update one service.

Why use Docker Compose?

Docker Compose is a tool used to configure the services in a multi-container Docker application [5]. In essence, when using Docker Compose, we still need Dockerfiles for each of the individual containers, and on top of that we also need to write a Compose file in which we configure all of the application’s services, for instance specifying the port-forwarding, build arguments and environment variables of each service. Using Docker Compose to manage the containers instead of managing all the containers manually has many advantages, including:

  • being able to build, start and stop all the services of the multi-container application with one command, instead of manually starting/stopping each container
  • being able to configure all of the application’s services in one place, the Compose file. This provides a better overview of the entire application and how each service is configured, and we wouldn’t need to pass these configurations in as runtime arguments each time we want to run the service, which is more tedious and more error-prone

docker-compose.yml

version: '3'
services:
influxdb:
image: influxdb:1.7-alpine
environment:
- INFLUXDB_DB=jmeter
jmeter:
build:
context: "jmeter"
args:
- jmeterVersion=5.1.1
environment:
- JMETER_TEST=default_test_plan.jmx
depends_on:
- influxdb
grafana:
build:
context: "grafana"
ports:
- "3000:3000"
depends_on:
- influxdb
- jmeter
  • Docker Compose creates a single network for all of the application’s service’s containers by default, and containers are able to discover other containers by their container name and talk to other containers via this network. This is extremely convenient in this use case because the JMeter container needs to talk to the InfluxDB container to pass test result data to the database and Grafana needs to talk to the InfluxDB container to retrieve test results to display. With the default network created by Docker Compose, I was able to configure JMeter to send data to InfluxDB by adding a InfluxDBBackendListenerClient to JMeter and specifying the InfluxDB URL as http://influxdb:8086/write?db=jmeter, and likewise configure Grafana to retrieve data from InfluxDB by adding an InfluxDB datasource with URL: http://influxdb:8086. Note that I don’t need to know what the ip address of the InfluxDB container is, and can directly send requests to http://influxdb, since by default Docker Compose creates each container with a hostname identical to their container name. This is great news for me as the developer since I can just use the default network created by instead of having to play around with Docker networking

In the following section, I demonstrate how to build and run this application with and without Docker Compose

Without Docker Compose:

We first need to build the Docker images for each service from their Dockerfiles. Note that we only need to build the images once at the beginning, and rebuild the images when there are changes.

  • docker build grafana to build the Grafana Docker image
  • docker build jmeter to build the JMeter Docker image

We need to perform the following every time we want to run the application.

  • docker run -p 3000:3000 grafana to run the Grafana Docker image with port forwarding
  • docker run -e INFLUXDB_DB=jmeter influxdb:latest to pull and run the InfluxDB Docker image with environment variable
  • docker run jmeter to run the JMeter Docker image:

Stopping the application requires essentially the same amount of effort, just replacing docker run with docker stop

With Docker Compose:

  • docker-compose up to build and run all the services
  • docker-compose down to stop all the services

Docker Compose is clearly the option that best satisfies the user-friendly constraint, with its concise commands that leave little room for errors. Its’ Compose file and default network creation functionality also makes it easier for developers to work with.

How did I choose my parent images?

Most Dockerfiles define a parent image; the final image that is built from the Dockerfile is the parent image after it has been modified by the instructions in the Dockerfile. There are many images hosted on Docker Hub that can be used as parent images to build on top of.

For InfluxDB and Grafana, I chose the official InfluxDB and Grafana Docker images, so that I wouldn’t need to install these softwares in the Dockerfile, and since official images are generally better maintained and more secure. I chose the InfluxDB image tagged with alpine in particular because Alpine image variations are more lightweight, and the influxdb:alpine image provided all the functionality I needed.

There is no official JMeter image on Docker Hub, so I chose Alpine Linux as the base image, in order to start with the smallest image possible, and manually installed JMeter in the Dockerfile.

Why user.properties file?

There are many things that a performance tester might want to configure when running the tests. The most obvious example that comes to mind is the URL of the web app to be tested, but other examples include the duration the test runs for, the number of concurrent users making requests to the web server, etc. The following section gives a comparison of several ways to implement allowing users to specify configuration values, and explains why I ultimately went with the user.properties approach.

1. Editing the test plan

The most direct way to set the values is to manually edit the test-plan.jmx file and hard-code in the desired values. The advantage of this solution is that it requires no implementation on my part. However its disadvantages are prominent; users would have to search for each property in the test plan and hard-code in the value at the appropriate place, and users who are not familiar with JMeter test plans might find navigating the .jmx file confusing and inconvenient. In short, this solution seemed the least promising to me since it clearly violated the “intuitive and user-friendly” constraint.

2. user.properties file

Another solution is for users to directly edit the user.properties file inside the JMeter test folder. The .jmx test plan then reads the properties specified in the user.properties file using the JMeter property function.

user.properties file:

# time it takes for jmeter to start all the threads in secondsrampUp=0# total number of threads to be executedthreads=2 # duration of the running test (in seconds)duration=1200 # url of the web app to put load onwebUrl=medium.com

example line in default-test-plan.jmx file with JMeter property function reading value of duration from user.properties:

<stringProp name="ThreadGroup.duration">${__P(duration,1)}</stringProp>

This is the approach that is the most user-friendly, intuitive and least error-prone (for a user with no background in JMeter or Docker) and I ultimately implemented this solution.

3. Environment variables

A third possible solution is for users to pass in the values as environment variables in the Compose file, similar to how users are currently specifying the name of the JMeter test.

docker-compose.yml with JMeter properties as environment variables:

version: '3'
services:
jmeter:
build:
context: "jmeter"
args:
- jmeterVersion=5.1.1
environment:
- JMETER_TEST=default_test_plan.jmx
- JMETER_RAMPUP=30
- JMETER_THREADS=5
- JMETER_DURATION=1200
- JMETER_WEBURL=medium.com
depends_on:
- influxdb

This solution and solution 2 are similar in that both solutions ultimately work by modifying the user.properties file, and differ only in user experience, with solution 2 requiring users to specify their desired configuration in the user.properties file and solution 3 requiring users to pass in their configuration as environment variables.

The advantages of this solution are that users wouldn’t have to decipher the .jmx test plan, and would only need to edit the environment variables in the Compose file. However, for a user with no background in Docker Compose, modifying environment variables in the Compose file is still less straightforward than setting properties in the user.properties file. Additionally, this solution would require that I implement logic to overwrite the user.properties file with the values of the environment variables. Overall, solution 3 seems to be less intuitive for the user and more work for me as the developer, which is why I ultimately decided on solution 2.

Product

In order to try out this setup for yourself, clone this repository. Update the user.properties file with your desired values, then run docker-compose up in the main directory. There should be load placed on your specified webpage right away, and you should be able to view the results in Grafana at http://localhost:3000 .

This solution is also configurable: you can use this setup to execute your own JMeter test plan by copying your-test-plan.jmx into jmeter/test/ and replacing the value of the JMETER_TEST environment variable, which should be ‘default-test-plan.jmx’, with the name of your new test (‘your-test-plan.jmx’ in this example) in docker-compose.yml.

You can also create your own Grafana dashboards to display the test results; you should be able to query the test results data by querying the database named “InfluxDB”, with measurement “jmeter”.

Please note that the focus of this solution is on automating the entire setup and packaging it into a Docker application, and not on performance testing. The default JMeter test plan is sufficient to put load on any specified web app, but if you want to actually conduct performance testing, it is recommended that you replace the default test plan with a more sophisticated test plan. Likewise, the default Grafana dashboard is more for demo purposes, and it is recommended that you create your own dashboards via the Grafana web UI for more insightful data visualizations.

Impacts

This Dockerized integration of JMeter, InfluxDB and Grafana provides a complete and customizable performance testing solution. It makes performance testing easily accessible for developers with little experience in this area, and for performance testing veterans, automating the setup process would still save a lot of time and effort. Additionally, because this solution can be run via command line, it can easily be integrated into an automated testing solution.

A potential weakness of the solution is that the default JMeter test plan and Grafana dashboard included, while functional, are very simple. As explained above however, the focus of this solution is on automating the setup process and packaging all the tools together with Docker, and not on providing a full-featured performance testing suite. Users wishing to conduct more serious performance testing are welcome to configure this solution with custom test plans and data visualizations.

References

[1] ‘Apache JMeter’. [Online]. Available: https://jmeter.apache.org/. [Accessed: 13-May-2019].

[2] ‘InfluxDB’. [Online]. Available: https://www.influxdata.com/products/influxdb-overview/. [Accessed: 13-May-2019].

[3] ‘Grafana’. [Online]. Available: https://grafana.com/. [Accessed: 13-May-2019].

[4] ‘Get Started with Docker’. [Online]. Available: https://www.docker.com/get-started. [Accessed: 13-May-2019].

[5] ‘Overview of Docker Compose’. [Online]. Available: https://docs.docker.com/compose/overview. [Accessed: 13-May-2019].

--

--