Using Artillery and GitHub actions for automated load testing

Mauro Centurión
Sep 4 · 5 min read
Image for post
Image for post

Introduction

During one of our latest projects, our client told us they would be expecting a specific amount of requests in a short amount of time (think something like 5000 in 10 minutes). Even though the load is not excessive, we decided it would be best to be one hundred percent sure we would not be having trouble once those requests started pouring in. Then Martín and I started automating this task.

What we decided to do

One of the important aspects of every application is its architecture. In our case, it being a web application, we were running a PERN stack (PostgreSQL, Express, ReactJS and Node.js) and hosting it in AWS, with S3 bucket + CloudFront for the frontend and ElasticBeanstalk for the backend.

One of the advantages of this solution is that we didn’t have to worry about our frontend availability, since S3 and CloudFront are responsible for it. So now we just need to check if our backend and database can handle the load. Enter: Artillery.io.

What Artillery does

Artillery is a “modern, powerful & easy-to-use solution for load testing and functional testing”, so it can both check that the backend is able to handle the load and check that the response is what it’s expected to be. It works by running scenarios in phases, so you can choose to run a light phase with not so many requests per unit of time and increase the request rate to simulate user access peaks.

The scenarios are highly customizable as well, as you can not only define the target endpoint but also the probability that the scenario is run. Other options to customize the scenario is the payload of the request and the expected response (e.g.: the response status, content type and even expected response payload keys).

But what does an artillery test file look like?

Image for post
Image for post
Artillery test files can be written using YAML

Here we have a simple test that tries to GET two endpoints, one of them is behind a request limiter and the other one is not.

We define two phases, the first starts with four requests per second and in the next five seconds it ramps up to ten, the second one keeps on sending ten requests for three seconds.

In each phase there are two scenarios running, each one requesting a different resource. As you may have guessed, we don’t expect the /limited requests to work, since that endpoint is protected by a request limiter, so we define the expected response status code to be 429 (Too Many Requests). On the other hand, we expect /unlimited to always work, returning a 204 status code.

In our case, the flow we were trying to test was someone visiting a user’s profile and performing an action, so our test looked something like this:

Image for post
Image for post

So you see our flow has two requests instead of one, that’s because in our app, every action will be preceded by a user profile view, and it’s important that the server fulfills both requests sequentially.

Automating everything

Initially, because we were close to the release date, we decided to manually run our load tests and build our results charts. This worked fine and we were able to ship the code with confidence. But then Martín and I decided to automate the process so we can make sure that the app will always be able to respond to a rapid succession of requests.

Thanks to GitHub actions, we can run tasks after some events take place in the repository. Some of these events are pushing a commit to a branch or opening a pull request, which are the events we are interested in.

Github Actions is a world of its own so we won’t go much in deep about how it works. If you want to know more about it, here’s a good article you can start with.

So our GitHub workflow is as follows

Image for post
Image for post

First of all, we pull the latest code and install the node dependencies with some of GitHub’s own actions. Then we use akhileshns’ action to deploy an app to Heroku. For this, we make use of GitHub secrets so that we can change our Heroku configuration without needing to deploy a new version of our workflow.

Last but certainly not least we run our tests. For this, we have our own script setup. Our script reads all the tests in the directory, then runs them one at a time and finally prints out a summary in list form. If any of the load tests fail then the script exits with a non-zero value and if all tests are successful then the exit value is zero. The end result is that the GitHub action will only pass if all tests pass, and you can see if a commit introduces breaking changes.

Other possible workflows

We presented a load test approach to artillery and GitHub actions. Other things you can do is integrating the actions with your unit testing library (be it Jest, Mocha, or whatever library you decide to use). You can also use artillery to do unit testing along with your load tests.

Another option is to have several workflows to do different types of tasks, i.e.: deploy your app to different environments depending on the branch, sending alert emails, automatically posting a review comment, etc.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store