Writing a load testing tool in Go

Aniket Gadre
Jun 13, 2019 · 4 min read
Image for post
Image for post

Motivation :

In one of my previous organizations we had a requirement of load testing Python and Scala based micro-services which interacted over RabbitMQ (using protocol AMQP 0.9 which is pretty different from AMQP 1.0)

We were already using Jmeter as our preferred load testing tool for Rest APIs. So I had few options to work with

Jmeter and RabbitMQ tool could have served the purpose to some extent but they were more suitable for async messages. Some of our micro-services used rpc (Remote Procedure Call) functionality of RabbitMQ which is more of a synchronous communication between Consumer and Publisher over queues.

Now whether the usage of RPC over message queues was a correct design choice or not is a topic of discussion for some other time :)

Conclusion was to not use Jmeter plugin or RabbitMQ Perf Test tool but to build something of my own

I was learning Go around that time and was super impressed with Go’s concurrency model. I thought it was a good time and opportunity to convert my Go learning into implementing something useful so I decided to build load generator tool in Go.

BTW there is a nice load generator tool written in Go called Vegeta but it’s mainly for HTTP

Before starting to build the tool, I wanted to confirm few things

Design :

Once the above things were confirmed, I started thinking about the bare minimum functionality this tool should have and I decided to follow Jmeter’s design for the same. I needed to control

That’s it. If I could manage these 4 things my tool should be ready for use.

Code:

First thing was to accept user input for No of User, Ramp Up time and Execution time. This was pretty straight forward, I could set it with command line flags

Image for post
Image for post

Go encourages having shorter variable names. If anyone is more interested in knowing more, here is one presentation

Second thing was to calculate how much time should Go wait before it starts next user thread, which was also straightforward as I had the total no. of users to load and time in which all user threads should be up and running, so wait time in between initiation of two consecutive user threads was

Image for post
Image for post

So for example if my ramp up time was 30 secs and I had total of 10 user I would spin off a new user thread every 3 secs.

Now I wanted something which could increment the user thread count as per the ramp up calculations. Best way was to initiate a goroutine for that job.

Goroutine is a lightweight thread created by Go’s runtime when any named or anonymous function is invoked with a prefixed keyword go

Below goroutine pushes user count into the channel every time its incremented but won’t initiate the actual user thread.

Channel object in Go is a communication medium for goroutines to interact with each other by sending and receiving messages and is defined by keyword chan

Image for post
Image for post

Next thing was to get a time channel which would help in indicating the end of execution duration

Image for post
Image for post

After this the final bit was to use the two channels defined in above code for controlling the flow and executing the required tests

This piece of code brings together all the core features of Go which makes concurrency so easy

select is a special statement in Go which blocks the execution until any one of the channels in case statements is ready for send/receive operation

Image for post
Image for post

And that was all I needed !! I just added all required test calls in the above for loop and was able to bombard RabbitMQ message queues with hundreds of users that to with extremely low memory footprint on my load generator machine !

For monitoring and reporting I pushed all metrics into InfluxDB and used Grafana for real time visualization

Go is an extremely powerful language when it comes to concurrency. I would recommend every Go enthusiast to read the book Concurrency in Go written Katherine Cox-Buday

You can find the full gist of the above code here

The Startup

Medium's largest active publication, followed by +717K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store