Benefits of benchmarking with Go

Photo by gustavo Campos on Unsplash

Good teams were doing their stops in around 4 seconds — but they could do better. Mercedes dropped that to 3.4s in 2011, McLaren took it on to 2.32s in 2012, Red Bull lowered that to 2.05 at the start of 2013 and broke the two-second barrier at the end of the season, cycling Mark Webber through the pits at the United States Grand Prix in a startling 1.92 seconds. And then… a pause.

Whenever I think of benchmarking I think about the pit stop crews in the F1 Formula; Always trying to beat the existing best pit stop time while also improving on their methods.

In software engineering, we also do the same.

In this post, we will be discussing the benefits of benchmarking on Go.

Problem

Imagine that you are writing a project in Go. You might notice that there are a lot of ways for us to concatenate strings in Go. Let’s say that you have two variables,

str1:=”I love”
str2:=”Golang”

To create

str3 := “I love Golang”

we can take a lot of different approaches:

• The basic str1+" "+str2
• Using the package fmtand calling fmt.Sprint(str1," “, str2) or fmt.Sprintf("%s %s", str1, str2)
• Using the package strings calling strings.Join([]string{str1,str2}, “ “)
• Even going down and dirty with bytes buffer

Surely all of them work differently behind the scenes. The question is: how different? Then, the follow-up question would be: which should I be using for my project? Since string concatenation is something that you see a lot of times in many projects, this can nudge your code to perform a bit better in the long run.

Here comes benchmarking to the rescue.

What is Benchmarking?

A good analogy of benchmarking is the pit stop team of many different F1 teams; Different teams have different numbers of crew, different approaches, and different technology. All of them contribute to the number of seconds it takes for a car to get its pit stop finished.

Benchmarking in software development is a process used to compare the same type of things under the same procedures. Benchmarking enables us to get a preview of our program’s performance and its potential bottlenecks. This can give us great insights on seeing how big of an impact a new feature/improvement would do on the codebase.

Golang has a built-in package to benchmark our code. With it, we can compare different approaches against their performances.

Benchmarking in Golang

Going back to the string concatenation problem. We’ll compare how each of these implementations perform by creating a simple code. Firstly, we create the functions that we want to benchmark in main_test.go and fill in the following:

It’s important to note that all of the functions must have the `Benchmark` prefix to mark it as a benchmarking function. If you don’t put it in there, then Go won’t consider running it for benchmarking.

Every benchmark function receives a *testing.B variable as an input. Let’s look a little bit into B:

type B struct {
    N int
}

B contains a lot of unexported fields, but it has this N field publicly available. b.N dictates how many times the benchmark will run the target code. It will vary b.N until the benchmark function lasts long enough to be timed reliably. You can set the number of iterations yourself by adding the benchtimeflag when running the command.

To run the benchmark, just call

go test -bench=.

Here’s the result

Now, what does this all mean? Let’s break down the output.

First, goos, goarch, pkg, cpu describes my system OS, architecture, package that I’m testing, and my CPU specs respectively.

Below that, we have 5 benchmark functions with –4 suffix denoting the number of GOMAXPROCS the benchmark is running on.

On the side of those words, we have some numbers. The ones right next to the benchmark names are the numbers of N iterations the benchmark function was running. Each benchmark is run for a minimum of 1 second by default. The N will be increased while the function is re-run again while the minimum requirements have yet been made.

What we are interested in looking at are the numbers besides that. Those numbers indicate the speed of each function. For example, fmt.Sprinttook 152 nanoseconds per operation, while the simple operator concatenation only took 0.3662 nanoseconds per operation. That’s roughly 415 times faster than the fmt.Sprint!

We can see how much impact in performance different string concatenation methods can give. A trim of a few hundred nanoseconds might be hardly noticeable for a single request. The benefit will show its impact in a large-scale environment.

In Tokopedia, we use benchmarking for both measuring and optimizing performance.

Whenever we are building a new feature and unsure of the code’s performance, we can estimate the viability of using the current implementation — considering the size of the traffic the feature is expected to receive.

As we get more requests, we also need to scale our throughput. Giving a significant optimization on our code logic can help in increasing the overall throughput of the system.

Optimizing our system at the right time is one of the ways to improve the overall performance of our existing services. Remember not to prematurely optimize things, as it can lead to slowing down development.

Conclusion

Benchmarking is a really powerful tool to analyze and optimize different approaches to a specific problem. In Tokopedia, it is crucial for us because every optimization counts. A slight improvement can greatly affect the performance of a server with high Requests Per Second (RPS) / high load in general.

In terms of optimization, there can be a lot of areas that can be optimized. However, we have to allocate our efforts where optimization can bring a huge impact. We will be discussing profiling next, in order to understand the flow of the program and determining which parts to optimize first.

As always, we have an opening at Tokopedia.
We are an Indonesian technology company with a mission to democratize commerce through technology and help everyone achieve more.
Find your Dream Job with us in Tokopedia!
https://www.tokopedia.com/careers/