Code Speed Matters: Leveraging Go Benchmarks for Continuous Improvement

Elevating Your Go Code’s Speed and Efficiency

Photo by traf on Unsplash

In my recent journey within a product team, I stumbled upon a fascinating realm — Benchmark Tests in Golang. Unlike a mere surface-level examination, this feature plunges deep into the intricacies of code performance, uncovering those elusive bottlenecks that impact efficiency.

As a Principal Engineer in Test, I view the exploration of code-level tests as a gateway to a profound comprehension of code coverage. It acts as a lens, enabling us to identify gaps in unit test coverage and fortify the resilience of our test suites.

Drawing from this newfound knowledge, I’ve assembled a brief introduction to Benchmark Tests in Golang.

Contents

What is Benchmarking?
How to write a Benchmark test?
Running Benchmarks with `go test`
Interpreting Benchmark Results
Code Example — Benchmark Results Comparison
Use Cases of Benchmark Tests
Conclusion
Reference Links

What is Benchmarking?

Benchmarking is the heartbeat of performance optimisation, the practice of running a program or set of programs to assess the relative speed and efficiency of an object. It’s our trusty guide in the pursuit of identifying bottlenecks, optimising code, and ultimately enhancing overall performance. Enter Golang, a language that not only embraces simplicity and performance but also boasts efficient memory management and concurrency.

In the realm of Go, benchmarking takes centre stage with the help of the testing package — a feature-rich toolkit designed to make writing and running benchmarks a breeze. Golang’s commitment to efficiency and speed makes it an ideal playground for crafting high-performance applications, and benchmark tests become our secret weapon in ensuring our code not only meets but exceeds expectations.

If you’re new to Golang, a quick primer on its testing framework might be beneficial before diving into the specifics of benchmarks.

How to write a Benchmark test?

When writing benchmarks in Golang, we utilise the testing package, and a common naming convention for benchmark functions is BenchmarkFunctionName. Let's break down a simple example:

Example:

package packageName

import "testing"

func BenchmarkExampleFunction(b *testing.B) {
    for i := 0; i < b.N; i++ {
        // Code to be benchmarked
    }
}

In this example, the for loop is a crucial part of the benchmark. The loop runs b.N times, where b.N is a value set by the testing framework to determine the number of iterations needed for a reliable measurement. It ensures that the benchmarked code is executed a sufficient number of times to capture accurate performance metrics.

Think of it as a way to simulate the code’s behaviour under various iterations, allowing us to observe its performance characteristics more comprehensively. The testing framework takes care of adjusting the loop count based on the time it takes for each iteration, ensuring accurate and consistent benchmarking results.

Running Benchmarks with `go test`

To execute benchmarks in Go, the go test command comes with the -bench flag. This flag takes a regular expression as an argument to select specific benchmarks for execution. Use the following command to run all benchmarks in the package:

go test -bench=.

The -bench flag takes a regular expression as an argument to select specific benchmarks to run.

Flags:

The go test command provides various flags to customize the testing and benchmarking process. Here are explanations for some commonly used flags:

-bench

• Purpose: Specifies a regular expression to select benchmarks to run.
• Usage: -bench=. runs all benchmarks in the package.
• Example: go test -bench=.

-benchmem

• Purpose: Includes memory allocation statistics in the benchmark output.
• Usage: This flag is used to gather information about memory allocations during benchmark runs.
• Example: go test -bench=. -benchmem

-run

• Purpose: Specifies a regular expression to select tests and examples to run.
• Usage: -run=TestFunction runs tests with names matching the regular expression.
• Example: go test -run=TestFunction

-count

• Purpose: Specifies the number of times to run each test or benchmark.
• Usage: -count=n runs tests or benchmarks n times.
• Example: go test -count=3 runs tests or benchmarks three times.

-cpu

• Purpose: Specifies the number of CPUs to use for the test or benchmark.
• Usage: -cpu=n uses n CPUs.
• Example: go test -cpu=4 uses four CPUs.

Interpreting Benchmark Results

When running Go benchmarks using the command

go test -run=BenchmarkName -bench=. -benchmem

The typical output statistics are as follows:

BenchmarkName XXXXX   XXX ns/op  XX B/op     X allocs/op

i.e. For BenchmarkName, the benchmark ran XXXXX iterations, taking an average of XXX nanoseconds per operation, allocating XX bytes per operation, and involving X allocations per operation.

Name of the Benchmark: This is the name of the specific benchmark function being run

Number of Iterations (b.N):

• Represents the number of iterations the benchmark function is executed.
• The value of b.N is automatically adjusted by the testing package to achieve a reasonable benchmark duration.

Average Time per Operation (ns/op):

• Indicates the average time taken for each operation in nanoseconds.
• Lower values are generally better, as they indicate faster performance.

Allocations per Operation (B/op and allocs/op):

• B/op represents the average number of bytes allocated per operation.
• allocs/op represents the average number of allocations per operation.
• Lower values are generally better, indicating more efficient memory usage.

Code Example - Benchmark Results Comparison

In this example, we explore the impact of code changes on the performance of a function by utilising Golang’s benchmarking capabilities.

Let’s consider a scenario with an Employee struct and a PrintDetails method, in a file called employee.go.

package employee

import (
 "encoding/json"
 "os"
)

type Employee struct {
 Name string
 ID   string
}

func(e Employee)PrintDetails() {
 encoder := json.NewEncoder(os.Stdout)
 encoder.Encode(e)
}

In this implementation of PrintDetails used json.NewEncoder to encode the Employee struct and print the result to the standard output.

Accompanying this code, we have a benchmark test file called employee_benchmark_test.go:

package employee

import "testing"

func Benchmark_PrintDetails(b *testing.B) {
 e := Employee{
  Name: "John",
  ID:   "12345",
 }
 for i := 0; i < b.N; i++ {
  e.PrintDetails()
 }
}

run the test using go test -v -run=Benchmark_PrintDetails -bench=. -benchmem -count=3

the result statistics would be like:

Benchmark_PrintDetails-10 134058 9592 ns/op 32 B/op 1 allocs/op PASS

lets say employee.go code is now changed to:

package employee

import (
 "encoding/json"
 "fmt"
)

type Employee struct {
 Name string
 ID   string
}

func (e Employee) PrintDetails() {
 message, err := json.Marshal(e)
 if err != nil {
  fmt.Println("error occured", err.Error())
 }
 fmt.Println(string(message))
}

The modified implementation uses json.Marshal to marshal the Employee struct into a JSON-formatted string and print the result.

Now the benchmark test run result statistics might look like :

Benchmark_PrintDetails-10 126278 9583 ns/op 112 B/op 4allocs/op PASS

Comparison of performance results Before and After code changes

*Note: Despite the “Before” version running more iterations, the average time per operation remained nearly identical to the “After” version.*

Here’s a summary of the changes:

• The number of iterations (Iterations) decreased slightly in the "After" version.
• The average time per operation (Average Time per Operation) also decreased slightly in the "After" version.
• The average bytes allocated per operation (Average Bytes Allocated per Operation) increased in the "After" version.
• The average allocations per operation (Average Allocations per Operation) increased in the "After" version.

These changes suggest that the “After” version is marginally faster in terms of time per operation but incurs higher memory allocations and bytes allocated per operation compared to the “Before” version.

In my opinion the 1st version is efficient, since it is almost as fast as the 2nd version with less resources.

Depending on your performance goals and requirements, you may need to balance these factors to achieve the desired trade-offs between speed and resource usage.

In the realm of Go benchmarking, there’s a handy command-line tool called benchstat that aids in comparing and analysing benchmark results. It enables you to easily compare the performance metrics of different versions of your code. Once installed, you simply run benchstat followed by the paths to the benchmark result files you want to compare. The tool presents a concise summary, highlighting any significant differences in execution time, memory allocations, and more. For a deeper understanding of its capabilities and statistical insights, you can refer to the official benchstat package documentation shared below.

Use Cases of Benchmark Tests

Benchmark tests are useful in below use cases:

• Identify Performance Bottlenecks — suppose there is a critical function in code and you want to identify potential bottlenecks. Benchmarks can help pinpoint areas of code that might need optimisation.

func BenchmarkCriticalFunction(b *testing.B) {
    for i := 0; i < b.N; i++ {
        // Code to benchmark
    }
}

• Compare Implementations — You have multiple algorithms or implementations for a specific task, and you want to compare their performance to choose the most efficient one.

func BenchmarkAlgorithmA(b *testing.B) {
    // Benchmark implementation A
}

func BenchmarkAlgorithmB(b *testing.B) {
    // Benchmark implementation B
}

• Regression Testing — Ensure that recent changes or optimizations do not negatively impact the overall performance of your code by comparing benchmark results before and after the changes.

func BenchmarkFunctionBeforeChanges(b *testing.B) {
    // Original implementation
}

func BenchmarkFunctionAfterChanges(b *testing.B) {
    // Modified implementation
}

• CI/CD Stage — Running benchmarks as part of the Continuous Integration (CI) pipeline allows developers to catch performance regressions early in the development process. The primary goal is to ensure that changes to the codebase do not inadvertently introduce performance issues, and if they do, they are detected and addressed promptly.

Jenkinsfile — stage example

stage('Run Benchmarks') {
            steps {
                // Run Go benchmarks with necessary flags
                script {
                    sh 'go test -bench=. -benchmem ./...'
                }
            }
        }

Conclusion

Benchmarks provide quantifiable data on how code performs under specific conditions, helping engineers make informed decisions about optimisations and ensuring the overall health of a codebase. Regularly running benchmarks as part of the development process can lead to more efficient and scalable software.

Reference Links

Go Testing Package Documentation

benchstat Documentation

Happy Testing.