Analyzing benchmarks with ease using benchcmp and benchviz Golang

The power of Golang toolkit has enabled developers to be pristine with the performance analysis and computing exceptions from their code. Now you don’t need a collection of mammoth data set or to take the cognitive load to use time complexity analysis to begin with to start digging deeper into your code performance.

Hmm, But I’m a genius, I could use time complexity analysis to compare two programs and see which performs better, then why should I use benchmarks?

Well, performance bottle necks could happen not just because of wrong choice of algorithms, but also due to poor selection or use of system calls. Benchmarks also helps you compare systems calls which otherwise is a cumbersome task. And please don't consider this to be replacement for time complexity analysis, but benchmarks are definitely a great way to easily understand which piece/version of code performs better. And also the performance difference widens up for large number of inputs or large inputs if a piece of code is algorithmically poor ( exponential or higher degree polynomial complex functions), definitely, understanding time complexity is really important in knowing how the running time is being influenced by size of input data .

Sounds good! But, how difficult is to write benchmarks in Golang?

It’s astonishingly simple to write benchmarks in Golang, I’m not gonna demonstrate how to write benchmarks here, lot of useful articles are already available. Here is one by Dev Cheney which show cases the ease with which benchmarks can be written in Golang. If you haven’t written benchmarks in Golang before do check that out.

Benchmarks doesn't reveal much with their absolute times values per operation, these numbers are machine dependent. Hmmm, then how could benchmarks be best used?

Yes, the absolute running time doesn’t reveal much, but benchmarks can be best used in cases where you gotta prove X performs faster Y, that is to compare various versions of your code to see which performs better.

Hmmm, that makes sense, but it’s still quite a lot of cognitive load to compare these numbers and then to arrive at a conclusion, is there a easier way?

Welllllll…. that’s what this article all about :D Benchmap makes it easier to compare between various benchmark results and benchviz make it even more easier by providing simple visualization.

Let’s take the simple benchmark example taken by Dev Cheney in his example article and see how performance comparisons can be made easy with benchcmp and benchviz.

Firstly, setup benchcmp and benchviz on your machine. It very simple, just use the following commands to let Go-get to set it up for you

go get github.com/ajstarks/svgo/benchviz
go get golang.org/x/tools/cmd/benchcmp

Here is the first version of the code containing inefficient way of calculating Fibonacci series

Here is the benchmark for the above code

Lets run the benchmark and what happens,

Hmm, as I had said earlier, the absolute time 370 ns/op doesn't reveal much. Lets just run it again and this time redirect the output to a text file to be used by benchcmp later

Lets change fig.go to achieve the same objective through different piece of code in the version 2

Here is the benchmark for the above code (contains a small edit)

Let’s run the benchmark again on the modified version of fib.go and redirect the output to a file called new (Again, we the output to feed it into benchcmp)

Now Lets use benchcmp and benchviz on the output of 2 benchmarks to see which performs better, once installed they’ll be located $GOPATH/bin

Here is the format to use them,

Here is the output.svg which illustrates the performance difference between two benchmarks

You can find all the code used for this article here https://github.com/Karthic-Hackintosh/Codes-From-My-Blog-posts/tree/master/122

Also as i had said earlier in the post, the benchmark is evaluated for constant value of input data (10 in the above cases), the performance difference widens up between the 2 versions with larger input values, this is where time complexity analysis gives more clarity on the behavior on the code, nevertheless Bechmark comparison is a great place to begin with before stepping into profiling the code to identify the hotspot and then analyzing the reason for the performance bottlenecks.

Thats it for now!!! Aaaaand in the next article I’ll be discussing in short about benchmarking header middleware of Caddy server ( www.caddyserver.com). Till then, Happy Coding!