Benchmarking in C# Using BenchmarkDotNet

Benchmark logo

What is benchmarking

Benchmarking a way of measuring the set of parameters related to code execution. With benchmarking we can compare the performance of code under different scenarios and take decision about which way to go.

Why benchmarking

Often when we write code, we are not really aware of the effect on overall performance on application and reason is that we are not aware of performance of different methods to solve a certain problem. so benchmarking can help us determining which way to go.

Benchmarking process can help us to know if some modification done to code has improved or worsened its performance. It is required for understanding performance metrics of the methods we have used in our application so that those metrics can be used during the code optimization process

let’s understand by simple example, say I want to search a item in dictionary there are many ways this can be solved, but we don’t know which one is efficient and faster.

after some search I found this great library called BenchmarkDotNet which can helps us doing the benchmarking in c#.

Benchmarking in C#

BenchmarkDotNet is a lightweight, open source, powerful .NET library that can transform your methods into benchmarks, track those methods, and then provide insights into the performance data captured. It is easy to write BenchmarkDotNet benchmarks and the results of the benchmarking process are user friendly as well.

Demo

The idea behind this article is not cover every aspect of benchmarkdotnet library but to provide the starting point.

Let’s continue with our example of searching an item in dictionary.

Imagine we have a dictionary which have lot of strings and we have to get a value of given key.

Create a console application “BenchmarkingDemo” in visual studio. and add the reference of BenchmarkDotNet nuget package.

Now add a class “DictionaryLookup” and add the following code

public class DictionaryLookup
{
   readonly Dictionary<string, string> MyDictionary = new();
   
   public DictionaryLookup()
   {
     for(int i =0;i < 100; i++)
     {
       MyDictionary[i.ToString()] = $"test_{i}";
     }
   }
}

Add a method to get the value by key.

public string GetValueByKey(string key)
{
    if (MyDictionary.ContainsKey(key))
    {
      return MyDictionary[key];
    }
    return null;
}

This will be our benchmark method.

Add some more ways of getting the value by key.

Note: I have added only three more ways for demo purpose but there might be other ways also.

public string GetValueByKeyWithLinq(string key)
{
 return MyDictionary.FirstOrDefault(a => a.Key == key).Value;
}

public string GetValueByKeyWithTryGet(string key)
{
 MyDictionary.TryGetValue(key, out string value);
 return value;
}

public string GetValueByKeyManual(string key)
{
 foreach (var item in MyDictionary)
 {
  if(item.Key == key)
  {
   return item.Value;
  }
 }
 return null;
}

Now we have to add benchmark class. Let’s add “BenchmarkDictionaryLookup.cs”

[MemoryDiagnoser]
[Orderer(BenchmarkDotNet.Order.SummaryOrderPolicy.FastestToSlowest)]
[RankColumn]
public class BenchmarkDictionaryLookup
{
    DictionaryLookup obj = new DictionaryLookup();
        
    [Benchmark(Baseline = true)] // mark this is as baseline method.
    public void GetValueByKey()
    {
      obj.GetValueByKey("test_99"); // i am doing it for worst case by searching the last item.
    }
    [Benchmark]
    public void GetValueByKeyWithLinq()
    {
        obj.GetValueByKeyWithLinq("test_99");
    }
    [Benchmark]
    public void GetValueByKeyWithTryGet()
    {
        obj.GetValueByKeyWithTryGet("test_99");
    }
    [Benchmark]
    public void GetValueByKeyManual()
    {
        obj.GetValueByKeyManual("test_99");
    }
}

Finally add the code in Program.cs

class Program
{
    static void Main(string[] args)
    {
       BenchmarkRunner.Run<BenchmarkDictionaryLookup>();
    }
}

Run the application in release mode. open cmd prompt and go to root of the application.

dotnet run -p BenchmarkingDemo.csproj -c Release

After benchmarking is completed you will see the following summary result.

Results are also stored inside folder “BenchmarkDotNet.Artifacts” under the application’s root folder

Result explanation

From results it is seen that our benchmark method “GetValueByKey” and “GetValueByKeyWithTryGet” ranks 1 with 8 ns to lookup, where as “GetValueByKeyWithLinq” ranks last with 1773 ns.

also the “GetValueByKeyWithLinq” method have some memory footprint as seen the screenshot.

by looking at the timing results you might say how it is going make difference but if this operation performed 1 millions times in day then it would make the difference.

So next time when you have to write the code to get value from dictionary then you know which method to use and which one to avoid.

Summary

BenchmarkDotNet is a great library that provides a simple way to make an informed decision about the performance metrics of your application.

Now you can try benchmarking with different methods and share your findings..

Share your thoughts below..!

Overview

GitHub - dotnet/BenchmarkDotNet: Powerful .NET library for benchmarking