.NET is fast, but I won’t be using it for my side project.

The latest (13th) Techempower benchmarks are out. For those of you who are skeptical of benchmarks, these are actually pretty good. They are as close to real-world as any benchmarks I’ve seen, and seem to be consistently well executed and analyzed.

This latest round is unique, as Microsoft has finally started paying attention to .NET performance. C# has risen in the ranks, but there are caveats.

The Plain Text Test

This is essentially a test of the HTTP stack for each technology. It’s a “hello world” plain-text HTTP get.

C# puts in its best performance with this particular benchmark, but it has 1,272 errors, so it’s a bit misleading. The other top contenders don’t have errors. And even here, C# is half as fast as the best Go implementation, and quite a bit slower than the best Scala and Java implementation.

Queries per second by language/framework:
Go:       3,602,407 
Scala:    3,054,880
Java:     2,914,627
C#:       1,822,366
Python:   975,225
JS:       467,246 
Elixir:   311,159
Express:  213,982
Ruby:     81,245
Rails:    3,907 (unicorn)

Other Benchmarks

The rest of these benchmarks test more real-world scenarios. DB querying, updates, etc. Let’s see how C# fares.

In the JSON serialization benchmark, C# is 3x slower than Scala’s best, 2.5x slower than Go, and slower than Node, too:

Scala:    556,192
Java:     555,922
Go:       463,413
Python:   420,820
Ruby:     377,344
JS:       304,372
Express:  186,548
C#:       185,435
Elixir:   107,966
Rails:    4,140 (unicorn)

In the multiple queries test, C# is 2x slower than Go, slower than JS, but beats Scala:

Go:       8,731
Java:     7,864
JS:       6,923
Python:   6,688
Ruby:     5,463
C#:       4,390
Express:  3,840
Scala:    3,777
Elixir:   1,945
Rails:    443 (unicorn)

Fortunes, which is a test of read performance has C# near the bottom of the pack of technologies I care about:

Java:     111,804
JS:       93,795
Go:       80,059
Scala:    53,310
Python:   45,230
Express:  34,691
Ruby:     32,118  
Elixir:   29,951
C#:       20,414
Rails:    1,900 (unicorn)

Data updates has C# performing quite well, slower than JS, Go, and Java, but still admirable:

Go:       3,050
Java:     2,873
JS:       2,841 (misleading, as this is mongodb, but the other contenders are SQL)
C#:       1,976
Elixir:   1,915
Scala:    1,639
Python:   1,354
Ruby:     732
Express:  374 (mysql)
Rails:    138 (unicorn)

Real World Implications

One thing I don’t know is how the Go, Java, or Scala benchmarks are set up. How realistic are their top-performing tech stacks? In other words, are those performances due to an unrealistically tuned stack that no one would really run in production? I don’t know. I’m also not personally interested in using any of those languages.

In the case of ASP, Express, Rails, and Phoenix, I can say that the numbers represent a real-world stack. If I were to deploy C# today, it would be using ASP.NET MVC Core (the stack used in these tests). If JS, I’d use Express. If Elixir, I’d use Phoenix. And if Ruby, I’d use Rails.

So, comparing ASP, Phoenix, Rails, and Express in these benchmarks, we get a good assessment of how real-world tech stacks size up. Let’s whittle the comparisons down to those:

Plain Text:

ASP:      1,822,366
Phoenix:  311,159
Express:  213,982
Rails:    3,907 (unicorn)

JSON serialization:

Express:  186,548
ASP:      185,435
Phoenix:  107,966
Rails:    4,140 (unicorn)

Multiple queries:

ASP:      4,390
Express:  3,840
Phoenix:  1,945
Rails:    443 (unicorn)

Fortunes:

Express:  34,691
Phoenix:  29,951
ASP:      20,414
Rails:    1,900 (unicorn)

Data updates:

ASP:      1,976
Phoenix:  1,915
Express:  374 (mysql)
Rails:    138 (unicorn)

Analysis

C# (ASP.NET MVC Core), Elixir (Phoenix), and JS (Express) are all solid performers and a safe bet if performance is your big concern. If performance matters, avoid Rails. In order to keep up, you’d have to run ~10x the server capacity with Rails vs most of these stacks (roughly estimated based on these numbers).

Performance isn’t the only thing that matters, though. Developer productivity is hugely important, and for that, your existing knowledge and your stack’s ecosystem is key. This is a bit subjective, but I think Rails might be the top-dog in an ecosystem war. Their online documentation is enormous and relatively consistent. There is a gem for *everything*, and getting up and running with a good test infrastructure is remarkably simple.

From what I’ve seen, Elixir and Node both come fairly close to Rails, in the ecosystem war, with Node being 2nd (there’s an npm package for just about anything), Elixir being in 3rd, but gaining ground.

ASP Core is dead last. When it comes to documentation and existing packages, ASP.NET MVC Core has to be at the bottom of the stack. Core is new, documentation is scattered, there are many dead ends (e.g. reading what appears to be a relevant article, only to find out that it is for a legacy ASP stack and no longer applies). C#’s package manager (Nuget) is a mixed bag, and last time I looked, a lot of the packages didn’t support .NET core. Trying to get up and running a month ago on Mac proved to be an exercise in reading obscure compiler errors.

That said, in my over 15 years of development experience, I’ve never found any dev environment that matches the productivity of Visual Studio + C# (and maybe F#) + Windows. So, that’s something to consider.

Conclusion

If I had to choose a tech stack today, it would be either Express (JS) or Phoenix (Elixir).

Depending on the app, Phoenix and Express both approach Rails’s personal productivity levels, and far outperform Rails in terms of scalability and resource usage. And while they aren’t as performant as Java or Go, they also don’t require you to write Java or Go, which is a big win in my book.

I’d avoid ASP.NET MVC Core right now, if you aren’t developing on Windows. Even though it performs well, the tooling and documentation for Mac/Linux development are lacking, and the experience (at least as of a few weeks ago) is painful.