jabak
jabak
Jul 25, 2017 · 1 min read

Nice post. Really informative and good tips for setting up the test to be more comparable. A few comments though.

Good to see that you keep the repetitions down to a few hundred. I’ve seen a lots of benchmarks serializing/deserializing the same object 100.000 times which is not a realistic scenario IMO. I know that even a few hundred repetitions is kinda unrealistic IRL but 100.000s of repetitions makes the benchmarks far more synthetic. Nevertheless you should test with a range of repetitions IMO. Let me explain.

A few months ago I did some testing myself using c#. I tested the built-in binary serializer, protobuf-net, Newtonsoft, Zeroformatter and DataContractSerializer.

What I found was that the binary serializer won hands down when serializing a few different objects to around a couple of thousand objects. The more objects serialized the smaller the gap became between the serializers. Protobuf and zeroformatter clearly won when serializing 10.000s of different objects in one go.

Maybe you could run your benchmark with let’s say 25, 100, 200, 500, 1000, 10000 and 50000 repetitions to see if that makes a diffrence.

Also try out MessageShark (https://github.com/rpgmaker/MessageShark)

    jabak

    Written by

    jabak