Benchmarking avro and fastavro using pytest-benchmark, tox and matplotlib

Lets begin…

Following code uses the pytest-benchmark, an extension for pytest.

  • ensure that pypy, pypy3, py27, py35 and py36 are installed on your operating system before running tox.
  • tox is responsible for running the pytest once for each python interpreter. This is declared in envlist variable of tox.
  • running pytest with -m “benchmark” only runs benchmark tests.
  • passing — benchmark-json=.benchmark-{basepython} to pytest tells pytest-benchmark to write the result of benchmark into files. Later I will use these files to compare results. For each run of tox, {basepython} evaluates to the version of python interpreter used in that run.
pytest-benchmark on pypy
pytest-benchmark on pypy3
pytest-benchmark on py27
pytest-benchmark on py35
pytest-benchmark on py36
  • There is a lot of information in those tables but the one that truly captures the pulse of performance is OPS (kops/s), which basically indicates the number of operations performed each second.
  • benchmark.pedantic gives control over rounds, iteration and warmup_rounds.
  • I have categorized my results into 2 logical buckets for ease of interpretation and comparison. This is achieved by the use of @pytest.mark.benchmark(group='...').
  • encoders group lists the benchmark results from fastavro schemaless_writer and avro writer.
  • decoders group lists the benchmark results from fastavro schemaless_reader and avro reader.
Plotting results from pytest-benchmark. Code inspired from https://matplotlib.org/gallery/units/bar_unit_demo.html

Conclusion

  • fastavro is multifold faster compared to avro, at-least in all the cpython interpreters.
  • general performance of pypy3 is much better than all other python interpreters.
  • avro is much faster than fastavro on pypy, this is really surprising.

Thanks

I want to give a shoutout to Scott Belden and Ryan for their insights with https://github.com/tebeka/fastavro/issues/195 which resulted in me writing this blog.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store