We speed up Membrane by reshaping Maps operations

We’re constantly working on improving the performance of the Membrane. It’s not easy, as it’s not obvious where to look for improvements. There are many factors that may affect performance and so it’s hard to point out just one that could be a bottleneck. We also don’t want to spend weeks or even months making changes that may not be as satisfying as we all expect. On the other hand, it could be pleasing when your work is making a real difference in the end. Here is a little story of our latest work on speeding up our pipeline mechanism.

It all began when we started to work on improving read and update time when using Elixir’s Map in Membrane. It’s crucial as we are using maps all the time to keep the state of processes and pass-through messages. Basically, all the data flow through the pipeline is done through maps. We’ve done many benchmarks, tested all provided access and update methods, and finally noticed significant differences between operation times using them.

The most popular way to access the value by key from the map is to type:

var = our_map.key

But the issue is that the syntax here is ambiguous. Using a function from a module stored in the variable will look pretty much the same:

var = module_name.func

So the compiler firstly needs to check what types of data it’s operating on. It’s not an expensive check, but done multiple times, and/or on a nested map might be noticeable

The other commonly used option is to use Map module functions:

var = Map.get(our_map, key)

That looks simple, but using a function from an external module (in that case Map module) always takes more time. Moreover, on compile-time, there are some limitations of optimizing code as Erlang allows to switch module implementation while running (that’s a cool trick but against performance in that particular case)

The fastest way to get data stored in a map in Elixir is to use pattern matching:

%{^key = value} = our_map

This syntax is well known in the Elixir community. It’s straightforward and self-descriptive, but it starts to be unreadable when it comes to nested maps. It becomes even more crooked when you need to update the value. In the Kernel module, there is a ‘update_in/3’ function but it’s based on Access behavior, which makes it quite generic, and also generates unwanted overhead.

To make it as fast as possible, we decided to create our own, Map specific, macro based on the same mechanics as ‘update_in/3’. We’ve compared it with other methods and the results are pleasing:

CPU Information: Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
Number of Available Cores: 8
Available memory: 16 GB
Elixir 1.14.0-dev
Erlang 24.3
Name               ips        average  deviation         median         99th %
fast_map        3.37 K      297.03 μs    ±18.08%         284 μs         513 μs
map             1.47 K      681.08 μs    ±25.72%         644 μs     1119.77 μs
Comparison:
fast_map        3.37 K
map             1.47 K - 2.29x slower +384.06 μs
Name                ips        average  deviation         median         99th %
fast_map         1.32 K        0.76 ms   ±123.00%        0.66 ms        1.74 ms
update_in        0.43 K        2.34 ms    ±76.97%        2.17 ms        4.11 ms
Comparison:
fast_map         1.32 K
update_in        0.43 K - 3.10x slower +1.59 ms

We also measured it with our internal tool for measuring the pipeline’s performance — it measures the number of messages per second sent through the pipeline:

Fast-map: PUSH: 57391.27 PULL: 41546.39 AUTODEMAND: 53491.85 [msg/s]
v0.9.0: PUSH: 46394.90 PULL: 27361.77 AUTODEMAND: 34696.72 [msg/s]

Then we tested Membrane Videoroom (one room with 21 participants) using BEAMchmark:

We introduced it in the v0.10.0 of Membrane Core, but due to Elixir’s bug, we found it wasn’t available right away. We found out that the code generated by our macro is freezing the compiler for some reason. We were debugging it for a pretty long time, blaming ourselves for accidental infinite recursion, but the problem seems to lay in elixir’s compiler type checking mechanism. The newest Elixir release v1.13.4 contains a fix for that bug and it’s already released (there are some inadequate type-check warnings while compiling a code, but those will disappear in Elixir v.1.14), we can finally announce a faster Membrane Core is ready to check!