Build your own GenServer in 49 lines of code
As you may guess, this article is not about rebuilding the Elixir GenServer. It’s already there and it works great. And that’s what interests me the most: why it works great?
Also, we will not discuss what GenServer does and its generic concepts. If you are not familiar at all with the it, I suggest starting with the docs. They do a great job explaining the basics:
- https://elixir-lang.org/getting-started/mix-otp/genserver.html
- https://hexdocs.pm/elixir/GenServer.html
The problem
I’ve been using Elixir for quite a while now. But I always had some doubts on some specific aspect of the GenServers: the asynchronous requests (handle_cast/2
). To be more explicit:
- is the GenServer state consistent across multiple async requests that take a various amount of time to execute (eg. running an expensive function inside one of the
handle_cast/2
implementation)? - are the async requests executed in the same order they were cast even if some of the casts take longer than others?
In other words, are GenSeerver async requests susceptible to race conditions?
Let’s find out
My confusion ended when my colleagues pointed me to the Erlang GenServer docs: http://erlang.org/doc/design_principles/des_princ.html#id63247
Among other useful info, you can find also a very basic implementation of the GenServer. I do not have Erlang experience, but I found the code readable enough. So I decided to rebuild it in Elixir, even add some extra features and shed some light on my dilemma.
MyGenserver
MyGenserver
, that’s what we’ll call the project:
mix new my_genserver
Before we start, don’t expect that we’ll end up with a fully featured GenServer. There will be many bits missing. But the basic functionality will be there at the end of our little experiment.
start_link / init
The GenServer normally starts with a start_link/3
function. For our experiment we will ignore the options
, so it will take only 2 arguments:
We pass a module and arguments to the start_link/2
function. The module
will be the one implementing the GenServer behaviour. Let’s call it Example, so we could start the server like this: start_link(Example, :ok)
.
spawn/3
creates a new process, running a function server_init/2
which does 2 things:
- expects that
Example
module defines functioninit/1
. It calls it and expects an{:ok, state}
response (congratulations! OurExample
can now implement theinit/1
callback) - starts a
loop/2
function, passing theExample
module and the initialstate
of the server. The loop is the heart of the GenServer and actually answers all my questions above
But let’s not rush into conclusions and continue the implementation.
call / handle_call
The call/2
function takes the MyGenserver
pid, and a request. It then sends a message to the pid, with 3 elements tuple:
- the action,
:call
self()
which is the calling process- request - the arguments we want to pass to the server
Inside the loop/2
there is a receive
function that listens to those messages and pattern matches on the action. In this case on {:call, _, _}
. We then expect that the passed module (Example
) will implement a handle_call/3
callback, that returns {reply, response, state}
.
Call makes a synchronous server request. The calling process is blocked until it receives a response from MyGenserver. So we send back the response with send(parent_pid, {:response, response})
. The call/2
receives and returns the response.
The last thing happening in the loop is to call itself, with the module (Example
) and the new MyGenserver state.
cast / handle_cast
As said above, the async server request, cast/2
is the actual reason I tried to rewrite this basic version of the GenServer. By this time however things are quite clear.
There’s nothing special about cast
. It is very similar to call
, except it’s not sending a response back to the “casting” process, and will not block it.
MyGenserver process mailbox manages the order of the functions execution, in a first in first out manner. And it’s sequentially consumed by the loop
function, which also keeps the state consistency.
It doesn’t matter if many async requests are cast to the MyGenserver
. And it doesn’t matter if some of those requests are expensive, time-consuming functions. They will be executed in the order they were cast. There will be no race conditions between them.
With the mystery solved, let’s quickly write the remaining callbacks.
stop / terminate
The same pattern as above, with one big difference. The loop is interrupted and the process will die with the specified reason.
handle_info
handle_info
callback does not have an associated API function. It catches all unmatched messages sent to MyGenserver and updates the state of the server.
The 49 Lines GenServer
Let’s put all of this together and see how it looks:
So, does it work?
It’s now time to see if MyGenserver
works. For this, we’ll write the Example
module referred above.
It’s a very basic example. All the requests take a number and add it to the existing state. The state is initialised in the init/1
callback with value 0
.
For the purpose of this demo, we also pass an order argument. We’ll use it to visualise the order in which the functions are called and executed (with IO.inspect
).
There are 2 types of requests:
- light — supposed to be very fast functions
- expensive — simulate some expensive operations that will take some considerable amount of time
We use a combination of those to test the state consistency and the order of the functions execution.
Testing
handle_call
We start an expensive call and a light call right after it. The test passes, but the IO.inspect
used in the example will give us some more insights:
REQUEST ORDER: : 1
PROCESS ORDER: : 1
REQUEST ORDER: : 2
PROCESS ORDER: : 2
Those are synchronous requests. The caller process is blocked until it will receive a response from the MyGenserver. Only after, a new request is sent to the server.
handle_cast
Same thing with the async requests. The test passes but let’s see the output:
REQUEST ORDER: : 1
REQUEST ORDER: : 2
PROCESS ORDER: : 1
PROCESS ORDER: : 2
The casting process doesn’t wait for a response. It immediately sends the second request. But, even if the first request is an expensive operation, the process order remains the same. The execution of the second request will not start until the first one ends.
handle_info
Same as above, but due to the nature of handle_info
we do not have a request order. The request is received directly through a process message.
PROCESS ORDER: : 1
PROCESS ORDER: : 2
terminate
Nothing much to add here. It monitors the newly created server so the test PID will receive the exit message from the server. Checks that the process is alive, stop it, checks the correct reason and that the process no longer exists.
combinations
This test is a combination of different requests. What we can see is the process order will always be sequential, no matter the type of request or the duration of the operation. This is exactly what aimed to demonstrate at the beginning of the article.
REQUEST ORDER: : 1
PROCESS ORDER: : 1
REQUEST ORDER: : 2
REQUEST ORDER: : 5
PROCESS ORDER: : 2
REQUEST ORDER: : 6
PROCESS ORDER: : 3
PROCESS ORDER: : 4
PROCESS ORDER: : 5
PROCESS ORDER: : 6
The Ultimate Test
With less than 49 lines of code, we were able to build our own basic GenServer
and understand how it works. Within MyGenserver
we implemented the server process loop. We proved the order of the requests execution and state consistency with an Example module and tests.
Now, as a final test. To check that our conclusions are correct, replace MyGenserver
with the real GenServer
in the example file. Don’t forget to use Genserver
at the top of the module. Now run the tests again and they will still pass.
You can see the full MyGenserver
, Example
and tests code on Github.
If you (as myself) had any doubts about casting requests behaviour, I hope this helps. And you will now use GenServer async requests with full confidence.