1,000,000 Sagas and 1,000,000 Events

Cizen is a library that supports building concurrent applications in Elixir. In this post, we try to start a million sagas and fire a million events with them.

For our example, I introduce a saga we’ll define, RelaySaga, and Relay event, which has two fields message and number. RelaySaga is started with a number parameter and works as described below.

If the given number is 0, RelaySaga:

• subscribes Relay event whose number field is 0.
• on Relay event, outputs the message field of the Relay event.

Otherwise, RelaySaga:

• subscribes Relay event whose number field is the same as the given number.
• on start, starts a new RelaySaga with a number parameter of (number-1).
• on Relay event, dispatches a new Relay event with a number field of (number-1).

If we start a saga %RelaySaga{number: 3}, the following sagas are started;

• %RelaySaga{number: 3}, which we started;
• %RelaySaga{number: 2}, which is started by %RelaySaga{number: 3};
• %RelaySaga{number: 1}, which is started by %RelaySaga{number: 2};
• %RelaySaga{number: 0}, which is started by %RelaySaga{number: 1};

and then, If we dispatch an event %Relay{message: "Hi.", number: 3};

• %RelaySaga{number: 3} dispatches %Relay{message: "Hi.", number: 2};
• %RelaySaga{number: 2} dispatches %Relay{message: "Hi.", number: 1};
• %RelaySaga{number: 1} dispatches %Relay{message: "Hi.", number: 0};
• %RelaySaga{number: 0} outputs “Hi”.

In order to ready to define the saga and event, run the following commands;

mix new massive_saga_attack
cd massive_saga_attack

and add cizen to your deps in mix.exs;

defp deps do
  [
    {:cizen, “~> 0.14.1”}
  ]
end

and then, run mix deps.get.

It’s ready. Open relay.exs and write the following code:

defmodule Relay do
  defstruct [:number, :message]
end
defmodule RelaySaga do
  @enforce_keys [:number]
  defstruct [:number]
  @behaviour Cizen.Saga
  alias Cizen.{Dispatcher, Event, Filter}
  require Filter # to use Filter.new/1
  alias Cizen.Saga
  alias Cizen.SagaID
  alias Cizen.StartSaga
  @impl true
  def init(id, %__MODULE__{number: number}) do
    Dispatcher.listen(
      Filter.new(fn %Event{body: %Relay{number: ^number}} -> true end)
    )
    if number >= 1 do
      Dispatcher.dispatch(Event.new(id, %StartSaga{
        id: SagaID.new(),
        saga: %__MODULE__{
          number: number - 1
        }
      }))
    end
    number
  end
  @impl true
  def handle_event(id, %Event{body: %Relay{} = relay}, number) do
    if number >= 1 do
      Dispatcher.dispatch(Event.new(id, %Relay{relay | number: number - 1}))
    else
      IO.puts(relay.message)
    end
    number
  end
end
defmodule Main do
  use Cizen.Effectful
  alias Cizen.Effects.{Subscribe, Start, Receive, Dispatch}
  alias Cizen.{Event, Filter}
  require Filter
  alias Cizen.Saga
  def main(num) do
    handle fn id ->
      perform id, %Subscribe{
        event_filter: Filter.any([
          # subscribes Saga.Started event from %RelaySaga{number: 0}
          Filter.new(fn %Event{
            source_saga: %RelaySaga{number: 0},
            body: %Saga.Started{}
          } -> true end),
          # subscribes Relay event from %RelaySaga{number :1}
          Filter.new(fn %Event{
            source_saga: %RelaySaga{number: 1},
            body: %Relay{}
          } -> true end)
        ])
      }
      perform id, %Start{
        saga: %RelaySaga{
          number: num
        }
      }
      %Event{body: %Saga.Started{}} = perform id, %Receive{}
      IO.puts("All sagas have started.")
      perform id, %Dispatch{
        body: %Relay{
          number: num,
          message: "Hello world!"
        }
      }
      # The final relay event has dispatched.
      %Event{body: %Relay{}} = perform id, %Receive{}
    end
  end
end
[arg] = System.argv()
{num, _} = Integer.parse(arg)
Main.main(num)

We can run it with mix run relay.exs 1000, and “1000” is the number of sagas and events (it actually starts 1001 sagas and dispatches 1001 events).

Now, we can start 1,000,000 sagas and fire 1,000,000 events with elixir --erl “+P 1500000” -S mix run relay.exs 1000000, but it does not work on my computer. It may be because my computer has only 8GB of RAM 😢, and we need about 8GB of free memories to start a million sagas.

Here is a memory and swap history when I start a half million sagas:

Memory Usage of a Half Million Sagas