Periodic tasks with Elixir — Part 2

Alternative implementation and dealing with drift

My post about performing recurring or periodic tasks in Elixir using receive timeouts and Tasks couple of weeks ago sparked quite a few conversations which was amazing! Thank you to everyone here on Medium, Twitter and in Reddit for sharing your experiences and suggesting alternatives.

Two of the things that came up most was using Process.send_after and concerns about drift caused by execution of the work. We’ll look at both of these in this post.

If you haven’t checked out the first post yet you can find it here: Periodic tasks with Elixir

Process.send_after/3

As soon as I had posted the first part couple of my colleagues immediately asked if there was a reason I’d not used Process.send_after/3 instead. I have to admit that I wasn’t even aware this existed.

Replicating a similar barebones module we developed in the first part would look like this

defmodule Periodic.Safter do
  use GenServer
  def start_link() do
    GenServer.start_link(__MODULE__, %{})
  end
  def init(state) do
    schedule_work()
    {:ok, state}
  end
  def handle_info(:work, state) do
    # do important stuff
    IO.puts "Important stuff in progress..."
    schedule_work()
    {:noreply, state}
  end
  defp schedule_work() do
    Process.send_after(self(), :work, 1_000)
  end
end

It appears this is originally from a stackoverflow answer by Jose Valim in 2015 it also appears in the GenServer docs for Receive regular messages

Very similar at first look and without a deep dive under the hood I’d imagine they work the same way as well so for me it comes down to semantics.

For the simplest use cases and where we don’t need to track state I’d still lean toward the original. Trading the ceremony of state handling that comes with GenServer with the empty receive block.

Dealing with drift

Both of the techniques used so far suffer from drift in time where the execution time of the work to be done shifts the start of the next.

Say we wanted to do a piece of work every 1 second but the work we are doing takes 100 milliseconds to complete. After 10 seconds elapsed we would have only done the work 9 times as each execution was shifted.

Most of the time the simplest thing you can do is just spawn a separate process to perform the work and let the current process get back to scheduling the next piece of work.

Let’s take out GenServer example from earlier

def handle_info(:work, state) do
  # do important stuff
  IO.puts "Important stuff in progress..."
  schedule_work()
  {:noreply, state}
end

We can simply do this instead

def handle_info(:work, state) do
  spawn_link(&do_work/0)
  schedule_work()
  {:noreply, state}
end
defp do_work() do
  # do important stuff
  IO.puts "Important stuff in progress..."
end

By using spawn_link/1 we keep the same behavior as before in terms of exit signal handling and what happens if the code that does the work crashes.

Spawning processes is super fast in Elixir so this should all but mitigate your drift. For fun I decided to test out how long the spawn_link/1 takes in the above example and it was about 10 microseconds on average 😍 Quick back of envelope math would suggest a drift of 1 second after 100,000 executions.

A more complex and a more accurate alternative would be to store the time of last execution in the GenServer state and work out the amount of drift and then use this as argument for the nextschedule_work call so it can be deducted from the wait time to compensate.

Existing libraries

Finally there’s of course some great packages around to solve this for you.

Erlcron seems to be a popular Erlang choice and provides easy(ish) to read job descriptions like these:

{{once, 3600},
    {io, fwrite, ["Hello, world!~n"]}}

{{daily, {every, {23, sec}, {between, {3, pm}, {3, 30, pm}}}},
    {io, fwrite, ["Hello, world!~n"]}}

Or Quantum for Elixir which appears to be well maintained and a popular choice. Quantum opts for describing your jobs in the config using the cron syntax

# Every minute
{"* * * * *", {Heartbeat, :send, []}}
# Runs every midnight:
{"@daily", {Backup, :backup, []}}

In the end both solutions seem to have some trade offs in terms of semantics that leave me still wanting a cleaner solution. Having surveyed the existing libraries I’m tempted to write a simple package for even cleaner syntax. Let me know if you’d be interested in that.

Hope these posts have helped you get some ideas on how you might tackle those periodic tasks in your next project and how to tackle the issue with drift.

Please click the 👏 icon below if you liked this post and follow me here or on Twitter @efexen to hear about new articles. If you haven’t already can I recommend checking out some of my other posts and let me know what you think 👍