When to use processes in Elixir - Part 2: Running concurrent tasks

There is no doubt that processes take a center seat in Elixir development. All the code we write runs in a process. In the previous instalment of this series, we talked about how processes are the only way to have a state in Elixir applications. To continue our exploration of when to use processes, we will talk about running concurrent tasks using processes.

Normally, all the code we write in Elixir ends up running on the same process. That means instructions, operations, and function calls are executed sequentially. Whenever you want to run things concurrently, you need to dispatch them on a process.

Before we proceed, let’s consider an example; in your application, you have a function that writes content to a disk. And a website endpoint that uses that function:

Spawning processes

When calling index of your website, you have to wait until the file is written to disk. But what if you want to write a file to disk and send an HTTP response without waiting for the write operation to conclude? In this situation, you can dispatch the write operation in a process by wrapping the call with spawn

Calling Utilities.write/2 now executes the write operation in a new process. It does that by calling spawn which takes a function executes it in a new process. The return value of Utilities.write is this newly created process id. Let’s try that.

Note: There is another variation of spawn that takes the module, function and arguments. It then creates a process to call that function on. There is also another variation of spawn; spawn_link that creates a link between the current process and the newly created process.

What if in some situations we want to know Utilities.write succeeded? If we want that, we can make Utilities.write inform us about its result by sending us a message. Let’s update Utilities.write to do that:

Utilities.write first captures the current process PID by calling self(). It’s important that this call is before we spawn as inside the spawn self will be the newly created process. Then, after the write operation is finished, we send the result to the target process with send(target, result_of_write).

Now to read the result of the write, we need to update our website endpoint too:

After calling Utilities.write we immediately call receive. receive will stop and wait until a message is sent to it. When Utilities.write writes to disk, it sends a message to the original process (the website) which will be received by the receive.

With this update, we can call Utilities.write and ignore the result, or we can call it and wait for its result by calling receive after it.

The main formula is:

• Capture the current process
• Spawn a process and execute work on it
• Send a message to the current process with the result of the work
• The current process calls receive to read the response

Elixir Task

Spawning processes and waiting for their responses is a very common pattern in Elixir. So common that Elixir provides the Task module that does exactly that. In our example above. If Utilities.write is an operation that takes some time, we can use Task to dispatch it on a process:

Now the Utilities.write will run in parallel and we won’t get its result. The call to Task.async returns a new Task structure. If we want the result of the write operation, we call Task.await on the created Task.

Calling Task.await..well..it waits for the task to be completed and returns the task original return value.

Behind the scene, Task does something very similar to what we did above. async spawns a process and execute the function on it. Let’s look at how Task.async works:

Let’s break down the important bits:

• owner = self() gets the pid of the current process. The one we want to sent the response to
• pid = Task.Supervised.spawn_link(owner, get_info(owner), mfa) creates a new process that will execute our function asynchronously
• send(pid, {owner, ref}) send the pid of the newly created process

Inside Task.Supervised.spawn_link the response of the async task is sent back to the current process. This is how the result of the task is read.

On the receiving end, Task.await will wait for the response of the Task.async and reply back to the current process (owner). Let’s look at Task.await code:

There is a bit of work done in the function. The important bits are:

Here the reply is received in Task.await and it then is used as the return value of calling Task.await.

To recap, Task does something similar to the formula we defined above. Which is; Capture the current process, spawn a new process to execute the work, the new process sends a message, and await calls receive to receive the sent message and returns it to the caller of Task.await.

This concludes our second instalment on “When to use processes in Elixir”.

• We saw how we can use processes to perform some work asynchronously.
• We also saw how we can use a dance of self, send and receive to wait until the newly launched process concludes.
• And we finally saw how to use Task module to get that for free.

As a final note, if your want to do some work asynchronously, and you also want to wait for it too. It’s better if you use Task module as it handles all the possible quirks and edge cases.