Concurrency Simple Enough to Use

or, Rao muddles his way through an explanation of multi-threading.

I have a sense that, to survive, a programming language has to be great at one thing. Nothing will ever be easier to deploy than PHP. Lisps will always be the best at creating DSLs. And Ruby will always have the lowest barrier to entry for beginners.

So what’s the one thing Elixir is great at? Well, Elixir, like Erlang before it, has a rock-solid, easy-to-use model for hardware-agnostic concurrency. Elixir’s architecture allows developers to use concurrency to solve problems in ways that would be untenably complex in other languages.

But first, it’s worth stepping back and defining what I mean by concurrency. At it’s simplest, concurrency in programming is the ability to run parts of your program in non-deterministic order, either in parallel or sequentially. It’s important to note that this is a distinct concept from multi-threading, which I’ll define here as the ability to run a program across multiple hardware processes in parallel. A concurrent program doesn’t necessary have to be multi-threaded, but a multi-threaded program must be concurrent to reach its full potential.

Much of the programming we do in traditional scripting languages is non-concurrent, in that it is meant to be run in a deterministic order and block execution, even during expensive operations. Even JavaScript, which uses the event loop model to allow for asynchronous execution, is not necessarily concurrent — events are processed in insertion order, and can be reasoned about as such.

Erlang has always had a different approach, however. Well before multi-threaded computing was widely accessible, Erlang prioritized building abstractions to support concurrency. Without getting too far into details*, the Erlang machine has two important parts:

  • A series of virtual processes meant to run concurrently. Each process manages it’s own state in isolation, and can send simple messages to other processes either synchronously or asynchronously.
  • A scheduler, in charge of deciding which process gets to execute its code, and when best to do garbage collection.

A mature Erlang or Elixir application may have thousands of processes running concurrently, each encapsulating its own state and the capability to coordinate with other processes if necessary. The scheduler ensures that the full capacity of the underlying hardware is utilized, optimizing process execution order as it sees fit.

It’s worth emphasizing that each process is entirely virtual, with no relation to the underlying hardware of your computer. Even on a computer with no support for multi-threading, the Erlang machine can mimic multi-threaded systems with it’s concurrency abstractions, even if the eventual execution happens sequentially. As multi-threading has become increasingly powerful on consumer machines, Erlang has migrated those same abstractions to take advantage of newer machinery.

This level of abstraction is often lacking in other languages in their approach to concurrency. Every instance of Ruby’s Thread class, for example, maps directly to an operating-system-level thread. Java threads generally follow a similar pattern. This makes monopolizing a thread expensive and generally unwise. In addition, most scripting language push the burden of shared memory management onto the developer. It’s up to you to ensure that threads are synchronized correctly, and don’t mutate shared state unexpectedly.

When dealing with threads in the Erlang machine, issues with hardware and memory management are mostly abstracted away by the process model. Concurrent code that runs sequentially on a single thread looks identical to code that runs in parallel on multiple threads, and the requisite optimizations are all handled by the scheduler.

The result is that there are very few barriers to using concurrency when writing Elixir code. For example, Let’s pretend we have a time intensive operation that increments a shared counter on completion:

def time_intensive_operation do
:timer.sleep(5) #sleep for 5 milliseconds
Counter.increment
end

defmodule Counter do
use GenServer

def start do
# initializes a counter with a count of 0
end

def increment do
# increments the count by 1
end

def read do
# returns the current count
end
end

Here’s the code to run that function synchronously 1000 times, predictably taking around 6 seconds to run.

Counter.start

Enum.map (1..1000), fn(_) ->
time_intensive_operation()
end

Counter.read
# 1000

and here’s the code for doing the above task concurrently, utilizing as many OS threads as possible:

Counter.start

tasks = Enum.map (1..1000), fn(_) ->
Task.async fn ->
time_intensive_operation()
end
end


Enum.map(tasks, &Task.await/1)

Counter.read
# 1000

The latter of which takes half a second to complete on a modern machine, and at scale, requires little to no tuning to make work.

Attempting the same in Ruby — having multiple threads alter a shared counter — would be difficult. Here’s the naive solution:

def time_intensive_operation
sleep(0.005)
@counter += 1
end

@counter = 0

(1..1000).map do |i|
Thread.new(i) do |i|
time_intensive_operation
end
end

puts @counter
# ???

As you may have guessed, there is no guarantee that the threads play well together. Indeed, if you execute the code above, you’ll see the threads non-deterministically trample over each other. Running this script will probably print a number between 10 and 900, but never 1000.

You can force the threads to line up sequentially with a mutex:

mutex = Mutex.new

@counter = 0

threads = (1..1000).map do |i|
Thread.new(i) do |i|
mutex.synchronize do
time_intensive_operation
end
end
end

threads.each {|t| t.join}

puts @counter

The above code removes any benefit of multi-threading, however, as it forces the threads to run sequentially. It will correctly print out the number 1000, but take the full six seconds to run.**

How does Elixir manage to use a multi-threaded approach while maintaining deterministic guarantees? Well, in the case, I’ve implemented the counter as a persistent GenServer process which manages its own state and evaluates messages it receives sequentially, in insertion order:

defmodule Counter do
use GenServer
  @name :counter
  def start do
GenServer.start_link __MODULE__, 0, name: @name
end
  def increment do
GenServer.cast @name, :increment
end
  def read do
GenServer
.call @name, :read
end
  def handle_cast(:increment, count) do
{:noreply, count + 1}
end
  def handle_call(:read, _from, count) do
{:reply, count, count}
end
end

Indeed, the counter’s state isn’t shared at all, and updates to it do not occur in parallel. The counter process is held distinct from the rest of the program, unable to be accidentally mutated. Other processes can communicate with the counter process, without fear of race conditions or write contention.

Such an architecture would be debilitatingly expensive in another language, as the counter would monopolize one of the few OS threads available. Instead, you would have to reach for something more complex like a mutex or a lock to ensure shared state is well-managed. In Elixir, however, there are many more opportunities available to experiment with process-based architecture, offloading most of your performance and safety concerns to the underlying Erlang architecture.

While this is a contrived example, the overall structure of the above code is characteristic of Elixir projects — concurrent whenever possible, blocking only when necessary. This is, of course, the inverse of most programming languages, and also what excites me the most about Elixir’s potential. The virtual process model makes it so that multi-threaded, concurrent programming techniques are the first things you reach for when writing Elixir.


*The details are interesting, though! This is a good high-level resource I’ve seen on the topic of Erlang’s scheduling architecture.

**I’m oversimplifying. There is a trick to making the Ruby code perform well, and it requires passing around the mutex to only lock when necessary.

def time_intensive_operation mutex
sleep(0.005)
mutex.synchronize do
@counter += 1
end
end

I’d posit that this mutex-aware complexity is a real barrier to using concurrency, however, and scares Ruby developers away from using it most of the time.