Rust — concurrency without regrets?
Rust offers the promise of “fearless concurrency” — delivering on it through memory safety. But this safety doesn’t guarantee code that is easy to maintain, or bug free.
If one is not “fearful” of complexity, concurrency can easily become a story of regrets. Can we get a “regret-less” kind of concurrency?
There is one particular approach to concurrency that I find only gets better with time, and I’d like to describe it in some details in this blog.
A sensible default: the event-loop
An event-loop is a loop that handles events — one at a time. An example of such a loop is the Web event-loop that runs Javascript on a web page. And since you know that Javascript code running in the browser is single-threaded — you might also be wondering why I bring it up here.
The reason I bring it up is because an event-loop might be your best default for modelling concurrency — the Web being a prime example.
Looking at this screenshot, I count about 200 threads in Web-related processes. What are those threads doing? Ever wondered how something like fetch(url) just works, and without blocking your Javascript code? How come adding an <img src="friday_night.jpeg"> to an HTML page actually loads an image?
And yet, despite this bacchanal of threads going on under the hood, we still refer to Javascript — and correctly so — as “single-threaded”. You will almost never be able to detect the concurrency of the Web platform from Javascript itself. How is this done? With the help of event-loops(give or take one per tab in your browser).
The Javascript on a page runs one “task” at a time. Some of those originate from a previously running task on the same event-loop. For example when a page first loads, a task is enqueued to fire certain events in a later iteration of the event-loop. In this case, running a task results in a another task being enqueued on the same event-loop — a form of single-threaded concurrency.
Other tasks are enqueued from other components running “in-parallel” to the event-loop where they are enqueued. When you do a fetch(url), the Javascript offloads this request to a dedicated “networking component”. That component is running in a different thread — perhaps even a different process — hence your code doesn’t block waiting for the response from the network. When the response from the network comes in, the networking component will enqueue a task back on the event-loop where the request originated.
The HTML standard defines “parallelism” as operations running “in-parallel” of an event-loop. The standard also does a good job of describing how the “event-loop” uniquely owns a “world of observable Javascript objects”, and that the only way to affect those is by “queuing a task” on that event-loop.
The “event-loop” here is not the prototypical event-loop described in this article, it is rather “an event-loop, running according to the processing model of HTML”. A concept in the HTML standard much closer to a “prototypal event-loop” is rather the parallel queue.
While the HTML event-loop is fairly convoluted, the concept of an event-loop itself is very simple — you can use it to model your concurrent Rust code, starting simple and making it as complicated as your problem requires.
The native thread: a simple building block.
To go about building our own event-loops in Rust, we need a simple way to represent a “running component”. The thread, a.k.a “native thread”, is a good bet.
Channels: your source of events
If threads can represent a “component”, how can we represent “events”, and “loop” over incoming ones? Enter — channels.
You can think of a channel as a thread-safe queue of events. Component can use those to enqueue events — in a form of message-passing — on each others event-loops. Note that “thread-safe” is only in the eye of the beholder here — from the perspective of a component’s event-loop, the others components are running concurrently — each component is actually running a single-threaded, sequential, event-loop.
Let’s take a look at a real-word concurrent component running it’s own “event-loop”: the background-hang-monitor in Servo.
As you can see from the snippet above, we have a BackgroundHangMonitorWorker “running until it drops” in a native thread.
We can also see that it makes available to the world a HangMonitorRegister, which appears to be a wrapper around a channel sender. It also seems that another channel sender, called constellation_chan is being shared with the worker. So far this follows the mantra of “communicate by sharing your sender”.
First, a word on what that component is supposed to be doing — its purpose is to monitor other components for “hangs”. Whenever one is noticed, it alerts another component called the Constellation. More details on it can be read in a previous article.
How do we monitor other components? Components send messages to the monitor, and the monitor can infer the activity/hanging of component from the messages it — or does not — receive.
The above is the event-loop of the monitor. A few features stand out — we’re selecting not only over our receiver, but also over acrossbeam_channel::::after, allowing the monitor to periodically wake-up even if no messages are received from components.
Also, we add a little while loop to drain the channel at each iteration — since we end-up doing some fairly heavyweight sampling of hanging components in some cases, the draining of the channel at each iteration hopefully reduces spurious hangs that could emerge from running a checkpoint at each iteration.
How do things look like from the perspective of a “monitored” component?
As you can see, this monitored component is also running an event-loop. Before blocking on the select , the component will notify the hang monitor the start of waiting for an event to come in — a bout of idleness following this shouldn’t be interpreted as a “hang”. When an event comes in, the component will notify the start of handling it by sending another message — if the monitor doesn’t hear back from the component for a while after, that will count as a hang.
This was an example of multiple components running an event-loop and communicating with each other by enqueuing events on each others event-loops. In this particular case, this was achieved using native threads and channels.
More complicated setups can be imagined. A component could own further threads of execution — a thread-pool for parallelizing some work, an async runtime for handling lots of concurrent tasks, and so on. Channels could be replaced by condvars and locks.
In all cases, the goal is using tools allowing you to clearly understand the logical constraints of the workflow at hand — going beyond the safety guarantees Rust offers.
Further reading:
- “Threads without Locks”, by Russ Cox. https://swtch.com/~rsc/talks/threads07/
For some practical examples of the techniques outlines in this post, see the two below posts:
