Image by andrey_l on Shutterstock

PART III

Fearless Concurrency with Rust

Asynchronous Concurrency

https://pragprog.com/newsletter/

In parts 1 and 2 of this series, we used multi-threading to hands CPU-bound loads. These are workloads that use all of the CPU-time you can give them, and can be sped up by dividing the calculations between cores in your system. Rust — and Rayon — provide great solutions for this type of workload.

In this part, we’re going to examine another form of concurrency: asynchronous code.

Note that asynchronous code does not mean multi-threaded. It’s quite possible to run an asynchronous program on a single thread and still benefit from its performance boost. Node.js is a great example of this approach.

Why Use Asynchronous Code?

Many workloads — especially in server programs — spend a lot of time waiting for other things. Your code may be waiting for a database to send it results, a file to open and process, or a network request to arrive. It’s certainly possible to spin off a thread for every request — and have that thread wait until a result arrives — but doing so can be very inefficient. Creating a thread requires server resources, and can be a relatively slow operation; the cost is measured in milliseconds, but on a heavily loaded server milliseconds can add-up quickly. You also can’t have more than 63,704 threads active at a time on Linux.

Asynchronous code uses a technique called green threads. Green threads (sometimes known as fibers) aren’t a full operating system thread: they represent a task and store a minimal amount of information about the task — and where it should send its result. Green threads may be (and often are) distributed between operating system threads — but they incur a significantly lower overhead than full threads.

How do Asynchronous Tasks Work?

An asynchronous task is a function marked with the async tag. When it executes, it runs until one of the following occurs:

• The task calls await on another asynchronous task.
• The task completes, optionally yielding a result.

Calling an async function doesn’t run it. It instead returns a Future. This represents a promise to coalesce into a result in the future — once you await the future’s result. When you call await, the future is added to your asynchronous green-thread scheduler, and the calling function sits idle until the future completes execution.

Consider the common scenario of a web user requesting frontpage.html from a server. If the requested page requires some content from a database, and some template application, the request goes through a number of steps:

Each await represents a time that the caller is idle: they are sitting doing nothing until another process completes. If frontpage.html is popular, managing this with a thread-per-task — with each thread spinning while it waits for a child task — quickly explodes into a huge number of threads. Worse, most of the threads would spend their life idling. Idle threads still have to be polled by the operating system scheduler — large numbers of idle threads can consume a surprising amount of CPU time!

Managing with asynchronous futures requires a significantly lower overhead as each task pauses (in a waiting state) until it receives results.

Build a Simple Asynchronous Web Service

Let’s use Rocket to build a simple web service that demonstrates asynchronous execution. There are several popular web frameworks for Rust; Rocket is a simple one to get started.

Create a new Rust project with cargo:

cargo init rocket-async-medium

Add Rocket to your dependencies in Cargo.toml:

We’re importing the json feature to help us mock (pretend to use) some database queries.

Now open src/main.rs and we’ll build a website. We’ll start with a main function, launching Rocket:

Rocket provides a convenient #[launch] macro that creates a main() function and performs initialization for you — before calling your rocket function. That’s all you need to start a basic web server with Rocket.

Loading an Index Template

We still need to define the index function:

There’s a few things to learn here:

• We use #[get("/")] to map the function to the website’s root.
• The function returns a RawHtml type — Rocket will accept a string, and ensure that the HTTP headers contain the correct content type.
• The function loads a file named index.html (we’ll write that in a moment), builds sections of the website with build_menu(), build_news() and build_footer() functions. It then replaces placeholders in the index file with strings returned from these functions.
• build_menu() and build_news() are asynchronous; build_footer is not. You can mix and match, so long as the calling function is itself asynchronous.

That’s a good representation of a typical templating process for a server-rendered website. All that remains is to build the content.

Make a new file named src/index.html and insert the following HTML into it:

If you’re familiar with HTML, this is very straightforward:

• The file is simple HTML.
• A style-sheet is defined inline, providing visual layout for the content.
• Three placeholders are present: !!MENU!!, !!NEWS!! and !!FOOTER!!. In the index() function above you are replacing these markers with site content.

Loading the Menu

Let’s build our menu. The menu will be a simple bar, loaded from a file. Create another file, named src/menu.html:

It doesn’t get much simpler than that! In src/main.rs , create a function to load it:

Again, this is quite simple: it loads the menu from disk and returns it as a string. Notice that it’s asynchronous — we’re using Tokio’s filesystem functions. This approach ensures that the function will quietly wait if there is any delay accessing the file — maybe because the server is busy.

Loading a News Feed

Now, let’s add some site content.

We’re going to pretend that we’re loading a news feed from a database. Instead of asking you to install a complete database system, we’ll store the current news feed in a JSON file. Create a file named src/news.json and paste some news into it:

This is a simple JSON file: it contains an array of objects, each of which contains a title and a summary.

Now go back to src/main.rs and add a function to asynchronously read this data:

We start by defining a structure that matches our news-feed format. Decorating it with #[Deserialize] allows Rocket (which embeds Serde) to decode JSON into NewsItem structures.

The build_news() function loads the news-feed file, and calls Serde to decode it. It then iterates the news array, using map and format! to transform it into HTML. The HTML for all items is then concatenated together with fold.

Finally — the footer. We’re going to always return the same string, and since we aren’t waiting on anything else we’ll make it a regular function:

Congratulations — you now have all of the elements required to run your webserver. Type cargo run and navigate to http://localhost:8000/. You should see your news site:

To recap, requesting this website:

1. Your web browser sends a request to 127.0.0.1 on port 8000.
2. Rocket receives the TCP connection and parses it.
3. Since “/” matches the route for index, Rocket calls your index function and awaits a response.
4. index calls fs::read_to_string to load the index.html template file and awaits a response.
5. index receives a response and calls build_menu (awaiting a response).
6. index receives the menu template and calls build_news (awaiting a response).
7. index combines the results, modifies the template and returns the HTML response.
8. Rocket re-awakens with the response and sends it to your web browser.

This is a lot of steps — but it’s very efficient. Each task pauses when it isn’t needed, consuming very few resources. You could easily serve hundreds of thousands of customers with your exciting news!

Improving Concurrency with Join

You may have noticed that some parts of index could run in parallel. The menu and the news don’t depend upon each other. Why not wait for both at once — Tokio can divide tasks between threads (transparently), so you may get a performance increase.

The join! macro executes all three futures (loading the template, building the menu and news feed) concurrently — and doesn’t wake up until all three have returned data.

When Tasks Block

All of the functions we’ve called have been async — they cleanly wait for another task to complete. Occasionally, you need to wait for a non asynchronous task — maybe a CPU bound task, or a device that doesn’t provide an asynchronous interface.

Directly calling blocking tasks inside your asymmetric function is a bad idea: the whole “future” pauses while it waits for a result. This can be very bad for performance; if you are running a single-threaded environment it can stall execution completely.

When you can’t avoid a blocking task, you can use Tokio’s spawn_blocking function as follows:

Using spawn_blockingwill spin your task off into its own thread — and pause your future until the thread returns. This approach keeps the other tasks running smoothly while your function executes.

Wrap Up

We’ve reached the end of the “Fearless Concurrency with Rust” series. In these three articles, we’ve covered:

• Using Rust’s threading primitives.
• Using Rust’s safety guarantees to avoid a bug that crept into similar C++ code.
• Using Rayon to make CPU-bound multi-threading easy.
• Using asynchronous execution in servers and other processes that spend most of their time waiting for other systems.

Rust lives up to its promise of fearless concurrency and offers a variety of methods to fit your needs. Please share in the comments how you are using concurrency in Rust.

Herbert Wolverson

Are you a Rust fan? You can save 35 percent on the ebook versions of Herbert’s books with promo code rust_2022 now through October 15, 2022. Promo codes are not valid on prior purchases.

Rust Brain Teasers

Hands-on Rust