Rust: The story of closures

The Rust programming language introduced many leading concepts in the programming language design landscape. The most famous features are the borrow checker, the ownership management and the trait system.

However, the fantastic expressiveness of the closures is generally underestimated. Yes, from the day Javascript introduces closures to main stream programming languages, these days closures become one of the basic features for almost all modern languages. However, Rust’s ownership rules result in some brand-new observations to closures and its position in programming.

Let’s start the journey now.

What is closure?

In a modern programming language, closures are playing important roles to abstracting functionality. But what are they and why are they important?

From the very beginning days of programming, people feel that they need a way to abstract out functionalities, so they don’t have to write the same piece of code again and again. They invented two ways to do so, one is called “macros” that automatically repeat the code without needing you to write them, and the other is “functions” that executes a specific code sequences to jump into a piece of code and go back. We will not talk about macros in this article. Let’s look at functions.

From functions to closure

Roughly speaking, a function is a piece of code that receive something as its “argument” and produce something as the “return value”. Out of this definition, there are quite a few questions to ask:

• When I call the same function with the same arguments, should I get the same result? In other words, can a function maintain its own state? Some programming languages allow this through “global variable” or “static value”, which allows them to access some intermediate calculation results in the last call and use these information to produce different results.
• If we can have states, can the function callers have access to the state so they can prepare the state before the function being called? Before closures, the solution of this is either “no” or “yes, use global variables”. The later is less preferred as developer believed uncontrolled use of global variables lead to terrible design.

Closures answers the second question in another way. It let you create a “function” with a specific internal state, so when it is called, the internal state will be used to decide what it actually do. This separation of “state initialize” time and “execution” time lead to flexible design and much better abstrations.

How Rust is different?

When we have closures, we still have some questions to ask

• How many times a closure can be called? In many programming languages, the answer is “I am not checking it. Call it if you dare”. Unless the function is side-effect-free (like in Haskell), calling a function more than a specific time can lead to problems (double-free for example), but most programming languages just leave this to the developer.
• If multiple calls are allowed, can a closure get called simultaneously? Again, this is important in multi-thread programming. But before Rust no other languages attempted to address this in the language level.
• When called again, would the new call use a modified internal state, or roll back to the initial state? Programming language handles this varguly, but most answer is the former as this is the most programmers expect.

Rust distingush closures and put them into different types. So developer have to know what they can do with/within a specific closure. This is good thing!

The Fn trait family

In Rust, closures classification includes two dimentions, focus on two set of abilities.

How can I use a closure and what the closure can do with its state?

According to the different answers of the above, Rust defines 3 closure types:

• Fn trait closures let the caller call on only a shared reference without restrictions, but the callee will not be able to modify the internal state, and they cannot move things out of the internal state. All Fn closures can be used as FnMut or FnOnce.
• FnMuttrait closures let the caller call on a unique reference, and can only be called subsquently. The callee are free to modify the internal state, and move things out if they want, as long as the internal state remains ready for the next call. All FnMut closures can be used as FnOnce.
• FnOnce trait closures can only be called once. The callee can do everything with the internal state, including destroy it completly.

Can I call the closure with the exact same internal state?

A closure can also be Clone or Copy + Clone . So we have 3 cases:

• Closures that is Copy + Clone: they are almost as powerful as Fn, because they can be called without restrictions. Combine with FnOnce it gives the best flexibility: the callee have no restrictions on what they can do with the state as well (but the state have to be copy)! However, the trade off is that everything you did with the internal state in the callee code, will be lost as the next call will be working on the initial state. There is more to say about FnMut + Copy + Clone, as there will be two ways to call the object, one is using the initial state and the other using the modified state. I will catch up this later.
• Closures that is just Clone: Like the above, but you now have to manually call clone method on the closure to keep a specific state.
• Closure with none of the traits: the closure hold a unique state, and you cannot replicate it.

Deeper duscussions

One thing I didn’t mentioned, is that FnOnce + Copy implies Fn. If you can copy an closure, and were only given a reference to the closure, you can still call its copy. If you write your own closure objects, you will see that you can always implement Fn on a FnOnce + Copy object.

The relationship between FnMut + Copy and Fn is more complicated. Let’s checkout this example

fn call_fn_once(f: impl Copy + FnOnce()) {
    f();
    f();
}
fn call_fn_mut(mut f: impl FnMut()) {
    f();
    f();
}
// Demostrates how to turn `Copy + FnOnce` into `FnMut`
// It also works for native `Copy + FnMut`, but it never
// mutates its state.
fn as_fn_mut(mut f: impl Copy + FnOnce()) -> impl FnMut() {
    move || (&mut f)()
}
//Demostrates how to turn `Copy + FnOnce` into `Fn`
//It always behave the same as calling the `FnOnce`
fn as_fn(mut f: impl Copy + FnOnce()) -> impl Fn() {
    move || f()
}
fn test(f: impl Copy + FnMut()) {
    call_fn_once(f);
    call_fn_mut(f);
    call_fn_mut(as_fn_mut(f)); //calls the converted`FnMut`
}
fn main() {
    let mut i = 0;
    test(move || {
        i += 1;
        dbg!(i);
    });
}

If you run the code in Playground, you will get

[src/main.rs:24] i = 1
[src/main.rs:24] i = 1
[src/main.rs:24] i = 1
[src/main.rs:24] i = 2
[src/main.rs:24] i = 1
[src/main.rs:24] i = 1

Here, although call_fn_once and call_fn_mut both calls the closure twice, but they do different things. When called in call_fn_mut it will modify the internal state, so you get different results on calls. However, when called in call_fn_once its ability to maintain its internal state were forgotten, and so all two calls returns the same value!

So we concluded that with an FnMut + Copy, you have an even stronger object than a FnOnce + Copy as you can now precisely control whether you want to work on an earlier state or on a brand-new state.

Technically speaking, when you define your own trait that is natually a FnMut + Copy, you can convert it to another FnMut + Copy that do not modify the internal state.

However, Fn + Copy on the other hand, didn’t provide any additional abilities. The restriction on Fn keeps the internal state unchangeable. As a result, this will be exactly the same as FnOnce + Copy.

(Next: Closures’ secret life)

(That’s enough for today, but I have more to tell in another day)