Rust: Closures’ secret life

Go to the profile of Earth Engine
Earth Engine

Last time I talked about closures. When talk about different types of them, the traditional terms base on “trait”s have a disadvantage that the set of Fn, FnMutand Fnonce are inclusive sets, so it is not easy to talk about different abilities of them.

I instead introduce six different disjointed set of closures in terms of the restictions apply to the caller and the callee. According to a reader’s advise, I would avoid name conflicts with the Rust traits.

  • A closure that does not have a state is a function. They can be called multiple times in parallel or reentranted, and as they don’t have a state, making a copy of them is just like copying a single pointer. They can also be casted to the system function pointer type fn and being used like any other functions. We call this Function.
  • A closure that can be called multiple times in parallel or reentranted, and the subsequence calls on the same object can mutate its state. As a result of the state required to be copiable, the closure can also be copied to obtain a snapshot of the current state so it will not be be affected by any other calls. Let’s call it ClosureMutCopy.
  • A closure that can be called multiple times in parallel or reentranted, and the subsequence calls on the same object cannot mutate its state. As a result of the state required to be copiable, the closure can also be copied and all copies always have the same state. Let’s call it ClosureCopy.
  • A closure that can be called multiple times in parallel or reentranted, and the subsequence calls on the same object cannot mutate its state. The closure is not copiable, and there is no restrictions on the type of the state. Let’s call it ClosureRef.
  • A closure that can only be called subsequently without reentrancy. Every call to the same object will ensure the object remains valid before the next call. The closure is not copiable, and there is no restrictions on the type of the state. Let’s call it ClosureUniq.
  • A closure that can only be called once. The closure is not copiable and there is no restrictions on the type of the state. Let’s call it ClosureOnce .
Closure types and their abilities

Table of Rust closure types

(Note, the ClosureMutCopy closure type can implement FnMut in two different ways. Also, when I use the word “meaningless”, I mean you can change the state, but the change of state will be lost after the call)

But how do I know which one my closure belongs to?

When you write a closure in Rust, the compiler decide which one your closure falls into, and then implement the correct traits for you.

I have did some experiement and here is the summary table:

Closure forms and their types

Notes:

  • If the state is not copiable, “By mut” implies “By ref” and “By val” implies “By mut”, which means a closure uses the state by both ref and mut, it is counted as mut. But with copiable state, it is “By mut” implies “By val”, not the other way round. In other words, if the closure body refers to the state both by mutand by value, it is counted “By mut”, so it will be FnMutCopyor FnMut.
  • There is only one way to make a ClosureMutCopy. The recipient is:
  1. You must use a movekeyword.
  2. You must mutate the state in the function body. (Note, even you finally moved the state out, as soon as your function body mutates the state, your closure falls into this type)
  3. Your state must be copiable.
let mut s = 0;
let closure = move || { dbg!(s += 1); };

The resulting closure is actually implented FnMut + FnOnce + Copy, not implementing Fnas it implies a snapshot-taking FnMut, which will confuses the user. If the user want an Fn they have to wrap explicitly (see into_fn in the experiment code).

  • There is also only one way to make a ClosureRef:
  1. You must use a movekeyword.
  2. You must access the state only by reference.
  3. The state must not be copiable.
let s: String="1".into;
let closure = move || { dbg!(s.len()); };
  • There are two ways to make a ClosureOnce:
  1. It does not matter you use movekeyword or not.
  2. You must access the state by value
  3. The state must not be copiable.
let s: String="2".into();
let closure = move || { dbg!(s.to_bytes()); };
let s: String="3".into();
let closure = || { dbg!(s.to_bytes()); };
  • There are also two ways to make a Function:
  1. It does not matter you use movekeyword or not. However, as this moveis no-op, I strongly suggest NOT to use it.
  2. You must not have any state at all.
let closure = |v| v+1;
let closure = move |v| v+1; //Not recommended
  • There are three ways to make a ClosureUniq:
  1. You can either have a movekeyword, or have a copiable state, but not both.
  2. You must access the state by mutable reference.
let mut s: String="4".into();
let closure = move || { dbg!(s+="."); };
let mut s: String="5".into();
let closure = || { dbg!(s+="."); };
let mut s=6;
let closure = || { dbg!(s+=1); };
  • In all other five cases, you get ClosureCopy.
let s=7;
let closure = move || { dbg!(drop(s)); }
let s=8;
let closure = move || { dbg!(s.clone()); }
let s: String="9".into();
let closure = || { dbg!(s.clone()); }
let s=10;
let closure = || { dbg!(drop(s)); }
let s=11;
let closure = || { dbg!(s.clone()); }

How should I declare my functions’ arguments?

This is a good question! I saw some commonly used crates like hyper made bad decisions in this topic.

For example, look at service_fn function:

pub fn service_fn<F, R, S>(f: F) -> ServiceFn<F, R> where
    F: Fn(Request<R>) -> S,
    S: IntoFuture,

In other words, it expect the user to pass a Fntrait closure. However, the resulting type ServiceFn<F,R> basically is an implementation of Service that defined as

pub trait Service {
    type ReqBody: Payload;
    type ResBody: Payload;
    type Error: Into<Box<dyn StdError + Send + Sync>>;
    type Future: Future<Item = Response<Self::ResBody>, Error = Self::Error>;
    fn call(&mut self, req: Request<Self::ReqBody>) -> Self::Future;
}

The method call accepts a &mut Self! This means the service can modify its internal state on call. But unfortunately the closure we passed to service_fn cannot. This limits the way the user can use this function. If it were service_fn_mut and accepts FnMutinstead, problem resolved.

Guidelines

  • If you only need to call the pass-in closure once, accept FnOnce
  • If you need to call it multiple times but only subsequently, accept FnMut
  • If you need to call it in parallel or enable reentrancy, accept Fn
  • If you need to use the closure in complicated calling model, accept Fn + Clone(Fn + Copyif you need performance garantee)
  • If you need to create snapshots of the closure, yet expect the function to update its state, accept FnMut + Clone( FnMut + Copyif you need performance garantee). A direct call closure() will update closure, and (closure.clone())() will not update.

(Enough for today)