Understanding the Execution Models of async JavaScript and React

You may have heard somebody mention React Fibers recently, and even more recently that Suspense lets you use async functions with React. Read on for a dive into what an async function is, and what they have to do with Fibers!

Async/Await

In 2017, JavaScript gained the async/await syntax, on top of 2015’s generator functions (function*/yield). This section explores what these language features do, and how they relate to React’s rendering model.

Function Calls

Let’s take a step back and discuss normal function calls. Take the following pair of functions, pretending that sleep(ms, v) is a function that takes ms milliseconds to return v:

function outer() {
  const y = inner();
  return y + 1; // <1>
}
function inner() {
  const w = sleep(1000, 4);
  return w + 2; // <2>
}
outer()  // after 1 second, returns 7

Every time a function is called, its stack frame is added to the stack. This consists of the function’s local variables along with the return pointer (r.p.) which indicates where in the caller to return to. outer returns to the event loop (labelled &loop), and &<1> refers to the line marked with // <1>.

In addition to the return pointer, there is also a global stack pointer (s.p.) that tracks the location of the top of the stack. All local variables are referenced by their distance to the top of the stack, for example references to y might be represented as sp — 4 bytes. The consequence of this is that we can’t run two functions concurrently, because both expect the stack pointer to be pointing to the top of their respective stack frame. From a practical perspective, this means the browser will hang while any function is executing.

Callbacks

To avoid this, JavaScript uses callbacks to process results from long-running functions. In this case, control returns to the event loop immediately, and the function is called at the discretion of the event loop, leaving the loop free to perform other tasks like handle user input in the meantime. For example, instead of sleep(1000, 4), we would call setTimeout(() => 4, 1000). However, this is still not quite what we want — outer would call inner, which would call setTimeout, which returns a number representing the timeout object. We would then perform our arithmetic on this number, which is probably not 4. Then, one second later the event loop would call our callback, and throw away the return value (which is the 4 we were after).

Promises

To solve this issue, Promises were introduced. A Promise represents a value which is not yet ready. In the meantime, callbacks can be registered with .then to be executed when the Promise completes, which occurs when its (internal) resolve method is called.

Breaking code up into callbacks (i.e. continuations) is not an intuitive way of structuring functions:

function outer() {
  const y = inner()
  return y.then(value => value + 1);
}
function inner() {
  const w = sleep(1000, 4)
  return w.then(value => value + 2);
}
function sleep(ms, value) {
  const timeout = new Promise(resolve => setTimeout(
    () => resolve(value), ms
  ));
  return timeout;
}
outer()  // Returns a Promise; after 1 second, its internal value will be 7

Every promise contains its value (which might not exist yet) and an internal list of callbacks that will be run when it resolves and its value is set.

Here, we create a new Promise w, and create a timeout that will resolve the promise after 1000 ms, setting its value to 4. Before the timeout expires, we create a chain of child promises using .then(fn). Each time this method is called, it creates a new Promise, and adds a callback to the parent’s internal list. This callback calls fn on the parent’s value and resolves the new Promise with the result. As a result, when the first promise in the chain resolves, all of the children follow one after the other, until we eventually hit the last promise in the chain and set its value.

Async/Await

Going back to our first example, what we really wanted was to suspend the execution of our current function when we need a value that is not yet ready, and then continue on when it is available. async/await lets us do exactly that — we still need to wrap the top-level call in a Promise, since the entry point is a non-async function (until top-level await is ready):

async function outer() {
  const y = await inner();
  return y + 1;
}
async function inner() {
  const w = await sleep(1000, 4);
  return w + 2;
}
function sleep(ms, value) { /* as above */ }
outer()  // Returns a Promise, as above.

Generators

Ignoring a lot of error handling, and assuming we only yield when absolutely necessary (i.e. when we have a Promise we are waiting for), we can replace async/await with generators and a helper function:

function* outer() {
  const y = yield* inner();
  return y + 1;
}
function* inner() {
  const w = yield sleep(ms, value)
  return w + 2;
}
function sleep(ms, value) { /* as above */ }
// Helper function to drive the generator
function step(g, next) {
  let {done, value} = g.next(next)
  // We always `yield` a `Promise` and `return` a concrete value
  return done ? value.then(next => step(g, next)) : value;
}
step(outer())  // Returns a Promise, as above

How are these functions different from our original implementation? When a generator hits a yield (or yield*) point, it saves its internal state and instruction pointer onto the heap and returns to its caller a function that resumes from this saved state (next)[1]. We can use this to suspend the execution of our function whenever we are waiting for a Promise, and then resume where we left off when it is ready — the step function does just that, by registering a callback that resumes the generator whenever it yields a Promise.

We’ve achieved our original goal here — the event loop is in control while we’re waiting and can handle user input, and our functions are still written in more-or-less the same style. There are some limitations, for example generators can only yield directly to their callers and as such can suspend only themselves, not any functions further up or down the call stack. In our example, inner suspends itself, and control returns to outer. outer sees that the yield* statement has returned an incomplete generator, and suspends itself, returning control to step.

Conclusion

Generators in JavaScript are resumable functions, a.k.a. coroutines. Specifically, they are asymmetric and stackless. Asymmetric coroutines maintain a caller/callee relationship, and can only yield control to their callers, while the rarer symmetric coroutines are capable of jumping to an arbitrary other coroutine (e.g. Python’s greenlets). Stackless coroutines can only suspend themselves — in the example above, inner yields to the event loop via outer, instead of directly suspending the entire stack as in stackful coroutines (e.g. Ruby Fiber and Lua coroutine).

React

NOTE: This section is about the internals of React, and some discussion about why it might be designed that way. None of this knowledge is necessary to use React.

What is React?

React is a JavaScript library for building interactive user interfaces. A UI written with React takes an input set of properties and renders to a tree of elements (usually HTML elements). In response to a user interaction, the UI can be re-rendered from different input properties. React’s responsibility in this is to render a UI given an input state as quickly as possible. This happens in two stages:

1. The reconciler figures out the minimal set of changes necessary to make the current page be equal to the output of the render
2. The renderer applies those changes using e.g. browser APIs, like Node.appendChild

For now, we’ll only consider function components, and focus on the reconciliation stage. React is most commonly used via JSX, an HTML-like shorthand:

<div {...props}>{children}</div> // is like:
React.createElement("div", props, children);
<Foo {...props}>{children}</Foo> // is like:
React.createElement(Foo, props, children)
// Usage:
ReactDOM.render(
  <Foo {...props}>{children}</Foo>,
  document.getElementById("root"),
)

The key point here is that <Foo /> does not actually call the function Foo, it just passes it to React.createElement. ReactDOM.render tells the reconciler which React Element to render, and which DOM Element it should be rendered into (from now on, Element refers to React Elements, unless stated otherwise). Unfortunately, the reconciler has a lot of work to do, which with large updates can lead to undesirable freezing of the browser while it figures out what the renderer needs to do.

Fibers

The aim of fibers is to break up a large diff operation into small chunks, returning control to the browser between each chunk so that the application can remain interactive. Let’s walk through a render of a simple set of components:

const Inner = ({text}) => {
  const [bold, setBold] = React.useState(false);
  // return a host component
  return <span
    className={bold ? "bold" : "normal" }
    onClick={() => setBold(bold => !bold)}
  >
    Go {text}
  </span>;
}
const Outer = () => {
  // return a host component
  return <div>
    // with some function components as children
    <Inner text="left" />
    <Inner text="forward" />
    <Inner text="right" />
  </div>;
}
ReactDOM.render(<Outer />, document.getElementById("root"));

This will render to the following HTML:

<div>
  <span class="normal">Go left</span>
  <span class="normal">Go forward</span>
  <span class="normal">Go right</span>
</div>

When we call ReactDOM.render, this triggers the creation of a graph of Fibers, one for each element (both function components and host components). Each Fiber contains information necessary for rendering (e.g. its props), but also one to three links to other Fibers — return to its parent, child to its first child (if it exists) and sibling to the next sibling (if one exists). Fibers also store information about queued updates, memoized results and so on, but these attributes are not relevant to control flow through the graph.

To do this, we take a handle to the <div id=”root”/> DOM Element (our host container) and wrap it as the fiber root. Then, it calls updateContainer which calls into enqueueUpdate, storing the element to render on the Fiber and then scheduleWork to trigger the work loop.

Apart from the initial render, the reconciler is also activated in response to a Hook (e.g. calling setState from [state, setState] = useState(…)). When the component is rendered from the parent’s beginWork call, setState gets a reference to the Fiber used to render it. When setState is called, the update is added to the Fiber’s update queue, and the root Fiber is scheduled and we enter the work loop (via a callback on the main event loop).

Work Loop

The work loop is responsible for processing individual fiber nodes in the correct order until a deadline is reached, at which point it bails out and returns control back to the browser’s event loop, before resuming with the next section of the fiber. It runs work = performUnitOfWork(work) in a loop, which returns the next unit of work to perform, until the timeout expires or it returns no work.

performUnitOfWork(work) in turn runs next = beginWork(work), which calls renderWithHooks(work) — this is the part where the user code is actually entered (i.e. the Element’s type is called on its props). The children are passed to reconcileChildFibers, and the result is assigned to work.child. If the render method returns a single child Element, the child is a single fiber, otherwise it’s a linked-list thereof. Finally, the first child element is returned as the next unit of work.

At this point, the work loop has the option of bailing out if the browser has an update to render. Otherwise, it will press on…

Assuming we’ve reached the deepest point in our tree (because there are no further children, next is null), we now call completeUnitOfWork(work). This will call completeWork on the current fiber and, unless completeWork returns more work, all of its parents until it meets one with a remaining sibling, at which point it will return the sibling back to the work loop. completeWork usually returns null, except in the case of Suspense Components — it may re-render the component with a new expiration time, triggering the fallback content.

Reconciliation

Fibers have given us a way to process the tree of components so that it can be interrupted frequently — namely between an element and its children, or between siblings. Now, what actually happens as each component is visited?

As mentioned above, the first time a component is visited is when a fiber begins its work on it. If the component is a React component, this phase calls its children’s render functions and reconciles them. This boils down to making sure the parent fiber has the correct number of child fibers of the correct type, deleting old ones and creating new ones from the result of the render function as needed. The second time a component is visited is when all of its children have been processed and it can be completed. At this point, any fiber corresponding to a host component or text is marked as having updates if necessary, while function components don’t require any further processing.

Finally, when the work loop completes we enter the commit phase, where the fibers are processed once more and their updates are committed to the DOM.

Suspense

Suspense is a new feature in React, built on top of Fibers. Its aim is to allow a component to indicate that it is not ready to render, and then trigger a re-render when it is, allowing developers to use async functions to fetch data for a component. However, components are still synchronous functions — to achieve this, React requires the async function to be wrapped in createResource. When the async function returns a Promise (strictly, a thenable), the resource throws the Promise, aborting evaluation of the render function. The Promise is caught by the work loop and has a callback registered to retry the render when it completes, while the work loop continues from the nearest <Suspense /> component.

Conclusion

React components form a tree, and using a recursive function to walk this tree is a natural first step, making use of the language’s implicit stack to track progress. Instead, React uses linked-lists (linked graphs? linked trees?), called Fibers. The next section will discuss some advantages of this decision.

Fibers vs Generators

Now that we’ve dug into both async/await and React, let’s dissect some of the design decisions in the latter. The previous synchronous reconciler used the function call stack to keep track of state as it recursively walked the component tree. This is straightforward and efficient, but the downside is that it is not possible to interrupt rendering and then continue later. It would be possible to interrupt from the middle of a render function by throwing an exception or returning a special error value, but the state of the in-progress work would be thrown away as the stack is torn down.

Using generators would be one option here — every component would yield if it is not ready but decides to suspend itself, and then finally return the completed reconciliation. The first issue is one of performance, as JavaScript generators can only suspend themselves (they are stackless) — in a deep tree you would need to push and pop each intermediate generator to and from the stack. An alternative approach would be keeping track of a manual stack using an array, pushing and popping elements as necessary.

The second is that it is not possible to snapshot the state of a generator[2]. One implementation might be that progress through a Fiber’s children is represented by a generator. So above, Outer would be a generator, that does a yield between its children. Suppose we have just finished reconciling “forward”, when the timer runs out and we return control to the event loop. At this point, the user clicks “forward”, and we return to the event loop. The generator implementation has no way to go back to the previous yield while preserving the state of its children, while in the case of Fibers we simply need to change the sibling pointer of “left”.

In summary, Fibers are a flexible way of implementing both resumable and cacheable function execution. They allow pausing the reconciliation of a React component, re-winding to a previous state and modification of a running tree. Understanding the structure of a Fiber and how it corresponds to the tree of components also gives some insights into possible performance pitfalls — although inserting into a component’s children only triggers O(1) DOM manipulations, the overall operation is still O(n), as the linked-list of children must still be walked. Their flexibility opens up a lot of opportunities — we’ve seen Suspense and Error Boundaries already. Stay tuned for more on UI frameworks and their implementation.