Om/Next: The Reconciler

The Reconciler is the heart of Om/Next. If you want to understand how Om/Next works, how its pieces come together, understanding the reconciler is a great place to start.

Note: This post is about architecture, not API. For details on how to use Om/Next, see the official or unofficial wikis, or the sample application. However, I hope this will be useful to help people get oriented to the API.

Reconciliation

The reconciler manages or “reconciles” data between React, your local client application, and the server. It provides a model for keeping your views in sync with the local application state and the remote server, and for responding to updates coming from the views.

Reconciler sits between React and local&remote state

In this post we will see how the model handles the following common cases:

• Updating views to changes in the application state
• Transacting new data into the application state
• Initial loading of data for the whole application
• Optimistic writes (immediately updating application state, but also going back to the server)

The Reconciler Knows All

The reconciler contains references to all the important components of Om/Next.

Reconciler knows about and orchestrates Om/Next subsystems

• The indexer. Contains pointers to every mounted Om/Next React component, and indexes them in various ways. Allows the reconciler to find the components that need to be updated in response to new data.
• Parse. A user-supplied function that resolves queries and transactions. Its outputs are sent to react components, and to the local & remote data sources.
• A set of remote targets for message forwarding
• Pointers to the local and remote data sources.
• Queues for updates that need to be reconciled.

Updating Views

Updating views in response to changes in local application state is the base case. Other scenerios will fall back on this for their final update step.

Updating the application state can happen in a number of ways: directly with swap!, via Om/Next’s transact!, or from data returning from the server. Regardless of their source, updates to application state will also place an entry in the reconciler’s queue, notifying it of work to do on the next render loop. Multiple updates may occur within a single frame, but reconciliation only happens at most once per frame.

Updating the app state will queue the reconciler for action

The reconciler is invoked on the next frame, and notices there is work in the queue. First, it uses the indexer to figure out what components need to be updated, based on information in the queue.

Indexer is used to figure out which components need updating

Next, the reconciler asks those components “what data do you need”? Om/Next, similar to Relay, lets components declare their data dependencies. In Om/Next, this description is plain EDN data similar to Datomic’s pull syntax and is called the query. You can think of the query as specifying the keys of a desired hashmap, and satisfying the query would be returning the populated hashmap containing those keys and their corresponding values.

The component’s query is handed to parse:

Reconciler gets query from component and passes it to parse

In the base case, parse satisfies the query using data from the local application state. The query’s result is stuffed back into the React component and the component is updated.

Parse resolves query using app state and results are stuffed into component

Now the view has been updated to reflect the updated application state.

Transacting changes

How does a component trigger an update to local application state?

Typically, an event handler in the component will invoke transact!, passing an an argument another kind of data-oriented message, known in Om/Next as a mutation.

Mutations are also passed to parse:

Components transact! mutation messages. Parse receives them.

The result of parsing the mutation message is to generate another update, which will update the application state and populate the queue:

Now we are back the the base case, and the reconciler will apply the updated app state to generate the updated view.

Going Remote, Message Forwarding, and Parse

In the two examples above, parse resolves query and mutation messages using the local application state. But how can we query data from the server? And what if we want to persist a mutation to the server?

The solution is to let parse forward messages it can’t handle locally to a remote server.

The way this works is that we specify to the reconciler a number of remote targets. These are just keywords that represent one or more remote services (for instance, one for static content that is HTTP-cachable and one for dynamic content that isn’t). The nil target represents the local application context.

Parse can forward messages to a number of remote targets

Given a message, parse can be invoked once for each target, which is passed as an additional argument to parse. The return value of parse is either applied locally (in the case of the nil target), or forwarded to the corresponding remote service. How messages are handled is application-specific, determined by the logic of the user-supplied parse function.

This architecture makes it very easy to express patterns like “I don’t know the value of this key, go ask the server,” or “I think the value is this, but ask the server anyway,” or “Update the local state with this function, but also pass the mutation to the server.”

Initial Load

One feature of Om/Next is that it minimizes the number of requests to the server. In particular, the initial load of data is done in a single request, thus minimizing latency (and tedious loading logic.)

Initial load can be achieved in a single request because Om/Next query messages are recursive. A component describes its data dependencies, including those of its own subcomponents.

The query of the root component in your UI will contain a tree of queries of all its subcomponents, its subcomponents’ subcomponents, and so on — and therefore contain the grand query necessary to populate the entire UI.

To populate the UI, then, we just need to satisfy the query of the root component. But there is a slight problem: how can we get the query of the root component, if we haven’t instantiated it yet? The simple solution of Om/Next is to put a static method on the React class, that lets us get the query for that class without needing an instance of the class.

The initial query comes from a static method on the root UI component class, asking for data on behalf of the whole UI hierarchy

Once we get the root query, parse figures what parts can be satisfied locally versus what needs to be forwarded to the server. The data retrieved locally is shown immediately; data coming from the server will be shown when it comes back. The initial application state will typically be a skeleton, enough to show some UI and perhaps a loading indictor while the rest of the data is being fetched via the forwarded query message.

Parts of the initial query may be satisfied from the local app state, and a React Tree is instantiated. In the meantime, the rest of the query is sent to the server to get the remaining data.

When the data comes back it fully populates the local application state, and triggers an update, returning us back to the base case:

Query results returning from the server trigger an update of the UI

Optimistic Update

Tweaking the behavior of parse also allows us to easily achieve optimistic updates.

Recall that parse lets us specify behavior for local and remote targets.

Mutation message transacted by component, received by parse

When a component transacts a mutation message, parse evaluates the message against all the targets, local and remote. Parse may determine that the mutation message should have some local behavior (apply a state change locally) and some remote behavior (forward the mutation message to the server.)

The local mutation is applied immediately, and triggers an update of the UI. At the same time, the message is sent to the server.

Mutation message can result in both an immediate local update, and a mutation message forwarded to server

Just as in the previous case, the result of the server request comes back and triggers another update round.

Server-side mutation returns to client and triggers an update

Mutation messages include not only the mutation instruction itself, but also what query should be re-run in order to display the result of the mutation; therefore the results coming back from the server look just like query results, and update the local application state.

Summary

The reconciler sits between your views and your data. It connects Om/Next React components with local data and with the server. When query and mutation messages come from components, it uses parse to figure out how to respond.

Parse is a user-supplied function that encodes the application-specific logic about how to respond to messages. It provides a simple and flexible system for balancing or delegating behavior between the local application and the server. Parse makes it easy to express desirable patterns, including bulk loading of initial data and optimistic updates.

I hope this has been useful. Have fun with Om/Next!