Let’s Build a UI with Qlkit and Graph Queries!

by Conrad Barski, author of “Land of Lisp” & CEO of ForwardBlockchain.com

Qlkit Github Page

Graph query language n. A language for querying servers (and other types of remote entities, like databases) that has the shape of a graph. Examples include Facebook’s GraphQL, Netflix’s Falcor, and Cognitect’s Datomic pull syntax.

In the last few years many large companies have started replacing RESTful server APIs with new types of APIs based on graph queries. However, graph queries can also be applied to other common development tasks, such as UI development.

Note: Graph queries are often confused with graph databases, though these are two completely unrelated ideas: With a “graph database,” the data has the shape of a graph. With a “graph query language,” it’s just the query that has the shape of a graph — the data itself still retains the traditional, flat relational model that you are most likely very familiar with, both in the client browser and the server.

The most well-known existing projects that have applied graph queries directly to UI development are Facebook’s Relay framework and the Om Next framework. Today our company released Qlkit, a ClojureScript web development framework with similar abilities that is heavily influenced by the design of both Om Next and Relay.

ClojureScript is a dialect of Clojure that compiles into JavaScript and can be used in lieu of JavaScript for writing UI browser code.

Where our framework differs from Relay and Om Next is that qlkit is meant to be, above everything, lightweight, unsurprising, and minimalistic. We have made an effort to maintain all critical features of these other frameworks (and improved on some of the features) while keeping the size of the library as small as possible: Qlkit consists of around 300 lines of source code, while offering everything that is needed for a modern UI app: a transactional client state model, as well server-side functions for query resolution —Additionally, rich component rendering can be added as well with a small accessory library.

Qlkit attempts to meet the “simplicity” guidelines from the seminal talk Simple Made Easy in being a single-purpose tool that can be easily composed, understood, and modified.

In this post, I will walk through a browser-based app that has a UI and see how simple code can be if all UI management tasks and all client-server communication are handled with a single uniform graph query language.

While this post walks through an example written in the qlkit query language, the ideas in this post can also be applied to applications built on Om Next, Relay or Falcor.

A Basic Todo-List App

For this example, we’re going to walk through the entire code for a basic todo list app, available to try out here. This app has the following features:

• It allows you to add new todo list items
• It allows you to delete todo list items
• It persists the items on a server
• It implements basic multiuser safety (i.e., items created on different browser clients are assigned temporary IDs that are replaced by server-generated IDs once persistence has completed)
• It leverages modern UI components for a modern appearance (in our case, we will leverage the Material-UI component library)

Self-Contained Components Through Graph Query Declarations

Our todo list app consists of two qlkit components, named TodoItem and TodoList. Here is the code that implements the TodoItem component:

To declare a component in qlkit, we need to (at a minimum) declare its query ➊ and also a React-like render function ➋ that outputs a description of the UI for the component.

A TodoItem component in this example app will need to know two different things about a todo item: its text, and its ID, which is why the component requests these two pieces of data as “inputs” into the component ➊.

The render function receives two important arguments:

1. The attributes of the component, which will be populated with the data the component requested. In this case, the attributes are the text of the component and the component’s ID.
2. The component-local state, which is a sort of “scratch pad” available to each component for writing the temporary state. We will discuss how to handle other, more permanent forms of state shortly.

The render function simply returns an EDN data structure that describes the HTML for the contents of the component. In this case, the component just returns a standard HTML li item ➌. However, you may notice that this list item also has some nonstandard attributes, such as :primary-text and :right-icon, which are made possible by extending qlkit through the qlkit-material-ui library, which transparently overrides the default li item with the more sophisticated Material-UI ListItem component.

(Note to advanced readers: The opinionated rendering algorithm and “DOM element sugar” demonstrated in this todo list app are completely optional and reside in separate packages named qlikit-renderer and qlkit-material-ui. Please take a look at this alternative example and you will see it is equally straightforward to build an app without this “fancy stuff” and which just uses default React rendering and sablono.)

Because data is supplied to this component in a standardized way (through the query declaration) and data is output from this component in a standard way (the EDN data structure returned via the render function), it is very straightforward to unit test qlkit components: You simply need to compare inputs with outputs in each test. The strong isolation and testability of qlkit components persist even as components are nested.

(Another noteworthy piece of code in the declaration of the TodoItem component is the declaration of the on-click event ➍ that is triggered when the user deletes the item. Ignore this code for now; we will discuss event handling in qlkit later.)

The TodoList Component

Here is the component that manages the entire TodoItem list and also lets users create new items:

Most of this code (again, besides the events declarations) should be straightforward for anyone familiar with React. The main novel aspect of this code is that it implements nested components (i.e., a single TodoList can contain many TodoItem child components). Three lines in this component have logic in them that connects the TodoItems to the TodoList:

• In the query declaration it embeds the query of the child component ➊ — after all, a todo list component shouldn’t have to “know” what fields each todo item needs; that’s the job of the component that manages individual items. This means that the child component’s data needs are addressed in a way that’s invisible to the parent component.
• In the todo list render function, the attributes passed in will contain not only the data for the parent component, but also the data that the child component requested: The qlkit-todo/todos value we’re extracting here ➋ contains both the data for the todo list and for all the todo items.
• When the todo list returns its HTML elements, it can simply refer to the child component TodoItem by name, and qlkit will appropriately insert the child component ➌. All the parent component is responsible for is forwarding the attributes to the child (i.e., taking the data collected here ➋ and passing it on to the child component here ➌ via the todo variable.)

By design, graph query languages are nestable: The final query at the top ➊ of this todo list component will look as follows:

 [[:qlkit-todo/todos {} [:db/id] [:todo/text]]]]

As you can see, this final query combines the data needs of both the todo list and all the todo list items into a single query that essentially says “I need the whole todo list, and for every item in the todo list, I need to know its ID and its text.” The empty map {} indicates that there are no parameters in the query for todo items (i.e., that we want all todo items). (A detailed description and full specification of the query language are included in our GitHub repo.)

Adding a New Item to the Todo List

OK, we’ve seen how a UI is generated with qlkit. Next, let’s take a look at how events are handled. To create a new todo item, we type the text into the input field, then store it as a permanent item when [enter] is pressed. Let’s review the code for TodoList to see how this happens:

In the on-key-down event for the input field ➊, we check if the key code is 13 ➋ (the [enter] key), and in that case, we need to perform two actions:

• We trigger an application-wide transact! event ➌, which will (in subsequent steps) place the new item into the application-wide client state, as well as into server storage. The arguments to the transact! event are similarly in the form of a graph query: The query performs a mutation called :todo/new! (the exclamation mark indicates that it’s a mutation), which qlkit will forward to query parsing functions, as we will see shortly.
• Every time a new item is created, we need to clear out the input box again to ready it for another item. We accomplish this by erasing the text from component-local state here ➍. This will automatically clear out the component, based on React’s “controlled component” mechanism.

Here is what the graph query will look like for the part of the transaction that generates the new todo item:

[:todo/new! {:db/id     "ff81aa5b-9cde-4399-a2fd-d648e8ef9503"
             :todo/text "Iron the curtains"}]

In this query, the query command of :todo/new! is followed by a map of parameters that describes things about the new todo item we’re adding. To add a new todo item, we need to first have a temporary unique identifier for the todo item, which is stored under the key :db/id here. (This ID will be replaced by a friendlier long-term identifier once we’ve managed to synchronize with the server). Additionally, we pass in the text for the todo as :todo/text.

So… what happens to this transaction once it’s submitted? To find out, let’s look at how a qlkit component (in this case, our TodoList component) gets mounted to the DOM tree:

As we mount the component to the DOM tree, we have the ability to set various options for how this component is connected to the outside world. First of all, we link it to the application-wide state atom that will represent the main client-side application state, in this case a Clojure atom named app-state ➊. Also, we assign a remote handler function ➋ — this is a function that we will see later that is responsible for forwarding graph queries to the remote server, the final storage destination for our todo items.

The last option we set as we mount our component is to link it to a bunch of parsing functions that we’ve written ➌. These parsing functions parse all the graph queries that originate from our components — this includes queries generated by the TodoList, the TodoItems, and whatever other components our app requires. Qlkit allows you to create four different types of parser functions on the client side:

• read parsers: These handle queries that read information out of the client-side application state.
• mutate parsers: These handle queries that modify the client-side application state.
• remote parsers: These parse queries to determine whether they require information (or involve mutations) that need to be communicated to the remote server.
• sync parsers: Once a server has engaged with a query, these parsers subsequently update the client-side application state based on the results received from the server.

Writing a Mutate Parser

Here again is the query we’re currently working to satisfy to create a new todo item:

[:todo/new! {:db/id     "ff81aa5b-9cde-4399-a2fd-d648e8ef9503"
             :todo/text "Iron the curtains"}]

We’ll want to update the client-side application state to add this todo item, even before the server has processed the new todo item. This is good practice, since it will give the user rapid feedback, regardless of what happens to the item on the server, which we will deal with later. (Updating the UI before a transaction has been fully processed is commonly referred to as an “optimistic UI update.”)

As we discussed, updating the client state based on a mutation query is accomplished by defining a mutate parser:

This parser receives three arguments: The query, an environment variable, and a reference to the state atom ➊. Right away we extract the parameters (i.e., theparams variable) from the query term, which will be found in the second slot. Next, we pull the todo item’s ID out of the parameters ➋. Finally, we update the application’s state by inserting the new todo item into it ➌.

With qlkit it is recommended that you keep your client application state flat and normalized, just as if it were a traditional database. Here is what the structure of the app state in our sample app would look like before we add the new item:

{:todo/by-id {0 {:todo/text "Walk the dog"
                 :db/id     0}
              1 {:todo/text "Pay the bills"
                 :db/id     1}}}

As you can see, the state contains a single item called :todo/by-id, which holds the collection of todo items, keyed by their ID. After the mutate parser executes, the state will look like this:

{:todo/by-id {0 {:db/id     0
                 :todo/text "Walk the dog"}
              1 {:db/id     1
                 :todo/text "Pay the bills"}}
              "ff81aa5b-9cde-4399-a2fd-d648e8ef9503"
                {:db/id     "ff81aa5b-9cde-4399-a2fd-d648e8ef9503"
                 :todo/text "Iron the curtains"}}

The new todo is now in the local state as we wanted, though it still uses a lengthy, collision-resistant temporary ID.

Rerendering the UI

After executing this parser function, qlkit will trigger a rerender of the entire component tree. (This is one significant difference between qlkit and Om Next, since Om Next goes to considerable effort to optimize for only partial rerenders of the component tree — However, React will still optimize the rerendering in qlkit in the usual way, so qlkit apps still have performance comparable to vanilla React apps.) To do this, qlkit will refetch the data for the components from the local client state, which, if you remember, is defined as the following query:

[[:qlkit-todo/todos {} [:db/id] [:todo/text]]]]

In processing this query, it will trigger the following read parser, which as we discussed is responsible for resolving reads of the local state:

Like components, parsers can also be nested: Just as we don’t want the TodoList component to have to worry about the “dirty details” of individual todo components, we don’t want the :qlkit-todo/todos parser to have to worry about the “dirty details” of parsing the query for an individual todo item.

As you can see, read parser arguments ➊ look the same as mutate parser arguments, with the sole exception that the state is passed in as a value, not a reference to the state atom: This ensures that a read parser doesn’t accidentally mutate the state.

Just as with the mutate parsers, read parsers support parameters in the query. In this case, the parser for :qlkit-todo/todos is written to handle both default and parameterized versions of the query:

;;default query
[[:qlkit-todo/todos {} ...]]]
;;parameterized query
[[:qlkit-todo/todos {:todo-id 123} ...]]]

In other words, this parser can address the query “Give me all the todo items” as well as the query “Give me the todo item with ID=123.” We will see later why it’s beneficial to handle both cases. To see if the :todo-id was provided, we check to see if it was defined in the parameters ➋ ➌. If the todo-id was specified, we hand over control to other parsers that know how to supply data for this individual todo item. This is accomplished by calling the qlkit function parse-children , which triggers all the query terms that are children of the current query ➍. Otherwise, we want to return all todo items and therefore start looping over the items ➎. For each item, we again hand over control to other parsers ➏.

Of course, the parsers for the child queries can function only if they are given enough context to do their job: It’s impossible to answer a vague question like “Give me the text of the todo item.” Instead we need to give more context, i.e. “Give me the text of the todo item with ID=123.” Therefore, before the parser initiates parsing of the inside of the query, it attaches the current :todo-id to the parsing environment ➏, which the child parsers will be able to reference.

Each term in a query has a parser defined for it. Here is the parser that handles the :todo/text term:

It’s a simple parser: It just grabs the todo-id out of the environment ➊ (the one we attached in the earlier step) and then gets the text for the todo item with this ID out of the application state ➋.

Our todo app now has all the information needed to refresh the UI, causing the new todo item to appear on the screen!

Persisting the New Todo Item on the Server

Often, the data we’re reading (or modifying) lives in the client-side application state of our application. However, other times it is on the server. In order to indicate what part of a query needs to be sent to the server (such as the todo items that are persisted on the server) we define a remote parser. Here is the remote parser that is triggered by the :todo/new! query term in our transaction:

Every remote parser takes a query term and the state as a parameter and then simply returns a new query, consisting of the version of the query that needs to be sent to the server. In the case of :todo/new!, we simply forward the existing, unmodified query to the server. In essence, our app is saying “Every time a new item is added to the todo list, let the server know about it.”

As we will see later, it is also common for a query to be trimmed before sending it to the server (when we want to say “These parts of the query can be satisfied by the client, but these other parts need to be sent to the server”). Other times it makes sense to add additional information to the query before sending it to the server (for instance, if we want to say something like “Before sending the todo item to the server, add a parameter to the query to indicate that the item was added by user bobsmith29, so the server can take this fact into account.”)

Communicating Between the Client and Server

Graph queries are a replacement for the more traditional RESTful web services methodology. With graph queries, the client app has richer mechanisms for describing the types of reads and mutations that need to be performed on the server, and multiple nested reads and mutations can be bundled in a single server round trip.

When you declare a graph query API for your server in qlkit, you only set up a single server endpoint, which is very different from a traditional RESTful API, which typically has many endpoints. (Note: If you need to connect to a RESTful service for legacy reasons, qlkit can support that as well — just use your preferred client-server logic and make sure the client state atom is updated appropriately.)

Here is the entirety of the browser-side ClojureScript code in our app for communicating with the server:

This function was assigned to our root qlkit component when it was mounted to the DOM tree. The function is called every time qlkit determines that a remote query needs to be sent to the server, which is accomplished by POSTing the query to a single server endpoint named “endpoint” ➊. (Note that this code is wrapped in a go block supplied by cljs.core.async, a library that implements Golang-style channels in ClojureScript. This is how asynchronous requests are usually handled in ClojureScript, in lieu of the “promises” that may be more familiar to the JavaScripters among you.)

When the server has completed processing the request, we first check that the request was handled successfully ➋ and if so, we read the result stored in the response and send it back to qlkit via a callback ➌, so that the data from the server can be incorporated in the client-side application state.

Here, on the flip side, is the entirety of our app’s server code for handling client requests:

All this request handler function does is read the graph query out of the request ➊, attempt to parse the query ➋, and then return the result of the parsing back to the client ➌.

In fact, the parsing of qlkit queries on the server makes use of the same library code as on the client, since Clojure and ClojureScript can share code: The same qlkit functions run on both the client and the server. On the server we therefore again use read parsers and mutate parsers to parse and satisfy queries. Here is the :todo/new! parser function on the server:

To keep this example app simple, the server simply stores the master list of todo items in a server-side state atom — in a “real” app, these items would of course be stored in some sort of database, instead.

First, this function extracts the todo item’s temporary ID and text from the query ➊. One important job of the server-side :todo/new! parser is to now assign a permanent ID to each todo item, which it accomplishes in a concurrency-safe manner by generating a new integer using a sequencer ➋. Next, it stores the todo item with server-side state, using the permanent ID ➌. Finally, it returns a vector, which associates the temporary ID with the permanent ID ➍: This will be sent back to the client to let it know about this new ID.

It should be noted that in qlkit (as in Om Next), it is usually a bad idea to return information from a mutation. Instead, mutations are usually followed by a read query term that is sent to the server as part of the same query. For many reasons we won’t go into here, this will lead to cleaner code when using a graph query architecture.

However, there are two exceptions to this rule (i.e., two situations in which it is appropriate to have a graph query mutation return a value):

1. In order to associate temporary IDs with permanent IDs (as we did in the mutate parser above ➍).
2. In order to report meta-level information about the mutation (such as whether it was successful or how long it took).

Handling the Response From the Server

After the server is done handling the new todo item, it sends the following response back to the client:

[["ff81aa5b-9cde-4399-a2fd-d648e8ef9503" 2]]

As we can see, this response lets the client know that the item’s temporary ID has been replaced by the server with an ID of 2. The client now needs to parse the original remote query, along with this result information, to properly update the client-side application state so that it matches the data maintained by the server. We do this by with the final type of parser function, the sync parser function:

A sync method receives the server results as an argument ➊— in this case it is receiving the temporary and permanent ID from the server’s :todo/new! mutation function, which we extract next ➋.

The job of the sync function is to update the client application state with the information from the server ➌. If you don’t use Clojure/ClojureScript regularly, the arguments to the swap! command here will likely look a bit daunting, but all that’s happening here is that all references in the state to the old temporary ID are being replaced with the permanent ID. You can see in this line ➍ how the temporary ID is being dissociated from the todo list. Finally a replacement item is inserted with the permanent ID ➎. The client application state is now fully in sync with the information on the server.

Deleting a Todo Item: Unlocking the Power of Graph Queries Through Nesting!

Because creating a new todo item is a “top-level” concern in our example app, the logic that we’ve looked at so far for routing the creation event for updating the server have been simple enough that they could have been handled just as easily using traditional methods (such as a more traditional component library and a traditional RESTful interface).

Graph queries really shine in situations where there is significant “local context” that influences actions in an application (or influences what data needs to be returned from a server request). In this next part, we’re going to look at the steps for deleting an existing item. This will allow us to see, with greater clarity, the benefits of building UIs with graph queries.

Here, again, is the code for our TodoItem component:

When the user clicks on the right “delete” icon, an on-click event handler is triggered ➊. This event triggers a :todo/delete!transaction ➋. This transaction appears very straightforward, but an observant reader may ask the following question: “Nowhere in this transaction is it specified which todo item is to be deleted. How will qlkit know how to properly process this transaction?” If you are used to a more traditional UI library (such as a library modeled on React Flux), you would expect the transaction to specify the ID of the item. However, with qlkit we can perform a “naked” deletion transaction ➋ where no ID is specified. In essence, with qlkit the component can say “When the user clicks the icon, please delete me” without having to reference an explicit ID. This is made possible through a feature of qlkit (also borrowed from Om Next) called “component-local transactions,” which helps to keep components fully isolated from external concerns.

How Component-Local Transactions Are Handled by Qlkit

When qlikit.core/transact! is called, qlkit works behind the scenes to determine what the current component is. Qlkit can determine where in the current UI component hierarchy this component resides and automatically expands the graph query in the transaction to reflect the context of the component.

For instance, here is the query we passed into transact! :

[:todo/delete!]

Qlkit will automatically convert this component-local query into a global query that looks as follows:

[:qlkit-todo/todos {:todo-id 2} [:todo/delete!]]

As you can see, in generating the global query, the precise context of the todo item becomes clear: We are deleting the todo item with todo-id = 2. Qlkit accomplishes this feat by keeping track of the environment each component was created in, which is a feature we call autoparamaterization. Read our FAQ for more info on this feature. In the case of a todo item, it exists in the context of a list of todo items, via the :qlkit-todo/todos read parser we saw earlier:

As you may remember, we added a :todo-id to the environment before calling the parsers for each individual item in the todo list ➊. Qlkit keeps track of these additions to the environment and automatically adds them as parameters when component-local transactions are executed.

Deleting the Todo Item from the Client Application State

As we’ve seen, deleting an item generates the following transaction that will need to be executed to update information on both the client and the server:

[:qlkit-todo/todos {:todo-id 2} [:todo/delete!]]

We already know how the :qlkit-todo/todos term is handled on the client: It calls the same read parser that we were just looking at:

Importantly, remember we discussed that this read parser can handle both default and parameterized versions of the query- In other words, it extracts the todo-id out of the parameters ➊ and then only parses queries for children if they have the correct ID. This is critical, since in this case we’re going to be calling :todo/delete! against todo items, and we certainly don’t want to delete the entire list of items!

(Notice how the same :qlkit-todo/todos read parser that adds the :todo-id to the environment is also the same function that needs to make use of the same parameter, at a future point in time. Our autoparameterization feature completely shields the remainder of the app from having to “know” about this implementation detail.)

Once an item with the correct ID is found, the :todo/delete! mutation is called:

This is a very simple function; it simply optimistically dissociates the todo item from the client application state ➊. Next, we need to delete this item from the server.

Deleting the Todo Item from the Server

The remote parser for our deletion transaction is also very simple:

It simply says “Any query that deletes an item needs to also be sent to the server.”

Consequently, the request we send to the server is unchanged (though it also gets a pass through the :qlkit-todo/todos remote parser, which also leaves it unchanged). It looks as follows:

[:qlkit-todo/todos {:todo-id 2} [:todo/delete!]]

On the server side, this query is again fed through read and mutation parsers, starting with the server-side version of the :qlkit-todo/todos parser:

This does the same thing as on the client, extracting the todo-id from the parameters ➊ (which may or may not be present, depending on whether this is the default or parameterized version of the query) and then parses the child todo item queries, as appropriate for the todo-id ➋.

Finally, this triggers the todo/delete! mutation on the server, to fully erase the todo item from existence:

Conclusion

A UI framework that relies on graph queries (as qlkit does) makes it possible to have a deeply nested component hierarchy with complex data requirements that may overlap between multiple components. Qlkit makes it possible to tame such complex UIs in the following way:

• Each UI component can ask for data (and perform mutations against data) in a very natural way. The component’s queries can pretend that the data is stored in a layout that exactly mirrors the UI hierarchy. This prevents any errant data transformation logic from “creeping into” your UI code.
• The “hard work” in the app happens in the parsers: These translate the root query from the UI (which, by the nature of a UI, is tree-shaped) into actions against the local state, against the server, or both.
• The client-side application state can be kept flat and fully normalized (i.e., the state doesn’t have to contain any duplication of data, even if multiple UI components need to access the same UI data).
• Each UI component has a context in which it exists, but the component does not need to burden itself with the details of the context. The framework ensures that any actions the component takes (in the form of transactions) are expressed in an appropriate way that specifies the context.
• Similarly, the locally processed queries and server queries need to have different contexts: A developer can write remote parsers that cleanly express rules such as “This is something only the client cares about; don’t send it to the server” or “The server needs to know this extra thing about the query (i.e., this extra parameter) before it is sent the query.”

The graph query system in qlkit completely insulates components from the implementation details of the data, which may live on the client or server or partially on both. Additionally, graph queries allow for highly efficient, atomic, and batched server requests that can be kept in sync with client-side data in a very predictable way. Given the benefits of graph queries for UI development, we at ForwardBlockchain hope that the release of our new qlkit library can help, either directly or indirectly, to increase the adoption of graph-query-based UI frameworks across the industry!

Qlkit definitely is a philosophical descendant of Om Next and we want to thank David Nolen, the author of Om Next, for his inspiration. We also want to thank the other folks who have provided valuable resources to the Om Next community, including António Monteiro and Tony Kay. Additionally, I’d like to thank Craig Ludington, Bruce Hauman for their work on the qlkit codebase, and Sophie Michals for feedback on this post.

- Conrad Barski

Conrad’s Twitter: @lisperati
Qlkit Github Page
ForwardBlockchain.com