What’s the Point of GraphQL and Falcor?

Stateless applications and declarative resource requests

I want to lead into this topic with an analogy to React. One of the major reasons React has become so popular is its declarative nature. All you do is specify the DOM structure what you want, and React will compute the most efficient DOM mutation to make it happen.

This pattern has been around for a while in the functional programming world and although it might not be quite as performant as meticulously mutating the DOM, it makes your programs much easier to reason about and gives you much more abstraction power.

The next thing people started to realize is the benefit of having stateless React components. When the entire user interface is computed from a single immutable state, the application logic is much easier to follow, components become more reusable, and other abstractions like time-traveling debuggers or undo/redo become trivial to implement.

Redux is a popular library for implementing this pattern. It has a great set of dev tools and when combined with ImmutableJS for immutable updates, it can be quite performant as well. Redux is heavily inspired by the Elm Architecture and I’d highly recommend reading about Elm.

The trend we’re starting to see is that our programs are becoming stateless. That is, the entire state of the application is held in a single serializable object, not spread across local variables. But part of our programs are still very much stateful and that has to do with data fetching.

Its hard to imagine fetching data that isn’t imperative. In React, you’re probably used to seeing something like this for HTTP requests.

fetchData: function(query) {
  this.setState({loading: true})
  fetch(url, {method: ‘get’})
    .then((response) => response.json())
    .then((data) => this.setState({data, loading: false}))
}

And what about subscribing and unsubscribing to resources over websockets? Whether you’re using a mixin or a high-order function, its likely this is tied to the lifecycle hooks of your user interface.

componentWillMount: function() {
  this.sub = Meteor.subscribe(‘chatroom’, this.props.roomId)
},
componentWillUnmount: function() {
  this.sub.stop()
}

This probably doesn’t make you cringe but I hope that after reading this article, it will. Not very long ago, we were manipulating the DOM in a similar way.

function newTodo(text) {
  var todo = document.createElement(“span”)
  todo.textContent = todo
  document.getElementById(‘todos’).appendChild(todo)
}

If the code above doesn’t strike a nerve with you, take a moment to read this article about declarative vs imperative programming.

The main problem with imperative data fetching is that our user interface are no longer stateless or pure because data fetching is tied to our render functions. Instead, we should specify the resource requirements of each component declaratively and parse the component tree to compute whether we need to fetch some data, subscribe to some chatroom, or unsubscribe to a chatroom.

When the actual side-effect (fetching data) is moved outside of the React component, we can start to leverage some interesting tricks. We can deduplicate requests and cache requests or subscriptions with simple high-order functions. And rather than send out 50 separate HTTP requests to the server, we can batch them together into one and send them all at once.

Once we get all these requests on the server, you may notice that we’re not necessarily being efficient with our database queries. This is a contrived example, but maybe the user profile component requests only the username field, and the profile picture sub-component requests the image_url field of the same user. Rather than run these queries separately, we ought to combine them into a single database query.

Currently, the best solution is to over-fetch in the profile component, anticipating that the profile picture sub-component will be there, but this isn’t very good for abstracting your view components. It would be great if we could simply add and remove components without having to keep track of the resource requirements of those components elsewhere in the app.

At this point we’ve discovered exactly the motivation for GraphQL. But rather than call it a day and settle with GraphQL, lets think about this problem some more.

Suppose components can have “domain” requests and “subdomain” requests. A domain request is a top-level request much like the user profile request, and the subdomain requests are typically just fields of that domain request, like the image_url. In GraphQL, these subdomain requests are called “fragments” or “edges”.

Using this pattern we should be able to declaratively specify the domain and subdomain requests of each component, much like we do with the virtual DOM, and compose them together into efficient database queries. So let’s suppose we create a monad that represents the domain requests and subdomain requests at some point in the component hierarchy. An empty request looks like this:

{ domain: [], subdomain: [] }

A chatroom component may have a top-level domain request so it will return something like this:

{ 
  domain: [
    [‘chatroom’, {roomId: 42}, [
      [‘id’, {}, []],
      [‘name’, {}, []]
    ]]
  ],
  subdomain: []
}

I’m defining a request as a recursive tuple consisting of [ name, argument, [ request ] ]. A field is the most simple subdomain request and doesn’t have any arguments or subdomains, but some subdomains can be far more complicated as you’ll see, so it helps to have the consistency even though it looks superfluous right now.

So perhaps some chatroom info sub-component requests extra information in the subdomain of the chatroom:

{ 
  domain: [],
  subdomain: [
    [‘members’, {}, []],
    [‘messageCount’, {}, []]
  ]
}

This can compose nicely into one database query as we compose these components’ requests together.

{ 
  domain: [
    [‘chatroom’, {roomId: 42}, [
      [‘id’, {}, []],
      [‘name’, {}, []],
      [‘members’, {}, []],
      [‘messageCount’, {}, []]
    ]]
  ],
  subdomain: []
}

Now lets consider the messages sub-component. Its resource request looks like this.

{ 
  domain: [],
  subdomain: [
    [‘messages’, {limit: 50, skip: 0}, [
      [‘id’, {}, []],
      [‘text’, {}, []]
    ]]
  ]
}

There may also be a user sub-sub-component to display the username and profile picture next to each message.

{ 
  domain: [],
  subdomain: [
    [‘owner’, {}, [
      [‘id’, {}, []],
      [‘name’, {}, []]
      [‘image_url’, {size: ‘sm’}, []]
    ]]
  ]
}

These requests can all be composed together going up the component hierarchy until we get back up the chatroom component and the accumulated request looks like this:

{ 
  domain: [
   [‘chatroom’, {roomId: 42}, [
     [‘id’, {}, []],
     [‘name’, {}, []],
     [‘members’, {}, []],
     [‘messageCount’, {}, []],
     [‘messages’, {limit: 50, skip: 0}, [
       [‘id’, {}, []],
       [‘text’, {}, []]
       [‘owner’, {}, [
         [‘id’, {}, []],
         [‘name’, {}, []]
         [‘image_url’, {size: ‘sm’}, []]
       ]]
     ]],
   ]]
  ],
  subdomain: []
}

Now suppose we also want a list of chatrooms in a sidebar. This component would have another domain request like this:

{ 
  domain: [
    [‘chatrooms’, {limit: 20, skip: 0}, [
      [‘id’, {}, []],
      [‘name’, {}, []],
      [‘members’, {}, []],
      [‘messageCount’, {}, []], 
    ]]
  ],
  subdomain: []
}

And when we render the chatroom component alongside the chatrooms list component, we can simply concatenate the domain requests.

{ 
  domain: [
    [‘chatroom’, {roomId: 12}, [
      [‘id’, {}, []],
      [‘name’, {}, []],
      [‘members’, {}, []],
      [‘messageCount’, {}, []],
      [‘messages’, {limit: 50, skip: 0}, [
        [‘id’, {}, []],
        [‘text’, {}, []]
        [‘owner’, {}, [
          [‘id’, {}, []],
          [‘name’, {}, []]
          [‘image_url’, {size: ‘sm’}, []]
        ]]
      ]],
    ]],
    [‘chatrooms’, {limit: 20, skip: 0}, [
      [‘id’, {}, []],
      [‘name’, {}, []],
      [‘members’, {}, []],
      [‘messageCount’, {}, []], 
    ]]
  ],
  subdomain: []
}

Pardon my lack of artistic skill, but here’s an illustration of exactly how these requests get composed together. On the left you’ll see each component in the hierarchy with its individual resource requests and on the right you’ll see how those resource fragments compose together into one giant declarative resource request.

We’ve now effectively extracted all the data fetching requirements of our app out of the user interface and into a serializable request. If we’re dealing with HTTP requests or some 3rd party API, we can simply map over a stream of these requests (much like we map over a stream of states with Redux) and do all the subscribing, unsubscribing and imperative mutations that we need to outside of the user interface.

But if we’re building an app from the ground up, why not just use JSON diff/patch to sync this request object with the server over websockets and let the server deal with managing all the subscriptions? Then we’d have an entirely stateless application!

I think this is the right direction to move but declarative fetching is only a quarter of the battle. The next challenge is parsing this request format into a database query we can actually run. Given the recursive nature of this format, you’d think it would be easy, but it’s actually proven to be quite a challenge for me and gotten me interested in Datomic as a result.

I’m not much of a database person so I’d love to hear feedback on how we could parse these recursively nested requests into a query for your database of choice, especially if your name is Rich Hickey.

We also need to think about how to cache all this data on the client. Falcor has a really simple solution with their concept of a JSON graph and I don’t think it would be terribly hard to implement.

Falcor is very similar to GraphQL in that it tries to represent JSON data as a graph. Falcor has a concept of “path sets” for declarative data fetching but GraphQL seems to put much more emphasis on query composition with React components via Relay.

Perhaps an even more elegant solution would be to use DataScript. In any case, the client cache is going to be another battle.

The last piece of this puzzle is perhaps the hardest — database mutations, reactive updates, and latency compensation. This is a very challenging problem and has huge implications for how this approach will scale. I will be writing another article soon, asking some questions that have been lingering in my mind about scaling.

For the most part, I reactivity will involve filtering all active queries for queries that may have stale data and re-running those queries for the latest results.

Finally, I thought I’d preemptively answer a looming question:

Why not just use Falcor or GraphQL?

That’s a good question. You certainly can! I just think they might be a better solution. One problem I have with Falcor and GraphQL is that they depend on their own domain specific language (DSL). Falcor has its syntax for path-sets and GraphQL has its own syntax entirely. And I’m not that saying the syntax is hard to understand — I actually really like the GraphQL syntax.

Primarily, I don’t like how these libraries are choosing not to use JSON primitives. This makes it really hard for someone else to get in there and hack on because I don’t want to have to parse all those strings!

The other, more important thing I really want to drive home in this article is that these are all relatively simple programming patterns. GraphQL is a great idea and its really just fancy monad!