GraphQL: The view from atop the cliff
GraphQL is a powerful, flexible technology that can revolutionise your approach to APIs and massively accelerate their development.
There’s just one problem: it’s incredibly difficult to fully understand.
In the last 3 years I’ve worked on three major API-based projects; an internal GraphQL API for a mobile game, the public, REST-based IX-API standard, and an internal GraphQL API for my current employer. Each of those, even the RESTful API, has made me appreciate the power and potential of GraphQL.
But dear gods, the learning curve is nothing short of a cliff.
The thing is, it’s not for a lack of documentation. There are some great GraphQL resources out there, including How To GraphQL and GraphQL Mastery, that show you how GraphQL works, and I’d advise you, in due course, to study at least one of them. What they don’t show you, however, is the GraphQL mindset you need. They show you the how, but they don’t give a deep understanding of the why.
And it’s probably a tricky thing to teach, and I’m not going to promise to give you a full comprehension of the Matrix right here, but maybe I can help address why GraphQL is so tricky to grasp, and pass on some useful hints.
Firstly, it’s worth addressing why GraphQL is so hard for so many of us to grasp. I think the biggest reason is that most of us see it as an evolution of REST rather than a revolution, so we keep trying to do things in an (albeit enhanced) RESTful way. I think the most important thing you can do when trying to start to understand the GraphQL methodology is to set aside the RESTful approach entirely and treat this as an entirely new solution.
Now — don’t run away! You’re probably as skeptical as I am of any technology that claims to entirely supersede the existing paradigm, and you’ve probably seen many of them display the success trajectory of Douglas Adams’ whale. Your existing coding knowledge is still relevant, and the methods you use to generate and consume data will be familiar. But to my mind, that’s not where we want to start.
So what does GraphQL give us?
Having cleared the decks and neatly stowed our preconceptions, where do we start. What is GraphQL?
- GraphQL is a method for interrogating ecosystems, not entities
- It’s a strongly typed, explicitly defined interface that’s both human- and machine-readable
- It’s a mechanism for getting all (exceptions exist) of the information you need at once in one query
- It’s a technology in which you program connections in the server, not specific-purpose endpoints
- It’s a tool that focuses on the system you need to know about, not how it happens to be stored in the database
- It’s a way of suddenly realising that you already have all the information you need for that new page available, even if you’d not previously considered that use case
That’s all wonderfully inspiring, and incredibly hand-wavy. What does it actually mean?
Let’s take some examples from the domain of IX-API. Now, I happen to feel this is an incredibly well-designed REST API, and that opinion is clearly entirely unbiased. But there were plenty of times during its development that I wanted to be implementing it in GraphQL.
Of course, if we had based in on this far-less-understood technology, we’d have completely killed its adoption, and adoption was key for that project.
What might we ask of a network description API that’s based on GraphQL?
Let’s say we wanted to display a network map of a single exchange point (a PoP in IX-API parlance) on a webpage. We’d need:
- List of all switches in the PoP
- List of all trunked Connections on the switches (a collection of physical ports)
- List of each port (Demarc) on each trunk
- The hardware details of each port
- The network details of each connection
So, the IX-API is deliberately very granular (see my previous post for some of the design decisions behind it), but I’m sure you can think of examples of similar complexity for displaying a user’s profile page with posts and comments, or any of the other standard scenarios. There’s a lot of disparate-but-related data here, and in traditional systems that either means one coding one really specific endpoint on the API server, or doing a lot of sequential network requests.
In GraphQL, you’d ask, in a single, strongly-typed query:
- Get me all switches
- For each of those, get me their name, manufacturer, etc, and all of their trunks
- Within the answer for each trunk, get me its name, network details etc, and all of its ports
- Within the answer for each port, get me its name and hardware details
All of that comes back from one network query in a map looking very much like a large, nested JSON object. All of the data you wanted, and only that data, is there in exactly the position you asked for it.
To relate that to our list of GraphQL features above:
- We’ve interrogated the ecosystem that is the network as a whole, rather than asking for each entity, looking at it, and then going back for more
- We’ve been able to ask about specific details of each entity as we go along, because the strong typing determines what properties and relationships everything we get back will have
- We’ve been able to ask for “related entities” each time we needed them, because we have predefined connections between related items
- At no point have we cared what the primary key or database ID of any item is, because we’re treating them as related items, not DB rows
Now let’s imagine we wanted a different query, to see what ports we could reach within a single hop from a given switch, and what their hardware details were.
We need two things we’ve not used before: firstly, an ability to enter the map at the location of single switch. And secondly, a way to ask for the “other end of the connection”. Once we’ve done that, we can ask:
- Given an existing switch
- Get me all trunks on this switch
- Within the data for each trunk, give me the trunk at the other end of its connection, if any
- Within the data for each remote trunk, get me its name and hardware info
Here, rather than an entirely new query, we’re using a combination of connections we’ve already used, new connections, and a new entry point on the map.
As we develop more queries, we develop more connections. Eventually, we come to ask a new question of the ecosystem, we find we already have every single connection we need to ask about. We don’t need to write any more server code, we just need to phrase the question.
And that’s when GraphQL really works. You’ve suddenly got a fully-mapped ecosystem, and you can see as much of it as you need at once. You can use that map to start anywhere, and get anywhere.
While getting to the top of the cliff is quite a hike, the view is well worth it. But can we make the journey any easier?
Well, yes. There are some processes and pointers you can follow to get you there faster:
Key pointers
On the server side, write connections, not query endpoints. A GraphQL server only has one endpoint, not so that you can collect disparate solutions in one place, but because you’re always interrogating a single, connected system. In concrete terms, this means you should thing twice before adding anything to the top-level Query type, and the question you should ask is “should I be able to get this via an existing relationship?” The Query type is for your entry points into the graph, and you shouldn’t need many of these — maybe one per type of entity you might want to traverse the graph from.
Use properties flexibly. Every entity property has the potential, without change of syntax, to be a resolver. GraphQL determines what is data type is accessible from each resolver, but it doesn’t care about how you obtain it. If it’s a static scalar from the DB row defining your entity, fine. If it’s the current temperature in a city on another continent, also fine. Just tell GraphQL that type ‘City’ has has property ‘currentTemperatureF: Float’, or ‘TestAttempt’ has ‘totalScore: Int’ and you can fetch it however you want. Plus, you can change your mind later without affecting the interface or any of your clients.
Don’t be shy of filters, either. Each resolver function takes named parameters, each strongly typed. You can use these parameters, for example, to specify a sort order, a filter on a specific field, or a limit (among other things) and because they’re named and (unless specified otherwise) optional, you can just name the ones you care about in any given request and not worry about whether it’s five or seven ordered parameters you need to skip.
Never terminate at an ID, or a list of IDs. Type User’s property Posts should be an array of Post objects, not a list of database IDs. In the latter case you’ve just built a brick wall across the map, and your API user has to find somewhere new to enter the map and start traversing again. If you only actually need to return IDs right now, and don’t want to code up resolvers for title, comments etc? Just define the Post object as containing the ID and nothing else — yet.
Don’t succumb to YAGNI. With the exception of the “no IDs” rule, you never need to add a resolver or field you’re not using yet. Addition of fields, types, queries, parameters is always backward-compatible in GraphQL, because they’re all named and all unused unless the query specifies them (special exception for required parameters). Removal of items that a client might already be using, isn’t. Add capabilities only once the use-case is actual and understood.
Make all relationships bi-directional (unless that really makes no sense). If you can ask for User.Posts = array of Post Items, each Post item should have an Author field = User object. Bi-directionality = better traversability = happy API consumers, particularly when that API consumer is you.
Note that bi-directionality does not inherently create loops! In fact you cannot create an infinite loop in GraphQL, because you’d have to have an infinitely large query to do so. When you specify the fields to be returned on a given type in a query, you’re only specifying them for that type at that level, not for everywhere they appear. So the ability to ask for User.Posts (with body, subject, author-with-name) is safe as well as convenient, and makes your front-end coders happy.
Make use of the type safety. Every GraphQL server exposes a fully-typed schema, and you can use that schema both as documentation and as a type map. Your language will likely have tools that read in a schema and spit out type definitions. If you’re really lucky (which in this case includes coding in TypeScript), you’ll find tools that reads the queries from your source code and provides typing data for exactly that data that you’ve requested.
The schema forms the contract between your server and clients, and a strongly typed schema makes for a strong contract. However, it’s also possible to create much weaker contracts, for example by specifying fields that can contain arbitrary JSON. Avoid this where possible, as it means your client can’t learn from the schema exactly what to expect. Instead, if you just need more types or subtypes, define them; GraphQL supports heterogeneous responses and subtypes via unions, interfaces, and per-type field requests.
Finally, know when to code defensively. The schema will tell you whether a property must contain a value (it’s marked with a !) if you asked for it, or it’s optional and might be null.
Conclusion
This — obviously — isn’t a complete guide to GraphQL. That would be much longer, and much of it would be better covered elsewhere. In particular, I’ve deliberately not mentioned security, performance or federation, all of which you’ll want to continue to research.
However, the pointers will (I hope) allow you to get into the GraphQL mindset and shorten your climb up the cliff. With luck, it might also save you from heading into a few dead-ends in the crags.
