Our GraphQL Experience

Photo by Raphael Koh on Unsplash

Here at Sainsbury’s, as well as building products for our customers, we also build applications to help support some of our 200k colleagues.

Last year, we began the development of a new application to support our planning team, and started to research what technology would be the best choice to build with. In the past, we’ve built out new applications using RESTful APIs and React, using Redux for state. We’d experienced a few pain points with this approach, and we wanted to see if there was a different approach that would provide our users and developers with a better experience.

Some of the team suggested GraphQL as a good solution. After a bit of experimentation, the team ended up using it and have now been working with it for around a year. We wanted to share some of our experiences, and explain some of the benefits we’ve seen as a team.

What is GraphQL?

GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data … it provides a complete and understandable description of the data in your API

A query language for APIs

GraphQL is usually positioned as an alternative to REST. Rather than multiple resources, with an endpoint for each, you query a single endpoint with a description of the data you want.

So, an example query could look like this:

query {
  products {
    name
    price
    related { # We can descend into nested objects
      name
      price
    }
    inStockAt(storeId: 50) # We can provide inputs to fields
  }
  stores {
    id
    postcode
    storeName: name # Aliasing too!
  }
}

There’s a few benefits over standard REST APIs:

• No over-fetching. The client specifies only the fields it’s interested in, and receives only that data back. This allows clients to avoid fetching massive JSON documents to pick out two fields. It also lets multiple clients interact with the same graph in different ways, depending on their use cases.
• Fewer requests to fetch related data. As you can traverse between different nodes in your graph, you can fetch multiple related resources in a single request — great for slow connections!
• Introspectable. It’s possible to query any GraphQL API to ask it about itself. Every compliant server has a __schema field which will allow you to retrieve information about the available fields and types. This allows for some really cool features, such as the automatic schema-browser provided as standard by many APIs.

GraphiQL provides traversable documentation for your API, powered by introspection queries.

The GraphQL specification also comes with support for field aliasing, mutations, input variables and fragments, among others. The GraphQL spec is a legitimately interesting read, and is still evolving today.

Describe your data as a graph

As clients can request any data they want, GraphQL needs a way of communicating what fields are available, and how the client should expect to receive them. This is done by way of a type system, or schema.

To reuse our products example from above, a schema might look like this:

Schemas are written in SDL, which was added to the GraphQL specification in 2018.

This strong typing adds a lot of value — queries can be validated at write-time as well as runtime, as long as you have a schema to reference against. This lets developers self-validate their client code while writing it.

A runtime for fulfilling API queries

The first two elements of GraphQL are purely specifications, but we don’t currently have a way of querying our graph and getting any data back. This is where GraphQL.js comes in.

GraphQL.js is the official reference implementation of a GraphQL server. It’s written in JavaScript, and acts as a guide as to how other implementations should act. It takes a schema, a query and, via a set of resolvers, it returns data that conforms to the schema.

A resolver is simply a function that returns data for a given field. If you request a field in your query, the resolver will be called and the data returned. If you don’t, the resolver will never be called. This means any expensive fields can be resolved independently, and only affect response times when asked for.

Different languages implement resolvers in different ways, but the essentials are the same.

An example of resolvers in GraphQL.js

Outside of this functionality, GraphQL does not specify any particular use case. Queries are generally sent over HTTP, but there’s nothing in the specification to require this. The implementation is also storage-agnostic; as long as your resolvers return data that conforms to the schema you’ve defined, you can return data from other APIs, database storage, flat files, or even memory.

This flexibility means GraphQL is a great technology to sit in front of a collection of microservices or other APIs, providing a common language over a disparate set of services.

Though the reference implementation is written in JavaScript, many other implementations exist in a variety of languages

Our Experience

We’ve learnt a lot over the last year, around how to think and work with GraphQL successfully. Some of the lessons we’ve learnt have been technical, and we’ve definitely made some mistakes along the way. However, there are also benefits to adopting GraphQL that aren’t purely technical, and have actually changed how we work as a team.

Schema-first Development

One of the biggest changes to our working practices has been how we approach new stories. We work in an agile fashion at Sainsbury’s, breaking down features into stories, and then tasks. After a few sprints, we noticed that pretty much every story was now starting with a task to define our schema.

As an aside, there’s another great lesson about making work visible here — possibly we wouldn’t have noticed this pattern if it hadn’t been staring us in the face every day.

This is how our board looks pretty much every sprint. Notice the big SCHEMA tasks!

This has been done before, to some extent. Technologies such as Swagger and JSON API already exist to help formalise API development, however they place much more of a focus on defining and enforcing a clear contract between API developers and consumers.

GraphQL feels different. Instead of discussing how we were going to build a certain feature, we very often ended up discussing how our data looked, how it was related to other parts of the business, and how best to represent these relationships.

Talking about a schema first helps you to model your business domain as a graph.

This has been really powerful — it’s allowed us to focus on how our clients will use the data, rather than how we provide it to them. It lets us concentrate on the “what”, rather than the “how”.

It’s also helped to identify a few issues much earlier in the development process. Talking about how our data needs to be structured has often let us find hidden complexity, and route around it.

Decoupled Teams

Traditionally, building out API-driven applications has led to some pain points for us. We’ve often been in situations where progress on the client has been stalled due to the requirement of a new endpoint, or a change to an existing one. The tooling that GraphQL has provided us has improved this massively.

Because we’re discussing and agreeing a strictly typed schema upfront, both the API and client can not just be built in parallel, but tested against a mocked version of each other. graphql-faker is a tool we now use regularly. It allows developers to either mock, or extend, a GraphQL API, based upon a schema definition file.

An example of how quickly you can mock out an API with graphql-faker.

You can point your applications at this mocked API, share mock SDL files between developers, and continue to work.

Similarly, backend developers only need to confirm that the API they’ve built conforms to the agreed schema. One of our developers built a graphql contract-testing tool to do exactly this, and warn if there are any discrepancies between the expected and actual schemas.

The configurable nature of graphql queries also helps to decouple teams. Once you’ve built out your initial types and fields, you quickly come to a point where new features can be built, tested, and delivered without any backend development, even when they have different data requirements.

It’s a really pleasant development experience, and we’ve definitely moved faster because of it.

Flexibility

Because clients only ask for the data they need, we’ve been able to be much more flexible in how we design and grow our API.

In a traditional REST setup, we’d have versioned our API and grown it over time with new releases, being careful not to remove functionality or introduce unnecessary. With GraphQL, we’re able to be much more liberal in how we approach this.

• No versioning. GraphQL explicitly avoids versioned APIs, instead expecting that your schema will grow and change as you add new fields to it.
• Non-breaking changes. As you only get the fields you request, adding new fields has no effect on any current clients. This means it’s quite easy to deploy a new field or type to your API, test it (possibly behind a feature flag), and be safe in the knowledge it won’t affect anyone else.
• Fields for your specific use case. With REST, there’s a tendency to try and reuse endpoints, adding new filters and parameters to them to make them more generic and reusable. In GraphQL, the opposite is true — you can create fields which can take generic filters, but it’s much nicer to just create the field that you need, with only the filters you need, and only call it when it’s required.
• Adapt and change with deprecations. Adding new fields is easy, but changing or removing other fields can sometimes be interesting. GraphQL makes this easier by allowing you to easily deprecate existing fields. Clients can then move over to the replacement over time, without a hard version cutoff.

Not a silver bullet

Though we’ve come to enjoy GraphQL a lot, there are definitely situations where it doesn’t make sense. It’s important to see it as a new tool to use, rather than a straight replacement for REST. You may want to think harder about using it in these cases:

• Applications with a large performance requirement. GraphQL is generally not slow by default, but there is an overhead to handling arbitrarily complex queries. It’s also harder to cache due to the changing nature of the queries. Some of this complexity can be handled by things like batching or query complexity analysis, but for some applications with severe performance constraints this probably isn’t appropriate.
• File export and delivery. GraphQL returns JSON, and only JSON, so it doesn’t really make much sense for times when you’re generating files such as reports or exports. Luckily you can easily mix your GraphQL endpoint with REST endpoints, so you can fall back to a standard endpoint for these cases.
• Authentication flows. Many OpenID or OAuth flows have specific requirements for clients, and require specific callbacks and endpoints to exist. Again, it’s easy enough to fall back to REST here when necessary.
• Apps with a single client and server. GraphQL shines when multiple clients and use cases are in play. If you’re building a single application backed by an API, it might make more sense to build something more constrained. However, GraphQL will still work perfectly well in this situations — it’s absolutely a personal and team choice.
• Server-to-server APIs. A lot of the benefit of GraphQL is tied up in the benefit it gives to frontend development — less requests, smaller payloads, etc. With server-only integrations, these benefits aren’t there. However, if you do have multiple consumers or use cases, it may still be worth investigating if GraphQL is a good fit.

Build things faster and better

Ultimately, we’re delighted with GraphQL as a technology. It’s not a silver bullet, but for our use case it’s ideal.

It’s allowed us to build the right thing much sooner, adapt when needed, and has driven more meaningful and valuable conversations about our data. It’s also put us in a great position to open this data up to more areas of the business, creating a common language around a fairly complex area.

GraphQL has a really exciting future, and hopefully will continue to evolve and grow — we’re looking forward to the next year!