GraphQL at our Museum: No REST for the Wicked

At MAAS Museum in Sydney, we have been busy building version 2 of our unified API for object data. This API serves data to our two big projects of the past year, a digital labels system for exhibition touchscreens and a new collection website. Both are essentially interfaces for object data, so having a single API to power both made sense.

Initially, we started off with a traditional REST API approach.

What is a REST API?
APIs are how different computer programs communicate with each other. REST is a popular type of API that communicates via HTTP, allowing websites to interact and enabling one website to get data from another. In our case, the API is a way to query our database of ~130,000 museum objects in various ways. Other websites can then use our API to search and get object data from our collection.

REST has been the standard for many years and has been adopted by most of the web. We even did a whole bunch of research on existing museum APIs.

Museum API research we conducted in early 2016.

Our digital labels project became the first consumer of our fledgling API. This web application displays content and images about objects, as well as enabling search within an exhibition. Much of the data comes from EMu, our collection management system (shoutout to Rowan Stenhouse for his work on ‘harvesting’ data from EMu).

The digital labels are currently in use at the Powerhouse Museum in the Icons exhibition (two iPad Pros) and Museums Discovery Centre (12 Dell Touchscreen PCs).

The digital labels client uses some pretty innovative technology too. We went with React, Redux and got server side rendering working, allowing us to build an universal (aka isomorphic) Javascript application. Article coming soon.

We had grand plans to power our new collection website with the same API. It was going to be the cornerstone of our new tech stack…

and then ditched months of work, gave up on good old reliable REST for a new kid who has barely run around the block — GraphQL.

It was an audacious move. Deadlines were looming, the existing API was working fine and we were about to embark on the collection website.

Firstly, some background on GraphQL: it is an open source project used in production by Facebook. You probably used it a few times today without even realising it. Essentially, it is a data querying layer that sits in front of a database (or databases). A client application decides what data it needs, makes a request using GraphQL’s language and gets a predictable response, no more no less.

Why is it called GraphQL?
It is inspired by the Graph Theory of Mathematics and has nothing to do with silly bar graphs or pie charts. A graph in mathematical terms consists of points (nodes) and connecting lines (edges). This turns out to be a useful way to visualise interconnected data. We got this theory because of The Seven Bridges of Könisberg.

Euler’s Figure 1 from ‘Solutio problematis ad geometriam situs pertinentis,’ Eneström 53 [source: MAA Euler Archive]

REST APIs work a little bit differently. Client applications will ask a server for data by requesting a URL. The server generally decides what data you get. So you may fetch more data than you need, and you may also have to contact the server again to fetch more data.

Even though REST has been the standard for years, we experienced quite a few pain points while building the digital labels. To illustrate, check out this page on our application:

No problem here, just a list of objects in one of our rooms at the Museums Discovery Centre. Each room is called Store 1, Store 2 etc. The endpoint to get the data is:

https://api.maas.museum/v2/set/store-1

‘Set’ is our way of saying a ‘group of objects’. We get back something like this:

{
  title: 'Store 1',
  slug: 'store-1',
  parentSlug: 'museums-discovery-centre',
  objects: [
    {
      title: 'Bima and Purukapali with the Spirit of their Dead Son Jinani'
    },
    {
      title: 'Little Known Facts diorama'
    },
    # More results here ...
  ]
}

All good right? But what if we wanted to navigate between different rooms within Museums Discovery Centre? Here is the menu we built:

So how do we get the data for the side menu with ‘Store 1’, ‘Store 2’, ‘Store 3’ etc? Eagle-eyed readers may have noticed a parentSlug field. This is precisely what we’ll use because we designed sets to have a parent-child relationship with other sets. In this case, ‘Store 1’, ‘Store 2’ etc sets all have a parent set called ‘Museums Discovery Centre’.

Let’s request the ‘Museum Discovery Centre’ set:

https://api.maas.museum/v2/sets/museum-discovery-centre

Here is the data:

{
  title: 'Museums Discovery Centre',
  slug: 'museums-discovery-centre',
  parentSlug: null,
  childSets: [
    {
      title: 'Store 1',
      slug: 'store-1'
    },
    {
      title: 'Store 2',
      slug: 'store-2'
    },
    ... # we use these to build the menu
  ]
}

This is quite useful, but we did have to fetch data twice to build our page.

GraphQL, on the other hand, can get this data in one transaction:

query {
  set(slug:"store-1"): {
    title
    slug
    objects {
      title
    }
    parentSet {
      title
      slug
      childSets {
        title
        slug
      }
    }
  }
}

If you haven’t seen a GraphQL query before, this might seem a bit strange. It is pretty much a JSON object without the commas. Notice how we chose which fields to return, even within nested fields (objects and parentSet).

The resulting data is the same shape as our query — predictable and all in one go:

{
  title: 'Store 1',
  slug: 'store-1',
  objects: [
    {
      title: 'Bima and Purukapali with the Spirit of their Dead Son Jinani'
    },
    {
      title: 'Little Known Facts diorama'
    },
    ...
  ],
  parentSet: {
    title: 'Museums Discovery Centre',
    slug: 'museums-discovery,
    childSets: [
      {
        title: 'Store-1',
        slug: 'store-1',
      },
      {
        title: 'Store-2',
        slug: 'store-2',
      },
      ... # we use these to build the menu
    ]
  }
}

Not bad hey? If you are still not convinced, have a look at an object page:

Information about the individual object is fetched from:

https://api.maas.museum/v2/objects/1234

You may have noticed the header retains the title of the current set for easier navigation. Luckily our digital labels url has information about the current set (‘store-1’).

https://labels.maas.museum/set/store-1/object/1234

Unfortunately, there is no way to get the title of the set, unless we fetch again from here:

https://api.maas.museum/v2/sets/store-1

Notice any issues? You’ll see that while we fetched the title of the set, we also got a list of objects that we didn’t need. This increases our data payload and seems highly inefficient.

{
  title: 'Store 1',
  # Everything underneath is not used for the object page
  slug: 'store-1',
  parentSlug: 'museums-discovery-centre',
  objects: [
    {
      title: 'Bima and Purukapali with the Spirit of their Dead Son Jinani'
    },
    {
      title: 'Little Known Facts diorama'
    },
    ...
  ]
}

For more complicated user interfaces, things can quickly get out of hand. Quite often, multiple API requests have to be made just to get a small snippet of data, while extraneous information gets discarded.

We stuck with REST when we launched the digital labels at Museums Discovery Centre in September 2016, but we still had one eye on GraphQL.

Then this happened — Github announced their new GraphQL API a few days after we launched the digital labels.

The GitHub GraphQL API

This was a huge endorsement, especially considering that Github’s own REST API implementation has long been hailed as the gold standard by developers worldwide.

This news gave us the impetus to try out GraphQL. We are a museum of technology after all (our previous director always told us to try new things and not be afraid to fail).

Rebuilding the API was a daunting task, but in the end it only took two weeks of intense googling, great open source tools and lots of head scratching.

Previously, with our REST API, we used a CMS called Keystone JS to manage data and help build the endpoints. We managed to get Keystone working with GraphQL by using some clever workarounds (big ups to my colleague Lachlan Gordon for working that one out).

We used a package called graphql-compose that quickly set up GraphQL queries with filters, sorting and pagination. It also opened up an easy way to write to our database via GraphQL mutations. Suddenly, we overtook our previous REST API.

Other benefits of GraphQL include:

• GraphiQL — an in-browser tool to query your data. A 20min demo of this was all it took to get the team on board. Here is an example.
• Self-documenting — you can describe each field and schema model within the code. These descriptions can be accessed by a special query called ‘introspection’ in GraphQL.
• Multi-Database — GraphQL is not tied to any particular database, so it is possible to query multiple databases in one go.
• Versioning — this is a common issue amongst REST APIs. However it is much easier to incrementally refactor in GraphQL without breaking client applications. Facebook has used GraphQL in production since 2012 and hasn’t needed to version once.
• Platform agnostic — you can build GraphQL servers with NodeJS, PHP, Ruby, Python, Java and more.
• Well documented — check out http://graphql.org.

While GraphQL won’t suit every situation, it works for us because our data is highly interconnected. We are also using React (another Facebook library) for our client applications, so GraphQL is a natural fit.

For the past few months, we have have incorporated GraphQL in our new online collection website (it is launching very soon, so keep an eye out for it!). Using GraphQL has allowed us to barely touch the backend code, letting us concentrate on the core application.

We even connected GraphQL to our unofficial REST API that runs on Wordpress (yes, you can combine REST and GraphQL together). This allows us to link collection objects to our Wordpress events, exhibitions, products or blog posts and vice versa.

With such easily accessible and interconnected data, the Museum has a strong foundation for new applications such as touch tables, teacher guides, chatbots, tour apps and VR interfaces.

We are even working on opening up our GraphQL API to the public once we’ve tightened the screws and completed some documentation. Hopefully the wider community can experiment with our API and build experiences we haven’t even conceived of yet.