illustration by adrien griveau

How your API could benefit from Hypermedia

Roy Fielding’s dissertation thesis on “Architectural Styles and the Design of Network-based Software Architectures” albeit published in 2000, is still a goldmine in 2015. For those who are not familiar with Fielding’s work, he is one of the authors of the Apache web server, he has worked on the first specification of HTTP, and he is the father of the REST acronym.

A REST architecture is an architecture that MUST (as in RFC 2119) respect the following architectural constraints:

• client-server
• stateless
• cacheable
• layered-system
• uniform interface
• code-on-demand (optionally)

HTTP and the 2000’s World Wide Web, fit perfectly in this description of a REST architecture (in fact, REST constraints seem to be chosen so that systems that would respect these constraints will look like the web ☺)

15 years later, everybody builds RESTful APIs. These APIs take full benefit from HTTP (and thus, from its REST-base architecture) but most APIs do not fully respect the “uniform interface” constraint.

In the Fielding sense, an API is really RESTful if it respects the uniform interface constraints:

• resources are unambiguously requested via URIs,
• representations of resources are manipulated (you’re always playing with a “view” of a resource, not the resource itself),
• messages are self-descriptive and self-contained,
• transitions and actions should be clearly exposed to the client by the server, via hyperlinks and hypertext. This point is often referred as HATEOS: Hypermedia as the engine of application state.

A classical website, like the one from which you’re reading this article, uses hypermedia as the engine of application state. You’re viewing representations of resources and, on each of them, a set of actions (going back to home, viewing a related article) is actionable via hypertext links. Clicking on one of these links will (hopefully) give you another view of another resource, modifying the application state.

Those links are materialized and explained by meaningful text and iconography to help you do what you want to achieve. The web everybody knows is fully compliant with the HATEOS constraint.

HATEOS constraints applied to APIs

Fine, but what’s the point with APIs? Let’s take an example with a widely-used API; the Twitter one. Twitter API exposes a way to retrieve your friends by doing the following call over HTTP

GET https://api.twitter.com/1.1/friends/ids.json?cursor=-1&screen_name=twitterapi&count=5000

Then, the reply (as documented here) will be something like:

{
  "previous_cursor": 0,
  "ids": [
    657693,
    ...,
    783214
  ],
  "previous_cursor_str": "0",
  "next_cursor": 0,
  "next_cursor_str": "0"
}

Without any knowledge of the twitter API structure, what actions are easily actionable from this response? NONE.

Server developers are responsible for providing the next actions a client can take from each point in the application.

This is exactly where HATEOAS comes into play.

By being non-HATEOS compliant, most APIs, including “big” ones, do not respect the uniform interface constraint as defined by Fielding. Does it means that the web (well, APIs) is broken? NO ☺

Now, two hard questions arise:

• “As a client developer, is it really possible to create an API client smart enough to use hypermedia relations to perform actions on websites without any knowledge of the website structure?”
• “As a server developer, can I describe my application logic via links and relations that would be self-comprehensive by any hypermedia client?”

For instance, if every social network would expose Hypermedia APIs, fetching my friends on each of the network would be as easy as:

client(any_website).auth(login, password).get_friends()

(here: our client would request the “/” of the social network api, get the Content-Type to parse the response, look at any available link relation to authenticate, and then look at any available link relation to retrieve friends.)

The (optimistic) answer for these two questions is YES. Hypermedia advocates put a lot of energy in standardizing the web to allow the emergence of Hypermedia APIs:

• by creating formats for our APIs: Atom+Pub, Collections+JSON, Siren, HAL, …
• by creating repositories of link relations (see schema.org)

To approach hypermedia APIs, it is important to understand that every media type (or Content-Type) should define a processing model.

Said otherwise, keep in mind this question: “What would a browser do with my response?”.

Hypermedia APIs remain not massively used

Sadly, there are a few limitations that could explain why Hypermedia APIs are not widespread:

• writing an exhaustive list of link relations seems unrealistic: even if efforts are made to simplify the description of new link relations (JSON-LD is a good example)
• APIs are not (quite) the web, they are designed to interconnect services and feed end-user applications. Using hyperlinks and describing link relations yield bigger (in size) replies and slightly more requests in a world when connectivity is still an issue in many cases (notably in mobility: smartphones, etc..)

Another reason for the rarity of such APIs might be the lack of commitment from WEB giants. Google might be the only one (as far as I know) to provide a Hypermedia API (their XML responses respect the AtomPUB format).

But there are a lot of things to learn and some benefits to take from Hypermedia-based APIs:

• definition of a processing model according to a Content-Type (or simpler: formatting of replies),
• comprehensive description of your API via links and their associated descriptors (especially standard ones as defined by the IANA).

Hypermedia API case in point

Let’s now go back to our previous example. As already described,

GET https://api.twitter.com/1.1/friends/ids.json?cursor=-1&screen_name=twitterapi&count=5000

would return the following response:

{
  "previous_cursor": 0,
  "ids": [
    657693,
    ...,
    783214
  ],
  "previous_cursor_str": "0",
  "next_cursor": 0,
  "next_cursor_str": "0"
}

Under a “hypermedia” light, this response has several limitations:

• I have a list of id, which are, as we can guess, our friends’ ids, but what can I do with these? How can I fetch their user information?
• As a human, I guess that next_cursor is something to fetch the next page of ids, but what is the associated URL? And is there something more generic to say that something can be used to fetch the previous and next page of results?

In fact this answer is full of conventions. These conventions are documented in a human-readable format and accessible via twitter’s developer website.

Hence these conventions should be coded by a human within a library, leading to a strong coupling between client and servers.

The lack of URL usage in answers leads to a high number of links hardcoded in client APIs.

As an example, here is the output of a search for “url” in a python twitter api client.

Let’s try to make this answer a bit more « Hypermedia ».

IANA standard relations type (http://www.iana.org/assignments/link-relations/link-relations.xhtml) defines « prev » (or « previous ») and « next » relations as:

• « prev »: indicates that the link’s context is a part of a series, and that the previous in the series is the link target.
• « next »: indicates that the link’s context is a part of a series, and that the next in the series is the link target.

Our previous answer could be rewritten as:

{
  "prev": "https://api.twitter.com/1.1/friends/ids.json?cursor=-1&screen_name=twitterapi&count=5000",
  "ids": [
    657693,
    ...,
    783214,
  ],
  "next": "https://api.twitter.com/1.1/friends/ids.json?screen_name=twitterapi&count=10&cursor=0",
}

Then for each id, you could decide to return a data structure that could give you useful information about the user: lookup (as described in twitter api), friends, timeline, etc… It could give the following rewrite of your answer:

{
  "prev": "https://api.twitter.com/1.1/friends/ids.json?cursor=-1&screen_name=twitterapi&count=5000",
  "ids": [
    {
      "657693": {
        "user_lookup": "https://api.twitter.com/1.1/users/lookup.json?user_id=657693",
        "friends": "https://api.twitter.com/1.1/friends/ids.json?user_id=657693"
      },
      ...,
      "783214": {
        "user_lookup": "https://api.twitter.com/1.1/users/lookup.json?user_id=783214",
        "friends": "https://api.twitter.com/1.1/friends/ids.json?user_id=783214"
       }
    }
  ],
  "next": "https://api.twitter.com/1.1/friends/ids.json?cursor=-1&screen_name=twitterapi&count=10&cursor=0",
}

But wait! Here “user_lookup” and “friends” are not described as standard relations links in the previous IANA link relation lists. That is true, and it is, as described earlier, one of the fallacies of hypermedia. Here, the fun fact is that even the Person « type » described by schema.org does not expose a « friends » link relation.

Finally, this response does not fit any existing format. With collection+JSON, the previous response might be:

{
  "collection": {
    "version": "1.0",
    "href": "https://api.twitter.com/1.1/friends/ids.json?cursor=-1&screen_name=twitterapi&count=5000",
    "links": [
      {
        "rel": "prev",
        "href": "https://api.twitter.com/1.1/friends/ids.json?cursor=-1&screen_name=twitterapi&count=5000"
      },
      {
        "rel": "next",
        "href": "https://api.twitter.com/1.1/friends/ids.json?cursor=-1&screen_name=twitterapi&count=10&cursor=0"
      }
    ],
    "items": [
      {
        "href": "https://api.twitter.com/1.1/users/lookup.json?user_id=657693",
        "data": [
          {
            "name": "user_id",
            "value": "657693"
          }
        ],
        "links": [
          {
            "rel": "user_lookup",
            "href": "https://api.twitter.com/1.1/users/lookup.json?user_id=657693",
            "prompt": "User Info"
          },
          {
            "rel": "friends",
            "href": "https://api.twitter.com/1.1/friends/ids.json?user_id=657693",
            "prompt": "Friends",
            "render": "collection"
          }
        ]
      },
      {
        "href": "https://api.twitter.com/1.1/users/lookup.json?user_id=783214",
        "data": [
          {
            "name": "user_id",
            "value": "783214"
          }
        ],
        "links": [
          {
            "rel": "user_lookup",
            "href": "https://api.twitter.com/1.1/users/lookup.json?user_id=783214",
            "prompt": "User Info"
          },
          {
            "rel": "friends",
            "href": "https://api.twitter.com/1.1/friends/ids.json?user_id=783214",
            "prompt": "Friends",
          }
        ]
      }
    ]
  }
}

(note: As mentioned earlier, this response is far bigger compared to the original one, this is a huge limitation of hypermedia responses)

To conclude, why would we consider this rewrite of a Twitter response as better?

I think that every API should have the following qualities:

• stable
• comprehensive
• adaptable

Providing links in your replies will let you change the link destination of a resource, making your API adaptable, and somehow stable (if client developers do not make any assumptions about your URLs)

Providing relations will make your links comprehensive and will make your responses self-descriptive.

Using a stable format will make the format of your replies predictable and, as a server developer, will give you a guide to forge responses.

Following this advice might let you build APIs that would be more adaptable and their clients might be more resilient to changes.

Useful Resources

If all of this makes sense to you, I truly recommend you read this book. Mike Amundsen is one of the best advocates of Hypermedia APIs and this book goes deep into standards description.

I should mention too traverson which is the coolest attempt to build a functional hypermedia client and HAL browser which is a project where you can play with HAL. By the way, you will find a short API client for HAL browser in one of my slide deck here.

This article is part of the publication Unexpected Token powered by eFounders — startup studio. All eFounders’ startups are built by extraordinary CTOs. If you feel like it could be you, apply here.

You like? Hit “Recommend” and subscribe to our collection!