Designing a RESTful API

While working on DataFire, I’ve had the chance to work with hundreds of different APIs to build out our set of integrations. I’ve seen a lot of good, bad, and ugly API designs, and would like to share what I’ve learned so far.

While I’ll go into a lot of particulars, there is a general theme to this article: when designing an interface, use standards wherever possible. This is just as true for APIs as it is for UX. Nothing scares new users more than an unfamiliar interface.

Requests

The first thing a developer will want to know is how to make requests to your API. Creating an API console or including cURL samples are a great way to get them started, but experienced developers should be able to understand the structure of a well-designed API at first glance.

HTTP Verbs

Generally, most endpoints will handle database CRUD — that is, the Creation, Retrieval, Updating, and Deletion of resources owned by your users. To use the common example of a pet store, we’ll need to be able to create new pets for sale, retrieve a list of pets for sale, update pets’ information when they get vaccinated, and delete them when they’re sold.

One of the most familiar aspects of RESTful design is that HTTP verbs, particularly GET, POST, PUT, PATCH, and DELETE, have semantic meaning. Generally speaking:

• Use GET to retrieve one or more items
• Use POST to create a new item
• Use PATCH/PUT to update items (I like to use PUT for upsert and PATCH for modifications)
• Use DELETE to remove items

Sometimes you’ll need to expose more operational endpoints - e.g. an NLP API might expose an endpoint that takes in some text and returns whether it has positive or negative sentiment, or the Heroku API might expose an endpoint that restarts your app. In these cases, the standard is to use the POST verb.

Routing

Generally, the path of the URL corresponds to a resource or set of resources on the server. Plural nouns work well, and correspond nicely to how we already think of files and directories. While there’s no standard here, the system I tend to use is:

• The first part of the path corresponds to a table or collection in your database
• The second part of the path is an ID or slug (more on slugs below) that identifies a particular item in that table/collection
• Additional parts of the path represent sub-fields inside that item.

For instance, our pet store API might have the following routes:

• GET /pets - retrieves all pets
• GET /pets/{id} - retrieves a particular pet
• POST /pets - adds a new pet
• PATCH /pets/{id} - updates a pet
• PUT /pets/{id}/owners - adds a new owner for the pet
• GET /pets/{id}/vaccinations - retrieves a list of the pet’s vaccinations

Slugs

An unfortunate name for something that’s meant to make URLs prettier. The idea is that instead of having URLs like

/pets/565df42677f8630521dbbd31

we can instead use human-readable handles like

/pets/Rover

This is a great way to make your API more usable, but can be tricky both from a technical and a UX perspective. You have to make sure slugs are unique, and don’t collide with IDs. My suggestion is, unless you already have a unique, human-readable ID field (like a username), hold off on slugs until v2.

Linking UI and API

This is a rare practice, but it’s done by some of the best APIs out there, most notably by GitHub. The idea is that the URLs on your website and the URLs for your API should map neatly to one another. For instance, to get all the commits for a particular repository you can visit

api.github.com/repos/{username}/{repo}/commits for JSON

github.com/{username}/{repo}/commits for HTML

This makes it easy for developers who are already familiar with your site to quickly understand and debug your API.

Responses

How your API handles requests - particularly in error cases - will have a strong effect on your API’s perceived usability and stability.

Format

The vast majority of modern APIs use JSON to communicate with the client, and I see no reason to deviate from that. You may want to provide XML or YAML output as an option, but JSON is far and away the preferred medium of machine-to-machine communication, and has built-in support or libraries in every major language.

HTTP Status Codes

The use of status codes is critical to RESTful API design, and is often overlooked. I see a lot of APIs that will return a 200 (success) response with the JSON {“error”: true}. This is Bad.

You should at least handle these five cases explicitly:

• 200 - Success
• 400 - Bad Request
• 401 - Unauthorized
• 404 - Not Found
• 500 - Server Error

There are plenty of others that are useful, and by handling each explicitly you give your developers’ applications the chance to react appropriately. Here’s a great article from Dropbox that goes deeper into how to think about status codes when designing an API.

On Wrapping Resources

The question here is whether the /pets endpoint should return exactly the resources requested:

[{
  "name": "Fido",
  "age": 2
}, {
  "name": "Rover",
  "age": 1
}]

or wrap them with metadata:

{
  "results": 2,
  "pets": [{
    "name": "Fido",
    "age": 2
  }, {
    "name": "Rover",
    "age": 1
  }]
}

While the latter adds a bit of overhead to working with the API, it makes things like pagination and hypermedia possible, and is more future-proof.

Authentication

This is probably one of the most crucial decisions you’ll make when designing your API. Authentication is a huge bottleneck for developers. If you don’t believe me, try using Twitter’s API without a client library.

While every API has its own requirements, the most general statement I can make here is to follow industry standards. Rolling your own authentication mechanism is not only a waste of time, it’s dangerous.

There are three major flows to consider, in order of increasing complexity and security:

API Keys

Issue each API client a unique string, which they pass along with every request to the API, often as a query parameter. This string acts as a username/password in that it uniquely identifies the user and grants access to all his/her resources.

This is somewhat insecure — if the key is compromised, the account is compromised. There is no scoping (e.g. making a particular key read-only) or ability to revoke access. That said, it’s very easy to use — clients can immediately start calling the API via their browser or cURL.

Basic Authorization

Basic authorization passes a username and password (or an API key and secret) on every request to the API. This should be as an Authorization header, with the base64-encoded string “username:password” as the value. This is a bit harder for clients to implement, but doesn’t require multiple requests to the API to establish a session.

OAuth 2.0

OAuth is generally used when API clients can act on behalf of other users — for example, Twitter uses OAuth to allow third-party sites to tweet on a user’s behalf.

The benefits of OAuth are extensive:

• Tokens are scoped, so they can be made read-only, or given access to a subset of information
• Tokens are temporary, so a compromised token will only last a short period of time
• Tokens can be easily revoked

Unfortunately, OAuth requires a fairly complex handshake between client and server before the API can be used. That said, it’s implementation is fairly uniform across APIs, and a number of libraries and services have sprung up to ease the process. At DataFire, we use Passport to handle our OAuth integrations.

Tooling

If you’re working with NodeJS and MongoDB, we’ve published an open source library, Jammin, which takes care of a lot of this for you. Specifically, it:

• Ties HTTP verbs to CRUD operations
• Automatically handles 200, 404, and 500 responses
• Is extensible via middleware to handle complex queries, lazy deletion, access control, and more

I also strongly suggest documenting your API in a machine-readable way. This allows products like DataFire to easily integrate your API, and allows you to link into a growing ecosystem of API tooling to help with things like documentation, testing, mocking, and client library generation.

Swagger seems to be the front-runner here, but no need to stress too much on this decision; we’ve put out another open-source library for converting between different formats. Here’s an article from Kin Lane discussing the merits of a few different formats.

Conclusion

While every API has its idiosyncrasies, designing your API with RESTful standards in mind will help keep your API stable and easy to use. The more familiar your API looks, the faster developers will understand what it can do and how to use it.

Have you designed an API recently? Can you think of cases where breaking REST patterns is helpful or necessary? Let us know in the comments!