Learning outcomes: By the end of this blog, you should:
- Understand what REST API is
- Know what makes it a good and bad REST API design
- Be able to design your own REST API
What is REST APIs?
REST APIs are a widely adopted communication method in backend applications, allowing them to expose their functionalities to clients such as frontend and other backend services. However, it’s important to keep in mind that REST is not the only option, and there may be other methods like gRPC or GraphQL that are better suited to the needs of your application. In some industries, specialised communication methods are preferred. For instance, Financial Information eXchange (FIX) Protocol is very popular in Finance.
Before diving into designing REST APIs, it’s crucial to understand the requirements and business reasons behind the software you’re developing. This involves working closely with stakeholders to clarify the requirements and avoid making assumptions. An effective software design should take into consideration both functional and non-functional requirements, and avoid over-engineering or under-engineering the solution. Is it practical to invest months in developing a comprehensive software when all we require is a basic HTML page for prototyping? Similarly, while designing a bill payment system for an electricity company with a million customers who pay a single bill per month, should we create a system capable of handling billions of transactions daily?
Now let’s say after thoroughly analysing requirements, with all the trade-offs have been addressed, we decide that REST APIs is the right choice. Let’s take a look at how good REST APIs should be.
Designing REST APIs
It’s not uncommon to come across teams that design APIs using arbitrary styles on top of HTTP, but still refer to them as REST APIs. However, APIs can only be considered as REST if they follow the principles of REST design. In fact, REST architectural style does not dictate the specific protocol to be used, although it is most commonly applied on top of HTTP. The adherence to conventions is crucial because APIs that deviate from industry-standard REST APIs may cause confusion and increase the learning curve for new developers and clients. In addition, these deviations can lead to integration bugs and negatively impact the overall efficiency of the system.
REST APIs have been around since the early 2000s. There are plenty of Goodreads talking about designing REST APIs. My favourite is the website https://restfulapi.net/. Checking-out the website will be one of the best investment you can make as an engineer, a big time saver.
At a glance, good REST APIs should use components of an HTTP request and response including the Verb, URI, Query, Headers, and Body for the request, as well as the Status, Headers, and Body for the response properly.
URI
RESTful URI should refer to a resource that is a thing (noun) instead of referring to an action (verb) because nouns have properties that verbs do not have — similarly, resources have attributes, e.g Movie.
Good REST APIs should include versioning to maintain backward compatibility. This means if changes are made from one version to another that could break the API, consumers have enough time to upgrade to the latest version. The most straightforward is to embed API version in URI:
- /api/v1/movies
is better than /api/movies
Use “singular” name to denote a single document (a.k.a. object), e.g:
- /api/v1/movies/{id}
- /api/v1/movies/{id}/rating
Use “plural” for collections and storing a new document
- /api/v1/movies
is better than /api/v1/movie
Use “verb” for controller. A controller resource models a procedural concept. Controller resources are like executable functions, with parameters and return values, inputs, and outputs. E.g:
- /api/v1/computers/{id}/shutdown
- /api/v1/cameras/{id}/turn-left
Use forward slash (/) to indicate hierarchical relationships. By convention, {id}
right after each resources refer to their identifier.
- /api/v1/movies/{id}/actors
~ /api/v1/movies/{movie-id}/actors
- /api/v1/movies/{id}/title
Use hyphens (-) to improve the readability of the URIs
-/api/v1/movies/{id}/release-date
GOOD
- /api/v1/movies/{id}/releaseDate
NOT GOOD
- /api/v1/movies/{id}/release_date
NOT GOOD
Query
As its name imply, Query is for querying data, e.g. to filter, and/or to paginate. I will not delve into pagination in this blog, but let me know if you’d like me to write about it in the comments. Instead, I will discuss how it should be used on the APIs level.
- Filter by fields
GET /api/v1/movies?director="Christopher Nolan"
- Paginate results with offset
GET /api/v1/movies?limit=20&offset=100
- Paginate results with cursor
GET /api/v1/movies?cursor="dXNlcjpXMDdRQ1JQQTQ="
- Sort results
GET /api/v1/movies?sort_by=-releaseDate
. We can use +/- for asc and desc
GET /api/v1/movies?director="Christopher Nolan"
HTTP/1.1 200 OK
Content-Type: application/json
[
{
"id": "897",
"title": "Inception",
"releaseDate": "2010-07-16",
"director": "Christopher Nolan",
"actors": ["Leonardo DiCaprio", "Ken Watanabe", "Joseph Gordon-Levitt"],
"summary": "A mind-bending thriller about a man who enters the
subconscious of his targets to plant ideas."
},
{
"id": "423",
"title": "The Dark Knight",
"releaseDate": "2008-07-18",
"director": "Christopher Nolan",
"actors": ["Christian Bale", "Heath Ledger", "Aaron Eckhart"],
"summary": "The Dark Knight of Gotham City begins his war on crime with his first major enemy being the clownishly homicidal Joker."
}
]
Query is good for not so complicated use cases. If you find a need of complicated query, GraphQL could be the better choice for your applications.
It’s also worth mentioning that Query is sometimes wrongly used for hierarchical relationships. For example, GET /api/v1/movies/123
and GET /api/v1/movies?id=123
have different meanings. The former requests a single Movie object with id=123
, while the latter requests a list of Movie objects with id=123
which just happens to have a single element.
GET /api/v1/movies/123
HTTP/1.1 200 OK
Content-Type: application/json
{
"id": "123",
"title": "The Shawshank Redemption",
"releaseDate": "1994-09-23",
"director": "Frank Darabont",
"actors": ["Tim Robbins", "Morgan Freeman", "Bob Gunton"],
"summary": "Two imprisoned men bond over a number of years, finding solace and eventual redemption through acts of common decency."
}
GET /api/v1/movies?id=123
HTTP/1.1 200 OK
Content-Type: application/json
[
{
"id": "123",
"title": "The Shawshank Redemption",
"releaseDate": "1994-09-23",
"director": "Frank Darabont",
"actors": ["Tim Robbins", "Morgan Freeman", "Bob Gunton"],
"summary": "Two imprisoned men bond over a number of years, finding solace and eventual redemption through acts of common decency."
}
]
Verbs
We should never use CRUD function names in URIs. Use HTTP verbs instead.
- GET /api/v1/movies
instead of GET /api/v1/listAllMovies
- POST /api/v1/movies
instead of POST /api/v1/createMovies
- GET /api/v1/products/{id}
instead of GET /api/v1/getMovie?id={}
- PUT /api/v1/products/{id}
instead of PUT /api/v1/updateMovie?id={}
- DELETE /api/v1/products/{id}
instead of DELETE /api/v1/deleteMovie?id={}
POST
is usually the choice for controller resources, even if the payload is empty
- POST /api/v1/computers/{id}/shutdown {}
- POST /api/v1/cameras/{id}/turn-left {args}
When updating resources partially is often, we should consider supporting PATCH
API too. It looks very much the same as PUT
but only requires changing fields.
Request and Response Body
The most widely used content-type for REST APIs is application/json. When it comes to naming conventions, using a consistent style is more important than whether it is lowerCamelCase
, UpperCamelCase
, or snake_case
.
If the size of data in requests and responses is not too large, it’s a good idea to consistently use the same data model for the same resource in all CRUD operations. This makes our jobs as engineers much easier. For example, to update a resource, the client just needs to GET it, make changes to the necessary fields, and then PUT it back. This is why it’s important to keep resource sizes reasonable but still include all relevant information for the client. If you are not so sure if your resources are too big, you can execute performance tests your APIs.
GET /api/v1/movies
HTTP/1.1 200 OK
Content-Type: application/json
[
{
"id": "897",
"title": "Inception",
"releaseDate": "2010-07-16",
"director": "Christopher Nolan",
"actors": ["Leonardo DiCaprio", "Ken Watanabe", "Joseph Gordon-Levitt"],
"summary": "A mind-bending thriller about a man who enters the subconscious of his targets to plant ideas."
},
{
"id": "1023",
"title": "The Dark Knight",
"releaseDate": "2008-07-16",
"director": "Christopher Nolan",
"actors": ["Christian Bale", "Heath Ledger", "Aaron Eckhart"],
"summary": "Batman, Gordon and Harvey Dent are forced to deal with the chaos unleashed by an anarchist mastermind known only as the Joker."
}
]
GET /api/v1/movies/897
HTTP/1.1 200 OK
Content-Type: application/json
{
"id": "897",
"title": "Inception",
"releaseDate": "2010-07-16",
"director": "Christopher Nolan",
"actors": ["Leonardo DiCaprio", "Ken Watanabe", "Joseph Gordon-Levitt"],
"summary": "A mind-bending thriller about a man who enters the subconscious of his targets to plant ideas."
}
PUT /api/v1/movies/897
Content-Type: application/json
{
"id": "897",
"title": "Inception",
"releaseDate": "2010-07-16",
"director": "Christopher Nolan",
"actors": ["Leonardo DiCaprio", "Ken Watanabe", "Joseph Gordon-Levitt"],
"summary": "Other description"
}
HTTP/1.1 200 OK
Content-Type: application/json
{
"id": "897",
"title": "Inception",
"releaseDate": "2010-07-16",
"director": "Christopher Nolan",
"actors": ["Leonardo DiCaprio", "Ken Watanabe", "Joseph Gordon-Levitt"],
"summary": "Other description"
}
Metadata
Including additional information with requested resources is common in APIs. For instance, returning details of a violation in the 400 Bad Request
response, or including metadata like a nextCursor
in a list. To handle this, it’s recommended to wrap the main data with additional fields. This approach makes unmarshaling the data easier, as the same model can be used for both successful and unsuccessful cases.
{
"data": {
// Generic struct of the requested resource
// can be null on failure
},
"metadata": {
"nextCursor": "optional, for list API only",
"message": "Something meaningful for human to read",
"errorCode": "ERR123"
}
}
Status code
It’s frequently seen that APIs always returns 200 OK
even on failures with the error codes in response body. I personally disrecommend this approach. The main reason which supports me is many REST clients built around the common understanding of Status Code, for instance Golang Resty:
resp, err := client.R().
SetHeader("Content-Type", "application/json").
SetBody(`{"username":"testuser", "password":"testpass"}`).
SetResult(&LoginResponse{}).
Post("https://myapp.com/login")
if err != nil {
// If the Status code is not 2xx, err is not nil
// indicating Login failed
}
// If the API returns 200 along with the error code in the body
// It makes our life a little harder
Another example of using Locust for performance testing.
class User(HttpUser):
@task
def login(self):
# If we return 403 on failure, this would be all we need
# as Locust can automatically categorize status codes for us
self.client.post(
"/login",
payload={"username":"user", "password":"p@ssword"}
)
def loginAlways200(self):
# However, if the API always returns 200
# We need to let it know that not all 200 responses are OK
resp = self.client.post(
"/login",
payload={"username":"user", "password":"p@ssword"}
)
if (resp.error is None):
recordSuccess(...)
else if (resp.error == ERR400):
recordFailure("Bad request")
else if (resp.error == ERR403):
recordFailure("Unauthorized")
else:
recordFailure("Unknown")
My recommendation is to comply with status codes’ definition:2xx
Success4xx
Client error, typically not retry-able5xx
Server error, typically retry-able
Headers
Other than default headers such as Authorization
Content-Type
, we can also use headers to exchange additional information, e.g. X-UserRole
on a system where one user can assume different roles with different privileges. By convention, custom headers should always start with X-
.
Location
header is typically used in returning 201 Created
for POST /resources
API. In fact, Spring ResponseEntity.created(URI location)
enforces Location header in Created
response. The value of Location response is the URI of the newly created resource, e.g. Location: /resources/1234
.
That’s a lot of information for the topic around designing REST API. If there is only one thing you could remember, it should be: Checkout this website https://restfulapi.net/ one more time and refer to it whenever you are in doubt!
Stock Exchange Application
It’s time to apply the techniques we have learnt so far to design REST APIs for our oversimplified Stock Exchange Application.
Let’s analyse the requirements to spot the APIs we need:
1. The ability to list new stocks and delist existing stocks, one at a time.
Listed Stock is a resource. There should be 2 APIs to list and delist Stock. In reality, Stock object might contain many information. However, in this oversimplified application, we will oversimplify it to:
Stock {
Symbol: String // Symbol is unique, hence can be used as ID
Company: String
}
2. Traders can submit orders to buy or sell listed stocks at specified prices. They can also cancel their unexecuted orders.
Order is another Resource with 2 APIs to submit and cancel it. Order should specify the Stock, quantity, price and its side of the trade.
Order {
Id: UUID
Symbol: String
Quantity: Int
Price: Long // Currency can be omitted for simplicity.
// In reality, Exchange typically operates around a single
// listed currency (Local currency to it). Hence, this
// is acceptable.
Side: BUY|SELL
}
3. The exchange will automatically match orders, but only fully filled orders at exact prices will be supported in the first version. Unfilled orders will be canceled at the end of the trading day.
This is an important requirement, however, it doesn’t ask for any APIs to be created. Instead, in only defines the trading execution — backend process.
4. Traders can check the status of their orders and view their historical trade data.
Checking Order status refers to a GET API on Order resource. It also tells that Order should have a field named Status.
Historical trade data means GET list of Orders in the past. We might need to support pagination on this API as data might be huge.
Supported APIs
Combining these analysis, we can come up with the APIs supported by the Stock Exchange application as below:
POST /api/v1/stocks
{
"symbol": "AAPL",
"company": "Apple Inc."
}
HTTP/1.1 201 Created
Location: /api/v1/stocks/AAPL
Content-Type: application/json
{
"message": "Listed AAPL successfully",
"data": {
"symbol": "AAPL",
"company": "Apple Inc."
}
}
DELETE /api/v1/stocks/AAPL
HTTP/1.1 200 OK
Content-Type: application/json
{
"message": "Delisted AAPL successfully"
}
POST /api/v1/orders
{
"symbol": "AAPL",
"quantity": 1000,
"price": 15101,
"side": "BUY"
}
HTTP/1.1 201 Created
Location: /api/v1/orders/f252e372-5a9c-4f71-82e1-c60e8421d353
Content-Type: application/json
{
"message": "Placed order successfully",
"data": {
"symbol": "AAPL",
"quantity": 1000,
"price": 15101,
"side": "BUY"
}
}
GET /api/v1/orders/f252e372-5a9c-4f71-82e1-c60e8421d353
HTTP/1.1 201 OK
Content-Type: application/json
{
"message": "Get order successfully",
"data": {
"id": "f252e372-5a9c-4f71-82e1-c60e8421d353",
"symbol": "AAPL",
"quantity": 1000,
"price": 15101,
"side": "BUY",
"status": "NEW"
}
}
DELETE /api/v1/orders/f252e372-5a9c-4f71-82e1-c60e8421d353
HTTP/1.1 201 OK
Content-Type: application/json
{
"message": "Canceled order successfully",
"data": {
"id": "f252e372-5a9c-4f71-82e1-c60e8421d353",
"symbol": "AAPL",
"quantity": 1000,
"price": 15101,
"side": "BUY",
"status": "CANCELED"
}
}
GET /api/v1/orders
HTTP/1.1 201 OK
Content-Type: application/json
{
"message": "Get order successfully",
"data": [
{
"id": "f252e372-5a9c-4f71-82e1-c60e8421d353",
"symbol": "AAPL",
"quantity": 1000,
"price": 15101,
"side": "BUY",
"status": "CANCELED"
}
]
}
Exercise 1. Design REST APIs for a library management application with resources such as Books and Authors.
Exercise 2. Design REST APIs for a online shopping application with resources such as Products, Cart, Orders, ShippingAddress and Payment.
That’s long enough, see you in the next blog post where we will beautify the APIs spec with OpenAPI!