Design Patterns for REST-APIs
Plus a quick primer on what they are…
Intended audience: API-developers, data modellers and (domain) architects.
Now how do we design good REST-APIs? It is not difficult as long as you stick to some simple patterns. It is basically just data modelling, which is a common skill based on solid principles. Following are the process and design patterns that will help you design good APIs.
The cheat sheet is here.
HTTP and REST
Things on the internet communicate using a protocol called HTTP. For instance, your browser sends HTTP-requests to a webserver and the webserver sends HTTP-responses back. REST is a HTTP design pattern, ‘the right way to do HTTP’. When implemented according to the REST pattern, HTTP is a client-server protocol with which clients can query and manipulate the resources of a server, like:
- ‘Create a new resource on this URL’
- ‘Return me the resource on this URL’
- ‘Delete the resource on this URL’
- ‘List me all your resources on this URL’
Resources can be anything from the content of webpages, to representations (proxies) of real-life things such as sensors, to data in a database and even the volatile results of calculations. In the REST model they all have an URL and a set of ‘verbs’ to create, change, update or delete them (‘CRUD’) and the server treats every request like it is the first it receives, no client history or state. Note that the ‘R’ in URL stands for Resource. Universal Resource Locator.
Clients can be anything from browsers, cars, apps, corporations to your smart home, it doesn’t matter, but not humans (who usually connect through man-machine interfaces like websites or apps). Servers can be anything too (but not humans either, although manual processes can be part of the implementation of a server). That’s why we call it the internet of things, the IoT. There are far more machines on the internet than humans.
It is fair to say that HTTP/REST conceptually maps everything on the internet, virtual as well as physical, onto URLs to make these things accessible for ‘other things’.
Everything that connects to the internet can be a server in one conversation and a client in another. That is why we call the internet a peer-to-peer structure, or ‘network’. The REST-API is the native interface or connector in this network. In other words: it is the best way to expose functionality or resources to the rest of the internet.
On the internet of things everything is either a REST-server or a REST-client. Or both.
and…
A REST-API is the collection of HTTP-requests that a REST-server can respond on.
and…
A RESTful HTTP-request is ‘a query or manipulation on a server-resource addressed by an URL’.
That simple, REST-HTTP.
Not everything talks HTTP (some talk SMTP for mail, others XMPP for chat and there are many more protocols). There are also other styles than REST (such as GraphQL and SOAP), but it is safe to say that REST and HTTP are the de facto standard on the internet of things (on the level of applications, engineers might want to look into the protocol stack called the OSI-model now). In principle the REST pattern can be applied to other protocols than HTTP too, but let’s not go there in this article.
Some examples
REST-API expose resources. An insurance company API would expose resources like customers, policies, quotes, covers, claims, etc. A connected home API would expose resources like locks, lights, solar panels, sensors, controls. And finally a piece of mathematical software could expose resources like algorithms, calculations or inputs. All with unique URLs like:
- https://nn.nl/api/v1/customers/piet@gmail.com
- https://207.13.46.59:8081/home/bedroom/main-light
- https://vw.com/car/serial-983-634-23/statuses/engine/sensor/lambda
So in the insurance company case, creating a new quick quote on a life-insurance policy could be a request like:
And to read the status of a light in your connected home:
Designing a REST-API in 6 steps
REST-API design typically is the domain of developers with strong data modelling skills. Which is logical once you realise REST is based on resources and those resources follow from a data model.
(See: datamodeling in UML, domain modeling and data modeling)
For an enterprise (or any other connected thing) to become ‘digital’ or ‘connected’ it must become a REST-server and expose its resources through the REST-API as its primary channel. To do this in a consistent way, the public resources (accessible and manipulable by clients through the API) must first be modeled by a domain model. This model then acts as the metaphor or map through which clients can interact with the API. The model can contain virtual as well as physical resources, and their associations. It describes ‘the domain’ that the server controls. See an example below. Don’t fall in the trap of creating a canonical model for the entire enterprise, that’s an anti-pattern. Use bounded contexts, maybe even per REST-server.
Build a domain model (ER-diagram or class-diagram) and publish this
Next, every resource (-type) in the model that must be exposed to the outside world, must be mapped on a URL. These URL’s must also reflect the (hierarchical) relationships they (can) have to the other resources, this makes the API intuitive.
Create an URL-schema that reflects the domain model
For this I created some design patterns you can use, see below this article. Next assign the required manipulations (the HTTP verbs such as POST, PUT, PATCH, DELETE, GET) to every entity.
Assign HTTP verbs to the entity
Next, create the request and response formats for all relevant entities in the model
Each entity gets its own set of requests (and corresponding responses)
In principle every entity in the domain model gets its own set of request and corresponding responses. The entire interface is than represented with an OpenAPI specification
Specify the entire API with a OpenAPI/Swagger file
A modern API should also broadcast events (notifications) for relevant state changes or resource mutation. There could be private events, ment for a specific API-client / user, and public events. In the Azure cloud this can be done using the Event Grid. A server can also accept registrations of webhooks for this and manage notifications itself.
Specify per resource which events are broadcasted and when
The meaning of the HTTP-verbs
Mapping resources onto URLs allows clients to manipulate a resource with one of the following verbs:
- POST, to create a new resource (or any other non-idempotent operation*)
- PUT, to replace an existing resource
- PATCH, to (partially) update an existing resource
- DELETE, to delete an existing resource
- GET, to retrieve an existing resource
*) In computing, an idempotent operation is one that has no additional effect if it is called more than once with the same input parameters. In math: f(f(x)) = f(x). Only the POST-verb is non-idempotent in REST.
The standard URL scheme
To reflect the domain model in the URLs of the API, we must first assign URLs to the ‘wholes’ in the model. These are the resources for which instances can exist on their own, the parents in a relationship.
Consider this domain model I created for Dutch Railways (it is a fictitious model for educational purposes).
By inspecting the model we can see ‘Orders’ is a root resource or parent because instances of orders can exist on their own (this follows from the multiplicity or cardinality of the relationships). Following the relationship to ‘TravelPlans’ we can see its instances are parts, or children, of ‘Orders’ and instances of ‘Travelers’ are parts of ‘TravelPlans’. There is a clear hierarchy which must be reflected in the basic URL-scheme for this API:
- /orders
- /orders/{id}
- /orders/{id}/travelplans
- /orders/{id}/travelplans/{id}
- /orders/{id}/travelplans/{id}/travelers
- /orders/{id}/travelplans/{id}/travelers/{id}
The only reason for this is understandability (‘API UX’). It is not forbidden to have some ‘flat URLs’ too. There probably even is a need for those in some uses cases. For instance, to list all ‘Travelers’, regardless to which ‘Travelplans’ they belong, the endpoint would be:
- GET /travelers
- GET /travelers/{uuid}
The returned entities should probably contain links to the parent ‘Travelplans’. Keep the API minimal (but complete).
The standard set of endpoints
Given these URLs, we can now add manipulations. A standard set for ‘orders’ would be:
- POST /orders (post/add a new order in the set of orders)
- GET /orders (get a list of orders)
- PUT /orders/{id} (replace an order)
- DELETE /orders/{id} (delete an order)
- GET /orders/{id} (get a single order)
This set is sometimes referred to as the Collection-Metaphor.
The OpenAPI definition
The standard way to present an API is in a Swagger or OpenAPI definition. The OpenAPI definition of the API that I created based on the domain model above is here:
You can copy-paste it into the Open-API editor here:
Implementing the server
On the implementation level, every response is transformed into a request in the following steps:
- authentication
- authorisation
- input validation
- <the actual transformation or resource CRUD>
- notification (broadcast the event)
- output filtering
Don’t use complex design patterns or class-models, a REST-SERVER is ‘just’ a pipeline.
Implementing asynchronous responses
When the server can’t deliver the response to a request synchronously (immediately) it needs to register a webhook from the client. The server can then call this webhook asynchronously. This webhook receives the delayed response without the need for the client to wait for it. Most probably it will be a POST method into the client. The webhook is an URL supplied by the client with the request.
A similar method can be used for generic notifications: clients can register webhooks that will be called whenever the state of a resource changes. The server can delegate the actual broadcasting of these notifications to a component like Azure Event Grid if large scale is needed.
These are in a nutshell the REST-API basics.
The design process in some infographics
The design patterns
The design dynamics / process
