Designing a REST API

How to Create a REST API That You Can Learn to Love

Designing a solid REST API is not as hard as you might think. But to do it, first you have to understand that REST is not a procedural API like those with which you are most familiar. It is an API for reading and writing resources, as well as manipulating and transferring their state across networks.

What Is REST?

Representational State Transfer is not a software engineering standard, but is rather an architectural style. REST has become popular because it can be used to implement networked and Web-enabled applications.

The term was first described by Roy Fielding, back in 2000, as part of his doctoral dissertation. Fielding is one of the principal authors of the HTTP specification. He is an authority on Network-based Software Architectures and co-founded the Apache HTTP Server project. As you will see, some of REST’s conventions are based on lessons learned from using HTTP.

REST was intended to describe an image of how a well-designed Web application might behave. It postulates a model through which a user or application can access, create, modify, or delete resources on a network.

A resource can be anything with a state for which there is a representation that can be transmitted over a network. That includes things like images, video, sound, text, spreadsheets, data tables, hypertext, relational and document database entities, computational results, and even process or program state.

A representation can be any serializable data format that expresses the detailed state and semantics of a resource, so that it may be transmitted across a network as a byte array and reconstituted at the other end without losing either meaning or precision. Unlike a Java object, a representation describes only data state, not behavior. Representations, unlike procedure calls, are independent of programming language and execution environment considerations.

REST takes its name from the proposition that a thing can be created or manipulated through its representation on the client side, and that the representation can be transferred to the server to set its state there. It is simple, clean, and elegant if understood and used properly.

The REST Information Model

Contrary to a common misconception, REST does not exist solely to interact with relational or document databases. It is well-designed to transfer the representations of many kinds of objects back and forth between clients and servers. That just happens to make it useful when we need to talk to a DBMS.

Why Do We Use REST?

Tim Berners-Lee, probably more than any single individual, was responsible for shaping the Internet into the World Wide Web, when he proposed the concept of hypertext on the Internet to CERN and then published the world’s first Website on 20 December 1990.

On the surface, a World Wide Web using hypertext seems to be a very simple idea, but it was a stroke of genius with enormous impact on the world as we know it today. Before the Web, the Internet was primarily the domain of academics and defense contractors (DARPA). Today, it is hard to imagine the developed world without the Web.

Representational State Transfer was another of those strokes of genius. REST extends the hypertext concepts of the Web with a clearly defined and standardized model for retrieving, transferring, and manipulating the state of complex objects across a network.

Fairly early on, HTTP and HTML became capable of handling media types other than just hypertext and images. But there were still serious limitations in dealing with representations of some types of objects.

As developers needed to create more complex and capable Web applications, the programming concept of AJAX (or Asynchronous JavaScript and XML) enhanced the ability to interact with server-based systems without freezing the UI waiting for responses. Having to manipulate the XML in JavaScript was a far from ideal solution; so many AJAX developers quickly replaced XML with JavaScript Object Notation (or JSON) which is native to JavaScript.

REST, combined with AJAX, provides a clean and architecturally consistent way to interact with complex objects residing on networked servers. Properly implemented, REST delivers a lightweight and implicit contract between clients and servers; which is a priceless benefit for both developers and maintainers.

From a runtime performance perspective, it also distributes much of an application’s workload from the few servers to the many clients — delivering built-in scalability. As you read further, you will see what a useful and powerful tool REST can be, not only for browser-based applications, but for any networked application needing to interact with a server.

If you are seriously interested in how REST can positively impact application development, you might enjoy reading Building Software Systems.

Popular REST Alternatives

In the opinion of some developers there are two superior alternatives to REST APIs that you might want to consider:

1. gRPC also known as Google Remote Procedure Call is an open-source remote procedure call system initially developed at Google in 2015 as the next generation of its RPC infrastructure. It uses HTTP/2 for transport, Protocol Buffers as the interface description language, and generates cross-platform client and server bindings (stubs) for many languages. gRPC’s complex use of HTTP/2 makes it impossible to implement a gRPC client in the browser, instead requiring a proxy.
2. GraphQL is an open-source data query and data manipulation language for APIs, and a runtime for fulfilling queries with existing data. It was publicly released by Facebook in 2015. It allows clients to define the structure of the data required, and the same structure of the data is returned from the server, therefore preventing excessively large amounts of data from being returned. But this has implications for how effective web caching of query results can be. The flexibility and richness of the query language also adds complexity that may not be worthwhile for most applications.

It is my opinion that, except under very limited circumstances, these are not useful and economical alternatives to REST APIs and neither alternative promotes the lightweight and implicit contract between requesters and responders that is a hallmark of REST. Here are some of my reasons:

1. [gRPC] When messages (and records) are not self-describing, the added overhead and complexity of a distributed schema management system is required. Protocol Buffers require just such a schema management system. A Protocol Buffer message is totally indecipherable without its schema definition, making version management a real challenge. Very few (but some) applications require the extreme message compression that Protocol Buffers can deliver. A combination of JSON and GZIP can deliver the message compression most applications require; without gRPC’s added complexity and cost. If compression requirements justify it, Protocol Buffers can be used with REST as well. It should be noted that GZIP is pretty much the standard compression mechanism for the entire Internet and is supported by both browsers and servers.
2. [gRPC] Remote procedure calls are an obsolete and less flexible alternative to REST messaging. Distributed and cloud-capable applications are easier to design and implement with a message passing metaphor than with a networked implementation of the old C function call. RPCs are strictly point to point and are not typically amenable to real-time redirection for load balancing or when network failures occur.
3. [GraphQL] Back in the 1980s when SQL databases were initially gaining popularity, we all rushed to put SQL queries into our app code. For many reasons — among which was creating a rat’s nest of undocumented dependencies — we learned that was a bad idea and started moving the SQL into a database access layer organized by data entities. That approach eventually evolved into the use of REST APIs for Web and networked applications. I guess that the excited proponents of GraphQL will have to relearn that lesson for themselves. Ad hoc queries and application data access are two very different things. And query languages and query errors are best handled outside your application specific code.
4. [GraphQL] You will see as we design our REST API, there are RESTful ways to refine both the resources selected and the number of rows and columns returned. This can be done within the REST API and without the added complexity of learning and using a separate query language.

You will need to be the judge of whether the complexities inherent in gRPC and GraphQL are offset by their benefits in your particular use cases.

REST Architectural Constraints

To meet the objectives of the REST architectural pattern, a RESTful implementation should enforce the following architectural constraints:

1. Decouples consumers from producers by promoting a lightweight and implicit contract between requesters and responders and lending itself to building client server and distributed computing solutions
2. Stateless existence by ensuring each request from any consumer contains all the information necessary to service the request, and session state is maintained only on the consumer side. Resource state is maintained on the producer side.
3. Able to leverage a cache by ensuring that the use of HTTP methods follows the rules of safety and idempotency specified by the HTTP standard. Most specifically that the GET method is always guaranteed to be safe and idempotent to facilitate cache management and that responses, explicitly or implicitly, identify themselves as cacheable or not. GET responses are implicitly cacheable and all others are not.
4. Leverages a layered system by ensuring that consumers do not need to know if they are directly connected to end servers or through intermediaries, enabling easier management of load balancing, caching, and security policies.
5. Leverages a uniform interface by presenting a clear, unambiguous, and implicit application interface with the following characteristics (see below).
6. Code-on-Demand (Hypermedia as the Engine of Application State (HATEOAS)) is an interesting part of the REST architectural pattern, however it is less often found in common usage. Used creatively, HATEOAS can enable a server to respond with complete executable Web components, not just data.

The Attributes of a Uniform Interface

A major strength of REST is the clear, unambiguous, and implicit API that it promotes:

1. Identification of resources: Individual resources are uniquely identified in requests. The resources themselves are conceptually separate from the representations that are returned to the consumer. For example, the server may send data from its database as JSON, which is not the server’s internal representation.
2. Manipulation of resources through representations: When a consumer holds a representation of a resource it has sufficient information to request that the producer create, modify, or delete the resource on the server.
3. Self-descriptive requests: A request includes enough information to describe to the producer how to process the request and uses HTTP methods to specify the desired intent.

The Challenge in Designing a REST API

The challenge is to define a clear, concise, and consistent API that meets both RESTful architectural constraints and application requirements — in an easy-to-use, economical, extensible, and sustainable fashion. It should enable a developer to read and write data (resources). Using a RESTful API, a developer should be able to create (POST), read (GET[uid]), list (GET[where]), update (PUT and PATCH), and delete (DELETE) resources.

Chapter 5 of Roy Fielding’s doctoral dissertation, Representational State Transfer (REST), describes in great depth the reasoning for the architectural constraints. It is written as an academic paper, not for popular consumption — but it can be well-worth the effort to read. In this post, we’ll touch on the practical application of those principles when designing a REST API.

So What Is a REST API and Where Can We Use It?

The expressed purpose of the REST architectural pattern is to access, navigate, and manipulate the state of resources in a networked environment, all while promoting a lightweight and implicit contract between clients and servers. Therefore, a REST API is first and foremost focused on things (the resources) before actions (the verbs). First we create or locate the thing on the network and then we can act upon its state.

Part of the elegance of REST is in the simplicity of the underlying concept. Because we can do anything we wish directly to a resource representation itself — rather than having hundreds of unique function calls to document and learn, we need only a finite number of specific REST calls. They are:

To some, DELETE (serverUrl, resourceType) might be considered a legitimate REST API call, but as it would delete all of the resources of a type, prudent implementations usually reject any DELETE request without a unique resource identifier. From another perspective, any request that changes the state of a resource should always contain a unique resource identifier. At a minimum an all-inclusive DELETE should at least require an explicit confirmation of intent in the call.

As you can see, a real REST API call is definitely not a remote procedure call. It is resource representation-oriented and therefore has a finite set of functions. REST promotes a lightweight and implicit contract between requesters and responders.

The important things that vary from call to call are the verb (HTTP method), the server identity, and the resource with which you are working. The pieces that need to be documented are the resource representations (effectively the data layout and acceptable values to represent the resource state); and any parameters you create to refine or modify behaviors.

REST views the entire Internet as one giant repository of resources to be accessed and edited. In fact, though most of us are too young to remember, that is how the original Web worked and why the HTTP methods POST, PUT, and DELETE exist. When the Web was pretty much limited to academia there was little need for security and almost anyone who knew and could access the server URL could:

1. POST a new resource (document or image) to a Web server.
2. GET a resource from a Website, change it, and PUT it back.
3. DELETE a resource from a Website.

Academic peer pressure was sufficient to minimize deliberate abuse. There was absolutely no chance of that idyllic state lasting after the Web went public.

Though originally intended for use on the Internet, REST is appropriate for interacting with resources in any networked environment. Roy Fielding was one of the principal authors of the HTTP specification, and REST is intended to take advantage of many of HTTP’s attributes. However, Fielding was careful not to limit REST to HTTP. REST and its close relative, Event Carried State Transfer, are usable with many different network transports.

How are REST and ECST related? REST is used for sending requests and ECST for posting events. We use REST to tell the target we want something to happen. We use ECST to tell subscribers that something has happened. They both have a target, and the payload for both is the identity and representation of the state of some resource, coupled with an HTTP method that specifies what we want to happen or what has happened. For REST, the target is a server and for ECST, the target is an event topic queue.

A Typical REST Scenario

Using REST, a client can:

1. Create a new resource of a specific type by POSTing its representation to a server.
2. GET a list of resource representations, from a server, using its resource type (with or without selection criteria).
3. GET a representation of a unique resource, from a server, using its resource type and unique identifier.
4. PUT a modified representation of a resource back onto the server using its resource type and unique identifier.
5. PATCH is a more network-efficient version of PUT that uses diffs instead of entire representations to transfer state (see JSON Diffs).
6. DELETE a resource from a server using its resource type and unique identifier.

If the client is written in JavaScript running on a browser it will typically work directly with a serialized JSON representation of the resource. If the server code is written in Java, it is usually convenient to work with a Java object representing the resource or with a hierarchical Java Map object loaded by parsing the JSON. Fortunately, it is fast and simple to serialize Java objects into JSON and deserialize JSON into Java objects.

The Power of Representations

Too often, people new to REST try to implement traditional procedural API calls with it. Unfortunately, that approach misses out on the intended benefits of the REST architectural pattern — including the power available through representations. Many of the criticisms of REST we hear are, in fact, criticisms of the misuse of REST. Let’s try to avoid that problem and realize the very real power of REST representations.

A representation can be any serializable data format that expresses the detailed state and semantics of a resource, so that it may be transmitted across a network as a byte array and reconstituted at the other end without losing either meaning or precision.

REST APIs invoke services. It is useful for an individual service to operate within the Domain-Driven Design construct of a bounded context. For the purposes of designing with a REST API, it is also useful to view a service as a resource and its representation as the logical data model of that resource.

As an example, let’s take a simple sales order. One approach might be to implement a resource, with its representation, for each entity involved in creating a sales order: SALESORD, COMPANY, CUSTOMER, PRODUCT, ORDLINE and INVOICE. We need those resources in our underlying database, so that’s probably a good idea.

However, that does make coding the Sales Order client fairly complicated because it requires coordinating a whole set of REST API calls to multiple resources — along with the implied requirement of user interface management. Fortunately, REST leverages a layered system, and with representations, can provide us a better way.

In this scenario, the Sales Order client makes REST requests only to the Sales Order Service using the Sales Order representation. REST’s layered services hide the data navigation complexities of the underlying data model from the client and enable it to work with a sales order as one unified document with customer and company information, multiple order lines that each contain a product, and ultimately with an invoice.

For clarity, this Sales Order representation has been left fairly bare-bones. But, hopefully, the underlying concepts are clear. To write the client using conventional procedural APIs, there would be many functions implemented by many different API calls, each one to learn and use — and many opportunities to misunderstand and to use them incorrectly. A change to any detail in any one of those API calls could break the client.

Using the REST architectural style, there is one representation, the Sales Order, to learn — and at most 6 standard REST APIs to use: create, list, get, put, patch, and delete. The REST concepts of leveraging a layered system and a uniform interface give us real advantages over procedural APIs when managing complexity. The Sales Order representation enforces all data integrity and relationship rules insuring that any validation error in the UI cannot make it into the underlying data stores.

This use of a composite structured resource representation and layered services has the added benefit of providing, at the top-level service, a way to manage database transactions in order to ensure that the underlying data stores do not get into an inconsistent state. An otherwise difficult challenge for stateless REST.

Version Management with REST

How we choose to manage the coexistence of multiple versions of our REST resources will impact the extensibility and sustainability of our solution. The following approach works well with JAX- RS 2.0 URI path handling.

Conceptually there are two versions to compare:

1. The client version is the version of the resource from the perspective of the resource consumer. It is always the first part of the request URI, i.e., /v1/customer/374. JAX-RS uses that URI part to route a request to the Java servlet and method that processes it.
2. The server version represents the version or versions of the resource that are understood by the server and ultimately identifies the Java servlet and method that will process the request.

A JAX-RS 2.0 pre-matching filter examines the client version and directs the request to the appropriate servlet method compatible with the version (or returns a 404 error code if there is no match). The client request can be matched to the latest service version with which it is compatible.

This approach can be implemented at a granularity as fine as an individual servlet method, i.e., multiple methods matching the same HTTP method, but with different versions, can co-exist in the same servlet.

This strategy requires that a catalog of service module versions, and the client versions they support, be available for the pre-matching filter to use. This can be implemented through a custom Java Version (supported client versions) annotation in service modules. The server builds the catalog by parsing the JAR files in its class path. This takes seconds and happens only once at startup.

REST Requests

A REST request is, in its implementation, an HTTP Request. Depending upon the programming language you are using, your implementation of a REST API helper function will vary. These are the important parts with which you will work. For example:

GET: sales.abc.com/v1/SalesOrder?offset=0&limit=100&columns=name,ordId,orderDate&where=[status=Pending]&orderby=name

• The HTTP method (POST, GET, PUT, PATCH, or DELETE) that identifies the requested action. The combination the method and resource path determine the other input parameters.

TABLE 1 — REST/HTTP Methods

• The server address (sales.abc.com) that identifies the specific REST server to which the request will be delivered.
• The resource path or URI (/v1/salesOrder) that identifies the specific resource type/service to which the request will be delivered. The client version (the service version expected by the client) is always the first part of the URI. The resource path can include the individual resource’s unique identifier (/v1/salesOrder/B214873) to GET, PUT, PATCH, or DELETE a specific resource.
• An optional query string (?offset=0&limit=100&columns=[ordId, name]&orderby=[name, ordId]) that provides additional input to the service.
• An optional body that contains the resource representation. The body is only required when the REST method is POST, PUT, or PATCH so that the state can be transferred to the server. The body is not part of the request line but is carried within a separate part of an HTTP Request.

Barring a network partition or catastrophic failure, any request that reaches the server will return a response code in its HTTP response.

REST Responses

A REST response is implemented as an HTTP Response. Depending upon the programming language you are using, your implementation of a REST API helper function will vary. These are the important parts with which you will work. For example:

• The response code that specifies the result of the request. Depending upon that code and the type of request being responded to, the other data carried by the response will vary. Typically, a successful GET request will also return an array of resource representations (because it is never safe to assume a query will return only one answer). Optionally, a POST can be programmed to return the newly created resource or it just the new identifier created for it. TABLE 2 — REST/HTTP Response Codes contains a list of the standard HTTP Response Codes and their RESTful use.

TABLE 2— REST/HTTP Response Codes

• An optional body that contains an array of the resource representations returned in the response.

REST Heartbeat

It is often useful for a REST client to confirm that it is connected to a responsive and healthy server. To support that need it is handy to implement a low-overhead “heartbeat” service that can be invoked to confirm the status of a server. It is customary to use the HTTP HEAD method for that purpose because, by default, it returns very little data.

REST Query Filters

Query Parameters (query string) are used to identify options to be used in processing the request. The absence of an option should not cause the failure of the request and, where necessary an option should have a default.

1. offset=n Starts a result set array of resources at the nth (zero-based) item. offset defaults to 0.
2. limit=n Limits the size of a result set array of resources to n. Defaults to an application assigned value (perhaps 20 or 50). A limit of -1 signifies unlimited (caution, might result in memory and/or network issues). If there are more resources remaining when the limit is reached, a response code of 206 — Partial Content is returned.
3. columns=comma-separated values list of columns to be selected. Defaults to *.
4. where=[key = value pairs separated by commas] Acts like a SQL “where” clause to filter a result set of resources. No default is needed.
5. orderby=comma-separated value list of one or more attribute names Sorts the result set of resources (honored by offset and limit). No default needed.

TABLE 3 — Sample Query Filters

When dealing with resources managed by a relational or document database management system, these parameters can implement some of the features so prized by GraphQL proponents, while honoring REST constraints. It is relatively simple to use these parameters to format server-side queries.

HTTP Headers

HTTP Requests and Responses implement the concept of headers that are part of the HTTP messages transmitted back and forth between clients and servers within requests and responses. They are essentially name/value pairs with Java methods for setting and getting them. They are very useful for passing additional information in calls. If the name begins with an ‘X’ and the value is a string, you can use them in any way you wish within your REST API. TABLE 4, below has some useful samples. “Location” has an already standardized meaning.

TABLE 4 — Sample HTTP Headers

REST API Call Helpers

Working directly with Java (or other) APIs to manipulate HTTP Requests and Responses is tedious and error prone. It is much more effective to create your own API call format in your implementation language to make REST requests and handle REST responses through the underlying HTTP APIs.

A very compelling reason to use REST API call helpers is that they can insulate you from HTTP considerations. REST and ECST do not require HTTP. In fact, ECST typically uses message queue technologies like Kafka or ActiveMQ Artemis. REST can use any network protocol you choose. Keep your options open.

The helper APIs translate between the call parameters and an HTTP Request and translate the HTTP Response into a developer-friendly RestResponse. There is only the one set of six API calls, because the only things that vary in use are the server address, the URI, the query string, and the body. For example:

RestResponse response = RestAPI.POST(String server, String URI, Object body);
RestResponse response = RestAPI.GET(String server, String URI, String queryStr);
RestResponse response = RestAPI.PUT(String server, String URI, Object body);
RestResponse response = RestAPI.PATCH(String server, String URI, Object oldBody, Object newBody);
RestResponse response = RestAPI.DELETE(String server, String URI);
RestResponse response = RestAPI.HEAD(String server, String URI);

Wrapping Up

Representational State Transfer is a useful and powerful model for connecting clients and servers across networks. A real REST API call is definitely not a remote procedure call. It is resource representation-oriented and therefore has a finite set of functions. REST promotes a lightweight and implicit contract between requesters and responders.

Too often, people new to REST try to implement traditional procedural API calls with it. Unfortunately, that approach eliminates the intended benefits of the REST architectural pattern — including the power available through representations. Many of the criticisms of REST we hear are, in fact, criticisms of the misuse of REST. Let’s try to avoid that problem and realize the very real power of the REST model.

For those who are interested in implementing a REST API in Java, I highly recommend RESTful Java with JAX-RS 2.0: Designing and Developing Distributed Web Services 2nd Edition for its coding knowledge. It is written for Java EE JAX-RS 2.0, but Jakarta EE REST libraries are virtually identical except for “jakarta” instead of “javax” package names.

If you found this article useful, a clap would let us know that we’re on the right track.

Thanks!