Returning errors in a REST API

As API developers, we read in almost every post about REST APIs that developers consuming our services are our customers, we have to treat them well and make their lives easier.

When you are using someone else software and you get an error it can be very, very frustrating:

1. Did you cause it?
2. What the real problem is?
3. Is there anything you can do to fix it and move forward?

We should do a good job at explaining other developers what went wrong, especially when they can do something to fix the problem and move forward.

Client errors vs Server errors

At a very high level, errors belongs with two different scopes when we deal with a REST api:

1. The Server: the client sent valid data, but some exception or error happened on the server. There is nothing the client can do about it. Typically such a request results in a 5xx response status code.
2. The Client: the client sent some inconsistent data with the request or a wrong url was used, and it is the client’s own responsibility to fix it: there is nothing the server can do. Typically such a request results in a 4xx response status code.

While in the first case we may not want to leak sensitive information to the client, in the second case it is our job to give as much information to the client as we can, in order for the client’s developer to easily fix the issue and move forward.

This post focuses on the second case, errors caused by the clients.

Disclaimer: I stole most of the ideas you will find below from other articles, common sense or endless conversations with members of the teams I worked in :D

Domain and endpoints

We will use a real example, one single domain object: JobCandidate.

A JobCandidate has only the following fields:

JobCandidate:
 - firstName : string : required
 - lastName : string : required
 - dateOfBirth : ISODate : required
 - withOwnCar : bool

Our very basic REST API provides the following endpoints:

1. GET: /job-candidates?[pageSize=50][&page=1][&orderBy=firstName-] to retrieve a paged list of candidates.
2. GET: /job-candidates/123 to retrieve a single candidate.
3. POST: /job-candidates --> { ...fields... } to store a new job candidate.

What can go wrong?

Actually there are many things that the consumer can get wrong and should be made aware of, even in such a simple REST API.

User is not authenticated

The request is unauthenticated and this endpoint requires authentication.

Not enough privileges

The client is authenticated but it does not have enough privileges to hit this endpoint.

Wrong url

GET: /jjjobs-crudinates

Url /jjjobs-crudinates doesn’t exist.

Wrong or unexpected query string parameter name

GET: /job-candidates?WRONGpageSize=50

Query string parameter WRONGpageSize does not exist.

Wrong query string parameter value

GET: /job-candidates?pageSize=fifty

Parameter pageSize must be numeric, cannot be a generic string.

Resource does not exist

GET: /job-candidates/123

Candidate with id 123 does not exist in the system.

Empty body in POST request

POST: /job-candidates
Request: null

A post request lack the json body representing the resource we intend to create.

Unaccepted method on url

DELETE: /job-candidates

You cannot send a DELETE request to a collection endpoint.

Required field in POST request is missing

POST: /job-candidates
Request:
{
  "firstName": "Luca",
  "lastName": "Abbati"
} 

Based on the domain definition we gave at the begin of this section, field dateOfBirth if required but it is not included in the request’s body.

Unknown field in POST request

POST: /job-candidates
Request:
{
  "firstName": "Luca",
  "whatIsThis": "Don't know"
}

Field whatIsThis is unknown.

Field type in POST request is not valid

POST: /job-candidates
Request:
{
  "firstName": "Luca",
  "lastName": 12
}

12 is not a valid last name for a job candidate.

Business rule violation

POST: /job-candidates
Request:
{
  "dateOfBirth": "2000-01-01",
  "withOwnCar": true
}

The candidate is not allowed to drive a car (she is too young) but we are setting withOwnCar to true, clearly a non-sense.

As you can see, even for such a simple API there may be many possible sources of errors in the interactions between the consumer and the server. If we want to consistently guide the user to a solution, we have to find a way to return errors which is compatible with all the use cases above and that is able to scale as requirements an business rules grows in number.

General consideration

Even before going deep into the various use cases we want to cover, we can already anticipate that:

1. Errors may appear at both url, method and body level.
2. Errors may be specific to a field in the body request, e.g. when I pass a lastName: 12 and 12 is clearly an invalid last name.
3. Errors may be caused by some business rule violation, e.g. when the user tries to set a dateOfBirth indicating a candidate less than 16 years old and at the same time she is providing withOwnCar: true. In this case both dateOfBirth and withOwnCar presents correct values (from the syntactical point of view), but their values, together, have no sense. The error in this case is referred to the request itself, and not to a specific field.
4. There may be more that one error in a single request, and whenever possible we would like to return info about all of them.

Error responses

As a first step I want to focus on what we want to tell the client:

• Whether it is a client error or a server error.
• If it is a client side error, it would be nice to provide a reference, e.g. a link or an error code, that can be used to know further details about what just happened and possible solutions.
• If the error is specific to a field, the field name (or path).
• A very short explanation of what caused the error. This message is not intended to be shown to the end user, e.g. in a web application. It is rather intended to give some immediate context to the developer using the API.

Http status code

We use http status codes to indicate whether it is a server side error ( 5xx status codes) or a client side error code ( 4xx status codes).

In particular, for the client side error codes we are interested in, I would not return too many different error codes, limiting the possible values to:

• 400 for validation errors that involves wrong data sent to the server,
• 401 when the user is unauthenticated and
• 403 when the user is unauthorized

Errors representation

Let’s start with the root level object. For sure there will be a field containing the array of errors, so let’s call it errors.

{
  "errors": [
    ... error 1 ...,
    ... error 2 ...
  ]
}

Each error will have a few fields that will be the same across different types of errors, while others may be very specific. I find a good idea to enclose them in an higher level field, e.g. data.

Representation of each element in the array of errors:
{
  "message": "Some human readable message",
  "code": "some.domain.error.name",
  "data": {
    "field": "optional field name, if error is related to a field",
    "otherField": "this can vary",
  }
}

Message: It is a textual field that you use to give a context or a very high level explanation of what happened.
*It is not intended to be shown to the end user*, so depending on the specific case, you can decide to be more or less specific here. But keep in mind that this can be considered “public”, so don’t write down plain passwords here :D

Code: The error code is a unique identifier of the error. Many of you may legitimately ask “why is it a string and not a number”? Well, there are a few reasons actually.
First, I have worked in projects when error codes returned by the API were (are) numeric and things get confusing quite soon. It is not uncommon that you write in the project chat/channel “hey guys, do you mind if I reserve 10xxx codes for job candidate specific errors?”. On the other hand, using error codes that follow a sort of namespace leave the door opened to countless different codes in the same context.
Secondly, reading the message email.format.invalid gives you a better indication of what the issue may be, rather than a numeric code 17456.
Of course you still have to write proper documentation to describe the error in a greater depth, but I find this solution very elegant and it scales very well, much better than numeric codes.

Data: Data can be anything, really, from null to an arbitrarily complex object. It varies for different endpoints and its exact structure and the meaning of each inner field MUST be properly and precisely documented in the api documentation.

As an example, here is how an error response may look like?

POST /customers?wrongParameter=12
Request:
{
  "firstName": 12,
  "dateOfBirth": "2000-01-01",
  "withOwnCar": true
}
Response (Http status code -> 400):
{
  "errors" : [
    {
      "message": "Field value is not a valid string",
      "code": "string.invalid",
      "data": {
        "field": "firstName",
        "invalidValue": 12
      }
    },
    {
      "message": "Unknown query string parameter",
      "code": "url.query.unknown-parameter",
      "data": {
        "parameter": "wrongParameter"
      }
    },
    {
      "message": "Candidates with own car must be 16+",
      "code": "candidate.with-own-car.too-young",
      "data": null
    },
  ]
}

Everything I wrote above makes no sense without a proper documentation as a companion. Document...document everything.

Last but not least, even if you feel that the concepts I described above are too complex and you feel you prefer to return something much simper, please at least be consistent. Take some time to make sure that ALL the errors returned have the same structure and follow the same naming conventions. It may seem obvious, but I assure you that it is not.