Design Your APIs for Humans, Not Just Other Systems

When people think about how others will experience their software, they immediately think about page mockups, application flow diagrams, and endless debates over where exactly the submit button should be placed. But the design of the software’s system-to-system interaction points often plays an equally strong role in the way the software’s experienced, albeit for a different userbase.

The rise of microservices & serverless architectures has placed increased importance on system-to-system communication. This is usually done through APIs, using a communication architecture called REST (though you’ll still find pockets of SOAP users in the wild). A common development mistake is underthinking the design of APIs that you expose to other systems, leading to a slapdash experience that can be confusing at best, error-inducing at worst, and can drive people away from interacting with your software.

Why the Developer Experience Matters

Most developers have a lot on their plate. You can see this in the number of defects that wreck havoc on production systems worldwide. Even with rapidly improving IDEs, automated testing tools, frameworks, and abstracted languages there are still thousands of defects being deployed to production on a daily basis. It’s not because developers are lazy, or because we’re a lot less smart than we think (though for some people that may still be true), but because creating complex software is fundamentally really hard.

Because creating software is really hard, most developers really appreciate anything that makes their jobs less challenging. Well-designed APIs fit into this category. A well-designed API is simple, intuitive, and clear. It’s easy to understand what exactly will happen if you call x endpoint with y data, with no unexpected side effects or gotchas. This means fewer mistakes are made by the developers creating code to interact with your API, and therefore fewer defects elevated to production.

But why should you care about defects in systems that interact with yours? That sounds like a “them” problem and not a “you” problem, right? In addition to not being very empathetic, this thinking runs into issues once your butt gets dragged into late-night production issues concerning interactions with your system. Furthermore, any problematic calls to your API could cause a whole range of issues in your own system, anything from system slowness & rejected requests to data issues if API calls are moving said data to an incorrect state. Even before all that happens, developer(s) from other system will likely have to communicate frequently with you to understand more about your unintuitive APIs’ function & effects, taking up your own valuable time & energy that could have been saved if your APIs were more clear.

Some Systems Live-or-Die Based on Who’s Calling Them

For some systems, having easily consumable APIs can make-or-break their success in the marketplace.

Barring overrides from management or “hard nos” from security & compliance departments, most developers have at least some agency over which tools & systems they leverage to build their software. We live in a beautiful era of software development where the DRY (Don’t Repeat Yourself) principle can be executed on a global scale. Where developers can solve common problems by leveraging existing solutions sourced from practically anywhere, allowing them to focus on domain-specific value delivery instead of wasting time building the same solution repeatedly (for instance, user authentication).

When developers are choosing solutions to help them solve said common problems, often times API design plays a major role in the decision making process. For most developers being able to call simple, intuitive, clear APIs to solve a common problem is ideal, since they’re now able to focus more on solving the important domain-specific problems ahead of them.

In the software industry where externally-facing services are only becoming more critical (pretty much everything’s “as a service” these days), developers choosing a competitor’s offering over your own due to poor API design can be a real blow to the bottom line.

Applying the Human Centered Design Process to API Design

Ok, let’s say you’re convinced that how you design your system’s APIs often plays a key role in its across-the-board success. But where do you start the design process? Before diving into HTTP verbs & response codes I’d recommend thinking about the people who are most likely to interact with your software via API calls, and applying the following steps from Human Centered Design (HCD) to start designing clear, intuitive, & useful APIs tailored for them.

1. Empathize — First off, you should truly understand the sorts of systems that are likely to interact with yours, and the subsequent people creating & maintaining said systems. This may feel a bit too “soft” for some, but remember that developers are people too, with a lot to think about and usually not enough time to think through it all. So try creating “user profiles” of systems that will (or are likely to) interact with your APIs, and if possible reach out to the people who build said systems. Describe the purpose of your system (and certain aspects of the data model & functionality if necessary), then work with them to understand how your system could be leveraged by their system to outsource non-novel functionality, continue a business flow or user journey, or simplify development tasks for them. Even better, if they’re already interacting with a similar system or have written code themselves to fulfill a purpose that your system can provide, see what their biggest pain points are & whether your system could solve them without sacrificing aspects of their current state that they like.
2. Define— Next step would be to define the problems that you’ll look to solve through your system’s APIs. These can be expressed through high-level user stories that feed off of the opportunity areas identified during the Empathize stage, alongside other potential CRUD (Create, Read, Update, Delete) needs concerning resources in your system. At this point you’re not looking to define specific APIs, but instead describe at a high level the interactions that consumers should be able to have with your system. For example: Instead of saying “consumers should have access to a Get User API and a Get Orders API”, I recommend defining the problem statement as “consumers need to be able to access users’ information, including their order history”. This leaves the door open to later design discussions, like whether the API(s) that fit this need should aggregate certain backend information vs. require separate calls, and whether they should allow arrays to be passed in (fetching orders for multiple users vs. one user at a time). An interesting question to consider is how many potential needs you should account for, on top of identified needs. There are many instances in API design where the team thinks: “Someone at some point may need to perform x operation on y resource(s), but no one’s asked for it yet”. Opinions on this topic range from allowing all CRUD operations on all resources by default (so long as they don’t pose security or data integrity concerns) to only building APIs that consumers (existing or potential) explicitly ask for. My take is that it’s a judgement call, dependent upon the odds of the potential need becoming actualized vs. the time it would probably take to deliver, expressed in both cycle time (time between the consumer asking for it & it being delivered?) & development time, alongside any security & data integrity concerns.
3. Ideate — For each of the problems identified in the Define stage, begin identifying a variety of potential solutions via APIs, and how each of those solutions would shape the system’s broader API landscape. The ideal state is usually a 1–1 mapping between resources and APIs (ex. a User API to handle CRUD operations on User resources, an Order API to do the same for Order resources, etc.). However, during the problem-driven ideation process you may find that a lot of 1–1 mapped solutions would require the consumers to make chained calls to various APIs in order to meet their identified needs. This can feel unintuitive to consumers unfamiliar with your data model, and can lead to performance issues. In that case aggregation (fetching or operating on multiple resources from one call) may become a commonly-used API design pattern in your system to reduce complexity & calling-load for consumers, even if it may yield increased brittleness, system load, and unexpected side-effects. By first identifying various solutions to each problem, then looking at the whole palette of options, your team can make more intelligent decisions about the system’s current & future API design philosophy, accounting for it when selecting which designs to use.
4. Prototype & Test — Prototyping can have two steps. First would be to define & run your team’s API designs past the consumers identified in step one to get feedback on their clarity, intuitiveness, and usefulness. Then once designs are tweaked (taking said feedback into account), it may make sense for your team to mock out these API definitions, allowing soon-to-consume systems to call said mocks and receive example data back while building their own integrations to your system. This goes beyond eyeballing & rubber-stamping, to allowing developers from other systems to experience your APIs and provide more accurate feedback on their clarity, intuitiveness, & usefulness. Once your team & all participating consumers feel good (or at least good enough) about the designs, feel free to start putting some actual logic behind those APIs.
5. Iterate — Iteration may be the best thing to come out of agile philosophy in my opinion, and it’s because humans are notoriously bad at predicting the future. Many bad products & designs had at least one person behind the scenes who thought they would work at time-of-creation, but turned out to be wrong. And even if they nailed that design initially, operating environments shift, and what seemed awesome at release can soon become become clunky, outdated, & effectively useless. Systems change over time as functionality is iteratively built & updated. As a result API designs that worked great at-creation may no longer enable consumers’ needs, or may no longer reflect the system’s underlying design. That’s why it’s important to reconsider the system’s existing APIs regularly, even running through a miniature version of the HCD process described above to ensure these APIs remain clear, intuitive, & useful.

Other API Design Considerations

• Arrays on Input? — For any API that’s used to perform CRUD operations on resources, the odds are strong that there’s a use case for consumers to provide an array of stuff to create, read, update, or delete all at once, instead of passing it one-at-a-time. By allowing an array of ids, new resource definitions, etc. in your API’s inputs, you can reduce the chattiness of intersystem communication, generally improving performance. On the flip side accepting arrays mean a lot more for you to think about. What should happen if the requested operation fails on one resource, but works on the rest? How many resources or ids should other systems be able to send you at one time? How much complexity would handling multiple resources or ids add to your system’s underlying logic? All pros & cons to weigh when deciding whether to handle parameters in bulk.
• Have a Method to Using HTTP Methods — HTTP methods aren’t hard to learn, and there’s a sea of websites that describe what each one’s meant for, yet far too many APIs expect a method that differs from the actual operation they perform. Furthermore, some teams will define an entirely new API to meet a need, when they could just add a new method to an existing endpoint. The fact that HTTP methods are standardized makes them a powerful tool, since you don’t need to define and teach consumers about new verbs & their meanings for them to interact with your APIs. By specifying that an endpoint accepts certain standard methods (ex. the User API supports GET, POST, and PUT) it’s made immediately clear to new consumers what they can (and can’t) do.
• Kowalski, Status Report! — On the other end of API calls, consider the range of results that may come from your API being called, and the status code said API should return for each. Everything from expected results (most likely 200 codes) to client-side failures like missing or incorrect parameters (400 codes) to server side failures (500 codes). Don’t be afraid to return codes that don’t end in two zeros, if they better represent the actual state resulting from the API call.
• Paint a Picture with Your Response Messages — In addition to accurate, meaningful status codes, look to provide more context around what happened (or is happening) through the response message given back to the caller. Especially in 400-style scenarios where the client called the API incorrectly. Why exactly did their call fail? What parameter(s) is/are causing the issue? Giving meaningful feedback to the consumer will save significant headaches, both on their side and yours (since you’ll be the first person they call at 2am when all calls to your service are failing).