User-Centric Web API Design

A Product Manager’s Approach

By Sue-Mun Huang

Originally created by developers, for developers, Application Programming Interfaces (APIs) make software development easier, faster, safer, and more resilient to change. But APIs have since become products in their own right, and lucrative ones at that. With APIs, businesses can not only improve their internal systems, but also quickly expand beyond their own platforms, while maintaining full control of external interactions with their content and data. For example, Facebook’s dominance in the digital ad space (generating a cool $39.9 billion revenue in 2017) is bolstered by its highly competitive marketing partner program, where partners build integrated tech solutions to make buying vast quantities of ad inventory easier and more effective. The best part? They also leverage the insights gathered through their partners’ innovations to improve their own product. Facebook has truly embraced the API economy, and they’re not alone.

APIs have had an enormous impact in a world which is becoming increasingly digital-first — but does it make sense for your business?

In short, an API is the interface of a system that is exposed to other systems, with a strict set of rules defining how these systems can interact with it. It promises a specific outcome based on a specific data input, without either system caring how that data is being used/produced by the receiver/sender. As long as both parties understand and obey the rules, everybody gets what they want. This reduces solution complexity, and eliminates the need to understand the inner workings of every system component.

APIs have had an enormous impact in a world which is becoming increasingly digital-first — but does it make sense for your business?

Building for Web

A Web API is simply an API which does its communication over the internet. Server-side web APIs are the most commonplace: these define the request-response system between a client (e.g., a browser) and a server (the main app itself). The client makes requests to a server by sending messages to pre-specified points of access (known as “endpoints”), which returns the appropriate response. The use of the REST (Representational State Transfer) architecture style is hugely popular for many reasons: namely simplicity, speed, efficiency, and its use of HTTP/HTTPS (what the world wide web uses to communicate).

These days, there are countless APIs serving a variety of use cases, but the most valuable ones put the end-user first. As a product manager, my main focus is on ensuring maximum benefit to the users, even in pure “engineering” projects. This is my approach.

1. Start With Why

Immerse yourself into the current systems and get to know them very well. Ask yourself (and everyone else):

• What is your source of truth, or is that the problem itself?
• What are the current pain points of the existing system?
• Is your business unable to expand or scale? Why? Why not?
• How does this affect the user? Which users?

You might need a Web API if:

• You need to integrate with numerous external systems
• You’re unable to sync user data across devices (poor user experience)
• Apps are using a lot of client-side logic to manipulate data (slow performance, poor battery life)
• You’re unable to provide partners with data safely and securely (limited user reach, unable to build integrated features)
• Apps are accessing the same/similar data from the same underlying system using different methods (inconsistent user experience, hard to roll out new features)
• The same data is being retrieved over and over again (expensive for business)

Is it not obvious that you need a web-based solution? Consider looking into other types of middleware.

While APIs make developers’ lives easier, the end-product is always geared towards end-users — as such, the API must be able to provide the functionality required to solve those users’ problems in the end-product(s).

2. Know Your “Who”

As with any product, take the time to understand who your stakeholders are, what they need and want to do, their pain points, and how they measure success. How do these translate into measurable metrics for the API?

There are usually three or more broad classifications of stakeholders:

• End-User: who will be interacting with the end-product, which is built on top of the API
• End-App Developer: who will be building those end-products, using the API you’re building
• Business: who own the data and the underlying systems

While APIs make developers’ lives easier, the end-product is always geared towards end-users — as such, the API must be able to provide the functionality required to solve those users’ problems in the end-product(s). The stories you write to develop an API will involve each type of user, but the overall features should prioritize the end-user.

3. Create an Elevator Pitch

You should be able to clearly articulate the problem, the potential benefits of the solution, and who the first adopters might be. Can you explain this to someone else in 20 seconds or less? Can you get buy-in from your potential beta partners to make the commitment on their end? Finding your general purpose and being able to clearly and confidently explain what you’re doing will go a long way. This is especially important for backend projects, as they are often poorly understood and their value harder to measure (compared to, for instance, a shiny new consumer-facing mobile app feature).

Step into the shoes of each user type, and try to understand what that user is trying to achieve and why.

4. Understand the Jobs To Be Done

APIs need to be able to support the workflows that a user will take in the end-product. To break down a general purpose (e.g., “make data access standardized across platforms to provide a consistent user experience”) into specific features, I love using the Jobs To Be Done (JTBD) thinking framework.

Step into the shoes of each user type, and try to understand what that user is trying to achieve and why. Write down any of the steps that a user has to take to achieve their desired outcomes. Think about the main Jobs To Be Done, and the related jobs to be done.

For example:

Main job:

• An end-user wants to listen to their favorite podcast on their drive to work

Related jobs:

• Discover new or interesting podcasts
• Keep track of which podcasts they’ve listened to
• Share their favorite podcasts
• etc.

I find this exercise helpful in keeping the “solve the end-user problem” purpose top of mind. It also helps to identify gaps in current systems and can highlight the need to build or integrate with new systems to supplement existing data.

5. Understand the Real-World Environment

APIs never exist in isolation. While APIs should be designed with the end-user as the main priority, to support an API such that it works not just today, but for the future, requires careful consideration of many other things:

• Integrations/Dependencies (third party systems, other internal systems)
• Data Abstraction (e.g., ORM layer)
• Performance Management (latency, uptime, request volume etc.)
• Security (encryption, sessions, secrets, firewall, auth etc.)
• Legal (data protection act, privacy policies etc.)
• Logging (short term, long term, enabling support)
• Monitoring (IT, dev ops)
• Monetization Strategies (access tiers, revenue sharing)

Do your homework early, and often. It’s especially important to understand if the API is intended for internal or external use: consider using authentication and authorization methods to prevent unwanted traffic.

6. Distill your Feature Set

The API should serve a primary purpose: addressing the “why”. The jobs to be done defines your “what”, and your feature set should define the “how”.

I like to define feature sets for each user type and then look for areas of overlap. The functional map should directly correlate to solutions which address each Job To Be Done for the end-user, for example:

End-User Features = API Functional Map

• User Authentication
• Content Management
• Email Service
• Device Management

User Preferences:

• Save my favorite podcasts `POST /v1/favorites/podcasts`
• View my favorite podcasts `GET /v1/favorites/podcasts`
• Update my favorite podcasts `PATCH /v1/favorites/podcasts`

As the Product Manager, you ultimately own the endpoint routes and their intended functionalities. Your engineers should help you organize your routes and select the best method for each.

For this example, there will also be non-endpoint features to support the success of the product for all stakeholders:

App Developer

• API Documentation
• Support System

Business / IT / Admin

• Logging Tools
• Monitoring Tools

Know where you want to go, but only build what you need and adjust course as you collect feedback; otherwise, you might find yourself building the wrong thing over time.

7. Define your MVP

Planning ahead is incredibly important. Know where you want to go, but only build what you need and adjust course as you collect feedback; otherwise, you might find yourself building the wrong thing over time.

Focus on what is the one most important JBTD for the most important user? Are there related JBTDs that must be accounted for to achieve this main JBTD in a user-acceptable way? Where are there no workarounds with the current system? Start here.

For example, the main JBTD above was to let the user listen to their favorite podcast on their drive to work. The bare minimum API functionality required to achieve this would be to make that podcast information available via an API call. It would be significantly more user friendly if the user was able to sort their search results, or filter by a specific subject, so they can actually find their podcast — but is this necessary? If you deem that the feature would not provide user value without these supporting functions, then include them in your MVP.

8. Define your Build Strategy

API projects are typically complex and difficult to completely decouple. Work with your engineers to understand if there are any dependencies that cannot be avoided, and plan your build accordingly. For example, in order to add the ability to sort through a list of podcasts, you must first be able to retrieve that list of podcasts — so build that first.

Furthermore, APIs are only useful when they are actually being used by some system — identify who your first adopters will be, and plan the order in which you build things around their needs. Be sure to consider versioning when defining your routes and your release plan.

9. Write Stories

Since the API is the product, then the routes, method, request, and response definitions are the business requirements. Your stories will initially be very high-level and vague, but as you work with your engineers and QA team, these details will take shape over time. Stories should be agreed upon by the whole team.

I’ve personally found the following format to be a good balance of keeping both end-user benefits in mind, while providing enough technical detail for developers:

Summary: As an {end user}, I should be able to {action}, so that I can {benefit}

Acceptance Criteria:

• GIVEN: An {end user} is {doing the action} in the {app}
• WHEN: The app hits the {endpoint route} endpoint with a valid request, containing {required request parameters}
• THEN: The app should receive a status {code}
• AND: In the response, the following information {response body}

Focus on maximizing the usefulness of the response while minimizing the number of requests required.

A well designed API should remove as much of the business logic and data processing as possible from the client-side. This is especially important for mobile devices, which have bandwidth and processing power limitations. Focus on maximizing the usefulness of the response while minimizing the number of requests required.

Where applicable, remember to include performance metrics for things which could affect the user’s overall experience, like response latency or rate limits.

API Specification

A good API follows a clearly defined and consistent standard for requests and responses. Open standards like JSON API are a great place to look for inspiration, but make sure it makes sense for your product. Logic and consistency within an API is more important than adhering to someone else’s standards. Define your own spec as you build, and refer to it constantly in your stories.

Even the smallest things can help your API feel more cohesive and consistent:

• Naming standards (e.g., plurals for objects, snake case for field names etc.)
• Case-insensitivity (e.g., for search terms or routes)
• Data formats (e.g., dates are always in “YYYY-MM-DD” format)

10. Build, Test, and Release in Iterations

This goes without saying. Get feedback early and often: alpha and beta partners are your best friends here. APIs are tricky to test, and it’s impossible to foresee all use cases. Early adopters usually highlight functional gaps or quick wins, which can be easily adapted into the next iteration.

At TribalScale, we use the Extreme Programming (XP) flavour of Agile methodologies, and deliver in weekly release cycles. This keeps the project moving in the right direction (even if that direction changes with user feedback). Each release should be able to deliver a tangible, incremental benefit to the user — if this is not possible in a week due to feature or implementation complexity, then try a slower cadence or revisit the size of your user stories.

While Agile principles state that functional software is more valuable than detailed documentation, a functional API is useless if no one knows how to use it.

11. Documentation

Since your product isn’t an app that someone can download and intuitively click around in, you must have documentation, and it must be intuitive to use. Ideally, any developer should be able to determine how to use different endpoints to build the features they want from your documentation alone. While Agile principles state that functional software is more valuable than detailed documentation, a functional API is useless if no one knows how to use it.

Invest the time and effort into doing this the right way: give detailed examples of requests and responses, explain what and why each endpoint does what it does, and provide a sandbox environment to let developers play with the endpoints. (Swagger is great for this.)

12. Monitoring & Logging

Monitoring and logging are important for APIs for many reasons, such as:

• Scaling server capacity as needed (up and down)
• Identifying opportunities to optimize API functionality
• Identifying opportunities for caching (e.g., Fastly)
• Security monitoring
• Health checks
• Troubleshooting issues

Good error logging also helps enable quick and easy assistance for users who contact support.

Invest the time in building a visual dashboard for monitoring — this goes a really long way in providing meaningful visibility to system health and ensures all stakeholders have access to the right data.

I strongly recommend implementing a robust logging system (think: transaction ids, error ids), that can be used to quickly find recurring errors. Good error logging also helps enable quick and easy assistance for users who contact support.

I’ve found this simple approach to be a great starting point:

• Detailed temporary console output for event diagnostics (e.g., Amazon Cloudwatch)
• Persistent storage of key events for longer-term analysis (e.g., Elasticsearch)
• Visual output to business-facing dashboards to monitor key metrics (e.g., Splunk)

Summary

Designing a good user-centric API requires extensive iterative research and collaboration between numerous teams and individuals. The end-user’s needs should be kept in mind at all times, the API should make developers’ lives as easy as possible, and the business should be able to easily understand why the API exists, and how it is performing as a product.

I’ve learned a lot of lessons along the way, and I have many more to learn. But whatever the industry or technology, I’ve learned that building a user-centric API is just like building any other product: it must serve a purpose by solving problems for its users. If the user has nothing to gain, there is no point. Invest your money elsewhere.

Over the last six years, I’ve worked with SOAP web services at Orion Health, a global provider of software solutions for hospitals and healthcare networks, before shifting to the world of REST at AdParlor, one of Facebook’s original Strategic Preferred Marketing Developers. Today, I’m the TribalScale API Product Lead for a large US media company, who produces and delivers localized audio content to 83 million weekly listeners. Together, we’ve helped them build the APIs and backend systems which power their suite of Web, iOS, Android, Alexa, Sonos, Cortana, Google Home, Chromecast, Roku, and FireTV offerings, as well as a number of partner apps.

Join our fast growing team and connect with us on Twitter, LinkedIn, Instagram& Facebook! Learn more about us on our website.