API Design Using Feedback Loops

We often think of API definition as beginning with the description file. The challenge with this approach is that API producers often get into the weeds on resource names and methods before actually determining what the API capabilities need to be. This can cause a lot of churn when designing the exact interface because the details and high-level concepts are being discussed simultaneously.

Many organizations opt for the fast, easy win by releasing an “MVP” into the market with the intent to build on those concepts from there. While that may work for traditional products (I would argue this is questionable but has become an accepted approach), it is far more challenging with APIs because of their role in applications that depend on them. If you are going to provide an API that is both desirable and stable, you need to ensure that your initial API release is functional, reliable, and usable. A death knell for APIs is releasing constant breaking changes or updates that require the app developer to rework their app.

Minimum Viable Product: Build a slice across, instead of one layer at a time. Diagram courtesy of Jussi Pasanen, with acknowledgements to Aarron Walter, Ben Tollady, Ben Rowe, Lexi Thorn and Senthil Kugalur.

A more efficient methodology is to use some of the same principles from the LEAN methodology to encourage constant (and early) learning. This allows you to refine the required vs desired capabilities before diving into the details of how to implement those capabilities. With some adjustments to fit the API space, you can apply similar principles to achieve the same end goal — understanding the value to the customer and constantly readjusting based on feedback loops.

Benefits of Feedback Loops

Taking the time to test your product ideas in the market before committing to code or a roadmap can be a big investment of time and money. But making that investment pays off in many ways. Let’s look at some of the benefits of this approach:

• Planning the capabilities first allows you to think strategically about your release plans and your potential partners. When you commit yourself first to the inner workings of the API, you become locked to the technology rather than the strategy.
• By gathering feedback on your roadmap and capabilities, you can be nimbler about changing both of those because you haven’t already committed to an API design and code. You can also have more confidence in the longevity of your API interface because you have done a lot of the forward-thinking already.
• Your potential API consumers are more invested in your releases because they have provided direction and strategic thought to help form your offering. You are building what they asked for and they know it.

Risks of Establishing Feedback Loops

It wouldn’t be honest if we didn’t admit that there are risks associated with putting your API concept out there before your product is ready. Depending on how risk averse your organization is, these are risks that should be seriously considered.

• You have to embrace a certain level of transparency. With you are iterating on the concept and garnering feedback, you must expose your ideas to the public. There are ways to mitigate this risk that we will discuss further on.
• If you ask for feedback on your API concepts, you need to be prepared to react to that feedback. Your product managers need a nimble mindset and your organization needs to react to changes in plans during this phase.
• You may have to sacrifice speed for quality. We’ll talk about various ways to gather feedback and react to it, but they all come with a time commitment and the risk of taking longer to release than you may be comfortable with. But you can also rest assured that you will be bringing an API to market that developers really want.

Techniques for Establishing Feedback Loops

Depending on the complexity of your API and the urgency to release it, you can use any number of techniques to get feedback on your API. Ideally, given the time and resources, you would incorporate a combination of techniques, each of which are designed to clarify and solidify your API concept more deeply.

Remember that a feedback loop is exactly that — it is intended to be part of an iterative design cycle. Let’s take a look at some feedback loops that can help you design the right API with the right value for your consumers.

Developer Focus Groups

These conversations are invaluable when you’re ‘kicking the tires’ on an idea. You can host these at any time during the ideation cycle — your API concept could be just the germ of an idea, with no real detail about the methods or limitations. In fact, it’s best not to showcase any known limitations so you can have a freer conversation. Here are some tips for organizing a developer focus group:

• To keep this conversational, the ideal size for this type of gathering is no more than 30 people.
• Make sure you include a variety of participants — enterprise devs, start-up devs, product managers, all from a variety of verticals.
• Be prepared to show your concepts and encourage honest feedback. Do not convince, do not justify; this is a listening exercise. Explore the feedback with questions, questions and more questions.
• Bring a scribe — you want to focus on the conversation; let someone else take notes.
• Assess and re-evaluate your target market based on the feedback (maybe start-ups are more interested in this capability so they don’t have to build it themselves, for example).
• Keep your ears open for messaging refinement — what phrasing/words helped the participants understand the capability the most? What was their first emotional reaction to your description and positioning? This isn’t just a chance to get feedback on the API you want to build, but also how to position it later.

Publicly Available Prototypes

Ideally you follow the focus group sessions with what you believe will be your first version of the API, but in prototype form. The goal of this prototype is to move from concept to reality by exposing the API interface along with some mock data in a playground setting. Let developers make calls to the API, encourage feedback on the interface design and payload construction. Some things to consider when you make API prototypes available:

• Your API design should still be as clean as possible. While you don’t want to invest so much time in the interface that you are loathe to change it, you also want to ensure that you haven’t exposed internal paths or resource names and that you have provided enough security to engender confidence in your ability to design an API.
• Provide some robust mock data. Developers want to see how your API performs under error conditions as well as under normal conditions. They want to see a full payload that gives them a real sense of how they would work with the API in a real development environment.
• If you have collaboration tools at your disposal, use them. There are several tools on the market that allow people to comment on your API description and engage in forum discussions. Gathering that information from your audience via a tool means that you can keep all the feedback in one area and have it stored permanently. If you can iterate in the tool itself by providing an updated version of the API description for developers to experiment with, even better.
• Publish your draft documentation along with the prototype. API documentation is not a separate deliverable — it *is* part of the part the API itself. During the prototyping phase, you can get real-time reviews of the documentation you provide so you can iterate on that as well. If you can provide the documentation on GitHub, you can even encourage your consumers to submit PRs with doc corrections and enhancements.
• Watch it in action. Now is the time to host some informal hackfests and even more focus groups. Each time you iterate over the prototype to incorporate feedback, you can bring a group together to validate the direction.

Beta/Invite-Only

Okay, so you’ve gotten feedback on the concept from focus groups before writing a single line of code. And you built prototypes that implement virtual APIs and first-draft documentation, and you iterated over those prototypes based on consumer feedback.

Now it’s time to hit production. This is where you really need to evaluate your risk tolerance. If your API does not involve secure information or mission-critical capabilities, you can deploy to production and open it for general use.

However, if you are in a more risk averse environment, you can continue your slow roll to General Availability by providing the API to a limited audience. This gives you yet another opportunity to gather feedback and make improvements before making it more widely available. Here are some general guidelines for the Invite-Only phase that can help you ensure your API is solid and meets market needs:

• Be selective about the API consumers you give access to. By now, you’ve tested the concept, the interface design and the market positioning. You know your target market and the roadmap. Identify up front the size of the developer community you want to provide access to and keep the guardrails up to make sure it stays within that range.
• Decide how you want to build that community — do you want to reach out to specific developers who were engaged in the earlier iterations? Or do you want to have a publicly available sign-up form that you use to select your initial community?
• Be prepared to make changes based on this first real-world trial. You may feel that you’ve already done enough iterations. But bear in mind that this is the first-time developers are trying to build meaningful production-ready apps using your API. They will find things they didn’t find before — Oauth issues, documentation inaccuracies, rate limitations, performance problems, etc. Be open and be available to take their feedback into consideration.
• Set a pre-defined time range for this phase. Being in beta is addictive. It’s the easy out for many API providers. But you’ve done your homework already — you’ve done focus groups, prototypes and now a beta. Make sure you have a target date for General Availability and have engaged sufficiently with your community to be ready to meet that date.

Versioning

Don’t. Just kidding.

In all seriousness, one of the reasons you went through the iterative work up front was to avoid excessive versioning. There are many philosophies about this and you can decide your own versioning strategy. But there are some important things to consider as you plan for this. Developers have a business dependency on your API. When you version your API, you run the risk of forcing them to version their app, which they may or may not have time and resources to do. Make it easier on them by following some simple principles:

• Communicate, communicate, communicate. Give plenty of warning before you implement a version change and be very clear about the requirements to update their app. Be transparent about your rationale for the new version.
• Make sure you provide a backwards-compatible option for those developers who cannot upgrade right away (or ever).
• Maintain support for previous versions.

API Quality is Important But Not Easy

APIs have become an essential part of the development stack. Having a disciplined product management approach to defining, designing and deploying them is critical. Don’t be afraid to take the time up front to build the right capabilities with the right interface — it will strengthen your offering in the long run.

DISCLOSURE STATEMENT: These opinions are those of the author. Unless noted otherwise in this post, Capital One is not affiliated with, nor is it endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are the ownership of their respective owners. This article is © 2018 Capital One.