API-first software development for modern organizations

Photo by Iker Urteaga on Unsplash

As more organizations move to the cloud, they implement processes to deal with new microservices architectures, containerization, and continuous delivery. Whether they’re adopting cloud services or transitioning to a cloud infrastructure, an API-first approach can help you manage the complexity of working in the cloud.

The traditional code-first approach to app development sometimes results in delays, rework, or a disjointed frankenstein-esque experience for the developer, especially in this cloud-driven landscape.

An API-first approach assumes the design and development of an application programming interface (API) comes before the implementation. Your team begins by creating an interface for their application. After the API has been developed, the team will rely on this interface to build the rest of the application.

By introducing new features as an independent service accessed by API, the rest of the app can be stitched together, along with any other future apps.

Design and build the API first, before building out the rest of the app.

In a world where speed-to-market holds a premium, why would you spend extra time to focus on the API first?

When the code comes first

With a code-first approach, you might start with your integrated development environment (IDE) to type out a few lines of code. Somewhere in the back of your mind, you know that eventually an interface will be created to deliver this service. However, your main focus is on what you call “core functionality” not the “interface”.

When it comes time to implement the interface (the API), the core service is mostly done so it drives the implementation of the API. In some cases, the API might need to be shoehorned into place to accommodate the behavior of the core service. For example, you can’t retrieve information the way you want to due to the way access is granted or the way a database is structured.

To the user who will ultimately use this API, it can feel tacked on, like an afterthought to the core service.

Besides a disjointed developer experience, this approach leaves your team vulnerable to bottlenecks.

Once a version of the API is completed, developers will commit the code to a shared repository and release it to a server environment. Only then can your dependent teams begin. At this point, there’s a host of other tools that you might use to consume the API, to develop test cases, write documentation, and provide feedback.

Testing and documentation wait for development before getting started.

This is a synchronous flow that’s frequently encountered in the traditional waterfall software development process. Delays at any point in time will hold up dependent teams and push back the overall delivery of the product.

Feedback is gathered later in the development process, so changes are more costly since valuable time and resources have already been invested into this earlier version of the product.

When the API comes first

With an API-first approach, instead of starting with code, you could start with design, planning, mocks, and tests.

As you might already suspect, this process is aligned with the popular agile software development principle of iterating rapidly. This allows the team and any other stakeholders to weigh in on the proposed direction and functionality, early and often.

Gathering feedback at this early stage allows you to make changes more easily before a lot of time and effort is sunk into the project. Using mocks or an API development environment makes the current state of the project more accessible to both technical and non-technical team members.

By also separating the design of the API from its implementation, the architect is constrained only by the data model and business logic. Your API can now evolve unfettered by any existing user interface or legacy engineering frameworks.

Once the direction has been solidified, the design then serves as the contract that all teams can begin working towards in parallel, and only then does coding officially begin.

Testing, documentation, and development can begin independently of other teams’ progress.

To drill down a little bit deeper on the API-first approach, let’s talk about API-first development and API-first design.

API-first development

This concept refers to developing the actual API first and foremost. When you’re developing new functionality, the functionality should first be exposed as an API in your organization. The developers responsible for the rest of the application will be the first consumers of this API. This ensures the quality, predictability, and stability to withstand web clients, mobile users, and other developer consumers. Other projects requiring this functionality can now independently consume the functionality via this API.

API-first design

This approach takes it a step further and requires planning the intended API’s functionality before building the API itself. What functionality will the API have? What data will it expose? What will the developer experience be like? How will it scale? How will we add new functionality in the future?

When people talk about API-first, sometimes they’re only referring to API-first development and sometimes they’re including API-first design as well. For the remainder of this article, API first will encompass both API-first development and API-first design.

Word of Caution: focusing on the design first does not mean ruminating endlessly on all the what-if scenarios. A good design should have the flexibility to adapt to changing circumstances and accommodate new insights. As some teams can attest, mandating that specifications drive development can backfire.

Most developers still prefer documenting existing code as specifications, rather than writing the specification first. An interface is intended to be an abstraction for interacting with a system, while the implementation is the way the system does its work.

Hyrum Wright observed that it’s problematic to fully separate an interface from its implementation, especially in large scale systems.

Hyrum’s Law differentiates between an explicitly documented interface that is planned for and an implicit interface that will become evident only with usage. Consumers will come to rely on behavior observed from both the explicit and implicit interface, at which point any changes to the implementation will violate their expectations.

With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.

- Hyrum’s Law

Relevant XKCD

Why API first?

Clearly, pausing to flesh out the API delays valuable building time. Is it worth it? While not all organizations have the liberty to fully plan out their work, there are several benefits for choosing this approach that potentially outweigh delaying your time-to-market.

• Earlier validation: getting early feedback on the design allows the team to pivot or adapt to any new inputs while the cost of change is still relatively low. This reduces overall cost over the lifetime of the project.
• Clear abstraction layer: showing only the necessary details to the intended user hides internal complexity. This allows others to ramp up more quickly when implementing the new service.
• Decoupling dependencies: by adhering to a contract between internal services, dependencies are decoupled so that work can progress independent of other teams’ work. Working in parallel minimizes the project’s overall development time.
• Faster growth: building out the API design at an early stage takes future functionality into account, laying the groundwork for expansion, and accommodates the ability to quickly scale to other apps, devices, and platforms.
• Freedom from constraints: focusing first on the API instead of the code and implementation frees the design from legacy constraints.

What does API first look like in modern organizations?

Some organizations launched as API-only businesses, like Twilio or Algolia, who are both known for offering their services most importantly as an API. Twilio virtualized traditional telecommunications infrastructure allowing developers to replicate any communications experience using APIs. The business recently announced Twilio Build, a partner program designed with an API-first approach.

Twilio’s unique API-first cloud platform is tailor-made to support an ecosystem of partners that are differentiated by the innovations they deliver for their customers.

- Ron Huddleston, Twilio

As some organizations adopt agile software development practices, an API-first approach allows other companies to optimize their total development time. For example, Etsy switched to an API-first approach after facing the common challenge of implementing their logic twice.

All of the code that was built for the website then had to be rebuilt in our API to be used by our iOS and Android apps.

- Stephanie Schirmer, Etsy

Besides increasing extensibility for new devices and features, Etsy’s upgrade optimized their platform’s performance by enabling concurrent API calls previously limited by their PHP code base.

Another example of an API-first company is Netflix. Accounting for a large percent of total internet traffic, Netflix also re-configured their API architecture to allow concurrency. In the process, they distributed their API development so that each client application team was capable of implementing and operating their own endpoints.

A single team should not become a bottleneck nor need to have expertise on every client application to create optimized endpoints. Rapid innovation through fast, decoupled development cycles across a wide variety of device types and distributed ownership and expertise across teams should be enabled.

- Ben Christensen, Netflix

In this context, many teams treat APIs as a separate product or platform. Today, many engineering teams have a completely separate group for building new API functionality and maintaining scalability of their APIs.

Dedicating resources for designing and implementing the APIs demonstrates the value these organizations have for an API-first approach. This is in stark contrast with the code-first mentality which might propose building the application and then retrofitting an API as an afterthought.

Ok, it’s time to try it out.

I love pushups 💪 — who doesn’t? Let’s build a web app that reminds us to get some exercise. I can picture it in my mind, kind of.

Gif by CLTerry

At this point, wecould probably just pull up our integrated development environment (IDE) and plunge right in and get started coding.

Instead, let’s walk through an API-first approach, and learn about one way to build a new product or feature in Postman.

1. Design the new feature
2. Get feedback about the new feature
3. Build the feature
4. Deploy the changes

#1 Design the API

Let’s open the Postman app, and log in.

Create a Postman collection. This will be used to group our requests, and will come in handy when we start testing our API. Let’s name this collection after our project called Pushup.

Create a Postman environment also called Pushup to store variables so that we can update a value in a single place and propagate the changes easily. Store our base url https://api.pushup.com/v1 under a key called base_url. When we’re ready for development, we can swap it out to https://localhost:3000/v1, for example. In addition to storing useful values for configuration, this would be a good spot to keep any sensitive and confidential information like API keys and tokens.

Store config values or confidential information in a Postman environment.

Let’s add a Postman folder to this collection. Similar to a collection, we can further group our requests in a meaningful way. You can nest these folders, and also dedicate variables and scripts to use within the folder scope. Let’s call our first folder Pushups after an endpoint we plan to make /pushups and follow a predictable REST architectural style.

1. GET request to retrieve a list of all types of pushups (/pushups)
2. GET request to retrieve an existing pushup (/pushups/:pushupId)
3. POST request to create a new type of pushup (/pushups)
4. PUT request to update an existing pushup (/pushups/:pushupId)
5. DELETE request to delete an existing pushup (/pushups/:pushupId)

Group requests in a collection and sub-folders.

Add a second Postman folder called “Get Started”. For developers new to this API, this folder can include a few requests walking through an example workflow. This is helpful for other team members to visualize what is happening, and also give you a head start on testing this API. Let’s duplicate some of our previous requests and drag them over to our new folder.

1. POST request to create a new type of pushup (/pushups)
2. PUT request to update an existing pushup (/pushups/:pushupId)

Create a separate folder to help others visualize an example workflow.

For every request, add an example illustrating what you’d like the response to look like. For example, we can add headers, HTTP status code, and a response body. These examples will come in handy later for mocking and documentation, before we have a working endpoint.

Add examples to demonstrate a response.

#2 Get feedback on the API design

The design is just about perfect! We’re actually ready to start developing the API, but it can’t hurt to ask our teammates what they think. Changing the API now will be much less painful than waiting until later once we’ve coded the whole dang thing.

Collaborate in workspaces: Go ahead and share the Pushup collection from your personal workspace to a team workspace so they can look it over. If anybody wants to make any changes, you can give them edit permissions.

Allow others read and write permissions for collaboration.

Activity feed: an activity feed displays updates to the collection, so that you can keep track of who is making changes, and what kind of changes.

Revert changes: if anyone makes any changes that we don’t like, restore the collection and roll it back to the point right after their change was made.

Review the collection’s activities and restore the collection to a previous point in time.

Mock the API: your boss loves pushups too, but he’s not part of your Postman team. We can mock up the responses using the examples that we previously saved for each request.

Mock the API to visualize intended behavior, all before you have a working endpoint.

Now your boss can visualize the intended behavior of these endpoints by hitting the mock endpoints, before the real endpoints are up and running.

Send a request to the mock endpoint along with the unique path in your saved example.

Turns out our teammates had some good feedback. Let’s implement some of their suggestions.

Update the /pushups endpoint to be inclusive of other exercises. Pushups are great, but what if we want to expand the functionality of the app to include burpees, arm circles, and anything else? Let’s find and replace* the text by replacing “pushup” with “exercise”, and similar instances of the word. In fact, let’s rename our collection as “Workout”.

*Note: Find and Replace is available in Postman’s Canary v6.2.2.

Find and replace text related to “pushups” to be more inclusive of other exercises.

Add a third Postman folder called “Account” for account-related requests. It would be nice to let other people use this app too, but we don’t want to mix up our workout schedules.

1. POST generate an access token (/oauth2/token).
2. GET request to retrieve the exercise history of a user (/account/:userId)

Defining what our POST request to create an access token could look like.

Now you can initialize and save the resulting access token as an environment variable so it can be used in subsequent requests.

In fact, add an Authorization header with our access token to all of our other Pushup API requests. Don’t forget to duplicate this “Create an account” request, and add it to the beginning of our “Get Started” workflow folder.

Include the access token as an Authorization header with all other API requests.

Time for another round of feedback. Our collection updates are made in real time, so our teammates can once again review the latest version of the collection in our team workspace. The examples are also updated in real time, so your boss can view the latest version of the mock responses too.

We can continue repeating this loop until everyone feels good about what we’re building. This allows us to iterate quickly. It’s better to update the design and the mock at this stage before we invest our time and resources into development.

Alright, we’re ready to start building!

#3 Build the API

I think the collection helped people visualize the app a little bit better. It seems like there’s a few more teammates who want to help out now. Fortunately, your boss has agreed to let them work on this for one week.

How can we do this in a week?!?

With parallel development.

By gaining alignment on the plan up front, we have established a public contract of sorts to avoid any major pivots and integration failures down the line. We’ve also decoupled a number of dependencies, so that no one is waiting on anyone else.

We have everything we need to get started.

• Document the API: your technical writer has everything they need to begin describing each endpoint and laying the groundwork for the API documentation. This API is a private one, and we’ll use it internally within our organization, but thorough documentation will be helpful for anyone working on the project and especially new team members. Furthermore, this API documentation will continue to socialize the purpose and behavior of the planned service.
• Write tests: Since we saved examples, your QA engineer doesn’t have to wait for development to get going. They can write tests and assertions for each endpoint, run each test against the mock responses, or create new examples to mock any external dependencies that are not yet available.
• Develop the backend: your back-end engineer will be helping out with the API and database development. Every request in Postman represents a server-side route that must be developed. The mock responses will also provide insight into how to set up the database and structure the database queries.
• Develop the user interface: you can handle the front-end development this time. Both back-end and front-end development will be referring to the mocks to develop their respective code. You know how and where to send client requests, what kind of server response to expect, and can make requests to the mock server until a time the real endpoints become available. As a shortcut, remember that you can use Postman to generate a code snippet for every request you plan to make to the server.

Generate code snippets

Once you’ve finalized and saved your requests in Postman, you can generate a code snippet in your preferred language or framework to add to your own application. Let’s walk through how to generate code snippets in Postman.

Identify the request

In your app, imagine there’s a button called “See all exercises”. If a user pushes that button, we’d like the client to make an authenticated GET request to the /exercises endpoint.

In our Workout collection (previously called Pushup), expand the Exercises folder, and click on the GET request to list all the exercises to load it up in the request builder.

Select an environment if you’re relying on any environment variables.

When you have the request working as you’d like it, click the Code link on the right side to open the GENERATE CODE SNIPPETS modal.

Use the Code link to generate a code snippet of the current request.

Select the programming language and framework

From the dropdown, select the language and framework that you’re using for your own app. Our team is using Node.js, specifically the Request framework, to develop our web app. You can update the snippet* in this editor.

*Note: the code snippet may include a Postman-Token header. It’s a holdover from the deprecated Postman Chrome app, and can be deleted in your own app.

Select your preferred programming language and framework to customize your code snippet.

When you’re ready, copy and paste this code directly into your IDE where you’re developing your own app.

If you’re using a Postman environment variable to store your secrets, Postman will use string substitution while generating the code snippet. In your own app, you will likely have a way to pull in environment variables so that you’re not hardcoding any secrets or leaking any sensitive information.

#4 Deploy the API

Finally, everyone is ready to deploy our app to a staging server where we will test the app before letting it go to production.

• Share the API documentation: your technical writer has finished documenting all the endpoints and team members can view the private documentation in a web browser as an easy reference.
• Run the tests: your QA engineer has swapped out the mock endpoints for the real ones already deployed to the staging server by the backend engineer. They added test cases in Postman, which we will run for any new build and must pass prior to any future deployments. We’re using Postman as the single source of truth for our API tests, API documentation, and how our APIs are being consumed.

For the API documentation and API tests in Postman, team members with write permissions have made their updates. Postman automatically merge d their changes into the shared collection so that everyone can work off the latest version. If we decide to undo any changes, we can restore the collection to a previous point in time.

• Deploy server-side code: your backend engineer has committed the code updates made in their IDE to Git and pushed them to our shared repository on GitHub. We’re using GitHub as the single source of truth for our code base, including the implementation for our APIs.
• Deploy client-side code: commit the code updates made in your IDE to Git, and pushed the client-side code to GitHub.

As part of our branching workflow, the server-side and client-side code must undergo a peer review before it will be merged into our main branch. Once approved, the code is integrated and released to the staging server environment for manual and automated testing.

For this particular project, we’ll use the Postman collection runner to execute the test suite written by QA. If any tests fail, we debug the issues until the code is running as intended. Only after all the tests have passed, will we release the code to production.

A closer look at API-first development where work progresses in parallel, avoiding bottlenecks.

In the future if we decide to set up a Continuous Integration and Continuous Delivery (CI/CD) process to automate our build, we’ll use Newman to handle test automation from our CI environment.

Rinse and repeat

Huzzah! It’s been 3 glorious weeks since we released the app to production, and we are pumped, figuratively and literally 🏋️

There’s been such a fitness buzz in the office, that we’re ready to expand the functionality of the app.

Since we followed an API-first approach, we’re set up favorably to handle new feature requests. The team can use the Workout API to build out additional application functionality or expand to other platforms. Alternatively, a new team can jump in to help out and get up to speed quickly by referencing our API documentation.

Let’s continue this API-first approach for our next feature. Adding another feature will follow the same process as when we first created the app.

1. Design the new feature
2. Get feedback about the new feature
3. Build the feature
4. Deploy the changes

Repeat as many times as necessary.

Want to check it out in the Postman app? Click the orange “Run in Postman” button below to download the Postman collection and environment containing these sample requests, or check out the documentation.

click this button to download the collection and environment

What is the single source of truth for your APIs?

We agreed earlier that our shared repository, in this example GitHub, is the single source of truth for our code base, including the implementation for our APIs. This code base is deployed on your own servers, a cloud provider, or on an API gateway if you’re managing an extensive system of services.

Even though the code lives in your source control system, teams will sometimes say that Postman serves as the single source of truth for their APIs.

With Postman Pro, we’ve created a single source of truth for all API endpoints that’s shared across the organization.

- David Esposito, BetterCloud

Some teams use Postman to write their API tests, write their API documentation, and then run their API tests before any build is permitted to release. Postman is the single source of truth for their APIs.

For a lot of teams, Postman is used as a reference for the latest endpoints, use cases, and examples of how their APIs should be functioning.

Although any updates in Postman still require one more step to update the code base, via code generation or other means, the data maintained in Postman serves as the single source of truth for their APIs.

This source of truth is valuable for all API stakeholders, including Product Managers, Sales, Customer Support, DevOps, or anybody who will interact with or consume the API.

Closing thoughts on API-first software development

We’ve reviewed a number of reasons why organizations choose an API-first approach for software development. Some organizations are under the gun to push something into the market, and doing it quickly outweighs any headaches they might have to deal with down the line.

For other teams, let’s make APIs a first class citizen. It’s conceivable that one day, if not today, components will be talking to each other and services will be relied on internally and externally. The world of code is only increasing in complexity.

Consider your API, one that is resilient, versatile, and future-friendly. Designing and building the API first will save time, resources, and headaches over the lifetime of your project.