Jan 1, 2022
Enabling an API-first approach
An API-first approach means that any and all software development starts with a mindset that the software capabilities being built, WILL be consumed by other software systems or mobile applications through APIs. This would require a fundamental shift in mindset of all stakeholders involved in building software systems.
API — quick intro?
API stands for Application Programming Interface. As the name suggests, it is an interface to interact with a software system. In other words, it is a mechanism to invoke a software system without really knowing how it is implemented.
An API specifies what input a software system will take and what output the software system will respond with.
In the above example of a simplified sample travel app, the team that is building the Make Reservation Service will invoke the Process Payment Service via an API. The Process Payment Service behind the scenes may invoke APIs for Visa, MasterCard, PayPal, Venmo, Zelle or any other payment provider. In order to process a payment, the consuming application will only need to know how to invoke the Process Payment API, what input to send and what output to expect in return, and does not have to worry about the underlying implementation.
By abstracting the complexity of implementation, the APIs provide for a simple mechanism to integrate heterogeneous software systems.
API Architectural Style
APIs usually follow one of the following architectural styles.
- REST: Representational State Transfer. Most widely used and designed to work with resources; follows principles such as client-server, stateless, can be cached, will work whether it is communicating directly with a server, or through an intermediary such as a load balancer.
- gRPC: Modern open source high performance Remote Procedure Call (RPC) framework. Typically uses XML or JSON for encoding. Designed to call methods or actions where the URI identifies the server, but contains no information in its parameters.
- SOAP: Simple Object Access Protocol. Uses XML as a format to transfer data. An older standard that is rigid and has strict rules for defining and processing messages.
The advent of GraphQL has extended the usability of APIs even further.
GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.
The above Google Trend illustration points to the growing popularity of RESTful APIs and GraphQL, which are both well suited for modern SaaS applications.
The OpenAPI Specification, formerly known as the Swagger Specification, is a widely accepted standard for defining RESTful interfaces. Whether teams have existing APIs or are building new APIs, standardizing on a specification like OAS would reduce complexity.
Typically a governance body like the Architectural Steering Committee helps standardize on the use of a specification along with defining other aspects of an organizations API standard like naming conventions, use of verbs, versioning (how many will the team actively support), deprecations etc.
Code-first approach
In a traditional code first approach, software engineering teams design and build software systems to meet the requirements and somewhere along the journey, build APIs. In this approach the core services built as part of the software system and the corresponding dependencies like the underlying data model, often drive the implementation of the API. Consuming systems have to wait for the API to be published before invoking the system. Often, teams find that they will have to change the APIs or the underlying implementation to satisfy the needs of the invoking system, potentially making this approach slow and expensive.
API-first approach
In an API-first approach, design & development of API comes before the implementation. Engineering teams, start by creating an interface and then use that to build the rest of the software system. Transforming to an API-first approach requires a fundamental shift in culture. Everyone on the product engineering team has to think about APIs before writing a single line of code.
In this approach the API has a better prospect of representing the true business capability and is not constrained by underlying dependencies like for example, the data model
Depending on the maturity of the organization, there are several people who could be involved in this lifecycle.
- Product Owner
- Architect
- Development teams
- Operations Engineer
- API Admin
Everyone of them has a role to play in the API-first approach. This is where the cultural transformation kicks in.
API Guild
Forming an API Guild is an effective means to promote the API-first culture. This guild should comprise of interested technologists as well as product owners across the organization. They would meet regularly to discuss, deliberate and help advance the API strategy within the organization.
Education
Within an organization, people have varying degree of understanding of APIs. Conducting training sessions for different teams and aligning everyone on the organizations API Strategy, what an API-first approach means, the tooling that will be used and other topics related to APIs is critical.
Tooling
The choice of tools can make it easy for people to embrace the API-first approach. Fortunately, the current marketplace provides for a wide range of choices. It is critical for the organizations technical governing body to select tools and make them available to the product engineering teams. This should cover various aspects of the API Lifecycle including design, test, discovery through a developer portal, API documentation, access via an API gateway or service mesh and monitoring.
Design
This is the first step of the API lifecycle. There are many characteristics to a good API design. While there are several wonderful articles that dive deep into good API design, I will list a few here.
- A good API is simple, self-describing, easy to read and work with, provides valid responses and handles errors gracefully.
- One way to help build good APIs is to design them keeping developer self-service in mind. Essentially every message should be self-descriptive, whether it is a request or a response.
- Adopt a consistent naming convention in URIs. Resource URIs should be based on nouns (the resource) and not verbs (the operations on the resource). Define API operations using HTTP methods (GET, POST, PUT, DELETE, PATCH).
Good
https://myapiexample.com/orders
Bad
https://myapiexample.com/create-order
- APIs should support multiple versions. Versioning is usually done with /v1/, /v2/, etc. added at the start of the API path.
- Every request should have valid response (either success or failure). Handle errors gracefully and return standard error codes (200, 400 series, 500 series)
Product Owners, Architects and Development teams can use tools like Postman, Insomnia, Swagger or Apigee to design APIs.
Testing
Once the API is designed, teams should have the ability to verify several aspects of the API, including, response format, error conditions, how secure the API is and response time. This is a critical step in the API-first approach as engineering teams need not wait for the API to be fully implemented. They can use mock testing framework to test the APIs and start on their development lifecycle. Tools like Postman, Insomnia, Swagger, Apigee or SoapUI among others provide good API testing capabilities.
Continuous Evolution
APIs evolve over time based on several scenarios that probably, were not considered at design time. Consumer driven contract testing helps alleviate this problem.
Contract testing allows for each software system to be tested independently from the other and that the contract is generated by the code itself, meaning the contract is always kept up to date with reality. Contract tests run faster, are easier to maintain, are repeatable and scalable as each software component can be independently tested. Pact and Spring Cloud Contract are frameworks that make the definition of contract tests easy.
Discoverability
As part of the API-first approach, it is critical for Engineering Teams to be able to discover APIs that exist within or outside a firm. This is made possible with a Developer Portal. Engineering teams should be provided an easy means to publish an API to the Developer Portal after due validations though a CICD pipeline. This will serve as the API catalog that teams can browse through to determining if they can reuse, extend or build a new API for their purpose. This also promotes better reuse of software assets, potentially leading to overall reduced costs.
After the initial design and test phases, Engineering teams will continue to implement, secure, publish, manage, observe, scale and retire APIs. During these phases, newer opportunities to reuse and share services would arise, which in turn would follow the API-first approach of Designing and Testing before implementing the APIs.
Takeaways
- An API-first approach requires a cultural change within an organization. This starts with education sessions as well as the creation of an API guild.
- Selection of tools is critical to make it easy for software engineering teams to adopt an API-first approach.
- Everyone on the software engineering team start by first designing APIs even before the first line of code is written, leveraging mock testing frameworks to help simulate tests.
- Software engineering teams leverage consumer driven contract tests to continuously refine and evolve the APIs.
- Developer portal provide for an API catalog allowing software engineering teams to be able to discover existing APIs as well as publish their APIs.
- Development teams can work in parallel. Dependent teams need not wait for the underlying implementation to be completed enabling rapid development & prototyping leading to shorter go to market times.
