API Lifecycle Management
Shaping an API Lifecycle Model for Your Organizational Context
A shared API lifecycle model and vocabulary will simplify governance, enhance productivity and underpin practical guidance for developers from design through to release.
The API Lifecycle model presented in this article is not intended to be prescriptive, rather it is meant to spark consideration, discussion and agreement on a tailored and fit-for-purpose API Lifecycle model.
This API lifecycle management model will discuss the following facets:
- API lifecycle phases
- API lifecycle states
- API version management
- Design tooling and documentation conventions
- Automation and CI-CD
API Lifecycle Phases
The API lifecycle is sometimes characterized in terms of production readiness, sometimes in terms of development stages. In an enterprise managed API context, an API lifecycle must encompass both of these perspectives, as well as the reality of multiple environments and pre-production consumer feedback and iteration.
Starting point
- Discovery: Discovery is outside of the API lifecycle, however it is the setup and the initial entrypoint. It is the process of demand generation, prioritization and planning that results in a prioritized backlog and kickoff of the design phase.
- Evolution: Observation, evaluation and client feedback on APIs deployed to either test or production environments may trigger design changes. Changes may be breaking or non-breaking.
Design
- Model: Through a process of broad stakeholder consultation, distillation and frequent iteration, a model of the domain is built. A collaborative, version controlled and readily accessible domain data modelling tool is recommended.
- Validate: The domain model “acts as a Ubiquitous Language to help communication between software developers and domain experts”, maximizing collaboration, tightening the feedback loop, and ensuring that the domain model remains the definitive source-of-truth.
- Document: JSON schema and prototype API specification(s) are generated from the model in conformance with API standards and security policies.
More: Modeling Coherent and Composable Business Resource APIs
Build & Test
- Build: The prototype OpenAPI document & JSON schema are inputs into the build phase. The API implementation is built to API design and security standards. The API is deployed to sandpit & Dev environments for iteration & testing, and may be initially back-ended by a mock service.
- Test: ‘Code-first’ API continuous integration and publishing automation can apply Policy-as-Code controls, enforce mandatory standards with specification linting, and validate security schemes. Deployment and regression testing validates security, function and performance.
- Evaluate: Testing, client engagement and analytics may inform an evaluation of the API resulting in either promotion or rework. In a certified CI/CD pipeline, this step may be entirely automated. The API may loop through several test environments and evaluation checkpoints before it is published to production.
More: Managing API Lifecycles with Model Driven, Integrated DevOps
Release
- Publish: APIs that survive the final test and acceptance stage-gate are published to production. APIs might be published to an internal, partner or public facing catalog as appropriate. If a previous version of the API exists, it is deprecated at this point.
- Support & Observe: The API is actively supported, tested and monitored. Client engagement (comments, issues and notifications) is encouraged via the client portal. Feedback and analysis may provide insights that shape the evolution of APIs and business processes.
- Deprecate: As APIs evolve and are republished as a new major version, the previous version must be deprecated — which means maintaining support for a period in order to allow existing clients adequate time to cut-over to the new version. No new clients are able to subscribe to a deprecated API. A retirement schedule should be communicated to subscribed clients.
- Retire: Once existing clients have cut-over to the new version, and/or the scheduled retirement date arrives, the API is retired, and will no longer be available. Calls to this API will result in a gateway error.
Security
It is important to note that securing APIs is not a step — but rather is embedded into each phase.
- Data is classified and regulatory controls identified during the modelling phase
- API development guidelines outline patterns and tactics to be employed by developers to secure APIs
- API specification linting is applied during integration into source control
- API management and API gateway security policies are enforced on test and production gateways
- API testing incorporates OWASP testing as well as targeted testing against applicable security controls
More: API Bites — Tactics to Secure Sensitive APIs
API Lifecycle States
API Lifecycle states are of particular interest to consumers, as they will have ramifications in terms of stability and support. The lifecycle states of an API are as follows:
- In Design — feedback encouraged, subject to change at any time.
- In Test — feedback encouraged; breaking changes may be made.
- Published — released to production, stable and actively supported.
- Deprecated —soon to be retired. New clients are unable to subscribe.
- Retired — the API has been shut down. The gateway will return errors.
Managing API Versions
A key aim of business resource APIs is stability. However, in the pursuit of continuous improvement, breaking changes are almost inevitable, and in order to avoid a ‘big-bang’ coordinated cut-over there will occasionally be a need to support more than one version of an API.
Versioning APIs
Versioned URLs are widely employed for the management of API versions. The scheme is somewhat limiting but uncomplicated, making it easy for clients to comprehend and control which API versions they interact with. This is the default API versioning mechanism in a specification-first managed API environment, and this bite-sized discussion makes the (low-risk) assumption that this scheme will be employed.
Versioned URLs must only include the MAJOR version as part of the URI, in the format ‘v{MAJOR}’, e.g.
/membership/v1/applicantsThe version number typically precedes the resource (and API specification) name in the path — it should never appear between the resource name and the resource ID.
Minor and patch versions (assuming SemVer versioning) must NOT be included in the URI as such changes are by definition backwards-compatible.
Versioning options and considerations are covered in the article Versioning Managed APIs.
Semantic Versioning
API Models and specifications should be managed with semantic versioning.
{MAJOR}.{MINOR}.{PATCH}
Ideally, domain modelling tooling would manage semantic versioning across all derivative artifacts.
As a guide the following rules should apply when incrementing the API version number:
- MAJOR version when incompatible or breaking API changes are made,
- MINOR version when functionality is added in a backwards-compatible manner, and
- PATCH version when backwards-compatible bug fixes are made.
Backwards Compatibility
Backwards compatible (minor version) changes include:
- Addition of a new field (non-mandatory) to a representation
- Addition of a new operation
- Addition or removal of HATEOAS links
- Additional support of a new media type (e.g. Accept: application/pdf)
The following changes are NOT considered backwards compatible:
- Removal of fields from representations
- Changes of field data types (e.g. string to boolean)
- Removal of an operation
- Removal of media type support
- New or tightened API security definitions
Test Environment Break-Fixes
Major version increments should apply only to breaking changes to APIs deployed to production, and this can create difficulties when deploying break-fixes to yet-unpromoted APIs in integrated test environments. In this case increment the minor version number and use API analytics to identify and notify active API consumers that are likely to be impacted.
Replacing a Major API Version
Justify, Document and Socialize. Introducing a new major API version will impact all consumers. Before deciding to introduce a new major version, API owners should explore all possible alternatives. A decision to introduce a new major version should be documented as an Architecture Decision and API owners should ensure that the potential impacts are discussed in agile planning forums.
Decouple Cut-over. The newly deployed version is effectively a new API, and consumers must subscribe to the new API version in order to use it. With each new major version of the API the older versions should be deprecated (no new subscribers) and maintained until all consumers have cut over to the new version. This can add introduce significant complexity on the development and support side.
Communicate a Retirement Schedule. Identify consumers and communicate an API deprecation and retirement schedule with sufficient notification. API deprecation and client notification are supported by most API management platforms.
Retire and Remove the API. Once there are no remaining subscribers OR when the scheduled retirement date arrives, raise a change to retire the API. Retiring an API is not the same as removing an API — retirement is easily reversible. A change to remove an API should follow service changes that remove support for the old API altogether.
Design Tooling and Documentation
API Documentation
The OpenAPI Specification (OAS) defines a standard, programming language agnostic interface description for REST APIs. Importantly, the specification is widely supported by API Management platforms, and by a number or domain modelling platforms.
Adoption of the most current, generally supported specification version is recommended. Take the time to research the level of support offered by commercial API management platforms — including both gateway and developer portal support for key features.
The “info” object “version” field MUST be contain the full semantic version number of the API— {MAJOR}.{MINOR}.{PATCH}
info:
version: 1.1.2API request and response examples SHOULD be included in the specification document to assist client developers.
OpenAPI document linting rules should be clearly defined. Spectral OpenAPI rules are widely referenced as a base OpenAPI document ruleset.
More on OpenAPI document and linting rules here:
Domain-Driven API Design Practice
An enterprise domain-driven API design practice is recommended.
The domain model “acts as a Ubiquitous Language to help communication between software developers and domain experts” (Fowler, M 2014, BoundedContext), maximising collaboration, providing the tightest possible feedback loop, and ensuring that the domain model remains the definitive source-of-truth.
API specifications are generated from the model, in conformance with API standards and security policies. These may be tool generated based on standards and policy-conformant rules or templates, or they may be ‘hand-crafted’ from templates. In either case, peer review is essential.
Ideally, domain/REST modelling tools will support source control, and manage semantic versioning across the model and its derivative artefacts — features that are helpful in maintaining the currency and traceability of published APIs.
Managing API lifecycles with model driven development, API generation and automation is discussed in Managing API Lifecycles with Model Driven, Integrated DevOps
More on API Design practice and tooling here:
Vertically Integrated DevOps
API enabled, interoperable business services often endure considerable friction and blocking engagements coordinating the delivery of new and updated interfaces. A well-defined process of model driven development, complimentary tooling and vertically integrated DevOps can substantially reduce friction between domain model development and delivery.
A Single Repository
API specification documents and config files should be co-located in a common repository with the service implementation. API Gateway CI-CD pipelines are triggered from the product pipeline.
CI-CD Pipeline Enrollment
Enterprise endorsed DevOps tools and platforms should be identified in API Lifecycle Management guidance, with links out to more detailed tool and platform-specific documentation and ‘getting started’ resources. Typically, onboarding into lower (dev) environments will present a low barrier, however onboarding an autonomous CI-CD pipeline will require tighter governance and accountability.
Compliance with Security Controls and API Design Standards
API design standards and security policy compliance is automated wherever possible. Continuous integration and delivery pipelines will apply Policy-as-Code controls, enforce mandatory standards with specification document linting.
Coordinated, Automated Deployment
As managed APIs are the interface to deployed microservices, DevOps tooling must ensure that new and updated interfaces and schemas generated by domain modeling tooling and implemented by a business service are published to relevant API management (and potentially event, security and distributed graph) platforms simultaneously with the deployment of the business service. Deployment and testing of APIs must be automated, domain-autonomous and as frictionless as possible.
When new versions of an API are deployed, automation must be able to detect and deprecate previous versions (no new subscribers allowed).
Vertically Integrated DevOps is discussed further in Managing API Lifecycles with Model Driven, Integrated DevOps
Registering API Resources with IAM
An API lifecycle model should introduce resource registration concepts and refer developer to guidance for the registration of new business resources, data profiles and resource access policies to API Management, Security Token Service (STS) and Identity Access Management (IAM) platforms.
Registration of API resources is further discussed in Securing APIs with an Integrated Security Framework
Wrap-Up
A shared organizational understanding of API lifecycles and its corresponding vocabulary will simplify governance and aid communication and productivity. An API Lifecycle model should be tailored to a particular organization context and widely socialized.
It is important to cover development lifecycle phases, lifecycle states and ramifications for consumers, guidance around API version management, design tooling, documentation conventions and automation/CI-CD. The model must encompass both the overhead and opportunities presented by multiple environments, and pre-production feedback and iteration.
