Bringing law and order to APIs with OpenAPI Specifications
A better way to build and manage APIs using schemas, descriptions, and style guides
A specification is a technical document that tells you how a thing works.
If you’re building cars, a specification tells you how a car works. It includes the most important details and perhaps a list of requirements that the end product should fulfill. Once the car gets built, the owner’s manual shows a driver how to operate and maintain the car.
Similarly, if you’re building APIs, a specification tells you how an API works, along with the most important details. The documentation shows a consumer how to work with the API.
We’ve previously talked about software development in an API-first world and also how to design APIs like you design user experience.
Now let’s see how API descriptions help us build and manage APIs.
A brief history of API descriptions
Since the beginning of APIs, various specification formats have evolved in an effort to standardize the way we describe them.
- Web Services Description Language (WSDL) was created by Ariba, IBM, and Microsoft as a machine-readable way to describe services in 2001.
- Web Application Description Language (WADL) was from Sun Microsystems as a RESTful equivalent to WSDL in 2009.
- Swagger was a project from 2011, acquired by SmartBear, then donated to the Linux Foundation’s Open API Initiative, eventually to be renamed OpenAPI Specification.
- RESTful API Modeling Language (RAML) was from MuleSoft and a group of other enterprise companies in 2013 to focus on API Design.
- OpenAPI Specification (OAS) is currently the market leader in specification formats and supported by a number of large enterprises.
These are a few of the formats that gained popularity in recent years. Their goals are all to standardize the way in which you describe your resources, and they’re all useful in different ways for developing APIs.
And the evolution continues. For example, some community members want to introduce more human-readable elements, describe more use cases for the end consumer, and find other ways to govern API design and development in a scalable manner.
Some confusing terms
In addition to a continually evolving landscape, there’s a bunch of ambiguous terminology. Industry experts don’t entirely agree on the terms, and they’re often used interchangeably when they really don’t mean the same thing.
Schemas — this is how data looks
Schemas include metadata referring to your data models and provide information about the resource representations that your API accepts or returns. For example, JSON Schema is a specification declaring the proper structure of JavaScript Object Notation (JSON) data used frequently with web APIs, and is used to validate the correct format for the body of a request or response.
API description formats — this is a way to describe APIs
These formats are sometimes called API specification formats, and are usually based on a specification format (such as OpenAPI Specification) that combines a schema language (such as JSON Schema) and other information to describe endpoints, headers, and all the stuff that’s not in the body (that’s covered in a schema).
API description document — this is how a particular API works
These documents are sometimes called API specifications, and is a file that contains all the stuff from your description format fleshed out with information related to your actual API. This file describes how the API works. It can be used as a contract to programmatically build workflows useful for API development, such as tests or documentation.
API documentation — this is how to use a particular API
Documentation talks about how to use an API, and will typically include a technical reference as well as functional guides for how to interact with the API across various use cases. An API consumer who reads the API documentation should understand how to use the API.
There are meaningful differences between all of these concepts. But once again, experts don’t always agree on these meanings, and they’re frequently referred to interchangeably or ambiguously. For example, when someone says “API specification”, they could be referring to either the format for describing APIs, the file intended to drive API development, or the file generated to document an API.
The important thing is to understand these basic concepts, and then determine what is helpful for your organization’s desired workflow.
If your head hurts from reading this, so does mine. Let’s forge ahead and walk through a couple examples.
Example #1: Building blocks
From the building blocks of modern software to the building blocks of life — let’s talk about APIs and DNA.
In biochemistry, DNA is the blueprint for every living organism. A schema establishes the rules for how biological molecules pair together to form chemical strands of a double helix 🧬
The human genome is a description format offering a complete genetic blueprint for building human beings. This mapping provides detailed information about the structure, organization, and function of the complete set of human genes.
My personal DNA sequence can be captured in a description file. My DNA follows the general rules of DNA, particularly that of a human’s DNA, and not an alligator or mushroom 🐊🍄
My documentation helps others understand how to interact with me. I must be fed 4–5 times a day, and I’ll get grumpy if you talk to me while I’m busy working with my headphones on.
Let’s walk through one more example — this time with our friendly, neighborhood Postman.
Example #2: Postman collections to describe an API
The concept of a Postman collection is an API description format (or API specification format), in the sense that it’s how you can describe an API. The underlying JSON representation of a collection must be formatted a certain way as dictated by a schema to ensure the collection is valid.
Now, imagine Twitter has an API. When Twitter uses a Postman collection to describe how their particular API works, Twitter’s collection becomes an API description document (or API specification). Maybe the Twitter team includes a few different examples to demonstrate authorization use cases, or adds some code to their collection to show off different workflows.
As an API description document (or API specification), Twitter’s Postman collection can now be used to generate code samples, virtualized services, or API documentation to help developers learn how to use their API.
Ok , enough with the examples — it’s time to get tactical!
Test APIs and lint specifications with OpenAPI Specification, Newman, and Spectral
Just like a clothing lint roller creates a more uniform and consistent appearance by removing bits of fuzz and fur from all of your surfaces, a code linter does the same.
You may have used a code linter to ensure that your code is error-free, readable, and contains impeccable syntax for whatever programming language you’re using.
We’ve already talked about writing tests in Postman and different ways to automate your testing. Now let’s use an API specifications linter and create a custom style guide to bring law and order to our untamed APIs.
We can either run these checks locally from the command line, automatically using git hooks, or as a build step during our continuous integration workflow.
Let’s run tests and lint specifications in our CI pipeline.
Step 0: Pre-requisites
If you want to skip to the punchline, go ahead and clone this example, and follow the README to try it out locally. Otherwise, follow these pre-requisites.
- You’ll need a Postman account to use the Postman API to programmatically access data stored in your Postman account. Skip this step if you’re working with standalone files of your Postman collection, environment, and OpenAPI specification.
- Make sure you have Node.js and a package manager like npm installed on your machine. Then, start a new project and install your dependencies locally.
$ npm init
$ npm install newman @stoplight/spectralNewman — an open-source library by Postman for running Postman collections and tests
Spectral — an open-source JSON / YAML linter that supports OpenAPI Specification and JSON Schema
Step 1: Lint an OpenAPI specification file locally
Download this OpenAPI specification file to your project directory. In your terminal, use Spectral to lint this specification file, and then decide which errors or warnings you want to fix.
$ ./node_modules/.bin/spectral lint cosmos.yaml
There’s a number of options for installing Spectral, and depending on how you’ve completed the installation or have Node set up, you may need to fiddle with it. If you plan to use Spectral across multiple projects, install it globally to run:
$ spectral lint cosmos.yaml
Step 2: Generate a Postman collection from the OAS file
In the Postman app, sign in to your Postman account, and then import the specification file as an API. Give this new API a name, select OpenAPI 3.0 and YAML from the options, and then hit Save.
Now you can automatically generate a Postman collection from your specification file. For now, we’ll just write API tests. But remember, you can also generate documentation, code samples, monitors, or even virtualize mock services in Postman.
💡 POSTMAN TIP: Import your specification file into the Postman app to generate a collection. The collection is the foundation for creating documentation, code samples, monitors, or mock services in Postman.
You’ll notice this newly generated collection describes the API just like the OAS file does, but it’s not quite actionable yet.
Create a new Postman environment with a key baseUrl and the value https://e8c086f7-a5c9-4752-b81f-bfac0d0dc5d2.mock.pstmn.io.
Now you can submit your first request — go ahead and try it out!
Step 3: Write some Postman tests
We won’t make any changes in this folder so that you can refer back to it. Instead, duplicate this folder to add some tests.
Under the Tests tab, add a few tests using the snippets on the right or writing your own from scratch. For the first request GET List all cosmos, let’s also randomly pick one of the items returned in the response, parse the ID, and then save that information as an environment variable so we can use it in the next request. Remember to hit Send to execute this code.
In the next request GET Info for a specific constellation, look under the Params tab and change the value of the cosmoId key from <string> to {{constellationId}}. This is the name of the environment variable that you set from the previous response, allowing this data to be accessed for this request.
Step 4: Run the Postman tests locally
In the Postman app, we can run all of our Postman tests using the collection runner. Another option is to export the collection and environment as JSON files to run tests from the command line using Newman.
$ newman run Cosmos.postman_collection.json -e cosmos.postman_environment.json
Step 5: Run tests and lint specifications in CI/CD
Instead of running our checks locally, let’s add it to our continuous integration pipeline. We’ll use a bash script, but you can execute this in any manner that you like. Create a new folder called bin/ to hold a new executable file called deploy.sh.
Once again, we’ll use Newman to run our collection and Spectral to lint our specification. Last time, we used static files located within our project directory, but a better practice is to pass these elements as a URL. Using the Postman API to dynamically retrieve the latest versions, means that we can make changes in Postman without needing to re-export the latest files.
Here’s an example of how you can structure your deploy script:
💡 POSTMAN TIP: Instead of exporting and then referencing static files, use the Postman API to dynamically retrieve the latest version of your collection, environment, or specification, and then pass these elements as a URL.
Once you’ve written the script, remember to add the path of the deploy script file to your package.json file.
...
"scripts": { "deploy": "./bin/deploy.sh"},...
Then you can run your deploy script from the command line.
$ npm run deploy
If you’re using a tool like Jenkins or Travis CI to manage your Continuous Integration / Continuous Deployment (CI/CD) pipeline, simply add a build step to execute this command.
If we experience any failures or errors, our code won’t deploy.
Step 6: Create a custom style guide
We’re testing. We’re linting. These are all good things!
But hold up. Just like how tests are only as good as we make them, linting is only as good as the rules we’ve defined.
If you work on a team, there’s probably style preferences and code requirements that everyone abides by. When pull requests are submitted, deviations are hopefully caught in code review, but frequently they’re missed. And I can already see there’s default lint warnings that I have no intention of ever addressing.
Let’s make a style guide by creating a new file called .spectral.yaml where we will define our custom rules to use with Spectral. For example, I want all of our URL paths to be camelCase, and not some ridiculous kebab-case!
Style guides ensure that everyone agrees to specified conventions. Creating a style guide in this manner also provides an easy way to enforce these conventions.
A final thought about API descriptions and linting
Using an API specification can be useful to standardize how data is exchanged between services. Furthermore, it can be useful to standardize how information is relayed between collaborating teams and throughout various phases of development.
Using a specifications linter allows you to save time otherwise spent on manual reviews. Creating a custom style guide allows you to align your team’s opinions about API design and documentation. And perhaps even more importantly, it’s also an easy way to automatically enforce these rules and govern the API development process.
Don’t waste customers time forcing them to try and figure out your inconsistencies.
Don’t waste all API developers time learning to memorizing style guides.
Don’t waste the API governance teams time reviewing APIs manually.
Don’t waste everyone’s time fixing inconsistencies in production later.
As with all of these better practices, the important thing is to understand the possible solutions, and then determine what is most helpful to achieve your organization’s desired workflow.
