Automating API Documentation

This article explains how we document our REST APIs in Condor Labs, in a maintainable and testable way, in this post you will find:

1. Introduction

2. Tech Stack

3. Scaffolding

4. Writing the API specification with OpenAPI

5. Generating the API documentation with ReDoc

6. Testing the API services against the API specification with Dredd

7. Setting TravisCI to validate the API specification

8. Enabling GitHub pages to publish the API documentation

1. Introduction

Our APIs are the components that define how apps or other clients must interact with our services, the capabilities of the services and the data types that are handled by our API. API Documentation it’s critical for its usability and maintainability, and for integration with other services and applications.

The APIs documentation must be clear, concise and simple to understand, However maintaining this documentation, sometimes become into a headache for the development team. Evolving the documents at the same pace that our services, involves extra work that some development teams prefer to avoid, in many cases because they don’t have enough time.

When we decided to start to document our APIs at Condor Labs, we had some things clear:

1. The process for documenting our APIs must be straightforward, the documentation must be done with some tools which allow us to generate the documentation in a simple way, without developers worrying about structuring, styling and formatting documents.
2. We need to have a way to validate the API definition against their implementation, so we could identify if there are differences between the services and their specification.
3. We should have a mechanism that allows us to protect the API's code against undocumented changes or unimplemented specifications, that is, an automatic way to validate our services before committing our changes in the master branches.

We selected the “spec-first development” approach which recommends to the teams, to write the specifications by hand and then handle this specification as a contract that the developers use when they write the code.

With that in mind, we choose some tools which allow us to specify the services in our APIs, to develop the services that match those specifications, and generate the documentation to the final users in an automatic way.

2. Tech Stack

We used the OpenAPI standard (formerly knowns as Swagger) to write the description of the services that the API must offer to users, we used ReDoc to generate the documentation for the final users and Dredd to validate that the API implementation match the specified contract.

Tech Stack for API documentation

3. Scaffolding

We created two new folders /docs and /spec in our project. We added the api-description-v2.yml file into /spec folder to write the API specification using YAML.

4. Writing the API specification with OpenAPI

OpenAPI is a specification that defines a standard for the description of REST API interfaces, which allows to discover and understand the capabilities of the services.

Why OpenAPI?

There are several REST specifications that have been developed to try to provide some standards about how REST APIs are described. The three most popular REST API specifications are as follows: OpenAPI (formally called Swagger), RAML, and API Blueprint.

In the early years of specifications, there was healthy competition between the formats. But now, without a doubt, the OpenAPI specification is the most popular, with the largest community and tooling.

The current version for OpenAPI is 3.0.1 but considering that most tools are not supporting this version yet, we chose version 2 for our documentation. You can learn about the OpenAPI V2 specification in this link.

Writing the API Specification

With OpenAPI, you have a JSON file, with a specific schema that defines their naming, order, and contents. This JSON file (often expressed in YAML instead of JSON) describes each part of your API.

We are not going to go deeper in the description of the Open API document elements. You can go to the official documentation in the GitHub repository OpenAPI V2 specification.

API definition

At the beginning of the api-description-v2.yml file we specified the version of the OpenAPI that we use in our document and the general aspects of the API:

Paths definitions

Some of the OpenAPI elements are paths, parameters, responses, and security. Each of these elements are actually an “object” that holds a number of properties and arrays. In our example we are going to specify the path for the Professions entity.

If you would like to read more about how to document your API using OpenAPI specification, you can follow this OpenAPI tutorial which shows you how to document an API step by step.

In this case, we defined the GET /professions method to get a list of professions and the POST /professions to create a new profession:

5. Generating the API Documentation with ReDoc

ReDoc creates a custom HTML tag from your OpenAPI specification, and provides you with a handful of attributes for defining and customizing their documentation, including specurl, scroll-y-offset, suppress-warnings, lazy-rendering, hid-hostname, and expand-responses — providing some quick ways to get exactly what you need, on any web page.

We used ReDoc CLI to generate the documentation from the API specification in our .yml file, so we installed redoc-cli like a development dependency in our project:

npm install --save-dev redoc-cli

Once we have installed redoc-cli we were able to use the next command to create the HTML documentation:

redoc-cli bundle ./spec/api-description-v2.yml --output ./docs/index.html

ReDoc take the specification in our OpenAPI document and create an index.html file with the page with the documentation and place it into the /docs folder.

So you can go to the /docs folder and open the index.html file in a browser and see how ReDoc has converted our plain specification in a great API documentation:

Automating the Documentation

We didn’t like to have to execute the command all times to see the changes in the API specification, so we configured our project to serve the documentation in a site and keeps the documentation updated while we modify the API definition in our specification file. To reach that we followed the next steps:

1. Create the script redoc-bundle in the script section in the package.json file to run the command we saw previously:

“redoc-bundle”: “redoc-cli bundle ./spec/api-description-v2.yml --output ./docs/index.html”

This is how it looks in the package.json file:

Now we could execute this npm script from our terminal:

npm run redoc-bundle

2. Serve the documentation page and enable hot-reloading to reload the page when the html file change

To serve the /docs/index.html and update it when there are changes we installed the live-server package like a development dependency in our project:

npm install --save-dev live-server

Then we created a new npm script to serve the HTML doc with live-server, which update the page in the browser each time the HTML file change:

“redoc-reload”: “live-server —-watch=docs/ --open=docs/”

This is how it should look in the package.json file:

3. Set a script to help us generate the documentation when we change the API specification

To generate the documentation in the/docs/index.html file when we update the specification in the /specs/api-description-v2.yml, we installed watch like a development dependency in our project.

npm install --save-dev watch

And we configured the redoc-watch script which allows us to monitor the changes into the specification file and then generate the documentation in HTML format:

"redoc-watch": "watch \"npm run redoc-blundle\" ./spec"

4. Run the scripts simultaneously

To run both scripts redoc-reload and redoc-watch, without requiring open two terminals we installed the concurrency package like a development dependency:

npm install --save-dev concurrently

And we created the api-doc script that allows us to run both scripts at the same time:

“api-doc”: “concurrently \”npm run redoc-reload\” \”npm run redoc-watch\””,

This is how it should look in the package.json file:

6. Testing the API service against the API specification with Dredd

Our next challenge was how to test our services from the specification, and we selected Dredd like our tool to reach it, Dredd is a command line tool for validating the API description against the backend implementation of the API.

What Dredd does is to evaluate the actions that can be performed on the resources (HTTP verbs) and, based on the requests/responses defined for each action and validate it against the specification.

The first thing we did was install Dredd as a development dependency:

npm install --save-dev dredd

Once we had Dredd installed as a development dependency in our project, we were able to validate our API against the specification. We run the API using npm start (this command serve the API in http://127.0.0.1:3000) and we run Dredd to validate our specification:

dredd ./spec/api-description-v2.yml http://127.0.0.1:3000

Dredd can be configured by many CLI options. It’s recommended to save your Dredd configuration alongside your project, so it’s easier to repeatedly execute always the same test run. We used interactive configuration wizard to create a dredd.yml file in the root of your project:

$ dredd init
? Location of the API description document: api-description-v2.yml
? Command to start API backend server e.g. (bundle exec rails server)
? URL of tested API endpoint: http://127.0.0.1:3000
? Programming language of hooks:
❯ nodejs
python
...
? Dredd is best served with Continuous Integration. Create TravisCI config for Dredd? Yes

Once the dredd.yml file is created in the root folder of your repository, you can modify this file when you need. Now you can only execute the dredd command into your terminal and run the API validation.

7. Setting Travis to validate the API specification

We configured command dredd as a npm script adding the following line to our package file:

“test:spec”: “dredd”

This is how it should look in the package.json file:

If you have TravisCI configured in your repository, you must add the following instruction to the current travis configuration file travis.yml

script:
  - CI=true npm run test:spec

In this way, we asked TravisCI to run dredd command. In our case, TravisCI runs these tests and when these pass, it allows the integration to the master branch, so when we try to integrate to master we can see the result of Dredd validations in the TravisCI dashboard.

8. Enabling GitHub pages to publish the API documentation

The most important part of documenting an API is to publish the documentation so our users can read and understand all the services and how to interact with them. To do that we used Github Pages.

In this case, we published our documentation site from a /docs folder on our master branch. you can review the official documentation for GitHub pages or to follow the next step:

1. On GitHub, navigate to your GitHub Pages site’s repository.
2. Under your repository name, click Settings.
3. Use the Select source drop-down menu to select master branch /docs folder as your GitHub Pages publishing source.
4. Click Save.

And in a few minutes you’ll have your API documentation published in the web!

That’s it! Now we can write the services specification, develop our services, and we have the documentation updated and a way to test the API services against the definition. We have protected the API project against not documented changes and we’ll have an updated documentation in GitHub pages each time we commit changes in the project. 🎉