API-First: Process and tools
In my blog, I had the opportunity to write about API-First generically. It’s a good exercise since it’s possible to focus on the concept, instead of tools and processes. But the goal of this article is to dive into how to put it in practice.
The problem to be solved is going to be explained below, but before falling into the temptation of thinking about web frameworks, data modeling, virtualization with Docker and Kubernetes, we’ll first focus on the specification.
With some luck, we’ll be able to produce an API that fulfills the needs of our stakeholders.
The problem
Let’s pretend we are going to build a “Feedly-clone”. Let’s start by interpreting the functional requirement:
As a user,
I want to visualize recent news from subscribed websites,
So I can be up to date with topics of my interest
With this user story, we can think about the process behind a practical application of API-First.
The process
Jennifer Riggins, on “How to Design APIs with API-First Design” brings a really practical and throughout vision about how the process should be defined:
- Plan: Before starting, pick a purpose and start drafting the API;
- Design and validate: Discuss the API with other stakeholders and enhance the API design. Prove the concept through mocks and understand how the API is going to be used;
- Turn it official: Build a specification following the plan and the design. Generate documentation based on the spec, improve your mock with use cases and release the specification;
- Test: Ensure the API works. Ensure the use cases are fulfilled. Test to ensure that additions aren’t going to break the current contract. Consider automated tests when possible;
- Implement: Not just you, but other stakeholders are free to code. Sooner you interact with each other, easier it gets to react to changes;
- Operate and engage: Publish it! Interact with your clients, learn from your mistakes, understand what still needs to be done, and repeat the process. API-First is also about business, not just technology.
Single source of truth
The approach used in this article follows the values presented in “Three Principles of API First Design”. More specifically:
Your API comes first, then the implementation
In this example, we are going to in fact separate specification from implementation. In other words, two different artifacts.
There are alternatives to this approach, for instance, you can use your web framework to auto-generate the specification based on some annotations for you.
No matter which option you get, something must be clear: There should be just one source of truth. In this case, it’s going to be the specification file written separately.
1: Plan
Let’s use REST as the communication standard between the server and the clients. Therefore, we can use the OpenAPI Specification to describe the API. In case you decide to use a different technology (for instance, GraphQL or gRPC) you’ll need to use a different approach to describe your interface.
Besides that, we are going to be “RESTful compliant”. We’ll follow recommendations regarding the path, status code, and verbs. The API versioning is going to happen on the path (e.g.: /v1/) and (for now) we’ll be presenting only JSON responses.
Imagine that after debating with other stakeholders, we end up with the following proposition:
[
{
"id": 1,
"title": "Title of article #1",
"pub_date": "date-with-timezone",
"summary": "Summary",
"content": "Full content",
"image": "image.png",
"channel": "ABC",
"url": "http://article-1.html"
},
{
"id": 2,
"title": "Title of article #2",
"pub_date": "date-with-timezone",
"summary": "Summary",
"content": "Full content",
"image": "image-2.png",
"channel": "XYZ",
"url": "http://article-2.html"
},
]And we are going to answer to the following endpoints:
GET /api/v1/channels/<id>GET /api/v1/articles/<id>
Let’s ignore out of scope complexities like authentication and pagination. Also, to keep things simple, let’s avoid dealing with date formats and let’s assume we are using UTC dates.
This is already setting some aspects of the API style guide. But it’s natural to let some details pass without an inner look.
2: Design and validate
This is the document describing with OpenAPI Specification the example mentioned before:
openapi: "3.0.2"
info:
title: Feedly-clone
version: v1
paths:
/channels:
summary: Set of channels, e.g. blogs, websites or podcasts
get:
responses:
"200":
description: List of channels
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/Channel"
/channels/{id}:
parameters:
- $ref: "#/components/parameters/id"
summary: Details of a channel
get:
responses:
"200":
description: Channel detail response
content:
application/json:
schema:
$ref: "#/components/schemas/Channel"
/articles:
summary: Set of items
get:
responses:
"200":
description: List of articles
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/Article"
/articles/{id}:
parameters:
- $ref: "#/components/parameters/id"
summary: Details of an article
get:
responses:
"200":
description: Article detail response
content:
application/json:
schema:
$ref: "#/components/schemas/Article"
components:
parameters:
id:
name: id
in: path
required: true
schema:
type: integer
schemas:
Channel:
properties:
id:
type: integer
name:
type: string
Article:
properties:
id:
type: integer
title:
type: string
pub_date:
type: string
summary:
type: string
content:
type: string
image:
type: string
channel:
type: string
url:
type: stringDesign
From now on we can start thinking from the use case perspective. Tools might vary here, depending on your team’s creativity.
To start with, the path articles/ is not aligned with the semantics behind the problem. The RSS format uses the name “item” for what we are calling “article”, so, if we want a ubiquitous language we may need to change it.
Another controversial topic in our first proposition is that the resource articles/ could be nested in channels/<id>/:
channels/<id>: Channel detailschannels/<id>/items: Item list for a specific channelchannels/<channel-id>/items/<item-id>: Item details
Let’s say that after consulting our mobile and web stakeholders we concluded using the nested resources. This is the exact type of discussion that this step should raise.
As our last topic, let’s face the over-fetching that is happening in the item list. No one is interested in showing the complete content in a list, so let’s remove the content key from the channels/<id>/items/<id> endpoint.
You can check the final spec here.
Validate
It’s time to validate the contract.
Possibility, the most practical way of doing this is by the usage of mock servers. We have different options for different platforms, my suggestion is the tool called Prism, written in Javascript and quite simple to use.
By executing Prism with the created specification we now have a server providing a REST API:
$ prism mock openapi.yml
› [CLI] … awaiting Starting Prism…
› [CLI] ℹ info GET http://127.0.0.1:4010/channels
› [CLI] ℹ info GET http://127.0.0.1:4010/channels/869
› [CLI] ℹ info GET http://127.0.0.1:4010/channels/857/items
› [CLI] ℹ info GET http://127.0.0.1:4010/channels/226/items/12
› [CLI] ▶ start Prism is listening on http://127.0.0.1:4010
$ curl http://127.0.0.1:4010/channels/1/items/1
{"id":1,"title":"Anthony Mackie fala sobre se tornar o Capitão América","pub_date":"Sat, 29 Feb 2020 19:21:37 +0000","summary":"Quero que o meu Capitão América represente todo mundo, não só um grupo específico de pessoas","image":"https://uploads.jovemnerd.com.br/wp-content/uploads/2020/02/anthony-mackie-capitao-america.jpg","content":"Conteúdo completo","url":"https://jovemnerd.com.br/nerdbunker/anthony-mackie-fala-sobre-se-tornar-o-capitao-america/"}That allows us to do new evaluations about our contract, and changing it is not as expensive as changing some Spring or Django code. It also allows other stakeholders to have something to develop.
3: Release the specification
Time to organize the release of the document.
While I was working at Loadsmart, we had the specification in a specific Git repository and a CI to publish the documentation and test based on the version of the file on master.
We have different options for this step. Maybe ReDoc is one of the most “beautiful” options out there. It’s capable of generating API documentation from OpenApi Specification files.
For the sake of brevity, I’m going to use Swagger Hub to publish my documentation: https://app.swaggerhub.com/apis-docs/kplaube/feedly-clone/v1
4: Test
Always test. Automatize when possible.
It’s time to see if the contract is working. Use cases might be needed to stress the API features. I don’t think it’s a good idea to test more than the contract here, but it is up to you to decide how deep your tests are going to be.
Dredd is a quite interesting tool that reads the specification and uses a service as a target, validating the requests and responses. It’s a good idea to add it to your build pipeline, as some sort of component tests.
Running it against our “mocked server” (thanks, Prism) everything should work out of the box:
$ dredd ./openapi.yml http://127.0.0.1:4010
warn: API description parser warning in /Users/klaus/Desktop/openapi.yml:66 (from line 66 column 7 to column 13): 'Parameter Object' contains unsupported key 'schema'
warn: API description parser warning in /Users/klaus/Desktop/openapi.yml:73 (from line 73 column 7 to column 13): 'Parameter Object' contains unsupported key 'schema'
warn: API description parser warning in /Users/klaus/Desktop/openapi.yml:102 (from line 102 column 7 to column 12): 'Schema Object' contains unsupported key 'allOf'
pass: GET (200) /channels duration: 63ms
pass: GET (200) /channels/1 duration: 20ms
pass: GET (200) /channels/1/items duration: 14ms
pass: GET (200) /channels/1/items/1 duration: 11ms
complete: 4 passing, 0 failing, 0 errors, 0 skipped, 4 total
complete: Tests took 116ms5: Implement
Finally, here it comes the fun part :)
Now it’s a matter of using your favorite web framework to write the API. When you are done, don’t forget to run Dredd again against your service. By the way, adding it to your CI could be a great idea!
6: Operate and engage
Your feature is ready. The product is documented. Now it’s time to “sell it”.
With API-First it gets easier to adopt the “eat your own dog food” way of thinking. Having other teams from your organization as users is easier to get feedback and restart the process described in this article with new requirements and possible fixes.
For third-parties, it’s crucial to have your API as part of the business, and it (and its documentation) must have a proper space in your company’s website. Twilio does that quite well.
You also have the option to publish your API in marketplaces. RapidAPI is a good example of that sort of platform which can allow you to connect with a broader number of consumers.
Don’t forget about the SDKs! It’s not enough to write just the backend, sometimes you might need to engage more and even write a client in a specific language for your customers. The Swagger Codegen can help you out with this task, generating SDKs based on the API specification.
Final considerations
We talked about the process, the steps, and some tools that will help you fulfill the requirements of each step. There is room for an article about a more practical way of putting everything to work together.
There is an interesting amount of tools and services regarding API-First. Its adoption might increase the complexity of your development lifecycle, but on the other hand, it’s some sort of “insurance” in a competitive world where APIs are now fundamental parts of the product strategy.
See you next time.
This post was originally written in Brazilian Portuguese for klauslaube.com.br.
