SDD — Schema Driven Development

A more efficient way of developing APIs, Schema first.

Schema Driven Development

Long Story Short (-_-)zzz

Building APIs can be a difficult and painful process. Cross-team collaboration is required and the post development back and forth is almost always expected.

If you care about quality and efficiency, there is a better way to develop APIs that will make your dev life more pleasant. By being more proactive in the initial stages of the development, you can create beautiful documentation, have the incoming requests validated and test your API responses to make sure they follow the correct API contract.

Problem (╯°□°)╯︵ ┻━┻

The typical development between the Backend Team and the Frontend or Mobile Team can be very long and frustrating.

The typical process can look something like this:

• Both teams agree on a new endpoint.
• The Backend Team takes their time to implement the endpoint.
• After the implementation, the Backend Team writes up documentation on what the new API contract is.
• The Mobile Team starts their development and the use of the new endpoint.
• The Mobile Team starts running into issues and has suggestions on the API changes.
• After some bickering, the Backend Team goes back and modifies the endpoint.

Goal ¯\_(ツ)_/¯

When collaborating on software development, communication is 🔑. Lack of good communication can make projects take a very long time and drive developers insane.

Beyond good communication, what would you like your dev process to enhance? There are few things you are guaranteed to get with Schema Driven Development:

• Upfront definition of the API contracts which improves the velocity for both Backend and Mobile teams. This allows the development to be done in parallel.
• A single source of truth for the API contracts that is used for request validations and the API documentation to eliminate discrepancies.
• Nearly no back and forth on the API changes after the endpoint creation.
• Improved morale (^_^).

Solution ᕕ( ᐛ )ᕗ

The Schema

First and foremost you’ll need to define a schema. To define the schema for your API contract you can use a simple JSON file.

Define your schema with Swagger in mind, which is explained below.

A good schema example can be found in a Swagger Tutorial. This schema is defined in YAML which would work only for Swagger Editor but JSON format would work for everything that you need. It will look something like this:

{
  "paths": {
    "/air_quality": {
      "get": {
        "tags": [
          "Air Quality"
        ],
        "description": "gets air quality index",
        "operationId": "getAqi",
        "produces": [
          "text"
        ],
        "parameters": [
          {
            "name": "lat",
            "in": "query",
            "description": "latitude",
            "required": false,
            "type": "string"
          },
          {
            "name": "lng",
            "in": "query",
            "description": "longitude",
            "required": false,
            "type": "string"
          }
        ],
        "responses": {
          "200": {
            "description": "aqi response"
          },
          "default": {
            "description": "unexpected error"
          }
        }
      }
    }
  }
}

The API Documentation

No one likes creating documentation, but what if you can create it seamlessly and, as a bonus, make it look pretty good?

Introducing: Swagger.

You can host the documentation internally in your app. If you’re using Rails, you can utilize an easy-to-use gem called swagger_engine which you can simply mount in your routes:

Now you’ve got Swagger UI in your app using an internal schema to build up a good looking API documentation.

Example of the Swagger UI documentation

Example of the Swagger UI documentation

The Validations

Now you have the schema and a way to present it as API Documentation. You can utilize that same schema to validate the incoming requests to make sure they’re following the correct API contract and won’t be blowing up your controller.

Use JSON Schema Validator, which is part of the json-schema gem, if you’re on Rails. An example of typical validation in the controller would look something like this: https://github.com/ruby-json-schema/json-schema#basic-usage.

You can simply reject all of the requests that do not conform to your expectations in the controller.

The Specs

You can now use the defined schema to test your responses. If you’re using Rails, you can extend json_matchers in your specs. Thank you Thoughtbot!

If at any point your response changes, the schema matcher should catch that and your specs will go red.

Matching the schema is pretty easy and an example can be found right here: https://github.com/thoughtbot/json_matchers#usage.

Summary

By collaboratively defining your schema ahead of time with the Mobile Team you are able to work in parallel on the implementation. As the Backend Team is developing the API endpoint, Frontend or Mobile Team is able to work on the feature that uses that endpoint at the same time. Both teams know exactly what to expect.

“And they have developed happily ever after…”

The implementation of Schema Driven Development results in eliminating the back and forth between the teams and speeds up the overall development process. The development feels a lot more like a collaboration than a hand off and at the end of the day, everyone is a little happier.