How and why JSON schema in Rails always saves the day

5 min readNov 29, 2016

At Maxwell Forest, we’ve been using JSON schema to validate our persistent data and API response. It’s saved us a whole lot of time, and helped us bring order to our database. In this blog post, I’ll talk you through the what and why of JSON schema, demonstrate how to apply JSON schema to Rails model validation, and also how to test your API endpoint with schema matcher.

First things first: What is JSON schema?

I recommend you read this and this to understand more about JSON schema.

However, if you’re lazy like me, just think of JSON schema as the spec of your JSON data. It basically helps you define how your JSON data should look.

Here’s an example of the simplest schema:

{
  "properties": {
    "name": {
      "type": "string"
    }
  },
  "required": ["name"],
  "type": "object"
}

This schema means your JSON object should have a name attribute with a string type.

For example, this is a valid JSON:

{
  "name": "Wayne"
}

And this is not a valid JSON, because the name is a number, but not a string:

{
  "name": 5566
}

Why we need JSON schema

So, why do we need JSON schema? What’s the benefit?

First of all, defining your data properly is never a bad idea. And there are at least 4 other benefits I can think of:

A JSON schema validates your JSON data structure, so you don’t mess up your database;
It help you validate your API’s response, especially REST-like API;
One rule works everywhere, which also helps your client validate their data; and
As a bonus it integrates with Swagger (if you use it)

Cool, so now let’s write some code with our beloved Ruby.

Applying JSON schema to Rail’s model validation

I have found two gems that help me validate JSON schema. The first one is the Ruby validator implementation of JSON schema called json-schema. The second one is the Rails validator implementation called activerecord_json_validator, which is based on json-schema.

In this example, I’ll use activerecord_json_validator to integrate our model level JSON validation.

Let’s assume we have User and Report. When we allow a user to send an error report to us that includes their system’s environment and save it in the data column as JSON, the migration file looks like this:

# db/migrations/xxxxxxxxx_create_reports.rb
class CreateReports < ActiveRecord::Migration
  def change
    create_table :reports do |t|
      t.references :user
      t.jsonb :data, null: false, default: "{}"
      t.timestamps null: false
    end
  end
end

We want our Report#data to have at least two keys: devise_id and version. A valid JSON, therefore, should look like this:

{
  "devise_id": "devise-id-is-a-string",
  "version": "5.56.6"
}

We can test our model validation by writing RSpec code like this:

# spec/models/report_spec.rb
RSpec.describe Report, type: :model do
  describe 'validates data column' do
    # We use Factory girl to create fake record
    subject(:report) { create(:report, data: data) }
    let(:valid_data) do
      {
        devise_id: 'devise-id-is-a-string',
        version: '5.56.6'
      }
    end

    describe 'valid data' do
      let(:data) { valid_data }
      it 'creates report' do
        expect { report }.to change { Report.count }.by(1)
      end
    end

    describe 'invalid data' do
      context 'when missing devise_id' do
        let(:data) { valid_data.except(:devise_id) }

        it 'raise validation error' do
          expect { report }.to raise_error(ActiveRecord::RecordInvalid)
        end
      end

      context 'when missing version' do
        let(:data) { valid_data.except(:version) }

        it 'raise validation error' do
          expect { report }.to raise_error(ActiveRecord::RecordInvalid)
        end
      end
    end
  end
end

Install the gem:

# Gemfile
gem ‘activerecord_json_validator’

Add validation into Report:

# app/models/report.rb
class Report < ActiveRecord::Base
  JSON_SCHEMA = "#{Rails.root}/app/models/schemas/report/data.json"

  belongs_to :user

  validates :data, presence: true, json: { schema: JSON_SCHEMA }
end

Add the JSON schema file:

(Note: I prefer to add the .json file into app/models/schemas/report/data.json)

// app/models/schemas/report/data.json
{
  "$schema": "http://yourdomain.com/somewhere/report/data",
  "type": "object",
  "properties": {
    "devise_id": {
      "type": "string"
    },
    "version": {
      "type": "string"
    }
  },
  "required": [
    "devise_id",
    "version"
  ]
}

Now all tests pass.

Test your API endpoints with schema matcher

Now it’s time to add some API endpoint response tests.

Assuming we have the user APIs GET /users and GET /users/:id, let’s define our response:

// GET /users
{
  "users": [
    {
      "id": 1,
      "name": "John John Slater",
      "email": "jjs@example.com",
      "is_good_surfer": true,
      "updated_at": "timestamp",
      "created_at": "timestamp"
    }
  ]
}

// GET /users/:id
{
  "user": {
    "id": 1,
    "name": "John John Slater",
    "email": "jjs@example.com",
    "is_good_surfer": true,
    "updated_at": "2017-02-01T10:00:54.326+10:00",
    "created_at": "2017-02-01T10:00:54.326+10:00"
  }
}

Now, let’s write some tests.

I’m using RSpec, and have found a gem called json_matcher. It’s also based on the other gem json-schema, so you can choose either one to implement your test. Check out this post to get a better idea about how to do this.

Ok, time to get our hands dirty.

First, let’s do the setup:

# Gemfile
gem 'json_matchers'

# spec/spec_helper.rb
require "json_matchers/rspec"

# spec/support/json_matchers.rb
JsonMatchers.schema_root = "controller/schemas"

Now, let’s write the tests:

RSpec.describe User, type: :request do
  describe 'GET /users' do
    let!(:user) { create(:user) }
    subject! { get '/users' }

    specify do
      expect(response).to be_success
      expect(response).to match_response_schema('users')
    end
  end

  describe 'GET /users/:id' do
    let(:user) { create(:user) }
    subject! { get "/users/#{user.id}" }

    specify do
      expect(response).to be_success
      expect(response).to match_response_schema('user')
    end
  end
end

This test will actually fail, as we haven’t added a JSON schema file yet.

So let’s now add a schema file for the user.

One little trick that you might notice here is how we add user object definitions to definitions, so we can reuse them. It’s a technique called reference — you put an object’s schema into the definitions block of your file, and use “$ref”: “#/definitions/<object>” to reference it.

It’s a useful practice to split your schema into definitions, as they facilitate schema reuse in the same way that writing small and concise methods can facilitate code reuse in your program.

// app/controllers/schemas/user.json
{
  "type": "object",
  "required": ["user"],
  "properties": {
    "user": {
      "$ref": "#/definitions/user"
    }
  },
  "definitions": {
    "user": {
      "type": "object",
      "required": [
        "id",
        "name",
        "email",
        "is_good_surfer",
        "updated_at",
        "created_at"
      ],
      "properties": {
        "id": {
          "type": "integer"
        },
        "name": {
          "type": "string"
        },
        "email": {
          "type": "string"
        },
        "is_good_surfer": {
          "type": "boolean"
        },
        "updated_at": {
          "type": "string"
        },
        "created_at": {
          "type": "string"
        }
      }
    }
  }
}

And here is an example of how we can use “$ref”: “user.json#/definitions/user” inside our app/controllers/chemas/users.json (just think of this as Rails view partial and you’ll get it 😎):

// app/controllers/chemas/users.json
{
  "type": "object",
  "required": ["users"],
  "properties": {
    "users": {
      "items": {
        "$ref": "user.json#/definitions/user"
      },
      "type": "array"
    }
  }
}

When we run our test now, it passes. Once again, the day is saved thanks to JSON schema. 😂

Homework for the curious reader

If this post doesn’t completely satisfy your curiosity, I recommend you seek out the following topics:

How to write a generic JSON schema test and apply it to all API endpoints; and
How to expose your JSON schema so you can share/use it either at Rails project or other repos.

References

And if you’re still not satisfied, check out these references. Cheers!

by Wayne Chu