Kong Plugin Configuration, how to define schema and test it

Red, Green, Blue, TDD loop

On the first article we have bootstrapped a plugin with a schema inspired from other plugins. We will see how to write tests, of course some will fail. We will conduct a deep dive on the schema helpers to improve our schema, and fix the code to have everything green.

First tests

As we have our plugin, time to test it, first thing first, create the directory for tests:

mkdir -p spec/medium-test

We will start by writing some test for the plugin configuration:

vim spec/medium-test/01-schema_spec.lua

If you are interested by only testing the logic, you will find it on the next article Kong Plugin: Easy Functional testing

I won’t describe busted syntax, you can find more info on https://olivinelabs.com/busted/#overview

First lines:

local PLUGIN_NAME = "medium-test"
local schema_def = require("kong.plugins."..PLUGIN_NAME..".schema")
local v = require("spec.helpers").validate_plugin_config_schema

As we want to test our schema first thing require it with

local schema_def = require("kong.plugins."..PLUGIN_NAME..".schema")

And here we are, first test helper usage from spec.helpers we have a validate_plugin_config_schema function.

This helper is documented:

--- Validate a plugin configuration against a plugin schema.
-- @function validate_plugin_config_schema
-- @param config The configuration to validate. This is not the full schema,
-- only the `config` sub-object needs to be passed.
-- @param schema_def The schema definition
-- @return the validated schema, or nil+error

So writing test to validate the configuration is quite easy \o/, time to jump in. First create a describe block, with information, plugin name and schema for example

describe("Plugin: " .. PLUGIN_NAME .. " (schema), ", function()
[....]
end)

and a first very very simple test, an empty configuration

it("minimal conf validates", function()
    assert(v({ }, schema_def))
  end)

Lets check our code with the linter pongo lint, this is the first call to pongo so it will download the docker images, you will get something like

Docker images download and build

Then the lint by itself:

Linter result

No error from the linter. Launch kong with pongo up,once again as it is the first time, pongo needs to download the docker images:

psql & cassandra kong-pong docker images download

A docker ps allows to check what happens, two containers are now running:

• kong-pongo_cassandra_1
• kong-pongo_postgres_1

We are now ready for the first test pongo run

pongo run

It is a success 🎉

Success

After such a success, we have to add some other test:

• a full config test

it("full conf validates", function()
    assert(v({
      header_name = "foo",
      header_allow = { "allow" },
      header_deny= { "deny" },
      mark_header = "x-mark",
      mark_action = "all",
    }, schema_def))
  end)

Some error cases, in a dedicated describe block

• An invalid mark action
• Asking to mark without providing the header name

Invalid mark action

it("mark_action invalid value", function()
      local config = { mark_action = "foo" }
      local ok, err = v(config, schema_def)
      assert.falsy(ok)
      assert.same({
        mark_action = 'expected one of: none, allow, deny, all'
      }, err.config)
    end)

assert.falsy(ok) check that the validation catches the error.
assert.same check the error description

Mark action without header

it("mark_action without mark_header", function()
      local config = { mark_action = "all" }
      local ok, err = v(config, schema_def)
      assert.falsy(ok)
      assert.same({
        mark_header = 'required field missing'
      }, err.config)
    end)

Same logic for this.

The 01-schema_spec.lua is now

test failure

Something should be improved in your schema module, there is no check on the schema coherence, just on the fields itself.

Entity check

Once again, no panic, everything is here to easily validate the configuration. This is not really testing a plugin but more validating the input.

Digging into the source code you can find Schema.entity_checkers

Entity checkers are cross-field validation rules.
An entity checker is implemented as an entry in this table, containing a mandatory field `fn`, the checker function, and an optional field `field_sources`.

In our example we are looking for a conditional validation
https://github.com/Kong/kong/blob/2.3.3/kong/db/schema/init.lua#L667

--- Conditional validation: if the first field passes the given validator,
  -- then run the validator against the second field.
  -- Example:
  -- ```
  -- conditional = { if_field = "policy",
  --                 if_match = { match = "^redis$" },
  --                 then_field = "redis_host",
  --                 then_match = { required = true } }
  -- ```

If we want to mark i.e. if the mark action is not none then the mark header field is mandatory:

entity_checks = {
    { conditional = {
      if_field = "config.mark_action", if_match = { ne = "none" },
      then_field = "config.mark_header", then_match = { required = true },
    } },
  },

Et voilà

Pongo test output

The default output for busted is quite short, if you prefer to have more details you can use gtest pongo run -o gtest

gtest output

TAP:

TAP output

You can use json et junit format to use it into our CICD

Château Beychevelle — France

Entity checker

The entity checker available are:

• at_least_one_of
• conditional_at_least_one_of
• only_one_of
• distinct
• conditional
• custom_entity_check
• mutually_required
• mutually_exclusive_sets

We have seen a specific usage of conditional, let’s deep dive all the others checker and look for other way to use conditional

at_least_one_of

The name is quite explicit, check if at least one of the field is set for example from https://github.com/Kong/kong/blob/2.3.3/kong/plugins/ip-restriction/schema.lua#L35

entity_checks = {
    { at_least_one_of = { "config.allow", "config.deny" }, },
  },

conditional_at_least_one_of

Quite similar with at_least_one_of but with conditional, conditional fields available are if_field, then_at_least_one_of, else_then_at_least_one_of. Only if_field is mandatory. For if_match see Fields validator.

A full example with custom error message from https://github.com/Kong/kong/blob/2.3.3/kong/db/schema/entities/routes_subschemas.lua#L12

only_one_of

Here we are with our example of select of the mark_action field. This is quite explicit, example from acl plugins

https://github.com/Kong/kong/blob/2.3.3/kong/plugins/acl/schema.lua#L37

entity_checks = {
    { only_one_of = { "config.allow", "config.deny" }, },
    { at_least_one_of = { "config.allow", "config.deny" }, },
  },

distinct

When the value cannot be the same, for example https://github.com/Kong/kong/blob/2.3.3/kong/db/schema/entities/upstreams.lua#L231

{ distinct = { "hash_on_header", "hash_fallback_header" }, },

conditional

If one field matchs a test then another test should match a test, see Fields Validator for details about the test.

if_field = "foo",
if_match = { ... },
then_field = "bar",
then_match = { ... },

https://github.com/Kong/kong/blob/2.3.2/kong/plugins/statsd/schema.lua#L93

custom_entity_check

You can define your own function to validate the entity

https://github.com/Kong/kong-plugin-proxy-cache/blob/master/kong/plugins/proxy-cache/schema.lua#L85

mutually_required

When several fields need each other, for example https://github.com/Kong/kong/blob/2.3.2/kong/db/schema/entities/certificates.lua#L26

{ mutually_required = { "cert_alt", "key_alt" } },

mutually_exclusive_sets

Two set should be profited set1 and set2

https://github.com/Kong/kong/blob/master/spec/01-unit/01-db/01-schema/01-schema_spec.lua#L2386

With this config, if a5 is set, a3 shouldn’t be set according to the first rule.
Neither a1 or a2, according to the second one.

Fields validator

Generic:
eq, ne, not_one_of, one_of

type-dependent:
gt, timestamp, uuid, is_regex, between,

Strings:
len_eq, len_min, len_max, starts_with, not_match, match_none, match, match_all, match_any

Arrays:
contains,

Other:
custom_validator, mutually_exclusive_subsets,

We can set the Fields validator or the Entity Checker directy when we declare the field, as we have done with mark_action and one_of

The schema helpers as header name are using this.

Header name is a string with a custom validator:

typedefs.header_name = Schema.define {
  type = "string",
  custom_validator = utils.validate_header_name,
}

Another type that will help us is ip_or_cidr

typedefs.ip_or_cidr = Schema.define {
  type = "string",
  custom_validator = validate_ip_or_cidr,
}

Piraillan, Cap Ferret — France

Next step

We have now all what we need to validate and test our configuration, time to test the logic itself.

Testing the schema is simplified due to the helper, no need to launch a full kong set up, just use the validation function as kong will do it.

The functional testing will need to have a running kong, some upstreams server (or mockup) and client to send queries to kong.

You will find this on the last article of this series.

https://medium.com/manomano-tech/kong-plugin-easy-functional-testing-67949957527b