Testing stateless API with behat

tl;dr https://github.com/gorghoa/ScenarioStateBehatExtension

Introduction

Behat scenarios are all about state. First, you put the system under test (SUT) to a special state through the `Given` steps. Then, you keep manipulating your system through `When` steps (still modfying the SUT state) and, finally, testing the resulting state via the `Then` steps.

When testing with behat a system like a single page app or a stateful website, the resulting state of a scenario is handled by the SUT itself (either by the browser, the php session, etc.).

But, when you are testing a stateless system, chiefly an API, then the resulting state of our steps is, by design, no longer handled by SUT. Leaving us with the responsibility to manage the resulting state in the Context classes.

Solutions

There are some existing solutions to handle the scenario’s state.

The most commonly admitted way to handle this problematic is this one:

Combining the facts that contexts are getting created for each scenario (ie. contexts instances are not shared between scenarios) and they are just plain old php classes. One can easily assign class properties through `$this`, each methods of the context can now access the same property in the Context class. Adding the fact that you can easily inject one Context instance to another (see http://docs.behat.org/en/v2.5/guides/4.context.html#using-subcontexts).

It works pretty well, but I there is some drawbacks that bothering me:

1. It’s kind of verbose, you have to inject contexts, writting setters, getters…
2. There is some coupling here between two contexts, fuzzing their responsibilities
3. You (the developer) **have** to know which Context is handling the piece of state you need
4. Steps methods are not stating their dependencies to a previously fragment of the previous state

The coupling induced with this method, is a kind of malicious. It’s malicious because it starts within the gherkin features files themselves!

Imagine those two scenarios:

scenario: I create a new post
Given an operator authenfied as a superadmin
And the operator creates a new post
Then this new post should be publicly visible
scenario: A new post is cronly imported
Given a cron operator
And the operator creates a new post
Then this new post should be publicly visible

The two important steps here are : “Given an operator authenticated as a superadmin” and “Given a cron operator”.

Both of them are contributing to the state by providing to following steps an operator which should be able to create a new post.

The rest of the steps are exactly the same, hence sharing the same context class method.

If all steps are written within the same context class, no problem. Just assign to a class property `$operator` the previously instantiated operator.

And we imagine easily a Context method like this:

/**
* @When the operator creates a new post
*/
public function createANewPost()
{
$operator = $this->operator;
$postManager->createPost($operator, ‘THIS IS SOME POST CONTENT’);
}

But, if you want to have three specifics context:

- WebContext, whose responsibility is to handle auth in a browser
- CliContext, whose responsibility is to handle auth from a CLI console
- PostContext, whose responsibility is to handle the posts manipulations

Both WebContext et CliContext have to give (probably through a setter) the operator instance to the PostContext.

PostContext have to be injected to WebContext and CliContext, so they can call the PostContext::setOperator() method.

Yeah… well… it works. But it’s quickly become tedious when you need other contexts that need an operator: You have to inject each new context to **both** WebContext and CliContext. Each new context will probably need to implement a kind of OperatorAwareInterface which have a `setOperator(Operator $operator)` method (because you don’t want CliContext and WebContext to know which method to call on each contexts), etc.

Sure you could play by adding a kind of “data” context whose responsibility should be to store state’s data. You know what? We kind of did that for you and bundled it in the form of a behat extension!

Introducing the ScenarioState behat extension

What if steps could, during their executions, provide some of the scenario’s state to a storage service, and then inject fragments of the step to following asking steps, directly as method parameters?

With two coworkers from https://les-tilleuls.coop/: Vincent Chalamon and Hamza Amrouche, we created the ScenarioStateBehatExtension.

Note, if you want more precise instructions about installation and usage, you should see the readme file of the extension.

# Usage

This behat extension will allow scenarios steps to provide and consume what I called “fragments” of the resulting state.

Each scenario get it’s own isolated and unique state.

Let’s say a feature like this:

Feature: Monkey gathering bananas
Scenario: Monkey gives a banana to another monkey
When bonobo takes a banana
And bonobo gives this banana to “gorilla”

See the “**this** banana”? What we want during the second step execution is a reference to the exact banana the bonobo initially took.

This behat extension will help us to propagate the banana refence amongst steps.

# Provide state fragment

You can publish state fragment thanks to the `ScenarioStateInterface::provideStateFragment(string $key, mixed $value)`
method.

/**
* @When bonobo takes a banana
*/
public function takeBanana()
{
$banana = ’Yammy Banana’;
$bonobo = new Bonobo(’Gerard’);
// Here, the banana `Yammy Banana` is shared amongst steps through the key “scenarioBanana”
$this->scenarioState->provideStateFragment(’scenarioBanana’, $banana);
// Here, the bonobo Gerard is shared amongst steps through the key “scenarioBonobo”
$this->scenarioState->provideStateFragment(’scenarioBonobo’, $bonobo);
}

# Consume a state fragments

To consume state fragments provided to the scenario’s state, you must add needed arguments to step’s methods using the `@ScenarioStateArgument` annotation, with the name of the state fragment this step needs.

use Gorghoa\ScenarioStateBehatExtension\Annotation\ScenarioStateArgument;
/**
* @When bonobo gives this banana to :monkey
*
* @ScenarioStateArgument(“scenarioBanana”)
* @ScenarioStateArgument(name=”scenarioBonobo”, argument=”bonobo”)
*
* @param string $monkey
* @param string $scenarioBanana
* @param Bonobo $bonobo
*/
public function giveBananaToGorilla
(
$monkey,
$scenarioBanana,
Bonobo $bonobo
) {
// note: assertEquals are purely fictive functions here
assertEquals($monkey, ’gorilla’);
assertEquals($scenarioBanana, ’Yammy Banana’);
assertEquals($bonobo->getName(), ’Gerard’);
}

Pros

  • Steps’ state dependencies are no longer coupled to specifics contexts
  • Take benefits from the php type hinting for methods arguments
  • Fails quickly
  • Concise

Conclusion

If you read carefully the installation part, maybe you see the `@RC` flag to the `composer require` command.

We are confident that the extension is pretty stable now. What we would like to have before releasing an official v1.0.0 is
more feedback from the community:

  • Does the idea/conception of this behat extension makes sense to you?
  • Does the extension seems ergonomic enough for your daily behat coding?
  • Maybe, have you solved the problem in a more elegant way already?

We would love to hear from you and feel free to contribute to the project: it’s MIT Licensed ;)