You’re Doing Approval Criteria Wrong!
Who hasn’t heard that phrase? I wager everyone ever to write stories has been told by someone, they’re doing it wrong.
In my line of work I see Backlogs for many different products, maintained by many different people, employing many different styles. And it’s been eye-opening.
I’d like to share some observations with you. Some concerning best practices, some concerning commonly seen mistakes. And I don’t want to just preach to you, I’d like to make you think about why something is a good or bad practice, understand the consequences of your choices in the matter, and be ready to adapt your workflow to your unique context.
For this 2nd installment, let’s talk about…
Denoting Approval Criteria
There are several kinds of approval criteria. What they all have in common though, is that they build the basic statements a story is evaluated against.
As such, we KNOW that they should all be observable, unambiguous, necessary, independent, and relevant.
Observable
A criterion is observable, when it can be seen by someone. In this regard, it makes sense to frame an approval criterion as a statement of an observer’s experience.
“the door shall be red” vs “the tester sees a red door”
Unambiguous
A criterion is unambiguous when its meaning is clearly defined. To ensure this is the case, we must make sure we check out statements for all the ways they could be misunderstood, and reframe them, or extend them, to clearly state what it is we expect, and what we leave to the best judgement of the team.
“the tester sees a red door” — what door? Where? What kind of red? Unless we give specifics, we must yield the decision to the team.
Necessary
A criterion is necessary only when it drives the value of a story. Can the value of the story be fulfilled without stating this criterion? Then it is not necessary.
Often times we feel like including criteria taken from the planned solution we have prepared, either during refinements, or otherwise. We feel those criteria are necessary to state the result we want, but really, they simply prescribe implementation detail.
How much of the implementation detail is actually necessary to state as approval criteria? — Usually none.
It is a good practice to record decisions on implementation details that have been made beforehand, but one may be careful not to state them as approval criteria. Approval criteria are serious business. They enter into the test catalog and are considered “hard” requirements not in the domain of the team.
“If X is not true, I will reject this story!” — This is what approval criteria mean.
So be mindful of what you might let slide, what is merely your wish or initial vision, and what is really necessary to deliver the value you seek.
Each less approval criterion you set forth makes your value less costly, but not including what you care for makes it more costly. Do invest some thought on this, it’s not trivial.
Independent
A criterion is independent when it can be true or false independently of other criteria.
“the fishbowl should be transparent and lightweight and at least 50cm in diameter” vs “the fishbowl is transparent”, “the fishbowl is lightweight”, “the fishbowl is at least 50cm in diameter”.
Relevant
A criterion is relevant when it adds something not already documented elsewhere. A story should not include redundant information. If e.g. a design document already describes what the screen looks like, what elements are found on it, and how they behave, the story need not make mention of these things as approval criteria. Instead, treat such documentation as a kind of approval criteria and agree with the team on how to treat them.
The same goes for technical details like technology choices, data structure prescriptions etc. These things should not be relevant to the delivery of the stated value of the story, and best left to domain experts as a function of value delivery, rather than a delivery object themselves.
Similarly, anything that exceed the scope of the story, i.e. global requirements, definition of done, and similar conventions, are not relevant to the story context, and should not need to be made into approval criteria for a specific story, but for the product as a whole.
How Should We Do It Then?
We need a way to write down our criteria in a way that is readable, quick to adjust, and clear.
The idea is to describe a sequence of events and their expected results which together make up a closed loop, un uninterrupted, non-branching, piece of user experience.
In this, we shall mention all of the necessary actions and observations from the perspective of an actor.
An actor can be anybody. The end user, the tester, the developer, the server administrator etc. We’ll want to make it clear who does what and who sees what. We’ll also want to make it clear in which order these events are expected to play out.
Enter: The Positive Flow Model
The structure is simple:
WHEN [condition/action]
THEN [state/observation]
We also inject the actor in there.
WHEN [actor] + [condition/action]
THEN [actor] + [state/observation]
WHEN the Customer walks in through the front door
THEN the Customer sees the front desk
AND the Customer sees the security gate
THEN WHEN the Customer approaches the security gate
THEN the Customer sees the security gate light up
AND the Receptionist sees the identity of the Customer
AND the Receptionist sees a prompt to grant or deny entry to the Customer
THEN WHEN the Receptionist grants entry to the Customer
THEN the Customer sees the security gate open
We clearly express the necessary steps, involved actors, objects, and interactions, as well as their sequence.
You may have noted that we do not describe the case where the receptionist denies entry. That is because this would create a branching behavior, a separate case, which we can describe separately.
WHEN the Receptionist denies entry to the Customer
THEN the Customer sees the security gate indicate denial of entry
AND the Customer sees the front desk light up
THEN WHEN the Customer walks up to the front desk
THEN the Customer sees a contact button
THEN WHEN the Customer presses the contact button
THEN WHEN the Customer speaks
THEN the Receptionist hears the Customer speak
Each line represents an observable or executable behavior. Every new observation or execution has its own new line, the sequence is indicated, including simultaneity.
Some of you may recognize this paradigm from Cucumber or Gherkin. The idea is very similar.
But What About The Details?
If we wanted to cover every relevant detail, every annotation, every input and output detail like formatting, data fields, every error case, etc., we’d create a long and difficult to read wall of requirements. It is therefore necessary to complement such a positive flow, describing the expected “happy case” behavior we want, with a set of limitations and additional context and decision guidelines.
This can be done in as simple a manner as to include a “Limitations” section with the approval criteria.
Here we may enumerate a series of more general statements.
- The security gate must be the only way the customer can get past the front desk
- The customer cannot enter the front desk
- The front desk must be clearly identifiable as such
- The customer can only pass the security gate if the receptionist granted them entry
- The security gate remains open for 20 Seconds after entry was granted
- The security gate works both ways, in and out, so does the front desk.
- No functionality is expected outside opening hours.
- We acknowledge the dependency on outside power supply.
These statements are often amended with the feedback from refinement sessions to better understand the scope of the story, its implications, dynamics, and edge case expectations. This exercise forces all involved to contemplate these, identify additional requirements, and decide whether to include them in this story, or treat them in a separate one.
So, What Should, And Should I Not State?
Sometimes we want to, or other people want us to, state things as approval criteria that arguably violate the heuristics set forth above. How should we deal with this?
Let us explore some typical cases:
Engineering Requests
Engineers may already discuss implementation details during refinements or planning meetings and subsequently ask for those things to be documented in the story. This is especially prevalent in teams that use JIRA or similar ticket systems where tickets can hold a lot of text. The temptation to include all sorts of documentation within such tickets is great, and many a Product Owner has fallen prey to it. User Stories are not the same as feature documentation though. Features exist mainly on a solution level, whereas User Stories exist at Implementation or Iteration level.
If you receive requests to please document solution-level information inside a story artifact, document it at solution level and include a reference to it within the story.
This is important because the Solution level documentation may still be negotiable and subject to change, even if the story remains untouched in its approval criteria. Such change should never lead to conflicts between documentations, there cannot be more than one source of truth on these matters.
Design Requests
Designers may want to have the behavior of the graphical user interface included in the approval criteria. Sometimes engineers and designers may not see eye to eye on these matters and one seeks to cement their wishes into the story’s approval criteria. It is generally recommended to not include standard behaviors of the GUI, such as transitions, button states etc. This ought to be documented with the design documentation accompanying screen designs and assets, and should ideally be left to designers to coordinate within the sprint.
Common visual quality standards may need to be agreed on beforehand, with their scope exceeding an individual story, such requirements would be considered “global”, and are therefore discouraged in stories. If necessary, do include a reference to the place these things are documented.
The behavior and structure of the front desk is documented here: [link]
The behavior and structure of the front door is documented here: [link]
The visuals for this solution are documented here: [link]
The technical documentation for this solution can be found here: [link]
Adding a list of references within the context of the story may generally be advisable. Just make sure the links remain current, and do version your documentation so it does not change needlessly during an iteration cycle.
You have other issues with formulating your approval criteria? Post your questions and comments …in the comments. I’ll be sure to take a look and maybe include your input with a future post.
Good read, good night!
