Risk Automation Pyramid

I know what you’re thinking; “not another Test Automation Pyramid article!”. Well, I’m afraid it is. Seb Rose recently wrote an article titled ‘Eviscerating the Test Automation Pyramid’. This article got me thinking about how I use the Test Automation Pyramid as a model to inform my testing.

To illustrate my thinking I’ll begin by sharing the original Test Automation Pyramid, first published by Mike Cohn in his book “Succeeding with Agile” in 2009:

The pyramid is simple. It argues that we should write more automated Unit tests than any other type as they are easier to maintain and provide fast feedback. Automated Service tests exercise a service layer of the application, but avoid the complexity of UI frameworks. UI (or end to end) tests often exercise every layer of the system under test but are notoriously brittle and difficult to maintain.

There have been many debates around the Test Automation Pyramid and its usefulness. A quick search engine query will return multiple articles about the model (there’s a certain irony of me adding yet another one!). Some people find it too limiting and believe it leaves too much unsaid and oversimplifies testing. It doesn’t cover exploratory testing, for example.

I like the Test Automation Pyramid. I like it with the caveat that it is ONLY used to consider which automated tests to write. I don’t believe one model can cover an entire testing strategy, although I’m happy to be proven wrong!

How I Use The Test Automation Pyramid

I wrote a Tweet about how I have used the Test Automation Pyramid in the past. I identify the risks of the functionality being tested and work my way from the bottom of the pyramid (lowest possible layer) to the top (highest possible layer). I ask myself at every layer: “can we mitigate the risk here?”

This is a simple addition to the original model, but it helps us to ensure that we are mitigating the risks we have identified at the lowest layer possible.

Here’s some notes I wrote using an example of a checkout page for an online retailer:

I have quickly identified a number of risks that we may identify when testing a checkout page. Heuristics for identifying risks is not something I’m going to go into here, but it’s definitely an idea for a future blog post.

A big risk to the business, in the above example, is that a customer cannot complete their order/checkout. This would result in a frustrated customer and lost revenue for the business.

Can we mitigate this risk at the Unit layer? Definitely. A number of unit tests can be written with a number of scenarios (hopefully spawned from a Three Amigos session!) to ensure that the component of code will successfully complete the checkout.

Can we mitigate the risk at the Service layer? Again, yes. We will want to ensure that any API and database calls that are required to complete a transaction can be completed successfully.

Can we mitigate the risk at the UI layer? Given our previous tests, we may want to add a test to ensure the “Checkout” button is there and can be selected. We’ve covered the majority of the functionality at the other layers, so an end to end test may not even be necessary. Either way, we’ve reduced the number of testing needing done at this layer as we’ve tested the functionality below.

I would then follow the same process with the other risks we have identified.

Final Words

Again, I want to highlight that the Test Automation Pyramid should not be used as a model for every test that you write. It should make up part of your test strategy, not all of it. It is an extremely useful model for when you’re considering the addition of automated tests, but exploratory testing ideas, for example, should be considered outside of the pyramid.

I welcome feedback on whether you use the Test Automation Pyramid and and how you use it.

TL;DR: Identify the main risks of the system you are writing automated tests for and aim to mitigate these risks at the lowest possible layer.