Automated security testing using language you already know

Harjit Sandhu
Jul 10, 2019 · 4 min read

When it comes to the conversation of security, there are many tools that can help us to aim for an optimum level of security.

Image courtesy of:

In short, threat modelling is the process of defining the risks in any given system, and then assigning controls given the best result in risk reduction. For more information please see OWASPs resource on the subject.

A challenge many businesses share is around measuring the outputs of threat model sessions and the controls put in place thereafter. One way to do so could be to use documentation in some sort of wiki, which we then collate. However, one concern with this approach is that once controls are put in place, we don’t have assurance to make sure these aren’t regressed.

Breaking this concern down, a security team might want all controls to be tested automatically, measurable and auditable. With this in mind, an opportunity arose to take threat model assurance forward.

In each of the threat modelling sessions we hold, we’re always asking our engineers ‘who’, ‘what’, ‘how’ and ‘why’. This draws parallels with the user stories, and the Gherkin syntax they’re already comfortable with.

This begged the question:

What if we could use Gherkin syntax and create real concrete test cases for real life threat model scenarios?

The process comes in a few stages. Take the following example:

An API that validates user credentials.

☝🏾 That looks an awful lot like a ‘Feature Case’, but we’ll come to that a little later.

Document the threat model

Using your favourite methodology, articulate the threat, and define controls.

The NCSC released a statement in November 2018 claiming that credential stuffing is on the rise: This article goes in to a little more detail about what this is:

Using the research above along with the ‘feature’ we defined, an example of a threat could be that an API that validates user credentials exposed over the internet may be susceptible to a brute force which is a class of credential stuffing.

The threat here, is the action, or the ‘when’ in our gherkin statement.

Controls could vary greatly, but for the purpose of this exercise let’s assume the following:
1 — Account lockout after three failed requests
2 — Filter on registration to stop registration with weak passwords
☝🏾 Those controls look awfully close to gherkins ‘Givens’, but, again, we’ll come to that a little later.

Defining risks as scenarios

Now we have one threat, and two controls (three if we include the case with no control in place). Written out as Gherkin, we may come up with something like the following:

Scenario: Baseline — no controls

Given no controls
When I enumerate known passwords
Then I gain access access a users account

Scenario: Control 1 — account lockout

Given locking account after failed logins
When I enumerate known passwords
Then I don’t gain access access a users account

Scenario: Control 2 — no weak passwords

Given no account creation with weak passwords
When I enumerate known passwords
Then I don’t gain access access a users account

The automation part

Once the engineers implement the specifications behind these scenarios we have failing tests and according to TDD/ATDD methodologies we can now work to turn those failures in to passes.

Once the verification of the controls are in place, we can report the outputs to the security teams in language which they can understand (threats and controls). As an added bonus we also have protection from regression. The below GitHub link shows an example of how this exact scenario could be implemented in real life (using stubs). Feel free to use the git repo as a base for any threat models you create.

The source code is available here:

How does this fit into the test landscape?

Having a large number of teams doing the same thing is not an efficient use of time. Internally we are working on building a repository of these automated scenarios that teams can use for free and can add to, based on secure design patterns available at OWASP or CIS. There will always be certain cases specific to a domain or technology that must be bespoke, or perhaps some test cases that don’t lend themselves to automation. That’s why there’s still a place for penetration and exploratory tests.

In summary

Using the above methodology we can create ‘fail scenarios’ on the threat models for common use cases (web app sql injection/DDoS/OWASP top 10 etc). We can give these out to each of our teams to configure and run against their own applications, and have security testing for free out of the box. Better still, we can gain improved visibility from a central security team to understand the risk landscape.

Some Extra reading:

• The post on ATDD by Raphael Yoshiga :
• OWASP wiki entry on Credential Stuffing :
• Wikipedia entry on Gherkin Syntax:

This blog post was written by Harjit. He loves to code and loves security. He’s always happy to have a discussion, even if it’s not related to software or security.

The ASOS Tech Blog

A collective effort from ASOS's Tech Team, driven and…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store