Automated security testing using language you already know
When it comes to the conversation of security, there are many tools that can help us to aim for an optimum level of security.
In short, threat modelling is the process of defining the risks in any given system, and then assigning controls given the best result in risk reduction. For more information please see OWASPs resource on the subject.
A challenge many businesses share is around measuring the outputs of threat model sessions and the controls put in place thereafter. One way to do so could be to use documentation in some sort of wiki, which we then collate. However, one concern with this approach is that once controls are put in place, we don’t have assurance to make sure these aren’t regressed.
Breaking this concern down, a security team might want all controls to be tested automatically, measurable and auditable. With this in mind, an opportunity arose to take threat model assurance forward.
In each of the threat modelling sessions we hold, we’re always asking our engineers ‘who’, ‘what’, ‘how’ and ‘why’. This draws parallels with the user stories, and the Gherkin syntax they’re already comfortable with.
This begged the question:
What if we could use Gherkin syntax and create real concrete test cases for real life threat model scenarios?
We’ve been working on refining this hypothesis to come up with a solution we’d much like to share.
The process comes in a few stages. Take the following example:
An API that validates user credentials.
☝🏾 That looks an awful lot like a ‘Feature Case’, but we’ll come to that a little later.
Document the threat model
Using your favourite methodology, articulate the threat, and define controls.
The NCSC released a statement in November 2018 claiming that credential stuffing is on the rise: https://www.ncsc.gov.uk/news/use-credential-stuffing-tools. This article goes in to a little more detail about what this is: https://www.wired.com/story/what-is-credential-stuffing/.
Using the research above along with the ‘feature’ we defined, an example of a threat could be that an API that validates user credentials exposed over the internet may be susceptible to a brute force which is a class of credential stuffing.
The threat here, is the action, or the ‘when’ in our gherkin statement.
Controls could vary greatly, but for the purpose of this exercise let’s assume the following:
1 — Account lockout after three failed requests
2 — Filter on registration to stop registration with weak passwords
☝🏾 Those controls look awfully close to gherkins ‘Givens’, but, again, we’ll come to that a little later.
Defining risks as scenarios
Now we have one threat, and two controls (three if we include the case with no control in place). Written out as Gherkin, we may come up with something like the following:
Scenario: Baseline — no controls
Given no controls
When I enumerate known passwords
Then I gain access access a users account
Scenario: Control 1 — account lockout
Given locking account after failed logins
When I enumerate known passwords
Then I don’t gain access access a users account
Scenario: Control 2 — no weak passwords
Given no account creation with weak passwords
When I enumerate known passwords
Then I don’t gain access access a users account
The automation part
Once the engineers implement the specifications behind these scenarios we have failing tests and according to TDD/ATDD methodologies we can now work to turn those failures in to passes.
Once the verification of the controls are in place, we can report the outputs to the security teams in language which they can understand (threats and controls). As an added bonus we also have protection from regression. The below GitHub link shows an example of how this exact scenario could be implemented in real life (using stubs). Feel free to use the git repo as a base for any threat models you create.
The source code is available here: https://github.com/ASOS/threat-model-automation
How does this fit into the test landscape?
Having a large number of teams doing the same thing is not an efficient use of time. Internally we are working on building a repository of these automated scenarios that teams can use for free and can add to, based on secure design patterns available at OWASP or CIS. There will always be certain cases specific to a domain or technology that must be bespoke, or perhaps some test cases that don’t lend themselves to automation. That’s why there’s still a place for penetration and exploratory tests.
In summary
Using the above methodology we can create ‘fail scenarios’ on the threat models for common use cases (web app sql injection/DDoS/OWASP top 10 etc). We can give these out to each of our teams to configure and run against their own applications, and have security testing for free out of the box. Better still, we can gain improved visibility from a central security team to understand the risk landscape.
Some Extra reading:
• The post on ATDD by Raphael Yoshiga : https://medium.com/asos-techblog/atdd-acceptance-test-driven-development-at-asos-81577568e4f2
• OWASP wiki entry on Credential Stuffing : https://www.owasp.org/index.php/Credential_stuffing
• Wikipedia entry on Gherkin Syntax: https://en.wikipedia.org/wiki/Cucumber_(software)#Gherkin_language
This blog post was written by Harjit. He loves to code and loves security. He’s always happy to have a discussion, even if it’s not related to software or security.
