Real-World Threat Modelling

Mike Goodwin, VP product security and architecture and Technical Fellow, Sage

5 Pragmatic Tips from Someone Who Has Experienced the Pain and the Pleasure of Threat Modelling

First the basics: What is application threat modelling?

Application threat modelling is a structured approach to help application builders find ways that an adversary might try to attack an application and then design ways to mitigate those attacks. The description of an application’s threat model is identified as one of the criteria for the Linux CII Best Practises Silver badge.

Defence-in-depth has been a key principle of security for a long time, but although most people have an intuitive understanding of this (usually based on an egg metaphor), it can be hard to put into practise when building an application. After years and sleepless nights, worrying and fretting about application security, I have found application threat modelling to be an exceptionally powerful technique for building defence-in-depth into applications. It is also excellent for finding structural security weaknesses, which are often subtle and hard to detect using offensive security approaches, especially if the flaws are deep inside your application.

There are several ways of doing threat modelling ranging from formal methodologies with nice acronyms (e.g. PASTA) through card games (e.g. OWASP Cornucopia) to informal whiteboard sessions. Generally, the technique has three stages:

1. Decomposing your application into its component parts, usually using a diagram.

2. Asking questions about the decomposed application to try to guess how someone might attack it — these are the threats.

3. Designing ways to block, avoid or minimise the threats — these are called mitigations.

This sounds easy, right? Three simple steps. But I have found that teams can struggle to be effective and efficient when doing threat modelling.

· They feel overwhelmed by the complexity of their models or of the methodologies they are using

· They often waste time going down rabbit holes that add no real value

· They find it hard to think of mitigations that they can confidently implement

If you want to do threat modelling but are finding it hard, the rest of this article tries to give you some simple and pragmatic tips that can help you make progress, harden your applications and gain experience and confidence in threat modelling.

Tip #1: Focus on trust boundaries

If your application is large or complex it can be hard to know where to start. Or, if you have limited time available to do threat modelling (and who doesn’t?) it is important to focus your attention on the areas that will give you the most value.

In my experience, trust boundaries are the richest source of good quality threats and mitigations, so they are a good place to start.

The first step is to get the boundaries in the right place. The most common problem I have seen is missing trust boundaries — that is, teams assume trust where they shouldn’t.

If you have two elements that communicate with each other, but that don’t have a trust boundary between them ask yourself:

How can I justify that trust? How is trust established?

Once you’re happy with the boundaries, start with the cross-boundary communications and data flow and the model elements on either end of the flow. Some good questions to start with are:

· How is the flow authenticated?

· What is the authorisation logic?

· What validation is needed?

· Are there any opportunities for DoS/DDoS?

· Are there any threats from automation?

And remember that trust is a two-way thing — you should pay attention to responses as well as requests.

Tip #2: Don’t worry about model grammar

Like a lot of security-related processes, threat modelling has its fair share of acronyms, methodologies and taxonomies — STRIDE, DREAD, PASTA, CWE ENISA Threat Taxonomy and more. I have seen teams wasting time and energy trying to conform to the detailed specifications of these different approaches. If you are in that situation, my advice is — RELAX!

· You don’t need to classify a threat accurately or precisely to design a mitigation for it — and its mitigations that count most in the end.

· Similarly, it is more important to know that a threat exists than to attach it the “correct” part of the model.

· A threat model does not have to be 100% complete to be useful — in fact, there is often a law of diminishing returns if you push for 100% completion.

· If part of your model is confusing, then maybe you can just ignore it, generate some useful threats and mitigations on other parts of the model and come back to the hard part later. Note: Some parts of a model are more security-critical than others. Don’t park the most critical parts unless you are sure you will have time come back to them later.

· If it helps you to add extra parts to your model to help people understand it, just go for it. And don’t worry about threat generation for these “contextual helpers”.

Even the most basic threat model built from a whiteboard sketch and some documented threats and mitigations is still useful and a valid exercise. Always remember — these methodologies and taxonomies are there to help and guide you — they are not your master!

Tip #3: Do incremental threat modelling

The earlier you can do threat modelling the better — ideally you would do it before the design of the application is crystallised too much. This is not always possible though and if you hear things like this from your teams:

“We have learned about threat modelling and we are eager to start, but we are in sprint 10 of the project. There is no time in the current sprint to model everything we have done in sprints 1 to 9.”

Or

“Our team is working on a new feature of a large mature product. We cannot model the whole of the product’s architecture, but we at least want to ensure the new feature does not make it less secure.”

Then you should consider incremental threat modelling. This is a technique where you treat the existing part of the application as a “legacy blob” that exists in the model as a sort of black box. You then model the new components that you are building and tease out just enough of the legacy blob for the model to make sense. I first heard about this technique at the AppSec EU conference in 2017 in an excellent talk by Irene Michlin. It was followed up with a blog post outlining the technique.

Tip #4: Detection can be a mitigation too

Sometimes it is hard to see how to fully prevent a threat. Or maybe you can see how to do it, but it will take more time than you have. Either way, there is a lot of value in raising alerts as a form of partial mitigation. At least this will allow you to take some tactical action in production to block the threat if it is detected (e.g. blocking IP addressed) and over time you can build up automated responses to harden your application.

So, make sure that if your application detects evidence of a threat it is logged. You should do this regardless of whether you can block the threat or not. Examples of this are:

· Exploit attempts of known vulnerabilities that have not been patched

· Failures of digital signature checks

· Invalid inputs

· Unauthorised access attempts

· “Unusual” activity

Obviously, you should also build the right alarms from your logs taking into account the threat severity, and make sure your team have the response playbooks, whether they are automated or manual.

Tip #5: Not all mitigations need to be code

Most of the people I have done threat modelling sessions with have been developers. There have been a good number of QA and operations folk involved too, but developers have been the clear majority. This is fine since developers usually have a really deep understanding of their applications.

Developers are mostly about the code though and as a result, they tend to focus on mitigations that are done in code. I’m not referring to infrastructure as code here, I mean application code like the validation of inputs etc.

Now, there are lots of good mitigations that can be done with code, it’s not the only option. Some threats can be mitigated using other forms of technology and even non-technical measures. For example:

To be most effective, think holistically about your application. It can often help to widen out your sessions to include people from different disciplines.

In conclusion

Application threat modelling can be a very powerful technique for hardening application designs, especially in terms of defence-in-depth, but teams often find it hard to get started. This article sets out five pragmatic tips to help your team make progress with threat modelling.

Read another blog by me on the Three approaches to offensive security and how we choose between them