Threat modeling Handbook #1: What is a threat model and why your service needs one
I remember a few years ago when I was newly interested in Application Security “threat modeling” was one of those things everyone was saying must be done, but no one was saying enough about “What” it actually looks like in practice. There were some guides and tutorials, but they usually jumped into a very detailed set of steps and requirements which was very hard to digest for someone new to the field.
A few years later after reviewing and contributing to hundreds of threat models, I can say from experience that there is no one good way to create a useful threat model, and while the steps mentioned in many of the guides are useful for specific situations, they could be useless or even distracting for other ones.
Hence, I want to take a different approach with this “Threat modeling” handbook which will span through this story and the next few ones. Before going into the details of “How” to create a threat model, I will start this story with “What” is a threat model and “Why” your service/application/feature needs one.
I’ve decided to take this approach because in my opinion understanding the problems we are trying to solve with threat modeling ( the “What” and the “Why”) is what allows us to efficiently adapt the “How” to the scope of what is being built in a way that solves these problems most efficiently.
What is a “Threat model”?
A threat model is ANY document that tries to answer the question “What could go wrong?” for a specific service, application, or feature, and then for each identified thing that could go wrong (which we will call a “Threat”), it tries to answer the question “What can we do about it?” (which we will call the “Mitigations”).
It doesn’t matter if your threat model has an architectural diagram or not, it doesn’t matter if it has trust boundaries or not, it doesn’t matter if it has data flow diagrams or not. As long as it answers the question “What could go wrong?” in an efficient way it is a “Threat model”. Of course, all of these things could help identify the things that could go wrong, but it is important to remember that they are tools that should be used only when they are useful.
What makes a good “Threat model”?
Again, there is no one good way to create a “Threat model”, but in my experience, a “Threat model” is most useful if it meets the below requirements which helps make it actionable, and possible to attach to the SDLC process and the CI/CD pipeline.
- “Threats” should be tied to the High-level “Risks” for your organization. Example of a High level “Risk”: Exposure of our customers’ PII data.
- Unlike High-level “Risks”, “Threats” are well-defined scenarios that could lead to the “Risk(s)” it is related to. Example of a “Threat”: An authenticated attacker is able to expose other customers’ PII data through SQL injection affecting input parameters to our API.
- Each “Threat” should have one or more “Mitigations”. Example of a Mitigation: Our code uses parametrized SQL statements for all queries to prevent SQL injection.
- Each “Mitigation” should have one or more “Verification” which documents how this mitigation could be verified. Example of “Verification”: Performed code review for all SQL statements, or SQL injection tested in a pentest, .. etc.
- Each “Mitigation” should have one or more “Continuous Test” (e.g. unit or integration test) whenever relevant. Example of “Continuous Test”: Using SAST tool X for every PR which covers SQL injection, and blocks the pipeline for SQL injection findings.
NOTE: Don’t worry if it not very clear at this point, In the next stories we will go through all of these things in much more details and with some examples.
Why you need a “Threat model”: What is beyond the happy path?
One of the main reasons comes down to our human nature, when building or creating something our brains tend to focus on “how things should work” (aka the happy path), but of course, in real life things don’t always work as they “should” and every time they don’t we learn something new when we investigate and understand the cause and fix the issue. In this process, we accumulate experience, and we learn how to make our code and design better to avoid the same issues in the future.
While this “learn from our mistakes” process could work for non-critical bugs and inefficiencies, it is definitely not suitable for security issues. Typically we never want to be in a position where our application has a vulnerability or a breach, and hence we need to train our brains to think about the security issues and misconfigurations that could affect our application before they ever happen. In other words, we want to learn from other people’s mistakes before they become our mistakes.
And that is what “Threat modeling” is about, having some dedicated time to exclusively think about “what could go wrong” instead of “how things should work”. That is why making “Threat modeling” a habit by adding it to the SDLC process trains our brains to foresee security issues we normally miss. I would even argue that being frequently involved in “Threat modeling” sessions makes developers much more aware of security than any security awareness training they go through once or twice a year.
Why you need a “Threat model”: Save time and effort
It could seem that adding “Threat modeling” to the SDLC process adds extra time and effort for each feature, but having seen the time and effort needed to re-design and re-implement features that had security issues (or even worse fix the issues after they are in production) I could honestly say that the value you get from “Threat modeling” vastly outweighs the time and effort spent. And the earlier “Threat modeling” is done, the less time and effort is needed to fix the security issues.
Why you need a “Threat model”: Measure Security
Security of an application or a feature is one of the things that are hard to measure. In many cases, there are no metrics around security, and if there are some metrics they usually are coming from tools like SAST and SCA which are known to have a high rate of false positives, irrelevant or un-exploitable findings, hence they don’t give a true measure of how secure an application is.
“Threat modeling” can help fix this issue as it defines a clear list of “Threats” and “Mitigations”, and by implementing tests covering the “Mitigations” we can continuously verify and measure the security of the application/feature, and we can also track metrics like “coverage of mitigations in test cases” and “number of failed mitigation tests” per application or feature.
Why you need a “Threat model”: Focus on the right priorities
There are many security tools out there today that we can use and add to our CI/CD, however, these tools if not tuned properly can lead to a large number of findings which consumes a lot of time and effort from both security and engineering teams. As mentioned earlier many of these findings could be false positives or at least not properly prioritized, meaning that this time won’t be necessarily spent on the most important issues.
This is why it is a better approach to start with “Threat modeling” and based on the “Threats” and “Mitigations” it generates we decide which security tools we need to use to verify the mitigations and how to tune them (e.g. which tests we want to enable/disable and how should we adjust severities). The other thing to consider is that some mitigations would need custom tests to be verified which is not covered in the out-of-the-box configuration of the security tools. Hence, deciding on the tests and the severity of their findings based on the threat model, significantly increase the likelihood of the alerts we are getting being true and properly prioritized, which reduces alert fatigue and ensures the security and engineering teams are spending time on the right priorities.
Conclusion
As you can see “Threat modeling” if performed efficiently and frequently can help solve a lot of problems before they happen, set the right mindset in the organization around security, and guide other security decisions related to tooling and tests more efficiently. Stay tuned for the next stories where we will dive deeper into “How” to create a “Threat model” and go through some examples.
