Cybersecurity Book Review: How to Measure Anything in Cybersecurity Risk
ACM.1 You cannot manage what you cannot measure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
⚙️ Part of my series on Automating Cybersecurity Metrics. The Code.
🔒 Related Stories: Cybersecurity and Related Books.
💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I’m going to walk you though end to end setup and capturing cybersecurity metrics in an AWS account. However, these principles apply to any environment. Along the way you’ll learn the nuances involved in securing a cloud environment and how to build a proper cloud architecture. In the end I hope to help you generate statistics related to your cloud Configuration.
I’ve spent a lot of time pondering risk and security metrics lately. The end of my last book covered the concept of leveraging security metrics to reduce the overall risk in an organization. As stated in one of the early chapters of How to Measure Anything in Cybersecurity Risk:
You cannot manage what you cannot measure.Hopefully, we can all agree on that.
Comparing and Contrasting Approaches to Cybersecurity Metrics
In my book, I pose twenty meaningful questions boards and executives can ask cybersecurity teams. Those questions were directly correlated with what causes data breaches and increases their impact. I provided a simple example of the measurement of risk within an organization starting with a single metric. That example can be extrapolated to apply the principle to produce metrics using other questions in the book.
My book was an executive level overview of where and how to start thinking about cybersecurity and risk for those new to the topic. The main point was how to measure cybersecurity based on misconfigurations and security problems within the environment.
How to Measure Anything in Cybersecurity Risk takes a look at cybersecurity metrics from another angle — predictions. The book introduces statistics and mathematical formulas to try to drill down into a highly analytical and quantitative approach to risk metrics that tries to quantify the chance that an organization will have a data breach.
Flawed approaches to measuring cybersecurity risk
Let me start right off the top by saying that anyone in cybersecurity should read or listen to this book. The people that listened to it as an audio book complained about hearing Excel formulas read to them, but I was listening more for the overall concepts than the details. If you really want to apply the formulas I’d recommend downloading the associated spreadsheet as even a hardcopy would not be as useful as looking at and using the formulas in the spreadsheet.
That being said, the formulas were not the most important points of the book for me, personally. I don’t foresee myself using them because I tend to focus on other aspects of cybersecurity metrics. I don’t want to predict your chances of having a data breach. I want to help you reduce your risk regardless of the probability of a data breach. If a breach is possible and you can prevent it within a reasonable budget, should you take the steps to mitigate the risk?
My approach has always been to focus on risk reduction through reducing the things that can cause data breaches or increase their impact. Alternatively, you may find it useful to calculate the statistical possibilities that your organization will or will not have one. Some people may find these methods helpful in obtaining funding from executives for cybersecurity initiatives. That particular calculation may help drive decisions about how much money an executive wants to spend. I take a slightly different approach to the problem but either approach is useful.
Whether or not you want to calculate the probability that you will have a data breach, here’s why you should read this book:
This book explains why qualitative metrics like RISK = LIKELIHOOD X IMPACT and “HIGH, MEDIUM, and LOW” are flawed. That includes the calculations provided on the OWASP website. As the book explains some of those calculations equate to oranges + bananas X horses / cats — things on which you should not be performing mathematical equations.
In a recent IANS Research presentation at a CISO Roundtable I presented an alternate formula to the one that always tends to be used (Risk = Likelihood X Impact). From my perspective, that formula is simply too subjective and doesn’t measure what causes data breaches accurately.
Why you should read this book
Understand the downsides of qualitative methods. Although this book is driving towards a different metric than the ones I’m generally after to help companies reduce risk, it clearly articulates flaws with the traditional approach. Qualitative methods are a biased shot in the dark. There is no evidence that this method is working. The technical reasons as to why this is true is covered in the book.
Your estimates might not be as accurate as you think. This book spends an almost inordinate amount of time explaining why and how your current estimations about data breaches may be wrong. Some people may find that drilling into how wrong people’s estimates are to be overboard but you should understand whether you can trust your estimates and how to improve calibration with reality if you are in the business of predicting cybersecurity risk.
Learn tools and methods to apply statistical analysis, probabilities, and actuarial methods to cybersecurity. I have some thoughts on using these methods versus others. For example, if you use a Monte Carlo simulation, the output is relative to the accuracy of your inputs that create the simulation. But in any case, it’s a good idea to understand your options when it comes to predicting cybersecurity risk. Then decide whether or not they meet your needs.
Understand the percentage of uncertainty. One really interesting concept in this book is the percentage of uncertainty and trying to reduce it. You may find it useful to be able to articulate the amount of uncertainty in your analysis. Percentage of uncertainty is interesting because you can never really be wrong. You can always say, “Well, I told you there was still a 10% chance we could have a data breach and I guess we fell into that 10% chance.”
One of the issues I see with percentages of uncertainty is that if you tell your CEO or the board you are 75% uncertain they may get frustrated with that answer and think that you are not good at your job even if you are right. Meanwhile, the person who overestimates their ability to measure risk will be perceived as more competent — even if they are wrong.
The irony of the percentage of uncertainty in this book is that this percentage of uncertainty is itself uncertain. The value is derived from asking security professionals for estimates. A more definitively accurate quantitative approach may not be possible without additional data (an objective I really think we should be driving towards in the cybersecurity industry). Absent the data, these statistical and probabilistic methods to come up with predictions that may be more useful than pure guesses that make up much of the qualitative methods in use currently.
You may be using the term “statistically significant” incorrectly. If you have used that term recently you may want to review the actual meaning in this book. This book introduces a number of statistical terminologies and methods which you should know and be able to discuss, if they arise.
Underlying data improves your estimates. What I found especially interesting was a scenario presented in the book where a security analyst or CISO presents to a CEO a very specific number equating to the risk of a data breach — reserving the right to change that estimation based on the results of an upcoming cloud penetration test — which is exactly what 2nd Sight Lab does. The data provided by that penetration test helps drive better and more accurate estimates and improved decision making.
That last scenario drives home my thoughts on security metrics — organizations can reduce cybersecurity risk by gathering the appropriate data that drives decisions focused on reducing data breaches. Whether you analyze the data using the methods in this book or some other methods I’m working on, you need to start by gathering and tracking the data related to your security configurations and vulnerabilities. This book touches on but does not go into too much detail on that point.
Overall, I think this book is highly relevant to changes that need to take place in the cybersecurity industry to help us better understand and reduce risk. The metrics described in this book and others can help organizations prioritize cybersecurity efforts more effectively with the objective of reducing data breaches and their impact.
Although I’m a huge proponent of cybersecurity metrics, I pose the following question: Should be be using statistics to make cybersecurity risk decisions? I’ll ponder that question in the next post.
Follow for updates.
Teri Radichel | © 2nd Sight Lab 2022
About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight LabNeed Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for PresentationFollow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab
