Image for post
Image for post

Measuring a red team or penetration test.

Quantifying “success” after an “unsuccessful” red team.

Ryan McGeehan
Jan 25, 2018 · 5 min read
  • Our security must be better than expected, because the security firm is amazing.
  • These findings are serious. But, we detected the attack on their first day.
  • These findings aren’t serious. Regardless, I’m worried we didn’t detect it.

How do I capture this information?

The previously mentioned opinions are usually verbally expressed in a briefing after an offensive engagement.

Scope the forecasts tightly with the engagement’s scope.

You should already have a reason for the offensive engagement. Perhaps you need to understand lateral movement from one network to another, or flesh out a certain class of vulnerability nearby a sensitive database, or the quality of your detection mechanisms.

  • The Red Team will be detected.
  • The penetration test will discover an exploitable SQLi.
  • The Red Team will obtain Domain Admin.
  • CERT will discover the “root cause” that began the assessment.

Select and train a diverse group of forecasters.

I discuss this in “Killing Chicken Little”. In general, you want your forecasters to have a little bit of practice, and generally be very intentional when forecasting. A little bit of training goes a long way.

  • 15% Red team will be detected in more than an hour.
  • 35% Red team won’t be detected.

Run your offensive engagement, and repeat a forecast.

If your scenario has a clear and measurable outcome, then your team will be able to anticipate the results and compare their forecasts with reality afterward.

Image for post
Image for post

Conclusion

I have long felt that security teams misunderstand the value of offensive exercises, and it can be hard to capture some of the “softer” areas of value they provide without having some method to measure them.


Starting Up Security

Guides for the growing security team

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store