Blind Leading the Blind: The Rise of Test-Centric Security

Alissa Knight
Jul 24 · 5 min read

When I was interviewed this morning by a journalist on what technologies financial institutions can use to shift the timeline left to detect breaches more quickly, it got me thinking. If we bring our car in to the shop regularly for tuneups and to ensure our security controls, such as the brakes are working as expected for example, then why don’t we test our security controls in our network to make sure they are working as expected as well? Where’s our tune-up for security controls?

Test-Centric Security

Enter test-centric security, a term very much being led by the folks at AttackIQ with their breach and attack simulation (BAS) technology. BAS solutions enable you to build a test-centric security program by testing your detective and preventative technical controls to make sure they work as intended, are not misconfigured, and have full visibility into your network and endpoints. BAS solutions have given us a wake-up call that we should be testing our controls to ensure they are working if many of the breaches over history have not already taught us to be doing.

You keep using that word, I dont think it means what you think it means

Now you may say that a penetration test does the same thing — but as someone who has been performing penetration testing for twenty years, I can tell you that this isn’t always the case. Also, many penetration testers request their IP address be whitelisted so security controls don’t limit the efficacy of the testing. The point of penetration testing is to “capture the flag” otherwise, gain domain administrative privileges or whatever the “crown jewels” happens to be — not to test the efficacy and visibility of security controls like “did our WAF stop all of the SQL injection attacks?” While security controls can be a deterrent during a penetration test, such as an EDR preventing the execution of a reverse shell, it shouldn’t be relied on for ensuring that your security controls aren’t misconfigured or operating as they should be. Take for example the numerous ways you can evade antivirus or disable the service completely using Metasploit.

When you go to sleep at night, do you wonder if someone added a rule to your firewalls that had no change request tied to it? Are you sure every network IDS sensor has all of the home networks specified properly? Is shadow IT on-prem and in your cloud also a concern? Is the cardholder data environment (CDE) really truly isolated from your corporate VLANs making it impossible to pivot to it from the conference rooms? This is where BAS comes in allowing you to design attack scenarios and have the system continuously run them attempting to do those very things.

Reporting Made Simple

The reports also vary differently from traditional vulnerability scanners and what you might see from a penetration test. BAS reports are more actionable, instead of the 90+ page vulnerability scan reports that you don’t know what to do with nor prioritize, BAS reports will typically be 1–2 pages and contain a prioritized list of what vulnerabilities were used to successfully execute the attack scenario.

Honey, Not Everything That Sparkles is Gold

So you’ve invested millions of dollars into your security controls and you’re not sure if they are configured properly or that they can see everything? Take this for what you will but — if it were my millions? I’d make buying a BAS solution to test them a de facto purchase along with my investment into those security controls just like I take my car in regularly for tuneups.

Analyst Outlook

I am in the process of writing several research papers on AttackIQ and the Breach and Attack Simulation product space at Aite Group. Feel free to read these upcoming reports or the previous report I wrote on XM Cyber at

So what’s your opinion? Do you use BAS solutions to test your security controls? Do you think it should be the de facto purchase along with any security control? Leave your comments in the section below!

Like & Share

As usual, if you liked this article, please support me by clicking LIKE and share it with your own feed! This is the best possible way that you can support me and my continued research. If anyone has anything to add or comment on in this article, please feel free to share it with everyone below in the comments section! Learn more about me at my homepage at, LinkedIn, watch my VLOGs on my YouTube channel, listen to my weekly podcast episodes, or follow me on Twitter @alissaknight.

About Me

I am a senior analyst with Aite Group where I perform focused research into cybersecurity issues impacting the financial services, healthcare, and fintech industries through the assessment of sector trends, the creation of segment taxonomies, market sizing, preparation of forecasts, and developing industry models. I provide these industries with a combination of syndicated and bespoke market research, competitive intelligence, and consulting services in the cybersecurity market through unbiased, objective and accurate research and content development. Out of my research into the contemporary cybersecurity issues affecting these industries today, I produce research reports and white papers, as well as provide advisory services that include inquiries, briefings, consulting projects, and presentations on study findings as well as bespoke speaking engagements where I often keynote at cybersecurity conferences, seminars, and roundtables annually.

Alissa Knight

Written by

Senior Analyst, Aite Group | Chairperson, Brier & Thorn Board of Directors

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade