MITRE released the results for Round 2 of their EDR evaluation scenario, this time emulating APT29. As you might have seen, nearly every vendor associated with the evaluation has issued a press release pronouncing their clear effectiveness and decisive victory over the competition. I want to avoid the marketing fluff and jump right into the data. What follows is an explanation of how I quantified the results, with layers of nuance that I hope will help customers find the right fit for their situation. Rather than provide a one-size fits all scoring methodology, I broke down results with clear lines of separation between human derived detection and machine only. If you’re trying to better understand the market or looking to make a choice for a new EPP tool, what follows should be especially relevant to you. I’ve provided the Github link to code and scoring files at the bottom. …
In my previous post, A Phishing Guide: Lessons Learned on the Journey to Detecting Phishing Domains, I laid out my experience building phishing detection algorithms and the associated challenges. In this post, I want to dig deeper into a specific use case where open source technology can be a force multiplier for any security team. I provide all the requisite information to get started on GitHub if you want to get straight to it, but I will take a deeper dive here to explain the project and the impetus for it.
As we all know, security breaches are becoming all too common from political parties to major health insurers and large technology companies. Industry research suggests that somewhere around 90% of breaches begin with some form of phishing. As I noted in my previous article, phishing encompasses a very broad set of tactics, techniques, and procedures (TTPs). This creates a need for a broad set of solutions to protect the enterprise in an environment that is increasingly less centralized with the explosion of cloud, IoT, and personal devices. Any single point solution, whether it be email or web gateway, will only see a snapshot of the attack surface. And any one organization will only see the attacks which have been directed at them, whether purposeful or opportunistic. …
By now, we have all seen an article claiming artificial intelligence (AI) is the solution to all of our detection problems or an unstoppable automated hacking force. AI has become the buzzword that every security company must include in their marketing material to keep up. So how do you separate strong technology from marketing hype? In this article I’m going to lay out common applications of machine learning (ML) in security and the questions you should ask every vendor to cut through the marketing hype and understand if their technology will be useful to you. I’ll start with a definition of AI to ground the rest of the post, followed by a brief analysis of techniques and what you can do to better vet vendor engagements. …
Welcome to the first post of a series that will chronicle my personal experiences building detection capabilities for some of the largest organizations in the world. When addressing detection capabilities, phishing is often the first subject that comes to mind and is an essential tactic used by proficient adversaries. Hence, the series will start here with A Phishing Guide: Lessons Learned on the Journey to Detecting Phishing Domains.
Here I will highlight my philosophies and document many of the common pitfalls to which the industry has succumb to. Opacity and lack of in-depth knowledge of phishing techniques has led to the general mistrust of advanced analytic vendors and practitioners in today’s market. This article will review phishing’s historically successful approaches and clarify the importance and techniques of building detection models to defend against these malevolent methods. I hope that the following insights are useful for those looking to build a robust detection platform and helpful to those working to figure out what the new “Advanced AI” vendors actually do (often this is actually not AI) and are trying to accomplish in the future. …
About