Building an ATT&CK Sightings Ecosystem

People love that ATT&CK is driven by real data. It’s not a theoretical set of possible things adversaries could do, it’s a compendium of things adversaries have actually done.

At the same time, the data that drives ATT&CK, like all data, is subject to bias. While it’s great for developing the knowledgebase and identifying adversary usage of techniques, it isn’t as well-suited to other common things that people want to do with ATT&CK.

Let’s dig in to some potential biases that inform the reporting in ATT&CK, and then we’ll talk about the things people want to do that these biases make difficult.

• Novelty bias: new and interesting techniques or existing techniques used by new actors get reported, while run of the mill techniques used over and over again do not.
• Visibility bias: organizations publishing intel reports have visibility of certain techniques and not others. For incident responders, responding to a fire in progress vice sifting through the ashes to find out what happened will give a different view as well. Not all techniques can be viewed the same way during or after an incident.
• Producer bias: some organizations publish much more reporting, and the types of customers or visibility they have may not reflect the broader industry/world.
• Victim bias: certain types of victim organizations may be more likely to report (or be reported on) than others.
• Availability bias: techniques that are easily called to mind are likely reported more frequently, as report authors think to include them more often.

We use intel reporting to show fact-of use of techniques in the wild. This isn’t exhaustive since not everything can or does get reported. We also make every effort to get useful public reports indexed into ATT&CK, but it does take some time. As such, ATT&CK is just an approximation of what adversaries are doing today and isn’t ground truth.

This doesn’t mean ATT&CK is flawed. Rather, it means that you have to be careful drawing too many conclusions based just on the data included in ATT&CK. Andy Applebaum talked about this in his post on finding related ATT&CK techniques, and Katie Nickels elaborated when she talked about choosing appropriate information in her post on ATT&CK for cyber threat intelligence. As an example, is a technique referenced many, many times in ATT&CK because it’s actually more common or just because it’s more interesting to read about? Does the fact that a technique has been used many times by state actors mean that you, a retailer facing mostly financial crime, need to worry about it? Those are important distinctions when your objective is to build a defensive security program using ATT&CK.

To address this and provide more data to users of ATT&CK to help them understand how techniques are used, MITRE will be launching a program to collect, aggregate, and publish “sightings” of ATT&CK techniques. We’re starting small with a pilot program involving data collection under NDA. This will help us understand what type of data we can get, how it can (and can’t) be combined or aggregated, and how the submitters want us to share it and credit them. If this type of data sharing is successful and provides useful information, then over time we’ll formalize the program and make it more straightforward to contribute automatically with defined rules for how to contribute and how we’ll use the data.

Building the program

We hope that the ATT&CK Sightings data will help answer some questions that are asked frequently:

• How do I know which techniques to start with first?
• As a company in the finance sector, do the attackers I face use different tactics from those facing retail or healthcare?
• How are attacks trending over time? Are older forms of attacks still in use?

If you’re interested in getting involved in the pilot, take a look at our program description and in particular the data format description. If you have any questions or if you have data to contribute, please reach out to us at attack@mitre.org.

©2019 The MITRE Corporation. ALL RIGHTS RESERVED. Approved for public release. Distribution unlimited 18–03730–6.