Threat Hunting Basics
What is threat hunting?
If you ask a group of information security professionals for the definition of threat hunting, then you’re likely to get multiple (potentially divisive) responses. There seems to be general agreement among threat detection and incident response (IR) practitioners on what hunting conceptually is, but there is often little discussion of how one performs it and what the outcome should be (and as with most things, as vendors and non-practitioners add their voices into the mix, the concept of hunting becomes muddy).
In the context of this post (and any other personal writing from me on this topic), threat hunting is threat detection that is driven by a person. This concept is analogous to, but also the opposite and complement of, a concept familiar to many who practice in the IR domain: threat detection that is driven by an automated system, such as intrusion detection systems (IDS) or Security Information and Event Management (SIEM) tools. While people are involved in the design and instrumentation of these automated systems, people are not the ones carrying out the act of detecting threats — this is the key difference.
A point that should be addressed is that analysts who conduct hunts often use automation to increase their effectiveness and efficiency — this is different than using an automated system to carry out the act of detecting threats. Examples of using automation in support of threat hunting includes using scripts to retrieve, normalize, and categorize data required for a hunt.
If one needs an explanation of how detection via an automated system differs from threat hunting (you’d be surprised how often this comes up in some places), then the simplest is this: an automated system produces detection results with low mental and physical interactivity from the analyst; threat hunting requires high mental and physical interactivity to produce detection results. High mental interactivity does not refer to the level of skill required to hunt, it refers to how engaged the analyst is with the process.
Here’s the million dollar question: is a machine telling you what to look at or have you chosen what’s most important to look for? If a machine directs you — and it doesn’t matter how stupid or advanced that machine is— then you won’t get any of the benefits of hunting (we’ll get to these benefits later). It’s also worth noting that in this context, a machine that offers you the “choice” of reviewing dashboard A, receiving reports B and C, and categorizing alerts D through Z also gives you none of the benefits of hunting.
Before anyone calls me out for disparaging automated detection systems (please don’t, that’s not what I’m doing), consider that both automated systems and threat hunting are important parts of a mature threat detection program and that one is not necessarily better than the other — in some ways, they’re two sides of the same coin. Automated detection systems are highly valuable and necessary, but they aren’t the same as threat hunting.
Isn’t “threat hunting” just a newer, shinier phrase for “incident investigation?”
The idea that threat hunting and incident investigation are the same is a common misconception and it’s easy to understand why: the line between the two is blurry and the skill gap between them is relatively low (if you can conduct an incident investigation, then you’re just a hop, skip, and jump away from being able to hunt).
Threat hunting isn’t the same as incident investigation because hunting is a pre-investigation activity. Recall the previously mentioned relationship between threat hunting and automated detection — both contribute to an organization’s threat detection program and both are analogous in that their purpose is to identify threats in the network. In the IR cycle, detection must precede investigation — you can’t investigate something you don’t know about.
The output of automated detection systems and hunting are the same: both produce potential investigation candidates. In the case of automated systems, potential investigation candidates typically come in the form of alerts, dashboard panels, or reports which must be triaged by an analyst before an investigation is deemed necessary. In hunting, potential investigative candidates are the results of a hunt (often a series of field-specific or type-specific values) that must be triaged by an analyst before an investigation is deemed necessary.
The takeaway here is that threat hunting is rooted in the threat detection domain, not the incident response (investigation) domain; it is a parallel process that exists alongside automated detection systems. Here is a crude flow chart that illustrates this idea (and hints at how hunting fits into the IR cycle — it’s an input):
Why would I need or want to do this?
I’m not interested in “selling” anyone on threat hunting — it’s up to you to determine if it’s something that’s worthwhile. With that in mind, it’s important to answer the question above by discussing the goals and benefits of hunting. Threat hunting has (at least) two high-level goals:
- Identify attackers operating unseen in a network
- Improve automated threat detection systems
Some may emphasize one of these goals over the other, but both are potential criteria for evaluating the outcome of a hunt. The first goal is fairly self-explanatory and commonly understood: if you do not hunt (or have a third party hunting on your behalf), then you likely rely on automated systems to tell you about malicious activity; if attackers bypass your automated systems (“detection failures”) or you have gaps in coverage (“detection gaps”), then you may not know when attackers are in your network. Hunting is a way to offset risk introduced by these failures and gaps.
The second goal may be less intuitive, but a primary goal of threat hunting (and I happily give credit to David Bianco for often mentioning this) is to identify new ways to detect attackers. This goal is a good example of why automated systems and threat hunting can peacefully coexist in the realm of threat detection: they create a positive feedback loop. If you hunt with the intention to discover new ways to identify attackers and then automate it, then you’re expanding your capabilities to automatically detect attackers; if detection is expanded in automated systems, then this reduces the number of things that need to be hunted for (i.e., you don’t need to hunt for what you can already detect).
This is creates a well-oiled threat detection program: automated systems will continuously improve while the scope of threat hunting can evolve based on the actions (or reactions) of attackers, not on an overabundant number of detection gaps.
A third benefit not directly tied to the two goals is that threat hunting is an opportunity to learn. If you start hunting, then you’ll learn:
- How your network really operates
- What data you need to do your job (detection/IR) effectively
- How to think creatively and abstractly
- Why scripting is a necessary skill in detection/IR
- Emerging attacker tactics, techniques, and procedures (TTPs)
… and probably a lot more.
Threat hunting processes
This post is (intentionally) very heavy on the “what” and “why” of threat hunting. The “how” of threat hunting is a more complex topic, and one that should probably be detailed in pieces that follow each step in the hunting process. Given that hunting is not a universally practiced activity, there are bound to be differences in the hunting process between analysts and teams (definitions on Google are dominated by vendors). Personally, these are the high-level steps I use to carry out a hunt:
- Identify what to hunt for
- Identify and collect data needed to carry out the hunt
- Identify most effective method(s) of processing data
- Apply method(s) to data, iterating based on quality of results
- Triage results for detection and investigation
Once again, it’s worth mentioning that this and any other writing on this topic represent my personal opinions on and experience with threat hunting and threat detection. If there’s interest, then it would be fun to describe each of the steps above in more detail.
