Hunter’s Tool Chest: Sysmon
The first entry in this series was about a network-focused hunting tool, so it seems fair that the second entry is about an endpoint-focused one. Sysmon is a free tool created by Microsoft that is capable of collecting process level metadata— including command line activity, network connections, and file modification.
Collecting process metadata is a capability that security operations teams can no longer afford to ignore— the data is extremely valuable, especially to hunting operations.
Scaling
Endpoint collection tools are inherently more difficult to scale than network collection tools— unlike tapping a network (where you may capture metadata from hundreds or thousands of systems with a single device), you have to scale with the number of endpoints you want to monitor. Given that Windows environments vary from site to site, I won’t suggest the best way for anyone to deploy Sysmon. You’ll need to figure out ways of getting Sysmon to your endpoints and getting Sysmon event logs to an aggregation server. If you’re interested, here are some resources for you to reference:
- Deploying software with Configuration Manager (deployment)
- Event Forwarding and You (event collection)
- Setting up Tiered Event Forwarding (event collection)
EVTX? XML? ?:|
One big issue I have with Sysmon is that it’s a Windows Event log (sorry, Microsoft). To make the log data usable, you have to convert it from EVTX to XML (and then from XML to something else, like CSV or JSON). This is a ‘your mileage may vary’ kind of problem— not insurmountable, but annoying.
In search of TTPs
Gripes with scaling and log format aside, Sysmon can’t be beat as a free Windows tool that allows you to hunt attackers in a surprisingly close way— by monitoring their activity on the command line. Think about any time you re-ran command line tools because you haven’t memorized their options or forgot the correct syntax for a specific parameter— these mistakes are the result of simple human error and attackers are prone to them as well. Even when mistakes aren’t made, chances are good that you — as well as any attacker in your network — have procedures that are followed to complete your work. The procedures can be simple, such as using a particular tool to achieve a specific goal, and they can be complex, like using a particular grep pattern to retrieve lines from a certain kind of document. These mistakes and procedures inevitably create patterns. Given enough data, the patterns emerge; given enough time spent analyzing the data, the patterns become attributable. Process metadata is particularly useful for identifying these types of patterns at a high level of detail. Hunting and identifying these patterns is a valuable method of fingerprinting attackers and, if used properly, can be used to identify them when they return.
Slice and dice
While valuable in its original form, Sysmon’s log data is most useful when it’s broken into individual elements and reassembled with intelligence applied to it. Think about it, a lot can happen in a single process execution— it can be decomposed into multiple parts, including program metadata (e.g., filename, filepath), program options (e.g., switch parameters), results (e.g., saved output, created files), and system interactions (e.g., connections to remote machines). Sysmon does some of this process break down for us (file creation, network connections), but command line decomposition (separating the program from the command switches, for instance) is something we have to do manually.
If we look at the process elements in different arrangements and apply a bit of intelligence to our hunting, then we can do some clever things. Let’s pretend that we know attackers like to bring common tools into our environment under different filenames — rar.exe becomes r.exe, mimikatz.exe becomes m.exe — but we can’t predict what filename each tool will have during any given intrusion. What can we do? One idea is to hunt for any processes that use common program options associated with these tools— specifically, setting a password for a Rar file or checking if Mimikatz has the correct system privilege (‘privilege::debug’). If we do that, then it becomes trivial to identify the tools, even if the attacker renames them every time they bring them into the network. Thinking about process metadata like this allows you to explore many hypotheses, and it even lets you answer interesting questions fairly easily. (One of those questions might be: is it abnormal for PowerShell processes to connect to the Internet? Answer: yes, probably.)
These are simple examples, but they illustrate how much power you gain from collecting this data and post-processing it— just think about all the actions attackers commonly perform in phase 7 of the kill chain (internal recon, privilege escalation, credential dumping, lateral movement, persistence, …) and all the process artifacts they leave behind!
If you made it this far, then I recommend downloading Sysmon and running it on a local Windows machine. If you aren’t into hunting and malware analysis is more your thing, it’s also nice as part of a dynamic analysis system.
