Endpoint detection superpowers on the cheap, Threat Hunting app
One of the reasons for me to create this app was that the endpoint is an often used entry way into a network. There are quite some Endpoint Detection & Remediation (EDR) solutions out there and most of them are quite good, however they can be costly and not everyone is able to afford that (yet).
I wanted to create an alternative approach to the detection aspect, using the MITRE ATT&CK framework and work with the existing environment. It allows you to leverage your existing data platform, in this case Splunk.
This is not a magic bullet.
It will require tuning and real investigative work to be truly effective in your environment.
That being said it's also a great opportunity to learn a lot about your environment and truly know what is running, therefore what is normal and what isn’t.
Currently my app is primarily based on Sysmon data. Sysmon is a free, powerful host-level tracing tool, developed by a small but epic team of Sysinternals /Microsoft employees. Sysmon is using a device driver and a service that is running in the background and loads very early in the boot process. This service also allows you to configure what is being logged.
I've created a modular configuration repository which I've blogged about here; MITRE ATT&CK, Sysmon and my modular configuration
Goal
I wanted to create an app that would assist in creating an investigative workflow approach for Threat Hunters.
Work with ML (Mandatory Learning) to truly get to know your environment.
Supply the user with tools to contextualise and investigate these events
And base it on MITRE ATT&CK because its a great reference point.
At the time of writing I've created 125 reports that look for 117 of the ATT&CK techniques. I cover what I can but please note every technique has multiple variations, I'll keep updating this whenever I encounter those.
All these reports write their results to a summary index which is the main source of the app. Currently the ATT&CK distribution looks like this;
Challenges
In creating the app I ran into several challenges, a lot of them are due to the fact I do not have a development background but some of the noteworthy ones were;
- Splunk didn't have a datamodel which I liked, so created one based on OSSEM a great model by Roberto Rodriguez
- Inventing a whitelist capability on 120+ searches took me quite some time and rebuilds to get it right
- Rebuilt the app over 4 times to address things I discovered didn’t work well, let’s call this Agile
- I have not discovered a way to generate a performant and reliable process tree yet
- I still want more selective filtering
Setting up
Download it here.
Before you'll be able to use the app you need to install some required apps, create the threathunting index and adjust the macros to suit your indexes. You'll need to install the following apps;
You also need to create all of the whitelists, I deliberately do not ship this in the app to avoid overwrites on future upgrades. They can be downloaded here.
The App
Opening the app you'll land on the overview page, this will provide you with a count of all triggers per ATT&CK category over the past 24Hrs as well as the top triggered techniques and the most impacted hosts.
Everything is clickable in the app and drilling down into for instance Credential Access will show the following overview.
ATT&CK Overview
This page is dynamically populated with all Sysmon event types that triggered indicators through the reports. In this case there were triggers on Process Creation, Process Access and Image Load event types. Most fields in the table have a drilldown action connected to it an overview of all fields is below;
I plan to add more drilldown actions over time, please tell me your ideas as well!
GUID drilldowns
The GUIDs are very powerful, these are unique IDs for a process which allows you to reliably track processes and look for its parents and children. Using the ProcessID (pid) is less reliable since they are reused quite quickly on an active system so this might mess up your investigation. Clicking these GUIDs will take you to a focussed dashboard showing you it's parent and 1 level of children. (I am still to find a proper way to generate the whole tree in a performance acceptable manner)
Regardless this page is a great overview of events in many cases. The left graph will show which processes are tied to that GUID, this should be only 1. In case of certain process injection techniques there might be a second one, which you'd want to check out. The Sankey graph on the right will show you a small process tree. It will look for 1 parent and 1 child of the used GUID. In this case we see PowerShell being launched. On top of that there many childs spawned from the PowerShell process which are visible in the Process Create section and other event types that would have triggered, similar to the ATT&CK overview this page will be dynamically populated with events, in this case relevant to the Parent Process GUID.
Clicking these processes will take you to a similar page but then based on the Process GUID.
Computer Drilldown
Clicking the host_fqdn / computer name will direct you to the Computer Drilldown. This dashboard has a punchcard timeline at the top.
This will provide you with a nice overview of all triggers on that machine over the (by default) last 24 hours. As you might have spotted the top line on commonly used ports shows a distinct, repetitive, pattern, which at 4 and 6 PM changes and at that same time there is a lot more triggers firing. This would be a good reason to dig deeper into these timeframes as this might indicate beaconing towards a C&C server. As with all previously mentioned dashboards this one will also be populated with all event types which were triggered by a report. One of this will be the Network Connection drilldown.
Network Connection Drilldown
Clicking on the source or destination IP will direct you to the Network Connection drilldown and show you all relevant connections to or from that IP address.
The graph on the left will show you the searches entity in the middle and all respective connections around it. The Sankey diagram will in this example be a bit more descriptive since it is showing also the weight, or amount of connections by making these bars thicker.Then a punchcard graph will show the connections to or from the searched IP over the past 24 hours.
Below that is a table containing all relevant triggers and finally all raw events. These events, when unfolded, will provide additional options for getting contextual information by utilizing workflow actions in the form of Shodan, GeoIP, ThreatMiner and so on.
Whitelisting
A crucial feature to this app is the whitelisting option. Since there are quite some reports which have quite generic parameters there will be a lot of trusted behaviour visible, which you, after validation, want to get rid of. I suggest to become good friends with your IT admins since they will be able to explain a lot you'll be seeing and then be able to whitelist.
Any _time field in any table is clickable and will take you to a whitelist editor corresponding to the respective event type. The form will be populated with all relevant fields from the table row, allowing you to edit them to add whitelists. This will be particularly useful in whitelisting for more than one machine or when a command line contains a dynamic value. The only thing that is required is to add a reason. I'd recommend to use a ticketing system to store your investigations and use the ticket number as the reason.
After submitting the event will be added to the lookup list, with the addition of the data and the user which added the whitelist entry for future reference.
Additional helper dashboards
Computer Investigator
This dashboard will provide an overview of all logs ingested for the searched machine on top. Below that it will show a similar timeline depicting the triggered events. This will help you pinpoint interesting timeframes where suspicious things might have been happening.
PowerShell drilldown
This dashboard will search for all PowerShell command lines and will generate tables for Base64 blocks found as well as a table for download or web connections initiated.
When clicking on a base64 block it will guide you towards the excellent CyberChef and automatically try to decode the block. From there you can further investigate that code block to see what has been executed on your machine.
More
There also is a Sysmon dashboard, showing tampering attempts. On top of that also a Lateral Movement dashboard showing various kinds of remote execution attempts by for instance WMI, PsExec and WinRM.
A demo video of the app is available on YouTube
Closing
This post covered deploying and maintaining Sysmon and its configuration, the former part covered;
- Part 1 — MITRE ATT&CK, Sysmon and my modular configuration
- Part 2 — Deployment and maintenance
- Part 3 — Detect Sysmon tampering
next topics will cover;
- How to start interpreting the logging to help you tune your config
- more… :)
