Five Elegant Brim Queries to Threat Hunt in Zeek Logs and Packet Captures

Threat Hunting is challenging — there’s an adversary trying to hide after all — so any tool that can speed up your time to insight should be in a hunter’s tool chest. And while advanced analytics, anomaly detection, machine learning and similar emerging approaches are without doubt powerful and have helped reduce the attacker’s advantage, there are times you just have to delve into the raw data. If you’ve worked in the trenches of advanced threat defense you know that at some point you’re going to have to get down in that trench, get muddy boots and use the digital bayonet — Seek and Destroy, or in our case, Search and Query.

Here at Brim Security, we’re working on something we believe gives threat hunters an edge — and cures the security pyramid of pain from the ground up. For example, we can take a really large raw packet capture and present it in the form of Zeek streams, maintaining Zeek’s rich data types and pairing it with a query language that’s suited to both analytics and sophisticated searching.

But before we get too caught up in technical details (which you can read more about over at https://github.com/brimsec/brim), what I really wanted to show you is how this all helps you to be more effective at threat hunting.

Minimal yet Elegant

This article is inspired by the great blog Russ McRee recently posted about using Brim and the Mordor Datasets to learn threat hunting. It got me thinking about what I would look at first if I initially knew very little about the data I am investigating.

I am a fan of minimal yet effective solutions. Anything that is short, quick, yet effective and elegant really gets me excited. Extending this to threat hunting, my ideal is to be able to form a hypothesis as quickly as possible, and then to validate the hypothesis in as few steps, tasks and time as feasible.

TIP! The Mordor Project provides pre recorded adversarial traffic. It’s a really useful repository of practise data for threat hunting, so if you haven’t checked it out yet, you can find it at https://mordordatasets.com/introduction.html

This brings me to ZQL, the search and query analytics language that drives Brim, and some of my favourite queries I use to start threat hunting. This selection of five power queries provides a quick overview of what’s in the data to allow me to quickly pivot into areas of interest.

TIP! You can find detailed installation instructions for Brim on Windows, Linux and macOS under https://github.com/brimsec/brim/wiki/Installation

Getting the lay of the land

The first thing I like to get a feel for is what my data contains, for example the types of traffic or protocols. Brim automatically creates Zeek streams from packet captures, but it also converts all ingested data into its ZNG data format. The beauty of this is that there are parsers that already label the logs with the correct data fields and types. The streams are also labelled according to categories of interest, for example based on specific protocol or service types such as DNS or HTTP. This means that we can easily get a complete overview of what types of traffic and alerts our data contains to identify suspicious activity and begin forming our hypothesis.

Show all Zeek streams

count() by _path | sort -r

Counts all records by the Zeek streams and sorts them, highest to lowest

Below you can see the Zeek streams of packet captures included in the Mordor Dataset pcaps for APT29 (a.k.a. Cozy Bear4):

Getting the lay of the land, using Zeek streams

TIP! Another quick hack when using Brim is that you can do a one-click pivot from any query into the relevant logs

TIP: One-click Pivot from query to logs

Which Domains are being queried?

Another set of observables I always like to see while forming a hypothesis is the DNS activity — especially if there are any suspicious domain name queries, indicating malware command and control for example. Once again, due to the ZNG format and contextualized Zeek streams, it’s a short trivial query:

Show all DNS queries

_path=dns | count() by query | sort -r

Counts all instances of specific queries in the DNS Zeek stream, and sorts them by the highest volume of requests to the lowest for a specific domain name

Listing all DNS queries in raw network traffic

What’s going on in SMB?

Next let’s take a look at that suspicious SMB and DCE/RPC traffic we noted when we got the lay of the land earlier.

Showing only SMB and DCE/RPC activity

_path=dce_rpc OR _path=smb_mapping OR _path=smb_files

Displays the DCE/RPC, smb_mapping or smb_files Zeek streams.

_path=~smb* OR _path=dce_rpc

A slightly looser version using globbing, showing any Zeek streams beginning with smb Or dce_rpc

SMB and DCE/RPC activity

We immediately see a lot of SMB and related activity, including a bunch of DCE/RPC remote procedure calls. The histogram also shows distinct bursts of activity. Right now we’re still not sure if this is a legitimate administrator going about their business, but there’s a lot going on, including disk access. What’s nice about this particular command is that you can adapt it quite easily to other mixed Zeek event types.

TIP! You can right click on any event to open a menu with functions to pivot with. You’ll especially find “Open Details” valuable. You get a full outline of the corresponding communication, correlated by the connection UID

TIP: Showing details and UID correlation

Are any files being distributed across the network?

We’ve seen that there is some file shenanigans being carried out, so we really want to know which files were transferred between the two clients — many attacks distribute malware and remote access trojans — so we want to see if we can quickly identify anything out of the ordinary.

Showing all files seen on the network

filename!=null

Display the filename field from any log if it is not null

This awesome little query really shows how powerful the combination of a smart data schema and a smart query language can be. We get a nice little list of all the files that were seen on the network. If you get an overwhelming amount of results, don’t forget the little trick we learned earlier for getting the lay of the land and DNS data. If you change the query to “filename!=null | count() by filename | sort -r”, it will count and give an aggregate of seen files and display them in ascending order.

Files seen on the network

Which URIs are being requested?

Annoyingly, none of the files being transferred on the network seem particularly malicious. But then if you recall, we also had that suspicious HTTP traffic, so let’s see if we can find anything that might stick out here.

Showing all http requests

_path=http | count() by uri | sort -r

Counts the amount of requests for a specific URI from the HTTP Zeek stream and sorts them in ascending order

This command is really just a variation of the DNS query we used earlier, except that it searches only in http data and then displays the count of URI requests in ascending order. What sticks out of course, is “python.exe” and the compressed “OfficeSupplies.7z” file.

Top requested URI’s

Conclusion

I hope these 5 short and concise, yet elegant and powerful queries inspire you to go and grab Brim and the Mordor datasets to have a play yourself. I also hope they’ve got you thinking about how you can begin your own threat hunts more effectively.

In the next article, we’ll be extending our toolbox with some more ZQL power queries,but this time to analyze some of the most interesting network activity, for example which ports and services are in use, or which assets talk the most or use the most bandwidth. If you’re already curious, throw this into Brim:

put total_bytes = orig_bytes + resp_bytes | top 10 total_bytes | cut id, orig_bytes, resp_bytes, total_bytes

In the meantime, you can get Brim at our website https://www.brimsecurity.com/ — it’s available for Windows, Mac and Linux, so we’ve got all bases covered, or follow us on twitter @brimsecurity