Threat Hunting Using PolySwarm

Paul Makowski
May 22 · 5 min read

Live & historical hunting features are available now!

As of today, you’re able to leverage familiar YARA rules to query against PolySwarm incoming sample stream (live hunting) or against PolySwarm’s entire archive of samples* (historical hunting).

This is big news for researchers and incident response (IR) teams that have identified a threat and want to expand their knowledge of the threat by identifying variants of known-malicious samples.

Threat hunting features are exclusively available to Enterprise (and above) customers. Visit https://polyswarm.network today to see the PolySwarm Marketplace in action, then use the 30 Day Trial button to get started on your free Enterprise trial!

*Historical hunts search the last 6 months of samples. Currently, this covers the entire PolySwarm archive.


Threat Hunting 101

So, you found some malware. Now you have more questions:

  1. When was this malware first seen in the wild?
  2. How well have AV engines / EDR agents detected this threat over time?
  3. Are there variants to this malware?

PolySwarm’s threat hunting capabilities provide you with the tools to answer these questions, expanding your knowledge of the threat and helping you identify previously unknown threat variants.

Below is a crash course on using YARA to detect suspicious files as well as running YARA rulesets against PolySwarm’s historical database (historical hunts) and PolySwarm’s live feed (live hunts). Hunting functionality is supported by polyswarm-api and will be coming to https://polyswarm.network soon!

As an Enterprise customer, you’re allotted:

  • 5 historical hunts (YARA ruleset executions) every 30 days. There is no limit to the number of YARA rulesets you can upload.
  • 1 live hunt at any given time.

Live & historical hunt features can be used simultaneously. We’re working on enhancements to https://polyswarm.network that will allow you to visually manage your hunts!

These are not hard limits— please reach out if you’d like to chat about higher-volume usage and view our account options for more details.

Test Locally

Before you deploy your rulesets on the PolySwarm marketplace, we highly recommend that you test your rules locally. Testing locally ensures:

  1. Your ruleset matches what you intend — and doesn’t match what you don’t intend. Results for any given ruleset are capped at 1000.
  2. Your rulesets are sufficiently performant*.

*We reserve the right to throttle execution on overly broad or malicious rulesets that consume inordinate compute resources.

Install YARA

YARA installation instructions vary by platform. Please consult YARA’s official documentation for instructions on your platform.

Once installed, run YARA to verify your installation:

YARA Matching Against the EICAR Test File

We’ll use Airbnb’s open source library of YARA rulesets to detect the EICAR test file.

Take a look at Airbnb’s YARA ruleset here. Their ruleset contains 2 rules:

  1. eicar_av_test: the textbook definition of EICAR
  2. eicar_substring_test: a looser interpretation of EICAR that simply substring searches the file for the EICAR string

Let’s run the entire ruleset against the canonical EICAR test file and against a file that is definitely not the EICAR test file:

YARA will print one line of output per rule matched against the file(s) you specify. In the above, both the eicar_av_test and eicar_substring_test in Airbnb’s eicar.yara match against eicar.com. Neither match against our not-eicar file.

Protips

When developing your own YARA rulesets, optimize for precision and speed. You probably want to detect malware variants — and not benign files. There are many articles on optimizing your YARA rulesets for speed — avoid heavy features like loops and regex whenever possible.

PolySwarm Live Hunting

PolySwarm’s “live hunting” feature allows you to install a YARA ruleset that is run against all incoming samples to the public PolySwarm marketplace.

There are only 2 steps:

  1. Install your ruleset.
  2. Query against your installed ruleset to retrieve any matched samples.

Okay, we kind of lied: you also need to have polyswarm-api installed. But you already had that installed, right?

Installing Your Live Hunt Ruleset

For convenience, set your POLYSWARM_API_KEY environment variable to match your API key:

Installing a live ruleset is as simple as:

Write down your rule_id- we’ll use this to query for results!

Querying Against Your Ruleset

Gathering results is as simple as querying against the rule_id:

All polyswarm-api commands support the --fmt json option — which will deliver results in a machine-consumable JSON format.

PolySwarm Historical Hunting

PolySwarm’s historical hunting runs your YARA ruleset against the last 6 months of PolySwarm submissions.

Again, only two steps:

  1. Execute the historical hunt.
  2. Query the hunt for results.

Executing a Historical Hunt

Starting your historical hunt is as simple as:

Querying for Results

Historical hunting takes time. You’re able to query for results at any time, but results may be incomplete if the hunt is still in progress. Future releases of polyswarm-api will notify when a hunt is in progress. For now, we recommend waiting at least an hour before considering the hunt complete.

Gathering results is as simple as querying against the rule_id:

More Features Soon!

We’re continually rolling out new features to make the PolySwarm marketplace more usable. We’re excited about these new hunting capabilities, but we’re not stopping here. Check back soon for the latest PolySwarm features, juxtaposed with pictures of cute animals.

Threat hunting features are exclusively available to Enterprise (and above) customers. Contact us today to get started on your free trial!

Happy hunting, ~Paul Makowski

PolySwarm

The world's first crowdsourced threat detection marketplace. Learn more @ https://polyswarm.io

Paul Makowski

Written by

CTO @ PolySwarm

PolySwarm

PolySwarm

The world's first crowdsourced threat detection marketplace. Learn more @ https://polyswarm.io