Live & historical hunting features are available now!
As of today, you’re able to leverage familiar YARA rules to query against PolySwarm incoming sample stream (live hunting) or against PolySwarm’s entire archive of samples* (historical hunting).
This is big news for researchers and incident response (IR) teams that have identified a threat and want to expand their knowledge of the threat by identifying variants of known-malicious samples.
Threat hunting features are exclusively available to Enterprise (and above) customers. Visit https://polyswarm.network today to see the PolySwarm Marketplace in action, then use the 30 Day Trial button to get started on your free Enterprise trial!
*Historical hunts search the last 6 months of samples. Currently, this covers the entire PolySwarm archive.
Threat Hunting 101
So, you found some malware. Now you have more questions:
- When was this malware first seen in the wild?
- How well have AV engines / EDR agents detected this threat over time?
- Are there variants to this malware?
PolySwarm’s threat hunting capabilities provide you with the tools to answer these questions, expanding your knowledge of the threat and helping you identify previously unknown threat variants.
Below is a crash course on using YARA to detect suspicious files as well as running YARA rulesets against PolySwarm’s historical database (historical hunts) and PolySwarm’s live feed (live hunts). Hunting functionality is supported by
polyswarm-api and will be coming to https://polyswarm.network soon!
As an Enterprise customer, you’re allotted:
- 5 historical hunts (YARA ruleset executions) every 30 days. There is no limit to the number of YARA rulesets you can upload.
- 1 live hunt at any given time.
Live & historical hunt features can be used simultaneously. We’re working on enhancements to https://polyswarm.network that will allow you to visually manage your hunts!
Before you deploy your rulesets on the PolySwarm marketplace, we highly recommend that you test your rules locally. Testing locally ensures:
- Your ruleset matches what you intend — and doesn’t match what you don’t intend. Results for any given ruleset are capped at 1000.
- Your rulesets are sufficiently performant*.
*We reserve the right to throttle execution on overly broad or malicious rulesets that consume inordinate compute resources.
YARA installation instructions vary by platform. Please consult YARA’s official documentation for instructions on your platform.
Once installed, run YARA to verify your installation:
$ yara --version 3.10.0
YARA Matching Against the EICAR Test File
Take a look at Airbnb’s YARA ruleset here. Their ruleset contains 2 rules:
eicar_av_test:the textbook definition of EICAR
eicar_substring_test:a looser interpretation of EICAR that simply substring searches the file for the EICAR string
Let’s run the entire ruleset against the canonical EICAR test file and against a file that is definitely not the EICAR test file:
$ wget https://raw.githubusercontent.com/airbnb/binaryalert/master/rules/public/eicar.yara$ wget http://www.eicar.org/download/eicar.com$ yara eicar.yara eicar.com eicar_av_test eicar.com eicar_substring_test eicar.com$ echo "this is not eicar" > not-eicar$ yara eicar.yara not-eicar
YARA will print one line of output per rule matched against the file(s) you specify. In the above, both the
eicar_substring_test in Airbnb’s
eicar.yara match against
eicar.com. Neither match against our
When developing your own YARA rulesets, optimize for precision and speed. You probably want to detect malware variants — and not benign files. There are many articles on optimizing your YARA rulesets for speed — avoid heavy features like loops and regex whenever possible.
PolySwarm Live Hunting
PolySwarm’s “live hunting” feature allows you to install a YARA ruleset that is run against all incoming samples to the public PolySwarm marketplace.
There are only 2 steps:
- Install your ruleset.
- Query against your installed ruleset to retrieve any matched samples.
Installing Your Live Hunt Ruleset
For convenience, set your
POLYSWARM_API_KEY environment variable to match your API key:
Installing a live ruleset is as simple as:
$ polyswarm live install eicar.yara Successfully submitted rules, rule_id: <YOUR_RULE_ID>
Write down your
rule_id- we’ll use this to query for results!
Querying Against Your Ruleset
Gathering results is as simple as querying against the
$ polyswarm live results --rule-id <YOUR_RULE_ID> Found 24 samples in this hunt. Match on rule eicar_substring_test File cabf86b068c066b70e7defead09d252832490bc7fe8a1b14093b7e86560566dc File type: mimetype: application/x-dosexec, extended_info: PE32 executable (GUI) Intel 80386, for MS Windows SHA256: cabf86b068c066b70e7defead09d252832490bc7fe8a1b14093b7e86560566dc SHA1: e56b040814ecb0426b61319d4ac1433b7cb263c4 MD5: 41faf9d72f3b3a588025f2c02011a51b Observed countries: Observed filenames: cabf86b068c066b70e7defead09d252832490bc7fe8a1b14093b7e86560566dc ...
polyswarm-api commands support the
--fmt json option — which will deliver results in a machine-consumable JSON format.
PolySwarm Historical Hunting
PolySwarm’s historical hunting runs your YARA ruleset against the last 6 months of PolySwarm submissions.
Again, only two steps:
- Execute the historical hunt.
- Query the hunt for results.
Executing a Historical Hunt
Starting your historical hunt is as simple as:
$ polyswarm historical start eicar.yara Successfully submitted rules, rule_id: <YOUR_RULE_ID>
Querying for Results
Historical hunting takes time. You’re able to query for results at any time, but results may be incomplete if the hunt is still in progress. Future releases of
polyswarm-api will notify when a hunt is in progress. For now, we recommend waiting at least an hour before considering the hunt complete.
Gathering results is as simple as querying against the
$ polyswarm historical results --rule-id <YOUR_RULE_ID> Found 24 samples in this hunt. Match on rule eicar_substring_test File cabf86b068c066b70e7defead09d252832490bc7fe8a1b14093b7e86560566dc File type: mimetype: application/x-dosexec, extended_info: PE32 executable (GUI) Intel 80386, for MS Windows SHA256: cabf86b068c066b70e7defead09d252832490bc7fe8a1b14093b7e86560566dc SHA1: e56b040814ecb0426b61319d4ac1433b7cb263c4 MD5: 41faf9d72f3b3a588025f2c02011a51b Observed countries: Observed filenames: cabf86b068c066b70e7defead09d252832490bc7fe8a1b14093b7e86560566dc
More Features Soon!
We’re continually rolling out new features to make the PolySwarm marketplace more usable. We’re excited about these new hunting capabilities, but we’re not stopping here. Check back soon for the latest PolySwarm features, juxtaposed with pictures of cute animals.
Happy hunting, ~Paul Makowski