Panther v1.0 Announced: Open Source, Cloud-Native SIEM

Powering secure, scalable, and open source detection and response initiatives for cloud-first organizations

We are excited to announce Panther v1.0 — an open source, cloud-native SIEM for modern security teams! Panther is a powerful alternative to traditional SIEMs like Splunk.

Note: Register for our on-demand webinar to see Panther v1.0 in action!

For years security teams have struggled to deploy and scale traditional SIEMs due to their high overhead, astronomical costs, and lack of flexibility.

Panther offers a modern approach to SIEM — it runs fully on top of cloud-native services and empowers smaller teams to detect, investigate, and remediate threats with fewer resources.

Panther is the culmination of years of experience building security tools at scale for some of the largest tech companies in the world, including StreamAlert at Airbnb and critical internal monitoring systems at Amazon.

With Panther, teams can continuously detect threats with log data, improve cloud security posture, and build a robust data warehouse to power investigations. And unlike products that require control over customer data and extensive knowledge of a domain-specific syntax, Panther is self-hosted and uses Python to enable powerful and flexible detection logic.

A few common use cases include:

• Detecting host-based compromise
• AWS log analysis
• Analyzing network traffic
• Threat hunting with indicators of compromise
• Continuous monitoring of AWS infrastructure

In this post, we’ll discuss Panther’s architecture and walk through a typical attacker scenario to learn how to detect and remediate threats in real-time.

How It Works

Panther leverages a serverless architecture and is built fully on top of cloud-native services offered by AWS such as Lambda, ECS, DynamoDB, S3, Cognito, and more.

Here’s a visual on how Panther fits in your security toolbox:

1. Panther receives security logs from clouds, networks, endpoints, and more
2. Panther also baseline scans cloud infrastructure to understand the state of your world
3. All of this data is received, parsed, analyzed, and saved to the data warehouse
4. Alerts are generated and dispatched to your team
5. Optional remediations are applied to misconfigured infrastructure

This design provides a holistic approach to SIEM, where logs are contextually joined with standardized fields, and infrastructure context can be gained by looking up cloud resource attributes in a single pane.

Example Use Case

To better understand how Panther can be helpful, let’s walk through a typical attacker scenario — SSH credentials are stolen providing access into a production machine. Once the attacker connects to the host, they begin to enumerate access and establish their foothold.

How can we detect, investigate, and remediate these behaviors?

Step 1: Preparation

The first step is to collect the proper data to power detections. In most cloud-focused organizations, this involves a combination of logs across various layers:

• Cloud: AWS CloudTrail, S3 Access, GuardDuty
• Network: VPC Flow, Switches/Firewalls, NIDS
• Endpoint: Osquery, Syslog, Auditd, CrowdStrike
• Application: SSO, Productivity Tools, Sales Applications

For this exercise, let’s assume we are collecting AWS CloudTrail, VPC Flow, and Osquery.

To find the suspicious login, we will write a rule that analyzes osquery data from the logged_in_users table:

$ sudo osqueryi
osquery> SELECT * FROM logged_in_users WHERE type = 'user';
+------+--------+-------+----------------+------------+------+
| type | user   | tty   | host           | time       | pid  |
+------+--------+-------+----------------+------------+------+
| user | ubuntu | pts/0 | 136.24.229.194 | 1584146846 | 9459 |
+------+--------+-------+----------------+------------+------+

The above information provides context on how users are logging into our systems. Using the osquery aws_firehose logger plugin, these results can be sent to S3 and analyzed by Panther.

In the example rule below, let’s ensure users are only logging in from centralized egress points, such as offices or VPNs:

import ipaddress
# Monitor the office IP Network
OFFICE_NETWORK = ipaddress.ip_network('192.0.1.0/24')
def rule(event):
# Only look for new entries
if event['action'] != 'added':
return False
# Make sure we are analyzing the right osquery table
if 'logged_in_users' not in event['name']:
return False
# Check that the host IP is present
host_ip = event['columns'].get('host')
if not host_ip:
return False
# Check that the IP is within the office network
if ipaddress.IPv4Address(host_ip) not in OFFICE_NETWORK.hosts():
return True
return False
# Group logins by user to track lateral movement
def dedup(event):
return event['columns'].get('user')

Panther rules also contain metadata to assist with triage, such as severity, log types, unit tests, runbooks, and more. This rule can be written directly in the Panther UI or uploaded programmatically with a CLI.

Step 2: Detect

After our rules are uploaded, Panther will immediately begin to analyze new logs. When suspicious login activity occurs, we will see the following messages in Slack:

Opening the Panther UI reveals the alert, context, and event details:

From this alert, we know:

• Time of the first event (2020–03–14 00:47:47)
• The IP the attacker used to connect (136.24.229.194)
• The host that was logged into (ip-172–31–84–73)

This is the starting point for our investigation. Using the standardized data fields, we can begin to pivot through all of our data to answer additional questions.

Let’s dive deeper.

Step 3: Investigate

After Panther parses and analyzes logs, it stores them in a data warehouse for long-term storage. During this process, common indicators (IPs, domains, etc) are extracted to allow for fast queries and quick searches across the log corpus.

Using the Panther Athena database, we can query all related logs to this IP:

sql
SELECT DISTINCT p_log_type
FROM "panther_views"."all_logs"
WHERE contains(p_any_ip_addresses, '136.24.229.194')
AND month=3 AND day=14
count  p_log_type
1 AWS.CloudTrail
2 AWS.VPCFlow

From here, you can find all possible instance IDs connected to the timeframe:

SELECT instanceid, COUNT(*) AS login_count
FROM "panther_logs"."aws_vpcflow"
WHERE srcaddr = '136.24.229.194'
AND dstport=22 AND month=3 AND day=14
GROUP BY instanceid
ORDER BY login_count DESC
instanceid   login_count
i-016e2cb69ac58c2d5 1

We can then look at this instance in Panther’s resource search, which provides all attributes and associated along with policy successes and failures that could indicate security lapses.

Looking into the attributes, we find the following:

With this public IP, we can query CloudTrail to find related API calls from this host:

SELECT eventname, COUNT(*) AS event_count
FROM "panther_logs"."aws_cloudtrail"
WHERE sourceipaddress='54.164.105.138'
AND month=3 AND day=14 AND errormessage = ‘’
GROUP BY eventname
ORDER BY event_count DESC

And the commands run with osquery with the private DNS name of ip-172–31–84–73:

SELECT *
FROM "panther_logs"."osquery_differential"
WHERE hostidentifier='ip-172-31-84-73'
AND month=3 AND day>=14
AND name LIKE '%shell_history'
AND decorations['username'] = 'ubuntu'
1 2020-03-12 06:58:06.000 aws sts get-caller-identity
2 2020-03-12 06:58:06.000 aws iam get-role --role-name TestDemoRole
3 2020-03-12 06:58:06.000 aws iam list-users --region us-east-2
4 2020-03-12 06:58:06.000 aws cloudformation describe-trails
5 2020-03-12 06:58:06.000 aws s3 ls --region us-east-1

Step 4: Remediate and Post-Incident

Now that we’ve answered all of our investigation questions, it’s safe to terminate the instance, rotate credentials, and fix ACLs related to the root cause. Panther makes this easy by also offering remediation capabilities.

Navigating to the associated security-group for the instance will show the following policies. As we can see, Panther had detected an ACL failure, and it was not fixed, which was one of the causes of the compromise. Simply clicking REMEDIATE will correct the resource in the affected account. This functionality is generally used during incident response for containment:

Finally, your team can update and push new rules and policies to prevent this from happening again. Your infrastructure will be hardened, and the monitoring cycle will restart.

Recap

In this scenario, we established how Panther can be used to:

• Detect attacks in real-time
• Alert the security team about a potential incident
• Investigate activity across all collected log data
• Improve and inspect cloud security posture

Run Panther!

Panther’s elastic architecture enables terabytes of data per day to be analyzed with low overhead and minimal cost. And best of all, Panther is open source!

Follow our Quick Start Guide to deploy Panther v1.0 today with built-in support for:

• Analyzing all AWS logs such as CloudTrail, VPC Flow, S3 Access, GuardDuty, ALB
• Analyzing logs from popular open source security tools like Osquery, Suricata, OSSEC
• 150+ Built-in detections based on CIS and security best practices
• A Powerful UI to create, update and tune analysis
• Fast queries across your data warehouse to power investigations
• Real-time AWS configuration monitoring
• Alerting support for Slack, PagerDuty, MS Teams, and more

For teams who need maximum performance, premium analysis packs, RBAC, and vendor support, contact our sales team about Panther Enterprise.

And to keep up with the latest updates, join us on Slack, follow us on Twitter, and star our project on Github!

(Originally published at https://blog.runpanther.io on March 25, 2020.

Thanks for reading! Subscribe here to receive a notification whenever we publish a new post.)

Register for our free, on-demand Webinar!

See v1.0 in action and learn how to get started with Panther in our on-demand webinar: Panther 101: Bootstrapping your Cloud SIEM.