Open Sourcing Bro-Sysmon

Published in

Salesforce Engineering

5 min readDec 18, 2018

Our newly open sourced project “Bro-Sysmon” was developed to enable Bro-IDS (Bro) to monitor Windows endpoint activities and was inspired by the Bro-OSQuery project. When analyzing network traffic, JA3 and HASSH fingerprints provide valuable network data points due to the increased usage of encryption. These tools create a fingerprint which changes based on the configuration of encryption libraries, the application itself, the underlying operating system, or evasion techniques. The fingerprints will change due to these nuances.

The question we sought to answer is how to definitively attribute a JA3 fingerprint to the process on a host. Bro-Sysmon focuses on integrating the Windows Sysmon ID 3: Network connection with the SSL/TLS analyzer of Bro to generate logging which contains the process ID, path to executable, and JA3 fingerprint. This work was inspired by Steffan Haas’ Bro-OSQuery which we have used to map JA3 fingerprints to MAC and Linux hosts.

Example output of mapja3.log created by mapJA3-Proc.bro script.

DESKTOP-SHFK4CF 2836 192.168.200.100 53223 35.231.36.130 443 54328bd36c14bd82ddaa0c04b25ed9ad faa88e75a7471aaa07850841e28f87f1 www.usualsusp3cts.com C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe — CN=usualsusp3cts.com CN=Let’s Encrypt Authority X3,O=Let’s Encrypt,C=US

How it works

We start with JSON formatted logs of Sysmon events.
The file is read by a Python script which will establish communication with Bro, parse the JSON logs, generate Bro events, and publish the events to the message bus.
Bro will subscribe to the message bus and raise events.
Bro scripts handle these events. The provided script will log the Sysmon events out to disk. There are also scripts included to perform the mapping of JA3 to process.

The use case

Let’s take a look at the analysis process. In this scenario, an analyst is reviewing newly observed domains on their network. The domain www.usualsusp3cts.com is observed. During the analyst’s research it’s noted that the domain is using a Let’s Encrypt Certificate.

Example logs from ssl.log.

1544648162.452073 CTbIwm1WsTyY0UVwD 192.168.200.100 53256 35.231.36.130 443 TLSv10 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA www.usualsusp3cts.com F — — T Fxm3n21gKDX2gxnK0e,FFzF7E3fy4C2Wa7qM3 (empty) CN=usualsusp3cts.com CN=Let’s Encrypt Authorit X3,O=Let’s Encrypt,C=US — — ok 54328bd36c14bd82ddaa0c04b25ed9ad faa88e75a7471aaa07850841e28f87f11544648495.863132 C39S4l1DLLpknJSOh9 192.168.200.100 53267 35.231.36.130 443 TLSv10 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA www.usualsusp3cts.com F — — T Fi89Ii46H5HLo2tE6d,FP9n7QotVOK9ONPte (empty) CN=usualsusp3cts.com CN=Let’s Encrypt Authorit X3,O=Let’s Encrypt,C=US — — ok 54328bd36c14bd82ddaa0c04b25ed9ad faa88e75a7471aaa07850841e28f87f1

Now that the host is identified, the analyst will need to transition to incident response mode and investigate the host. This may require using an enterprise tool, engaging technical support for remote desktop access or shipping the hard drive to the analyst’s office for investigation. These manual processes can take hours to weeks to complete.

Response time is precious and logs are perishable. If the collection process takes too long then essential logs can be overwritten. During the investigation of the host the analyst comes across the following Sysmon Event. The event details provide the ProcessId and Image which generated the network traffic.

With this information gathered the analyst is able to see what the command and control of the victim looks like.

Source Address: 192.168.200.100
Destination Address: 35.231.36.130
Domain Name: www.usualsusp3cts.com
JA3: 54328bd36c14bd82ddaa0c04b25ed9ad
Process: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Our overworked analyst will have to perform this data collection process for every host that appeared communicating with that destination IP. So how can we shorten the length of time and the manual process of this data collection?

Answering the original question

Recall the original question, how to map JA3 and HASSH fingerprints to applications on hosts? Streaming events from a Windows host and converting the logs to JSON provides closer to real-time capabilities for data collection. Below is the JSON formatted output of the Event Log from the victim host for the process id 3952.

'event_data': {
  'DestinationHostname': '130.36.231.35.bc.googleusercontent.com',
  'DestinationIp': '35.231.36.130',
  'DestinationIsIpv6': 'false',
  'DestinationPort': '443',
  'DestinationPortName': 'https',
  'Image': 'C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe',
  'Initiated': 'true',
  'ProcessGuid': '{9A5530DB-7A19–5C11–0000–00105331C300}',
  'ProcessId': '3952',
  'Protocol': 'tcp',
  'SourceHostname': 'DESKTOP-SHFK4CF',
  'SourceIp': '192.168.200.100',
  'SourceIsIpv6': 'false',
  'SourcePort': '53223',
  'User': 'DESKTOP-SHFK4CF\\Kai',
  'UtcTime': '2018–12–11 04:52:43.748'
}

Now that we are receiving host events and processing them with Bro-Sysmon, Bro is receiving these Sysmon logs and can perform additional logic on them. The concept is to keep track of process identification numbers (PID) with their associated information like image location and network connections details. The Sysmon Event ID number 3 will contain the PID and a network tuple of the source IP address, source port, destination IP address, and destination port. This tuple will be the key for referencing the PID information.

This is the record that gets added to the tableNetConns table. Reference trackNetConns.bro for details.

[192.168.200.100,53223,35.231.36.130,443] = [‘3952’, ‘C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe’]

Now when Bro is instructed it can use the conn_id to lookup the host details of this connection. Custom scripts such as mapJA3_Proc.bro can combine host logs and network logs. This technique can also be used to extend current logs with the hash provided by Sysmon. Below is what the new log looks like, showing the network connection with the PID and file location.

DESKTOP-SHFK4CF 3952 192.168.200.100 53223 35.231.36.130 443 54328bd36c14bd82ddaa0c04b25ed9ad faa88e75a7471aaa07850841e28f87f1 www.usualsusp3cts.com C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe — CN=usualsusp3cts.com CN=Let’s Encrypt Authority X3,O=Let’s Encrypt,C=US

This same process can be leveraged to create mapping of HASSH fingerprints with host process information. The network logs contain additional information like client and server strings which can be added to the logging for context.

Considerations

This is a simple architecture. In larger environments, it is important to choose the appropriate technology to ensure delivery and queueing if necessary. The log transport can be replaced with Kafka if needed to ensure robust transport. The JSON parsing and Bro event generation can be rewritten in C++ or other Broker supported language for speed if necessary.

Be aware of proxies and network address translations (NATs). Both of these will modify the network tuple on the external side. I’ve been working on a technique to combine the JA3 TLS fingerprint with the HTTP User-Agent which is done by monitoring network activity on both sides of the proxy.

Conclusion

Bro-Sysmon aims to extend the powerful network monitoring capabilities of Bro by adding host-level visibility. Many network session flows can be extended or enhanced to provide a comprehensive view of how a host actually generated the traffic. Leveraging network and endpoint data to identify and review suspicious activity is essential for Security Analysts and Detection Engineers to efficiently identify bad actors.

Be sure to check out the GitHub repo. You can contact me via email or @4a7361.

Credits

Bro-Sysmon concept was conceived and developed by Jeff Atkinson (@4a7361). Special thanks goes out to Kevin Thompson (@bfist), @tenzir_company and @0xHosom.