Security Automation at BlackHat Europe 2022: Part 1

In part 1 of this double-header, we look at the build and configuration tasks for the conference.

It’s been called one of the most dangerous networks in the world, and there are many good reasons why each Black Hat conference has its own IT infrastructure built from the ground up.

There are training classes, where attendees learn offensive security techniques, from hacking infrastructure to attacking the Linux kernel, exploiting IIoT, and abusing directory services. There is the Arsenal, where researchers demonstrate the latest techniques, as well as briefings from experts in a variety of security domains. Then add hundreds of eager and interested attendees, who are not only learning from the content at the conference, but may have their own tricks to bring to the party too.

Roll Your Own

A dedicated infrastructure that does not rely (as far as is possible) on the venue’s own network and security capabilities is the only feasible way to host this kind of community of keen security professionals. Building an infrastructure per conference means that a multi-disciplined team, from a variety of vendors and backgrounds, must find ways to make the build as streamlined as possible. Automation is key to the approach.

The Black Hat team chose Palo Alto Networks to provide network security for all three of their conferences during 2022, renewing an annual partnership which now spans 6 years. The partnership includes Palo Alto Networks supplying their staff to work in the conference NOCs, configuring and operating several PA-Series hardware next-generation firewalls (NGFWs). In 2022, the partnership expanded to include the use of Cortex XSOAR to automate security operations.

Automating the Build Process

The build happens in a short period of time; the core infrastructure went from cardboard boxes to “live” in just over one day for the Europe 2022 conference. A design including complete segmentation of each conference area (including segmenting each training class, the Arsenal, the exhibiting vendors, the registration area, the NOC itself, and more), requires a lot of IP subnets and VLANs, multiple wireless SSIDs, and several DHCP servers and scopes. Some DHCP scopes require reservations, particularly where infrastructure components require predictable IP addressing, but there are too many of them for configuration of static addressing to be feasible. And change happens; IT security is a fast-paced industry, and we knew from experience that we would be adding, moving or changing the configuration data as the conference progressed.

With a single source for all of that configuration data, and a PAN-OS network security platform with plenty of automation capability, automation was inevitable, the only choice was the flavour!

Step forward Ansible. With its task-based approach, its ability to bring in configuration data from almost any structured source, and a collection of modules for idempotent configuration of PAN-OS, it was the perfect match for the requirements.

All of those segmented subnets needed configuring with IP addresses, as well as security zones. Here you can see some excerpts from a playbook execution, where Ansible observed modifications in the configuration data source, and changes were made to only to the required items, with the rest being of the configuration left in original state:

This is important; the initial configuration would not be the final configuration, so when re-executing Ansible to make incremental changes, we only want to make modifications where they are needed. This approach also speeds up the processing time for changes.

Below you can also see a long (and truncated, for brevity) list of DHCP reservations required for some of the infrastructure components. They are being configured with a single Ansible task; this is a list of MAC addresses and IP address that definitely does not want to be configured by hand!

The PAN-OS next-generation firewalls are the DHCP servers for every subnet, and at scale, such a large quantity of DHCP servers is also something which nobody would want to configure by hand, so again, Ansible did that for us automatically:

Automatically Keeping an Eye on Suspicious Hosts

It is rare that the Black Hat team has to take any action against a conference attendee; the majority of seemingly malicious activity is usually part of the trainings, a demo in the Arsenal, or something else “expected”. Occasionally attendees approach or cross the line of acceptable behaviour, and during those instances and investigations it is very useful to be able to view the historical data across the conference.

User-ID provides a huge benefit when the network should include known and authenticated users, but at Black Hat conferences, that is not the case. There is no authentication past the pre-shared key to join the wireless network, and no tracking of any person that attends the conference. However, we chose to modify the user-to-IP mapping capability of User-ID to become MAC-to-IP mappings. Being the DHCP server, the PAN-OS NGFWs knew the MAC address of each host as it requested an IP address, so we routed that information into the mapping database. This meant we were able to observe a host machine (without any knowledge of the person using it) as it moved throughout the conference. Even if the machine left the network and joined again later (after lunch!?) with a new DHCP IP address, or if the machine moved between different wireless SSIDs and hence different IP subnets.

Should action be required when a machine is exhibiting unacceptable behaviour, one option is to utilise network security controls based on the MAC address of the host, instead of the IP address. These controls would be applicable no matter which network the host moved into.

Part Two

The second part of this double-header will focus on the operations side of the conference infrastructure, as the team (below) move into threat hunting mode. Carry on reading here…