Windows Active Directory data collection with the new SecOps Collection Agent

In the dynamic landscape of cybersecurity, proactive threat detection and response hinges upon efficient log collection and analysis. For organizations relying on Microsoft Windows Active Directory, comprehensive log monitoring is paramount. Traditionally, tools like NXlog have been employed for this purpose (official Google Cloud SecOps documentation for AD data collection with NXLog is availalble at this link). However, as security threats evolve, so too must our strategies.

This article explores a modernized approach to Windows Active Directory data collection and ingestion in Google Cloud’s Security Operation platform (SecOps), leveraging the new SecOps Collection agent (aka BindPlane OTEL Agent).

Architecture

This diagram illustrates the recommended foundational components to collect and forward Microsoft Active Directory context information (users and devices) to Google Security Operations (SecOps) SIEM. Please find below the reference architecture for logs collections.

The following configurations are part of the previous architecture:

• PowerShell script is created and configured on each Microsoft Windows AD server to collect USER_CONTEXT and ASSET_CONTEXT data.
• SecOps Collection Agent is installed on each Microsoft Windows AD server to send data to the Chronicle Forwarder installed on the central Microsoft Windows or Linux server.
• Google Security Operations forwarder is installed on the central Microsoft Windows or Linux server to forward log data to Google Security Operations.

Prerequisites

The following prerequisites should be met for proper ingestion of the Windows Active Directory logs in Google SecOps:

• All systems in the deployment architecture are configured with the UTC time zone.
• The Google Security Operations Parser supports logs from the following Microsoft Windows server versions. Microsoft Windows Server is released with the following editions: Foundation, Essentials, Standard, and Datacenter. The event schema of logs generated by each edition does not differ.
  - Microsoft Windows Server 2019
  - Microsoft Windows Server 2016
  - Microsoft Windows Server 2012
• Google Security Operations parser will parse and normalize data retrieved from the User Context and the Asset Context. It supports logs generated with English language text and is not supported with logs generated in non-English languages.

Windows AD servers configuration

This section deals with Microsoft Windows AD servers configuration for proper AD logs collections and forwarding to Chronicle Forwarder.

PowerShell script setup

First it is recommended to configure all systems with the UTC time zone. Then on each Microsoft Windows Active Directory server, create and configure a PowerShell script to collect log data to an output file. The SecOps collection agent will then read the output file and send data to the Chronicle Forwarder installed on the central Microsoft Windows or Linux server.

Create the PowerShell script replacing the placeholders available in the following example. Change the value of $OUTPUT_FILENAME to the location where the output file should be written. This file will be read by the SecOps Collection agent. Data must be stored in JSON format. Set encoding to utf8. Use the -Filter parameter, rather than the -LDAPFilter parameter when calling the Get-ADUser and Get-ADComputer cmdlets.

# Set the location where the log file will be written
$OUTPUT_FILENAME="C:\chronicle\logs\ad_logs.log"

If (Test-Path -Path $OUTPUT_FILENAME) { Remove-Item -path $OUTPUT_FILENAME -ErrorAction SilentlyContinue}

# USER_CONTEXT: Gets all Active Directory users and their properties.
Get-ADUser -Filter * -properties samAccountName | % { Get-ADUser $_.SamAccountName -properties * | ConvertTo-JSON -compress | Out-File -encoding utf8 $OUTPUT_FILENAME -Append }

# ASSET_CONTEXT: Gets all Active Directory assets and their properties.
Get-ADComputer -Filter * -properties samAccountName | % { Get-ADComputer $_.SamAccountName -properties * | ConvertTo-JSON -compress | Out-File -encoding utf8 $OUTPUT_FILENAME -Append }

Recurrent Task configuration

Create a recurring task that runs the script to fetch and write data to the output file. Open the Task Scheduler application.

Task Scheduler

1. Click Create task.

2. Enter a name and description for the task.

3. Select the “Run with highest privileges” checkbox to make sure all data is retrieved as well as “Run whether user is logged on or not” and “Do not store password” as per the following screen.

Task basic configuration

4. In the Triggers tab, define when you want to repeat the task (it is recommended to run the task on a daily basis).

Task triggers configuration

5. In the Action tab, add a new action and provide the path of the file where the script is stored. Program should be “powershell” and the argument “-File PATH_TO_SCRIPT”

Task actions configuration

SecOps collection agent configuration

Install the SecOps Collection agent on each Microsoft Windows Active Directory server by downloading the latest version of the installer from the following url:

https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi

Run the downloaded MSI file. Keep settings as default, make sure that “Enable OpAMP management” is kept disabled.

SecOps Collection agent installation

Wait until the installation is finalized as per the following screenshot.

SecOps Collection agent installation finalization

By default, the agent configuration is at

C:\Program Files\observIQ OpenTelemetry Collector

SecOps Collection agent configuration file location

Edit the config.yaml file in your favourite editor (remember to run it as Administrator) and replace the content with the following:

receivers:
  filelog/ad:
    include: [ C:\chronicle\logs\*.log ]
    start_at: beginning
    delete_after_read: true
    operators:
      - type: json_parser
processors:
  batch:
exporters:
  chronicleforwarder/windowsad:
    export_type: syslog
    raw_log_field: body
    syslog:
      endpoint: FORWARDER_IP:FORWARDER_PORT
      transport: tcp
service:
  pipelines:
    logs/windowsad:
      receivers:
        - filelog/ad
      processors: [ batch ]
      exporters: [ chronicleforwarder/windowsad ]

Remember to replace FORWARDER_IP and FORWARDER_PORT with the IP address of the centralized Chronicle Forwarder and FORWARDER_PORT with the port exposed by the Chronicle forwarder collector for WINDOWS_AD logs (see next section for a sample chronicle forwarder configuration of WINDOWS_AD collector).

Restart the agent and check the status with the following commands (run them as Administrator).

sc stop observiq-otel-collector
sc start observiq-otel-collector
sc query observiq-otel-collector

SecOps Collection agent logs are available in the C:\Program Files\observIQ OpenTelemetry Collector\log folder.

Central Chronicle Forwarder configuration

Please have a look at the official documentation to install and configure the forwarder on either Linux or Microsoft Windows according to the centralized server/VM operating system. Then configure the system with the UTC time zone.

Configure the Google Security Operations forwarder to send logs to Google Security Operations. Here is a sample Forwarder configuration for sending WINDOWS_AD user and devices context logs to SecOps.

Be sure the port for the syslog collector is the same configured on the SecOps collection agent and that connectivity is guaranteed between each Windows AD server and the chronicle forwarder.

  - syslog:
      common:
        enabled: true
        data_type: WINDOWS_AD
        batch_n_seconds: 10
        batch_n_bytes: 1048576
      tcp_address: 0.0.0.0:10518
      connection_timeout_sec: 60

If everything is working fine, whenever the PowerShell script runs, the following logs should be available on the Chronicle Forwarder:

Chronicle Forwarder logs for ingested data

showing WINDOWS_AD logs has been received via syslog and uploaded successfully. Those logs will be also available on the SecOps instance searching for USER_CONTEXT or ASSET_CONTEXT event types.

Summary

The emergence of a SecOps Collection agent built upon the OpenTelemetry Collector signifies a major advancement in log collection and management for Google Cloud SecOps. This architecture fosters seamless integration with a vast ecosystem of observability tools, providing unparalleled capabilities for our customers. In this article we provided a step-by-step instructions for integrating Active Directory data with Google SecOps leveraging the new SecOps Collection agent.