Using EventHub and elasticsearch to get out Azure ApplicationGateway and WAF logs(1)

Azure ApplicationGateway Configuration

ref: https://learn.microsoft.com/en-us/azure/application-gateway/media/how-application-gateway-works/how-application-gateway-works.png

The following configuration script is from GitHub Copilot

az network application-gateway create --name myAppGateway --resource-group myResourceGroup --location chinaeast --min-capacity 2 --sku WAF_Medium --http-settings-cookie-based-affinity Disabled --frontend-port 80 --http-settings-port 80 --http-settings-protocol Http --routing-rule-type Basic --servers

Add a domain to the application gateway and add listeners for ports 80 and 443

az network application-gateway frontend-port create --gateway-name myAppGateway --name myFrontendPort80 --resource-group myResourceGroup --port 80
az network application-gateway frontend-port create --gateway-name myAppGateway --name myFrontendPort443 --resource-group myResourceGroup --port 443
az network application-gateway frontend-ip create --gateway-name myAppGateway --name myFrontendIP --resource-group myResourceGroup --public-ip-address myPublicIPAddress
az network application-gateway http-listener create --frontend-port myFrontendPort80 --frontend-ip myFrontendIP --name myHTTPListener --resource-group myResourceGroup --host-name myou.cvte.com
az network application-gateway http-listener create --frontend-port myFrontendPort443 --frontend-ip myFrontendIP --name myHTTPSListener --resource-group myResourceGroup --host-name myou.cvte.com --require-server-name-indication true

Add a backend pool to the application gateway and add a backend server

az network application-gateway address-pool create --gateway-name myAppGateway --name myBackendPool --resource-group myResourceGroup
az network application-gateway address-pool address add --gateway-name myAppGateway --pool-name myBackendPool --resource-group myResourceGroup --address

Add an HTTP setting and a path mapping rule to the application gateway

az network application-gateway http-settings create --gateway-name myAppGateway --name myBackendHttpSettings --port 80 --protocol Http --resource-group myResourceGroup --probe myHealthProbe
az network application-gateway probe create --gateway-name myAppGateway --name myHealthProbe --path /healthcheck --protocol Http --resource-group myResourceGroup
az network application-gateway rule create --gateway-name myAppGateway --name myRule --resource-group myResourceGroup --address-pool myBackendPool --http-listener myHTTPListener --http-settings myBackendHttpSettings --rule-type Basic

Add an SSL certificate to the application gateway

az network application-gateway ssl-cert create --gateway-name myAppGateway --name mySSLCert --resource-group myResourceGroup --cert-file /path/to/cert/file --cert-password myPassword

Add an HTTPS setting and a path mapping rule to the application gateway

az network application-gateway http-settings create --gateway-name myAppGateway --name myBackendHttpsSettings --port 443 --protocol Https --resource-group myResourceGroup --probe myHealthProbe --host-name myou.cvte.com --ssl-cert mySSLCert
az network application-gateway rule create --gateway-name myAppGateway --name myHttpsRule --resource-group myResourceGroup --address-pool myBackendPool --http-listener myHTTPSListener --http-settings myBackendHttpsSettings --rule-type Basic

Add a WAF rule to the application gateway

az network application-gateway waf-config set --gateway-name myAppGateway --enabled true --firewall-mode Detection --rule-set-version 3.0 --resource-group myResourceGroup

Configure the logging for the application gateway to store access logs and WAF logs, and send them to an event hub. Also, configure the storage account and event hub to retain logs for 1 day. Note the related permission issues.

az network application-gateway diagnostics set --gateway-name myAppGateway --resource-group myResourceGroup --enabled true --retention-days 1 --log-analytic workspace myLogAnalyticsWorkspace --log-analytics true --event-hub myEventHub --event-hub-rule myEventHubRule --storage-account myStorageAccount

Note:

Application Gateway has specific headers: X-ORIGINAL-HOST and X-AppGW-Trace-Id. These headers can be used for analysis when capturing packets.

Issue with obtaining ota client IP:

1. By default, Azure Application Gateway obtains X-Forwarded-For (XFF) in the format IP:PORT, and the port information needs to be removed using rewrite. The service-side obtains XFF to get the client IP, so it is necessary to obtain the correct X-Real-IP header.
2. The backend server obtains the client IP using X-Real-IP and customizes ota rules based on the client IP. Therefore, it is necessary to obtain the correct X-Real-IP header.
3. Add set_real_ip_from 10.162.10.0/24; to the nginx configuration on the backend server (this subnet is the subnet where Azure Application Gateway is located).

In Azure Portal → Application Gateway → Rewrites → Rewrites Set (Add Action)

Reference document: common-scenarios-for-header-rewrite