Next-Gen Cloud Network Security: TLS inspection with NGFW Enterprise

Greetings, everyone! Welcome back to the NGFW Enterprise series. Today, we’ll delve into the inner workings of TLS inspection and explore some configuration strategies.

In our previous discussion, we gave you a comprehensive tour of the NGFW Enterprise components and their interconnectedness. However, we deliberately left out one crucial element: TLS inspection. Let’s rectify that now!

In the last episode, we explored the vast landscape of NGFW Enterprise resources. Today, we’ll dive deeper into the specifics, focusing on TLS inspection policies and other key components we haven’t yet covered.

To successfully inspect TLS traffic, there are two crucial steps:

1. Create a TLS inspection policy. This policy acts as the bridge, linking NGFW Enterprise to other GCP services, responsible for the actual inspection process.
2. Activate TLS inspection for specific traffic patterns within your Firewall Policies. This step ensures that the inspection applies only to the traffic you deem necessary.

In the following sections, we will delve into both of these aspects in detail.

TLS Inspection Policies, CAS and Certificate Manager

TLS inspection policies are project-bound, regional resources that enable TLS inspection at your Firewall Endpoints.

The resources that NGFW Enterprise needs to enable TLS inspection functionalities.

They connect NGFW Enterprise to two additional GCP services:

• Certificate Authority Service (CAS): This service empowers NGFW Enterprise to respond to clients making TLS requests by providing them with the necessary certificates.
• Certificate Manager Trust-Configs: This service offers an optional repository for your own (self-signed) certificates within your chain. By storing them here, you ensure that NGFW Enterprise trusts them (we’ll clarify this concept further on in the article).

NGFW Enterprise acts as a “man-in-the-middle”, effectively splitting the clinet-server connection into two segments.

Similar to traditional Network Virtual Appliances (NVAs), NGFW Enterprise acts as a “man-in-the-middle,” effectively splitting the connection into two segments: it intercepts and terminates TLS connections from clients, while simultaneously establishing a new connection to the intended servers. The response NGFW Enterprise (CAS) provides to clients depends on the response it receives from the servers:

• A CAS-signed certificate: If NGFW Enterprise trusts the server’s certificate, it returns a certificate signed by CAS intermediate CA. CAS is backed by Google’s Hardware Security Module (HSM) and generates new intermediate CA certificates every 24 hours.
• A self-signed certificate: If the server’s certificate is invalid, expired, or otherwise not trusted (neither public nor configured within the Certificate Manager trust-config), NGFW Enterprise returns a self-signed certificate.
• “Reset by peer” error: If the Server Name Indication (SNI) sent by the client doesn’t align with any Subject Alternative Name (SAN) present in the server’s certificate, the connection is terminated with a “Reset by peer” error.

By default, TLS inspection policies are configured to inherently trust all public Certificate Authorities (CAs), along with any additional certificate you’ve explicitly defined within your Certificate Manager trust-configs. The trust in public CAs is grounded in the Mozilla Root Program, and as such, the list of trusted CAs may evolve over time. If you wish to deviate from this default behavior, you have the option to configure your TLS inspection policies to disregard public CAs by setting the exclude_public_ca_set argument to true.

Firewall Policies

After configuring your TLS inspection policies, your Firewall Endpoints are ready to inspect TLS traffic. However, you’ll still need to explicitly instruct NGFW Enterprise on which specific traffic to inspect. This is achieved by enabling the tls_inspect (boolean) option within your Firewall Policy rules.

For instance, you may opt to perform TLS inspection exclusively on your ingress traffic, while leaving egress traffic untouched.

Trusting CAS certificates

It’s important to note that your VMs within GCP won’t automatically trust the certificates issued by CAS. You’ll need to take proactive steps to “manually” distribute the CAS CA certificates to your clients. This can be achieved through various methods, such as utilizing GCE startup-scripts or employing third-party tools like Active Directory.

Keep in mind that intermediate CAS CA certificates are regenerated every 24 hours. Therefore, it’s advisable to push the root CAS CA certificate to your clients to ensure consistent trust.

While this process is commonplace when working with traditional NVAs, it often catches people off guard that it doesn’t happen seamlessly within GCP. Hence, it’s worth highlighting this aspect.

Use-cases

In this section, we’ll explore some typical use-cases to gain a deeper understanding of how TLS inspection operates and how to configure it effectively (or, in some scenarios, why you might choose not to!).

Egress traffic

Let’s imagine a scenario where you want to inspect TLS traffic originating from your client VMs within GCP, specifically targeting a TLS-secured destination, such as the Google website.

In this scenario, a client in GCP connects to https://google.com and we enable TLS inspection.

• The client VM initiates a request to https://google.com, which is intercepted and terminated by NGFW Enterprise.
• NGFW Enterprise establishes a new request to https://google.com.
• Google responds with its original, publicly trusted certificate. Assuming you haven’t modified the default configuration of your TLS inspection policy, NGFW Enterprise will inherently trust Google’s public certificate.
• Since Google’s certificate is trusted, NGFW Enterprise will return a certificate signed by its intermediate CA to the client.

Ingress through Application/Proxy Load Balancers

Let’s explore a scenario where you want to expose a GCP service (for instance, a web server hosted on a VM) to the internet. Your goal is to inspect incoming traffic before it reaches the server.

For this initial use-case, we’ll leverage an Application Load Balancer. The load balancer will present the public certificate to clients, proxy the connection, and act as a client in relation to both NGFW Enterprise and the server. We’ll later discuss why this is the preferred deployment method when inspecting TLS traffic.

Now, let’s break down the flow step-by-step:

You want to expose a service in GCP. Some clients connects to it through an Application Load Balancer and you enabled TLS inspection.

• Clients from the Internet connect to the application and land on the external Application Load Balancer (public IP).
• The load balancer terminates the TLS connection and proxies it towards the destination. During load balancer configuration, you have two choices: deliver unencrypted HTTP traffic to the server, or re-encrypt it and send HTTPS traffic, ensuring end-to-end encryption. While there’s no TLS inspection involved in the former scenario, we’ll focus on the latter and observe how NGFW Enterprise handles it.
• As traffic flows from the load balancer to the end destination, NGFW Enterprise intercepts and terminates the connection, mirroring its behavior in the previous egress use-case.
• NGFW Enterprise establishes a new TLS connection to the server, which responds with a certificate. Let’s assume the worst-case scenario: NGFW Enterprise receives a self-signed certificate that you haven’t uploaded to the Certificate Manager trust-config.
• Due to the self-signed certificate, NGFW Enterprise returns a self-signed certificate to the Application Load Balancer.

It’s important to highlight that Google Application Load Balancers don’t validate backend certificates and they consistently return the appropriate “client-facing” certificates to the clients.

The traffic would follow the same path if you used Proxy Load Balancers (both internal and external), for example, to manage non-HTTP(S) TLS traffic.

This approach is the recommended way to expose your services and inspect TLS ingress traffic with NGFW Enterprise.

Ingress through public IPs and Passthrough Load Balancers

You still aim to expose your service to the internet, but this time you opt for a Passthrough Load Balancer. The same flow (and potential problems) would arise if you directly assigned a public IP to your VMs. Let’s analyze the outcome.

You want to expose a service in GCP. Some clients connects to it through a Passthrough Load Balancer and you enabled TLS inspection.

• Clients connect over the Internet and reach the Passthrough Load Balancer (public IP). This load balancer doesn’t proxy the traffic, which is forwarded as-is to the end server.
• NGFW Enterprise intercepts the call, terminates the TLS connection, and establishes a new connection to the end server.
• Let’s assume the best-case scenario: you’ve configured a publicly trusted certificate on the server. The server responds to NGFW Enterprise using this certificate.
• Since NGFW Enterprise trusts the server’s certificate, it returns a CAS-generated certificate to the clients. However, as we’ve discussed, clients, even those within your GCP organization, won’t automatically trust this certificate. Consequently, clients on the Internet won’t trust the destination (unless you manually trusted the root CAS CA certificate).

Therefore, if your goal is to inspect ingress TLS traffic, I strongly advise against exposing your services using Passthrough Load Balancers or by directly assigning public IPs to your VMs.

Combining NGFW Enterprise and HTTP proxies

Many users ask whether it’s possible to use NGFW Enterprise and do URL filtering in egress.

Cloud Secure Web Proxy (SWP) supports the encapsulation of TLS connections towards an end server through a proxy but NGFW Enterprise still doesn’t.

However, you can still combine NGFW Enterprise with explicit proxies like Cloud Secure Web Proxy (SWP) or other third-party solutions. Just ensure you disable TLS inspection (tls_inspect) on your NGFW Enterprise egress Firewall Policy rules.

Terraform Support

The Terraform Google provider is ready to accomodate your NGFW Enterprise deployment. Here is the list of resources:

• network_security_security_profile
• network_security_security_profile_group
• network_security_firewall_endpoint
• network_security_firewall_endpoint_association
• network_security_tls_inspection_policy

We recently added support for NGFW Enterprise and TLS inspection in Fabric FAST:

• We created a CAS dedicated Fabric module.
• We updated the security stage: you can now create private CAS CAs, Certificate Manager trust-configs and TLS inspection policies.
• We added a new network-security stage, where you can reference the TLS inspection policies created in the security stage and create the rest of the NGFW Enterprise resources.

Conclusions

In conclusion, TLS inspection with NGFW Enterprise offers a powerful way to enhance your network security posture in GCP. However, successful deployment requires a clear understanding of its mechanics and careful configuration, especially when dealing with ingress traffic and load balancers. By following the best practices outlined in this article, you can leverage NGFW Enterprise to effectively inspect TLS traffic while ensuring a seamless experience for your users.

Stay tuned! My next article with further insights on NGFW Enterprise and related firewall policies design will be out soon!