Costless, Automated, Trusted Certificates on Palo Alto Networks Firewalls

Accept Risk and Continue? Man-in-the-middle attack?

Nope, just a self-signed certificate from the firewall.

Seeing this screen annoys me and reminds me of how much I hated renewing certificates every year.

I used self-signed certificates generated by the Palo Alto Networks firewall for GlobalProtect VPN service. I used it to access my lab and over the years, I’ve had ran into a few challenges issuing and renewing certificates on the firewall.

The self-signed firewall generated certs are valid for one year and require I manually renew them on the firewall. They are not issued by a trusted Certificate Authority so you will always get the cert warning. There have been a few times where I forgot to renew the certificates and everyone was locked out of the lab.

In this quick how-to I will guide you through the steps I took in order to automate the certificate renewal process on a Palo Alto Networks Next-generation Firewall using a free trusted Certificate Authority’s(CA) certificates.

Objectives

• Issue a trusted certificate
• Automated renewal and deployment of certificates

LetsEncrypt

LetsEncrypt is a free trusted Certificate Authority that uses the ACME protocol to automate the process of verification and certificate issuance.

There are a couple ACME clients available to issue certificates. LetsEncrypt suggests using CertBot. At first, I was using CertBot to verify and issue certificates with LetsEncrypt. CertBot ideally runs on the sever that the hostname resolves to and requires port 80 or 443 to be open to receive verification from the ACME servers.

This option didn’t work for me because I can’t run CertBot on the firewall and I didn’t want to leave ports open on my network to a server hosting CertBot.

When renewing certificates, I would port forward 80 and allow inbound access to CertBot. Then once I was done I would manually remove the NAT rules, port forwarding policy and firewall rules. This process had to be done every 90 days because LetsEncrypt certs are only valid for 90 days.

There is another client available called acme.sh. This client allows us to automate renewal and I will talk about how it has helped me automate the entire process of renewing and deploying certificates.

Install acme.sh

The acme.sh client is available at:

https://github.com/acmesh-official/acme.sh

Install acme.sh client on a machine behind the firewall. following the install guide documented on the GitHub page. We need to allow the acme.sh client outbound connectivity to the internet to access the ACME servers, your DNS provider, as well as the Firewall/Panorama management interface. Unlike CertBot, this machine does not require inbound connections I’ll explain why in the next section.

There are several methods to installing acme.sh I used the install from git option.

git clone https://github.com/acmesh-official/acme.sh.git
cd ./acme.sh
./acme.sh --install

This is what we should see after running the install command.

If you get the error for standalone server mode. You can ignore it, as we don’t plan to use it in standalone mode.

Create Firewall API Administrator

In order to make changes to the Firewall we need to setup an Administrator for the sole purpose of importing certificates and committing.

On PAN-OS 9.1 and newer you can create a user with an administrator role that has API access to import and commit. Be sure to remove all the GUI access.

Create an Admin Role Profile with only XML/REST API. We only need “commit”, and “import” permissions.

Device -> Admin Roles: REST API Commit and Import Only

Create a new Administrator and assign the profile we just created to that user.

Device -> Administrator: Assign user to the role we created

On versions prior to PAN-OS 9.1 we will need to provide an administrator with Superuser access because only a Superuser can import a private key in these versions.

Issue Certificate

We don’t want to expose the acme.sh client to the internet. So, I decided to use the DNS API options available from acme.sh. This option allows LetsEncrypt to verify the domain by creating a temporary TXT record on the DNS provider.

I’m using CloudFlare as my DNS provider and CloudFlare DNS is supported by acme.sh client. So, in my case, the acme.sh client requires outbound internet access to connect to the CloudFlare API to verify and issue a cert.

We need to provide the API credentials to make the API call. Follow the documentation on how to include API credentials for your DNS provider:

https://github.com/acmesh-official/acme.sh/wiki/dnsapi

After you have configured the credentials for the DNS provider you can continue with issuing a certificate.

Below is the CLI command for issuing a certificate with CloudFlare DNS API. The flag —-dns dns_cf is specific to CloudFlare API. Refer to the acme.sh documentation on how to use other DNS providers.

acme.sh --issue --dns dns_cf -d example.com

Once the certificate is issued acme.sh will take care of automatically renewing the certificate every 60 days. We only need to run this command once manually.

Deploying Certificate to Palo Alto Networks Firewall

Now that we have a certificate. We need to import the cert to the firewall. This can be done through the firewall GUI, but that requires a human to remember to deploy the cert after renewal. The acme.sh client has a feature that can automatically deploy the cert after renewal via deploy-hooks.

I wrote a deploy-hook function for acme.sh that uses the firewall’s API to import the certificate and commit on the firewall. The acme.sh client will run this deploy-hook after every renewal. This completely eliminates the human interaction of deploying the renewed certs.

On the acme.sh client that we just installed. Set the username, password and firewall IP/hostname that we setup earlier as environment variables.

export PANOS_USER="your_cert_user"
export PANOS_PASS="your_password"
export PANOS_HOST="10.0.0.1" //Can also be hostname

The acme.sh deploy-hook only commits changes made by this user. Uncommitted changes by other users on the firewall won’t get committed by mistake.

We can now run the deploy-hook called panowith acme.sh. The —-insecure flag is to not verify the certificate and get us past the default self-signed certificate on the management interface of the firewall.

acme.sh --deploy -d example.com --deploy-hook panos --insecure 

I recommend configuring the firewall/Panorama to use a hostname with a trusted certificate so that you don’t need to use the --insecure flag. This will eliminate the possibility of a man-in-the-middle trying to steal our certificates as we deploy them. This can be done after we deploy our first trusted CA certificate on the firewall.

If we need to troubleshoot the deploy hook we can append —-debug to the end.

acme.sh --deploy -d example.com --deploy-hook panos --debug

After a successful deploy we can remove host, username and password environment variables as they will be base64 encoded and stored by acme.sh. To change the stored variables, we will need to set all the environment variables and re-run the deploy hook.

After configuring the host and domain of the firewall to a trusted hostname. We can now configure a SSL/TLS service profile that uses the new cert. I recommend changing the environment variables of PANOS_HOSTto the trusted hostname and re-run the deploy command with out --insecure flag. This will insure that future renewals will use the hostname of the firewall and verify its authenticity.

acme.sh --deploy -d example.com --deploy-hook panos 

Don’t forget to delete the environment variables that you set.

Automated Renewals Complete

Now that you have successfully issued, and deployed your certificate to the firewall The acme.sh client will handle the renewal and deployment of the certificates every 60 days. You no longer need to worry about renewing the certificates on the firewall. One less task we need to take care of.

Enjoy!

Documentation on how to run this deploy script is also available on the acme.sh wiki:

acmesh-official/acme.sh