Getting Let’s Encrypt to app servers / firewalled networks

Does Let’s Encrypt (LE) support installation on servers with “random domain names”? The first impulse is to simply answer NO, you can’t do it, that’s the point of HTTPS, it makes sure you always talk to the right server. However, network configuration is a kind of magic.

We use Let’s Encrypt on test and demonstration servers for our cloud encryption service. The picture shows one of these servers — more at https://enigmabridge.com/hsm

Because of this network magic, the correct answer is YES, one can install SSL certificates on servers with different domain names or a random IP address. It does need a bit of network configuration to get it working as expected though.

Note: before I go into details, I should mention that everything becomes simpler, from the networking point of view, when you can use your DNS records to prove the control of your servers’ domain names.

The first task is to “link” the random domains or IP addresses of your servers. You don’t have to frown on this as there are legitimate reasons and it is a common networking task. We need it for scalability, the resiliency of large web services, use on internal networks, testing, or just for ad-hoc networking. When you think about it, e.g., CDN (content delivery networks) will direct hundreds or thousands of domain names to the same IP address and then correctly return the right web page. They couldn’t do it without exactly this kind of configuration.

Note: While this is a quite different topic, you can also use wildcard or multi-domain certificates. Multi-domain, or SAN certificates, are certificates, which may contain tens of (unrelated) domain names. You can install such a certificate (and its private key) on any of the domains it contains. As a matter of fact, you will also become trusted for all other domains from the security point of view. The security, protection of those domains from each other is done purely with the network configuration.

Certificates on “random” servers

Let’s say “one domain” (name in the certificate) is called MASTER and “another domain” (where you want to install the certificate) is SHADOW.

Getting a certificate for the domain MASTER. If you want a certificate with the domain name MASTER issued by a web-trusted certification authority (CA) you need to “control” this domain! Any such CA will require you to prove that you control the MASTER domain. The proof can be a response to an email, adding a file to a MASTER web server, or adding an authorization data to the MASTER’s DNS records. This proof is the reason why setting up HTTPS/TLS can become very complicated.

However, it also allows tricks which may be potentially dangerous, but also useful for the certificate setup in firewalled, private networks.

Getting a MASTER certificate to the SHADOW server

If you expect me to say that you simply copy the certificate and its private key between the two servers, I will not. Of course, you can do, but you create a weak central point which is never a good idea unless you reinforce its security. And when I say reinforce, I mean some pretty strong encryption of keys you store there. It is also much harder to automate as it requires granting admin privileges between servers.

The certificate issuance process can be broken down into two main parts. 1. authorization, where you show you control the domain name (actions 1–3), and 2. certificate issuance (actions 4–5).

As you are likely to use Let’s Encrypt (LE) these days, I will focus on this use case.

When you request a new LE certificate, you need to get an authorization (“authz”). It’s a token stored by Let’s Encrypt for your “account key” and a particular domain name. Once it’s created, it will remain valid for several days. So long as this token (authz) is valid, you can request certificates for the given account and domain name without a need to prove you control the domain name.

Please have a look at a simplified depiction of the issuance process above. For our purposes, we need to “split” the process in the middle, where the “split” action is nothing more than using a command line option “renew” the second time round.

First things first though — we need to set up the system, which is basically moving the account key to all servers where we want to run the certification requests. In our case to the MASTER, and the SHADOW servers.

The whole process, then looks like this:

1. once only — copy the account key from the MASTER to the SHADOW server.
2. whenever you need a new certificate for the domain MASTER on the SHADOW server, you:
3. request a new certificate on the server MASTER; and
4. request a new certificate for the same domain on the server SHADOW with “renew” switch, if it exists. This must be done within a few days.

In some situations we the roles of SHADOW and MASTER servers are reversed. For example, when the MASTER servers are behind a firewall and can’t be access directly from the internet. In this case, the router separating your network from the internet needs to be set up so that the certificate authorization reaches the correct server in step 3.

Using certificate on the SHADOW server

At this point, the SHADOW server has a valid key and a certificate for the domain name MASTER. The next step is to make web browsers, or any other clients accept this certificate. You can do that either by configuring the server or clients. Either way, what we need to do is to bind the domain name MASTER to the IP address of the server SHADOW.

Configure client(s). You can define the IP address for a particular domain name on any computer — Linux, Mac, Windows in so-called “hosts” file. When computers resolve IP addresses from domain names, they will first look into their “hosts” file as it has the highest priority for DNS resolving. This means whatever you set here will ensure that the computer will use.

You want to connect to the server SHADOW when you type the MASTER domain into the computer’s web browser. So if the SHADOW server has IP address 23.23.23.23 and you change your “/etc/hosts” to contain:

23.23.23.23 MASTER

When you now type the MASTER domain to your browser, the request will be sent to the IP address 23.23.23.23, i.e., the SHADOW server. Here the browser finds the certificate for MASTER. Bingo, as the web browser has the MASTER address and it gets a MASTER certificate, you get a green padlock in the browser.

Configure server. You can set either a proxy, your network router, create a redirect from your MASTER server to the SHADOW server, or setup a local DNS server (any network router can be used for this as well).

A redirection on the MASTER server means that web browser requests will reach the MASTER, but it will forward it to the SHADOW server. This is what some large web services do, especially when they have several SHADOW servers to handle large traffic. Again, anyone trying to connect to the MASTER domain will see “secure connection” even though the requests hit a SHADOW server.

If you want to set up local DNS, you can run own DNS server(s), or use a router separating your network from the internet. The router then provides a “hosts” file, where you have to manage the MASTER name to the SHADOW IP address mapping, or is a source of your local DNS server location.

As you can see, it’s all about networking and “bonding” domain names to IP addresses of the destination (SHADOW) servers. You may also now have a bit better idea about where the power of certificates ends and where we have to depend on skills of network administrators to get things right.

If you want to get a little bit more insight into the configuration of a particular server, you can use our https://keychest.net spot check as it does not follow redirects. It will show if there’s a redirect in place though.

Try the Professional HTTPS/TLS monitoring service KeyChest.net to keep on top of your certificates and to keep 100% HTTPS uptime. The public cloud service is free and allows you monitor thousands of certificates within minutes (YouTube video — 49 seconds).