Create a HTTPS-only Custom Domain CDN using Googles CDN, Cloud Storage and letsencrypt

6 min readOct 4, 2018

This short article explains in 7 steps how to migrate a Google Cloud Bucket to a Custom Domain enabled, HTTPS-only CDN. For that we use the Google CDN, Google Cloud Storage and Letsencrpyt.

UPDATE: There is a new way to set this up without letsencrypt and use google managed certificates instead. That is much easier to setup and you have no more maintenance regarding certificate renewals, but it is currently a beta feature. Use with caution.

UPDATE: I added a section on how to renew the certificate.

My main application runs on AppEngine and I love it. But there are some asserts that are changed by my application like article images and info graphics that change over time and can’t be part of the static content I usually deploy as static content within my AppEngine application.

In the past I used the Google Cloud Storage Buckets ability to serve content directly with https like product images https://storage.googleapis.com/kontaktlinse/air-optix-aqua-6-400x400.jpg. That was okay as long I only used it for product images and stuff displayed on the website. But when I started to create info graphics, PDFs and other stuff that is more related to my business, I don’t want to share that with the generic “storage.googleapis.com” domain in Social Networks or want it to be displayed in the Google image search.

Sadly, Google Cloud Storage doesn’t support SSL for custom domains — So I was looking for a way to bring my content available under a custom domain with SSL, backed by a Google Cloud Bucket.

The Solution, described in this article: Enable Google Cloud CDN with a Google Cloud Storage Bucket as origin and configuring a SSL-certificate with Letsencrypt.

Concrete: all content in the Google Cloud Bucket http://storage.googleapis.com/kontaktlinse-staging should be served by the Google Cloud CDN under https://cdn-staging.kontaktlinsen-preisvergleich.de.

Step 1: Check preconditions

You need a Google Cloud Bucket that is accessible via HTTP, like mine under http://storage.googleapis.com/kontaktlinse-staging/air-optix-aqua-6-400x400.jpg

It is important that the bucket or the elements that should be served have public access! In my case I simply gave “allUsers” the role Storage Object Viewer.

Step 2: install certbot

I’m running on a mac, so I just use

brew install certbot

Step 3: Create a letencypt DNS Challenge to prove that it is your domain

Usually, with letsencrypt you use a challenge & response (ACME) cycle based on HTTP to prove that you control the domain. But the ACME challenge is not supported by Google Cloud Storage and because of that, impossible for us to use. But there is another way to prove domain ownership, the dns challenge. It requires you to add a TXT Record to the Nameserver. Now run certbot to create a challenge…

sudo certbot certonly — manual — preferred-challenges dns

Step 4: Update your DNS with the TXT Record

Add the TXT Record mentioned that shows letsencrypt that you own the domain.

Step 5: Wait until the DNS has updated.

Can take up to 24h, but I was lucky and after 20 minutes the process succeeds.

Now we have a certificate, chain and key unter /etc/letsencrypt — that’s all we need for HTTPS in the Loadbalancer.

Step 6: create the HTTPS Loadbalancer with Google Cloud Storage Bucket as Backend

In the Cloud Console, go to Network Services / Load Balancer and on the right click “Create Load Balancer”, then click “Start Configuration” in the HTTP(S) Load Balancing part.

In the Backend Section, select “Create a backend bucket”

Select the bucket and enable the cloud CDN!

The Host and Path rules can be ignored. In the Frontend section reserve a new static IP and create a certificate by copy and pasting the content of the files in /etc/letsencrypt/… into the appropriate fields.

This should end up in a summary like this:

We now have a loadbalancer with enabled Google Cloud CDN running HTTPS only for a domain we created the certificate for. There is only one last step to do…

Step 7: Add a Nameserver entry for the cdn

At last we need to add an A Record to the nameserver to direct staging-cdn.kontaktlinsen-preisvergleich.de to our loadbalancer (35.227.236.123 in my case). Then wait again until the DNS has updated everywhere and we are ready to serve!

Summary

It was quite easy to migrate the Google Cloud Storage Bucket to a Google Cloud CDN backed Loadbalancer using a custom domain and HTTPS-only. I’ll add another article on how to renew the certificate, which will expire in 3 months. If you have questions or problems, please write me here or on twitter.

UPDATE: Renew the certificate

I just received an eMail, that my certificate is going to expire on Jan 1st — of course I am already in the Christmas holidays. But here is a quick guide how to renew the certificate for the configuration given above.

Sadly, a simple certbot renew is not possible due to the manual process required to make the dns challenge work. So we simply do the same steps as above to get a new certificate:

I simply created a new DNS challenge with the command:

sudo certbot certonly — manual — preferred-challenges dns

I added the TXT entry to the DNS of my provider and make the challenge succeed.

In the Google Cloud Console / Network Services / Load Balancing select edit in the Load Balancing Tab for the given Load Balancer.

Select Frontend configuration, click on Edit and in the Certificate Selectbox click on Create a New Certificate.

It is a good idea to add the expiration date to the certificate name. Add the certificate, chain and private key as we did on initial creation. Then click on Done!