Raspberry Pi: Cloudflare Tunnel High Availability ( HA ) with replicas

Increasing Service Availability by running Cloudflare Tunnel replicas on two Raspberry Pis

Raspberry Pi: Cloudflare Tunnel High Availability ( HA ) with replicas

Intro

In the previous post, I shared how to install and utilize Cloudflare Tunnel to expose the service in a private network ( or local network ) to an external network ( or Internet ).

The Cloudflare Tunnel that I set up has been working well except for the service availability. When the Raspberry Pi was down for maintenance, the service that was exposed to the Internet was not accessible. There should be high availability of the service.

This post is a note of how I configured Cloudflare Tunnel with two replicas in Raspberry Pis by using the Ansible Playbook I wrote.

Researched the offered High Availablity in Cloudflare Tunnel

Cloudflare Tunnel offers high availability (HA) features to ensure that our applications remain accessible even during infrastructure failures or network disruptions.

There are more than the two listed below, but these two are the most mentioned in Cloudflare Tunnel’s official documentation.

cloudflared replicas ( FREE )

When to use?

• To provide additional points of availability for a single tunnel.
• To allocate failover nodes within your network.
• To update the configuration of a tunnel without downtime.

Load Balancer ( PAID )

When to use?

• To intelligently steer traffic based on latency, geolocation, or other signals.
• To implement failover logic if a tunnel reaches an inactive state.
• To get alerted when a tunnel reaches an inactive state.
• To distribute traffic more evenly across your Cloudflare Tunnel-accessible origins.

cloudflared replicas vs. load balancers

Cost

To me, the biggest difference between these two was the cost. The Cloudflare Tunnel with replica is free, but the Load Balancer is charged based on multiple factors ( the base price is $5/mo )

Functionality

Major functionalities I noticed in Load Balancer were the smart traffic routing based on the latency and the evenly distributed traffic across multiple Cloudflare Tunnels ( note: Load Balancer needs more than one Tunnel )

I love the Load Balancer approach, but since I don’t need the offered functionalities from Load Balancer, I picked cloudflared replicas.

Cloudflare Tunnel High Availablity — Replicas vs. Load Balancer

Intro how I set more than one Cloudflare Tunnel Instance

NOTE: In short, cloudflare replicas uses one Tunnel. ( This is an important notion to understand. )

The way I setup cloudflared in the previous post, I did all the configuration in the Raspberry Pi where I wanted to run the Cloudflare Tunnel. Therefore, the related Cloudflare Tunnel config was left in the Raspberry Pi.

To create a new Cloudflare Tunnel replica instance, I had to copy those Cloudflare Tunnel configs to the second Raspberry Pi.

It works, but I wanted to have an easier way than manually downloading from the existing Cloudflare Tunnel configs and uploading them to the second Replica. ( I wanted to have an automated way. )

So, I decided to use my MacBook as a workspace to create the initial Cloudflare Tunnel configs. And, those configs were uploaded to the Raspberry Pis by Ansible scripts I built.

Raspberry Pi: How to expose Web Services in Private to Public with Cloudflare Tunnel — DNS routing

Section 1: Create Cloudflare Tunnel Configs

NOTE: Although I use my MacBook to create ( or initiate ) the Cloudflare Tunnel configs, it doesn’t mean I will run Cloudflare Tunnel on my MacBook. ( well. why not? but, that’s not what I want )

In this “Section 1: Create Cloudflare Tunnel Configs” section, I will do

1. Install cloudflared by using brew
2. Login to Cloudflare to ( it will create cert.pem )
3. Create Cloudflare Tunnel ( it will create <tunnel_uuid>.json )
4. Create DNS Route — Routing Traffic through Public Domain ( it will create CNAME DNS records )
5. Create a Cloudflare Tunnel configuration file
6. Confirm the list of files under ~/.cloudflared

1. Install cloudflared by using brew

If you don’t use MacBook or brew, then use the package tool you normally use on your machine. Since I mostly use brew , I used brew to install cloudflared.

# note: after installing the pkg, open a new terminal
#         if cloudflared is not reachable.
brew install cloudflared

Install cloudflared on MacBook by brew

cloudflared

2. Login to Cloudflare

To interact with Cloudflare Tunnel API, the authentication has to be done first. Here are the steps.

1. execute cloudflared login . This will open a browser and ask you to login to Cloudflare if you’re not logged in.
2. Pick the list of domains that you want to use to configure with Cloudflare Tunnel. ( I picked one that I wanted to try )
3. Confirm ( Authorize ) your selections in the domain
4. Check the path of the created cert.pem ( For me, it was under /Users/core/.cloudflared/cert.pem )

cloudflared login

Cloudflare Tunnel: Pick the domain you want to use.

Cloudflare Tunnel: Confirm your domain selection

Cloudflare Tunnel: Confirm your domain selection

3. Create Cloudflare Tunnel

In the previous step, the cert.pem was created after the Cloudflare authentication.

In this step, the Cloudflare Tunnel can be created with the command below. Since a Cloudflare Tunnel can be used for many domains, the Tunnel name doesn’t have to be specific unless you want.

In my case, I used “wowamazon” as a Tunnel name.

Once the Tunnel was created, I was able to check it on Cloudflare Tunnel Web UI like the screenshot below.

NOTE: The Cloudflare Tunnel creation will create a Tunnel credential ( <tunnel-uuid>.json

cloudflared tunnel list
cloudflared tunnel create <tunnel-name>

# Get tunnel info
cloudflared tunnel info <tunnel-name>

# In case, the tunnel has to be removed.
# -f can be used to force to remove it
cloudflared tunnel delete <tunnel-name>
cloudflared tunnel delete <tunnel-name> -f

Cloudflare Tunnel: create one by using cloudflared

Cloudflare Tunnel: created Tunnel can be found on Cloudflare UI as well

4. Create DNS Route — Routing Traffic through Public Domain

After the Cloudflare Tunnel creation, I created the DNS route with the domain I wanted to route through the Cloudflare Tunnel.

The DNS Route creation also created CNAME records under the domain wowamazon.party like the screenshots below.

Also, the Cloudflare Tunnel was updated with the Routes I just added like the screenshot below.

The created CNAME’s content has the format below.

<tunnle-uuid>.cfargotunnel.com

When you create a tunnel, Cloudflare generates a subdomain of cfargotunnel.com with the UUID of the created tunnel. You can treat UUID.cfargotunnel.com as if it were an origin target in the Cloudflare dashboard.

IMPORTANT: Unlike publicly routable IP addresses, the subdomain will only proxy traffic for a DNS record or a Load Balancer pool in the same Cloudflare account. If someone discovers your subdomain UUID, they will not be able to create a DNS record in another account or system to proxy traffic to the address.

ref: https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/routing-to-tunnel/lb/

cloudflared tunnel route dns <tunnel-name> <domain>

## For myself.
## NOTE 1:
# on macOS ( especially with zsh ), the double quote is required.
# The double wasn't required when I ran the command in Raspberry Pi directly.
## NOTE 2:
# The created DNS route can be removed only through Cloudflare WebUI.
cloudflared tunnel route dns wowamazon wowamazon.party
cloudflared tunnel route dns wowamazon "*.wowamazon.party"

Cloudflare Tunnel is updated with the two DNS routes.

DNS records have two CNAME based on the created DNS routes

IMPORTANT to note

• currently, the created CNAME DNS record can be removed only through Cloudflare dashboard WebUI. ( The feature request is still open. )
• the created CNAME DNS record can be seen only through Cloudflare dashboard WebUI.
• The DNS Rounte creation will fail if the CNAME entry already exists. ( Each DNS route creation will create a CNAME.

Cloudflare Tunnel: the DNS route can be removed only by removing DNS record on WebUI or API.

5. Create a Cloudflare Tunnel configuration file

It is time to create a Cloudflare Tunnel configuration file ( config.yml ).

I created Cloudflare Tunnel configuration file in the path where cert.pem was created. ( in my case, ~/.cloudflared/config.yml )

The content I used like below. I configured the ingress to route all traffic to the local “http://127.0.0.1:8080” where the docker was running and listening. ( Please change the content based on your context. )

And, the value of “tunnel” is the UUID I got when I created the Cloudflare Tunnel. ( please update those Tunnel UUID with yours. )

For the credentials-file, I used “/etc/cloudflared/tunnel_cred.json”. The path is set like that because the config will be uploaded to Raspberry Pi later by Ansible Playbook.

NOTE: The service does not have to be in the same machine where the Cloudflared Tunnel runs.

tunnel: e992f289-0a5f-40d4-8273-d0508316226c
credentials-file: /etc/cloudflared/tunnel_cred.json

# ref: https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/configure-tunnels/local-management/configuration-file/
ingress:
  - hostname: wowamazon.party
    service: http://127.0.0.1:8080
  - hostname: "*.wowamazon.party"
    service: http://127.0.0.1:8080

  - service: http_status:404

6. Confirm the list of files under ~/.cloudflared

If everything is done properly, there will be three files under ~/.cloudflared like the screenshot below. They are

• cert.pem: cloudflared credential
• e992f289–0a5f-40d4–8273-d0508316226c.json: cloudflare tunnel credential ( <tunnel-id>.json )
• config.yml: cloudflare tunnel config

Cloudflare Tunnel: the list of files under ~/.cloudflared

Section 2: Setup Cloudflare Tunnel on two Raspberry Pis by using Ansible Playbook with the locally created Cloudflare Tunnel Config

In the previous section 1, the required Cloudflare Tunnel configs were created. In this section 2, I will share how to set up Raspberry Pi by using Ansible Playbook.

If you have created your Raspberry Pi’s bootable SSD ( or micro SD Card ) with the described steps in the below post, it will be easier to follow and run the Ansible script. If not, you need to set up an SSH Key login.

Raspberry Pi 5: Build bootable SSD — macOS

This section will cover these.

1. Install Ansible CLI on MacBook
2. Git clone Github moon-rapi repo ( Ansible Playbook )
3. Prepare Cloudflare Tunnel Config
4. Run ansible Playbook

1. Install Ansible CLI on MacBook

To run Ansible Playbook, Ansible CLI has to be installed. The Ansible CLI can be installed by using brew .

brew install ansible

2. Git clone moon-rapi repo ( Ansible Playbook )

Clone the Github moon-rapi repo. Once the repo is cloned, go into the moon-rapi directory.

git clone https://github.com/Gatsby-Lee/moon-rapi.git
cd moon-rapi

3. Prepare Cloudflare Tunnel Config

In this step, let’s copy the Cloudflare Tunnel configs that we created in section 1.

1. create the .key directory in moon-rapi directory.
2. copy cloudflared cert.pem
3. copy cloudflare tunnel config.yml
4. copy cloudflare tunnel credential JSON.

If everything is done correctly, there will be three files under moon-rapi/.key directory. The three files will be copied into the target Raspberry Pi by Ansible Playbook.

mkdir .key

cp ~/.cloudflared/cert.pem .key/cloudflared_cert.pem
cp ~/.cloudflared/config.yml .key/cloudflared_tunnel_config.yml
# use your own tunnel credential
cp ~/.cloudflared/e992f289-0a5f-40d4-8273-d0508316226c.json .key/cloudflared_tunnel_cred.json

Provision Cloudflare Tunnel Config to moon-rapi/.key directory.

4. Run Ansible Playbook

Here is the Ansible Playbook.

moon-rapi/ansible_playbook/setup-cloudflared-on-debian-template.yaml at main · Gatsby-Lee/moon-rapi

This Ansible Playbook will do these in the target Raspberry Pis.

1. Install the required packages ( and necessary packages )
2. Install cloudflared package
3. Create /etc/cloudflared directory
4. Copy the files that were provisioned in the prevision step to /etc/cloudflared directory
5. Install cloudflared service

In my case, the two target Raspberry Pis were

• using 192.168.128.21 and 192.168.28.22
• using core user

cd ansible_playbook

ansible-playbook --inventory <target-raspberry-pi>, \
--user <username> \
--private-key ~/.ssh/<ssh-key> \
setup-cloudflared-on-debian-template.yaml

Run Ansible playbook to the target Raspberry Pi

5. Confirm Cloudflare Tunnel Connectors

Once the Ansible Playbook runs successfully, the running Cloudflare Tunnel can be found by executing the command below.

cloudflared tunnel info wowamazon

The list of running Cloudflare Tunnel connectors can be found

Next

Raspberry Pi: Cloudflare Tunnel: Design High Availability in Services

Posts you might be interested

Raspberry Pi: Install Docker by Ansible on arm64 — Debian Bookworm

Raspberry Pi 5: Picked one from Local Computer Store

Raspberry Pi 5: Picked up 27W USB-C Power Supply