Consul + Fabio + Your App

Mike R
Mike R
Jun 27, 2018 · 7 min read

+ Your Application

Sysinfo: Centos 7, Consul 0.9.2, hypothetical Pong application

Background

Need a good way to rout users to a custom application (lets call it Pong, running on Port 8300 (via Apache on port 80). There are 3 nodes running Pong and these will be monitored by Consul

Fabio comes in as an add-on to Consul and does the actual Routing and maintains its own Routing tables based on Consul data.

Fabio is great because you dont need to update Routing table, its completely automated. It reads whatever Service and host is on Consul and works off that.

Note:

This article is not a good solution for apps that need persistent sessions. Consul and Fabio do not support session persistence. This setup is best used for small applications and microservices. For large applications like Splunk or anything Java-based, use HAProxy to balance nodes.

Also, if this setup does not ensure COMPLETE 100% failover as users are routed to 1 single Fabio instance. For true 100% failover, you will need to setup 3 instances of Fabio + a Virtual IP that binds them together and have users access that 1 Virtual IP.

Basic structure:

1. Users go to a shared (virtual) IP, lets say “pong.company.com” -> this is resolved by DNS to 10.155.20.5

2. Virtual IP then passes the request to one of the 2 Consul/Fabio servers

3. Fabio reads healthcheck data from Consul, determines there are 3 active Pong

Configure Consul server cluster

Lets create a 2 node Consul cluster. Consul will monitor application service health, and Fabio will run on top of each Consul and get its routing tables from Consul data.

1. on each Consul server, install Consul
yum install consul

2. generate an Encryption key & Master Token
consul keygen
UAvkAzdjGfQ7J2NlgkrJMA==

geneate a Master Token

uuidgen
dbef8b5a-6110-4575-bf61-dda1c21ca339

3. create Consul dirs
mkdir -p /etc/consul.d/server
mkdir /var/consul

4. add Consul user + group
groupadd consul
useradd consul -g consul

5. change permissions

chown -R consul:consul /var/consul

Consul Config

6. on the 1st Consul node, create new Boostrap config,

validate the syntax

consul validate /etc/consul.d/*

fix any validation errors

Startup Service

7. create Startup Service

8. Do the same for 2 remaining Consul servers, change “bootstrap”: false

9. Start service, Web console should be up at <IP>:8500

UI should be available

Create ACL on Consul cluster

Login to Web UI, click on config gear, enter Master ACL token, click Close, this will save the Master token access between your browser and Consul cluster (this is better than using username/password)

Now click ACL button, update Anonymous Token ACL to let Read-Only access

Now create a new ACL for Pong service,

service "" { policy = "write" }
key "pong/" { policy = "write" }
node "" { policy = "write" }
session "" { policy = "write" }

get the Token ID of this Pong ACL policy

go back to each Consul node, change the consul.json setting, change to “Deny”

"acl_default_policy": "deny"

restart Consul service on all server nodes

Configure Consul agent on each Pong instance

on each Pong node, install Consul

Configure Consul agent on each Pong instance, for ACL token add the Pong ACL token from above

pong01> vim /etc/consul.d/client/consul.json

Service Health Check

pong01> vim /etc/consul.d/pong.json

This will check the Pong app running on Port 8300

To start Consul, add the same start up scripts as for Consul Server

start Consul

Client should register itself and its Pong service with the cluster

Try stopping Pong and watch for Consul to show the service as Orange (failing)

Fabio config

To rout users to any of the 3 operating Pong instances, you will need Fabio to read the health status of each instance (from Consul)

on each Consul sever, create Fabio user + group

get the Binary
wget https://github.com/fabiolb/fabio/releases/download/v1.5.9/fabio-1.5.9-go1.10.2-linux_amd64 -O /opt/fabio/bin/fabio && chmod +x /opt/fabio/bin/fabio

On each Consul server, Add Fabio properties file, update ui.addr and regitstry.consul.addr, by default Fabio will listen on port 9999

Fabio Properties

vim /opt/fabio/fabio.properties

Create a Fabio startup script

set permissions
chown -R fabio:fabio /opt/fabio

systemctl enable fabio.service
systemctl daemon-reload
systemctl start fabio.service

check Fabio service

journalctl -u fabio --no-pager -n100

You should now see Fabio service listed on Consul

Fabio should be listening on port 9998, open up browser to <IP of Consul Server>:9998

But there should be no routes yet

Fabio Routes

We will create 1 Route for our Apache thats running on each Pong server.

Fabio will route all requests coming to Fabio hostname, port 9999, it will route to <IP of Pong server>:80

We now have a total of 2 Health Checks, 1. Pong 2. Apache

Consul monitors both, but Fabio will create a route only for Apache,

Apache service check

lets add the Apache healthcheck, we will want to rout users to our Apache (which will use Reverse Proxy using certs to proxy users to Pong Web via HTTPS). Note: Fabio also supports TLS authentication but that is outside the scope of this article

add a new Apache health check, the Tag is using the “urlprefix-” to tell Fabio to create a Route, followed by a slash,
Fabio will route any requests coming to Fabio host (port 9999)

restart Consul client on the Pong node (make sure the Pong healthcheck passes, otherwise Fabio wont show the route)

you should now see Fabio display a proper route

Check the available routes using API

curl -s http://10.185.20.180:9998/api/routes

[{"service":"apache-svc","host":"","path":"/","src":"/","dst":"http://10.185.20.173:80/","opts":"","weight":1,"cmd":"route add","rate1":0,"pct99":0}]

Test Routes

on the Consul/Fabio host, tail Fabio output,

journalctl -u fabio -f

Turn off either Apache or Pong service, watch the routing table get updated automatically

In the browser, try going to <IP or Hostname of Consul/Fabio host>:9999

It should redirect you to <Pong host>:80, and from there, Apache will take over and reverse proxy you to Pong service running on its own different port

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade