CloudFlare tunnel with Hashicorp Nomad with Consul connect

First thing we need is to start our nomad and consul services ( i will use dev mode for this article )

nomad agent -dev-connect
consul agent -server -dev

Then, go to to CloudFlare choose your account and go to “Zero Trust” section. Go to “Access” and then “Tunnels”. Create new Tunnel, and choose “docker” version. Then, grab “token” and put in job or nomad env/vault.

Then, you need to create Public Hostname. I will use dom.tld as domain.

Eeasiest way to create wildcard domain would be to create some.dom.tld with target “http://127.0.0.1:8080” and then rename dns record to “*” and create *.dom.tld Public Hostname to “http://127.0.0.1:8080”. Once you done, run this job.

job "cftunnel" {
  type = "service"

  group "cftunnel" {

    network {
      mode = "bridge"
    }

    service {
      name = "cftunnel"
      provider = "consul"
      address_mode = "auto"
      tags = ["traefik.enable=true"]
      connect {
        sidecar_service {
          proxy {
            upstreams {
              destination_name = "traefik-consul"
              local_bind_port  = 8080
            }
          }
        }
      }
    }

    task "cftunnel" {
      driver = "docker"

      config {
        privileged = true
        image        = "cloudflare/cloudflared:latest"
        command      = "tunnel"
        args         = ["--no-autoupdate", "run", "--token", "$TOKEN"]
      }

      resources {
        cpu    = 100
        memory = 256
      }
      template {
        env = true
        destination = "local/env"
        data = <<EOF
TOKEN=[get token from nomad vars, consul kv or vault or just put it here]
EOF
      }
    }
  }
}

Now we need something to listen on http://127.0.0.1:8080 so I will use traefik container with connect support . Consul connect forwards traefik service port to CloudFlare tunnel container and binds to localhost:8080 port so all *.dom.tld comes to traefik.

job "traefik" {
  type = "service"
  meta {
    version = "1"
  }
  group "traefik" {
    network {
      mode = "bridge"
      port "http" {
        to = 8080
      }
    }
    service {
      name         = "traefik"
      tags         = ["traefik.enable=true"]
      port         = "http"
      provider     = "consul"
      address_mode = "auto"
      connect {
        native = true
      }
    }
    service {
      name         = "traefik-consul"
      port         = "8080"
      provider     = "consul"
      address_mode = "auto"
      connect {
        sidecar_service{}
      }
      tags         = ["traefik.enable=true"]
    }

    task "traefik" {
      driver = "docker"

      config {
        privileged = true
        image = "shoenig/traefik:connect"
        ports = ["http"]
        args = [
          "--providers.consulcatalog.connectaware=true",
          "--providers.consulcatalog.connectbydefault=true",
          "--providers.consulcatalog.exposedbydefault=false",
          "--entrypoints.traefik.address=:${NOMAD_PORT_admin}",
          "--serverstransport.insecureskipverify=true",
          "--api.insecure=true",
          "--entrypoints.web.address=0.0.0.0:${NOMAD_PORT_http}",
        ]
      }

    }
  }
}

And finnaly, we need something to serve. I will use 3 nginx services with html contents of Nomad Allocation ID.

job "nginx" {
  datacenters = ["dc1"]

  group "nginx" {
    count = 3
    network {
      port "http" {
        to = 80
      }
      mode = "bridge"
    }

    service {
        name         = "nginx"
        provider     = "consul"
        address_mode = "auto"
        port         = "80"
        tags = [
          "traefik.enable=true",
          "traefik.http.routers.${NOMAD_ALLOC_ID}.entrypoints=web",
          "traefik.http.routers.${NOMAD_ALLOC_ID}.rule=Host(`yoursubdomain.yourdomain.com`)"
        ]
        connect {
          sidecar_service {}
        } 
      }
    task "nginx" {
      driver = "docker"

      config {
        image = "nginx:latest"
        ports = ["http"]
        volumes = [
          "local/index.html:/usr/share/nginx/html/index.html"
        ]
      }

      resources {
        cpu    = 500 # 500 MHz
        memory = 256 # 256MB

      }

      template {
        data        = <<EOF
{{ env "NOMAD_ALLOC_ID" }}
EOF
        destination = "local/index.html"
        perms       = "0755"
      }
    }
  }
}

replace “yoursubdomain.yourdomain.com” with your real subdomain.dom.tld and refresh few times. it Traefik should balance through 3 nginx services.

And that’s it. No exposed ports. And you can bind any domain to your tunnel’s hostname as CNAME and define it in task tag for Traefik.

If you have any questions, write in commands and i will try to answer.

#hashicrop #nomad #consul #cloudflare #tunnel #zerotrust