Standing up Rancher

We wanted to be able to manage our infrastructure and our applications. We used Docker Cloud to do so, but since we have a lot of virtual machines hosted on Azure, the cost would increase quite fast.

The decision was made that Rancher was a good fit for our needs because it can manage and provision docker-machines from Azure Virtual. But getting it properly setup was not an easy task, not for the product itself, but for the dependencies all around.

I must mention that we really enjoy all the functionalities that Rancher offers and also the speed at which they reply to any issues posted on their github repos. Thanks a lot for your hard work and your great product! :-)

NOTES:

The Services Definition

version: '2'
services:
haproxy:
image: m21lab/haproxy:1.6.2
links:
- letsencrypt
- rancher
ports:
- '80:80'
- '443:443'
restart: always
volumes:
- /var/run/docker.sock:/var/run/docker.sock
volumes_from:
- letsencrypt
  letsencrypt:
image: m21lab/letsencrypt:1.0
environment:
DOMAINS: 'YOUR_DOMAIN'
EMAIL: 'RECOVERYEMAIL@YOUR_DOMAIN'
LOAD_BALANCER_SERVICE_NAME: 'haproxy'
OPTIONS: '--staging'
  db:
image: m21lab/rancher-mysql:5.7
restart: always
environment:
- MYSQL_ROOT_PASSWORD=MYSQL-ROOT-PASSWORD
- MYSQL_DATABASE=rancher
- MYSQL_USER=rancher
- MYSQL_PASSWORD=MYSQL-RANCHER-PASSWORD
volumes:
- ~/rancher-mysql-data:/var/lib/mysql

rancher:
image: m21lab/rancher:1.5.5
links:
- db
expose:
- 8080
restart: always
environment:
CATTLE_DB_CATTLE_MYSQL_HOST: db
CATTLE_DB_CATTLE_MYSQL_NAME: rancher
CATTLE_DB_CATTLE_USERNAME: rancher
CATTLE_DB_CATTLE_PASSWORD: MYSQL-RANCHER-PASSWORD
MYSQL_SERVICE_NAME: 'db'
VIRTUAL_HOST: 'LETSENCRYPT_DOMAINS_ENV_VAR, https://LETSENCRYPT_DOMAIN_ENV_VAR'
#Redirects HTTP => HTTPS
FORCE_SSL: "true"

Services

HAProxy

This service is the load balancer which also handles the SSL traffic. I reconfigures itself when Let’s Encrypt certificates are received. More information in the previous article

Let’s Encrypt

This services is used to generate valid CA signed SSL certificates for the domain hosting the registry. More information in the previous article

MySQL

All is needed is to create a database name, root password, username and user password. Add a volume to store the database itself and it’s configured. The rancher database will be initialized if it’s not already existing or updated if there is a version upgrade.

Rancher

The rancher service configuration is mostly:

  • Linking the db service
  • Set environment variables to map database, database username and password
  • Set environment variables to receive traffic from the haproxy service and to force it to be using SSL
  • Set the MYSQL_SERVICE_NAME environment variable which will be used by the image to ensure the database service is up and running prior to start the rancher service

How to start it

Locally

If you just want to run it locally, you won’t want to use the haproxy/letsencrypt setup, so just add a port mapping on the rancher service :

services:
rancher:
ports:
- '80:8080'

Then run

docker-compose up -d db rancher

Publicly accessible machine

Important Assumption: You are running those docker commands from (or using docker-machine as explained in this article) a machine publicly accessible by the domain you want to get SSL certificates for.

  • Update the letsencrypt service’s DOMAINS environment variable
  • Update the rancher service’s LETSENCRYPT_DOMAINS_ENV_VAR with the same domain

Then run

docker-compose up -d

Problems encountered

Minimum 1Gb of RAM for the VM: It matters!

We used an Azure Basic AO instance with version 1.1 and it was running smoothly even tho the minimum requirements were 1Gb RAM. Once we upgraded to 1.3, we did not think about upgrading the VM: it was working before! And at some point, the system just got really slow until it rebooted all by itself without any obvious logs from Rancher or the Azure portal.

Self Signed Certificates: great! SSL, but …

When we started to configure this server, we used a self signed certificate. This lead to multiple problems because:

  • Every hosts added to Rancher would have to be manually configured to trust the self signed CA certificate.
  • Not even the host itself needed to be configured, but a specific folder used as volume by the rancher/agent container needed to be updated too: /var/lib/rancher/etc/ssl/ca.crt
    NOTE:
    The doc was only updated in version 1.3, it wasn’t mentioned in version ≤1.2
  • The Docker registry was also using a self signed certificate, which forced us to add our own registry as “unsecure registry” when creating a host which modifies the parameters passed to the docker-engine the host runs
  • Our NPM repository was also using a self signed certificate which was not trusted by any of the container and all the projects were modified to include the strict-ssl=false configuration in the .npmrc file
  • Every user of the Rancher portal would have to trust the company’s self signed certificate

Provisioning Azure VM from Rancher 1.1 to >1.1

We started using rancher/server version 1.1.0 a while back, and when we tried to provision an Azure virtual machine, it was required to create an SSL certificate and upload it to the Azure Management Portal in order to give rancher the rights to provision Azure resource on behalf of our subscription.

Surprise! once we got that up and running and decided to upgrade to 1.3, this method had changed! Rancher uses the Azure docker-machine driver which now required an Active Directory Application with a Service Account attached which will grant the rights to create resources in the Azure Subscription. Obviously, we need the IT in order to do so since it had to be done in the default Active Directory of the Subscription.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.