Ptarmigan Labs
Published in

Ptarmigan Labs

Hey Butler, please meet Docker

Before the summer the Butler tool turned two years old — time flies!

Over those years I have installed, tweaked and upgraded a fair number of Butler instances… Not a problem per se, but maintaining a production grade Butler instance does assume a certain level of experience around Node.js, Linux, networking etc.

The most recent Butler version (v2.2) attempts to make it easier to deploy and operate Butler. This is achieved by deploying Butler as a Docker container instead of a regular Node.js app.

The Docker image (from which a container is created) contains exactly the same Node.js app that you can run right on your server or laptop — i.e. there is no functional difference what so ever between running the Node app natively, and running it as a Docker container.

There are some significant benefits of running Butler under Docker:

  • No need to install Node.js on your server(s)
  • Make use of your existing Docker runtime environments, or use those offered by Amazon, Google, Microsoft etc
  • Benefit from the extremely comprehensive tools ecosystem (monitoring, deployment etc) that is available for Docker
  • Updating Butler to the latest version is as easy as stopping the Butler container, doing a “docker pull ptarmiganlabs/butler:latest” and then starting the container again.
  • Fewer dependencies on whether you run Butler on Windows, Linux, cloud servers etc. With Docker, things usually just work.

Continuous Integration — CI

As mentioned, the ecosystem around Docker is amazingly rich. This means it is possible (even trivial) to automate creation of new Docker images when the GitHub project updates.

After reviewing several different free online CI (continuous integration) tools I settled with Codefresh. Some other services had more generous free tiers, but Codefresh really shines when it comes to ease of use, easy to get started and integration with things like Github and Docker Hub.

Creating a Codefresh pipeline just takes a few clicks, but results in

  • Automatic building of a new Docker image every time a new release of Butler is done on Github.
  • The new image is automatically tagged and pushed to Docker Hub. From there the latest version can be retrieved by anyone, by simple running “docker pull ptarmiganlabs/butler:latest”

Right — that link above is a referral link. Feel free to use it if you want to try out Codefresh, as it gives me some extra builds with them each month.
Those builds will come in handy when I keep working on the Butler tools. Thanks!

Hands on with Docker

Let’s take a look at what it looks like setting up Butler as a Docker container. The commands below are done on a Mac, but will be very similar on Linux and Windows. The instructions are also available on the Butler documentation site.

First let’s create some directories where the Butler config file and Sense certificates will be stored:

proton:~ goran$ mkdir /Users/goran/butler
proton:~ goran$ cd /Users/goran/butler
proton:butler goran$ mkdir -p config/certificate

Next, let’s pull down the docker-compose.yml file from the Butler Github repository. This file tells Docker how to create a container from a Docker image, and how to configure the container.

Let’s also take a look at what the docker-compose.yml looks like.

proton:butler goran$ wget
--2018-09-26 16:27:14--
Resolving (
Connecting to (||:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 565 [text/plain]
Saving to: 'docker-compose.yml.1'

docker-compose.yml.1 100%[========================================================================================================================================>] 565 --.-KB/s in 0s

2018-09-26 16:27:15 (6.34 MB/s) - 'docker-compose.yml.1' saved [565/565]

proton:butler goran$ cat docker-compose.yml
# docker-compose.yml
version: '2.2'
image: ptarmiganlabs/butler:latest
container_name: butler
restart: always
- "8180:8080" # REST API available on port 8180 to services outside the container
- "9997:9997" # UDP port for session connection events
- "9998:9998" # UDP port for task failure events
# Make config file accessible outside of container
- "./config:/nodeapp/config"
- "NODE_ENV=production"
driver: json-file
proton:butler goran$

Now it’s time to copy the certificates you have exported from the Qlik Sense QMC into the config/certificate folder.

You should also make a copy of the template configuration file available in the Github repository. Place the file in the config directory, rename it to “production.yaml” and edit it as needed for your local environment/systems.

You now have these files:

proton:butler goran$ pwd
proton:butler goran$ ls -la
total 8
drwxr-xr-x 4 goran staff 128 Sep 26 16:36 .
drwxr-xr-x+ 59 goran staff 1888 Sep 26 16:24 ..
drwxr-xr-x 4 goran staff 128 Sep 26 16:36 config
-rw-r--r-- 1 goran staff 565 Sep 26 16:25 docker-compose.yml
proton:butler goran$
proton:butler goran$ ls -la config/
total 8
drwxr-xr-x 4 goran staff 128 Sep 26 16:36 .
drwxr-xr-x 4 goran staff 128 Sep 26 16:36 ..
drwxr-xr-x 6 goran staff 192 Sep 26 16:36 certificate
-rw-r--r-- 1 goran staff 1861 Sep 26 16:36 production.yaml
proton:butler goran$
proton:butler goran$ ls -la config/certificate/
total 32
drwxr-xr-x 6 goran staff 192 Sep 26 16:36 .
drwxr-xr-x 4 goran staff 128 Sep 26 16:36 ..
-rw-r--r--@ 1 goran staff 1166 Sep 26 16:36 client.pem
-rw-r--r--@ 1 goran staff 1702 Sep 26 16:36 client_key.pem
-rw-r--r--@ 1 goran staff 1192 Sep 26 16:36 root.pem
proton:butler goran$

Almost there!

Run “docker-compose up” to create and start the container:

proton:butler goran$ docker-compose up
Pulling butler (ptarmiganlabs/butler:latest)...
latest: Pulling from ptarmiganlabs/butler
f189db1b88b3: Pull complete
3d06cf2f1b5e: Pull complete
687ebdda822c: Pull complete
99119ca3f34e: Pull complete
e771d6006054: Pull complete
b0cc28d0be2c: Pull complete
7225c154ac40: Pull complete
7659da3c5093: Pull complete
32189a059676: Pull complete
e5e9e893e38f: Pull complete
0e4e951cc7f3: Pull complete
22a23c1ada03: Pull complete
4ad2d8e68c8e: Pull complete
4f5cfbe37ef7: Pull complete
79c448ba92be: Pull complete
Digest: sha256:7b69a49e46677a3d8528c4a9bb29f4b82ebadcc9b708341e7d16f5ab31051ed7
Status: Downloaded newer image for ptarmiganlabs/butler:latest
Creating butler ... done
Attaching to butler
butler | 2018-09-26T19:02:01.847Z - debug: Server for UDP server: localhost
butler | 2018-09-26T19:02:01.851Z - info: REST server listening on http://[::]:8080
butler | 2018-09-26T19:02:01.871Z - info: UDP server listening on
butler | 2018-09-26T19:02:01.871Z - info: UDP server listening on
butler | 2018-09-26T19:02:01.875Z - info: Connected to MQTT server, with client ID mqttjs_d3b2b024
butler | 2018-09-26T19:02:01.918Z - info: MQTT message received
butler | 2018-09-26T19:02:01.918Z - info: qliksense/notification/handleExecutionResult/bffb268f-f87f-41c8-a5e7-f80ba6106ad4
butler | 2018-09-26T19:02:01.919Z - info: bffb268f-f87f-41c8-a5e7-f80ba6106ad4

That’s it — you now have a fully functioning Butler instance running. The REST API is available on http://localhost:8180 (assuming you ran the above commands on your local computer). We can test this by hitting the ping endpoint using cURL (open a new terminal and run the command there):

proton:~ goran$ curl "http://localhost:8180/v2/butlerPing"
{"response":"Butler reporting for duty"}
proton:~ goran$

Mission accomplished.

Next in turn is to do the same exercise for the rest of the members of the Butler family… Stay tuned.

Originally published at on September 26, 2018.




At Ptarmigan Labs we look at how the great business intelligence platform Qlik Sense can be made even better. We develop open source tools in this and other fields, but might spice things up with an occasional post about embedded electronics, Kubernetes and other cool stuff.

Recommended from Medium

Promise Technology Pegasus3 R4 RAID Storage REVIEW

Freelance, Boiler plate, Up-to-date and Alternatives with Antoine Wolff.

How to Install Git on your machine

Nuture the Bright Spots: Dutch Exploratory Workshop on Testing 2018

Download Quicktime Pro For Mac Yosemite

How to Create a Database Tables in Canonic

Tech Talk 5 — SMART System in MAP

How to Become an DevOps Engineer in 2020

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Göran Sander

Göran Sander

Sense sensei, hacker, maker, outdoor geek, family man. Thoughts expressed here and on are my own.

More from Medium

Setup MongoDB using docker-compose

CI-CD Operations with Github Actions

Build API with AWS Lambda, container, and API gateway

Container deployment with Aws lambda

Setup Custom AWS lambda (λ) function dependencies using Docker containers