Installing CouchDB 2.0 with HTTPS / SSL For Free

Using Haproxy and Certbot/Let’s Encrypt on Centos 7

I’ve wanted to start writing about some of the things I get up to as a web & app developer when running a small business. I try to have a broad knowledge ranging from server setups and configuration to front end coding. It’s nice to keep things interesting and my aim is to try and blog about setups and techniques that aren’t currently just a few taps of your keys on Google to find.

So here it goes…

Background

I’ve been on the search for an easier way to build apps using the latest technology. I began making apps a few years ago with Phonegap/Cordova and re-using some of the codes I use for websites, but it’s become cumbersome. Ionic 2, built on Cordova, looks very promising and is also using Angular 2 which is exciting!

As data is such a key part of apps, I wanted to see what the best solution was to move over from synchronising using a combination of SQLite, PHP and MySQL. I couldn’t find a good 2-way synchronising solution using an RDB which led me onto NoSQL document based databases.

I looked at the 3 most popular ‘Document store’ databases at http://db-engines.com/en/ranking so decided to look at:

  • MongoDB
  • Couchbase
  • CouchDB

MongoDB only has a master-slave synchronising option, which isn’t ideal, whereas Couchbase requires you to spend a significant chunk of money every year if you want to enable SSL/HTTPS. I really want to make sure that data is all encrypted as it is transferred so I chose to go with CouchDB. It’s created by Apache and seems very powerful. The only downside I can see is nice SQL-like query language which Couchbase and MongoDB have.

Installing CouchDB in conjunction with Haproxy gives will give you a fast and secure installation which is easily expandable with load balancing if you decide to go for a multi-cluster setup:

  1. Installing CouchDB 2.0 on Centos 7
  2. Run CouchDB as a Daemon
  3. Install and configure Haproxy
  4. Setup single node using Fauxton
  5. Installing CertBot / Let’s Encrypt
  6. Creating a Certbot hook for Haproxy
  7. Generate SSL certificate
  8. Setup automatic SSL certificate renewals
  9. Configure Haproxy for HTTPS

The Process

Step 1. Installing CouchDB 2.0 on Centos 7

There are a few articles out there explaining to use the EPEL repo and use yum to install CouchDB — I have EPEL with Centos 7 and there was no CouchDB package available. So a fairly simple manual install is required.

Starting with the official installation instructions is always a good first step: http://docs.couchdb.org/en/2.0.0/install/unix.html

Make sure you are have administrative privileges or write ‘sudo’ before this snippet:

yum install autoconf autoconf-archive automake \
curl-devel erlang-asn1 erlang-erts erlang-eunit \
erlang-os_mon erlang-xmerl help2man \
js-devel-1.8.5 libicu-devel libtool perl-Test-Harness

I found for my installation I also needed to install a couple of other dependancies

yum install gcc-c++ \
erlang-reltool \
erlang-erl_interface \
erlang-jiffy \
erlang-snappy

Let’s create the user that will be used by CouchDB

adduser -r --home /home/couchpotato -M --shell /bin/bash --comment "CouchDB Administrator" couchpotato

Next we have to make this user’s home directory and change to the newly created user. Additionally you shouldn’t really allow this user outside of the user’s own home directory

chown 751 /home
mkdir /home/couchpotato
chown -R couchpotato:couchpotato /home/couchpotato
sudo -i -u couchpotato

Time to download CouchDB, unpack, configure and make the files.

mkdir ~/tmp
cd ~/tmp
wget http://apache.mirror.digitalpacific.com.au/couchdb/source/2.0.0/apache-couchdb-2.0.0.tar.gz
tar -xzf apache-couchdb-2.0.0.tar.gz
cd apache-couchdb-2.0.0
./configure
make release

We should create a permanent directory for CouchDB and copy your final files to your home directory and update permissions.

cp -R rel/couchdb ~/
find ~/couchdb -type d -exec chmod 0770 {} \;
chmod 0644 ~/couchdb/etc/*

The default configuration for CouchDB is to bind to the localserver on 127.0.0.1. We will use Haproxy as the middleman between CouchDB running locally and your public IP.

Step 2. Run CouchDB as a Daemon

We want to be able to run CouchDB without keeping a terminal window open. Fortunately this isn’t hard to do as described here: https://www.jamescoyle.net/how-to/2527-add-systemd-startup-script-for-couchdb

CouchDB recommend “runinit”, but as it wasn’t available in the yum repo, and because this is so simple, I went with this option.

Log out of your couchpotato user and make sure you are a root user.

logout

I have made some modifications to the systemd service configuration. You can run this command below:

cat <<EOT > /etc/systemd/system/couchdb.service
[Unit]
Description=CouchDB 2.0 service
After=network.target

[Service]
Type=simple
User=couchpotato
ExecStart=/home/couchpotato/couchdb/bin/couchdb -o /dev/stdout -e /dev/stderr
Restart=on-failure
ExecStop=/bin/kill -TERM \$MAINPID
EOT

That will have created a file /etc/systemd/system/couchdb.service with the information required to run CouchDB as a service. As described in the original article you can now start CouchDB and run it every time your server reboots

systemctl daemon-reload
systemctl start couchdb.service
systemctl enable couchdb.service

You can also see the satus and most recent lines the server has output by running

systemctl status couchdb.service

Congratulations, your CouchDB server is now running in the background. You could also run the same commands in future with

service couchdb start/stop/status

Step 3. Install and Configure Haproxy

Haproxy was recommended to me on a freenode chat here http://webchat.freenode.net/?channels=couchdb by a CouchDB dev @janl — without the advice to use Haproxy, this installation would have loads of issues. Huge thanks!

Once put on the right path, this has made the world of difference and is a perfect solution to run CouchDB over HTTPS. I couldn’t find a lot of documentation about it, but I’ve been reliably informed that CouchDB is well tested using Haproxy and it will provide you will load balancing capabilities should you wish to have a multi-cluster setup.

So to get started with Haproxy, let’s move to our /tmp folder

cd /tmp

Make sure we have all the dependancies installed for Haproxy (the installation process until Haproxy has been installed is as seen here: https://gist.github.com/ryzy/1d22a3de8798732c2541)

yum install gcc perl pcre-devel zlib-devel

We also want to install OpenSSL 1.0.2 — there’s no need to remove the system version of OpenSSL. The following commands will fetch, unpack, then make and install OpenSSL 1.0.2

wget -O /tmp/openssl.tgz https://www.openssl.org/source/openssl-1.0.2-latest.tar.gz
tar -zxf /tmp/openssl.tgz -C /tmp
cd /tmp/openssl-*
./config --prefix=/usr --openssldir=/etc/ssl --libdir=lib no-shared zlib-dynamic
make
make install_sw

I have opted to install the latest version of Haproxy. You can always check and download the latest version using wget and the links available here: http://www.haproxy.org/#down

wget -O /tmp/haproxy.tgz http://www.haproxy.org/download/1.7/src/haproxy-1.7.1.tar.gz
tar -zxvf /tmp/haproxy.tgz -C /tmp
cd /tmp/haproxy-*
make \
TARGET=linux2628 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 \
USE_PCRE=1 USE_PCRE_JIT=1 USE_OPENSSL=1 \
SSL_INC=/usr/include SSL_LIB=/usr/lib ADDLIB=-ldl \
CFLAGS="-O2 -g -fno-strict-aliasing -DTCP_USER_TIMEOUT=18"
make install

Great, so now to complete the installation we just need to move a few files as describes here: https://www.upcloud.com/support/haproxy-load-balancer-centos/ — for the commands below, make sure you’re still in the haproxy unpacked tar location.

cp /usr/local/sbin/haproxy /usr/sbin/
cp examples/haproxy.init /etc/init.d/haproxy
chmod 755 /etc/init.d/haproxy

We need to add the user for haproxy as well

sudo useradd -r haproxy

Now we have Haproxy installed, we need to configure it. CouchDB comes with a sample configuration, so let’s copy it to the correct location before editing:

cp /home/couchpotato/tmp/apache-couchdb-*/rel/haproxy.cfg /etc/haproxy/haproxy.cfg

Time to configure Haproxy so you can access Fauxton web interface — these are the lines to modify so you can access CouchDB externally on port 15984:

vi /etc/haproxy/haproxy.cfg
#Bind inbound requests on port 15984
bind *:15984
#Update your first server to listen to CouchDB locally on port 5984
server couchdb1 127.0.0.1:5984 check inter 5s
#Comment out the other 2 servers - single node setup
#server couchdb2 127.0.0.1:25984 check inter 5s
#server couchdb3 127.0.0.1:35984 check inter 5s

Finally let’s start Haproxy so we can access our CouchDB Fauxton interface remotely.

service haproxy start

Now you should have CouchDB and Haproxy running to bind a public port to CouchDB running locally.

Step 4. Basic setup using Fauxton

As long as your firewall is not blocking port 15984 you should be able to being up the web interface in your browser: http://YOUR_IP:15984/_utils/

Go to Setup > Configure Single Node

Type in your desired administrator username and password then click Configure Node

Congratulations, you now have a CouchDB server running over HTTP with an admin user to prevent unauthorised changes.

You could test this by opening a new CLI window and running

curl -X PUT http://YOUR_IP:5984/newdatabase

You’ll receive the response {“error”:”unauthorized”,”reason”:”You are not a server admin.”}

Remember to restrict access to your databases by adjusting their permissions. You can do this via Fauxton. Go to your database then click permissions on the left.

Step 5. Installing CertBot / Let’s Encrypt

Now we have configured CouchDB, we need an SSL certificate. Let’s Encrypt is a “Free, Automated and Open Certificate Authority” — in short it’s amazing! It’ll give you a free SSL certificate (up to 5 certificates every 7 days) and each certificate lasts 90 days.

You will need to have a domain name though.

If you already have a domain elsewhere, just configure a subdomain to point to your server in your DNS settings (if a domain isn’t already pointing to the server’s IP).

To get a certificate, you need to install CertBot. It is as easy as running the following as running the following with administrative privileges (start the command with ‘sudo’ if you need to) https://certbot.eff.org/all-instructions/#centos-rhel-7-none-of-the-above

yum install certbot

You’re ready to create your SSL certificate! Before we start using up our certificate allowance, let’s also create a bash script that will be run when you generate and renew the certificate.

Step 6. Create CertBot hook for Haproxy — Bash script

Haproxy will need your fullchain.pem and privkey.pem in a single file. So we need to write a very simple script that can sort this our for us. Firstly, let’s create a new empty certificate file.

touch /etc/haproxy/cert-haproxy.pem
chmod 600 /etc/haproxy/cert-haproxy.pem

Now you have the empty file with the correct permissions, we can create a script that will run when CertBot generates or renews a certificate.

REMEMBER: Replace YOUR_DOMAIN to the domain name that you have pointed at your server’s IP address
cat <<EOT > /etc/haproxy/cert-hook
#!/bin/sh
DOMAIN="YOUR_DOMAIN"
FULL_PEM="/etc/haproxy/cert-haproxy.pem"
echo "Certbot renewal hook running as user: '$USER'..." >&2
echo "RENEWED_DOMAINS=$RENEWED_DOMAINS" >&2
echo "RENEWED_LINEAGE=$RENEWED_LINEAGE" >&2

if grep --quiet "$DOMAIN" <<< "$RENEWED_DOMAINS"; then
cat $RENEWED_LINEAGE/fullchain.pem $RENEWED_LINEAGE/privkey.pem > $FULL_PEM
echo "PEM updated $FULL_PEM" >&2
systemctl restart haproxy
echo "Haproxy restarted" >&2
fi
EOT

Next, make the script executable

chmod +x /etc/haproxy/cert-hook

Voila. This script will now copy the your certificate including the full CA chain and your private key into a file for Haproxy to use. It will also restart Haproxy so your new certificates will be used straight away.

Step 7. Generating your SSL Certificate

For my purposes, the server I installed CouchDB on is not a web server. Therefore I am able to run CertBot in standalone mode which temporarily starts a server on port 443. For other modes see: the CertBot instructions: https://certbot.eff.org/ (there is automation for Nginx and Apache if you are already running a web server, please be aware you may need to modify the renew-hook script to temporarily shutdown your web services as the certificate is requested and validated)

The command I ran for these circumstances was

replace “YOUR_DOMAIN” with your domain name
certbot certonly --standalone -d YOUR_DOMAIN --renew-hook "/etc/haproxy/cert-hook"

CertBot will then generate and authorise your certificate. It will save a symbolic link to your generated files here: /etc/letsencrypt/live/YOUR_DOMAIN/. Your “renew-hook” script will copy the contents of the certificate, CA certificate and private key to /home/couchpotato/couchdb/certs/ after they have been created.

Step 8. Configure SSL certificate automatic renewals

Change to a root user and create a CertBot service which will run your renewal script:

cat <<EOT > /etc/systemd/system/certbot.service 
[Unit]
Description=Lets Encrypt Automated Renewal

[Service]
Type=oneshot
ExecStart=/usr/bin/certbot renew --quiet --agree-tos --renew-hook "/etc/haproxy/cert-hook"
EOT

And then create your certbot.timer file:

cat <<EOT > /etc/systemd/system/certbot.timer
Description=Daily renewal of Let's Encrypt's certificates

[Timer]
OnCalendar=daily
RandomizedDelaySec=1day
Persistent=true

[Install]
WantedBy=timers.target
EOT

Enable the timer:

systemctl daemon-reload
systemctl enable /etc/systemd/system/certbot.timer

For more info: https://wiki.archlinux.org/index.php/Let%E2%80%99s_Encrypt

You can run your renewal script to test it if you want to, but remember you only get 5 certificates every 7 days:

/usr/bin/certbot renew --force-renew --agree-tos --renew-hook "/etc/haproxy/cert-hook"

You can also check that your timer is running with the command:

systemctl list-timers

Step 9. Configure Haproxy for HTTPS

Now we have our certificate file we can easily reconfigure Haproxy to use SSL encryption. I was directed to this as a standard SSL configuration for Haproxy within the chat: https://gist.github.com/rnewson/8384304

vi /etc/haproxy/haproxy.cfg
#Update your bind configuration within frontend http-in
bind *:15984 ssl crt /etc/haproxy/cert-haproxy.pem no-tls-tickets ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA384:AES128-SHA256:AES128-SHA:AES256-SHA256:AES256-SHA:!MD5:!aNULL:!DH:!RC4
#Add these lines beneath bind, still within http-in
reqadd X-Forwarded-Proto:\ https
# Distinguish between secure and insecure requests
acl secure dst_port eq 15984
# Mark all cookies as secure if sent over SSL
rsprep ^Set-Cookie:\ (.*) Set-Cookie:\ \1;\ Secure if secure
# Add the HSTS header with a 1 year max-age
rspadd Strict-Transport-Security:\ max-age=31536000 if secure
# Redirect HTTP to HTTPS
redirect scheme https code 301 if !{ ssl_fc }

Now we just have to restart haproxy and make sure it starts up when you restart your server next.

service haproxy restart
chkconfig haproxy on

Voila! You now have a CouchDB single node instance setup so it can only be accessed remotely via HTTPS on port 15984

Other Considerations

Additionally you’ll probably want to setup additional users and roles: http://docs.couchdb.org/en/2.0.0/intro/security.html#authentication-database