Hosting a Wagtail site on Digital Ocean with CapRover.

Silicon chip. Image © Chris Guy

I’ve recently been working with the open source Django based Wagtail CMS and wanted to share my experience of setting up a development environment using PostgreSQL with Docker and then creating a production instance hosted on Digital Ocean using CapRover.

Note: the steps below have been tested using an M1 Mac with Python 3.10.4 installed.

Initialise local dev environment

In this example I first want to create a Docker container for the Wagtail application and PostgreSQL database.

Create a new local directory for your application and cd into the directory using your preferred terminal app.

Setup and activate the virtual environment

python3 -m venv venv
source venv/bin/activate

Use pip to install pipenv

pip install --upgrade pip
pip install pipenv

Create a new top-level folder named mysiteand then install Wagtail and django-environ with pipenv. Note, I’m using pipenv to ‘pin’ the package versions that I wish to use.
Start a new app named base within mysite

pipenv install wagtail==3.0.1 django-environ==0.9.0 
wagtail start base mysite

Finally, deactivate and remove the virtual environment. This is no longer needed as the environment will be containerised in the next section using Docker.

deactivate
rm -r venv

Dockerise

cd into the mysite folder.
Important: the mysite directory is the site root and must also be the git root directory.

Move pipfile and pipfile.lock into mysite

Modify the supplied Dockerfile to fit your own requirements. My final Dockerfile looks like this:

# Use an official Python runtime based on Debian 10 "buster" as a parent image.
FROM python:3.10-slim-buster

# Create and set the directory where the source code is stored.
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# Set environment variables. Note language set to en-GB.
ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV LANG en_GB.UTF-8
ENV PYTHONIOENCODING utf_8
# Install system packages required by Wagtail, Django and Psycopg2.
RUN apt-get update --yes --quiet && apt-get install --yes --quiet --no-install-recommends \
    build-essential \
    libpq-dev \
    libmariadbclient-dev \
    libjpeg62-turbo-dev \
    zlib1g-dev \
    libwebp-dev \
    python-dev \
    && rm -rf /var/lib/apt/lists/*

# Copy the source code and install dependencies
COPY Pipfile Pipfile.lock /usr/src/app/
RUN pip install --upgrade pip
RUN pip install pipenv && pipenv install --system

Build the container with docker build .

Add a local build script

Create a directory named utils at the root of your project and add a shell script file named run_local.sh then add the following to the shell script:

#!/bin/sh

echo 'Starting server'
python manage.py runserver 0.0.0.0:80 --settings="$DJANGO_SETTINGS_MODULE"

Docker Compose

Compose is a tool for defining and running multi-container Docker applications. In this example we will use a docker-compose YML file to define and run the Wagtail application and Postgres database and set a persistent volume to hold the database data.

Create a new file at the root of the project named docker-compose.yml

Add the definitions and environment variables. My final version looks like this:

version: '3.8'
services:
  web:
    build:
      context: .
      dockerfile: Dockerfile
    command: /bin/sh "utils/run_local.sh"
    volumes:
      - ./:/usr/src/app
    links:
     - postgres
    ports:
     - "80:80"
    environment:
      - DEBUG=True
      - ENVIRONMENT=${ENVIRONMENT}
      - DJANGO_SETTINGS_MODULE=${DJANGO_SETTINGS_MODULE}
      - DATABASE_URL=${DATABASE_URL}
      - SECRET_KEY=${SECRET_KEY}
      - BASE_URL=${BASE_URL}

  postgres:
    image: postgres:14
    ports:
      - "5432:5432"
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    volumes:
      - db_data:/var/lib/postgresql/data/

volumes:
  db_data: {}

Environment variables

I’m using a .env file to securely store the environment variables. To do this create an empty .env file to the root of your project.

** Important ** ensure that .env is added to the .gitignore file to prevent the sensitive information being added to your repository.

Add the password and other credentials to the .env file as the example below

ENVIRONMENT=local
DJANGO_SETTINGS_MODULE=mysite.settings.dev
SECRET_KEY=secretkey-goes-here
POSTGRES_USER=database-user-name-goes-here
POSTGRES_PASSWORD=database-password-goes-here
POSTGRES_DB=database-name-goes-here
DATABASE_URL=postgres://user:password@host:port/database
BASE_URL=http://localhost
WAGTAILADMIN_BASE_URL=http://localhost

Next, open the settings files in mysite/settings and add these imports to the top of base.py

import environ
from django.core.exceptions import ImproperlyConfigured

add this section to base.py just below the import statements

ENV_PATH = '/'
ENVIRONMENT = os.getenv('ENVIRONMENT', 'production')

env = environ.Env(
    # set casting, default value
    DEBUG=(bool, False)
)
# only read from .env if local or test
if ENVIRONMENT == 'local' or ENVIRONMENT == 'test':
    env.read_env(env.str('ENV_PATH', '.env'))

# False if not in os.environ
DEBUG = env('DEBUG')

then change the database settings as follows:

DATABASES = {
    'default': env.db(),
}

Still in settings, open dev.py and change the SECRET KEY line to read

SECRET_KEY = os.getenv('SECRET_KEY', ImproperlyConfigured('SECRET_KEY not set'))

Psycopg2 (database adapter)

We are going to use the source distribution of psycopg2 and not psycopg-binary as this is intended to be a production site. The build prerequisites have been included in the Dockerfile described above.

First start the server with:

docker-compose up

And then install the database adapter psycopg2 using the following in a separate terminal window.

docker-compose exec web pipenv install psycopg2==2.9.3

Once the database adapter has been installed, return to the first terminal winodw, stop the server (control+c) and rebuild using:

docker-compose up --build

Create Custom image model (optional)

At this point you may want to consider creating a custom image model which allows additional fields to be added to images. For further info see the Wagtail documentation.

Run migrations and create an admin user

Start the local server:

docker-compose up --build

And run migrations (in a new terminal window):

docker-compose exec web python manage.py migrate

Create an admin user:

docker-compose exec web python manage.py createsuperuser

You should now be able to open the local site at http://localhost

Basic Wagtail installation running in Docker

And open the admin site at http://localhost/admin

Wagtail Admin Site

Gunicorn

Gunicorn is a Python WSGI HTTP Server for UNIX. Version 20.1.0 can be installed from PyPI using:

docker-compose exec web pipenv install gunicorn==20.1.0

To configure Gunicorn for use in production, create a new shell script file in the utils directory named run.sh then add this config script to the file:

#!/bin/sh

python manage.py collectstatic --noinput
python manage.py migrate
python manage.py update_index

gunicorn base.wsgi --bind=0.0.0.0:80 --timeout 10000 --workers 3

Note the variables in the above configuration :-

• ‘base’ is the name of the app created earlier
• timeout is set to 10000 Milliseconds
• number of workers is set to 3 (calculated using workers = (2 * cpu) + 1)

Setup CapRover and Digital Ocean

The next step is to setup CapRover and your Digital Ocean hosting. The instructions for this are detailed in the Getting Started page. I would recommend using the one-click app on the Digital Ocean marketplace and by taking this route you can skip stright to Step 2: Connect Root Domain

Outline of basic steps:

1. Register a domain name (e.g. example.com)
2. Purchase a cloud server instance from Digital Ocean and install CapRover. Click the ‘Create a Droplet’ button on the CapRover getting started page or search for ‘CapRover’ in the Digital Ocean marketplace.
   Select your hosting options and create a droplet.
3. Assign a ‘Reserved IP’ via the Digital Ocean Networking menu (optional)
4. Set up a non-root user and configure SSH access. See Initial Server Setup with Ubuntu 18.04.
5. Connect the root domain – configure the DNS of your domain to set *.something.example.com to point to the IP address of your Digital Ocean droplet or to the Reserved IP if you set one up. In this example I’m using *.wagtail.example.com
6. Install CapRover CLI on your local machine. You should now be able to login to the CapRover server dashboard at https://captain.wagtail.example.com

CapRover Server Dashboard

Create the database

Before deploying the code, we need to create the database. From the CapRover server dashboard click Apps, then open One-Click Apps/Databases and search for ‘postgresql’.

Select PostgreSQL from the One Click Apps

Give the PostgreSQL app a name. This should be different from your application name.

Set the Postgres version (14 in this example) and choose a username, password and default database name. These credentials will be used by your application to connect to the database. Click Deploy.

Once the PostgreSQL app has been created, click on the app name to edit and go to App Configs. Click add Port Mapping and map an unused server port to the database container port (5432) e.g.

Port Mapping example

Click Save & Update.

Add the environment variables

Firstly we need to update the production settings for the mysite app.

I’ve updated settings/production.py with the following settings:

from .base import *

try:
    from .local import *
except ImportError:
    pass

SECRET_KEY = os.getenv('CR_SECRET_KEY', ImproperlyConfigured('CR_SECRET_KEY not set'))

hosts = os.getenv('CR_HOSTS', ImproperlyConfigured('CR_HOSTS not set'))
try:
    ALLOWED_HOSTS = str(hosts).split(',')
except Exception as e:
    raise ImproperlyConfigured('CR_HOSTS could not be parsed. {}'.format(e))


# Production security settings
SECURE_BROWSER_XSS_FILTER = True
X_FRAME_OPTIONS = 'DENY'
SECURE_SSL_REDIRECT = True
SECURE_HSTS_SECONDS = 31536000
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_REFERRER_POLICY = 'same-origin'
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

Next, login to the CapRover server dashboard, click Apps and type a name for your application, I’m using production as my app name in this example.
Click Create New App.

Once the app has been created, click the name to edit and go to App Configs. Add the following environmental variables:

CR_SECRET_KEY=your-secret-key-goes-here
ENVIRONMENT=production
DEBUG=False
DJANGO_SETTINGS_MODULE=base.settings.production
CR_HOSTS=production.wagtail.example.com,example.com,
DATABASE_URL=postgres://user:password@host:port/database
BASE_URL=https://example.com
WAGTAILADMIN_BASE_URL=https://example.com

The CR hosts should be a list of URLs that your site will be available on. For the above configuration to work the DNS for example.com would need to be changed so that the A record points to the server IP address.

The port number in the database URL should be the one you choose in the port mapping section above (8240 in this example).

Add domains and enable HTTPS

In the CapRover server dashboard open the setting for your ‘production’ app.
Click ‘Enable HTTPS’ for your production.wagtail.example.co.uk domain. At this point you can also add the base domain example.com and enable HTTPS for that as well.

Captain Definition file
The next step is to create the captain-definition file. This is a key component of the CapRover system is explained here.

At the project root create a new file named captain-definition and populate it with the lines from your Dockerfile. My finished example looks like this:

{
  "schemaVersion": 2,
  "dockerfileLines": [
    "FROM python:3.10-slim-buster",
    "RUN mkdir -p /usr/src/app",
    "WORKDIR /usr/src/app",
    "ENV PYTHONUNBUFFERED 1",
    "ENV PYTHONDONTWRITEBYTECODE 1",
    "ENV LANG en_GB.UTF-8",
    "ENV PYTHONIOENCODING utf_8",
    "RUN apt-get update --yes --quiet && apt-get install --yes --quiet --no-install-recommends build-essential libpq-dev libmariadbclient-dev libjpeg62-turbo-dev zlib1g-dev libwebp-dev python-dev && rm -rf /var/lib/apt/lists/*",
    "COPY . /usr/src/app/",
    "RUN pip install --upgrade pip",
    "RUN pip install pipenv && pipenv install --system",
    "COPY ./utils/ /usr/src/utils",
    "EXPOSE 80",
    "CMD sh /usr/src/utils/run.sh"
  ]
}

Deploy the code

In your terminal app cd into your application root directory then run

caprover deploy

and follow the on-screen prompts to deploy your code to the production app.

You should now be able to view your site at https://production.wagtail.example.com/ and https://example.com

Wagtail production site sucessfully deployed

If things aren’t working as expected, click on the production app within the CapRover server dashboard and go to the Deployment tab. Here you will see the application logs. In depth troubleshooting is available on this page.

Resources and thanks

All the code from this demo is available here.

Django for Professionals by William S Vincent has proved invaluable to me — well worth a purchase.

The CapRover Slack is a good soource for community information. As of course is the Wagtail Slack channel.

Lastly big thanks to Russell Kirkland for all the help :)