A Simple Recipe for Django Development In Docker (Bonus: Testing with Selenium)

adam king
adam king
Jan 28, 2018 · 9 min read

Docker brings a host of advantages to software development, but I like it for two reasons: it decouples your code from the host operating system, and it allows you to easily plug in companion servers which can run a database or testing solutions like Selenium Grid Hub.

With Docker, you can run your Django project on an Ubuntu server in a container on your laptop, and because Docker is available for Mac, Linux, and Windows, your choice of operating system really comes down to preference. When it comes time to push your code to a staging or production server, you can be sure it’ll run exactly the same as it did on your laptop, because you can configure a Dockerfileto exactly match these environments.

docker-compose, in combination with Docker, allows you to create a network of servers on your dev box. You can wire your Django app to a dedicated database from the get-go, instead of building your application on SQLite and then migrating over later. It’s also easy to add in new servers or features: drop in a Redis database, maybe play around with a load balancer or cache. I’ve recently started using Selenium Grid for testing, and it’s easy with docker-compose. There are pre-built Docker images for pretty much anything you can think of, and the risks of experimentation are extremely low — just delete the entry from your docker-compose.yml file if you decide not to use it.

First Steps — the Dockerfile

Let’s get started and build a Dockerfile which will be used to create the container that’ll hold your Django project.

# My Site
# Version: 1.0
FROM python:3# Install Python and Package Libraries
RUN apt-get update && apt-get upgrade -y && apt-get autoremove && apt-get autoclean
RUN apt-get install -y \
libffi-dev \
libssl-dev \
libmysqlclient-dev \
libxml2-dev \
libxslt-dev \
libjpeg-dev \
libfreetype6-dev \
zlib1g-dev \
net-tools \
vim
# Project Files and Settings
ARG PROJECT=myproject
ARG PROJECT_DIR=/var/www/${PROJECT}
RUN mkdir -p $PROJECT_DIR
WORKDIR $PROJECT_DIR
COPY Pipfile Pipfile.lock ./
RUN pip install -U pipenv
RUN pipenv install --system
# Server
EXPOSE 8000
STOPSIGNAL SIGINT
ENTRYPOINT ["python", "manage.py"]
CMD ["runserver", "0.0.0.0:8000"]

Without getting too deep in the weeds about creating Dockerfiles, let’s take a quick look at what’s going on here. We specify some packages we want installed on our Django server (The Ubuntu image is pretty bare-bones, it doesn’t even come with ping!). The WORKDIR variable is interesting — in this case it’s setting /var/www/myproject/ on the server as the equivalent to your Django project’s root directory. We also expose port 8000 and run the server.

Note that in this case, we’re using pipenv to manage our package dependencies. If you’re not using pipenv, you’ll want to build a requirements.txt file and substitute the corresponding lines with something like this:

COPY requirements.txt .
RUN pip install -r requirements.txt

docker-compose lends a hand

Now that we’ve got a Dockerfile, we could build our image on the command line and spin it up, but because we plan to use docker-compose, let’s go ahead and start a simple docker-compose.yml:

version: "2"
services:
django:
container_name: django_server
build:
context: .
dockerfile: Dockerfile
image: docker_tutorial_django
stdin_open: true
tty: true
volumes:
- .:/var/www/myproject
ports:
- "8000:8000"

Now we can run docker-compose build and it’ll build our image which we named docker_tutorial_django that will run inside a container called django_server. Spin it up by running docker-compose up. You’ll see something like this in your console:

It works!

Open up a web browser and point it to 127.0.0.1:8000 and you’ll see the home page. Awesome!

An interactive debugger inside a running container!

Before we go any further, take a quick look at that docker-compose.yml file. The lines,

stdin_open: true
tty: true

are important, because they let us run an interactive terminal. Hit ctrl-c to kill the server running in your terminal, and then bring it up in the background with docker-compose up -d. docker ps tells us it’s still running:

We need to attach to that running container, in order to see its server output and pdb breakpoints. The command docker attach django_server will present you with a blank line, but if you refresh your web browser, you’ll see the server output. Drop import pdb; pdb.set_trace() in your code and you’ll get the interactive debugger, just like you’re used to.

Explore your container

With your container running, you can run the command docker-compose exec django bash, which is a shorthand for the command docker exec -it django_server bash. You’ll be dropped into a bash terminal inside your running container, with a working directory of /var/www/myproject, just like you specified in your Docker configuration. This console is where you’ll want to run your manage.py tasks: execute tests, make and apply migrations, use the python shell, etc.

Take a break

Before we go further, let’s stop and think about what we’ve accomplished so far. We’ve now got our Django server running in a reproducible Docker container. If you have collaborators on your project or just want to do development work on another computer, all you need to get up and running is a copy of your Dockerfile, docker-compose.yml, and requirements.txt or Pipfile. You can rest easy knowing that the environments will be identical. When it comes time to push your code to a staging or production environment, you can build on your existing Dockerfile — maybe add some error logging, a production-quality web server, etc.

Next Steps: Add a MySQL Database

Now, we could stop here and we’d still be in a pretty good spot, but there’s still a lot of Docker goodness left on the table. Let’s add a real database. Open up your docker-compose.yml file and update it:

version: "2"
services:
django:
container_name: django_server
build:
context: .
dockerfile: Dockerfile
image: docker_tutorial_django
stdin_open: true
tty: true
volumes:
- .:/var/www/myproject
ports:
- "8000:8000"
links:
- db
environment:
- DATABASE_URL=mysql://root:itsasecret@db:3306/docker_tutorial_django_db
db:
container_name: mysql_database
image: mysql/mysql-server
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=itsasecret
volumes:
- /Users/Adam/Development/data/mysql:/var/lib/mysql

We added a new service to our docker-compose.yml called db. I named the container mysql_database, and we are basing it off the image mysql/mysql-server. Check out http://hub.docker.com for, like, a million Docker images. We set the root password for the MySQL server, as well as expose a port (host-port:container-port) to the ‘outer world.’ We also need to specify the location of our MySQL files. I’m putting them in a directory called data in my Development directory.

In our django service, I added a link to the db service. docker-compose acts as a sort of ‘internal DNS’ for our Docker containers. If I run docker-compose up -d and then jump into my running Django container with docker-compose exec django bash, I can ping db and confirm the connection:

root@e94891041716:/var/www/myproject# ping db
PING db (172.23.0.3): 56 data bytes
64 bytes from 172.23.0.3: icmp_seq=0 ttl=64 time=0.232 ms
64 bytes from 172.23.0.3: icmp_seq=1 ttl=64 time=0.229 ms
64 bytes from 172.23.0.3: icmp_seq=2 ttl=64 time=0.247 ms
64 bytes from 172.23.0.3: icmp_seq=3 ttl=64 time=0.321 ms
64 bytes from 172.23.0.3: icmp_seq=4 ttl=64 time=0.310 ms
^C--- db ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.229/0.268/0.321/0.040 ms
root@e94891041716:/var/www/myproject#

Adding the environment variable, DATABASE_URL=mysql://root:itsasecret@db:3306/docker_tutorial_django_db Will allow our Django database to use a real, production-ready version of MySQL instead of the default SQLite. Note that you’ll need to use a package like getenv in your settings.py to read environment variables:

DATABASE_URL=env('DATABASE_URL')

If it’s your first time running a MySQL server, you might have a little bit of housekeeping: setting the root password, granting privileges, etc. Check the corresponding documentation for the server you’re running. You can jump into the running MySQL server the same way:

$ docker-compose exec db bash
$ mysql -p itsasecret
> CREATE DATABASE docker_tutorial_django_db;
etc, etc

Cool, right? Say goodbye to any database migration headaches when it comes time to push your code to a staging or production server.

Bonus Section! Testing with Selenium Grid

Selenium Webdriver is the industry standard for UI testing, and with Docker, it’s easy to add a fully-functional Selenium test suite into your project. Let’s pop into our docker-compose.yml file and add three more services:

version: "2"
services:
django:
container_name: django_server
build:
context: .
dockerfile: Dockerfile
image: docker_tutorial_django
stdin_open: true
tty: true
volumes:
- .:/var/www/myproject
ports:
- "8000:8000"
links:
- db
environment:
- DATABASE_URL=mysql://root:itsasecret@db:3306/docker_tutorial_django_db
db:
container_name: mysql_database
image: mysql/mysql-server
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=itsasecret
volumes:
- /Users/work/Development/data/mysql:/var/lib/mysql
selenium_hub:
container_name: selenium_hub
image: selenium/hub
ports:
- "4444:4444"
selenium_chrome:
container_name: selenium_chrome
image: selenium/node-chrome-debug
environment:
- HUB_PORT_4444_TCP_ADDR=selenium_hub
- HUB_PORT_4444_TCP_PORT=4444
ports:
- "5900:5900"
depends_on:
- selenium_hub
selenium_firefox:
container_name: selenium_firefox
image: selenium/node-firefox-debug
environment:
- HUB_PORT_4444_TCP_ADDR=selenium_hub
- HUB_PORT_4444_TCP_PORT=4444
ports:
- "5901:5900"
depends_on:
- selenium_hub

Here we’ve added three services based upon the following Docker images: selenium/hub, selenium/node-chrome-debug, and selenium/node-firefox-debug. Run docker-compose up -d and then docker ps. You should now have five running docker containers. If you jump into your Django container ( docker-compose exec django bash), you can ping selenium_hub to see that it’s been added to your Docker network. On your web browser, navigate to http://127.0.0.1:4444 and you’ll see the built-in web server that comes with Selenium Grid Hub. Click the link at the top right, “Perhaps you are looking for the Selenium Grid Console,” and you should see that two nodes are connected to the Selenium Grid — one running Chrome, one running Firefox. Using Selenium Webdriver, we can run tests on either browser, or both.

The Chrome and Firefox images, in addition to functioning as Selenium nodes, also each run a VNC server which you can use to watch your Selenium tests as it drives the corresponding web browsers. macOS comes with a simple VNC client built-in called ‘Screen Sharing’, but there are many clients available. We mapped those servers to ports 5900 and 5901, respectively, so you can just fire up a VNC client and connect (the password is secret) to 127.0.0.1:5900 or 127.0.0.1:5901 to watch the webdriver in action . Note that if you’re not interested in using VNC to see the output in real-time, there are Docker images called selenium/node-chrome and selenium/node-firefox that you can use instead. You can still issue Selenium commands which’ll take a screenshot for you.

Write a Test!

Let’s write a quick test to watch it all work.

from django.test import TestCase
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
class SeleniumTest(TestCase): def setUp(self):
self.chrome = webdriver.Remote(
command_executor='http://selenium_hub:4444/wd/hub',
desired_capabilities=DesiredCapabilities.CHROME
)
self.chrome.implicitly_wait(10)
self.firefox = webdriver.Remote(
command_executor='http://selenium_hub:4444/wd/hub',
desired_capabilities=DesiredCapabilities.FIREFOX
)
self.firefox.implicitly_wait(10)
def test_visit_site_with_chrome(self):
self.chrome.get('http://django:8000')
self.assertIn(self.chrome.title, 'Django: the Web framework for perfectionists with deadlines.')
def test_visit_site_with_firefox(self):
self.firefox.get('http://django:8000')
self.assertIn(self.firefox.title, 'Django: the Web framework for perfectionists with deadlines.')

Open up a VNC session to 127.0.0.1:5900 and 127.0.0.1:5901 and from your Django container, run python manage.py test to watch webdriver in action! Now it’s easy to quickly run through your application with both Chrome and Firefox. Unfortunately at this time, we don’t have images which can run Safari or Internet Explorer browsers. Maybe in the future?

A few final notes

You will need to make sure that your containerized server is permitted in your ALLOWED_HOSTS setting in settings.py:

ALLOWED_HOSTS = ['*'] if DEBUG else [...staging/production/externally accessible hosts here]

Selenium tests take a good deal of time. I recommend adding a tag to your Selenium tests:

from django.test import TestCase, tagclass SeleniumTest(TestCase):  @tag('selenium')
def setUp...
@tag ('selenium')
def test_visit_site...

That way, you can exclude those tests unless you want to see them:

python manage.py test --exclude-tag=selenium

Developing a Django application with Docker and docker-compose provides reassurance that what you’re building on your dev box will work straight away in a production environment. It frees you up to use whatever operating system you choose as a development environment. It makes your job easier when collaborating with other devs. It encourages experimentation: just add an entry into your docker-compose.yml file. When combined with Selenium Webdriver, you can add integration tests to your unit-tested models, forms, serializers, and views. Give it a try, and let me know if you agree.

zeitcode

We build great software: Web/iOS/Android

adam king

Written by

adam king

zeitcode

zeitcode

We build great software: Web/iOS/Android