Starting Serverless with Django

8 min readDec 23, 2018

Hi! If you want to jump straight into the code, it’s all right here: https://github.com/codeluggage/django-serverless

Each of the steps in this post are commits in the repo, so you can follow along if you like. Here are the overview of the steps we’ll go through:

The concept of Functions as a Service is beautiful: Abstract away the management of server infrastructure - spend that time on the application instead.

One of the many reasons I love Serverless is because it makes life simpler. Despite the extreme power we are wielding, all we really need is a yaml file, and a function.

The yamlfile describes the services we want to use, the functions we want to deploy, and the access they have. The functions are just code.

Serverless is powerful, yet simple. Photo by Jaelynn Castillo on Unsplash

Django and FaaS (Functions as a Service) are great together. Zappa, an AWS + Python framework supports Django out of the box. It is an excellent project and I recommend it - if you only ever plan to use AWS and Python.

As we want to stay cloud agnostic, we’ll use the Serverless Framework instead. Today you can deploy to 8 different cloud providers, including the very flexible Kubeless. Because you can deploy Kubeless anywhere, it’s more like (7+n) cloud providers.

Each cloud provider will have a slightly different configuration, but still similar. The complexity of deploying our code to the different cloud providers has been abstracted away - like the cogs in a clock.

FaaS is complex on the inside. Photo by Shane Aldendorff on Unsplash

You can follow these steps with your own existing Django project, or create a new one. If you are using your own, simply replace serverless-django with your own Django project name.

Creating a new Serverless project

To begin a new Serverless project, the easiest way is to simply run the serverles create (or sls for short) command. It takes a template or a remote URL to a template, and a name. The template says which cloud provider to use - each of which requires its own authentication. Here are some examples:

sls create --template=openwhisk-python --path=serverless-django
sls create --template=aws-python3 --path=serverless-django
sls create --template=spotinst-python --path=serverless-django

Once we run the command for the cloud provider of our choice, we’ll have a new folder called serverless-django with a Python project. Beware: if you don’t give it a name (serverless-django in this case) then the project will be created in the current folder. It creates a .gitignore file which would overwrite your own .gitignore file - if you have one.

Environment variables

Applications almost always require secrets of some kind. Usernames, passwords, private keys - the list goes on. Django requires several, and the solution is elegant in Serverless. Either…

1) Place all the environment variables in its own file (yay!)

2) Put each environment variable in serverless.yml (nay!)

Option 1 lets us store all our secrets in one place. We can keep it out of source control (for example by adding it to a .gitignore file), so we don’t upload it by accident. We can store it in a password vault, letting us share it securely with our team and onboard new members easily.

The choice is clear: we’ll go with option 1 and place all our environment variables in its own file, which we’ll call variables.yml. Let’s create the file in the same folder as serverless.yml, and add one environment variable called THE_ANSWER to it:

THE_ANSWER: 42

Next, let’s add a reference to it in our serverless.yml so our functions can read it. At the top level of your serverless.yml, indented all the way to the left, add:

custom: ${file(./variables.yml)}

This makes all the variables from our variables.yml file available with ${self:custom}. Now we can make these variables available to our function. Under your function hello:, add:

environment: ${self:custom}

This adds the entire reference to custom to our function. We can now access all our variables in our function! After these changes, our serverless.yml looks something like this (depending on which cloud provider you chose):

A serverless.yml file with environment variables

Let’s see how this would look if we only wanted to add a single variable. Instead of referring to the entire ${self:custom}, be specific:

environment:
  THE_ANSWER: ${self:custom.THE_ANSWER}

This also lets you use a different name for the environment variable inside the function, so it’s not always the same as the one in your variables.yml file.

Now, let’s make sure everything works like expected. We’ll print the value of the environment variable from inside the function. In our handler.py we’ll use the os module to access the environment variable. Replace the entire handler.py file with this:

Our new handler.py, returning THE_ANSWER

Deploying

Now we can deploy and then run the function to see the answer in the cloud:

sls deploy
sls invoke -f hello

This gives us: “The answer is: 42” — which it is, of course.

To recap, so far we have:

Set up Serverless, deployed a function and executed it
Set up a safe way to use secret environment variables
Accessed environment variables in our code

These pieces set the stage for the main event: Django!

Packages

If you’re adding Serverless to an existing Django project, feel free to skip down to the Plugins section.

Serverless uploads our local folder to the cloud provider, so it can all run in the function we define. To install everything in the environment we want, we’ll use virtualenv. Let’s install Django so it can be used in our function:

virtualenv venv --python=python3
source venv/bin/activate
pip install django

The default database SQLite is a simplistic database that saves to disk, often used on mobile devices. That is not what we want for this project. 1) it is not as compatible with cloud provider docker images which is where functions run and 2) it is not the kind of database we would want to use in production. Instead, we will use PostgreSQL:

pip install psycopg2-binary

Everything installed with pip is not saved anywhere except the installation in virtualenv folder. To properly store what we have installed, we can call pip freeze to get a full overview. The standard convention is to save this to a file called requirements.txt so we can check it into source control:

pip freeze > requirements.txt

Creating a Django project

Now that we have PostgreSQL available, we can configure Django to use it. Let’s create our Django project, and call it something like mydjangoproject. It can have any name, as long as it’s not a reserved word like python that confuses the system.

django-admin startproject mydjangoproject .

This gives us a full Django project in the mydjangoproject folder. The two most interesting files for us are mydjangoproject/settings.py and manage.py. To change the database, we’ll replace the entire DATABASES object in mydjangoproject/settings.py with this:

Using environment our variables, we tell Django to use PostgreSQL.

Now all we need to do is define these environment variables we’re using: DB_USER, DB_PASSWORD, DB_HOST and DB_PORT, and we are up and running. Anything you add to your variables.yml file is available by calling os.environ.get() after deploying.

To connect, you need an existing database running and open to outside connections. In general, functions as a service do not have a set IP that they will always run from. This makes it harder to restrict access so only your function can access the database, but you can read about it here.

Plugins

Django and other Python web frameworks use WSGI (Web Server Gateway Interface) to handle web requests. Serverless is modular and Logan Raarup wrote a package that elegantly solves this called serverless-wsgi. Although we’re working with Python, Serverless packages are installed with Node.js. By running sls plugin install, Serverless will set up a package.json and package-lock.json for us, and update serverless.yml to include the plugin:

sls plugin install -n serverless-wsgi

While we’re at it, let’s also install the serverless-python-requirements package that will bundle our dependencies in our requirements.txt file for us:

sls plugin install -n serverless-python-requirements

The plugin section of our serverless.ymlwill now look like this:

Plugins for requirements.txt and for serving WSGI.

Routing

If we tried to access Django through HTTP now, it would fail because we haven’t allowed any URL’s (hosts in Django terms) in Django yet. We could technically open one and one, but to keep this post simple we’ll make all URL’s available in both Serverless and Django. Let’s make our Serverless function accept any request:

Now all requests will be sent to our hello function in the handler.py file. Let’s also change the ALLOWED_HOSTS in mydjangoproject/settings.py so Django itself allows all URL’s:

ALLOWED_HOSTS = ['*']

When Django starts up on a regular server, it begins in manage.py and simply keeps running. By comparison, Serverless runs through the entire life cycle and then quits. It is event based, and is extremely efficient because it only runs when there is something to do.

The key in making Django run in Serverless is to send requests through manage.py like usual, and make our function behave like manage.py by changing our handler.py:

Connecting Django and Serverless

By calling execute_from_command_line with ['manage.py', 'runserver'], our Serverless function will run exactly like Django on a traditional server would.

From now on, we don’t really have to touch either of these areas again, unless we want to completely restrict specific HTTP requests. Any URL added to mydjangoproject/urls.py will now be available right away after deploying.

Migrations

Finally, we want a way to run migrations whenever we want. Usually, this would be done by SSH’ing into a server and running the migrations there, or perhaps through an admin panel. As usual with Serverless, this is very simple.

Let’s make a new, private function for migrating that only we can access. By default, Serverless functions without any event to trigger them are inaccessible. The only way to run that function is with serverless invoke -f on our own machine, or logging into the cloud website and executing it there.

Let’s add this new function with no events to trigger it under functions: in our serverless.yml:

Next, we’ll need to add the actual function for migrate.handler, so let’s create a new file called migrate.py:

Somewhat strange double try to make sure settings and include errors are reported properly

This will run our migration with all the environment variables in variables.yml.

At this point, we’re pretty much set! With the code we’ve written we can add any new URL’s to Django and it’ll be available immediately after deploying. When we need to, we can migrate with our own private function.

Overview / TLDR; (Too Long, Didn’t Read)

Thank you for reading! I truly hope this was useful :)