Django Channels 2.0 and AWS Elastic BeanStalk

TL;DR This article will help in:

1. setting up daphne and worker in the instance
2. setting up redis in separate instance
3. serving static and media files

And here is the story.

As a learner, we do not get to hear much about the how to setup Django Channels 2.x into an AWS Elastic Beanstalk (EB) server.

There is a good [article] on how to setup Channels 1.x into AWS, and the majority of the content here will just include a handful of addons on top of that, like, it misses out on serving the static/media files, which I spent several hours to figure, because I skipped `option_settings` document in AWS.

Note: AWS by default supports wsgi servers, not asgi, which is the one we need. And when launching a python instance, proxy server defaults to Apache.

Note: Please keep the aforementioned [article] open in the next tab.

Working of Django Channels Application

Channels introduces a view like entity called consumer which we primarily use for asynchronous operations. Traditionally with WSGI, requests were directly passed down to Django server by proxy(like nginx) and processed by the view based on the URL it matches. But with Channels, it passes through an interface like Daphne(ASGI server) which is responsible for distinguishing protocols like HTTP and Websocket and sending down to proper consumer, in which case it can be a traditional HTTP Consumer (Views) or Websocket Consumer.

Setting up message broker for production

I am going to almost re-iterate the steps of this [article] to setup Redis as our message broker.

1. Launch instance from AWS EC2 console.
2. Search redis 4.0 and choose one by Jetware.
3. Review and Launch
4. ssh into your Redis instance. You will need following information.

- logs: /jet/log/redis/redis.log
- conf: /jet/etc/redis/redis.conf
- dir : /jet/prs/redis/data 
- dbfile : dump.rbd

5. alter your conf file to change address from 127.0.0.1 to 0.0.0.0, and your port to 6379(which might be default to same).
Restart the server:

redis shutdown: redis-cli shutdown
redis start   : redis-server /jet/etc/redis/redis.conf

6. Now add an inbound TCP Rule to its security group from aws console.

Type: Custom TCP Rule, Protocol: TCP, Port Range: 6379, Source: 0.0.0.0/0

We are going to use its Public IP for later.

Fixing the Apache proxy server

Since most of us will be using some sort of Token for authorization, we need to add following to the apache’s config.

WSGIPassAuthorization On

By default, Authorization header is not passed through to Django.

Add the Redis as your Channel layer

REDIS_PUBLIC_DNS = os.environ.get("REDIS_PUBLIC_DNS", "The-ONE-you-copied-earlier")

if REDIS_PUBLIC_DNS:
    CHANNEL_LAYERS = {
        "default": {
            "BACKEND": "channels_redis.core.RedisChannelLayer",
            "CONFIG": {
                "hosts": [(REDIS_PUBLIC_DNS, 6379)],
            }
        }
    }

Run “Daphne” and “Worker” daemons

Create a folder called .ebextensions and create a file as shown in the given gist.

This will ensure the daphne and worker process fireup after deployment to the AWS.

Configuring the load balancer

I personally used aws-ebcli to create environment for the deployment. During which, we need to choose Application Load Balancer as our Load Balancer.

The problem I hit with the mentioned article was my static files were not being served somehow.

It is considered a good practice (indeed), to let proxy handle the serving of static and media files. So we will need to create a new config to add the followings into apache’s conf.

Add the following content as 098_staticfiles_alias.config into your .ebextensions :

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/post/098_static_files_alias.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash

      static_files_conf="<VirtualHost *:80>

      Alias /static/ /opt/python/current/www/static/
      <Directory /opt/python/current/www/static/>
      Order allow,deny
      Allow from all
      Require all granted
      </Directory>

      Alias /media/ /opt/python/current/app/media/
      <Directory /opt/python/current/app/media/>
      Order allow,deny
      Allow from all
      Require all granted
      </Directory>

      </VirtualHost>"

      echo "$static_files_conf" | sudo tee /etc/httpd/conf.d/static_alias.conf

      sudo chown apache:apache -R /opt/python/current/app/static/

      # to update the directory directive
      sudo apachectl restart
      # we can remove wsgi.conf since its not going to be used
      # sudo rm /etc/httpd/conf.d/wsgi.conf

Note: Mind my Alias may not match upto yours, so edit accordingly. Mine settings were,

STATIC_URL = '/static/
'STATIC_ROOT = os.path.join(BASE_DIR, "..", "www", "static")
MEDIA_URL = "/media/"
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

Now the alb.config looks like follows.

option_settings:
  aws:elbv2:listener:80:
    DefaultProcess: http
    ListenerEnabled: 'true'
    Protocol: HTTP
    Rules: static,media
  aws:elbv2:listenerrule:static:
    PathPatterns: /static/*
    Process: webserver
    Priority: 1
  aws:elbv2:listenerrule:media:
    PathPatterns: /media/*
    Process: webserver
    Priority: 2
  aws:elasticbeanstalk:environment:process:http:
    Port: '8000'
    Protocol: HTTP
  aws:elasticbeanstalk:environment:process:webserver:
    Port: '80'
    Protocol: HTTP

Extra

Furthermore, you can collect your static files to S3 bucket using django-storages package very quickly. All you need is a S3 bucket, and furthermore use CDN (using CloudFront) for quick delivery for static files.

On the other hand I chose to keep Media files in the default Django instance, uploading into S3 seemed like a latency issue, though I cannot verify yet. But, we can still use CDN, to serve those files. And my final settings looked like,

"""
MEDIA FILES
stored locally in the django server, however served by cdn dedicated for elb
"""
MEDIA_URL = os.environ["MEDIA_CDN_URL"]
"""
STATIC FILES
collected into S3, and served by dedicated cdn for s3
"""
# to upload static files to s3
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
# AWS_S3_CUSTOM_DOMAIN = os.environ["STATIC_CDN_URL"]