Using S3 for Django Media Files

As you scale your Django web application, you’ll eventually want to explore storing media files in a storage service like Amazon S3. Django media files typically encompass user uploads and downloads. For example, at Prompt, documents our customers upload, as well as PDFs of feedback we generate, use Django’s media file storage.

Storing media files off of your server in a service like S3 is safe — even if your web server gets hit by a nuke, your media files will still be accessible — and becomes cost effective as you aggregate lots of media files.

This tutorial covers how to setup Django to use S3 for all media files. We’ll use Boto, a python interface for accessing Amazon Web services, and django-storages, a module that can swap out where Django stores media files with just a few lines of code.

This tutorial assumes you already have an AWS account. If you don’t, you can sign up for the free tier which allows for 5GB of S3 storage.

1. Install Python Boto with command: pip install boto3
2. Install django-storages with command: pip install django-storages
3. Copy the code from the settings.py snippet below into the settings.py file in your Django project. This sets S3 as your default file storage solution (for Django media files only, you’ll need to find another tutorial if you want to host static files on S3 as well). We’ll fill out the missing settings as we create them.
4. Create a new S3 Bucket by clicking “Create Bucket” on the Amazon S3 page: https://console.aws.amazon.com/s3/. Save the name of the bucket you just created in the associated setting you just pasted in settings.py
   You can think of an S3 bucket as a hard drive. Within this S3 bucket, you can create folders. Indeed, if you’ve set the upload_to attribute on any of your Django file fields, then a folder bearing that name will be created in your S3 bucket to hold uploaded files.
5. Next, we’ll create a new user who will be able to read and write to the newly created S3 bucket. Go to the IAM section on S3 and in the “Create Individual IAM users” section, click “Manage Users”: https://console.aws.amazon.com/iam/home. Enter a new user name. Make sure that “Generate an access key” is checked and click “Create”.
6. After creating the user, you should be able to view security credentials. These credentials include the Access Key ID and Secret Access Key. Copy both into the associated settings in the settings.py file of your django project.
7. We need to grant our new user access to the S3 bucket that we created. First, we need to create a policy that grants access only to our created S3 bucket (and no other facets of AWS). In the IAM interface, click Policies and then Create Policy. Copy and paste the IAM Policy from below. Be sure to replace <bucket_name> with the name of the bucket you created in step 4. Do not change the Version string.
8. Next, in AWS’s IAM screen, click Users and then select the user you created earlier. Under the Permissions tab, click Attach Policy. Attach the policy that you just created.
   Note that you can grant the user full access to all of S3 by selecting and attaching the “AmazonS3FullAccess” policy. This is, of course, a bad idea for production, but perhaps something to consider when testing.

Your Django project should now use S3 to host all media files. Files uploaded via file fields on models will automatically be written to the correct folder in your S3 bucket (determined by the upload_to field attribute). If you want to interact with the files in your S3 bucket, check out the Storage section of the S3 django-storages documentation. Using django-storages, you can manually create, read and delete files in your S3 bucket (talk about a party).

This allows you to, for example, write downloadable files to a Downloads folder in your S3 bucket, that is publicly accessible (you’ll need to create an S3 policy that looks like this).