This is the third and last part of our little post series where we explore how to build a Flask JSON API for uploading files to Google Cloud Storage.
In Part 2 we added an avatar image and thumbnails to the User model, implemented the upload endpoint, and saw how to test our file upload with the configured in-memory file storage. In this last part, we are going to customize upload locations and file names and rewrite the depot config in order to allow multiple depots for different file types.
This is going to be a rather extensive tutorial. If you want to dive into the code along with reading: it’s available on Github.
The depot setup that you’ve seen in Part 1 and Part 2 works if you want to use only a single depot, that means: if you want to upload all the files to the same directory in Google Cloud Storage. However, maybe you will have another file to be uploaded for a User, e.g. a profile background image. And what if you wanted to store the avatar images in your Google Cloud Storage in an
/avatars directory, and the profile background in a
/backgrounds directory? Fortunately, depot comes with a way to achieve this.
Customizing the upload
Before we look into some config adjustments to allow different upload directories, let’s add the new
profile_background property to our User model:
Don’t forget to run your migrations to get this column created in your database table, too. If everything is set up in the model, let’s move on to the
Multiple depot storages
In order to store files in different directories in Google Cloud Storage you can configure multiple depot storages. In our case we want to keep the basic configuration (such as the depot backend, endpoint_url, bucket and access credentials) and only adjust the subdirectories for the avatar images and profile background images.
You can configure another storage by using
DepotManager.configure()which takes a depot name and a config dict as parameters. So, first of all, we will add two different constants —
BACKGROUND_DEPOT — for our depot names to the
/config/depot.py file. By adding a
depot.prefix to the config you can adjust the path prefix for the uploaded file. The prefix not only allows to set a custom the directory:
but you can also set prefixes for you file names:
The entire configured prefix is put in front of your uploaded file name and Google Cloud Storage will use this full path to store the file.
Given our depot config from Part 2, here is how we need to change it in order to configure multiple storages:
We define a
DEPOTS dict, loop over its items and merge the custom depot config with the default config we already defined before. Then the
DepotManager configures a new storage for each of our config items.
With this setup we have two different storages in place, but our user model will still upload all the attachments to the default storage. The default storage will be the
avatar storage, because this is the one we set up first. So, to get our images stored in the wanted directory we need to set the
upload_storage argument when creating the
Now you can also see, why we added our depot names as constants in the first place. You can just import them and reuse them to set the upload storages without having any magic strings around.
Implementing the endpoint for uploading a profile background image would be pretty much the same as we did in Part 2. Also setting up tests for this endpoint will look mostly the same as for the avatar upload. I’ll leave this part to you, the kind reader. If you don’t know where to start, please have a look at the previous post or find a DRY-ed version of the endpoint implementations in the Github repository.
Multiple upload services
In this tutorial’s examples we only had a look into how to adjust the storage directories for a single storage backend — Google Cloud Storage in our case. Of course you don’t need to reuse the default config, as we did. There is nothing that stops you from calling another
DepotManager.configure() when setting up your depots to store other attachments in e.g. Amazon’s S3 or a MongoDB GridFS. Also make sure to have a look at depot’s API documentation if you need further information about what storage classes are available and how to configure them in your depot setup.
Wrapping it up
With these final words, we will finish our little post series about how to upload files to Google Cloud Storage via a Flask API. You’ve seen how to setup a Flask app to provide file upload endpoints that store the uploaded content in Google Cloud Storage. We’ve looked into a local testing setup with depot’s in-memory storage, we’ve implemented automatic thumbnail creation for the uploaded image, and we’ve seen some ways to customize your storage directories and file prefixes.
I hope you enjoyed the reading and you learned something useful. Please leave some feedback and also don’t forget to share your new knowledge with your colleagues.
Happy coding, happy file uploading!