This is the second part of our little post series where we explore how to build a Flask JSON API for uploading files to Google Cloud Storage.
In Part 1 we got an overview of our flask app and added a filedepot configuration that allows us to handle multiple upload environments (testing/development, production). In this part we are going to add an avatar to our User model and–backed by some automated tests–implement the actual file upload endpoint.
This is going to be a rather extensive tutorial. If you want to dive into the code along with reading: it’s available on Github.
As mentioned in Part 1 we assume we already have a User model defined. So, in order to attach an avatar image to a user, we need to do two things:
- Adding an avatar column to the User model , which stores the URL to the uploaded image
- Adding an endpoint for uploading the image
Let’s take a look.
Adding an image
The filedepot library comes with an
UploadedFileField class, which we will import and use to create our avatar column. This class will take care of uploading the file when assigning a value and committing the model. It also provides access to the file url. You can pass an upload_storage (i.e. depot name) to the
UploadedFileField’s constructor to use a custom configured depot. Right now, we only have a single storage with the name “avatar” configured in
my_app/config/depot.py. So, by default the
UploadedFileField object will use this storage and you don’t need to provide the upload_storage explicitly. Hence, adding the avatar boils down to only a couple lines of code:
(In the code snippet, the
db.Modelbase class comes from FlaskSQLAlchemy. The imported
db is an instance of
Don’t forget to create a migration and upgrade your database!
Adding the upload endpoint
Model-wise we are well-prepared now. In order to make our uploading functionality available for an API client, we still need an upload endpoint.
I like test-driven development, so let’s write some tests first and then implement our endpoint. One tricky part would usually be mimicking a real file upload in our tests and allowing our test to be run without any real connection to the external storage service. Luckily, we already took care of this part by configuring our depot memory storage in Part 1.
There’s one thing I like to do when testing IO or file-related logic: in order to work with data that is as close as possible to our expected data I create real empty sample files and load them when running the tests.
For the image upload let’s put a single pixel PNG image into a test fixtures directory:
tests/support/fixtures/test.png. Additionally you can add some small helper functions into a
tests.support.helpers module that will facilitate loading the image in the tests:
load_file_data() function will provide us with a tuple of the format
(binary data, file name) which we can just assign to our user’s avatar property. Let’s say we want to use
POST /users/<user_id>/avatar as our image upload endpoint, then our test function looks like this:
We load the 1-pixel image data and post it as multipart form data to our just made-up endpoint. Below we have two simple assertions to check the response code, and whether there is some URL in the user’s avatar now.
If you run the test with
pytest tests/users/test_views.py you will get a 404 Error. This is expected, because we did not yet implement the upload endpoint, hence, the route is not yet available. Let’s add the endpoint implementation — it’s quite some lines of code, so we will slowly walk through it and see what it does. (For the sake of simplicity, we skip all authentication decorators etc. that you’d probably like to add in your real app.)
It starts with a couple of imports. Just ignore these for now, and let’s have a look at line 17. Here we define a
POST /users/<user_id>/avatar endpoint (the URL prefix
users is defined for all endpoints in the
url_prefix argument when creating the blueprint). Before we can upload the actual image, we should check a couple of things:
a) whether the user with the given
user_id is available. If not, we immediately abort the request with a 404 (Not Found) response.
b) whether a file to upload was provided. If not, we abort the request with a 402 (Bad Request) response. We get our avatar file from the
c) whether the given file has one of the allowed MIME types. Therefor we define a set of allowed types and check whether the given file’s MIME type is among them. If it isn’t a PNG or JPEG image, we abort the request with a 402 response.
If all checks passed, we assign the avatar to our user’s avatar property and commit the user. On commit, filedepot will take care of uploading the image, and adding the URL and some metadata to the
users.avatar column in our database.
In the end we respond with the user JSON, which (after you did some adjustments to your user serialization) now also includes the avatar image url. The example app uses marshmallow for serialization, but you can use whatever suits you best to create the user JSON data.
user.avatar returns an instance of depot’s
UploadedFileclass. This object provides us with a couple of methods to get the uploaded file’s data, such as
content_type, and an
uploaded_at timestamp. You can see a full list of the provided methods in the depot documentation. In the example app I just used
user.avatar.url to provide the avatar image URL in the response JSON.
Adding a depot middleware
pytest tests/users/test_views.py again, you should now see the upload test passing. But, wait, we now get another Error:
RuntimeError: No WSGI middleware currently registered
The reason is: unless our configured depot storage supports serving files via direct HTTP access (i.e has a not-None
public_url) we need to configure a WSGI middleware that can serve the uploaded file. Because we use the
MemoryFileStorage to run our tests, this is the case and we need to provide a middleware. You can read up on when the middleware is needed in more detail in the respective part of the depot documentation.
The good thing is, it’s basically a one-liner, so let’s quickly configure this middleware:
Then we call the
make_middleware() function during our Flask app setup, and we are good to go:
With this we have a basic avatar upload endpoint in place —Run the test again and it should pass. Hurray!
Of course, you could add some more tests to cover all the failing file upload cases with the checks we’ve seen above. If you like, I’ll leave this as an exercise to you ;).
Before we come to the end of this post, we’ll cover a common use case for image uploading. You might want to upload an image and auto-generate different-sized versions an the fly. So let’s briefly look into how you can create thumbnails for your uploaded avatar image.
Depot comes with the possibility to add custom filters to an
UploadedFileField. It also provides a
WithThumbnailFilter class which we will use to automatically create two resized versions of our uploaded image. For this we add some thumbnail size tuples to our User class and pass a filters list in the
WithThumbnailFilter requires the Pillow package to be able to process the resized thumbnail images. So we need to add pillow to our
pip install -r requirements.txt, or in case you use pipenv just run
pipenv install pillow.
When we now upload an image, depot takes care of resizing the original image to the configured size and uploads the thumbnails along with the original image. You can access the thumbnail URLs by auto-generated functions of the format
thumb_<size>_url, i.e. in our case we can call
thumb_128x128_url to get the small and medium thumbnail URL respectively.
We will quickly add two lines to our endpoint test to make sure that an
avatar_medium property is available in the JSON response. Also don’t forget to add the two properties to your JSON response in your endpoint implementation, i.e. to the UserSchema if you used marshmallow.
That’s it —after the tests passed we will now get some thumbnail images in our JSON response.
With these features in place we will finish Part 2. In the third and last part we will look into how to customize upload locations and will adjust our depot config to allow multiple depots. There is going to be some more things to explore, so leave some feedback if you learned something useful already and click next.