Uploading multiple assets to AWS during CRUD operations using NodeJS, mongoDB, multer and Postman — Part 2

TLDR: This article provides a guide to uploading multiple files to AWS (s3) before storing the URL of the uploaded asset in your DB in one single request.

In the first post in this series by Otoloye Oyeniran, we did a run through of setting up the application with the basic files needed, and we also demonstrated how to upload a single asset to AWS using the multer middleware before your controller performs the CRUD operation.

In this part, we’d take it a step further by showing how we can upload multiple assets in the same request as well as how to make the AWS upload function composable for reuse in other endpoints. Let’s begin…

Before you continue, please check the first post here so as to have proper context.

The first stop for us will be to abstract the upload function into a separate module.

Here, we need three major packages aws-sdk, node’s fs module and dotenv for parsing environment variables we wish to hide from public view/access.

In lines 6–13 of upload-to-s3.js, we implement the basic configuration for aws. That configuration was covered in Part 1 of this series.

Now, we make the uploadToS3 function return a Promise. I particularly love creating time-dependent functions this way mainly because at the point of call of the function, you can call a .then on it and perform any necessary action. This suits us well because after we call this function, and our upload happens, and the url to the asset on s3 is returned, then we can go ahead and save along with other fields to the DB.

I love abstractions so much. It ensures that the code is modularized and more understandable. And for this reason, I abstracted a checkMulterParams function out of the upload-to-s3. What that function does is to check the type and number of files we need to upload and perform necessary operations on it. Let’s take a look at that file.

The multer middleware takes in a series of configuration options for when you want to upload a single file, or an array of files, or an object whose keys point to each file.

.single(fieldname): Accepts a single file with the name fieldname. The single file will be stored in the req.file object when the middleware is added to your route.

.array(fieldname[maxCount]: Accepts an array of files with each of the files having the name fieldname. Note that fieldname can be any name/key you want the file to be in the DB. The request will optionally error out if more than maxCount files are uploaded. The array of files will be stored in req.files.

.fields(fields): Accepts a mix of files, specified by fields. It is an object with arrays of files that will be stored in req.files. Each file in the array should be an object, and so it would seem that when all is broken down, it comes down to multiple single file uploads, and the data structure for a single file is an object.

Armed with this information, we could make .fields the default type we’d use. Afterall, as explained in the last paragraph, it ends up coming down to a single file upload.

The checkMulterParams function is multi-functional. It checks if we have specified array or fields or single on the multer middleware, and the logic handles the breakdown of data to be passed to the uploadToS3 function.

We return an array of objects from the checkMulterParams no matter the type of data passed in. You could call this stage normalization of data. It is useful because it makes our uploadToS3 function worry only about the upload, and not data structure.

Notice also that we make the uploadToS3 function take an object as argument. This ensures that the function takes only a single parameter which we can stuff with as many key-value pairs as possible. What this helps to do is ensure that we don’t have to worry about positional parameters at the call site of the function.

Finally, we are ready to use the uploadToS3 function in our controller. Here goes:

First, we have our server

Then we have the user routes file. You see that we specify fields in the multer middleware like so:

The user model is shown below. It contains the possible fields we have defined for our user.

Finally, the user controller is as below.

You will notice that we have three conditionals. Each of them for instances in which either one or the other or both of the fields we need to upload assets for are specified. We do this so that we don’t call the expensive uploadToS3 method if it is not absolutely necessary. Also, you see that we use Promise.all. It is a handy Promise method that takes in an array of promises and does not return a result until the promises are completed (I use completed here because each individual promise can be resolved or rejected).

We also assign the req.body object to a variable userRequestObject and assign the values of the urls that point to our assets only when we have them. It is until we obtain these urls that we can then create a new user as seen on line 43 of the users-controller.js file.

Fire up postman at this point and navigate to the /signup route. If all goes well, you should see the message: user created successfully`

I did not do so much with handling the possibility of failure of any of the promises here with a .catch, and this is something you’d need to look into, else, a failure somewhere might lead to unwanted complications.

The journey was long, but I hope it was worth it.

The complete source code can be found on GITHUB here

Feel free to post your feedback and questions below.