Uploading files to AWS s3 using Nodejs, multer, and Postman. — Part 1

Storing your static files in your MongoDB or any database you’re using is a terrible idea and considered a bad practice. You want to be able to access your data in a fast, controlled, and easy to filter manner.

Saving blob data into a database will traditionally cause a lot of speed, bloat, and scaling issues.

Ideally, you want to separate your static files from your database and save these files in a bucket like AWS S3 or a caching CDN. Then place a record in the database that will help you fetch it from S3. This keeps your dataset small and your heavier data you don’t really need in the database outside of it while giving you a fast and easy way to identify which of those files you care about.

Simple case study to understand all I said here 👆🏾👆🏾👆🏾

• A new user selects an image to upload as an avatar.
• API uploads the image to S3 and returns the URL or identifier of the image
• Save the URL to the user document/table in your database
• To access the avatar content, simply fetch it with the URL stored in the user document/table in your database.

Let do this!

NodeJS — JavaScript runtime built on Chrome’s V8 JavaScript engine.

AWS S3 — Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance…

Multer — a node.js middleware for handling, multipart/form-datawhich is primarily used for uploading files. It is written on top of busboy for maximum efficiency.

Postman — is a powerful HTTP client for testing web services.

Steps

Clone the repo for the full source code here

1. Create your S3 account refer to this article by AWS for help
2. Setup your node API

3. Create a user model

4. Create user controller and route

Bucket names must be unique across all S3 users

S3 upload method returns errorand data in a callback, wheredatacontains location, bucket, and key of the uploaded file.

Multer allows you to upload files in this example we only consider a single file. Visit multer to see other methods supported.

4. Finally, let test by uploading our file through postman.

Choose form-data and enter the field avatar and select file you want to upload.

If all goes well you’ll get a response showing the location of the avatar. Note that this URL(location) is also saved in your local MongoDB. To view it, connect to your MongoDB client (I used Robo 3T).

At this point, the upload to s3 is tightly coupled to one single route. If you need to upload in more than one place, you might want to ensure you are not repeating the bulk of the logic, and so abstracting the s3.upload()to a util module might come in handy. In the second part by Oreoluwa Aboluwarin, we will do the work of abstracting that, and also configuring our CRUD operation for more than one asset (image) upload at a time.

I hope you found this article useful, feel free to post your feedback and comments.