Uploading a File to Amazon Web Services (AWS) S3 Bucket with Node.js

Keith Weaver

In the post, I’m going to outline how to upload a file to an AWS S3 bucket with Node.js. I’ve covered this topic extensively on my Youtube channel and I’m hoping to get it in writing. Before I get started, if you need to see it happen please checkout: “AWS S3 with Node.js — Amazon Web Services End to End Guide”, “File Upload with Node.js — MERN”, or “Uploading a File to Amazon Web Services (AWS) S3 Bucket with Node.js”. In addition, before I get started I want to mention this is a Node.js tutorial but I will be using this MERN boilerplate code (Tutorial is Node.js, not MERN related but this code works really well).

Setup

I will be starting by cloning the MERN boilerplate code to my working directory (desktop).

git clone https://github.com/keithweaver/MERN-boilerplate.git

Then navigate into the folder and install the starting packages.

cd MERN-boilerplate
npm install

You’ll need to copy the config/config.example.js to config/config.js . Change the Mongo values to work for your environment. For me, my dev environment is no username or password ( localhost:27017/db_name ) and the production I add a username and password.

Next, File Upload

I will be using Busboy for the file upload. The first portion of the code is relatively simple. You need to install three packages:

npm install busboy --save
npm install connect-busboy --save
npm install busboy-body-parser --save

Next you will need to setup Express to use Busboy. In server/server.js add the following imports:

const busboy = require('connect-busboy');
const busboyBodyParser = require('busboy-body-parser');

Then below const app = express(); add:

app.use(busboy());

And below:

app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());

Add:

app.use(busboyBodyParser());

Next, you need to create a new endpoint for uploading. Create a new file in server/routes/api/ called file.js or something like that.

Add the following code which will handle file upload:

const Busboy = require('busboy');module.exports = (app) => {
// The following is an example of making file upload with
// additional body parameters.
// To make a call with PostMan
// Don't put any headers (content-type)
// Under body:
// check form-data
// Put the body with "element1": "test", "element2": image file
app.post('/api/upload', function (req, res, next) {
// This grabs the additional parameters so in this case passing
// in "element1" with a value.
const element1 = req.body.element1;
var busboy = new Busboy({ headers: req.headers }); // The file upload has completed
busboy.on('finish', function() {
console.log('Upload finished');
// Your files are stored in req.files. In this case,
// you only have one and it's req.files.element2:
// This returns:
// {
// element2: {
// data: ...contents of the file...,
// name: 'Example.jpg',
// encoding: '7bit',
// mimetype: 'image/png',
// truncated: false,
// size: 959480
// }
// }
// Grabs your file object from the request.
const file = req.files.element2;
console.log(file);
});
req.pipe(busboy);
});
}

Next, we will try this file upload code. Start the server:

npm run start:dev

Open a REST Client like Postman and being entering the known information. It’s a POST request and the endpoint is http://localhost:8080/api/upload . The parameters are element1 and element2 . There are no headers.

When you press “Send”, you won’t see a response. The request didn’t have a response but if you check your terminal you’ll see file content.

File upload is done! If you get undefined then you probably have the wrong headers.

Adding AWS S3

Before getting started, you’ll have to setup an IAM User and S3 Bucket with proper permissions. If you need a hand with that, check out AWS S3 with Node.js — Amazon Web Services End to End Guide.

Next, I will add the actual package for AWS S3. Run:

npm install aws-sdk --save

Add this to the top:

const AWS = require('aws-sdk');
const Busboy = require('busboy');
const BUCKET_NAME = '';
const IAM_USER_KEY = '';
const IAM_USER_SECRET = '';
function uploadToS3(file) {
let s3bucket = new AWS.S3({
accessKeyId: IAM_USER_KEY,
secretAccessKey: IAM_USER_SECRET,
Bucket: BUCKET_NAME,
});
s3bucket.createBucket(function () {
var params = {
Bucket: BUCKET_NAME,
Key: file.name,
Body: file.data,
};
s3bucket.upload(params, function (err, data) {
if (err) {
console.log('error in callback');
console.log(err);
}
console.log('success');
console.log(data);
});
});
}
... other code ...

Below the console.log(file); add:

// Begins the upload to the AWS S3
uploadToS3(file);

The full code can be found on this Gist. Nothing changes with the Postman request to work.

Keith Weaver

Written by

keithweaver.ca

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade