File upload on Koa or Express Nodejs to Cloudinary | S3 | File System with Code sample

4 min readJul 25, 2019

Introduction

Back in my early days, I have always stored all the files in the backend server and there was no progress indicator whatsoever or maybe I didn’t have requirements that needed those but later on, I started utilizing the XMLHttp progress events and later migrated to Angular, React and Vue.

In this post, I will be writing how to upload files to Cloudinary, S3 bucket and back-end file storage using Node js server and angular 8 application and the source code used for this post, is available on the GitHub repositories below.

Since we are discussing 3 different channels to do this, I will not be writing about media transformation but rather just focusing only on upload.

I am very excited so let’s get started.

At the end of this post, it should be very easy to -

Upload files to S3, FileSystem & Cloudinary server.
Get the transfer percentage of the upload in real-time

Although we can transfer files directly from the Client (Angular) to Cloudinary etc, for the sake of this post, we will be transferred to the node server and from there to those third-party media servers.

Freebie at the end of the post

Just, as usual, I have already prepared a little freebie at the end of the post, pointed to a repository with working sources and an example on how to upload to Cloudinary, S3 & File system using Nodejs & a sample angular project.

Installation & Configuration

First of all, before we continue, there are some of the libraries and dependencies to download in order to make our lives a little bit easier.

Install Cloudinary library npm install cloudinary --save
install AWS S3 library npm i aws-sdk --save

Configuring Cloudinary Server

In order to use Cloudinary server, you will need an account with the necessary token to connect and upload your files.

Create an account and get API details.

To connect to Cloudinary we will be needing only the cloud_name api_key api_secret

Configuring Amazon AWS with S3

Just like Cloudinary, you will need to create a bucket on Amazon S3 and create Access key security credentials.

To connect to S3 bucket we will need the secretAccessKey accessKeyId region and bucket.

By now we have already all we need to connect to those services

Create the configuration file

Putting all the configuration together, we will have a file like this

const config = { 
      cloudinary: { 
           cloud_name:'', 
           api_key: '', 
           api_secret: '' 
      }, 
     aws: { 
           config: { 
              secretAccessKey: '', 
              accessKeyId: '', 
              region: '' 
           }, 
          bucket: '', 
          url: 'amazonaws.com' 
     }, 
     fileSystem: { 
          dir: 'public/images', 
          url: 'https://127.0.0.1:5533' 
     } } module.exports = config;

Connecting to Cloudinary

Per Cloudinary NodeJs configuration, to establish a connection -

const cloudinary = require('cloudinary').v2 cloudindary.config(configuration.cloudinary)

Connecting to S3

To connect to Amazon S3

const aws = require('aws-sdk'); 
aws.config.update(configuration.aws.config); const s3Instance = new aws.S3();

Uploading via S3

We are using NodeJs on Koa or Express, whatever you choose and sending our files via Angular 8

Blob file transfer on koa are available at ctx.request.body.file and can be converted to buffer and passed directly to S3 putObject method

Converting to buffer

fs.readFile(
            ctx.request.body.file.path, 
            (error, buffer) =>{ 
          
      if(error){ 
          console.error(error) 
          return 
       } 
     // buffer to send 
})

You can also use the node fs.promises API introduced on version > 10

const objectParams = { 
        Bucket: configuration.aws.bucket, 
        KeyName: fileName, 
        Body: buffer 
} s3Instance.putObject(objectParams).promise();

This is all that is required to upload using S3, few things to keep in mind though that ctx.request.body.file is for koa and express we will be using npm install --save multiparty for this post and code found below.

Uploading via Cloudinary

Although Cloudinary has a lot of transformation and good stuff within the SDK for this post, we will only be focused on uploading files only and leave those for later posts.

cloudinary.uploader.upload(ctx.request.body.file.path, 
(error, res)=>{ 
    if(error){ 
         console.error(error); 
    }     // res - > cloudinary upload response 
});

Response sample is available here and that's all that is required for simple upload on Cloudinary.

Uploading to file system

This is one of the easiest to do, you just have to copy the file from a temp directory to a directory of your choice dirOfChoice.

fs.rename(ctx.request.body.file.path,dirOfChoice),
(error,res)=>{      if(error){ 
        console.error(error); 
        return; 
      } 
     //completed 
})

Although there are few codes to run in order to make sure all works okay etc but this is just the basics of uploading directly to the server.

Request file on Koa & Express

On koa, files are available on ctx.request.body.file using koa-parser middleware and express using multiparty with following code

const getFiles = (req) =>{   return new Promise((resolve, reject) =>{       new multiparty.Form().parse(req, 
       async (error,fields, files)=> {            if(files && files.file && files.file.length){          
               resolve(files.file[0]) 
           }            reject(error); 
       }) 
  }) 
}

Front-end for the upload

We will be using a simple Angular 8 application to do the upload but I won’t be writing the code here, but you could find the complete Front-end source here.

You can also accomplish the same with other libraries like Vue, React and Vanilla JS XMLHttp.

putting it all together, all the back-end source and working samples used for this post are available here.

Let’s keep the post going. Let me know what you think in the comment box below and feel free to share and use the source for your projects.

Originally published at ElectroSoft on July 25, 2019.