There are bunch of npm libraries to handle File upload in Nodejs like multer , formadible , GridFs for mongodb etc. but its essential to understand how file upload is handled behind the scenes without any external library with plain Nodejs.
In this blog we will learn how to handle files in Nodejs using streams and buffers. first lets understand what are streams and buffers as they are the backbone of any file upload in Nodejs. also apart from file upload this are very important concepts to understand in Nodejs.
Streams: They are collection of data similar to array or string. a good analogy to understand streams is to consider them as water flowing through a pipe where water can be considered the data that can be either string , binary etc while the pipe is the memory such that the data continously keep passing through it.
You might have seen the request-response objects in nodejs , they are a type of streams.there are certain types of streams depending upon what we can do with that stream but that’s a topic for a diffrent blog, for now understanding what stream is enough.
Buffers: Javascript by default doesn’t work well with binary data , so nodejs provides buffer module that can be used to handle binary data effectively.
Buffers can be considered as an object where we can manipulate and store binary data.
Now lets understand why this two concepts are important for any file transfer.
Lets create a simple frontend with JS as below with only an input for file upload.
const inputButton = document.getElementById("inputtag");
const buttonn = document.querySelector(".btn");
inputButton.addEventListener("change", async (e) => await updateFile(e));
buttonn.addEventListener("click", () => uploadFile());
let data = null;
async function updateFile(e) {
const file = e.target.files[0];
const formData = new FormData();
formData.append("image", file);
data= formData;
}
async function uploadFile() {
try {
const imageData = await fetch("http://localhost:5000/upload", {
method: "POST",
headers: {
"Content-Type": "multipart/form-data",
},
body: data,
});
} catch (err) {
console.log(err);
}
}
The above code is very simple to understand , we have few elements in our html DOM which we are calling and setting the input file in the FormData provided by JS which is used to handle file’s.
Notice that while making fetch call to backend we are specifying ‘Content-Type’ as ‘multipart/form-data’ this means that the file data will go as a continuos stream.
Backend Part (Nodejs)
Lets now head to server where we are specifying “/upload” endpoint and corrosponding logic to handle the incoming file stream from frontend.
import express from "express";
import fs from "fs/promises";
import cors from "cors";
import path, { dirname } from "path";
const PORT = process.env.PORT || 8000;
const app = express();
app.use(cors());
app.listen(PORT, () => {
console.log("Server running at Port", PORT);
});
Above we have just created a simple express server listening to port 8000.
Now lets create an endpoint to listen at /upload.
app.post("/upload", async (req, res) => {
let data = [];
req.on("data", (chunk) => {
// collect incoming stream chunks in an array
});
req.on("end",()=>{
// do something after collecting all parts of incoming file stream.
});
});Since on upload endpoint we listening to a incoming stream, we are using req.on to listen to events like “data” and “end” which refers to what to do while the data stream is being received and when it is completely received.
app.post("/upload", async (req, res) => {
let data = [];
req.on("data", (chunk) => {
data.push(chunk);
});
req.on("end", () => {
let fileData = Buffer.concat(data);
fs.writeFile(
path.join(__dirname, "example.pdf"),
fileData,
"base64",
(err) => {
if (err) {
res.statusCode = 500;
}
}
);
});
});In the above code we are initially receiving the incoming stream chunks and pushing them to an array. finally when the stream end’s we are joining all the parts pushed to the array in a completely formed buffer fileData.
Remember we are using buffer here since the incoming file data is in stream which comes in chunks of data that is held in buffer.
Finally , after having whole fileData , we are writing it to a file using fs module.
Notice we are using base64 as an encoding type since media files are in base64 and finally saved to example.pdf.
That’s it. This is how we can handle file upload in nodejs without using any external library.
One thing to note here , we are specifying the destination file as example.pdf thus it will write the incoming file as pdf if we are receiving any other type file the name and extension of destination file should be set accordingly.
Follow me on Linkedin and Twitter for More such intresting blogs.
Linkedin : https://linkedin.com/in/himanshukhemot
Github: https://github.com/Himanshuub2
Twitter: https://twitter.com/himanshu_b2