In the HTTP protocol, when communicating data from a front end app to a express.js back-end server, data parsing is a primordial step where beginners may have a lot of issues.
This article is aimed to give a strong understanding about what data parse is, the different ways of parsing data in express.js when using nodejs and http protocol and to prevent some bugs where the parsing may be the problem (specially in file uploading).
When trying to get request data, you may have been confronted to the problem of “empty of undefined req.body”, this article may help you out.
If you’re using nodejs, you certainly have an idea about how http request works. Http request allows communication between clients and servers. A request is sent from the client side to the server side, this one receive the request, treat it and send a response back to the client. With that being understood let’s go straight to the point.
When sending a request to a server, the client may put some data into it, for create or update a resource, this is usually done using POST and PUT request. In that case, the server is suppose, when receiving the request, to extract the data from it so he can do what he is suppose to do.
But usually, the data associated to incoming request bodies are not in a format, in a structure easily exploitable for the server. That is why the format of those incoming data have to be modified so that they will be operable to the server. That ! Is data parsing.
Parsing data, is basically changing there structure from a type to another one that the server can treat. When using express.js, there is different way and modules used to parse incoming request, these libraries may be different depending on whether or not the request transport uploads files. We are going to cover all the cases here with these ones :
– body-parser
– busboy
– busboy-body-parser
we will be sending data from a html form to a express.js app.
Before starting with the first one, we need to talk a little about how those libraries usually work in general.
All of them basically return specific “middlewares” (function that have access to the request and the response object) depending on some conditions.
When sending data from the client (our form in this case), not only data are sent, but also headers that contain information about the request. One of them, named “content-type”, give information about the format of the data that are associated in the sent request . This headers is very important because, each parsing “middlewares” is specialize for a specific format or range of format of data. Keep that in mind, because at sometime you will have to set the “content-type” header before sending the request otherwise, well…it won’t work.
Body-parser :
Body-parser is one the most famous module for parsing data even though he has some limits.
For install it you just need to use an npm or yarn command inside your node js app directory :
| sudo npm install body-parser
if you get some issues with npm, use yarn :
| sudo yarn add body-parser
We can now import it and use it in our code :
| const bodyParser = require(‘body-parser’);
body-parser come with a lot of methods for parsing data, but you will mainly use two of them: bodyParser.json() and bodyParser.urlencoded() :
in this code we simply console log the req.body (which is the object were parser put the data).
as we see it earlier, data are sent from the front end in a certain format.
In this case (photo 1) data are sent by the form. We specify the method (POST) and the url with the “action” attribute. The format of data can be set using the enctype attribute which default value is :
application/x-www-form-urlencoded
let’s see the result :
We try to remove the line 7 to see what is going to happen ? Let’s do it :
as you can see, we get an empty object, it’s because the adequate parser was not called.
We saw that body-parser is good for parsing regular form data. But it doesn’t handle form data with uploads file. To be simply, if we add an input with attribute type = “file” to our form, body-parser will certainly do his job with a valid enctype, but this won’t be really efficient when you wanna do things with files like getting there type, extension or save them in local folder. And the reason is simple, the best format for form data that include file is “multipart/form-data”. And this type is not handle by body-parser. To parse the incoming data so you can efficiently use the files, you will need other module.
Busboy :
Busboy is a node.js module for parsing specially incoming HTML form data. It does not seem easy the first time but it’s really not big deal.
To install it :
| npm install busboy
or
| yarn add busboy
Now, let’s import it :
| const Busboy = require(‘busboy’);
Write the next code and let’s talk about it :
This is what we basically do in this code :
first of all, we create an instance of Busboy with the line 9 and we pass the request headers as an object (and at this point you should understand why we have to pass the headers ). This instance will contain the logic that that will be called to handle the form. And at the end we simply pipe the busboy instance to the request so he can work on it. and between those to line, we are going to write our logic.
Note : at line 4, we also import a function from the built-in util module. it will be useful to access to some of the form data. Don’t forget to pipe the busboy instance to the request, if you don’t the server won’t respond.
busboy work kinda like an event listener. For all the type of input, their are different “event” that you can listen to and link to them functions that you can customize depending on what you wanna do with the data, those functions take multiple variables containing information about the input.
for example this code will take care of all the input field expected the file inputs :
As you can see, it’s like we were listening to an event and the function we pass take many variable in which some of them are optional. And we use the inspect() function to get the value of a field. submit the form to see the result.
This code was for regular field handling. let’s go to the file inputs :
When we get how it works, the rest is not tricky business. There is a lot things you can do with busboy like save directly file in folders. All the additional information about it can be found in the documentation
busboy-body-parser :
this is certainly one of the simplest way to parse incoming data whether it’s multipart/form-data bodies or not.
use npm or yarn command to install it :
| sudo npm install busboy-body-parser
| sudo yarn add busboy-body-parser
Import it on your code :
|const busboyBodyParser = require(‘busboy-body-parser’);
And basically all you need to do is call it in an app.use before any routes. What this does is that It will add regular fields to a req.body object as per body-parser but will also add uploaded files information to a req.files object.
don’t forget to specify the enctype of the form tag to multipart/form-data, unless you do that, req.body and req.files will be undefined. Busboy-body-parser is specially for multipart/form-data and as we’ve seen it, the default value of enctype is “application/x-www-form-urlencoded”.
let’s see the result :
if we modify our form to add a file input (don’t forget to add a name attribute to the file input, otherwise req.files will be empty:
Fill the form and submit it. you will see that all the regular field are in the req.body with their value and all the file information in req.files.
Really hope that this guide help you out !
For more information about all the modules presented here :
body-parser: https://www.npmjs.com/package/body-parser
busboy: https://www.npmjs.com/package/busboy
busboy-body-parser: https://www.npmjs.com/package/busboy-body-parser
