For developers coming from a backend language like Ruby or Java, you might have heard of the term middlewares. Even applications are referred to as middlewares. When I started learning to code, I never understood why these were called middlewares in the first place, leave alone enterprise application architecture.
But then came my introduction to NodeJS/ExpressJS. I failed to understand, now, how can an entire middleware application resided within an application. Wouldnt it make the application heavy? Wouldnt it be impossible to do work if it were needed to be on a different server? Wouldnt it make the application slower with higher response times and a memory sucker (Sorry, but many Enterprise Java Apps are…, and I still love and acknowledge Java as the “GO” to language)? Wouldnt it need a lot more CPU power and RAM (Vertical infrastructure scaling)? I, now, thought that Middlewares must have got a new integration / best practice architecture to make development easier and more maintainable. Well, seems like I was reading wikipedia and Java developers a lot; who at that point dominated the enterprise market (and may be still be doing so).
Forgive me for the over-curated terminology junk above and let us forget the above details. Let us make it simple to understand middlewares in the context of ExpressJS, and understand the meaning here based on what middlewares actually do.
Middlewares are simple functionalities for some job that reside between the client facing server and your business logic handler. The processing may be before or after the business logic handler. It may be an application (in the same / different OS based on infrastructure architecture), OR for that matter a modular functionality (a function) that does a specific job/processing that your application core is incapable of doing (or designed to be so, to be modular, understandable, and/or maintainable).
Middlewares in ExpressJS are application function based functionalities that reside in your application before/after a request is handled by the Route Handler. Middlewares in ExpressJS are lean and, generally, focus specifically towards manipulation of requests and responses based on some needs of the enterprise application. However, there are more complex ways a middleware may be used in/with an ExpressJS application like it is done in Java/Ruby/Python based applications in enterprise architectures. The concept does not change here. To re-iterate, middlewares are functionalities that sit in the middle doing some job/processing.
Now, that we understand what middlewares are for, let us understand middlewares always have a cost to processing and memory consumption. Ah, sorry again. What I meant is, middlewares will be using CPU and memory — whatever the scenario. Middlewares will definitely slow down the system or the application based on how heavy the work is. But, due to the core non-blocking asynchronous nature of NodeJS application servers, the processing “feels” faster and is, definitely, more efficient over a span of concurrent requests. For people who are not aware of non-blocking asynchronous Eventloop of NodeJS, please have a look at this NodeJS documentation. I will write a separate post for Eventloop and non-blocking asynchronous handling later.
Now, let us take a look at how we can write a ExpressJS application function based middleware. When we finished the Route Creation post, we ended with this code somewhat below, which we will extend.
A simple middleware is a function with some functionality. So let us write that:
function(req, res, next) {
console.log(‘My logger ’, req.originalUrl);
next();
}
Notice that I wrote the function with the argument next that is extra. next is just an method that tells the middleware that “I am done processing and now please forward it to the next middleware or to the next handler (it can be the route handler)”.
Now, lets put it in our application server before we have defined the routes using a method called as app.use(middlewareFunction), like below:
app.use(function(req, res, next) {
console.log(‘My logger ’, req.originalUrl);
next();
});
That is it. Our simplest middleware is ready. The work of this middleware is that it consoles out “My Logger xxxSpecificRequestPath” for every request. Let us look at our code again:
Why not extract the anonymous function and put it such a way that it can be extracted into a different file later? Here is what it would look like:
const myFirstMiddleware = function(req, res, next) {
console.log(‘My logger’);
next();
};app.use(myFirstMiddleware);
Let us look at the code again:
Remember, the middleware will be applied to all the routes by default. But what if I want to run a middleware only for a specific path/route? In such a case, you will have to pass the middleware function before your route handler like below:
const myRouteMiddleware = function(req, res, next) {
console.log(‘My logger ‘, ‘only for ‘, req.path);
next();
};app.post(‘/’, myRouteMiddleware, function(req, res) {res.status(200).send({status: “running”, time: Date.now()})});
In this case, the middleware myRouteMiddleware will be run only for a HTTP POST request at URL path /. Let us look at the code again:
One important point to remember is that the ExpressJS middlewares are triggered in the order in which they were defined in the code. The ones defined first will be triggered first. So for the HTTP POST request to path / will result in the below order of execution:
// Console log in Server
// My logger /
// My logger only for /
// Response to the client
// {status: “running”, time: 1538448845388}}
Now that we know how to define middlewares, let us answer some important questions:
Wouldnt it make the application heavy?
No, it does not. We will not be using it for heavy work. Even if we do then it will not impact other requests since NodeJS server has quite high concurrency levels managed by the NodeJS Eventloop.
Wouldnt it be impossible to do the job/work if it were needed to be on a different server?
In such a need, we can redirect a server request to an external server and close the response to the client right in the middleware. Take a look at the middleware code below (note that the code below is asynchronous):
Or, may be just end the request-response HTTP life cycle by responding to the request from the middleware itself. Look at the code below:
The code above is less efficient since it waits for the POST to finish before being passed to the route handler, which is blocking. But the middleware ends the request-response HTTP cycle from the middleware itself whenever there is an error in the middleware’s task.
Wouldnt it make the application slower with higher response times and a memory sucker (Sorry, but many Enterprise Java Apps are…, and I still love and acknowledge Java as the go to language)?
Definitely, every processing has a cost to CPU and memory; whether Java or NodeJS. But since NodeJS has non-blocking asynchronous nature, other concurrent requests to Node Server are not hampered much even if one request is slower. This makes NodeJS quite efficient while managing high concurrency avoiding immediate vertical scaling that, generally, is required due to blocking nature of synchronous applications. So response times are not highly impacted unless you have a very very low end server for big and heavy duty applications.
Wouldnt it need a lot more CPU power and RAM (Vertical infrastructure scaling)?
It will for heavy duty middlewares. Even after that ExpressJS is lighter and less of a memory sucker than many other frameworks out there. But, ofcourse, you can mess up the server by badly written synchronous code in ExpressJS/NodeJS as well. Remember, we saw an example of that. It is important to decide before (based on business logic) which middlewares need to be synchronous and which ones can be asynchronous.
We understand Middleware a lot more now. But, what jobs/purpose/processing are they generally used for?
Well, the answer is that its a big list. In a simple application, it can be for logging, converting the request buffer body into readable format, authentication, authorisation, session/cookie verification, input validation, etc. But, there are ways where middlewares can be used along with a PUB-SUB, asynchronous, do-it-later models for logging, bulk email sending, simple or batch job processing, concurrent scheduler triggers, alerts, other enterprise logic not needing a response, etc. The list can go on.
To summarise, going along the post, we learnt:
- What middlewares are?
- What middlewares are, in context to ExpressJS application?
- How to create ExpressJS middlewares (and how to pass the processing to next middleware or route handler)?
- How to apply middlewares to entire application or only to a specific route?
- How to work with middlewares, and how some simple code changes make the middlewares blocking or non-blocking asynchronous?
- How we can close the request-response lifecycle without passing on to a route handler?
- What are the pain points of an application, and how NodeJS/ExpressJS resolves some main ones if not all?
- What are some of the common use-cases of ExpressJS middlewares?
In the next blog, we will use this knowledge to create a secure ExpressJS application and server that can read request bodies to do processing. We will also use some external npm packages that can be used as middlewares to ease the process of creation of the application.
Understanding the request from the client with the help of middlewares: https://medium.com/@ganeshsurfs/expressjs-series-understanding-the-request-from-the-client-with-the-help-of-middlewares-a50960c4239f
ExpressJS Series — All Blogs https://medium.com/@ganeshsurfs/expressjs-series-links-9e038be8d78b
Let me know how I did, and if you learnt something new. Do leave your comments, and dont forget to like the article.
