My way of structuring a Node JS App
The article which I was looking for from a long time when I was hunting all over the internet to search for an article which can help me out about structuring a Node JS app
But thanks to our great developer community which helps me out with great articles which makes myself capable to write this one. I’ve tried my best to accommodate all the best practices in building this app structure.
This article gives an overview of structuring a Node JS app mainly focusing on REST API implementation.
Here is my repo for detailed understanding. Lets start folks…
Starting with the root folder, this is how our Node JS structure looks like from the root.
├───.docker
│ └───data
│ └───mysql
│ ├───db_name
│ ├───mysql
│ ├───performance_schema
│ └───sys
├───app
│ ├───commands
│ ├───contracts
│ ├───errors
│ ├───helpers
│ ├───http
│ │ ├───controllers
│ │ │ └───api
│ │ │ └───v1
│ │ ├───middlewares
│ │ │ └───policies
│ │ └───responses
│ ├───models
│ ├───providers
│ ├───repositories
│ ├───services
│ ├───tasks
│ │ ├───DatabaseNotification
│ │ └───Imports
│ │ ├───College
│ │ └───Student
│ ├───transformers
│ ├───utilities
│ │ ├───AWS
│ │ ├───fcm
│ │ ├───Logger
│ │ ├───mail
│ │ ├───paginator
│ │ └───passport
│ └───validators
├───config
├───database
│ ├───migrations
│ ├───raw-query
│ └───seeders
├───public
│ ├───images
│ └───samples
├───routes
├───storage
│ ├───conversations
│ ├───imports
│ └───logs
├───views
│ └───emails
│ ├───auth
│ ├───conversations
│ └───enquiries
├───uploads
│ └───images
├───app.js
└───package.json— app.js
This file is the starting point of our node js app, which acts as a bootstrap file from where our app gets the ignition and its responsible for loading all the main resources & modules required for running the app completely and efficiently.
The “app” Directory
This directory holds all of the application’s code which is needed to run the app. It consists of many other different directories which are required to give a proper structure to our app, by following some of the best coding design patterns and practices.
Now, talking about its sub-directories:
The “app/http” Directory
As the name depicts this directory holds the code which interacts directly to the “http layer” of the app. This directory is a collection of 3 sub-directories.
These are the following directories:
- Controllers
Every app has this directory, and most of you out there already know that its a part of an app which follows MVC or MVC like pattern. This directory actually holds all the controllers which are responsible for receiving requests from mainly routes & middlewares and to serve them back as a server response.
The controller should do only one thing handle request from the route and return an appropriate response in a required format.
- Middlewares
Middleware allows you to define a stack of actions that you should flow through. Express servers themselves are a stack of middlewares. Actually, the whole app acts as a stack of middlewares which is executed in a sequence.
But in my structure, they play a major role in modifying request and response objects of the app as per needed configurations. So it consists of policies too which acts as a guard to our application.
For example, let me elaborate on some of the common and important middlewares:
— auth.js: Contains an authentication method to check the token coming in the request is valid or not.
— request-interceptor.js: It acts as an interceptor to request object. It is responsible for adding certain important methods and values to the request object which is further sent to controllers.
and many more… - Responses
Its a collection of all type of responses which app is responding back to all the incoming requests. To make this simple, understand it as a response-interceptor as you’ve seen in the request part.
For example, I always used to make these common responses which make it easy to return back responses further in the app with just a single line of code.
Models — Represents the main data source of our App
For this article, I’ve used sequelize (a promise based ORM for Node JS). Models are the most popular part of software design patterns. The model represents the data and does nothing else. It's totally independent of other layers in the app.
To make the code more readable and developer-friendly I’ve created custom classes for each of the models in our app, as you can look into the models directory which consists of these classes — User.js, Student.js, etc.
Each model class consists of many different methods supported by sequelize for example — defining table structure, associations/relationship, instance, and static methods which can be used all over the app for manipulating CRUD operations with the database.
The index.js inside this directory acts as a heart of all the model classes as its responsible for running, initializing, configuring all the models and their relationships, etc while our app is starting from scratch.
Repository — Data Abstraction layer for manipulating CRUD operations.
This is an implementation of a repository designing pattern. Mainly repositories are created to handle all sort of CRUD operations with the database. This directory holds bunch (equal or more than model classes) of repository classes for each of the model out there.
It has a BaseRepository which has a RepositoryContract attached to it. The RepositoryContract is built to guide all the repositories to define certain pre-defined important methods which help us to interact with our database.
All the repositories extend the properties of BaseRepository which itself extends RepositoryContract. Due to this implementation, all the repositories which are built in the future should define all the methods defined in the RepositoryContract. To avoid writing the same definition multiple times I’ve moved all the common methods for all the repositories into a single file — BaseRepository so that every time we create a new repository class we just have to extend BaseRepository to include all the common repository methods.
Services
Services encapsulate your business logic and make the controllers pretty thin. Basically, the controllers use the service layer to get the domain models that are then transformed into view models.
The main use of creating a service layer in-between Controller & Repository is to connect them both by obeying business logic and making the whole process encapsulated from the outer world.
According to me, a service contains a group of repositories, validators, tasks and other helper methods to implement a business logic and make the application run in a specific manner.
As we’ve covered all three things — Controllers, Services & Repositories, I would like to share the flow between these three starting from the incoming request.
Transformers
A transformer is a class that represents a single model that needs to be transformed into a JSON structure. The transformers give us the ability to define a specified set of JSON structure so that we can maintain consistency in all our public APIs.
When building an API it is common for people to just grab stuff from the database and pass it to json_encode(). This might be passable for “trivial” APIs but if they are in use by the public, or used by mobile applications then this will quickly lead to inconsistent output, which actually leads to a bad practice of writing APIs.
An example of UserTransformer is as follows:
A transformer mainly helps us to hide the specific fields from being exposed to the public, hence leads to a customized response as a developer needs in the end. Each model/entity should have at-least a transformer of its own so that a model can be exposed to the public with a set of defined JSON structure.
Validators
Performing validations at the params level can save your application from performing operations deemed to be invalid due to input params. Having parameter validations in place prevents errors due to invalid input and increases the security of your application in general.
That's why we use validators as a separate layer which acts as a security layer for our application.
I believe that — validating params and erroring early in the request lifecycle frees resources for the next request sooner, adds a layer of security and prevents invalid data from reaching the backend processes.
I’m too much inspired by Laravel’s way of validating parameters. Actually, I found it the best way to implement validations in the app, which forces me to build validations in a laravel way.
I’ve used a package named — validatorjs which helps me to achieve the validations done in that way. I’ve added many more custom validations into it which helps to achieve a bunch of validations which is not included in this package.
The best way of implementing the validation in this way is that its super-customizable. You can mold your validation logic and data in the way you like and you can also specify your custom human-readable error messages which each rule you define.
The BaseValidator here contains a bunch of helpful methods like — getRules(), getMessages(), getAttributeNamesForHuman(), validate() .
As soon as the validation fails the class throws a ValidationException which tells our global ErrorHandler to throw back an error response with validation messages with a status code of 422.
I bet that you’ll like the validation flow if you go through the whole implementation once.
Providers (or Service Providers)
It's nothing but just a way of bootstrapping some of the required services/components in our app. From the word “bootstrap”, I mean registering things when the app is started so that the available sources are already loaded before executing that piece of service/component.
Similarize it with the concept of Service Providers. These providers will be loaded only for once when the app is started. For demonstration purpose, I’ve used custom validators to be loaded and be available when there is a need for validating a data whenever a request arrives.
Tasks
Tasks are the short pieces of code or a type of standalone functionality which avoids unnecessary bloating of methods.
In brief, they are the standalone, reusable piece of code which are responsible to execute an independent task/functionality.
Tasks will be used whenever we need to achieve something without having its dependency on others. To explain you let me just throw some easy examples where we can fit these.
Processing/(e.g: compressing, resizing, type conversion) of images in the system can become a task so that we can rely on our tasks to complete the image processing without bloating our services for those standalone tasks.
In the following example, I’ve used tasks to prepare/create data for notifications and send them later (e.g.: notifications need some amount of specific information to create readable lines of text, so to accommodate data for that we use tasks).
Routes
Routing refers to how an application’s endpoints (URIs) respond to client requests. HTTP routes open the gateway for the outside world to interact with your application using URLs. This directory contains different files for different endpoints to make it easier to manage to route in large apps.
The basic routing Object is defined as:
[`<http-method> <uri-path>`]: {
action: Controller.methodName,
name: "unique-short-api-name",
middlewares: [
"auth.jwt", // middleware method name
]
},
...The basic example of routing is
All the routes are defined in an Object key-value pairs so that they can run in a certain defined process. That's my way of making routes work in a node JS app. A route is an object where the key represents a combination of http method and URI path combined with whitespace in-between.[http-method] [uri-path]
The value part of a route object is also an object with some key-value pairs which defines controller method instance, unique alias/name of route & middlewares which is an array of string (which actually are the method names of middlewares defined inside middlewares/index.js).
{
action: Controller.methodName,
name: "unique-alias",
middlewares: [...]
}Utilities Directory
This directory consists of all the external libraries being used in our project. Libraries like — Passport Authentication, Custom Logger, AWS Helper Class, Cloud Messaging (FCM), Mailer Class, Paginator, etc.
This directory generally contains helpers classes which are extended by the external libraries we want to use.
Config Directory
This directory contains the configuration for the whole app. The configuration could be for: global values, language-based messages/string, models, the path for directories, custom setting, etc.
Later we can create a helper method which is responsible to return a specific value from a config directory based on the arguments passed. For example:
let appName = Helper.config('settings.app.name');The above example depicts that there would be a file named settings inside our config directory and we’re fetching an object named app which has a property named name.
// File: settings.jsreturn { app: { name: "My App",
... },
....};
This is how it becomes so useful to store config values inside our config directory.
Database Directory
This directory is responsible for holding all the scripts related to DB and its transactions. Majorly its divided into 3 sub-categories:
- Migrations
It’s like a version control system for your database schema. For now, its paired with the ORM which we’re currently using — Sequelize. It's a good practice to create migrations so that no-one misses anything related to database schema updates in your repo. - Raw Queries
As we all know, great, scalable, optimized and big apps can’t be written without writing complex and optimized raw queries. So according to me we can places those raw queries in a separate module so that we can monitor all the raw queries from a single point of source.
We can use ES2016 features to write formatted queries in a string and can use patterned-slugs to replace those with actual parameters which can be guided by our code. For example:
module.exports = {
getAll: `SELECT * FROM {tableName}`
}- Seeders
For testing and filling our database to get started with the app, we need our database with some dummy entries. For those, we can take the use of seeders provided by different ORMs. Sequelize comes with the seeding capability so we can place seeder scripts in this directory
Uploads Directory
The uploads directory may be used to store user-generated files, such as profile avatars, that should be publicly accessible. You should create a route which points to this directory. You may create a route for user avatars which by default takes a user’s ID and returns avatar for the matching user from this directory.
Think of this directory as a directory where you can upload any of the data which is uploaded into your system and can be used in future using different routes. This strategy helps you to secure the storage path from the outer world as no one can know from where our server is serving a particular file.
Storage Directory
Logging is a major part of any server-side app. We need different log files to store different logs for our system. This directory is where we can place all the logs generated by our system on the basis of chunked files in different/specific directories inside the storage directory.
Views Directory
All the views are stored inside this directory. No matter whether a view is related to a front-end template or a mail template. I found this as a general practice to store view related files inside a views directory.
Public Directory
As the name depicts this directory is responsible for housing your assets such as images, Javascript, CSS/SASS, etc.
Just for the sake of your knowledge if you check app.js you can see that I’ve made this directory static and accessible to the outer world, that's the reason its available to the outer world.
// Exposing directories from server to outer world
app.use("/public", express.static(path.join(__dirname, 'public')));Docker Directory
Docker nowadays is a must use tool to deploy our app on different machines without any hassles. That's the major reason I’ve included that directory in this article. So many people have confusions regarding the placement of docker files.
I’ve found a way of storing docker related stuff inside .docker directory. It actually also used for storing docker container’s data inside it.
├───.docker
│ └───data
│ └───mysql
│ ├───db_name
│ ├───mysql
│ ├───performance_schema
│ └───sys
├───Dockerfile
├───docker-compose.yml
├───.envThat’s all folks, that's my way of structuring a Node JS App. I hope this helps you in structuring your own Node JS App. Looking forward to getting your suggestions. 🙂 ✌
