Building my own NodeJS MVC Framework — Part 1

Hello, all. Recently I’ve re-started my NodeJS studies, as I’m joining a new company and I’ll be working daily with NodeJS instead of Ruby on Rails (my main language of expertise). As a way to improve my knowledge, I’ve always enjoyed building and rebuilding things. I’m a hard believer that learning comes from creativity, so, let’s create something!

Quick notice 1: This will be a long tutorial!
Quick notice 2: This is not intended in any way to replace a well stablished framework such as ExpressJS or SailsJS, it's designed for learning purposes only, use something stable for your production code.
Quick notice 3: For this I've used Node version v8.11.4 and NPM version 5.6.0

One of the most important tools for a modern developer is a web framework, and as someone with strong Ruby on Rails backgrounds, I’d like to build and mimic my on Rails. As this process have started, I have choose to follow a few ground rules:

• Keep it simple. No giant integrations, huge helpers or super complex tasks.
• Do MVP (minimum viable product). Less is more.
• Be opinionated
• Easy to use (really easy)
• Include all the fun stuff: template parser, ORM (Object Relation Mapping) and etc.

With my ground rules set, I’ve decide to draw (or code) my final framework when it’s done. This is not what I’ve ended achieving, but very similar in initial designs. As my study progressed I realized more and more about all the tweaks. :) This is what learning is all about. So, let’s follow first my folder structure. This framework will *not* be designed to production, it’s for learning only. Here is the folder structure:

my_framework/
 - config/
 - controllers/
 - models/
 - public/
   - images/
   - javascripts/
   - stylesheets/
 - src/
   - middlewares/
 - tests/
 - views/
   - layouts/

Easy, right? Typical as most of the MVC frameworks should be. The src/ folder is where the core code will be located. In the future you can outsource this as a new package, but it's not the goal of this tutorial.

Next step. How our app will work? Let’s create a simple blog. A blog have posts, that’s enough for now. No comments, authors or authentication. Just posts, we’re building from the ground, let’s keep it simple. Let’s just design our main components to work in this “theoretical” framework.

First, the model Post. at models/post.js :

Let’s think about this design for a second. It’ll be more complex in the future.
First: setup() . Why? I want a callback to mount the model before any execution, as a included {} would work in Rails. Second, I could do magic attributes autoload stuff but I want to keep it simple, so, we need to tell the engine what attributes do we have. The API design should be this.setAttribute(name[string], type[constant/string]) . To keep it limited, we'll be exporting the available constants at Model.Attributes.
Second, one and simple hook. We can validate the attribute we want and just send a method to be parsed later during execution time. No worries about it for now. This should be enough for our model related data.

Second step, the controller: PostsController. Let’s just show an index action, which will be consumed from localhost:port/posts and list all our posts.

Here we have the first tweaks from my learnings through this goal. When fetching data, we need async/await. Otherwise we’ll hit some bad errors in the middle. If we don’t tell index() to be async and await for Post.all(), we’ll have an unhandled Promise, and it’ll not work. After a lot of tries, the solution is pretty neat, right?
As an opinionated framework, the controller name and method(action) will provide the view template. PostsController.index()should be views/posts/index.ejs . Why ejs ? It'll be the template parser we'll be using. As I've defined in my ground rules: "Keep it simple. No giant integrations, huge helpers or super complex tasks.". Build our own template parser would take a lot of time and efforts, and we already have our hands full.
You can read more about EJS here: http://ejs.co

Third step, the view:

Now, with the power of ejs we can use dynamic data inside simple views. That's awesome! linkTo is a custom method we'll be handling later. For now, we're just looping our posts and showing a table. Good enough!

Last step, the routes:

Yes. Just that! But that says a lot of what we’re going to achieve. First, Router will have 4public methods: get post put and delete . Each one mapping the correct method to fetch from the request. The first parameter will be the url to match (we're going to make it better, don't worry :)). And, at last, we're saying which controller we want and which action to treat. Now. That's simple enough to start building an awesome app.

Second step: The real framework.

We already have our final design of functional application, to build the real framework, we need to think more deep about what at the minimum core a framework should handle to provide a good structure. I’ve worked initially with the bare minimum, route and response. Now, I’ve listed what we’ll be using in this tutorial:

• http to handle the browser request/response
• fs to load internal files.
• ejs to handle our template parsing
• node-static to serve static files (css, js, 500.html)
• pg to handle the database connection. We'll be using PostgreSQL as our database.

And, our framework should be responsible for this:

• A general logger (debug, info, error)
• Post parsing (if we submit a form, we need to be able to handle the body data chunks)
• Cookie parsing (persist data)
• Flash handling (we want to exchange messages that should live for just one day)
• Data handling. Providing an easy to use ORM and connecting to the database. The ORM should be able to query, insert, update and delete data.
• Routing!
• Autoload (for the minimal magical piece, we’ll be loading all the controllers to be magically found). This will keep the Router short and simple.
• Response. It should be smart enough to handle 200 (success), 301 (redirection), 404 (not found), 500 (error) and custom responses.
• Static file serving. node-static will be responsible for this, but we have to tell it how to act.
• And, finally the most fun part in my studies: middlewares. Post parsing, cookie parsing and flash handling are actually middlewares. And we can always build new ones to happen before and after the controllers, adding a really neat execution structure.

So. We have a lot to do, and in my studies I’ve started by the ORM. I could test it and execute it before having the bare-bones for the framework. So, let’s do this as I did. So, begin by prototyping what our model could be able to do (although will be a lot of different dynamics, it will not be that complex. and we’re going to have hooks!)

So, here is a huge prototype of everything we’ll be able to do with our ORM, this should be enough to build some interesting apps.

Ok, that’s a lot to do. But I promise it won’t be that complicated. We can do some small tricks to handle the static calls (Model.find(1)) and use private data to handle our hooks, methods and variable definitions. I haven’t coded the best possible and most beautiful solution. But it works and is fun to do.

Now, let me list our private variables, they’ll be extremely important for this to work:

• this._tableName (posts, in our case)
• this._beforeSaveHooks ([])
• this._afterSaveHooks ([])
• this._validationHooks ([])
• this._attributes ({})
• this._changedAttributes ([]) // to only update what we did actually changed
• this.errors ([]) // or general error controller

Here, a list of what we will define at constructor level:

• this._tableName = this.constructor.name.toLowerCase() + “s”; // will transform Post to posts.
• this._attributes[‘id’] = {type: “integer”, value: null }; // our id
• this._attributes[‘createdAt’] = {type: “datetime”, value: null}; // createdAt timestamp handle
• this._attributes[‘updatedAt’] = {type: “date time”, value: null}; // updatedAt timestamp handle

Here a quick list of our static methods (accessed through Model.{method}): find findBy first last all where create

Our chaining methods (return this): where limit orderBy

Public methods (return true or false and update the model data): save update destroy .

Good enough. Now, let's move to Part 2 and start writing our ORM.