How I build a DNS RBL with NodeJS on 4 days, and what i learn on the way of…

…without prior knowledge of MEAN

For a long time ago, i heard about NodeJS, and their miraculous properties. Even that, i dont really think is something that can “change my world”, just another fad. About two weeks ago i read the amazing “How I built an app with 500,000 users in 5 days on a $100 server” and was really interested on it. Summer holidays and the need to spend to much time waiting on a hospital did the rest.

Previous status

My previous status about MEAN proyects is so simple: I had no idea. I heard about it on diferent ways but… never use it, read about it, be courious about it, thinking on it or whatever. I have a good JavaScript skill and can do very good things with JQuery, thats all.

The idea

On sysadmin work, be attacked by remote computers is like breathe, you dont need to think on it to it happens. When you run more than 20 servers, atacks alerts comming whitout stop (i get hourly more than 500 emails about remote guys or scripts trying to log or send mail or hack me or… etc etc) so i realize on get my own DNSBL to easy block all remote attackers.

Basically, i only need 2 components:

  • DNS server who act as a RBL Server
  • Web page to add new IPs, inform users, show IP status, ask for IP unblock, etc

So, this is a perfect idea to test the easy-to-do-server-things NodeJS

Heating

First of, i think is important read about it, so i get 2 books:

and, more important, i spend about eight hours looking into others code. Just go to Github and search for “nodejs demo”, read, try to understand others code, search google for doubts, etc. There is no best learning way.

After about 14 hours of heating, i think i ready to jump into.

Concentration

Before start a new proyect, i always do a “5-minutes-of-concentration-on-what-the-hell-i-going-to-do”. Its really important to me, so i can made a mental map of proyect, define the important things, etc

For this proyect, i focus on one thing: JUST MVP. My target is demonstrate myself that i can do that, just for fun and learning, so i force myself on doing something functional. No design, no UI, no nothing. Just a small MVP of a RBL system.

Develop environment

So far this proyect should be run on Ubuntu Xenial VPS, i made a Vagrant machine and copy all installation steps on a simple shell script. That way, i can reproduce it on final VPS without trouble.

Other ways like use Puppet or Chef seems to be more complicated.

Node libraries

I want to use the less number of nodejs libraries, to avoid reading to much api docs and made it more simple. Final libraries looks like:

  • dnsd: Core library on RBL server
  • whois: To add whois information on new IPs
  • mongoose: Simple object modeling for mongo, used to validate data
  • mongo: For simple querys, i use mongo pure querys
  • express: The web environment
  • pm2: In global install, to handle al daemon options.

Development proccess

I start on RBL server, who seems to be more easy. That was a big mistake. So far i not really understood events oriented mode, i spend about one hour trying to solve a stupid problem related with functions order. Once i understood where the problem are, i changed my development structure (separate functions on diferentes files, etc) and program all the RBL server on one single file, who has about five threaded functions, and no more that 70 lines. This code sucks too much, but is (relative) easy to read and follow, and protect me from my own knowledge emptiness.

Build the RBL server took me about 2 hours of real work, after two fully rebuilds. First mission acomplish ;)

Once server is done, its time to move to web page. I use a simple template which only has three folders:

  • controllers: One file wich functions related to url. In example, file ip.js handle all /api/ip related functions.
  • models: One file witch have mongoose object by mongo collection.
  • views: All views files. In my case, i use .ejs templates because is more related to my view concept. Next step is Jade.

I develop the API part on about 30 minutes. Validation API request are simple, and so far i use mongoose, when validate fails i just return mongoose errors, so its really simple work on it.

Also, express are really API friendly. Most part of work is get data, use it on mongoose object, and return something like return res.status(500).send(err) or res.status(200).jsonp(object). Simple API function file should be about 70 ~ 100 lines of code, which is easy to see and mantain, even by a noob like me ;)

Web part are a little bit complicated, because i want to use Angular without previous reading of it. When i decided to drop Angular and use pure HTML and POST/GET requests, it took about 3 horus get all running on. Again, really easy to do.

Conclusions and advices

About 4 days of working on, and it is alive !. My current code sucks, its dirty and need a full review. But it works, and this is just amazing due the time i use on it, and my poorly knowledge.

I can really explain who much fun and easy is develop on NodeJS once you understood they basic principles. Also, it runs really fine on a simple VPS. Simple tests say it can handle more thousands DNS request without any load on a cheap VPS

Even that, there are somethings you should take care of:

  • You must know and understand really well how events work, how NodeJS use CPU times, and why this is so important.
  • Try to find a library before program some common function. NodeJS has billions of librarys who can help you.
  • Any library on NodeJS has his own API and docs, and should be read carefully. It is complex and boring, but you can avoid hours of debug doing it.
  • Do it fast. Avoid the “I want to use that library/framework or watever”. Please, simplifly. Use pure mongodb commands instead of Mongoose, etc etc.
  • Dont be afraid of use to much threaded functions. Yes, your code dont look “expert”, but you avoid problems. Once it works fine, its time to clean it.
  • Start always with the less features as you can, and grow slowly.

Is NodeJS the best language ever ?

well, it depends. What you wanna do ? An easy MVP ? An complex web proyect like facebook ?

NodeJS seems to be really easy to use, but it can be a big headache if you dont really understood it. Also, you always going to need a VPS to run your app, and this is not open to anyone. I cannot find a good “shared NodeJS hosting” until now (maybe its time to made a MVP on “shared NodeJS hosting ;-)”. Code on NodeJS is code in JavaScript, and JavaScript sucks to much. Last, but not least, some things than is fully supported in languages like PHP are complex to do in NodeJS.

As always, all language or plataform has good and bad things, and i now sure that NodeJS is not “the solution to everything”. But its so cool, and you should know it.

And, for now, i really love it.