How I build a DNS RBL with NodeJS on 4 days, and what i learn on the way of…
…without prior knowledge of MEAN
For a long time ago, i heard about NodeJS, and their miraculous properties. Even that, i dont really think is something that can “change my world”, just another fad. About two weeks ago i read the amazing “How I built an app with 500,000 users in 5 days on a $100 server” and was really interested on it. Summer holidays and the need to spend to much time waiting on a hospital did the rest.
On sysadmin work, be attacked by remote computers is like breathe, you dont need to think on it to it happens. When you run more than 20 servers, atacks alerts comming whitout stop (i get hourly more than 500 emails about remote guys or scripts trying to log or send mail or hack me or… etc etc) so i realize on get my own DNSBL to easy block all remote attackers.
Basically, i only need 2 components:
- DNS server who act as a RBL Server
- Web page to add new IPs, inform users, show IP status, ask for IP unblock, etc
So, this is a perfect idea to test the easy-to-do-server-things NodeJS
First of, i think is important read about it, so i get 2 books:
- Introducción a NodeJS a través de koans (in Spanish)
- MongoDB: the definitive guide, 2nd edition
- MEAN.io docs
- Aprende cómo crear un API REST usando Node.js, Express y MongoDB (in Spanish)
and, more important, i spend about eight hours looking into others code. Just go to Github and search for “nodejs demo”, read, try to understand others code, search google for doubts, etc. There is no best learning way.
After about 14 hours of heating, i think i ready to jump into.
Before start a new proyect, i always do a “5-minutes-of-concentration-on-what-the-hell-i-going-to-do”. Its really important to me, so i can made a mental map of proyect, define the important things, etc
For this proyect, i focus on one thing: JUST MVP. My target is demonstrate myself that i can do that, just for fun and learning, so i force myself on doing something functional. No design, no UI, no nothing. Just a small MVP of a RBL system.
So far this proyect should be run on Ubuntu Xenial VPS, i made a Vagrant machine and copy all installation steps on a simple shell script. That way, i can reproduce it on final VPS without trouble.
I want to use the less number of nodejs libraries, to avoid reading to much api docs and made it more simple. Final libraries looks like:
- dnsd: Core library on RBL server
- whois: To add whois information on new IPs
- mongoose: Simple object modeling for mongo, used to validate data
- mongo: For simple querys, i use mongo pure querys
- express: The web environment
- pm2: In global install, to handle al daemon options.
I start on RBL server, who seems to be more easy. That was a big mistake. So far i not really understood events oriented mode, i spend about one hour trying to solve a stupid problem related with functions order. Once i understood where the problem are, i changed my development structure (separate functions on diferentes files, etc) and program all the RBL server on one single file, who has about five threaded functions, and no more that 70 lines. This code sucks too much, but is (relative) easy to read and follow, and protect me from my own knowledge emptiness.
Build the RBL server took me about 2 hours of real work, after two fully rebuilds. First mission acomplish ;)
Once server is done, its time to move to web page. I use a simple template which only has three folders:
- controllers: One file wich functions related to url. In example, file ip.js handle all /api/ip related functions.
- models: One file witch have mongoose object by mongo collection.
- views: All views files. In my case, i use .ejs templates because is more related to my view concept. Next step is Jade.
I develop the API part on about 30 minutes. Validation API request are simple, and so far i use mongoose, when validate fails i just return mongoose errors, so its really simple work on it.
Also, express are really API friendly. Most part of work is get data, use it on mongoose object, and return something like return res.status(500).send(err) or res.status(200).jsonp(object). Simple API function file should be about 70 ~ 100 lines of code, which is easy to see and mantain, even by a noob like me ;)
Web part are a little bit complicated, because i want to use Angular without previous reading of it. When i decided to drop Angular and use pure HTML and POST/GET requests, it took about 3 horus get all running on. Again, really easy to do.
Conclusions and advices
About 4 days of working on, and it is alive !. My current code sucks, its dirty and need a full review. But it works, and this is just amazing due the time i use on it, and my poorly knowledge.
I can really explain who much fun and easy is develop on NodeJS once you understood they basic principles. Also, it runs really fine on a simple VPS. Simple tests say it can handle more thousands DNS request without any load on a cheap VPS
Even that, there are somethings you should take care of:
- You must know and understand really well how events work, how NodeJS use CPU times, and why this is so important.
- Try to find a library before program some common function. NodeJS has billions of librarys who can help you.
- Any library on NodeJS has his own API and docs, and should be read carefully. It is complex and boring, but you can avoid hours of debug doing it.
- Do it fast. Avoid the “I want to use that library/framework or watever”. Please, simplifly. Use pure mongodb commands instead of Mongoose, etc etc.
- Dont be afraid of use to much threaded functions. Yes, your code dont look “expert”, but you avoid problems. Once it works fine, its time to clean it.
- Start always with the less features as you can, and grow slowly.
Is NodeJS the best language ever ?
well, it depends. What you wanna do ? An easy MVP ? An complex web proyect like facebook ?
As always, all language or plataform has good and bad things, and i now sure that NodeJS is not “the solution to everything”. But its so cool, and you should know it.
And, for now, i really love it.