Migrating from PHP to Node.js
A UI Developer’s experience.
I’ve been developing web apps in PHP for years but recently have been building more iOS apps. When I move back to web development I really miss the elegance of Objective-C and it’s MVC structure. I was able to find a similar structure for my client side JavaScript but my server side code was left to the confines of my comfort zone and PHP. The great thing about PHP is its ubiquity and low barrier of entry. You can get a web app up and working in almost no time however the code itself can quickly devolve to a mountain of spaghetti code. After almost every job I do using PHP I tell myself it’s time to start working smarter. I’ll then start investigating new frameworks for PHP and think about jumping ship completely to new stack that uses Java, Ruby or Python. I’ll start tinkering with the new approach and build out simple apps. What inevitably happens though is a new job comes in and I resort back to my domain knowledge of PHP. This is partially due to me, but at the same time its problem of the platform I’m trying to learn. I’ve found the entry points to make solid apps using other languages much higher than I did with PHP. This is partially why PHP is such a monster in the industry and why Rails and Django are typically relegated to the purists.
On top of all the popular alternatives to PHP there is a relatively new kid on the block called Node.js. When I started tinkering with Node a few years back it was refreshing to lean on the JavaScript knowledge I had to create server side code. However Node never seemed like it would be a realistic competitor to the big languages when it came to writing complete server side code; it was just too new and unproven.
Recently I took another look into using Node as my primary server side language and I was blown away with it’s progress and community. It has matured enough to be used as a essential tool for any web developer, but especially for UI/front-end developers. It was time to dig into Node again and see if it would break me away from PHP once and for all.
Node 101
Getting Node up and running on my MacBook was as simple as you could hope. Here is how the OSX install looks:
//Install homebrew then Node
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"brew install node
After that you’re ready to run JavaScript code on your server. The Node ‘Hello World’ app shows the basic code structure while at the same time illustrates that it’s a very capable web server. With only 2 lines in the terminal you’ll have all the power of JavaScript at your disposal and a web server to boot. Try it out, open a text editor and put in this code:
var http = require('http');http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(1337, '127.0.0.1');console.log('Server running at http://127.0.0.1:1337/');
Save your file as app.js and from the terminal run:
node app.jsLoad up your browser and goto http://localhost:1337 and voila!
This gets you to where you might ask, “ok, now what?” At least that’s what I did when I first gave it a crack. I didn’t know or understand how to apply any of my existing knowledge on how to build out a real proper web app from this example.
NPM
Node gets really powerful when you start adding to it. The Node Package Manager* or NPM is included with the install of Node. The community has created some invaluable apps, tools, frameworks and middleware to help UI developers focus on the key tasks at hand; making great user experiences. If you are experienced with composer when building your PHP web apps then using NPM won’t be much of a reach for you. There are some great sites out there to help with understanding the ins and outs of NPM so I’ll just mention a couple of key things. NPM uses a file called package.json to outline the metadata of your app and most importantly your web app software dependancies. You can edit this file manually and fill in the dependencies on your own just like you would with composer’s composer.json file. You can also install packages from the command line like so:
npm install <package_name> --saveI typically add the ‘— save’ option so the package gets added to the dependency list in my package.json file.
Beyond your web app, NPM houses some really great server side apps that you will want to use to make your life easier. You can install packages within your local app directory or you can choose to install packages globally on your system. Installing them globally will give you access to the package from the command line (you will have to add the NPM bin folder to your terminal’s PATH). Installing a package globally looks like this:
npm install -g <package_name>After learning how to manage packages there are some essential ones you’ll want to install before getting your feet wet with Node.
Building and Porting Websites
I haven’t been playing with Node for too long but I have come across some essential things to get me moving in the right direction. Here are a selection of the ones I found most helpful.
Express
Perhaps the single most important component of building successful web apps in Node is Express. It’s a middleware package that builds on the foundations of Node by adding some all important structure to building web apps with it. Node itself delegates a lot of the web server responsibility to the programmer. Express builds upon a middleware package called Connect that fills in this gap so developers don’t have to worry about the little details of parsing input and serving pages. On top of that Express provides a very nice framework for a developer to follow and create rich and sustainable code upon. Express has become the de-facto standard for building web apps in Node. I won’t be building a web app without it. There is a fantastic guide over at ExpressJS.com to get going: http://expressjs.com/guide.html
Note: I was informed that there is a new framework being built by the Express team called KoaJS. Definitely worth checking out: http://koajs.com/ Thanks @jeswin!
On the Express homepage there is a link to a very helpful book as well. It’ll run you through a very in depth overview of Node and builds up towards making web apps using Express. I found it very helpful in understanding the barebones of Node before jumping into Express. Have a look: http://www.manning.com/cantelon/
Nodemon
Running and testing a Node app is quite a lot different than building one in PHP. Typically with PHP you hack through some code and then head over to the browser and refresh. Every now and then you might need to modify the Apache config and restart the web server. The process in Node is quite a lot different. Since the web server and application are bundled into one tight little package, changing the code meant having to restart the whole web app; which also meant restarting the web server. When I was learning how to build the basics it seemed like there was an endless amount of killing the process, hitting the up arrow to goto my last command and then hitting enter to restart the app ([Ctrl+C][Up][Enter] cycle). This quickly became a giant pain in the ass. For people to make meaningful work in Node this had to improve. Fortunately this is where Nodemon comes to the rescue. Install Nodemon using NPM globally and instead of starting your app using:
node app.jsus the following:
nodemon app.jsThis app will monitor changes in the code and automatically restart Node for you. Very straightforward and very essential. Have a look at the git repo for more details: https://github.com/remy/nodemon
Swig
When I built PHP web apps my template engine of choice was Twig from the good folks at SensioLabs. It was a component of the Symfony framework and made putting together front end HTML a breeze. Twig was an essential tool for me and if I was going to port any of my existing projects to Node, I had to re-use the view templates I already had. Swig made this transition incredibly simple. Almost every part of Twig that I used was available in Swig. I had to make a couple minor modifications to the template code, but overall it was ready out of the box. It’s completely compatible with Express and has a very helpful guide to getting going here: http://paularmstrong.github.io/swig/docs/.
There is a direct port of Twig to Node but unfortunately that project has stalled. Swig is very actively updated so I opted to use it going forward.
Note: I assume others might find this helpful given Twig is one of the most popular template engines in PHP. Beyond Swig there are numerous template engines available for Node. All of the popular ones have the same or similar functionality; it’s just about syntax differences for the most part.
MySQL for Node
When you read about Node most of the time it’s paired with a NoSQL type databases like MongoDB. Going forward I will most likely use Mongo simply due to the amazing Mongoose middleware. It makes model management within web apps incredibly straightforward. However, given that I have a few legacy projects to manage and port over, I want to keep as much in place as I could. This meant keeping my existing MySQL schemas and data in place. There are a few MySQL packages available but I found the one by felixge incredibly full featured. Have a look here for more information and usage: https://github.com/felixge/node-mysql
PassportJS
Almost all web apps these days has some sort of connectivity with third party APIs. To authenticate with these services has its challenges. Fortunately there are numerous packages available in PHP to make accessing these APIs straightforward. With Node you really only need to concern yourself with PassportJS. It’s an Express compatible package that allows you to authenticate just about anywhere on the web. You can even use it to authenticate your own local login and manage sessions if need be. I found this tool invaluable when authenticating with other services like Twitter, Instagram and Facebook. From the software’s wiki, it says it can authenticate with 140 different services. A must have for any web app developer. Have a look at the details here: http://passportjs.org/guide/
Starting the engines
After understanding the Node fundamentals and getting familiar with the new stack I was ready to jump in and start creating apps. It took about a day or so to read through the NodeJS in Action book and work through a few examples using Express. With a basic understanding with how the mechanics worked, the next task was getting one of my old projects ported over. For a small to medium sized multilingual site the porting process took about 2-3 days to do. The speed at which I port sites would no doubt increase as I become more familiar with the process and structures of Node and Express. Here are a couple things that are worthy notes for others moving from PHP.
Synchronous vs Asynchronous
One of the biggest changes I had to wrap my head around was converting my synchronous PHP code into asynchronous JavaScript code. Since Node is built on a single thread non-blocking event loop system, any action you do won’t wait for a blocking action to return before executing the next line of code. All lines of code are executed immediately one after the other. The idea that the requested data will be available on the next line of code is not a guarantee. This is where your front-end JavaScript experience will help. Just like the client side, events are handled via callback functions and the same goes for Javascript code on the server. To create synchronous code using JavaScript you’ll need to nest your sequence of events in callbacks. Here is an example:
//PHP
$sql1 = "SELECT user_id FROM users WHERE email=?";
$stmt = DB()->prepare($sql);
$stmt->execute( array($email) );$sql2 = "SELECT photo_id FROM photos WHERE user_id=?";
$stmt = DB()->prepare($sql);
$stmt->execute( array($user_id) );
Just what you would expect. PHP will execute and wait for a result of the first query and then use the data received in the second. In Node you would have to nest the queries via the completion callbacks like so:
var sql1 = "SELECT user_id FROM users WHERE email=?";
connection.query( sql1, [email], function(err, rows)
{
var sql2 = "SELECT photo_id FROM photos WHERE user_id=?";
connection.query( sql2, [rows[0].user_id], function(err, rows)
{
console.log(rows); //output all the photos
}
}You might think that this type of code will grow to be absolutely unwieldily when you have a lot synchronous code to execute, and you’re right it does. Using promises solves this “Pyramid of Doom” problem. I won’t go into promises in this article but there is a great package for Node called Q and it gives you an idea of how promises will clean this up. Have a look here: http://documentup.com/kriskowal/q/
Bugs are really bad
Since Node is a single thread system it’s important to keep in mind that when your code has an uncaught exception it will bring down Node for everyone not just the person who caused the crash. This is different than PHP and Apache/Nginx because the thread that handled the request dies, not the entire web server process. In a dev environment this is totally fine and probably the encouraged so you can properly catch and fix issues. However in the production environment this can be very disruptive since your web app will be taken down for all users visiting your site. The way around this is to create a listener for all uncaught exceptions and to handle them gracefully (see note below). The code looks like this:
process.on('uncaughtException', function (err) {
console.log('Caught exception: ' + err);
});This will ensure your app stays up even when the unexpected happens.
Note: I’ve read a little deeper into this issue and it appears that handling uncaught exceptions this way isn’t ideal. Using domains appears to be the better solution. Read about it in the Node API docs. Another option would be to use a tool like forever to start the app with. It will automatically restart the server when the unexpected happens.
Rewrite Rules
One of the powerful parts of web app development is the ability to rewrite messy URLs into nice clean ones. This makes search engines happy and makes your users even happier. With PHP it didn’t care what the URLs looked like, it just wanted input to work with. The re-writing of the URLs was managed by Apache (or Nginx). When using Node to build web apps, Express’ route mapping is the special sauce to directing requests. I found managing routes through my server side app code a very welcome change. After getting comfortable with this idea you’ll never want to write another routing rule through your <VirtualHost> entry ever again. Here is a small example:
RewriteRule ^/blog/([A-Za-z-]+)$ /blog/entry.php?slug=$1Using Express’ route mapping:
app.get( '/blog/:slug([A-Za-z-]+)', blogController.entry );No more running around managing request route mapping between your web server and your web app code. <Applause>.
Running multiple Node sites on one server*
Since Node is both a processor of code and a web server built into one, it presents a challenge when running multiple sites in a single environment. Running a Node web server requires you to select a port to run requests through. Once you run a site on port 80, you can’t map a second site to the same port. One might just tell you to get a new VPS for each website given they are so affordable these days however in a lot of cases a single VPS is complete overkill for one site. In these cases you can choose to run a standard web server like Apache or Nginx and pass the requests through to Node via proxy. I opted to proxy using Apache rewrites because I use them so often. Here is what an example <VirtualHost> entry looks like:
<VirtualHost *:80>
ServerName example.com
ServerAlias another-example.com
DocumentRoot /my/folder/
RewriteEngine On
RewriteRule ^(/.*)$ http://%{HTTP_HOST}:1337$1 [P]
</VirtualHost>For Nginx it would looks something like this:
server {
listen 80;
server_name your-domain.com;
location / {
proxy_pass http://localhost:{YOUR_PORT};
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection ‘upgrade’;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}You can also choose a pure JavaScript approach by using the Node-Http-Proxy package. Use what makes sense for you.
*Note: I’ve been getting a lot of feedback on this topic and most are saying that using Nginx isn’t as big of a bottleneck as I originally made out and in some ways using it as a proxy will help more than it’ll hinder. My point was supposed to be about getting multiple node instances running in the same environment so I’ve re-written this point to be more helpful. I’ll leave the performance debate for the community as I haven’t tested much in a busy production environment. Thx Reddit!
Seek and you shall find
One of the most exciting parts about programming with Node is the community that surrounds it. There is such much happening in the space and it’s happening very quickly. When I was porting my first site over I would run into a snag and ask “How do I do this in Node?” I would simply google whatever my problem was and sure enough within seconds I would have answers. I never came across a deal breaker problem that just wasn’t easily solvable using Node. It’s a testament to the progress it’s made. If you run into a problem, just google it, no doubt someone has a solution.
Wrap Up
After spending about 3-4 days with Node and its counterparts I was completely blown away with how easy it was to get running and most importantly how easy it was to get to work making things. The barrier of entry for developing web apps using Node is still not as low as PHPs but it’s pretty close! Even saying that, the value you get from understanding package managers and MVC frameworks are essential to be a modern day web developer. Those two concepts are probably the only things keeping Node+Express’s barrier of entry higher than PHP’s. Node+Express force you to learn these concepts before you can get cooking and that is a really good thing for a developer and the code they write.
Finally, I think its important to understand why writing server side code in JavaScript is such a great thing for a web developer and specifically the UI/Front-end web developer.
Full Control
Most UI web developers already have an intimate knowledge of JavaScript and it’s nuances. Taking advantage of this knowledge when preparing server side code increases productivity. In addition, my experience tells me business logic/back end developers typically loath putting the UI code together for building web sites. They would rather focus on the engineering of the core functions of the app. Creating apps using Node allows the UI developer to take over the UI portion of the server side app. The business logic can be completely separated from the UI and accessed through API calls. Separating the back-end from the front-end allows each developer choose the stack they want to work with to get their job done.
There is a great article that goes into more detail about the idea and I couldn’t recommend it more: http://www.nczonline.net/blog/2013/10/07/node-js-and-the-new-web-front-end/
Given this recent experience I have committed to moving all future projects to Node (where possible) and committing to a full JavaScript stack for UI development. It’s been an essential language for building amazing user experiences and now it can be fully leveraged to make great server side code as well. I honestly believe it will become the dominate language for building front-end web app code going forward. I applaud and thank the community of developers who put their hard work into making these amazing tools so guys like me can use them to make great web sites.
I am a front-end developer and photographer. I support creative industries with great concepts, code and craft. Check my work out at HeadlandsDigital.com or connect with me on Twitter @IAmMattQ.
