Debugging Node.js Applications in Cloud Foundry
The other day some colleagues of mine and I were tasked with testing out the process of debugging a Node.js application running in Cloud Foundry.
Before I go any further into the details of how this was done, I feel compelled to share why and when it should be done. First and foremost, this is never, ever to be done in production. Dropping a breakpoint into a single-threaded application running in production with tens of thousands of active consumers is not a good idea. By not good I really mean catastrophic. Many kittens will die. Unicorns will lose their horns. The sky will fall.
Even if you don’t use a breakpoint, interacting with the debugger (especially with Node) can have adverse effects on the app — everything from degraded performance to outright crashes or hangs. Did I mention you should never do this in production?
What I’m describing in this post is how to attach a debugger to an application running in a testing environment. In production, you’ll need to rely on application monitoring tools, metrics, and log analysis for diagnosis. Cloud native applications deployed in production are like satellites in orbit — you can’t treat them as though they are reachable with a tether.
The application we intend to debug has to have the node-inspector module in it, which, when you run npm install, will add the node-inspector module in the node-modules/.bin directory. The dependency in package.json looks like this:
We’re going to need this dependency later.
Now that we have an app to debug, we need to push it to Cloud Foundry with an altered start command. This start command runs the Node application with the debugger active. This exposes a debugger port inside the app’s container on port 5858.
cf push myapp -c “node --debug server.js”
At this point, if your app started, you should see that your app is responding as it would normally, and you also have a trace message (you can see this with the cf logs CLI command) that looks like the following:
debugger listening on port 5858
Note that the port indicated in that trace message only applies inside your container. We’re going to have to do a bit of (somewhat dangerous) poking and prodding in the container to expose the debugger port outside.
Now that we’ve got our app running in debug mode, we’ll need to ssh into our container:
$ cf ssh myapp
$ export PATH=$PATH:/app/.heroku/node/bin
Here we’re adding the path to the node executable to our shell’s path so that we can invoke node scripts like node inspector. Next, run the following command from your /app/node_modules/.bin directory:
$ ./node-inspector --web-port=9090
If node inspector starts properly, you’ll see the following message in your ssh output:
Visit http://127.0.0.1:9090/?port=5858 to start debugging.
At this point, we’ve got our node application running internally on some port (usually 8080). We’ve got a debugger responding internally on port 5858. And we also have a web proxy for the debugger (node inspector) running internally on port 9090.
To make this at all useful, we need to expose the node inspector outside the container, usually on our local workstation via localhost. This allows us to use the URL we saw from the earlier trace message in a local Chrome tab.
To do that, we’re going to use cf ssh to create a tunnel for port 9090.
cf ssh -N -T -L 9090:localhost:9090 myapp
This will sit in your terminal and keep the ssh tunnel active until you Ctrl+C out of it. So you might want to do this in a separate terminal tab just to keep things flexible.
Note that we don’t need to tunnel any of the other ports. Ports 8080 and 5858 are fine running internally in the app’s container. The Cloud Foundry router is taking care of getting regular HTTP traffic to the app (port 8080 inside), and the node inspector proxy is going to talk internally to the port 5858 debugger.
Now we’re ready to debug! (finally!)
Next hit your application using its Cloud Foundry host and domain, e.g. http://myapp.my.domain.com. Now you’re remotely debugging a Node app in Cloud Foundry.
Many thanks to Rohit Kelapure for helping me walk through these steps and overcome my base level of disdain for all things Node.js.