Grown-up Node: Looking at Logging
I’m Rami, an engineer at Social Tables, where we build award-winning software for meetings and events, using the latest web technologies like SVG, HTML5, and Node.js! Among other things, I work on our server infrastructure, which includes taking care of our logs…
Logging is not the sexiest of topics. No, there’s no use in denying it, it’s true! Lots of people get excited about NoSQL (MVCC! Vector clocks!), or about shadow DOMs (React.js! Web Components!), but it’s relatively rare for someone to get excited about logging.
And yet logging is crucial for a production application, and any “grown-up” application platform needs to have a solution for it.
I Missed The Memo: Why Is Logging So Important?
Essentially, logs are our primary window into a running application, especially in production. In fact, in cases where we’re trying to inspect application state in the past (say, when debugging a past failure) they are our only window into the application. Thus logs are invaluable for debugging: finding places where application code or data is in an unexpected or invalid state.
They can also be the basis of monitoring, keeping detailed records of timings (for performance tuning), memory usage, uptime, and other aspects of application state. In addition, they are great tools for analysis and insights after the fact, such as building up a picture of users’ activity from a log trail.
Common Factors
At this point most of the best practices have already been distilled out of years of industry experience building web applications. Node, as a younger ecosystem, has incorporated a lot of them: a hierarchy of log levels (where only important errors are output at error level, those and less-crucial warnings at warn, all of those and more at info, and so on), a multiplicity of output options and formats, a hierarchy of loggers (where one logger is a ‘child’ of another and inherits much of its configuration), and so on.
Some Different Approaches In Node
Log4js is a deliberate port of the venerable Apache Log4j Java APIs, which are well-known and battle-tested ways of providing logging for an application. It provides everything mentioned above: a hierarchy of log levels, a wide range of configuration options, and a few helpers (like a built-in Connect middleware).
And a more involved example of a config.json:
Intel aims to be more powerful, more configurable, and more Node-focused than Log4js, which still has a faint whiff of Java around its API. Intel provides as many handlers and filters, full support for named loggers and complex configuration, but also fully hooks into the Node ecosystem. It provides a simple way to override the global console and pull in third-party modules’ output into your logging system, or even intercept use of the popular debug instrumentation library.
Another strong contender is a system used internally at Joyent (corporate stewards of the Node platform): bunyan. Bunyan takes a different approach to log output and consumption — it produces logs by default in JSON format, and provides a command-line tool to present them for humans if necessary. It codifies the assumption that there will be an ecosystem of tools on the other end to parse and analyze logs, and encourages more detailed logging. It also provides cool features like runtime log snooping, alongside an array of tools like custom serializers.
YAL is designed from the ground up for a more sophisticated setup, where streams of log data are being distributed to several remote destinations. It builds on top of axon to provide message-oriented network transmission and provides a handy server component for the receiving end. It’s simple and lightweight, and assumes you otherwise have everything set up the way that you want.
All Right, What Do You Recommend, Then?
What kind of logging setup you decide on depends very much on your project and its needs. Here at Social Tables, we needed simplicity, to drive developer adoption of good logging practices; the flexibility to output to streams, to ensure performance doesn’t take a hit; and ease of analysis, since we’re using logs as a fallback for metrics and monitoring as well as to build a story of user interactions. We chose lugg, a wrapper around the bunyan tool I’ve already mentioned, that brings the ease of use of debug instrumentation to structured logging (and combined it with our own syslog setup for aggregation).
What Do I Do With All These Logs?
So now you have logs. Great! There’s a plethora of things that you can do with your log data, as I’ve already suggested — debugging, alerting, low-level monitoring, extracting metrics, and more. You can send them to aggregation and analysis services like Splunk or Loggly, or build your own on top of tools like Fluentd or Kibana. There’s a wealth of insights in your logs — go find them!
