Node.js Configuration 101

We’ve established that configuration is important. Great! But how should you use configuration in your Node.js projects? Our approach to configuration is described below. It’s not the only way to configure your project, but this is how we’ve chosen to do it.

This is part two of a four part series — you can start with part 1 below!

Why Configuration Matters

Configuration Automation

We’ve open-sourced our configuration module, which is our way of giving back a little to the community. It’s exactly the same one we use in our projects. You can find it at https://github.com/myob-oss/node-config. It’s also published on npm and you can install it by running the following command:

npm install --save @myob-oss/config

The module we’ve written was inspired by other configuration libraries (such as Ruby on Rails, Apache Sling and other Node.js configuration modules). What we were after was a simple and predictable configuration module — one where the running configuration was easy to understand, and straightforward to change settings.

Our configuration module is nothing new, or overly fancy. It’s just mildly opinionated and written for a single purpose. The configuration files are not able to be changed by the application that is using them. The configuration files then become artefacts which makes understanding the running configuration very easy. In our production environments, the automatically generated runtime.json file has all the configuration the application uses. If we want to know what settings the application is using are we just look at that file. Simple! Predictable!

The main power and flexibility of our configuration module boils down to the concept of overlays. By this I mean that you can overlay configuration from a default environment, with settings for the machine environment, and finally you can overlay a runtime environment over the top of the other two settings for the final configuration settings.

Our configuration module expects all configuration files to be stored in the /config directory at the root of your project. The configuration files are in JSON format. There are two “magic” files that it uses: default.json and runtime.json. The default.json contains settings, which are common across all the environments, and the runtime.json file contains settings which are specific to the machine that the project is running on.

Each project also has environment configuration files: dev.json, test.json and production.json (we have others for the other environments we run). These environment files usually contain the database connection details and service addresses that correspond to each environment. We use the NODE_ENV environment variable to determine which environment file to use.

We don’t store passwords or credentials in any of our configuration files which are tracked by source control. You should never do that! This is to make sure that developers don’t have access to production database details or any other sensitive access information.

The best way to visualise how the overlay system works is with an example. Imagine we had a default.json file with the following settings:

{
  "foo": "bar",
  "barry": "has hair"
}

And also imagine we had a test.json file with the following settings:

{
  "barry": "has a chrome dome",
  "alice": "is a coder"
}

And finally imagine we had a runtime.json file with the following settings:

{
  "alice": "is an awesome coder",
  "foo": "baz",
  "fruit": "mango"
}

Then the final configuration result for NODE_ENV=test would be:

{
  "foo": "baz",
  "barry": "has a chrome dome",
  "alice": "is an awesome coder",
  "fruit": "mango"
}

Here you can see that the value for “barry” has been overlaid from the default.json value by the environment test.json value, and the value for “alice” has been overlaid from the environment test.json by the runtime.json value. Obviously these are made-up settings, but they should give you the basic understanding of how the overlay system works.

As the configuration files are in JSON format any key can have a value which can be: a string, a number, an array, or an object. Having objects is great for organising sections in the config files:

{
  "express": {
    "host": "localhost",
    "port": 1337
  },
  "mysql": {
    "host": "192.168.56.101",
    "port": 3306
  }
}

We don’t store the runtime.json files in our source code repositories (we ignore the file in out git repositories). This is due to three reasons.

First, the runtime.json file is generated with our auto-deploy process, and we don’t want local developer settings accidentally running on our servers.

Second, we don’t want to break the projects on our local machines (or machines of other developers) by updating our code and the configuration changes without us realising it.

Third, passwords and credentials may be stored in the runtime.json file and we don’t want that information in our repositories.

Every developer likes to work a little bit differently. I’ve never met two developers who work in exactly the same way. Our configuration module affords the developer the freedom to set up their machine the way they want. Anything that’s different from the base environment they can add in to their own runtime.json file and it won’t interfere with anyone else.

All of our team’s Node.js projects use the same configuration module. Once a developer has learned how to use it, they can then easily understand how all the other Node.js projects are configured. They’re able to quickly and easily make changes to the configuration or add new options as required. The best thing is that they only need to learn one type of configuration.

I hope this has been useful to help understand some of the inner workings of the way that our team configure our projects.

The next post in this series will explain how our configuration gets generated automatically when our apps get deployed.

In the meantime, feel free to drop any questions or feedback into the comments section below!