Config management for Node.js based on runtime environment variables

Managing configs for your apps often gets tricky and ugly in this world of microservices when you have different upstream APIs and different configs for dev, staging, QA and production environments. If you lookup at npmjs.org there are different config loaders and managers available as modules, but none of them seems to follow a well defined and unified universal approach.

I started following this approach of runtime env variable based config management for my Express.js and Node.js apps, which I have been using at scale for the past 3–4 yrs. The concept is simple as based on NODE_ENV environment variable, whether it is set to staging ,production , etc. a similarly named JSON config file is loaded and parsed by the app server when it is instantiated.

Let’s setup a simple Express.js app with this approach. This approach can be adopted in other platforms like Go-lang where instead of a JSON file, YAML files can be used to store configs. You can go through the example app https://github.com/jinmatt/foo-config-app

# Using express generator to scaffold a basic app
$ express foo-config-app
# Using yarn to install dependencies
$ cd foo-config-app
$ yarn

This gives you a folder structure:

.
├── views
├── bin
| ├── www
├── node_modules
├── public
├── routes
├── views
└── app.js
└── package.json
└── yarn.lock

If you look at bin/www when you do npm start the server is going to start on port 3000 or a port set by the env variable PORT

var port = normalizePort(process.env.PORT || '3000');
app.set('port', port);

Say we want our server to run on port 3000 in staging and on port 80 on production environment. Let’s build a config store for this:

$ mkdir config
$ touch config/staging.json
$ touch config/production.json

Staging config: staging.json

{
"server": {
"port": 3000
}
}

Production config: production.json

{
"server": {
"port": 80
}
}

Now let’s build our simple config loader:

touch config/index.js

vim config/index.js:

const fs = require('fs');
const path = require('path');
const NODE_ENV = process.env.NODE_ENV;
let configBuffer = null;

// Init config_buffer according to the NODE_ENV
switch (NODE_ENV) {
case 'production':
configBuffer = fs.readFileSync(path.resolve(__dirname, 'production.json'), 'utf-8');
break;
case 'staging':
configBuffer = fs.readFileSync(path.resolve(__dirname, 'staging.json'), 'utf-8');
break;
default:
configBuffer = fs.readFileSync(path.resolve(__dirname, 'default.json'), 'utf-8');
}

let config = JSON.parse(configBuffer);
module.exports = config;

Edit startup script bin/www to use this config:

#!/usr/bin/env node

/**
* Module dependencies.
*/

var app = require('../app');
var debug = require('debug')('foo-config-app:server');
var http = require('http');
const config = require(__base + 'config');

/**
* Get port from environment and store in Express.
*/

var port = normalizePort(config.server.port);
app.set('port', port);
...

In app.js I have set global.__base = __dirname + '\'; so config module can be loaded as require(__base + 'config') instead of specifying the absolute path. Read more on this here.

So now when you launch your app with staging or production NODE_ENV it’s going to load the corresponding config based on the env name and start the server on port 3000 if staging or on 80 if production:

$ NODE_ENV=staging npm start# OR$ export NODE_ENV=production
$ npm start

If NODE_ENV is not set, a default config will be loaded and can be used as the development environment configuration.

Similar to the above port configuration you can expand and scale your app to manage API and database endpoints based on runtime environments.

A example config file in production may look like this:

{
"platform": "mobile_web",
"robots_file": "robots_deny_all.txt",
"paas_redirect_url": "/order/confirmation",
"server": {
"port": "3000"
},
"render": {
"isMinifiedHtml": false
},
"static_assets": {
"host": "/static"
},
"newrelic": {
"agent_enabled": true
},
"ui_api": {
"hostname": "http://stencil.staging.xxxxx.com",
"endpoints": {
"home": "/home",
"suggestions": "/search/suggestions",
"search": "/search",
...

— jinmatt

ex-CTO at Mindhelix, Inc. Love Node.js backend horrors! Release Engineering fanatic and Docker enthusiast.