Electrode Confippet

Electrode, an open sourced platform by WalmartLabs, is used for building powerful web and mobile applications. Electrode Confippet is one of the many modules available in Electrode Web and is used for managing configurations for Node.js applications. In this blog post, we will take a deeper look at Electrode Confippet.

Why Confippet

In software engineering, when we are developing web applications or services using SOA or microservices architecture, we will have properties of key/value pairs which change quite frequently. Few examples of these changing properties can be URL:port, connection timeout, response timeout for a particular service we depend on, DB connection string , various framework settings etc., The benefits of using a configuration file over hard coding it directly in the application code is, we have a centralized repository in terms of configuration files which hosts all these properties. Changing a value in these properties files is many folds faster than searching across the entire code base to figure out where to make the change. It provides a uniform interface for reading configuration properties across the entire application. We can visually think of confippet as a container providing two operations:

load — Loads a bunch of configuration files where loading order of files is defined by its order value, with values being merged and conflicting values for a given key gets overridden by files having a greater order value .

get(key) — Retrieves an Optional value, for a given key.

Hit the ground running

Confippet composes files in the config/ directory following the convention of node-config files and its usage is very simple.

1. Go to a folder and do npm install electrode-confippet --save
2. Create a folder called configand add 3files: default.json , development.json , production.json .

default.json

{
  "settings": {
    "db": {
      "host": "localhost",
      "port": 5432,
      "database": "clients"
    }
  }
}

development.json

{
  "settings": {
    "db": {
      "host": "development-db-server"
    }
  }
}

production.json

{
  "settings": {
    "db": {
      "host": "prod-db-server"
    }
  }
}

3. To use electrode-confippet, we need to do,

run.js

const config = require("electrode-confippet").config;
const db = config.$("settings.db");

4. We can run in 2 modes:

• node run.js — This will load default.json first and then development.json which will make sure, values present for the same key in development.json will override the value coming from default.json. Load order of confippet : default.json < development.json . The value of settings.db will be development-db-server .
• NODE_ENV=production node run.js — This will load default.json first and then production.json which will make sure, values present for the same key in production.json will override the value coming from default.json. Load order of confippet : default.json < production.json. The value of settings.db will be prod-db-server .

Especially, confippet loads the files depending on order value with values being merged, and conflicting values for a given key gets overridden by files having a greater priority. Can we try to explain this behavior of confippet using concepts found in other areas in programming ?

• Inheritance and Overriding — In traditional Object Oriented Programming, inheritance enables new objects to take on existing properties. In the above example, development.json and production.json are able to reuse key/value pairs from default.json . Again in Object Oriented Programming, overriding is a feature that allows a subclass or child class to provide a specific implementation of a method that is already provided by one of its super-classes or parent classes. In the above example, value for host in default.json is overridden by development.json and production.json .
• Ordering of events — Ordering of one event happening before another happens in many other areas in programming. For example, database replication on the slaves have to happen in the same order as they got applied in master to guarantee ACID . Ordering of writes happened in the redo log is captured in the master and exactly replayed in the same order in the slaves. In the above example of confippet, ordering of file loading is done based on their order value using which reusing and overriding of key/value pairs is achieved.

Accommodate diverse perspectives

As mentioned earlier, confippet automatically loads default.json , development.json or production.json depending on NODE_ENV environment variable under config folder following the convention of node-config files. But different applications would like to load more (or) less files with different file names in a different set of order . WalmartLabs uses OneOps to deploy our applications in our cloud environment. Every application have different set of cloud environments like oneops-development , oneops-staging , oneops-production etc., and they would like to have different configuration files loaded depending on which cloud environment their application is running. Confippet is extremely configurable and can be customized for a variety of clients needs. To achieve this, we need to extend providers by adding OneOps profile as shown below and passing the additional config to Confippet.presetConfig.autoLoad api.

Being able to extend confippet depending on diverse client needs is one of the shining features which makes it extremely powerful over node-config. Popular libraries and framework generally accommodate diverse perspectives. For example, Spring Framework embraces flexibility and is not opinionated about how things should be done. It supports a wide range of application needs with different perspectives. We can read more about Spring Framework design philosophy here.

We can do a lot more customization using the composeConfig feature of Confippet. For more information please read here

Electrode Server + Confippet — A powerful combination

Electrode Server provides a standardized node web server that can be used to serve your web application using Hapi.js atop Node.js. Electrode Server uses Electrode confippet to bootstrap the web application. In the configuration files, electrode server standardizes the way we configure log settings, client configuration to talk to downstream services, to register UI configuration which we can be accessed by front end code, various plugins to bootstrap index page, collect metrics etc., A gist of how the configuration looks like is shown below. In our production ready applications, these configuration files are in the order of 100 to 300+ lines depending on the application.

Economies of scale

In WalmartLabs, Electrode is used by the entire company to power front end web applications. As mentioned above, Electrode Server along with Electrode confippet standardizes the bootstrapping of web applications. Likewise, Electrode has a plethora of modules which helps developers building web applications in a variety of ways. So a developer using Electrode framework in one team can seamlessly work in a completely different electrode web application powering a different page in the website and the ramp up time is very minimal. The engineer is already familiar with how electrode modules work together, how electrode artifact build is generated, process to get electrode build deployed to production using OneOps and also how electrode applications are monitored in production. The only missing piece is the domain knowledge of the new application. In essence, Electrode helps in standardizing how web applications are built, deployed and monitored in production which is a powerful model providing huge economies of scale and helps organization innovate much faster. Electrode becomes part of a shared vocabulary, making even more teams using it to build newer web applications, creating a strong feedback loop between applications teams and electrode platform team to continuously improve the Electrode platform and eventually becoming a Snowball effect. Write once, run anywhere was a slogan created by Sun Microsystems to illustrate the cross platform benefits of the Java language. In a similar vein, Electrode epitomizes the fact that a powerful platform which enables developers build, deploy and monitor web applications in a standardized way provides huge economies of scale and truly help run organizations very efficiently.

Conclusion

Electrode confippet which is used for managing configurations is a flexible, extensible general purpose library which can be used in a myriad of ways to build production ready applications. We showed how Electrode server uses Electrode confippet to bootstrap the web application in a consistent way across the organization. I sincerely hope that this blog post was able to pique your interest to start looking into Confippet module, which eventually pushes you to look deeper into Electrode ecosystem and hopefully use Electrode in your company to build powerful web applications as we do in Walmart Labs.

Thanks to Madhav Deverkonda and Alex Grigoryan for helping me review and publish this blog post.