Electrode Confippet
Electrode, an open sourced platform by WalmartLabs, is used for building powerful web and mobile applications. Electrode Confippet is one of the many modules available in Electrode Web and is used for managing configurations for Node.js applications. In this blog post, we will take a deeper look at Electrode Confippet.
Why Confippet
In software engineering, when we are developing web applications or services using SOA or microservices architecture, we will have properties of key/value pairs which change quite frequently. Few examples of these changing properties can be URL:port
, connection timeout
, response timeout
for a particular service we depend on, DB connection string
, various framework settings
etc., The benefits of using a configuration file over hard coding it directly in the application code is, we have a centralized repository in terms of configuration files which hosts all these properties. Changing a value in these properties files is many folds faster than searching across the entire code base to figure out where to make the change. It provides a uniform interface for reading configuration properties across the entire application. We can visually think of confippet as a container providing two operations:
load — Loads a bunch of configuration files where loading order of files is defined by its order value, with values being merged and conflicting values for a given key
gets overridden by files having a greater order value
.
get(key) — Retrieves an Optional value, for a given key.
Hit the ground running
Confippet composes files in the config/
directory following the convention of node-config files and its usage is very simple.
- Go to a folder and do
npm install electrode-confippet --save
- Create a folder called
config
and add3
files:default.json
,development.json
,production.json
.
default.json
{
"settings": {
"db": {
"host": "localhost",
"port": 5432,
"database": "clients"
}
}
}
development.json
{
"settings": {
"db": {
"host": "development-db-server"
}
}
}
production.json
{
"settings": {
"db": {
"host": "prod-db-server"
}
}
}
3. To use electrode-confippet
, we need to do,
run.js
const config = require("electrode-confippet").config;
const db = config.$("settings.db");
4. We can run in 2 modes:
node run.js
— This will loaddefault.json
first and thendevelopment.json
which will make sure, values present for the same key indevelopment.json
will override the value coming fromdefault.json
. Load order of confippet :default.json < development.json
. The value ofsettings.db
will bedevelopment-db-server
.NODE_ENV=production node run.js
— This will loaddefault.json
first and thenproduction.json
which will make sure, values present for the same key inproduction.json
will override the value coming fromdefault.json
. Load order of confippet :default.json < production.json
. The value ofsettings.db
will beprod-db-server
.
Especially, confippet loads the files depending on order
value with values being merged, and conflicting values for a given key
gets overridden by files having a greater priority
. Can we try to explain this behavior of confippet using concepts found in other areas in programming ?
- Inheritance and Overriding — In traditional Object Oriented Programming, inheritance enables new objects to take on existing properties. In the above example,
development.json
andproduction.json
are able to reuse key/value pairs fromdefault.json
. Again in Object Oriented Programming, overriding is a feature that allows a subclass or child class to provide a specific implementation of a method that is already provided by one of its super-classes or parent classes. In the above example, value forhost
indefault.json
is overridden bydevelopment.json
andproduction.json
. - Ordering of events — Ordering of one event happening before another happens in many other areas in programming. For example, database replication on the slaves have to happen in the same order as they got applied in master to guarantee
ACID
. Ordering of writes happened in theredo
log is captured in themaster
and exactly replayed in the same order in the slaves. In the above example of confippet, ordering of file loading is done based on theirorder
value using which reusing and overriding of key/value pairs is achieved.
Accommodate diverse perspectives
As mentioned earlier, confippet
automatically loads default.json
, development.json
or production.json
depending on NODE_ENV
environment variable under config
folder following the convention of node-config files. But different applications would like to load more (or) less files with different file names in a different set of order
. WalmartLabs uses OneOps to deploy our applications in our cloud environment. Every application have different set of cloud environments like oneops-development
, oneops-staging
, oneops-production
etc., and they would like to have different configuration files loaded depending on which cloud environment their application is running. Confippet is extremely configurable and can be customized for a variety of clients needs. To achieve this, we need to extend providers
by adding OneOps profile as shown below and passing the additional config to Confippet.presetConfig.autoLoad
api.
Being able to extend confippet depending on diverse client needs is one of the shining features which makes it extremely powerful over node-config. Popular libraries and framework generally accommodate diverse perspectives. For example, Spring Framework embraces flexibility and is not opinionated about how things should be done. It supports a wide range of application needs with different perspectives. We can read more about Spring Framework design philosophy here.
We can do a lot more customization using the composeConfig
feature of Confippet. For more information please read here
Electrode Server + Confippet — A powerful combination
Electrode Server provides a standardized node web server that can be used to serve your web application using Hapi.js atop Node.js. Electrode Server uses Electrode confippet to bootstrap the web application. In the configuration files, electrode server standardizes the way we configure log settings, client configuration to talk to downstream services, to register UI configuration which we can be accessed by front end code, various plugins to bootstrap index page, collect metrics etc., A gist of how the configuration looks like is shown below. In our production ready applications, these configuration files are in the order of 100 to 300+ lines depending on the application.
Economies of scale
In WalmartLabs, Electrode is used by the entire company to power front end web applications. As mentioned above, Electrode Server along with Electrode confippet standardizes the bootstrapping of web applications. Likewise, Electrode has a plethora of modules which helps developers building web applications in a variety of ways. So a developer using Electrode framework in one team can seamlessly work in a completely different electrode web application powering a different page in the website and the ramp up time is very minimal. The engineer is already familiar with how electrode modules work together, how electrode artifact build is generated, process to get electrode build deployed to production using OneOps and also how electrode applications are monitored in production. The only missing piece is the domain knowledge of the new application. In essence, Electrode helps in standardizing how web applications are built, deployed and monitored in production which is a powerful model providing huge economies of scale and helps organization innovate much faster. Electrode becomes part of a shared vocabulary, making even more teams using it to build newer web applications, creating a strong feedback loop between applications teams and electrode platform team to continuously improve the Electrode platform and eventually becoming a Snowball effect. Write once, run anywhere was a slogan created by Sun Microsystems to illustrate the cross platform benefits of the Java language. In a similar vein, Electrode epitomizes the fact that a powerful platform which enables developers build, deploy and monitor web applications in a standardized way provides huge economies of scale and truly help run organizations very efficiently.
Conclusion
Electrode confippet which is used for managing configurations is a flexible, extensible general purpose library which can be used in a myriad of ways to build production ready applications. We showed how Electrode server uses Electrode confippet to bootstrap the web application in a consistent way across the organization. I sincerely hope that this blog post was able to pique your interest to start looking into Confippet module, which eventually pushes you to look deeper into Electrode ecosystem and hopefully use Electrode in your company to build powerful web applications as we do in Walmart Labs.
Thanks to Madhav Deverkonda and Alex Grigoryan for helping me review and publish this blog post.