Better Software Through Well-Defined Configuration Context

Giving configuration values a well-defined context will improve documentation, infrastructure, onboarding of new developers, and service/application interoperability (to name a few).

Many configuration libraries are available for basically every language out there. This post is not about those. This post is about setting up the proper context for each configuration value.

Why do we need a configuration context?

Advantages of Context in Configurations

Hierarchical Key/Value Store Configuration Sources

A possible source for application and service configurations is Hierarchical Key/Value stores.

Although this post does not focus on any specific cloud provider, and both Google and Azure offer similar services, consider the SSM Parameter Store service.

Instances of SSM Parameter Store exist at the regional level within AWS. As such, key-names within Parameter Store could clash with key-names from other applications if not correctly scoped. Providing context (a proper hierarchical key name) is a great way to remove potential name clashing.

Environments

Applications and services will run in different environments.

• Development Environments — An application or service may be running in different development environments such as dev, stage, and production.
• Instance Environments — A custom configuration context for one instance of many instances running within a product cluster, for example, debugging one instance and needing to enable in-depth logging.

Often, environment scope is defined using different files (like settings-dev.yaml, or settings-prod.yaml). However, scoping via files can make things like new configuration values, documentation, and sharing similar configuration values difficult. We will touch on this in another post.

Our recommendation is not to rely on files as a scope mechanism for configuration context. Instead, rely on hierarchical key-names.

Configuration Context

Scoping Using Hierarchical Key-Names

Key-names are a great way to scope a configuration value giving a clean, well-documented context.

• The Root (company) — Anything at the “root” of the key-name is available to all company services and applications.
• Platform — Configuration values at this scope will be available to all services and applications for a given product offering.
• Compute — Configuration values at this scope will be available to a specific application, service, micro-service, and lambdas.
• Environment/Instance— Configuration values at this scope are available to a specific application, service, micro-service, lambda, or a particular instance.

NOTE: A single configuration file with multiple applications and services (Compute) defined within it (each application and service probably has its own Github repository) is not recommended. It is also not our intent to have a single configuration file with multiple Platforms defined within it.

Instead, the intent of having Platform and Compute as part of the configuration context is to prevent the clashing of configuration names.

Example Configuration Context

Hierarchical key-value store like SSM Parameter Store:

platform/compute/environment/configuration   value
superCrm/userApi/dev/db/port                 3005
superCrm/userApi/dev/db/host                 localhost
superCrm/userApi/stage/db/port               5432
superCrm/userApi/stage/db/host               db.company.com

Hierarchial file format like JSON:

{
  "superCrm": { // platform
    "userApi": { // compute
      "dev": { // environment
        "db": {
          "port": 3005,
          "host": "localhost"
        }
      },
      "stage": { // environment
        "db": {
          "port": 5432,
          "host": "db.company.com"
        }
      }
    }
  }
}

Hierarchial file format like YAML:

superCrm:
  userApi:
    dev:
      db:
        port: 3005
        host: localhost
    stage:
      db:
        port: 5432
        host: db.company.com

NOTE: The database configuration schema provided in the above examples is for simplicity and not necessarily recommended.

Usage of This Configuration Context

We can use this configuration context in most libraries. Let us try an example using the config configuration library.

We control the context of our service by using environment variables: CONFIG_PLATFORM=superCrm, CONFIG_COMPUTE=userApi, and NODE_ENV=db.

const config = require('config');
const PLATFORM = process.env('CONFIG_PLATFORM');
const COMPUTE = process.env('CONFIG_COMPUTE');
const ENV = process.env('NODE_ENV');
const dbConfig = config.get(`${PLATFORM}.${COMPUTE}.${ENV}.db`);

We can then easily switch which context to pull configuration values from: ${PLATFORM}.${COMPUTE}.${ENV}.db.

Conclusion

Having a well-defined context for configuration values can help with pain points like documentation, infrastructure, onboarding new developers, and service/application interoperability.

However, the recommendations introduce a few pain points: long key-names, a few more environment variables, and additional code to switch contexts (as shown above).

A way to resolve these introduced pain points would be to:

• Support for multiple configuration sources.
• A way to share the same configuration values between different contexts: a _shared reserved key.
• A .get that automatically considers the context:(lib.get(`${PLATFORM}.${COMPUTE}.${ENV}.db`) would be equivalent to lib.get('db'); ).

These features are part of config-core: a feature-rich but straightforward multi-source hierarchical configuration and settings solution.

[1] Photo by Danny Avila on Unsplash.