envloader: An Open Source Environment Configuration Tool
Originally published at https://curology.com on March 26th, 2021 by Olivia Kim, Sr. SRE.
We are excited to announce that we have released an open source version of our internal tool, envloader, which is now available on packagist and can be installed via Composer! envloader is a PHP tool designed to simplify and standardize the way that application environments are configured. With envloader, you can manage your configurations by loading parameters from AWS Parameter Store and using them to generate .env files for each of your environments.
Why envloader?
At Curology, we maintain several different application environments (development, testing, staging, production, etc.) that require unique .env files corresponding to environment-specific parameters within AWS Parameter Store.
Before centralizing our configurations in AWS, engineers would have to interact with various CI/CD tools and hosting platforms to manage configurations. Updating configurations within multiple platforms was time-consuming and prone to errors, as it was common to forget to update one of the affected tools. Additionally, as our engineering team grew, syncing local .env files across the team became increasingly difficult, as configurations would often become out of date. Engineers would have to explicitly announce their changes as well as manually update their .env files when other members of the team made changes. This process did not scale well and often caused local development environments to break, since we would frequently forget to broadcast configuration updates.
We use envloader to fetch parameters from AWS Parameter store and load them into .env files at deploy time, or before running our application. This provides a simple, secure, and unified method of configuring our app that is also consistent across all of our environments. This standardized workflow minimizes discrepancies between the different stages of our application, while also simplifying local development and setup.
envloader in Action
Configuring envloader
Let’s take a look at how envloader works with a simple example. First, we’ll need to create a configuration file, which will contain all the information that envloader needs to run. By default, envloader will search for a configuration file named envloader.json:
The parameterList contains the parameters envloader should fetch from AWS. The parameters should be specified in the form name:version so that envloader knows which version of the parameter to fetch.
The parameterPrefix is prepended to the parameter names in the parameterList. For example, the first parameter we will fetch is /envloader-test/foo:1.
You can learn more about all the configuration options here.
Generating the .env File
The generate command will generate a .env file at the location specified by the envPath option in your configuration file:
Showing the Generated Values
Once we have generated our .env file, we can use the show command to display the key, value pairs of the secrets we have generated, along with the corresponding parameter names in AWS. By default, the parameter values are obfuscated:
envloader Features
Versioning
envloader enforces explicit versioning to allow for an auditable and reproducible workflow. Entries in the parameterList must be specified in the form: name:version.
At Curology, we check our envloader configuration file into version control so that our application always accesses the correct parameter values based on the code revision that is in use.
When a new developer initially pulls the application code, they simply run the generate command to configure their development environment instead of having to manually set various environment variables.
When configurations need to be created, deleted, or updated, developers can modify them within AWS Parameter Store without affecting any of the running application environments since the code references specific parameters and versions. Once the parameters have been updated in AWS, developers simply edit the parameterList in the envloader configuration file by adding or removing parameters, or updating the versions of the parameters they are modifying. On deploy, the generate command will load the updated list of parameters.
Versioning also allows for quick and easy rollbacks to older parameter versions if needed. In the event that something goes wrong, developers can simply rollback to a previous code revision, which contains the list of corresponding parameters and versions to load for that specific revision.
Overrides
In addition to loading values from Parameter Store, you can also tell envloader to incorporate values from an override .env file. The values in your override file will take precedence over the values in Parameter Store when running the generate command. Overrides can be useful when you want to test changes locally before adding them to AWS.
In your configuration file, specify the path to your override file using the configuration option envOverridePath:
After running the generate command, we can see that the entry OVERRIDDEN="new value" from our .env.override file will override the parameter /envloader-test/overridden=old value. This results in OVERRIDDEN having the value new value instead of the value in AWS, old value.
Additionally, the override value EXTRA is also included in the generated file, despite not overriding a parameter in the parameterList.
The generated .env file will still contain the overridden values from AWS for your reference; however, they will remain commented out.
Substitution
You may also use variable substitutions within your parameters. For substitution to work properly, you must first specify referenced parameters in the parameterList before listing the parameters that reference them. The ordering of the parameterList will be preserved in the generated .env file.
To reference another parameter, use the form: ${PARAMETER_TO_REFERENCE} ( where PARAMETER_TO_REFERENCE is the name of the parameter you are referencing) in capitals, without the version and parameterPrefix.
For example, let’s say we have the following parameters in AWS Parameter store:
In your configuration file, ensure that the referenced parameter (/envloader-test/simple) appears before the referencing parameter (/envloader-test/substitution) in the parameterList:
After generating the .env file, we can see that the variable SUBSTITUTION has successfully referenced the value of SIMPLE (hello):
