Code-First Configuration approach for Python

GitHub repo: https://github.com/davidohana/kofiko-python
PyPi package: https://pypi.org/project/kofiko

In the Code-First approach, you first define your data-model in plain code. You can start working with that model immediately, and only later you worry about schema definitions, bindings, and other necessities. The mapping between the domain model and external entities like database tables/fields usually relies on conventions.

What I like about this approach is that it let you focus on the most important things first, and not less important, you have all the convenience of a modern IDE when defining your model — refactorings, code completion, type checks, etc..

After my last attempt in creating a configuration library for Python, I was still not satisfied and looked for a way to implement the code-first approach for configuration. The outcome is a Python package named kofiko — “Kode First Konfiguration” (which is also a funny ape from an old Israeli children book series). I am pretty satisfied with the outcome, which is described next.

First, you define the desired configuration settings in plain Python code. Configuration entries are defined as static attributes of a class, with a default value which also defines the type of each entry. You may define many configuration classes — each class serves as a different config section. It can be located anywhere in your code.

In the example above, for example, the log bootstrapping code that uses the config class above is not interested in any configuration options other than log-related, so its reasonable to define a dedicated config class for log and locate it side-by-side with the bootstrapping code.

The @config_section decorator above is actually registering this class with Kofiko. Once kofiko.configure() function is called, it will look-up for configuration overrides in various sources and set the value of relevant attributes of the class accordingly.

Currently, Kofiko supports the following configuration sources:
(1) Customization functions
(2) .INIfiles
(3) Env. vars

Customization functions allow you to stay in the code-first approach, by defining simple Python functions that override selective variables in the default configuration classes. This is useful for example in cases where you have multiple deployments. You can create a customization class for each deployment.

You have to register the customization function with Kofiko using the @config_custom decorator, the same way we did with the configuration class.

Kofiko also supports the familiar .INI format. You can specify one ore more in filenames to search for overrides. Kofiko maps the config class name to an ini section and the attribute name to an ini option:

Note that you can even omit the Config keyword from the section name.

The last supported override source is environment variables. Kofiko will look-up for env keys that matches the following convention app-prefix_section_option, and override config attributes accordingly. For example, we can run our app like this:

log_file_out_folder=../log-staging elastic_env_name=staging mirror_batch_hours=1 python my_app.py

In this case, we didn’t use any app-specific lookup prefix, but we can always opt for use such, in order to prevent collisions with other env vars.

Also, note that the lookups in ini and env-vars are case-insensitive by default. And if you don't like my default conventions for override lookups, Kofiko also allows you to customize those with your own.

One of the nicest things about Kofiko is that you don't have to do type-conversion any more. Kofiko will use the default value for each config attribute as a type-hint and will try to convert the text value read from untyped override sources (ini and env) to the same type. It will fallback to string only when unable to convert.
In addition to the basics (str, int, float & bool ), I added support for parsing list (comma delimited by default) and dict in the format key1:val1,key2:val2. The type conversion for list values and dict keys and values is done using the first element in the default value for the relevant config attribute (if exist).

database_endpoints=host1,host2 database_quotas=logs:300,alerts:50 python my_db_client.py

Bootstrapping

In the initialization code of your Python app, all you have to do is call the static kofiko.configure() function. You may specify a customization and/or ini files for lookup like this:

overrides = kofiko.configure(
   customization_name="prod", 
   ini_file_names="../cfg/prod.ini")

After this call, attributes values in all config classes are overridden from relevant override sources, and you can use those config classes directly in your code.

The overrides return values holds a dictionary of all values that were changed from their defaults. It might be useful to log or print this.

How to get it

The source code for Kofiko is available on GitHub under the Apache-2.0 license. You can also install it from PyPi:

pip install kofiko

I hope you will like this little configuration monkey, and please comment and tell me what you thought about it.