Using Macros to Handle Environment Variables in Elixir

Here on my team at Manheim, we work in three or four, or maybe five, languages. We use Ruby and Elixir for building apis, Groovy for automating Jenkins, Bash for automating anything that isn’t automated in Ruby, Elixir, or Groovy, and Java when we have no alternative.

And of course we configure everything with environment variables, which means we like using features of Sinatra and Rails that allow you to define settings, like the rails-config gem.

We missed this feature when we moved to Elixir. In Elixir, you usually run all your configs through the config/ directory. This is fine, but the code Application.get_env(<lots of arguments>) is a little tiring versus Settings.name. It is true that you can define these things at the top of the file with ‘@’ arguments, but still. Not the nicest API.

So we wrote a convenience package called env_helper to give us a slightly nicer API for environment and application variables. It relies on Elixir metaprogramming to work, and we basically created it through trial and error. On the other hand, it’s about twenty lines of code total, so it seems ripe for line-by-line analysis.

Essentially, the goal was to create a way to easily write functions in a module that returned either an environment variable if it was set, or a default value if not. Obviously, this can be done by writing out the function, but we wanted to have a simple API, preferably something automatic that read from a yaml file. We didn’t get that, but we do have a simple API that allows you to define two types of variable:

system_env(:port, 12345)
app_env(:port, [:app_name, :port], 12345]

Either of those will create a function called “port” that returns either the environment variable PORT (in the case of system_env) or the application variable config :app_name, port: in the case of (app_env). An incremental improvement, but an improvement.

So, how does this work? We define two macros:

defmacro system_env(name, alt) do
  env_name = Atom.to_string(name) |> String.upcase    
    quote do      
      def unquote(name)() do
        System.get_env(unquote(env_name)) || unquote(alt)
      end
    end
end
defmacro app_env(name, [appname, key], alt) do
  quote do
    def unquote(name)() do
      Application.get_env(unquote(appname), unquote(key)) || unquote alt
    end
  end
end

So how does this work? If you aren’t familiar with Elixir or Lisp macros, this might appear somewhat opaque. The first line of app_env:

defmacro app_env(name, alt) do

Simply declares that we’re going to define a macro called system_env that takes two arguments, name, and alt. No problem.

The next line:

quote do

Will return the Abstract Sytax Tree (AST) of the next code block. The AST is the form the code is in when it’s handed to the compiler. Here’s an example in iex:

iex(1)> quote do: 1 + 1
{:+, [context: Elixir, import: Kernel], [1, 1]}

That’s pretty easy to interpret, the function being called is :+, the context is Elixir Kernel, and the arguments are 1 and 1.

Our case is a little more complex, so let’s look at the next line:

def unquote(name)() do

So we want an underlying representation of a function definition. But what does that unquote do? Let’s look at iex again:

iex(4)> a = 3
3
iex(5)> quote do: 1 + a
{:+, [context: Elixir, import: Kernel], [1, {:a, [], Elixir}]}
iex(6)> quote do: 1 + unquote(a)
{:+, [context: Elixir, import: Kernel], [1, 3]}

In the first case, where we don’t unquote a, we get the literal :a as an argument. In the second case, when we unquote a, the variable a is evaluated and we get its value, 3.

So in our example, the AST that is built will use the value of the variable name, not the literal string name, when building this function. The next line:

Application.get_env(unquote(appname), unquote(key)) || unquote alt

Is going to return the ast of the Application.get_env function, using the variables appname and key as the arguments. If this returns nil, it will return the value of the variable alt. The AST for a similar statement looks like this:

iex(7)> app_name = :brick
:brick
iex(8)> key = :house
:house
iex(10)> alt = "foundation"
"foundation"
iex(11)> quote do: Application.get_env(unquote(app_name), unquote(key)) || unquote(alt)
{:||, [context: Elixir, import: Kernel],
[{{:., [], [{:__aliases__, [alias: false], [:Application]}, :get_env]}, [],[:brick, :house]}, "foundation"]}

So there’s a tree with a root of :||, in the Elixir Kernel context, which takes two arguments. The first argument is a tree representing the call to Application.get_env, the second argument is the literal string “foundation”.

So, when we call the macro app_env with arguments, the AST to define a new function will be created and compiled, resulting in a new function built using the variables we passed.

So the API looks like this:

defmodule Settings
  import EnvHelper #this is the module with the macros defined
  system_env(:base_url, "localhost")
end

Which results in a new function Settings.base_url, which returns either the environment variable BASE_URL or “localhost:9000”, depending on whether the environment variable was set at compile time. And of course I can call both system_env and app_env as many times as I need, to define all the defaults for system and application environment values that I need.

At some point I wanted to define these variables in a YAML file, and I think from this example we can see how this would be done, by loading the file, parsing it into an array of arguments, and then iterating through that array, calling the macro for each new set of arguments. Adding that and a few extra pieces of metaprogramming and leveraging OTP, and we could build an app that looked for the YAML file and defined functions if it found the file.