SaltStack: Extending the Pillar

Writing your own external pillar modules

One of SaltStack’s many strengths is it’s extensibility. It’s possible to write custom states, execution modules, grains, pillars, renderers and returners all in python and all with the power of SaltStack’s internal modules. Let’s take a look at writing a custom module that decorates the pillar with information from an external resource. For this exercise, we’ll create an external pillar that reads data from a Redis database.

Configuration

We’ll start by creating our external pillar module file. In your Salt base or root directory, create a folder called “modules” and a sub-folder called “pillar”:

/srv/salt $ mkdir -p modules/pillar

Next, we’ll create our pillar file:

/srv/salt $ touch modules/pillar/redis_pillar.py

And finally, lets tell the master server where to find our custom modules and which external pillar(s) to load by adding the following to our master configuration:

# path to load custom modules from
extension_modules: /srv/salt/modules
# external pillars to load
ext_pillar:
  — redis_pillar:
      port: 6379
      host: localhost
      password: superSecretTacos

This particular module has a dependency on the Redis python module so we’ll also install that with pip. You may want to use virtualenv in your own environment, but to simplify this example we’ll install the module globally:

/srv/salt $ sudo pip install redis

Writing the Module

Now that we have everything configured we need write the logic for our pillar. For the purpose of this demo we’re going to create a really simple module but keep in mind you have python, the pillar data, __grains__ and __salt__ all at your disposal here:

# -*- coding: utf-8 -*-
"""
Pull pillar data from Redis
"""
import redis
def ext_pillar(minion_id, pillar, **kw):
 data = {}
 # setup redis connection
 host = kw.get(‘host’, ‘localhost’)
 port = kw.get(‘port’, 6379)
 pswd = kw.get(‘password’, None)
 r = redis.StrictRedis(host=host, port=port, password=pswd, db=0)
 data = { 'redis_data': r.hgetall('my_hash_key') }
 return data

There’s a few things happening here. First, you’ll notice that we import the Redis module and define a method called ‘ext_pillar’. This is a special method name that SaltStack will call every time a minion updates it’s pillar data. The ext_pillar method receives the requesting minions id, a dictionary of all the pillar data already compiled for the minion, and the arguments specified in the master configuration (in this case, Redis connection info). The values we return from this method will then be merged back into the main pillar’s data and sent to the minion.

Potential Use Cases

There are all kinds of use cases that can benefit from custom pillar data. Any information that’s defined outside of configuration management like dynamic application parameters, cluster information or realtime infrastructure state can all be represented with an external pillar. And because the module is defined outside of the normal jinja / yaml space, you have the flexibility to pull data from anywhere on the network or the file system. This makes external pillars a valuable extension in the SaltStack ecosystem.

You can read more about the external pillar modules in the SaltStack Docs. Happy Hacking ☺