Configuration Management with Salt Formulas and Pillar
I’ve just started implementing a comprehensive configuration management solution at my workplace using Salt. The use-case is not simplistic: this is the kind of environment I face:
- Ubuntu servers
- CentOS servers
- Hypervisors and VM’s (Linux and Windows)
- User Laptops (Windows, Ubuntu and MacOS)
- Multiple locations around the world, resulting in slightly differing core config like NTP and Directory servers, routes and firewall rules.
- Grouping servers by teams
- Managing special cases
The core idea is that there’s a small subset of things which will be common across all systems (like an admin user), and the rest of the stuff will vary based on the category of machine.
I come from the Puppet world, and this looks like a roles-and-profiles problem at first glance. I just need to find the equivalent Salt paradigm, which can achieve the same result with grains, roles and the Pillar. The Puppet world also provides something awesome: modules, maintained by the community, which are comprehensive pieces of reusable code that cover most use cases you can think of. These modules provide the best way of cleanly separating out config from code — you don’t change a single line of module code, you just interact with it by passing parameters to its classes. These parameters are typically passed as key-value pairs written in YAML and interpreted by Hiera, the Puppet tool that understands their nested hierarchies.
Salt has it’s equivalent paradigms but a lot of the SaltStack documentation around seemed geared more towards using it as a one-shot remote execution tool, and though there is plenty of documentation around states, it wasn’t trivial to build up a state tree. Hence this post. Of course, I’m quite new to Salt so it’s quite possible there are better ways of doing things. If you happen to know any, feel free to comment.
I don’t believe in over-simplified contrived examples which make up the bulk of documentation, so let us take a simple but real-world use case.
Let’s say you want to configure a single non-root user with sudo privileges across all your Linux servers. Let’s break this task down:
- On the master, create the following directory and file structure:
/srv/
├── formulas
├── pillar
| └── top.sls
└── salt
└── top.sls- Generate a suitable hashed password (I used “password” as the password for an example — do not ever do that on production systems):
# openssl passwd -1
Password:
Verifying - Password:
$1$4V3gQKAM$DzpMP5K42pGBPsNeUl8QW.- Download the users formula from the Salt community’s collection of formula repos. These formulas cover the overwhelming majority of use cases, are battle-tested and maintained by dedicated teams of contributors who write documentation and fix bugs too. Never reinvent the wheel unless it’s absolutely necessary. Use community formulas as much as possible.
# cd /srv/salt/formulas
# git clone https://github.com/saltstack-formulas/users-formula.git
# mkdir /srv/salt/pillar/users
# cd /srv- Copy the pillar example from the formula to your local pillar:
# cp formulas/users-formula/pillar.example pillar/users/init.sls- Use it as the template to add a user, so the file now looks like this:
users-formula:
lookup: # override the defauls in map.jinja
root_group: roottestuser:
fullname: Test User
password: '$1$4V3gQKAM$DzpMP5K42pGBPsNeUl8QW.'
absent: True
createhome: True
groups:
- admin
- sudo
The password field here should be populated with the hash generated by the openssl command above. Remember to quote the hash with single quotes so as to avoid any weirdness arising out of shell variable interpolation.
- Next, we need to make the formula code available to the salt execution environment by adding to the file_roots variable in the master config:
# vim /etc/salt/masterfile_roots:
base:
- /srv/salt
- /srv/formulas/users-formula
- Every addition of a community formula requires this additional entry in the master conf file followed by a restart of the master:
# systemctl restart salt-master.service- Now for the highstate code (top.sls) for the root and the pillar:
base:
'G@kernel:Linux':
- usersThe “base” corresponds to an environment
- Finally, test out the changes against one minion to make it simpler. The show_highstate function in the state module is Salt’s equivalent of Puppet’s noop — it just tells you what would be done without actually doing it.
# salt '<minion_ID>' state.show_highstate<minion_ID>:
----------
users_/etc/sudoers.d/testuser:
----------
__env__:
base
__sls__:
users
file:
|_
----------
name:
/etc/sudoers.d/testuser
- absent
|_
----------
order:
10005
users_testuser_admin_group:
----------
__env__:
base
__sls__:
users
group:
|_
----------
name:
admin
- present
|_
----------
order:
10000
users_testuser_sudo_group:
----------
__env__:
base
__sls__:
users
group:
|_
----------
name:
sudo
|_
----------
system:
True
- present
|_
----------
order:
10001
users_testuser_user:
----------
__env__:
base
__sls__:
users
file:
|_
----------
name:
/home/testuser
|_
----------
user:
testuser
|_
----------
group:
testuser
|_
----------
mode:
750
|_
----------
makedirs:
True
|_
----------
require:
|_
----------
user:
users_testuser_user
|_
----------
group:
testuser
- directory
|_
----------
order:
10002
group:
|_
----------
name:
testuser
- present
|_
----------
order:
10003
user:
|_
----------
name:
testuser
|_
----------
home:
/home/testuser
|_
----------
shell:
/bin/bash
|_
----------
password:
$1$4V3gQKAM$DzpMP5K42pGBPsNeUl8QW.
|_
----------
gid_from_name:
True
|_
----------
fullname:
Test User
|_
----------
remove_groups:
False
|_
----------
groups:
- testuser
- admin
- sudo
|_
----------
require:
|_
----------
group:
testuser
|_
----------
group:
admin
|_
----------
group:
sudo
- present
|_
----------
order:
10004
- Once you’ve gotten this far and the previous command throws no errors, apply the high state using:
# salt '<minion_ID>' state.highstateand if that runs without errors, you should be able to login to the minion machine with the testuser credentials.
Congratulations! You have just completed a fairly typical configuration management task — push a user out on to a set of machines. And you’ve done this in a scalable way, with community formulas and Pillar.
You can similarly add community formulas for SSH, iptables, docker, nagios and a whole bunch of other things you would typically need on any server.
