Chef Automate with Policyfiles: Uncharted Territory

I’ve been working with Policyfiles in Chef for over a year now and using them in production for about the same amount of time. Deciding to go with Policyfiles versus the traditional environment/role workflow with Chef was really a decision that was made because of the ability to have an immutable Chef artifact for deployment. If you are unfamiliar with them, check out Michael Hedgepeth’s great article at http://hedge-ops.com/policyfiles/.

The Problem

For our cookbook deployment, we’ve been using Teamcity because that was already in use. If you’ve ever used it, Teamcity is not really geared towards a good Chef workflow. I had been wanting to try out Chef Automate for a while and with the Opsworks integration, this made things much easier. I began exploring the Automate interface and pretty soon I wanted to try out the Workflow feature. The only issue was that everything in the documentation and the public delivery-truck cookbook was geared towards using traditional Chef environments and roles along with Berkshelf. At this time, there is nothing public from Chef about how to use Policyfiles with Chef Automate.

Finding A Path

I started out by trying to modify the delivery-truck cookbook code to use Policyfiles. This was a huge mistake as I didn’t actually have any real experience with Chef Automate, save for the tutorial on the LearnChef site. After that realization, I converted my cookbook back to Berkshelf and created temporary Chef environments for testing. This way, I was able to experiment with how the generated build cookbook pulls in the delivery-truck recipes and also the methods from delivery-sugar. It took a lot of reading through both repositories before I was able to get the concept that this really is just using Chef code, and specifically a decent amount of execute blocks and libraries, to build and deploy the cookbook. It all seems like magic at first but there is a lot of documentation in the code to easily see what’s happening.

After I was able to better understand how all of the cookbooks worked with Chef Automate as one process, I was then able ready to tackle the Policyfile problem again.

Forging Ahead

After about a week to clear my head, I went back to addressing the using Chef Automate with Policyfiles. The first thing I did was change back to using Policyfiles and removed the Berkshelf integration. I’ve done this process a couple times now and it’s pretty trivial to go between the two. The bigger issue was how to change to using policy_groups and policy_names instead of environments and roles. For my workflow, I just modified my nodes using knife to have a policy_group with the same name as the Chef environment and the policy_name was the name of my policy. If you have those filled in on the node’s client.rb file then it should automagically run in Policyfile mode. Here’s what my “supermarket” node looks like that has the “supermarket” policy_name in the “delivered” policy_group

{
"name": "supermarket",
"chef_environment": "delivered",
"normal": {
"tags": [
]
},
"policy_name": "supermarket",
"policy_group": "delivered",
"run_list": [
]
}

After setting up the nodes, the next step was to modify my build cookbook. After a bit of trial and error, the changes that I needed to make were really just to the deploy.rb and provision.rb recipes. The rest of the recipes should pretty much just work as most of them either do nothing or already handle the Policyfiles. For example, the unit.rb recipe run Chefspec and that is configured in the spec_helper.rb file to use either Berkshelf or Policyfiles.

For the deploy.rb file, all I did was copy the code from delivery-truck and change the search to look for the policy_group instead of the chef_environment

search_terms = []
get_all_project_cookbooks.each do |cookbook|
search_terms << "recipes:#{cookbook.name}*"
end
unless search_terms.empty?
search_query = "(#{search_terms.join(' OR ')}) " \
"AND policy_group:#{delivery_environment} " \
"AND #{deployment_search_query}"
  log "Search criteria used to deploy: '#{search_query}'"
  my_nodes = delivery_chef_server_search(:node, search_query)
my_nodes.map!(&:name)
  delivery_push_job "deploy_#{node['delivery']['change']['project']}" do
command 'chef-client'
nodes my_nodes
end
end

The provision.rb file was also pretty easy as all I need that one to do is push to the policy_group. Since my groups were named the same as the Workflow environments, everything just works

execute "push_policy_to_#{delivery_environment}" do
command "chef install; chef push #{delivery_environment}"
cwd workflow_workspace_repo
end

Honestly, these changes could probably be incorporated into delivery-truck, along with some conditional code, fairly easily.

Conclusion

I did not worry about the publish part of Workflow as I was having issues with getting the Chef Vault items configured properly to push to the Git repository. I’m also still testing so I’m not pushing to my internal supermarket but looking at the delivery-truck code, I think this should all work without modification.

That’s pretty much all there is to using Policyfiles with Chef Automate. This was really just the very basics as a proof of concept and it works for me. I’m very excited to see the integrations in the Visibility section of Automate but I don’t have that in Chef Management now so it’s not a huge issue for me. I hope this helps others to adopt the Policyfile workflow a bit sooner as I really enjoy it and I think it works very well.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.