Ansible Custom Facts (Part 2)

Creating Custom Ansible Variables

In my previous Ansible Facts (Part 1) post, I gave a quick rundown on accessing the basic Dynamically Discovered System Variables that Ansible gathers during the setup phase if gather_facts: true is enabled in your playbook.

I’ll post an example using Packer to bootstrap the AMI using the amazon-ebs builder along with the Ansible provisioner soon, so we can pull more of these concepts together.

In this post, I’d like to expand a bit more on how you might gather custom facts and make them available to Ansible roles for playbook tasks and templates. After a fair amount of google soul-searching and sifting through the Ansible documentation, I found the reference to the Local Facts (Facts.d) to be accurate, but not very cohesive in pulling all of the concepts together in a bit more depth.

Say I want to grab some dynamic system environment variables that are not discovered dynamically by Ansible’s setup module; for example some AWS EC2 instance metadata like the AMI, VPC or Subnet ID. Or maybe I want to run a script to gather details about an S3 bucket, RDS database, or a DynamoDB table that I want to use in my playbooks to bootstrap my instance.

So in this example, let’s customize our MOTD (Message of the Day) banner to give users a quick overview of EC2 instance metadata, system info and usage stats when they first login. When you are working with a large environment of hundreds, or even thousands, of hosts this data at a glance can be very useful.

Now, there are a few different ways we could populate the MOTD banner dynamically, but that could be another blog post in itself; I’ll be showing you two different ways. Here is an overview of the playbook structure we’ll be using for this example:

Here is our basic base.yml playbook to bootstrap our base instance configuration:

- name: Setup the base AMI
  hosts: all
  become: yes
  become_user: root
  become_method: sudo
  gather_facts: true
  user: ec2-user
  roles:
   - base

Here is the profile.sh.j2 template we will use to create /etc/profile.d/profile.sh. Feel free to borrow and customize to your liking:

#!/bin/bash
# This script will gather dynamic EC2 instance meta-data and export as environment
# variables, export to facts.d directory as custom ansible variables to make them 
# available for ansible configurations as-needed, and will print out the MOTD 
# (Message of the Day) banner with quick system info at a glance that users will 
# see upon login.
# Gather instance meta-data
REGION=$(curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | grep region | awk -F\" '{print $4}')
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
INSTANCE_TYPE=$(curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | grep instanceType | awk -F\" '{print $4}')
ENV=$(aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" "Name=key,Values=env" --region=$REGION --output=text | cut -f5)
AZ=$(curl -s http://169.254.169.254/latest/meta-data/placement//availability-zone/)
AMI=$(curl -s http://169.254.169.254/latest/meta-data/ami-id)
PUBLIC_IP=$(curl -s http://169.254.169.254/latest/meta-data/public-ipv4/)
INTERFACE=$(curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/)
SUBNET_ID=$(curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/${INTERFACE}/subnet-id)
VPC_ID=$(curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/${INTERFACE}/vpc-id)
# Export as environment variables for later reference. Default AWS Regions makes AMI Roles work without needing to specify region.
export AWS_DEFAULT_REGION=$REGION
export INSTANCE_ID=$INSTANCE_ID
export INSTANCE_TYPE=$INSTANCE_TYPE
export ENV=$ENV 
export AZ=$AZ 
export AMI=$AMI
export PUBLIC_IP=$PUBLIC_IP
export SUBNET_ID=$SUBNET_ID
export VPC_ID=$VPC_ID
# Export as custom ansible variables to /etc/ansible/facts.d/aws.fact 
printf "{\"REGION\":\"$REGION\",\
\"INSTANCE_ID\":\"$INSTANCE_ID\",\
\"INSTANCE_TYPE\":\"$INSTANCE_TYPE\",\
\"ENV\":\"$ENV\",\
\"AZ\":\"$AZ\",\
\"AMI\":\"$AMI\",\
\"PUBLIC_IP\":\"$PUBLIC_IP\",\
\"SUBNET_ID\":\"$SUBNET_ID\",\
\"VPC_ID\":\"$VPC_ID\"}" > /etc/ansible/facts.d/aws.fact
# Basic System Info
OS=`head -1 /etc/issue`
HOSTNAME=`uname -n`
ROOT=`df -Ph | grep xvda1 | awk '{print $4}' | tr -d '\n'`
ROOT_TOTAL=`df -Ph | grep xvda1 | awk '{print $2}' | tr -d '\n'`
# System Load
MEM_USED=`free -t -m | grep Total | awk '{print $3" MB";}'`
MEM_TOTAL=`free -t -m | grep "Mem" | awk '{print $2" MB";}'`
SWAP_USED=`free -m | tail -n 1 | awk '{print $3}'`
SWAP_TOTAL=`free -m | tail -n 1 | awk '{print $2}'`
LOAD1=`cat /proc/loadavg | awk {'print $1'}`
LOAD5=`cat /proc/loadavg | awk {'print $2'}`
LOAD15=`cat /proc/loadavg | awk {'print $3'}`
# Print to stdout for MOTD banner
printf "
###############  System Summary  ##############
===============================================
   Operating System: $OS
   Hostname: $HOSTNAME
   Environment: $ENV
   EC2 Instance ID: $INSTANCE_ID
   EC2 Instance Type: $INSTANCE_TYPE
   EC2 Region: $REGION
   EC2 Availability Zone: $AZ
   Public IP: $PUBLIC_IP
   Private IPs: {{ ansible_all_ipv4_addresses | list | join() }}
   VPC ID: $VPC_ID
   Subnet ID: $SUBNET_ID
   AMI: $AMI
===============================================
   ROOT Disk Space..........: $ROOT out of $ROOT_TOTAL remaining
   CPU usage...........: $LOAD1, $LOAD5, $LOAD15 (1, 5, 15 min)
   Memory used.........: $MEM_USED / $MEM_TOTAL
   Swap in use.........: $SWAP_USED MB out of $SWAP_TOTAL MB
===============================================
"
Hopefully the script is pretty self-explanatory along with the comments as you read through it. But I'd like to specifically point a couple of things out that we'll come back to in a moment:

• Note the section that is creating the /etc/ansible/facts.d/aws.fact file. Once you create the facts.d directory, you can either place an executable script that produces INI or JSON output, or you can place a static INI/JSON file there called whatever.fact. In our example we are calling it aws.fact.
• In the last section where we are printing to stdout for the MOTD banner, note that we are calling one of the Ansible dynamically discovered system variables for the Private IPs in Jinja2 format as:
• {{ ansible_all_ipv4_addresses | list | join() }}
• You will see this value readily available in my previous Ansible Facts (Part 1) example. Even though in our example we have only one private IP address, we are sorting the values as a list and joining them because the value type is an array. You can play around with the formatting with multiple IP addresses to display them in a comma-separated fashion, by newline, etc.

Now by placing the above script into our /etc/profile.d/ directory, it will dynamically update the MOTD banner. Each time a user logs on, here is what they will see:
[ec2-user@ip-10-229-20-22 ~]$ sudo -i
###############  System Summary  ##############
===============================================
   Operating System: Amazon Linux AMI release 2017.09
   Hostname: ip-10-0-2-30
   Environment: dev
   EC2 Instance ID: i-0b310c1d236123456
   EC2 Instance Type: t2.small
   EC2 Region: us-east-1
   EC2 Availability Zone: us-east-1b
   Public IP: 34.234.123.45
   Private IPs: 10.0.2.30
   VPC ID: vpc-83b025e8
   Subnet ID: subnet-26b1234a
   AMI: ami-1d080816
===============================================
   ROOT Disk Space..........: 15G out of 20G remaining
   CPU usage...........: 0.00, 0.00, 0.00 (1, 5, 15 min)
   Memory used.........: 390 MB / 2001 MB
   Swap in use.........: 0 MB out of 4095 MB
===============================================
And here are the profile.yml tasks for this play in our playbook:
- name: Set profile.d script to set AWS Region for everyone.  Makes AWS AMI Roles work.
  template: src=templates/profile.sh.j2 dest=/etc/profile.d/profile.sh mode=0755
- name: Create ansible facts.d directory for custom variables 
  file: path=/etc/ansible/facts.d state=directory
- name: Touch aws.fact file with proper permissions 
  file: path=/etc/ansible/facts.d/aws.fact state=touch mode="a=rw"
- name: Run profile.sh to seed custom MOTD
  shell: /etc/profile.d/profile.sh
- name: Re-run setup to gather ansible_local custom facts from facts.d directory 
  setup: filter=ansible_local
- debug: msg="Ansible variables ---- {{ ansible_local.aws }}"
Notice a couple of things here:

• We are re-running the Ansible setup module to gather our custom facts that we just populated in our facts.d/ directory. These custom facts will be collected as a part of the ansible_local hostvars, so we are using a filter instead of running the full setup module over again.
• Our debug statement is printing the output of these new custom facts so we can validate their existence during the playbook run. We are specifically referencing our aws.fact file by calling {{ ansible_local.aws }} and it will output:
• TASK [base : debug] *************************************************************
ok: [34.229.139.53] => {
"msg": "Ansible variables ---- {u'AMI': u'ami-1d080267', u'PUBLIC_IP': u'34.229.139.53', u'SUBNET_ID': u'subnet-21b3264a', u'REGION': u'us-east-1', u'INSTANCE_ID': u'i-0b310c1d236567032', u'INSTANCE_TYPE': u't2.small', u'ENV': u'dev', u'VPC_ID': u'vpc-83b025e8', u'AZ': u'us-east-1b'}"
}

I promised to show two different ways to populate our MOTD banner. For the sake of this example, let's see how we could use these custom facts now that Ansible has access to them.
We could use an motd.yml play like so:
- name: Get OS distro
  command: "head -1 /etc/issue"
  register: OS_DISTRO
- name: Re-run setup to gather ansible_local custom facts from facts.d directory 
  setup: filter=ansible_local
- debug: msg="Ansible variables ---- {{ ansible_local.aws }}"
- name: Configure motd file 
  template: src=templates/motd.j2 dest=/var/lib/update-motd/motd owner=root group=root mode=0644
Along with the motd.j2 template to be placed at /var/lib/update-motd/motd as shown above, which the system symlinks to /etc/motd:
######  System Summary  ######
   Operating System: {{ OS_DISTRO.stdout | list | join() }}
   Hostname: {{ ansible_hostname }}
   Environment: {{ ansible_local.aws.ENV | default('undefined') }}
   EC2 Instance ID: {{ ansible_local.aws.INSTANCE_ID }}
   EC2 Instance Type: {{ ansible_local.aws.INSTANCE_TYPE }}
   EC2 Region: {{ ansible_local.aws.REGION }}
   EC2 Availability Zone: {{ ansible_local.aws.AZ }}
   Public IP: {{ ansible_local.aws.PUBLIC_IP }}
   Private IPs: {{ ansible_all_ipv4_addresses | list | join() }}
   VPC ID: {{ ansible_local.aws.VPC_ID }}
   Subnet ID: {{ ansible_local.aws.SUBNET_ID }}
   AMI: {{ ansible_local.aws.AMI }}
And just remove the section of our profile.sh script above printing to stdout:
# Print to stdout for MOTD banner
Now we still have our custom environment variables being exported, our custom facts being populated to our facts.d/ directory, but we are just using a second playbook and an motd.j2 template referencing all of our custom facts.
Notice we also generated and registered another custom fact on the fly in our motd.yml play here:
- name: Get OS distro
  command: "head -1 /etc/issue"
  register: OS_DISTRO
And referenced it in our template here:
Operating System: {{ OS_DISTRO.stdout | list | join() }}
And there you have it -- a few different ways to leverage the power of Ansible variables!
To summarize, we touched on:

• Using the

Ansible dynamically discovered variables gathered by the setup module
Local Facts
static
script
JSON or INI
generating and registering a custom fact
And we learned a couple different ways to dynamically update our Amazon Linux MOTD banner.
Please post any comments or questions below, I'd love to hear from you! I hope this post saves a few cycles of legwork out there for someone.