File flags in Ansible, two implementations

At least once and at most once

(this post consolidates my previous research in this topic. I already have presented some of it earlier).

There is a specific pattern for Ansible: If there are too many complicated notifications coming from different roles, use file flags.

Those flags consist of two parts: ‘file flag handlers’:

- name: one of the handlers
  file: path={{some_restart_flag}} state=touch

and flag processing code.

The key advantage for flag processing is that it provides some nice idempotent property: if playbook fails before performing actual restart (or other action), the file flag persists, and will be processed at next run. Moreover, if flag is placed at /var/run directory, it will be purged at next reboot (if handler is just service restart, that’s what we want, if handler is more complicated, flag may be stored in a more persistent location).

As in all cluster systems, there are two ways to handle notifications: at least once and at most once. If everything is OK, both methods do the same. But if there are errors, there is a huge difference at the next launch (so calling ‘retry policy’):

• try again (at least once)
• don’t try again (at most once)

‘At most once’ implementation

- name: Process restart flag
  file: path={{some_restart_flag}} status=absent
  notify: actual handler

It does two things at once:

1. remove restart flag
2. send notification to real handler.

Moreover, if handler failed (f.e. service not found, wrong service name, no ‘become’, etc), it will not try again, therefore, ‘at most once’.

‘At least once’ implementation

Some operations are idempotent and need to be performed, and there is no penalty of trying again. For example, if we need to ‘make this host master’ in some cluster configuration, we are completely fine to make it master twice, or even few times. But we definitively want to do it master at least once.

Implementation (I’ll continue to name flag ‘restart’ for a consistency sake):

- name: Check restart flag
  stat: path={{some_restart_flag}}
  register: restart_status
- name: Handling restart flag
  block:
   - name: Restarting {{service}}
     service name={{service}} state=restarted
   - file: path={{some_restart_flag}} state=absent
  when: restart_status.stat.exists

As you can see ‘at most once’ implementation is more concise, while ‘at least once’ is more robust.