Ops Scripting w. BASH: Frequency

Tracking Frequency in BASH (Bourne Again Shell): Part I

Operations oriented roles, almost always require skills in automation and scripting with shell programming as apart of that requirement. These days with the popularity and installed base of Linux, Bash has become ubiquitous.

Bash combined with command line tools, is regulated for small automation chores, data structures are supports strings, integers, and arrays. However, starting with Bash 4, you can use more advanced data structures like associative arrays, otherwise known as hashes, to create data structures that are used for things like tracking frequency.

This problem and later solutions, show how to use a BASH associative array to track frequency, and along the same show some cool tricks that you can use with Bash.

But first you need to make sure Bash 4 or higher is available on your system…

Getting Bash 4+

If you have a recent Linux distro, such as Debian, Ubuntu, CentOS, Bash 4 or greater is already installed.

macOS

If you are on macOS (aka Mac OS X), you will have to install this, such as first using Homebrew, installing bash, then configuring it to be the default shell:

brew install bash
sudo bash -c 'echo /usr/local/bin/bash >> /etc/shells'
chsh -s /usr/local/bin/bash

Windows

On Windows 10, we can use MSYS2 (Minimal System 2), which you can install manually from the website, or install a package manager like Chocolatey and then install it from command shell or PowerShell in Administrative mode:

choco install msys2

After this, you can bring up a console environment with Bash 4 or higher running.

The Problem

The primary goal for this exercise is to be able to process text files that have rows and columns, such as comma or colon separated fields. You’ll extract the data and store it in a data structure than can be used later to generate a report.

In this problem, we’ll essentially print out a formatted summary showing the shell and number of users that use that shell.

You’ll be supplied a local copy of passwd file as data input, which you must open, cycle through each line, and then save the results into an associative array called COUNTS. With this COUNTS, you’ll then print out a report.

The Data

Here’s the passwd file you can use, save this locally in the same directory with your script.

The Output

When generating a report, the output should look like this:

Shell Summary Report:
==================================================
Shell # of Users
----------------- ------------
/bin/bash 3 users
/bin/false 7 users
/bin/sync 1 users
/usr/sbin/nologin 17 users

The Code

Here is sample code to get you started. It also contains the code for creating the report, using some Bash tricks (see below for code walk through topics)

Code Notes

These are some of the techniques used in the code above, presented here in case you are not familiar with these:

Formatted Output

We printf command, which behaves like the C-Language counterpart:

printf FORMATTED_STRING VAR1 VAR2 VAR3
printf "%s %s %s\n" VAR1 VAR2 VAR3

Repetition

You can use repetition to print out, a line for example, using this technique:

printf '=%.0s' {1..10}

This will generate ten equal symbols. This itself can be wrapped into a sub-shell $() or `` it is needs to be concatenated with another string.

Enumerating Keys

The keys and values of an associative array in Bash can be enumerated with this notation:

KEYS   = "${!ASSOC_ARRAY[@]}"
VALUES = "${ASSOC_ARRAY[@]}"

If you put these into a subshell, you then sort the keys.

SORTED_KEYS = $(echo ${![ASSOC_ARRAY[@]} | sort)

Referencing a Value

You can extract a value given a key like this:

VALUE = ${ASSOC_ARRAY[$KEY]}

The Conclusion

For now, I’ll just present the problem, and next article I will present some solutions. With the sample code, you can get the following takeaways for Bash

  • Creating an Associative Array with declare -A
  • Enumerating Keys and Values from an associative array
  • Referencing (looking up) an item from the associative array
  • Formatted output with printf
  • Repetition using printf "%.0s=" {1..10} trick