Scripting tricks: Bash

Photo by Shahadat Rahman on Unsplash

So you want to step up your Bash scripting game?

Let me help you out. I am listing all the useful tricks and code snippets I use frequently in my scripts.

I know I could always google those and get my work done. But wouldn’t it be nice to have everything in one place?

I assume you already have written bash scripts in the past, so I won’t go in too much details about basics.

Let’s get started.

Writing a Function

This is how you can define a function which takes one argument and call it

#!/bin/bash
function usage {
    echo "You just called usage() function";
    echo "And you passed this argument: $1";
}
# I am going to call the usage() function now
usage "Hello World";

This is what it prints

>>./bash_demo
You just called usage() function
And you passed this argument: Hello World

I mostly use that function to print the script usage when user passes help argument or doesn’t pass correct arguments.

Here you learned how to write a function and how to pass arguments to it.

Send an email from command line

If you run long running tests or you just want to be notified when your script finishes, instead of constantly checking your terminal, you can just send yourself an email with some relevant information.

You can so that in a single line with “mailx”.

let’s say you want to send contents of a log file to yourself, you can do it like this

cat my.log | mailx -s "My subject" $USER

If you don’t have a log file, you can simple echo something and pipe it to mailx like above

echo "hello" | mailx -s "World" user@some_domain.com

There are more feature mailx provides, like adding cc, attachment etc.

You can look those up easily.

Arrays (and For loop too)

Initialize an array and loop through it

MODES=("READ" "WRITE" "UPDATE" "KILL" "DANCE");
for MODE in ${MODES[*]}
do
    echo "Current mode= $MODE";
done

It prints

>>./bash_demo
Current mode= READ
Current mode= WRITE
Current mode= UPDATE
Current mode= KILL
Current mode= DANCE

Here you learned how to initialize an array and how to use for loop to loop through it.

Split a string into an array

IFS=', ' read -r -a array <<< "$string"

Check if an element is in an array

This is kind of an advanced usage. But learning it doesn’t hurt.

function containsElement {
    local e match="$1"
    shift
    for e; do [[ "$e" == "$match" ]] && return 1; done
    return 0
}
MODES=("READ" "WRITE" "UPDATE" "KILL" "DANCE");
MY_MODE="DANCE"
containsElement "$MY_MODE" "${MODES[@]}"
echo "$?"

$? stores the return value of the function. You can also assign it to a variable if you want to use it later.

Parsing Command Line Arguments

Step up from the usual $1 checking.

If your script always requires at least one argument, use this

if [ "$1" == "" ]; then
    echo "ERROR: Please pass correct arguments";
    usage;    
    exit 1;
fi

This calls the usage function and prints how to use the script. Then exits with status 1.

To loop through the arguments, use a while loop.

This is how my complete script looks like

#!/bin/bash
mode=""
verbosity=0
function usage {
    echo "Example usage";
    echo "bash_demo -m DANCE -v 0";
}
if [ "$1" == "" ]; then
    echo "ERROR: Please pass correct arguments";
    usage;
    exit 1;
fi
while [ "$1" != "" ]; do
    case $1 in
        -m | -mode )
            shift
            mode="$1"
            ;;
        -v | -verbosity )
            shift
            verbosity="$1"
            ;;
        -h | -help )
            usage
            exit
            ;;
        *)
            usage
            exit 1
            ;;
    esac
    shift
done

At the top, I have some global variables initialized with some values.

Then I have my usage function with an example usage.

I am checking if I have at least one argument with the if statement there.

And finally I am running a while loop through all the arguments and updating my global variables with those values. If user doesn’t pass all the args, they retain their initialized values.

The last case takes care of args which are not listed in the while and prints usage too.

Here you also learned about while loop, case statements, shift and if condition.

Now your scripts won’t look like those written by 12 year olds or 75 year olds!

Time taken by your script

You could always use the time command when running the script. But many times we need to call our script from some other script, so it is good to have this snippet ready.

duration=$SECONDS;
echo "Total Time: $(($duration / 60)) min $(($duration % 60)) sec";

Get date in the format you need and get current work week

TODAY=`date +%Y-%m-%d`   #will get you "2020-01-29"
THISWW=`date +%V`   # will get "05"
LASTWW=`date -d "last mon" +%V`  #will return last work week

Check if a file/directory exists

FILE_PATH="/some/path/to/my/file.txt"
if [ ! -f  $FILE_PATH ]; then
    echo "ERROR: $FILE_PATH File not found!"
    exit 1;
fi
DIR_PATH="/some/path/"
if [ ! -d $DIR_PATH ]; then
    mkdir $DIR_PATH;
    echo "creating directory $DIR_PATH";
fi

Note the spaces between ‘[‘ and ‘]’ in the if statement. And don’t forget the “fi”

Check if your log file had errors

LOG="something.log"
if [ $(grep -c -i "Error" $LOG) -ne 0 ]
  then
    echo "FAILED ";    
    echo " Finding Errors . . .";
    echo "";
    grep -h -i -A 3 error clog; 
    exit 1;
  else
    echo "Success";
  fi

Here, the first grep looks for “error” string inside the log file.

The “-c” returns the count of matches

“-i” makes it case insensitive.

The if condition checks if the number of errors is 0 or not. If not, then we go inside the if statement.

There, we are again greping for error string with different arguments this time.

“-h” suppresses file name

“-A 3" prints 3 lines after the matching line including the matching line.

We can also print 5 lines before the matching line with “-B 5”

Here you just learned something new about grep

Include another script

MY_DIR="$(dirname "$0")"
. $MY_DIR/include_script

Get all the files and directories in an array in $PWD

Simple

ALLDIR=(*)

Now you can loop though the array an ignore files with .xml and .txt extension like this

for MODEL in ${ALLDIR[*]}
do
    if [[ ! $MODEL == *".txt"* ]] && [[ ! $MODEL == *".xml"* ]] ; then   #ignore .txt and .xml
        echo "$MODEL";
    fi
done

That’s a simple example of bash regex

Read cmd output into an array

Read it into a variable and read variable into array

VAR="$(<command>)"

my_array=("$VAR")

Split string and read last two elements

LAST=`echo $VAR | awk -F'/' '{print $(NF)}'`
SECOND_LAST=`echo $VAR | awk -F'/' '{print $(NF-1)}'`

Splitting at ‘/’

Check out the other post where I describe useful unix tricks and shortcuts here:

Generic Engineering tidbits

Other useful links

bash:tip_colors_and_formatting - FLOZz' MISC

Why won't the terminal display new lines?

Scrolling is disabled all of a sudden in terminal

Linux Shell Scripting Tutorial - A Beginner's handbook

Bash progress bar

Splitting Files by Context: csplit (Unix Power Tools, 3rd Edition)

date -d 'previous Monday" to display the preceding Monday