Terragrunt root selector: automatically select the best root directory base on file changed

We are using terragrunt to build the multi-environments infrastructure. ref: https://faun.pub/building-multi-environment-infrastructure-with-terragrunt-7ee1074004aa

• Simplified tree view:

in the infrastructure folder:

We used to execute following terragrunt commands to plan and deploy the infrastructure.

cd infrastructure
terragrunt run-all plan
terragrunt run-all apply

As there are more and more resources configured (more than 1000 resources per environment), the CICD pipeline take more than 30 mins to be executed.

I develop a script called terragrunt_root_selector — only ‘cd’ into the particular module and run terragrunt commands.

Here is the logic:

• files changed in one module: ie, dev/module-1 — terragrunt run all /dev/module-1

cd dev/module-1
terragrunt run-all plan
terragrunt run-all apply

• files changes in different modules — terragrunt run in the environment level

cd dev
terragrunt run-all plan
terragrunt run-all apply

I develop these steps in the CICD pipeline to find out what files have been updated in the commit, then change the terragrunt root accordingly:

   steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Get changed files
        id: changed-files
        uses: tj-actions/changed-files@v39

      - name: Decide Terragrunt root directory, plan or apply
        working-directory: ${{ env.TOOLS_BASE }}
        id: terragrunt-decision
        run: |
          bash terragrunt_root_selector.sh "${{ steps.changed-files.outputs.all_changed_files }}" "${{ github.event_name }}" | tee -a "$GITHUB_OUTPUT"
          echo "DEBUG_MODE=false" | tee -a "$GITHUB_OUTPUT"

• tf-action/changed-file@v39: returns a list of modified files in the commit
• terragrunt_root_selector.sh [list of files]: will return what is the best terragrunt root base on the list of files passed in:

terragrunt_root_selector.sh:

#!/bin/bash
single_module_list="module-1,module-2,module-3,module6-4,module-5,module-6"

#FUNCTIONS:
filter_paths_with_hcl() {
    local filtered_paths=()
    for path in $1; do
        if [[ "$path" == *".hcl" && "$path" != *"lock"* ]]; then
            filtered_paths+=("$path ")
        fi
    done
    echo "${filtered_paths[@]}"
}

is_common_config() {
    local second_part=$(echo "$1" | cut -d'/' -f2)
    if [[ "$second_part" == "common-config" ]]; then
      return 0
    fi
    return 1
}

is_networkhub_directory() {
    local path="$1"
    if [[ "$path" == *"network-hub"* ]]; then
      return 0
    fi
    return 1
}

is_top_level_directory() {
    local slash_count=$(echo "$1" | grep -o '/' | wc -l)
    if [ "$slash_count" -lt 2 ]; then
      return 0
    fi
    return 1
}

check_string_in_list() {
    local target="$1"
    local list="$2"
    IFS=',' read -ra strings <<< "$list"
    for str in "${strings[@]}"; do
        if [ "$str" == "$target" ]; then
            return 0
            return
        fi
    done
    return 1
}

get_second_to_last_directory() {
    local path="$1"
    IFS='/' read -ra path_components <<< "$path"
    local index=$((${#path_components[@]} - 2))
    local second_to_last="${path_components[$index]}"
    echo "$second_to_last"
}

get_first_two_components() {
    local path="$1"
    IFS='/' read -ra path_components <<< "$path"
    first_two="${path_components[0]}/${path_components[1]}"
    echo "$first_two"
}

process_multiple_paths() {
    local is_same_directory=0
    local second_part=""

    IFS=$' ' read -ra paths <<< "$1"

    for path in "${paths[@]}"; do
      if is_common_config "$path" || is_top_level_directory "$path" || is_networkhub_directory "$path"; then
        is_same_directory=1
        break
      fi
      local current_second_part=$(echo "$path" | cut -d'/' -f2)
      if [ -z "$second_part" ]; then
        second_part="$current_second_part"
      elif [ "$second_part" != "$current_second_part" ]; then
        is_same_directory=1
        break
      fi
    done

    if [ "$is_same_directory" -eq 0 ]; then
        echo "TERRAGRUNT_ROOT=infrastructure/$second_part"
    else
        echo "TERRAGRUNT_ROOT=infrastructure"
    fi
}

process_single_path() {
  local path=$1
  local second_to_last_dir=$(get_second_to_last_directory "$path")

  if is_common_config "$path" || is_top_level_directory "$path" || is_networkhub_directory "$path"; then
    echo "TERRAGRUNT_ROOT=infrastructure"
  elif check_string_in_list "$second_to_last_dir" "$single_module_list"; then
    echo "TERRAGRUNT_ROOT=$(dirname "$path")"
  else
    echo "TERRAGRUNT_ROOT=$(get_first_two_components "$path")"
  fi
}

#MAIN:
filtered_result=$(filter_paths_with_hcl "$1")
number_of_paths=$(echo "$filtered_result" | awk -F ' ' '{print NF}')
if [ "$number_of_paths" -gt 1 ]; then
  process_multiple_paths "$filtered_result"
elif [ "$number_of_paths" -eq 1 ]; then
  process_single_path "$filtered_result"
else
  echo "TERRAGRUNT_ROOT=NO_VALID_FILE"
fi

• Exception: “common-config” and “network-hub” folder also under “environment/” folder, so if changes in these two folders will also trigger terragrunt root changed to environment folder as well