How to Handle Cancelable Github Actions which Require Status Checks to Pass Before Merging

If you use Github Action for your CI/CD Process and require some status checks before merging your feature branch to the target branch, you can encounter with the Cancelable Action status always succeeded. In this blog, I’ll try to explain what is the problem and how you can solve it.

Photo by Dim Hou on Unsplash

In Github actions, you can set some properties at job level and step level. For example, we have an action named E2E Tests in the below code block and it will be triggered whenever a Pull Request ,target branch is master, is created. The configuration properties like timeout-minutes can be set both job level and step level.

name: E2E Tests

on:
  pull_request:
    branches:
      - master

jobs:
  e2e-test:
    runs-on: ubuntu-latest
    continue-on-error: true
    timeout-minutes: 1
    outputs:
      e2e: ${{ steps.run_e2e.outputs.e2e }}
    steps:
      - name: Run E2E tests
        id: run_e2e
        run: |
          sleep 100
          echo "e2e=success" >> $GITHUB_OUTPUT

      - name: If action is canceled or failure, then exit with code 1
        if: failure() || cancelled()
        run: |
          exit 1

  send-slack-notification:
    runs-on: ubuntu-latest
    needs: [e2e-test]
    steps:
      - name: Send slack notification
        run: |
          e2e_result="${{needs.e2e-test.outputs.e2e}}"
          if [[ -z "$e2e_result" ]]; then
            e2e_result="failure"
          fi
          echo "E2E Result: $e2e_result"

In our example, we try to run our E2E tests and according to the result of them we will send slack message for notification. For passing first step, we use continue-on-error as true. It guarantees that the second job will be run always (no matters the first job’s status).

Let’s examine the problem;

If continue-on-error: true and timeout-minutes: 1 are defined at job level and in the first step we wait nearly 100 seconds (greater than 1 minute), the operation will be canceled.

We expect that because of job is completed by cancelation, step will throw exit code 1 and job status is failure but job status is succeeded.

- name: If action is canceled or failure, then exit with code 1
  if: failure() || cancelled()
  run: |
    exit 1

If you require e2e-test job as status check requiring in Branch Protection Rules, Pull Request behaves as mergeable even if E2E Job operation has timed out.

Solution

If we define continue-on-error as true at job level and timeout-minutes at step level like the following code block, the e2e-test job will fail because of step has timed out as expected. Thus, Pull Request will not behave as mergeable because of requiring status check of e2e-test job in branch protection rules.

jobs:
  e2e-test:
    runs-on: ubuntu-latest
    continue-on-error: true
    outputs:
      e2e: ${{ steps.run_e2e.outputs.e2e }}
    steps:
      - name: Run E2E tests
        id: run_e2e
        timeout-minutes: 1
        run: |
          sleep 100
          echo "e2e=success" >> $GITHUB_OUTPUT

Conclusion

If you use Github Action for CI/CD process and when you apply some Branch Protection rules like requiring some action’s status must be succeeded for allowing to merge source branch to the target branch, you need to define timeout parameters at the step level for handling cancelation and timeout errors.

Thanks for reading. If you have questions or comments regarding this article, please feel free to leave a comment below.

Would like to get in touch? Reach me out on LinkedIn:

https://www.linkedin.com/in/muhammedsaidkaya/