Automated Post Deployment User Flow Testing using Datadog and Github Workflows

User flow testing is a critical aspect of usability testing that helps us evaluate that the main UI flows of our website are working correctly or we are encountering some drop-off points.

While deploying your code also, it is pretty essential to check that these major flows of our website are working correctly post deployment.

Datadog Synthetic Browser Tests is one way to implement user flow testing in our website which can be linked to Github Workflows for automated post deployment checking.

STEPS TO ADDING SYNTHETIC BROWSER TEST IN DATADOG

1. Navigate to UI monitoring on Datadog
2. Go to Synthetic Tests
3. Create a New Test -> Browser Test
4. Do some initial configuration-

i) Setup Local and Global Variables — these can be used for setting your custom website URL from your workflow or something that is generic to many of your flows. You also get options to generate random alphanumeric sequences, timestamps etc using these variables for eg- {{ numeric(n) }} generates a numeric string of n characters, {{ timestamp(n, unit) }} generates a timestamp in one of Datadog’s accepted units with a value corresponding to the UTC timestamp the test is initiated at +/- n units.

ii) View Port Setting- your flows may differ depending upon test is running from a laptop large, tablet or a small mobile

iii) Setting from which servers the tests should be initiated (remember to keep these to the places nearest to your website servers or where you have strong user bases for location based checks)

Apart from these you can set up monitors for tracking these tests in the configuration.

Once you are done with your configuration you can navigate onto the recording section where you can stepwise record your clicks , typing instructions to test your UI flow completely.

Image showcasing recording screen for Synthetic Browser Tests (credits: Datadog)

You get options to add assertions to check whether a particular element is present on the DOM, some particular text is displayed on the website etc.

You can also set parameters for waiting for some time before failing the test in the advanced section of an event to prevent test failures in cases where the actions may take some time to load (maybe due to a corresponding API call)

Refer to this video from Datadog to get better insights on the same:

Video showcasing the entire journey of Synthetic Browser Testing (credits: Datadog)

For further exploration of Datadog Synthetic Browser Tests refer these docs: https://docs.datadoghq.com/synthetics/browser_tests

INTEGRATING SYNTHETIC TESTS IN THE GITHUB WORKFLOWS

The second step after creation of these synthetic tests is to integrate them in our Github Workflows. One important thing to consider here is that these tests must run only after completion of the deployment and not during that, so we can’t run this workflow on a branch push or on making a pull request.

We can have 2 cases in here :

1. Deployments are automated in the Github workflows itself

In this case we can simply call the Datadog synthetic test on completion of the workflow associated with deployment.

Here is a sample code on how you can implement this:

name: Datadog Synthetic Browser Tests
on:
  workflow_run:
    workflows: ["your_deployment_workflow"]
    types:
      -completed
jobs:
  run-datadog-test:
    runs-on: ubuntu-latest
  steps:
    - name: Checkout code
      uses: actions/checkout@v2
    - name: Run Datadog Synthetic Tests
      uses: Datadog/synthetics-ci-github-actions@v1.1.0
      with:
        api_key: ${{secrets.DATADOG_API_KEY}}  // add secret keys to your Github  
        app_key: ${{secrets.DATADOG_APP_KEY}}
        public_ids: 'id1, id2'                 // add ids for the synthetic tests you added       

Post this you can also add more steps related to success or failure of the tests for eg sending notification to a slack channel maybe.

2. Deployments are not automated in the Github Workflows

In such case, you can use Github Workflow API’s to send workflow dispatch post requests and trigger them by the Webhooks present in your deployment service.

Check this out for dispatching a workflow on Github through Github APIs- https://docs.github.com/en/rest/actions/workflows?apiVersion=2022-11-28#create-a-workflow-dispatch-event

Here is the sample workflow code for the event to work on dispatch:

name: Datadog Synthetic Browser Tests
on:
  workflow_dispatch:

jobs:
  run-datadog-test:
    runs-on: ubuntu-latest
  steps:
    - name: Checkout code
      uses: actions/checkout@v2
    - name: Run Datadog Synthetic Tests
      uses: Datadog/synthetics-ci-github-actions@v1.1.0
      with:
        api_key: ${{secrets.DATADOG_API_KEY}}  // add secret keys to your Github  
        app_key: ${{secrets.DATADOG_APP_KEY}}
        public_ids: 'id1, id2'                 // add ids for the synthetic tests you added

You can locate public ids for your tests here (credits: Datadog)

For more details on integrating synthetic tests to workflows you can refer to this: https://github.com/DataDog/synthetics-ci-github-action