Automation Test Instrumentation with Katalon + CircleCI
This is the second part from the series about our journey to implement an automation testing environment in Tunaiku. You can read the first and third part here.
Continuing from my previous article about running tests from device farm, I would like to talk about instrumenting a scheduled automation tests and running them in multiple devices at once in this article. Also, as the title says, I will use CircleCI and Katalon Runtime Engine (KRE) to run the scheduled tests.
Before we start…
We need to create GlobalVariable in our Katalon project, in order to accept values from our CircleCI job later. Here is an example of the global variables which I used in my project:
It is okay to leave the default value with empty string, since we want to override these values when we run the KRE later.
Next, we can create a custom keyword in Katalon to start a Browserstack session like this.
Maybe you are wondering, why not make those global variables as parameters? Exactly! At first, I think for the same thing too. But then the test case will be know about how the environment will be running because it need to pass the variable. I think it would be better if we isolate the configuration in another place. Thus, I used the global variable directly from the keyword. Also, we can easily rewrite the test environment if we run it from KRE. So, we can determine the test environment when we run the job in CircleCI pipeline.
Now, we can use the keyword to start the session in our project!
Create CircleCI configuration file
We can start creating the configuration for CircleCI now. First, let me show you an example of the configuration.
From the example above, I want to give a little bit of explanations about why I am creating it that way.
Why am I using the docker image instead of orbs? Because I want to create a reusable command when creating the jobs. If I use orbs, we cannot parameterize it when creating the jobs. By using command, we can give it some parameters, and then define the values when creating each job for each device.
Why are there two workflows executing the same job? Because each workflow serve different purposes. scheduled_tests is our scheduled workflow, which pipeline will be started based on the cron configuration. And the other one is a workflow for pipelines that run when there is a new commit inside the branch, or using REST API.
What is run_upload_reports command for? This command is to be used to upload the reports data, which is generated by the test suite command, to Katalon TestOps. More about TestOps can be found in a later post, so stay tuned!
Remember that we have to upload our application in order to be used in our Browserstack session? We can upload it once from our local directly, but then we have to do it manually every time there is a new version. It is quite not practical, isn’t it? To make things easier, we will add a command to upload our app directly from our app build job.
Add command to upload Application
Below is an example of our config to build the application.
Let’s say from the example above, we are building our application using bundle_app command. Steps for the bundle_app command may vary for each project, hence I am not including it in the example. The command will create the .apk file at/app/build/outputs/bundle/staging. We will use those path in our deploy_to_browserstack command via path_bundle parameter. We also define our custom_id here via app_name parameter.
After deploy_to_browserstack command finished, we will trigger our test job via run_automation_pipeline command. Inside, we are calling CircleCI Pipeline API with the branch name as the request body. This will start the pipeline in our test project.
Finally, we are defining our nightly workflow, which will run our pipeline based on the cron configuration. Haven’t we already defined a cron based job inside our test workflow? Well, yes. But, we don’t exactly know which version of the app that run inside those sessions. It’s more practical to start the test pipeline from the application pipeline, so we can be sure that each time the test pipeline run, it will use the latest version of our application.
That is all from me about automation test instrumentation using CircleCI and Katalon Runtime Engine. I already mentioned Katalon TestOps in this article, but I think it will be a separate topic for another time. You can visit Tunaiku Tech to read another cool articles written by my colleagues in Tunaiku.
Cheers and see ya!
