Running Automation Test in Device Farm
This is the first part from the series about our journey to implement an automation testing environment in Tunaiku. You can read the second and third part below.
Tester: Hey, the behavior of the scrolling in this page is quite strange. Can you look into it?
Developer: Well, it works perfectly in my test device. I cannot reproduce that. Must be an isolated case.
Tester: I know, but I am afraid this will impacting many users with the same device, and I cannot give it a pass.
You may have similar experience with the situations above, especially if you are developing products in the mobile segment. These days, the mobile device segment is heavily fragmented, from the notches, screen resolution, and their respective flavor (MIUI, One UI, AOSP, iOS). This may become cumbersome to test the behavior of each device. Thus, device farms come to help.
A device farm is a service, where they provide a wide variation of devices which will be available for us to use in our testing scenario. It will run our tests in a real, or emulated, devices inside the farm. Currently, there are many device farm providers are available, like Kobiton, Saucelabs, AWS Device Farm, and Browserstack. For this post, I will use Browserstack as an example.
Running a Katalon Test Case in Browserstack
At Tunaiku, we use Katalon Studio as our testing framework. Since Browserstack provides the URL to start a session using Appium Webdriver, the compatibility with Katalon Studio is quite good. There is a drawback for this though, since Browserstack only accepts an application which is hosted in their own servers to run in a session, therefore, restricts our storage options for the applications. I will share about how to run a test case in a Browserstack session in a few steps below.
Uploading Application File to Browserstack
Before we run any test script in Katalon Studio, first we need to upload our application file to Browserstack server. To do this, we can use the app automate REST API. For example, below is how to upload the files from command line using curl.
curl -u “<browserstack-username>:<browserstack-apikey>” -X POST “https://api-cloud.browserstack.com/app-automate/upload" -F “file=@C:\\home\experiment\flutter\circleci_katalon_test\flutter-test.apk” -F “custom_id=<your-custom-id>”From the example above, there are some things we must look upon. First is the -u flag value. this indicates our username and API key in Browserstack. You can find those values from app automate dashboard. Next, from first -F flag value. This indicates the local path of our AUT (Application Under Test). Last is the -F value containing the custom_id of our AUT. This custom_id will be used inside our test script to indicate which file we want to run in the Browserstack app automate session.
Connecting our test script to a Browserstack App Automate Session
Inside our test script, we need to change our way to start the application. Typically, we will do it this way:
This will directly opening the the app inside any selected devices which connected to our local machine. But, in order to create a new session in Browserstack, we will utilizing DesiredCapability and AndroidDriver class. Below is an example in how to do that.
Check Session in Browserstack Dashboard
To check if the session is successfully created, go to the dashboard.
This will show you a list of your existing projects, and it contains a project with the same name with the ones you declared in your DesiredCapabilities option before.
Click the projects to see list of all created sessions under the projects. This will show general information about the sessions, like status, devices, and duration.
Click a session to show more detailed information about the sessions. You can see and download video of the session to see how the tests running just like how it will run in a real device. You can also view some logs, like device log and appium logs.
This concludes my share about how to running a Katalon Studio test case in a Browserstack session. I hope this will adding more knowledge about automation testing in general to you. You can visit our Engineering blog to read more article about automation testing or engineering in general inside Tunaiku. Thanks for reading and see you later!
