How we test AppsFlyer SDK — part 2

Intro

In the previous blog post, we looked at how to create a test scenario. We saw how we developed the app for maintaining the scenarios, and how we build it with a script.

In our case, after we build the app, we need to run it on real test devices.

In this blog post, we will first talk about automation commands. That will help us understand what is needed in the automation code to run an app functionality for a test device. And then, we will discuss what a test device is.

Automation commands

One of the first things those new to mobile and just getting into Android learn is ADB commands and how to install an APK on a test device. Same goes for iOS with iOS-Deploy. So, we know what commands we need to run over our test devices.

Note: For those of you that do mobile automation and usually use Appium, you will see that we took a different approach and avoided using Appium for launching and running test scenarios.

Abstract functions that will be implemented for both iOS and Android

We start by installing the application.

Android:

@Override
public Integer installApplication(String packageName) throws IOException {
    Reporter.write("install application on device: " + device.getDeviceName());
    StringBuilder stringBuilderInstallApp = new StringBuilder();
    int resultInstallApp = runCommand(new String[]{"adb", "-s", device.getUdid(), "install", packageName + ".apk"},
        (String) TestData.getInstance().getTestData(TestDataKeys.APPS_FOLDER), stringBuilderInstallApp);
    int didInstall = runCommand(new String[]{"echo", "$?"},
        (String) TestData.getInstance().getTestData(TestDataKeys.APPS_FOLDER), stringBuilderInstallApp);
    if (didInstall == 0){
        Reporter.write("***Installation success***");
    }
    if (stringBuilderInstallApp.toString().contains("No such file or directory")) {
        Reporter.write("App wasn't installed, No such file or directory");
        return Constants.ERROR_COMMAND_EXECUTE;
    } else if (stringBuilderInstallApp.toString().contains("INSTALL_FAILED_ALREADY_EXISTS")) {
        Reporter.write("App wasn't installed, Application already exists");
        return Constants.ERROR_COMMAND_EXECUTE;
    }
    return resultInstallApp;
}

iOS:

@Override
public Integer installApplication(String packageName) throws IOException {
    Reporter.write("install application on device: " + device.getDeviceName());
    StringBuilder sb = new StringBuilder();
    String[] command = new String[]{IOS_DEPLOY, "--id", device.getUdid(), "--nostart", "--bundle", packageName + ".app"};
    int result = runCommand(command, (String) TestData.getInstance().getTestData(TestDataKeys.APPS_FOLDER), sb);
    if (!sb.toString().contains("100%")){
        return Constants.ERROR_COMMAND_EXECUTE;
    }
    else {
        Reporter.write("The application should be installed");
    }
    return result;
}

As you can see, we need the path to .app and .apk files in both cases. And we also need the device ID and device name. Finally, we also need the advertising ID for each device, since the apps uses it for the automation web service REST API (will be explained in part 3 of this blog series).

A few cool commands that we frequently use:

1. launchApplication — We use this to separate the install from launch in some cases (like deep linking). This is because when we do deep link tests we want the deep link action to launch the app and not the launch itself.
2. runDeepLinkCommands — In Android, we have ADB open a webpage with a scenario as a query string parameter. Within the page, we have a javaScript function that clicks the link with the scenario name. In iOS, we have a bridge app that runs a unit test as a command on a real device, to open the same webpage with the scenario as query string parameter.

Testing devices

We know that it's possible to install an app using commands. And we have a code that executes these commands for both iOS and Android. All that we need is the APK/app path, device name, and UDID. Additionally, a simple list of device objects is needed for iOS and Android. The device name is passed as a parameter from Jenkins (the same way as the SDK version).

We already have an apps repo (part 1) and we have an automation repo that installs the app over test devices. Now, let’s say we have 30 test devices, and each install will take about one minute. Does this mean we have to wait 30 minutes for all the devices to have the app installed? No! The trick here is to have a Jenkins job for each test device that calls the automation with a test device name as a parameter. This means that we have separate the automation and apps build jobs from the actual run per device.

The Jenkins pipeline flow for running parallel test devices

Result

And now you have it:

• We have commands that can be triggered by automation code.
• We have objects for devices properties in order to run the commands over the connected devices. As a result, we have an automation run for each device for running in parallel.

Next challenge: Once apps are installed, how do we make the apps run the selected scenario (if you remember in part 1 we talked about multi-scenario apps)?

In part 3, we will talk about executing scenarios on our apps with REST API.