How we test AppsFlyer SDK — part 3

Intro

In part 1 we discussed multi-scenario apps, and in part 2 we talked about how to run these apps on real test devices. Next up: we need to figure out the best way to tell the app what scenario is currently being tested. So here goes: Part 3 will be about our approach to testing apps scenarios with rest API. We’ll also talk about how we do it with iOS 14.

Executing scenarios on our apps with rest API

In the beginning, we used Appium, the most common tool in existence for executing mobile automation test scenarios. We mapped a few page objects, and since we write our own apps, it was very easy to map the elements. However, we found that there were a few disadvantages to this solution, mainly because we used UI automation and UI element detection.

We did some rethinking and asked ourselves: Is this what we really need? If we write our own apps, why do we need the Appium server to tell us what to do with our own apps? And in our case, the SDK is not for UI purposes, so all we need is a way for the automation to pass some data to the app. Data such as which testID and scenarioID to run. The app should pass the AppsFlyerID, and outcome of the apps scenario.

The sequence of a scenario

In the end, we decided to use Python Flask to write a simple web service with these routes:

@app.route('/test-id-epoch', methods=['POST']) - Used by automation
@app.route('/what-test-id', methods=['GET'])- Used by app
@app.route('/app-uid', methods=['POST']) - Used by app
@app.route('/what-app-uid', methods=['GET']) - Used by automation
@app.route('/test-scenario', methods=['POST']) - Used by automation
@app.route('/what-scenario-id', methods=['GET'])- Used by app
@app.route('/test-outcome', methods=['POST'])- Used by app
@app.route('/what-outcome', methods=['GET'])- Used by automation

The app for testing (from part 1) can still run a manual test when clicked. The difference is, that when the app is launched, a request is sent to the web service asking if there’s a testID waiting. And for that, it passes the device ID (IDFA). In the event there is a testID, the app asks which scenario to run, and uses the existing logic to navigate to the relevant test, execute it, and run the scenario.

The app from part 1

When the app is launched (in appDelegate didFinishLaunchingWithOptions) it calls the web service with completion handlers.

Here’s the flow of how this all works:

1. Automation test is initiated with a device.
2. Automation test sends the testID (epoch) for the device IDFA to the web service.
3. Web service logic maps the testID to the device ID (IDFA).
4. Automation test sends the scenario ID for the testID.
5. Web service logic maps the scenario to the testID.
6. Automation test launches the app.
7. App asks the web service for the testID, according to the device IDFA.
8. App asks the web service for the scenario ID, according to the testID.
9. App informs the UI and the testFunctionManager that it received a scenario ID.
10. App’s testFunctionManager runs the scenario it receives, informing the web service of the AppsFlyerID and the completion it received from AppsFlyer.

Once the automation test receives the outcome, we move on to the validation stage, which will be discussed at greater length in part 4.

iOS 14 use case

The above test design and flow worked really well. The scenarios were running, and our tests were much faster and more stable. But then came iOS 14! And in iOS 14, if you want to use the IDFA, the user needs to approve the AppTrackingTransparency (ATT) alert by Apple.

if #available(iOS 14, *) {
            ATTrackingManager.requestTrackingAuthorization { (status) in
            }
        }

Because of the iOS 14 changes, we had to use UI automation again. And what would we do to install and approve the ATT alert? The answer was right in front of us, especially since we already used commands to run unit tests from our automation.

The unit test:

- (void)testGrantPermissiontrue {
    //     UI tests must launch the application that they test.
    XCUIApplication *app = [[XCUIApplication alloc] init];
    [app launch];
    int i = 0;
    id springboard = [[XCUIApplication alloc] initWithBundleIdentifier: @"com.apple.springboard"];
    NSArray *myArray = @[@"OK", @"Trust", @"Allow Tracking"];
    id allertAllowButton;
    while (i <= 2){
        for (id object in myArray) {
            allertAllowButton = [[springboard buttons] elementMatchingType:XCUIElementTypeButton identifier:(object)];
            if ([allertAllowButton waitForExistenceWithTimeout:5]) {
                [allertAllowButton tap];
            }
        }
        i = i + 1;
        NSString* myNewString = [NSString stringWithFormat:@"%i", i];
        NSLog(@"%@~~~~~~~~~~~~~~~~`iteration - ", myNewString);
    }    NSPredicate *predicate = [NSPredicate predicateWithFormat:@"label BEGINSWITH 'testDidFinish'"];
    XCUIElement *element = [app.staticTexts elementMatchingPredicate:predicate];
    [self expectationForPredicate:predicate evaluatedWithObject:element handler:nil];
    [self waitForExpectationsWithTimeout:90 handler:nil];
}

The unit test command:
/usr/bin/xcodebuild test -workspace /Users/leoniddiner/Desktop/Automation/sdk-automation-test-app-source-code/IOS/universallinks/universallinks.xcworkspace -scheme universallinks -destination "platform=iOS,name=iphone 8" -only-testing:universallinksUITests/universallinksUITests/testGrantPermissiontrue

And there you have it! An efficient way of handling all iOS alerts during tests. We used an XCUITest for that. And if you remember from part 2, we saw how our automation can run commands, and even be used to run a unit test bridge app. The same thing was applied here; all we had to do was use the install command and give it a little twist in the case iOS 14.

Result

• Our automation is much more stable.
• We decreased the maintenance time, since there’s no need to handle UI elements in page objects.
• We found a new way to handle UI obstacles.

Next challenge: Yes, we have a way for the automation to talk to the app through a web service, with the app triggering the SDK scenarios. But we now need a way to verify that the data being sent to our servers is the correct data.

In part 4, we’ll discuss validating the SDK request payload using validation services.