Open in app

Sign in

Write

Sign in

https://static.visiondirect.info/media/wysiwyg/Fastbanners.jpg

Detox — Gray Box Testing

Selvakumar Subramanian

--

Gray Box End-to-End Testing and Automation Framework for Mobile Apps

Detox does Gray box to allows the test framework to monitor the app from the inside and actually synchronize with it.

What is Gray Box Testing?
Gray box testing is the testing of a software application using an effective combination of both White box testing & Black box testing method. This is a nice & powerful idea to test the application.

The leading native gray box drivers are developed by Google — EarlGrey for iOS and Espresso for Android

EarlGrey is a native iOS UI automation test framework that enables you to write clear, concise tests.

Espresso — A key benefit of using Espresso is that it provides automatic synchronization of test actions with the UI of the app you are testing. Espresso detects when the main thread is idle, so it is able to run your test commands at the appropriate time, improving the reliability of your tests. This capability also relieves you from having to add any timing workarounds, such as Thread.sleep() in your test code

Detox does not rely on WebDriver, since this is not the web. Detox communicates with its native driver (which extends EarlGrey and Espresso) using a JSON-based reflection mechanism, this allows a common JavaScript implementation to invoke native methods directly on the device.

Espresso idling resources

An idling resource represents an asynchronous operation whose results affect subsequent operations in a UI test. By registering idling resources with Espresso, you can validate these asynchronous operations more reliably when testing your app.

Architecture

The sequence diagram below shows the general communication scheme between the components in Detox.

https://github.com/wix/Detox/blob/master/docs/Introduction.HowDetoxWorks.md

Design Principles

Detox does not rely on WebDriver

Detox is built from the ground up to integrate with native layers of your mobile app directly. We try to avoid generic cross-platform interfaces that are often the lowest common denominator. We want to optimize per platform.

Detox does gray box, not black box

Theoretically, it sounds better to test exactly what you ship as a black box. In practice, switching to gray box allows the test framework to monitor the app from the inside and delivers critical wins like fighting flakiness at the core.

Detox relies on EarlGrey and Espresso

The leading native gray box drivers are developed by Google — EarlGrey for iOS and Espresso for Android. Detox relies on them using a JSON-based reflection mechanism which allows a common JavaScript implementation to invoke their native methods directly.

Friendly Protractor-like API for tests

Tests in Detox are implemented in human-readable JavaScript and can even be shared between platforms. This easy to use API completely abstracts the complex native driver invocations taking place under the hood.

Detox controls devices through low-level APIs

Let’s take iOS simulators for example, which are difficult to control efficiently since multiple concurrent instances aren’t supported. Detox uses AppleSimulatorUtils (another opensource library by Wix) to work around these issues and support test shading.

Built from the ground up for mobile and React Native

Detox is inspired by web testing methodologies but is not a direct translation of a solution designed for a different platform. Detox is built from the ground up for native mobile and has deep first-class support for React Native apps.

Detox relies on websockets for communication

Communication between the test script running on Node.js and the tested app running on device uses websockets. This provides true bi-directional communication and is much faster and resilient than REST-like protocols.

Both tester and testee are websocket clients

The test script running on Node.js and the tested app running on device are both clients. This allows one side to disconnect without affecting the other. A separate proxy websocket server is used to connect them.

Expectations run on the testee, not the tester

Traditionally, test frameworks evalutate expectations in the test script running on Node.js. Detox evaluates expectations natively directly in the tested app running on device. This enables operations that were impossible before due to performance reasons.

Test Life Cycle

Detox is test runner independent, we can use mocha/jest/chai etc.

Add Detox to React Native Project

Install Detox command line tools (detox-cli)

npm install -g detox-cli

Install Detox
In React Native project, go to its root folder (where package.json is found) and type the following command.

npm install detox --save-dev

Install a test runner
Use any JavaScript test runner. Detox CLI supports Jest and Mocha out of the box

npm install mocha --save-dev

Add Detox config to package.json

"detox": {
    "test-runner": "mocha",
    "specs": "e2e",
    "runner-config": "e2e/mocha.opts",
    "configurations": {
      "android.emu.debug": {
        "binaryPath": "android/app/build/outputs/apk/debug/app-debug.apk",
        "build": "pushd android && gradlew assembleDebug assembleAndroidTest -DtestBuildType=debug && popd",
        "type": "android.attached",
        "name": "yourdeviceID"
      },
      "android.emu.release": {
        "binaryPath": "android/app/build/outputs/apk/release/app-release.apk",
        "build": "pushd android && gradlew assembleRelease assembleAndroidTest -DtestBuildType=release && popd",
        "type": "android.attached",
        "name": "yourdeviceID"
      }
    }
 }

android.attached for Android Device and android.emulator for Android Emulator

Create your first test (I have used mocha)

detox init -r mocha

  • Create an e2e folder in your project root
  • Inside e2e folder create mocha.opts (for mocha)
  • Inside e2e folder create init.js file
  • Inside e2e folder create app.spec.js

Add Detox dependency to an Android project

In android/settings.gradle add:

include ':detox'
project(':detox').projectDir = new File(rootProject.projectDir, '../node_modules/detox/android/detox')

In android/app/build.gradle add this to defaultConfig section:

defaultConfig {
      ...
      testBuildType System.getProperty('testBuildType', 'debug')  //this will later be used to control the test apk build type
      missingDimensionStrategy "minReactNative", "minReactNative46" //read note
      testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
      ...
  }

Please be aware that the minSdkVersion needs to be at least 18.

In android/app/build.gradle add this in dependencies section:

dependencies {
	...
    androidTestImplementation(project(path: ":detox"))
    androidTestImplementation 'junit:junit:4.12'
    androidTestImplementation 'com.android.support.test:runner:1.0.1'
    androidTestImplementation 'com.android.support.test:rules:1.0.1'
    ...
}

And in android/build.gradle you need to add this under allprojects > repositories:

buildscript {
    repositories {
	     ...
        google()
        ...
    }
}

Create Android Test class

You need to add the file android/app/src/androidTest/java/com/[your.package]/DetoxTest.java

Build your app and run Detox tests

Build your app

detox build

If you have more than one configuration(package.json) we need specify the configuration as below

detox build --configuration android.emu.debug

Run the tests

detox test

If you have more than one configuration(package.json) we need specify the configuration as below

detox test --configuration android.emu.debug

Troubleshooting for Android Devices

react: 16.6.1
react-native: 0.57.5
mocha: 5.2.0
detox: 9.1.2

a) I have used Android Mobile (Redmi Note 4) version 7.0 to test, while executing the detox test, got an error in application installation.
detox[69260] ERROR: [exec.js/EXEC_FAIL, #8] failed with code = 1, stdout and stderr:

The above error is due to installing the APK with -g (grant all runtime permissions), fixed by updating the below code in node_modules\detox\src\devices\android\ADB.js

let childProcess;
    if (apiLvl >= 24) {
      childProcess = await this.adbCmd(deviceId, `install -r -g ${apkPath}`);
    } else {
      childProcess = await this.adbCmd(deviceId, `install -rg ${apkPath}`);
    }

to

let childProcess;
    if (apiLvl >= 24) {
      childProcess = await this.adbCmd(deviceId, `install -r ${apkPath}`);
    } else {
      childProcess = await this.adbCmd(deviceId, `install -rg ${apkPath}`);
    }

b) Security Exception for INJECT_EVENTS permission

Reference:
[Android]:https://developer.android.com/training/testing/espresso/idling-resource#java
[Detox]:https://github.com/wix/Detox
[Detox]:https://github.com/wix/Detox/blob/master/docs/Introduction.Android.md
[Detox]:https://github.com/wix/detox/blob/master/docs/APIRef.Configuration.md#test-runner-configuration
[Blog]: https://hackernoon.com/detox-gray-box-end-to-end-testing-framework-for-mobile-apps-196ccd9564ce
[Youtube]: https://youtu.be/GgFFeI70PWw

Full example project in GitHub https://github.com/selvakumardreams/DetoxExample

 by the author.

--

--

Selvakumar Subramanian

Written by Selvakumar Subramanian

34 Followers

https://www.youtube.com/@hopelabsdevops