Photo by Luis Villasmil on Unsplash

React Native Device Testing w/ Appium, Node, and AWS Device Farm

At JetClosing, we put a considerable emphasis on testing, especially for our mobile applications. We use React Native to quickly build out features for both iOS and Android. React Native provides a wonderful framework for building cross-platform applications; however, it not only increases the testing scope but also increases the difficulty for writing true end-to-end integration and automated UI testing on real devices.

We wanted to avoid using different languages and massive frameworks such as C# and Xamarin and stay within the Node ecosystem. Fortunately, there are existing frameworks available for testing mobile applications with Node: Appium and WebdriverIO. Appium allows us to use the WebDriver protocol to send/receive information to/from the device. WebdriverIO provides an API for Node that will allow us to connect to the Appium server and run our tests. Armed with these two, we are able to not only test locally on emulators and simulators but also in the cloud on real devices thanks to AWS Device Farm and their support for Appium with Node.

Our goal for this article is to help you understand how to test your React Native application in Device Farm using Appium and WebdriverIO. We will cover the following topics:

• Setting up your environment for testing.
• Allowing components in our app to be selected in our tests.
• Creating the Android APK and iOS IPA/App needed for both local testing and Device Farm.
• Testing with WebdriverIO and Appium locally.
• And finally, running our tests in Device Farm.

We are making a few assumptions in the article:

• You are developing on a Mac (iOS and Android).
• You have a React Native project (non-Expo or ejected).

Last, here are the versions we will be using:

• Appium 1.9.1
• React Native 59.1
• React 16.8.3
• XCode 10.1

If you would like to set up a new project, please follow the instructions here: https://facebook.github.io/react-native/docs/getting-started. You will want to select React Native CLI Quickstart. Once your development environment is set up and you are able to run your application, you are ready to begin configuring the application for testing.

I have also set up an example repository that contains a React Native application and tests. Check it out here: https://github.com/names144/RNDeviceFarm

Installing Appium

Before we begin testing in Device Farm, we will want to run our tests locally on simulators/emulators so we do not waste time and money on things such as syntax errors.

For this article, we will be using Appium 1.9.1 and the CLI utility. Appium can be installed as a desktop application and/or as a CLI utility. I highly recommend the desktop application, as it gives you the ability to view your components and their properties in a simple interface. You can download the application from their Github.

To install the Appium CLI, simply run the following:

npm install -g appium@1.9.1

It is also highly recommended you install appium-doctor to ensure your system has all dependencies required by Appium.

For more information on installing and configuring Appium, please see: http://appium.io/docs/en/about-appium/getting-started/

Setup Component Selection for Tests

To test our application, we will need the ability to select components to perform validation checks and actions. Unfortunately, there isn’t a standard way to set a component as selectable by the Webdriver testing framework within React Native for both Android and iOS. On iOS, we can use the attribute testID, while on Android we will need to use accessibilityLabel. For more information on this issue, see this Github thread: https://github.com/facebook/react-native/issues/7135.

To remedy this, we will need to create a utility that will apply the correct attribute for the current platform.

Example of creating a cross-platform selectable ID for testing.

This gives us a simple utility for setting the selectable ID needed when doing our testing.

Create Android APK

Creating and testing the Android APK is fairly straightforward. We will want to create an APK file that can be used by Appium and AWS Device Farm. To do this, we can run the following from the root of the project:

cd android && ./gradlew assembleRelease

The APK should be located at android/app/build/outputs/apk/release/app-release-unsigned.apk. Don’t worry about signing this APK: Appium will sign it and install it on the emulator before testing (Device Farm will do the same before running tests).

Create iOS IPA/App Build

Creating an IPA/App file for iOS is a bit more involved. We will need to create an IPA for Device Farm and a .app file for our local simulator testing.

To create the iOS app for local testing, open your Xcode project file from the ios directory.

Once your project is opened in Xcode, select the device you would like to run your tests against locally.

For my testing, I used an iPhone X. Appium will run the tests locally on this device.

Now that we have our device selected, we can tell Xcode to build our application for release, so we can target our production build with our tests. Go to Product -> Scheme -> Manage Schemes and change your scheme Run Build Configuration to Release.

Changing the build configuration to Release to target our production build.

Next, we will want to build the app for the simulator device we selected. Select Product -> Clean and Product -> Build. This will create a .app file in your Xcode build output directory. The default should be ~/Library/Developer/Xcode/DerivedData/. Once your app builds and you have verified the .app file exists, you are ready to test on the simulator locally.

For Device Farm, we will need to create a signed IPA file. Device Farm will re-sign our IPA file before installing on their devices, but we still need to archive our app and sign it. We can choose either Development or Ad Hoc signing when archiving our app.

Before we are able to archive we will need to change our device from a simulator device to Generic iOS Device.

To archive our app we need to select the Generic iOS Device.

Now, go to Product -> Clean and Product -> Archive. Once the app is archived, select Distribute App -> Development or Ad Hoc -> Uncheck Bitcode Option -> Automatic Signing.

Use either Development or Ad Hoc for Device Farm.

You should now have an IPA file that can be used for Device Farm testing.

Create and Run Tests Locally

For our tests, we will be using WebdriverIO and Appium. WebdriverIO not only allows us to communicate to Appium with the WebDriver protocol but also gives us a test runner to easily run our tests.

Before we begin writing tests, we will create a new directory that will hold our node_modules needed for our tests and test runner. Device Farm requires a zip file containing our tests and their dependencies.

From the root of your project, create a new directory for testing and add a package.json so that we can install our needed packages.

mkdir integration && cd integration && npm init -y

We will need to install a few packages so that we are able to run our tests in Device Farm.

npm install --save webdriverio @wdio/cli

Now, we need to set up our Webdriver configurations. The Webdriver CLI allows us to quickly create a configuration file through a few questions.

npx wdio config

This will install the dependencies needed based on your selections.

An example of the CLI configuration generator.

Feel free to change any of the values in the wdio.conf.js file that is generated. To view my configuration files, please see: https://github.com/names144/RNDeviceFarm/tree/master/integration

After we have configured our Webdriver test runner, we can test that everything is installed and working by running the test driver.

First, start the Appium server and your emulator/simulator, then run the following:

npx wdio ./wdio.conf.js

We get an error because we have no tests!

If you are able to run the test runner and see an error about missing tests, you are now ready to write some tests.

Add a test file in the test directory for the Webdriver test runner.

We select the testable ID on the component and make assertions, send commands, etc.

Now, we are ready to run tests against our application. Add the cross-platform test ID attribute to a component in your application and build the APK or .app file. In the wdio.conf.js file, update the capabilities to include information about the app location, emulator/simulator, etc.

Below is an example of a configuration for Android.

If you have everything set up correctly, running the test runner again should yield the following:

Example of a successful test run for the local test.

You can also watch your emulator as the tests are running!

Watch your tests run on an emulator!

Run Tests in Device Farm

After successfully running our tests locally on the simulator/emulator, we are ready to upload to Device Farm!

Before uploading, we will need to prepare our tests and bundle everything into a zip file. To do this, we will use npm-bundle.

npm install --save-dev npm-bundle

Now we need to add bundle dependencies to our package.json:

Running the package NPM script will bundle our package and then put it into a zip file that will be ready to upload to Device Farm.

Before you run the pack, you will need to remove the capabilities from the wdio.conf.js file. Remove all attributes except for maxInstances as Device Farm sets the capabilities needed.

After we have updated the wdio.conf.js file and ran npm run package we should have a zip file that is ready to be uploaded to Device Farm. Open Device Farm, create a new project or choose an existing project and click Create a new run.

Upload your APK or IPA file that we built previously.

Upload your zip file and edit the Appium YAML configuration to run the tests. In this example, we have an NPM script that runs ‘npx wdio wdio.ios.conf.js’ for iOS.

Select the device(s) you want to run your tests against.

You can also adjust network speed, directories, install other apps, and more. I left the device state as the default state and started the run. If everything goes smoothly you should see the output from the tests in the AWS console and eventually a successful test run!

AWS Device Farm stream of our test on a real iPhone device.

All greens indicating our tests passed in Device Farm.

Armed with the information in this article, you should now have the ability to test your React Native applications not only on emulators/simulators but on real physical devices in AWS Device Farm.

You can check out my example repository here: https://github.com/names144/RNDeviceFarm

This article was inspired by https://medium.com/@dabit3/testing-mobile-applications-across-hundreds-of-real-devices-with-appium-node-js-aws-device-farm-b749aefd651a

Thanks for reading and if you have any questions/comments please leave them below!