Testing Cordova Apps with Appium

Testing is an important part of creating a consistent and reliable experience for your app’s users. Previously on The Web Tub we have talked about unit testing your Cordova apps. This is a great way of testing your code, but it does not directly test the overall app experience. This is where Appium can improve our test suite. It helps us to mimic real user actions in the app itself to ensure that everything is working as expected.

In this post, I will outline how to run Appium tests on both Android and iOS Cordova apps.

What is Appium?

Appium is a “test automation framework for use with native, hybrid and mobile web apps”. It runs as a separate app on your device (or emulator), and accepts commands from your computer which it then uses to control your app. These commands use the WebDriver protocol, which is a standard way of controlling automators such as Appium.

On the computer side, there are many tools available to control Appium. One of the most popular and easiest to set up is WebdriverIO, which we will use. Since we write Cordova apps using Javascript, it makes sense to write our tests using Javascript too. But, you can actually write your tests in any language if it has a library which supports the WebDriver protocol.

Setup

First let’s install all the libraries we need. Ensure your project has a package.json file so this will work. If it does not, run npm init and follow the instructions to generate the file. Now, let's install!

npm install appium appium-doctor webdriverio wdio-appium-service wdio-jasmine-framework wdio-spec-reporter --save-dev

Once these are installed, edit your package.json file so that the script block looks like below. This will make these applications easier to run.

"scripts": {
  "appium": "appium",
  "appium-doctor": "appium-doctor",
  "test": "wdio tests/wdio.conf.js"
}

Appium requires quite a few things to be set up correctly on your computer. Luckily, the appium-doctor program helps you figure all this out. Run it using npm run appium-doctor. If you see an X next to anything, follow the instructions listed under "Manual Fixes Needed". Make sure everything has a tick next to it before you proceed.

Additionally, we have installed Jasmine for WebdriverIO, which allows us to write our tests to work with Jasmine. WebdriverIO also supports Mocha and Cucumber if you prefer to use those. Finally, we have installed the Spec reporter, which will nicely display the results of our tests.

Sample App

Before we can start testing, we need something to test! Run the following commands to create a Cordova Android app. Don’t worry, we will test an iOS app later!

cordova create AppiumTest
cd AppiumTest
cordova platform add android

Replace the contents of www/index.html with the code below. This will give us a simple app with a button which shows and hides a box below it. We can then test this button's functionality using Appium.

Save this file and then build the Android app:

cordova build android

Configuration

We have installed everything needed, but we still need to configure it. WebdriverIO needs to know where to find our test scripts, and what kind of device we want to test on. The easiest way to configure WebdriverIO is using a config file, typically called wdio.conf.js. Create a folder called tests and save it in there.

Writing a test

We are ready to write our tests! Create another folder inside tests called specs. Now, we can add our first test file, to be saved as tests/specs/button.js.

Now all we need to do is connect our device or start up an emulator, and run:

npm run test

The first run can take a minute or two while Appium prepares the device, but subsequent tests will be faster. If all goes well, you will see the tests running on your device. Don’t worry if you don’t see much activity in the terminal. It only outputs the test result, unless you set logLevel in wdio.conf.js to another setting such as verbose. When your tests have run you should see a result like this:

Tests successfully passing on an Android 8.1 device

You may see an error like “Failed to start Chromedriver session: A new session could not be created”. This is due to Appium shipping with the latest Chromedriver, which allows us to control our app remotely. To test on older (or very new) Android versions, you must download another Chromedriver, and tell Appium to use it in your capabilities.

You can download all Chromedriver versions here. Unzip the downloaded file and copy to your project. Create a .chromedrivers folder and move the file here. Rename it to include its version number. Then in wdio.conf.js, inside your capabilities object, you can add a line like this:

chromedriverExecutable: '.chromedrivers/chromedriver-2.36'

See the Appium Chromedriver documentation to find out which version of Chromedriver you will need. To figure out which version of Chrome / Webview your app is using, check navigator.userAgent within your app.

Breaking tests

Now let’s confirm that our tests actually work by breaking our app. Go back to index.html and change the CSS so that it looks like this:

.hidden { display: block; }

Rebuild the app with cordova build android, and re-run the tests with npm run test. You will notice the test now fails because the button no longer hides the test box. You will see a result similar to below.

Failing test on Android 8.1 device(ignore the “suite1”, this is a known bug in wdio-jasmine-framework)

Testing on iOS

Testing on iOS devices and emulators is very similar to Android. The only change required is to the WebdriverIO configuration. First let’s set up the iOS platform in our Cordova project. Ensure that you have changed the CSS back so the button hides correctly again. Also, make sure that you have Xcode installed and everything set up for building a Cordova iOS project.

cordova platform add ios
cordova build ios

All we need to do is change the capabilities we request in wdio.conf.js. Make sure that the version and deviceName match what you can find in Xcode.

capabilities: [{
  platformName: 'iOS',
  platformVersion: '11.2',
  deviceName: 'iPhone 8', // this will launch the iPhone 8 emulator
  app: './platforms/ios/build/emulator/HelloCordova.app',
  autoWebview: true,
  autoGrantPermissions: true
}],

Run the test as before, using npm run test.

Taking Screenshots

You can also take screenshots using Appium. This can be a very useful way of automatically generating the screenshots you need for the app stores. Create a new file called screenshots.js with the following content:

Continuing from here

Take a look at the WebdriverIO API, which outlines all the things you can control in your tests. It is quite thorough, so you should not find much that you can’t test.

All the code for this post can be found in this GitHub repository.

Good luck with your testing!