How to run parallel tests with Appium on Android using Genymotion Cloud (SaaS)

Published in

Genymobile

3 min readOct 22, 2018

Genymotion Cloud hosts cloud-based Android emulators running in a SaaS fashion or as virtual images on AWS, GCP or Alibaba Cloud (PaaS). In this article, we will focus on SaaS. This allows you to test your applications using multiple device versions ( Android 4.1 to Android 9.0).

Launching automated tests can take a long time, so with this tutorial, you will learn how to parallelize your tests in order to detect bugs as soon as possible and spend less time on your test runs.

Requirements

Python 3 environment
A Genymotion Cloud account
Genymotion gmsaas command line tool installed and configured
Android SDK platform tools with adb installed
A working Appium install

We’ve used Python throughout this tutorial but you can use your desired language; simply refer to the Appium documentation for Appium clients written in other languages.

Launch several devices in Genymotion Cloud

gmsaas recipes list lists recipes that can be started. UUID is the identifier used when starting an instance.

This step will create and launch several Genymotion devices

#start a Google Nexus 5X - 8.0 device
instance1=$(gmsaas instances start bf1a9765-c743-4cdc-95ec-c40a74493055 device_8.0) #start a Google Nexus 6P - 7.1 device
instance2=$(gmsaas instances start 00cd578a-14de-4f47-b4f1-d454d613d9da device_7.1) #start a Google Nexus 6 - 7.0 device
instance3=$(gmsaas instances start a0a9c90a-b391-42f4-b77b-ae0561d74bbe device_7.0)

The instance UUID is printed on standard output once an instance is started.

Note : Those commands only work on Unix shell like Bash or Zsh, if you are on Windows, consider using bash.exe (https://docs.microsoft.com/en-us/windows/wsl/install-win10) and call gmsaas.exe instead of gmsaas.

Once devices are started, you need to connect those devices with adb.

port1=44 && port2=20000 && port3=30000gmsaas instances adbconnect $instance1 --adb-serial-port=$port1 
gmsaas instances adbconnect $instance2 --adb-serial-port=$port2 
gmsaas instances adbconnect $instance3 --adb-serial-port=$port3

An adb tunnel will be set up and the devices in the cloud will be seen as local devices. So you will be able to see your devices with the adb devices command :

> adb devices
List of devices attached
localhost:10000 device
localhost:20000 device
localhost:30000 device

Keep the port numbers handy, as you will need them to configure the Appium server.

If you need a persistent adb port, you can add the parameter --adb-serial port (optional)

Start appium server

Since Appium 1.7, it’s easy to do parallel testing using only one appium server. Before that, we had to start N appium servers in order to test N devices in parallel.

So let’s start an Appium server, simply using the basic command:

appium

Write your tests in Python

(Again, we’ve used Python throughout this tutorial but you can use your desired language.)

We’ve chosen to use Pytest as our test framework. It has several useful plugins like:

xdist: for running tests in parallel.
rerunfailures: for rerunning failed tests an arbitrary number of times to reduce build failures due to flakey tests

Here is a simple python script:

How to run Python tests in parallel

Now it’s time to run tests on all devices!

If we run the python script as follows:

pytest test_example.py

pytest will execute tests on one device at a time. This is not what we want. In order to have results as soon as possible, we need to execute all the tests at the same time. To do that, we can take advantage of the pytest-xdist plugin and run the following command :

pytest -n 3 test_example.py

-n : number of worker processes you want to use. Here we start the suite with 3 processes

That’s it! You’ve been able to run parallel tests on several Genymotion Cloud SaaS virtual devices using appium and pytest.

If you also want to run on PaaS devices, it is easy, just start and stop the devices using the other providers tools (i.e EC2 APIs/CLI for AWS). You can also refer to tutorials for the setup.

Many thanks to Jonathan Lipps, one of the Appium maintainers and founder of Cloud Grey, for reviewing the article!

To learn more about Genymotion Cloud see the following resources :

To run parallelize your tests using Java rather than Python, see Ellinor Kwok project on Github.