Speeding Up Android Test Automation with Scalable Infrastructure

Linux VM-based Setup for Android Application Automation

The mobile applications we develop at VMware support a wide variety of devices and OS versions, from Android 4.4 to Android 11.

As the number of mobile applications grew in our organization, it became difficult to test each application on a variety of devices and OS versions to provide fast feedback. This was a bigger problem on Android due to the high level of fragmentation.

So, what did we do to solve this problem?

One of the approaches, VMware has taken to scale Android-based test automation is by making use of emulators in Virtual Machines (VM).

In this post, we describe the approach we took to spin environments on the fly with emulators of various OS versions, which helped us scale our test automation and test infrastructure creation while providing better stability, speed, and reliability.

Goal

Most of the time, automation scalability is linked with hardware and device availability; As the number of test suites, tests, and the build pipelines grow, we needed to increase the number of build machines and devices to keep our overall build time under control.

On average, we see 80 merges in a single project in the development branch in any given month. Our BATS (Build Acceptance Tests) consisting of ten test cases would take ten minutes each, which would mean 800 mins of tests being run per month, per team on a dedicated environment. It would require a dedicated build machine and a mobile device.

Instead of opting to add additional physical machines and devices to include more OS versions in our pipeline, we chose to create a highly scalable and consistent environment consisting of Linux Virtual Machines (Ubuntu flavor), for running our Android-based automation tests on Android emulators.

Getting Started

To achieve our goal of a VM capable of running Android-based Espresso and UI Automator tests, these are primary steps involved:

1. Creating and setting up a virtual machine, which acts as a test environment and can also function as a build environment.
2. Setting up the emulators inside the test environment.
3. Converting the VM to a template.

In the subsequent sections, we are going to go over the steps for creating a VM and setting it up with the required configurations.

Step 1: Creating and setting up a virtual machine

Creating the environments, which we can clone and use, involves the following steps:

1. VM Creation
2. Display Setup
3. VNC Setup
4. Java Setup
5. Android Studio Setup

VM Creation

Using vSphere, we took a basic Ubuntu template and configured a VM according to our needs.

One of the pre-requisites is for the VM to support hardware acceleration.

Instructions for creating VMs can be found here:

Expose VMware Hardware Assisted Virtualization

Hardware acceleration can be enabled using the instructions seen here.

Display Setup

To begin, we would want to update all the available repositories and the packages on the created VM.

SSH to the Linux machine and run the script below to upgrade the packages, install a package management tool, and a light-weight display manager.

One way to install a GUI on a Linux machine is by using the command below:

sudo tasksel

Reference: https://phoenixnap.com/kb/how-to-install-a-gui-on-ubuntu

Reboot the machine for the GUI screen to show up post-reboot.

VNC Setup

We can install VNC and set the default password by using the below commands:

To configure the x11VNC beh avior during the bootup, update /etc/systemd/system/x11vnc.services file with the configuration below using sudo permissions:

The above setting would make the VNC pick the default display, use the VNC password configured in the path ‘/etc/x11vnc.pass’, and listen on the port 5900.

Reference: https://linux.die.net/man/1/x11vnc

Run the script below to start x11vnc by default on reboot:

Setting up Java on the VM

We can install Java on our VM using the below script:

Verify the Java version on your machine:

java -version

Modify .bashrc to update JAVA_HOME:

Setting up the Android Studio on the VM

Using the VNC Viewer, we can connect to the device using the IP address and the port number 5900 to do the remote logging.

Once logged in, we can click on the Ubuntu software and search for Android studio, select and go ahead with the installation.

Once installed, open the Android Studio and go ahead with the default installation steps.

At this stage, we are ready to install the Android SDK and the command tools needed to install the emulators. Click on the tools and open the SDK Manager and installation reference can be seen below:

https://developer.android.com/studio/intro/update

Open .bashrc and update ANDROID SDK and ANDROID_HOME path as below:

We need to copy the adb to the local path, by doing the following:

sudo cp $ANDROID_HOME/platform-tools/adb /usr/bin

Step 2: Setting up the emulators inside the test environment

Here, we set up the emulators needed for our automation environment on the VM and clone the final template for replication and scaling up of test setups.

It involves the following steps:

1. Setup KVM and install the emulators
2. Preparing the emulators for tests

Setting up KVM and creating the emulators

Before we start the emulator installation, we would need to set up the Android KVM and give the required permissions to the user, by running the below script:

To create emulators please refer:

https://developer.android.com/studio/run/managing-avds

Once the required emulators are created, they can be accessed by using the following commands:

Prepare the devices for the Espresso/Instrumentation tests

Test Services and Test orchestrator would be required on the device to run Espresso and UI Automator tests.

Follow the commands below to install the required services.

Reference: https://developer.android.com/training/testing/junit-runner.html#ato-command-line

Step 3: Converting the VM to a template

We now have all the required installations and are ready to convert VM to a template.

Depending on the framework's need and if any other features are needed, they can be added to the VM.

Once done, we can convert the VM to a template by following the steps below:

Clone a Virtual Machine to a Template in the vSphere Web Client

Conclusion:

At this point, we have a fully capable VM with the ability to run the Android-based tests on them and create new VM’s on the fly.

Prior to this solution, our team was using Mac minis to host the Android emulators and real devices, which were not scalable as our test suite size increased and our build pipelines evolved.

We were able to create a new scalable Android emulator test environment through code with this approach which has the same benefits, speed, and stability as the real host machines.

In the next post, we will cover how to build, distribute, and test Android applications on these VM’s. We will also show, how to run tests in parallel with multiple emulators.

References

1. Virtual Machines |VMWare. Deploying Virtual Machines
2. Android Automation | Android. Android writing automated tests with UIAutomator
3. Android Automation | Android. Android automation with Espresso
4. Automation | Gupta, Amit. “Efficient UI Test Automation: Decoupled Element Identifiers Services” European Journal of Advances in Engineering and Technology, vol. 5, no. 2, May. 2018, pp. 136–141, ejaet.com/PDF/5–2/EJAET-5–2–136–141.pdf