Android UI Automation, Jenkins and GenyMotion

So recently I came across an interesting topic that I believe most of the people who are working with Android apps are seeing: How to run UI Automation in your CI. 
In particular, in our company, we are using Jenkins to driving most of these works. When the codes get pushed, Jenkins will kick off a build, start a GenyMotion VM in AWS, run the UI tests against the VM, stop the VM, publish the HTML reports. 
I’ll walk you through all the setup in details below, the most difficult bit in this task is how to use a VM in the cloud so we could scale up and down by demand. AWS VMs or other cloud providers’ VMs just couldn’t run Android Emulator. I read about Ravello but couldn’t find more information about Ravello in AWS anymore after they came to Oracle. 
I have talked with a lot of friends in the Android dev community and most of them are using their own physical boxes and setup VM in there which we don’t really want to go there yet. We will have our own box though when we run tests against our new Android devices (soon :) ).
We also tried the AWS device labs and Google firebase test labs but they are expensive since they are using real physical phones, also they provide a limited number of way to set up the device which won’t work for us.
So only another option I can find is GenyMotion. And they are really nice. For our current need, we just need the VM in a t2.small, so the total price is only about $0.15 an hour. We would spin up and down multiple VMs if needed and this is helpful for test sharding.

So here are the details on how to set it up:
Big pictures, there are three main components

1. Android app and Android tests. We are using Espresso and UI Automator to test our apps and the system. (https://developer.android.com/training/testing/espresso/)
2. GenyMotion VM to run the tests on
3. Jenkins to orchestrate the build, tests, publishing, and reports

#1 Setup Espresso and UI Automator for your test:

We choose these frameworks since they are suggested by Google and also provide us with everything we need for our tests. In our company, we have a few apps and also have changes in the system itself and UI Automator would take care of all UI simulation we need. I heard good things about Appium as well but we don’t have cross-platform apps so we didn’t go that route. 
There are multiple instructions on how to set up and use Espresso/UI Automator and we found the official documents are very useful as well https://developer.android.com/training/testing/espresso/setup

#2 Setup GenyMotion in AWS

1. Go to GenyMotion, Android 8.0 (By the time you read it, if they have newer VM version please feel free to use it, the setup step should be similar)
https://aws.amazon.com/marketplace/pp/B07CXZMC9J?qid=1534822994547&sr=0-3&ref_=srh_res_product_title&cl_spe=C 
2. Click Continue to Subscribe

3. Click continue until you see this page. Don’t be scared of the price, it’s for m4.xlarge instance which you don’t really need (yet?). t2.small is good enough for us and it’s the cheapest option

Click Continue to launch then you will be able to customize the instance, VPC, security group, keypair value etc. on the next page. And hit launch.

4. Give step #3 a few minutes. After your EC2 instance of GenyMotion is ready, you will need to set up your VM to your needs. 
In our case, we need Google play service and few initial apps. To install the app in GenyMotion VM, it’s as easy as drag and drop or just using adb command.
To enable adb, please follow their instruction here https://www.genymotion.com/help/cloud-paas/tutorial/enable-disable-adb/ (basically ssh to the VM and run ‘setprop persist.sys.usb.config adb’)
Then now you could use `adb connect <ipaddress>` and install everything. 
These tutorials are good for installing apps and google apps as well
https://www.genymotion.com/help/cloud-paas/tutorial/install-app/
https://www.genymotion.com/help/cloud-paas/tutorial/install-gapps/ 
If you don’t have luck with installing google play service, below are steps that work for me

- Download google play service zip online.
- Copy archive adb push archive.zip /sdcard/Download/archive.zip
- Flash archive adb shell /system/bin/flash-archive.sh/sdcard/Download/archive.zip
- Reboot adb reboot

5. To avoid setup steps again, just create a snapshot of this VM

aws ec2 create-image — instance-id i-12341f21492894f34 — name genymotion_snapshot

Next time, you can just spin up a new VM using this snapshot. And to use the VM (finding public ip address, connect to vm, etc.) you could find more information in #3 below

#3 Jenkins

Since this isn’t an article about setting up Jenkins with Android (Please feel free to ping me if you think one detail tutorial like that could be useful, I’ll spend some time and create one in the future), I’ll link to one example of setting Jenkins up and go in details on how to run Jenkins with GenyMotion.
Personally, I found this article is quite good in case you never set up a Jenkins with Android before https://appsgit.com/jenkins-continuous-integration-continous-delivery-android-ios-mobile-applications/ 
To run Jenkins with GenyMotion, I basically using a shell script to connect Jenkins box with the VM using ADB command and run the tests there.
Note that, all the steps below are after you get the Android app build in Jenkins.

Below are the high-level steps:
a. Build the app
b. Start GenyMotion VM and connect to VM

c. Run tests
d. Stop the VM
e. Publish the results

a. Firstly, build the app

clean
:app:assembleDebug
:app:publishApkSnapshotPublicationToWonderMavenReleaseRepository

b. Secondly, start and connect to the VM:

One note in piece, since GenyMotion using adb version 1.0.39, you will need to use same adb version in your Jenkins box and I got it here https://dl-ssl.google.com/android/repository/platform-tools_r25.0.4-linux.zip

Also please make sure to update your region to correct value.

# Start emulator with tag name == androidemulator01
_instance=$(aws ec2 describe-instances — region us-west-2 — filters “Name=tag:Name,Values=androidemulator01” — query Reservations[].Instances[].[InstanceId] — output text)
aws ec2 wait instance-stopped — instance-ids $_instance — region us-west-2
aws ec2 start-instances — instance-ids $_instance — region us-west-2
aws ec2 wait instance-status-ok — instance-ids $_instance — region us-west-2
# Get the Public DNS
public_dns=$(aws ec2 describe-instances — region us-west-2 — instance-ids $_instance — query Reservations[].Instances[].[PublicDnsName] — output text)
# adb in GenyMotion is in version 1.0.39 so we need to use the same one in jenkins box
/home/ubuntu/platform-tools/adb connect $public_dns
/home/ubuntu/platform-tools/adb devices
/home/ubuntu/platform-tools/adb wait-for-device

c. Thirdly, run the tests using Spoon
We could run using Gradle command ‘connectedAndroidtest’ but I want to have better reports, logcat and also sharding capability. I’m using Spoon (https://github.com/square/spoon) for running tests. To do that, I’ll have to install the app and test by myself and also uninstall them.

installDebug
installDebugAndroidTest
spoonDebugAndroidTest
uninstallDebug
uninstallDebugAndroidTest

d. Last but not least, stop the VM so we save $$

Please make sure to update your region to correct value.

# Stop emulator
_instance=$(aws ec2 describe-instances — region us-west-2 — filters “Name=tag:Name,Values=androidemulator01” — query Reservations[].Instances[].[InstanceId] — output text)
aws ec2 stop-instances — instance-ids $_instance — region us-west-2

e. Publish the reports

That’s it. Hopefully this could help you set up a CI pipeline with Jenkins for Android app and run UI automation on it cheaply.

Like what you read? Give Minh Dong a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.