Managing Android Virtual Devices during test session

In this post I would like to present you our solution for handling AVDs during our Android test sessions at Azimo.

Why bother using emulator?

Performing tests on real android devices seems to be very justified and good choice. Users are running your app on real devices - not on Android AVDs after all. Additionally you are able to:

• Reproduce real conditions under which your app is running (weather, location, network status, overheating, battery level and more).
• Devices has different hardware setups.
• Devices has different memory state.
• You can test your apps performance on various Android system mods (i.e. Cyanogen).
• You have access to build-in sensors and devices.

Those are for sure big advantages and I believe we should put them to use. Yet there are cases when AVDs can be more efficient.

• AVDs are easily customisable - you can freely test your app on different API variants without need to perform system downgrade or avoiding system updates on some of your devices.
• They are free and always available - real devices require you to use money and time in order to get them. AVDs needs to be configured and downloaded.
• No need to perform maintenance - always ready to launch your tests, you can also start AVD by using memory state image so your device is always the same as on first launch.
• No need to charge battery.
• No hardware related issues - sometime it happens that even if device is plugged to USB cable then it might get loose. That will make your device disconnected from ADB even for millisecond and force your testing session to quit and fail with error.
• You can easily duplicate them - if you wanted to perform stress tests and run 500 instances of your app at once, how would you do that? It is easier to create one AVD config and copy it to single virtual machine which you can start as many times as you want than buying 500 devices and connecting it to large number of PCs which also need maintenance.

Our solution

At Azimo we try to automate everything that is possible to be automated. It hastens our development by saving time on repeatable tasks. Consequently we have more time for some research and rocket science.

To launch our automation we use tool called fastlane. It helps us to switch between test sets and variants, launching any kind of tests on any flavour of our app, deploying app to Google Play Store along with descriptions and screenshots, distributing our *.apk to third party services, sharing it with other teams and more.

For managing AVDs, we have created fastlane plugin called automated-test-emulator-run. It wraps command for launching tests in process that prepares new set of AVDs, waits until they start and deletes them after test session finish. You can specify how many AVDs should be launched and apply precise configuration to each of AVD created/launched that way. Launch configuration is stored inside JSON config file. You can have many config files and run different variants on various device setups.

Let’s get started

In this paragraph we would like to present you how to make use of our plugin, step by step. If you have fastlane already added for your project skip first two steps.

Step 1

Install fastlane on your PC. Tutorial how to do it is available here.

Step 2

Check if you have applied fastlane to your project properly. Enter root of your project. You should have folder with name fastlane added and files such as Fastfile and Appfile added. Navigate to root of your project in terminal and insert following command:

fastlane

You should see output similar to this:

If you can see it, that means fastlane has been correctly added and configured for your project. Lanes android test , android beta , android deploy are simply examples. You can use them as model to create your own ones and then delete/modify them later.

Step 3

Time to add plugin. Make sure your terminal is currently pointing at root of your project. Insert command below to your terminal:

fastlane add_plugin automated_test_emulator_run

If this is your first plugin, fastlane will additionally ask if you want to add Gemfile for your project. Allow it, and new files (Gemfile, Gemfile.lock, Pluginfile) should appear in your project. To make everything work smooth and fast we would recommend to launch

bundle update

to clean up your dependencies. To check if plugin was successfully added to the project type

fastlane

again and you should be able to see additional frame with title Used Plugins and the name of our plugin below:

If you can see this, then we are ready to use the plugin!

Step 4

It’s time to edit fastfile and create lane that will launch our tests. We will create most basic config. fastfile is written in Ruby language. Let’s clean up unnecessary code generated after init and start writing our own lane. Now we have simple fastfile containing only one lane ready for next implementations.

Step 5

Now we would like to make use of the plugin and wrap the simplest ./gradlew connectedAndroidTest command in AVD launch process.

Configuration of the plugin might look a bit complicated. That is because AVDs has a lot of options you can optionally set during AVD creation/launch action and we support most of them. Person who is not too familiar with it might indeed get lost but we will start with the most common and simple option.

There are two configs you need to prepare in order to setup plugin correctly:

1. Config of plugin inside fastfile
2. Config of AVDs you would like to launch during test session and that should be done inside JSON file

Ad. 1: Preparation of basic plugin config

Plugin config has currently 2 necessary parameters and 7 optional ones. You can find them at the bottom of automated_test_emulator_run_action.rb file, under value “key”. Each parameter has short description of what it does.

Required parameters:

- AVD_setup_path - path to JSON file containing configuration of AVDs you want to use in your test session. It is a core part of this plugin and MUST be provided and AT LEAST ONE AVD must set.

- test launch command: gradle_task or shell_task - depending how you launch your tests you can provide either command communicating with gradle or with shell (e.g. other lane of fastlane). Plugin won’t allow you to have both parameters used at the same time.

Consequently if we set those parameters to fastfile our lane may look like this:

As you can see ./gradlew has been dropped and you simply need to pass the command. If you need more examples of lanes that are using our plugin then they are available under this link on our GitHub.

Ad. 2: Preparation of JSON file

By storing config of AVDs in JSON files you can run as much AVDs as you want. Additionally each of them can have different and unique config. Furthermore you can have few JSON files and each test starting lane connected to different file. So if you want to perform tests with usage of shards on few AVDs, which use API 16 and launch the same test set on AVDs with API 23 - all you need to do is to create 2 separate JSON files and use them in different lanes (or in single lane then after first session will end the second will start). Simple as that.

We have prepared JSON file formula that has to be filled by you:

{
    "avd_list":
    [
        {
          "avd_name": "",  

          "create_avd_target": "",
          "create_avd_abi": "",
          "create_avd_hardware_config_filepath": "",
          "create_avd_additional_options": "",  

          "launch_avd_snapshot_filepath": "",
          "launch_avd_launch_binary_name": "",
          "launch_avd_port": "",
          "launch_avd_additional_options": ""
        }
    ]
}

You can add as much as you want AVD configs to “avd_list”. Parameters are divided into three groups: unique “avd_name” which is used always, parameters used during AVD creation and parameters used during launch.

You might feel lost now if you never launched AVD from terminal but we will try to help you out. We will explain now the minimum you must do in order to create simple AVD config.

Required parameters:

- avd_name - name of your AVD, it will be used during AVD creation and launch. You have to make sure this name is unique, otherwise in case AVD with name you have already set existed on your PC then (accordingly to plugin setup in fastfile) you could either re-use existing AVD and your config wouldn’t be applied or delete existing AVD and replace it with the one specified in JSON file. In a nut shell: make sure AVD names you set in JSON file are meant to be used only by this plugin to avoid confusion.

- create_avd_target and create_avd_abi - we will talk about those fields at the same time because in case of AVDs they are very strongly dependant of each other. create_avd_target is an Android API Level that you want to use and create_avd_abi is an Application Binary Interface - in more human language, it is an architecture of CPU you would like your AVD to have. Those parameters are necessary to set. To deal with it open your terminal and type

android list targets

What you will see is a list of your currently support for system platforms (API levels) and ABI images. You won’t be able to use API levels and ABIs which are not available for this list. Furthermore each API level has separate ABI images dedicated only for this system platform.

In our case, we have installed support for android-16 (JELLY BEAN) and android-23 (M).

Each platform has some ABIs paired with it. ABI name consists of two parts divided with / character.

• Difference between ABIs starting with default and google_apis is that first one is “clean” while second one has Google Play Services installed and included.
• Difference between x86_64 and x86 is that first one emulates 64-bit CPU while second one 32-bit CPU.

To setup your targets correctly open your terminal and type:

android

It will open Android SDK Manager where you can download all required components to run your emulator. In our case:

What you need is SDK Platform. Google APIs and System Image (ABI) you want to have attached to your system platform. We do not recommend using armeabi as it is 10 times slower than x86 type architectures because it can’t use HAXM acceleration.

Finally, what you need to insert to JSON file are names of your targets and ABIs attached to them from android list targets . For example

"create_avd_target": "android-23",
"create_avd_abi": "google_apis/x86_64",

- launch_avd_launch_binary_name - simple but not too obvious field. In general when you launch your AVD you do it by calling binary responsible for it. Those binaries are located in <your SDK location>/tools

So why is this parameter even needed? To allow you to use every kind of CPU architecture you wish.

Example: Docker containers does not support acceleration for x86 and x86_64 ABIs. You are left with slow armeabi but and you won’t be able to launch AVD with that CPU architecture if you use emulator binary, you will need to use emulator64-arm instead.

If you use x86 or x86_64 architectures you are free to use emulator binary like this:

"launch_avd_launch_binary_name": "emulator",

With knowledge we have just shared, we will create JSON file that will cause two AVDs to launch. One using API 16 and second using API 23:

{
    "avd_list": [
        {
          "avd_name": "Test-Emulator-API16",
          "create_avd_target": "android-16",
          "create_avd_abi": "google_apis/x86",
          "create_avd_hardware_config_filepath": "",
          "create_avd_additional_options": "",  
          
          "launch_avd_snapshot_filepath": "",
          "launch_avd_launch_binary_name": "emulator",
          "launch_avd_port": "",
          "launch_avd_additional_options": ""
        },
        {
          "avd_name": "Test-Emulator-API23",
          "create_avd_target": "android-23",
          "create_avd_abi": "google_apis/x86_64",
          "create_avd_hardware_config_filepath": "",
          "create_avd_additional_options": "",
          "launch_avd_snapshot_filepath": "",
          "launch_avd_launch_binary_name": "emulator",
          "launch_avd_port": "",
          "launch_avd_additional_options": ""
        }
    ]
}

Save it in file with *.JSON extension and provide link to it in fastfile.

Step 6

Now when we have fastfile configured and JSON file with AVD configs created, all there is left to do is to launch our tests. Type in terminal

fastlane automation_integrationTests

and observe as your tests are running on fresh AVD instances.

HAXM (Hardware Accelerated Execution Manager)

To make your AVDs work correctly, you need to be aware of HAXM. It is a mechanism provided by Intel which speed ups Android app emulation. It allows you to use x86 and x86_64 emulator images. If you used any of those then probably you have HAXM already installed on your PC via Android-Studio SDK manager.

What is important to know. HAXM by default is allowed to use 2GB of your PC RAM. One AVD instance, by default takes ~1GB of RAM . So with default HAXM configuration you won’t be able to launch more than 1–2 AVDs. In order to fix that please, download HAXM installer from Intel website and then configure it according to your needs.

Additionally take a look at currently supported system versions. If your system version (of PC which you try to run AVDs on) is different than supported ones - it is possible to notice some malfunctions like AVDs freezing on startup or even black screens (if you are using Mac).

Final words

Our plugin provides you with more functions than we have mentioned in this post. We created only most basic, minimalistic config. If you are curious how to use other parameters or you would like to see more complicated JSON file then we have some descriptions and examples on our GitHub. Furthermore feel free to create issues and ask questions.

Happy testing!

Towards financial services available to all

We’re working throughout the company to create faster, cheaper, and more available financial services all over the world, and here are some of the techniques that we’re utilizing. There’s still a long way ahead of us, and if you’d like to be part of that journey, check out our careers page.