How to run Android apps in AWS Device Farm
This blog post describes how to prepare, build, and run Android apps in AWS Device Farm. The article is intended for mobile engineers and QEs, and provides the documentation of the solution that should make our life easier if there is a need to replicate it in the future. This blog post was written by Vladimir Valouch and Zdenek Papez.
The story of our challenges
We have an app that measures the performance of scheduled tasks on various mobile devices. We need to be able to run our app on as many physical devices as possible and at any time. The app needs to be able to login to our system and interact with it.
Due to the requirement for physical devices, we sought a platform with a good variety of physical devices and a way to run the app. Fortunately, the search was simple. Many similar services exist, such as AWS Device Farm, Google Firebase Labs, Azure DevLabs, Sauce Labs, and more. Another option was to build our mobile farm, but that would avert our attention from our primary goal. AWS Device Farm is a service providing managed access to many physical mobile devices and web browsers for testing. The essential part for us is that there is no need to maintain those devices or browsers. It is straightforward to use, it has devices that we need, and we are already using other services in AWS; therefore, AWS Device Farm was our final choice.
Another challenge of ours was determining how to pass credentials to mobile devices that are running in the cloud. Simple solutions like using environment variables in a Docker container are not applicable as mobile devices cannot run Docker containers. Furthermore, we wanted to avoid compiling the credentials directly into the app. Thus we searched for a solution how to pass values from the command line when starting the test.
The app preparation
If your app needs access to different user credentials, test specifications, test scale filters, or other external parameters, you can provide them by implementing a custom AndroidJUnitRunner.
Your custom testInstrumentationRunner must be configured in the build.gradle file for the given Gradle module:
android{
defaultConfig{
testInstrumentationRunner “my.pkg.ParametrizedTestRunner”
}
}Then you can provide those parameters in your adb shell am instrument command as described in Using the am instrument command:
adb shell am instrument -w -e USERNAME testUser -e PASSWORD pass123 my.pkg.test/my.pkg.ParametrizedTestRunnerBefore that, you should check that your instrumentation package exists on your device via: adb shell pm list instrumentation
If your package is not listed, install it using gradle task:
$ gradle :{$project}:installDebugAndroidTestBuild APK files for AWS
The following command builds the needed APK files for AWS Device Farm.
#Run the command in the base dir of your project
./gradlew clean build cCIt should generate for you your-app/build/outputs/apk/ folder, which contains both *-debug.apk and *-androidTest.apk.
The project creation in AWS Device Farm
This section describes how to create a project in AWS Device Farm and configure it for the run. We use a browser to manually complete the setup below, but I strongly encourage you to check the Boto3 documentation for Device Farm for a way to automate what I am going to show.
Firstly login to AWS in your browser, then go to the “Device Farm” section and click on “Create a new project”. The next step is for providing “Project Name”, so fill in anything you prefer and click “Create project”.
I would suggest going to “Project Settings” and optimizing the “Execution Timeout” after the project creation. I believe that for initial testing, the value of 5 minutes should be more than enough. I recommend creating something like “Tiny Device Pool” for the initial test setup.
The run configuration
Here we set up the run on the device itself, and the result of the setup is scheduled to run against the device pool of your choice.
Go to your project and click “Run”. In the next step, upload the app APK that we created in “Build APK files for AWS” and click next to get to the “Configure” step. To run parameterized instrumentation in Device Farm, you need to choose Run your test in a custom environment during the “Configure your test” step.
The selection of the custom test environment allows you to adjust the adb shell am instrument command and provide your custom -e parameters and runner class, as described above.
“Service name too long” problem
If your parameters are too long, you could encounter “Service name too long” error. That is due to a limitation that adb shell accepts only input text up to 1024 bytes.
The solution is to save your long command into a file, push it to the device, and then run it using adb shell sh.
See the complete TestSpec yaml that is doing this trick in Device Farm:
The final words
If you find anything missing in the article or have a question about more details, please leave a comment. This blog post is a filter and sorted collection of information that we found in various discussion forums, official documentation, and our own experience. I want to say a big thanks to Chris Ismael, Jakub Snor, Robert Reed and Zdenek Papez, for their help with the article’s review.
