Rejoice, Android developers, we present you Virtual Device Testing for Android! 🎉
We started development as we received several reports of timed out emulator boots, unreliable test results with fake fails and hanging tests. Plus, earlier you had to run device tests one by one, so testing could be quite tiresome. 😫
Enter Virtual Device Testing for Android! From now on you can run Android emulators reliably and get conclusive test results presented in logs, video and screenshots on Bitrise. You can run tests parallelly on multiple devices which is much faster, so we’ll spare you some time to do some more programming instead! (Or watching an episode of Silicon Valley. You decide.)
After testing with the Android Emulator you only got one html file heralding success or announcing failure and the test took about 20–30 minutes. From now on testing will take significantly less time, and in addition, the results will be more informative and comprehensive. Beside the usual html file, the result contains a video screen capture of the whole test, screenshots, an activity map, raw and stylized logs per device to overload you with information. It’s quicker and easier to debug as you’ll have detailed reports and you’ll see what causes the fail and when and why (?!!!)
Use the shiny new features
Activate
You can activate this separately for each app on the app’s Settings tab:
Add the step
Then you’ll need to add the step called Virtual Device Testing for Android to the workflow where you want to run your emulator tests (that is all your testing workflows). We support both Instrumentation tests and Robo tests, the latter is the default.
Build & Test
If you add the step and start a build, you’ll find a new tab on the Build page of the app, next to Logs and Apps & Artifacts.
See the awesome test results overview…
When the build in concluded, you’ll see the list of devices you ran your tests on, this is basically the Dashboard for your tests.
… and the details per device
If you click on a device, you’ll see the test results, the status of individual test cases, a video of the whole test, screenshots, an activity map, stylized (easy to read) logs, and a list of all the files generated.
Or download all the test result files
At the bottom of the test results, there are all the files that were generated during the test for you to download:
The tests run on Firebase Test Lab and we display the results we get from them. You have unlimited device minutes for free until this feature is in beta.
Robo or Instrumentation?
We support both Instrumentation tests and Robo tests, but it is only possible to have one of them in a workflow, so if you want both, you’ll need set up two workflows.
Robo
Robo is the default, so you’ll only need to add the Virtual Device Testing for Android step after the Gradle Runner step and this is it.
Instrumentation
For Instrumentation tests, you’ll need change your primary workflow:
- Add an extra task assembleDebugAndroidTest in your gradle-runner’s gradle task input, so it will be something like:
assembleDebug assembleDebugAndroidTest. - Add Virtual Device testing for Android step after your gradle-runner step.
As a minimum, in Virtual Device testing for Android step set Test APK pathinputs to your built debug apk (e.g.:./app/build/outputs/apk/app-debug-androidTest-unaligned.apk) Make sure to configure only one section of the step, which matches your desired test type!
(APK path is exported by the gradle-runner step, and it is set as the default for the step's input, so if you want to use a different APK then don't forget to change the step's input accordingly!)
We hope you’ll like this as much as we do! Try it, test it, love it and you know, we loooove feedback, so ping us and share the love! 💖
Originally published on Bitrise Blog.
