Why You Should Automate Your Tests
The Nudge Rewards tech stack includes a mobile app and a Rails backend. The server team is responsible for maintaining the backend and API that is used by the mobile app. One of the challenges that the server team faces is having to test that the mobile application is working as intended on every deploy of the backend codebase. Lo and behold, Amazon’s Device Farm is up to that challenge!
Device Farm is part of the AWS package. It has the ability to run UI automated acceptance tests against an executable Android or iOS file. Automated UI tests can be thought of as a script that clicks through parts of the application and verifies that the right elements show up.
Our solution involves having a Travis postscript that automatically deploys to alpha environment and then runs the UI tests against the mobile app which checks that everything is in order. Amazon has a set of built-in fuzz tests that will randomly click through your app. But, if you want a little more specificity, Device Farm supports Java JUnit, Python Appium, Calabash and UI Automator for writing custom tests.
When we first tried to implement this, we started by writing the tests using Appium. Getting Appium working in the first place proved challenging. It seems like some of the system libraries used by Appium have not been updated in some time. Finally, once it was up and running, it worked well with the Android APK but we had issues using its UI Inspector tool with our iOS app. So, we ditched Appium and decided to try Calabash instead.
Calabash was super easy to setup, and it uses the Cucumber framework for specifying tests, which are easy-to-read. Further, Calabash provides a ruby shell console that allows for easy exploration of the UI elements.
Using Amazon’s AWS API is pretty straightforward and is comprehensively documented. Instead of using vanilla HTTP requests, we decided to use the AWS SDK for making API calls, which conveniently has a ruby gem called ‘aws-sdk’, allowing integration into Rails applications.
If you’re considering automated UI acceptance testing, it might be worth looking into Device Farm/Calabash! Depending on your application, your tests may differ. Our tests walk through the core features of the application like logging in, clicking ‘Forgot Password?’ and referring other users. One thing to keep in mind is that this is simply a way to test your UI more than anything. For example, if you are testing a ‘Forgot Password?’feature, Calabash will not help you test that an email with instructions for resetting the password was sent.
The end result of this exercise? Considerable time savings for the server team from having to manually test the mobile application!
Originally published at dev.nudgerewards.com on December 9, 2016 by Parth Naik
