Same test cases for Android and iOS automation — pros and cons

If your app looks similar on both the platforms it is tempting to desire an automation suite which has same set of E2E tests. Especially if you are using a tool like appium which has a unified protocol for communication to both iOS and Android.Architecturally it would look something like this

Benefits of this approach

1. Test cases are more visible and high level, especially if you are using BDD correctly.
2. Would be much easier to hook up CI system, test runs from a single repo
3. Management would be able to see test results and coverage in the same report

But, having created 2 such frameworks in Ruby/Cucumber, I am currently leaning over to the other side based on the following problems.

1. Both platforms are not really uniform, no matter how similar your app looks they are based on two different code bases and operate on different OSes. Back button handling, settings navigation, touched etc eg: Notifications and alerts are handled pretty differently on both platforms
2. UI tree in iOS includes all the laid out elements, even if they are not in view (needs scroll), on Android UI tree contains only visible elements, so one has scroll mechanically and then find elements.
3. Having 100% feature parity isn’t really achievable with different teams, working on their schedules on deadlines. Hence automation has to keep that in mind and have conditionals/tags for those. For instance, logout may need an additional confim dialog in one platform but not the other.
4. For adding a simple test one has to write initial test cases, middle layers, helper functions and that starts to take toll, especially in v2 and later on where platforms/features diverge much further.
5. Even if we have proper interfaces and architecture, avoiding conditional logic is difficult making the code less maintainable.
6. Things get pretty complicated once we introduce tablets, landscape and portrait modes. eg; text entry in android landscape mode is quite different

What are your thoughts?

Originally published at mobiletest.engineer on January 8, 2016.