Mobile Automated UI tests history
Mobile apps development is extremely dynamic today. Evolution in electronics and communication technologies is very fast and concurrence between device constructors ans operating systems is very tough. Mobile apps increase in number and complexity, we have a lot of new OS versions to improve existing features and add new ones. That implies another sector to be at least as dynamic : test and quality tools adapted for mobile apps. We will review a short history of automated functional test tools for mobile apps in this article.
The first steps
Before the iPhone/Android generation, people could enjoy mobile devices with java mobile (J2ME) virtual machine. There were some big differences between devices and sometimes some bugs in JVM implementation. So the solutions were to invest in some portability tools, a big part of the testing job was mainly reported there, but it implied to be dependant of one commercial solution and a lot of limitations. With a VM over another VM, it was impossible to exploit the device capacities at its best. For sophisticated applications like games, usually, a adaptation was done with development and manual test phases alternatively for every new device in the market, that was very expensive.
Then, with the first iOs and Android devices, the need of testing and quality was important and most of it was done manually at the beginning. Some UI automation tools appeared quickly as libraries we could include in test projects; for example, first versions of Robotium for android or Iosdriver. The users of that tools had to look on the source code to be able to write UI tests, apk or ipa had to be adapted most of the time, so independency between testers and developers was not possible and there were some random connexion or material problems. These reasons, and others, made really difficult to make automated UI tests more profitable than manual tests.
Modern tools
In 2013/2014 mobile UI automation open sources projects were developed with the possibility to access element IDs directly without looking at the source code.
Follow the links to access to 2013 Selendroid and IosDriver github with inspector in beta versions.
Those tools aimed to do what Selenium was doing for web sites, the first stable versions were quite acceptable for basic sanity tests. At this point, ideas to make UI automated tests more efficient were to make it easier to write. One of the way was to make possible to describe the test in “natural” language (gherkin) so BDD (behavior driven development) is used. Jbehave and Cucumber are two ways to use BDD.
Calaba.sh interacts with cucumber and is still used for mobile UI automation today.
In fact, if the automated test is written by a dev, BDD does not make the tests to be implemented faster. If the test is written by a PO, it implies a very high level of maturity in specification and development process to work. Anyway, BDD can be use to make the test report easier to read. Another idea to speed up test implementation is record and replay tools. Basically, you start the record, do the test, the tools generates some code you just have to copy/paste. Even if it looks interesting, this generates a code hard to maintain, some portability problems (different screen size, etc), and asserts (what we have to check to decide if the test has failed or succeed) are not presents in generated code.
So, in order to obtain a mobile UI test code as maintainable as possible, the page object model from Selenium to automate test for web page has been adapted, it has also the advantage to be useful for another challenge : make cross platform tests. As we have a factory pattern for the different pages of the app, we can have a factory for android and Ios drivers as we have for chrome, firefox, safari, etc in Selenium.
Now, we can say we have some stable cross platform mobile UI automation tools with inspection and replay modules, with the will to let test automators free to use BDD or not and to choose the language to write the tests, and plug-ins for continuous integration. Appium and Calaba.sh are the most widespread concurrents (2014 best open source application development tool Bossie Award for Appium). We can still find some problem on these tools especially with new versions of OS, but developers are very reactive. There are also some automation tools specialized for a platform or a language that can be useful for specific projects and organizations. On our side, we often have to develop in agile mode in parallel for Android and iOS, so we develop UI smoke tests and some feature tests depending the size of the project in Java using Appium with nightly builds with Jenkins on real devices.
Cloud testing services
Even if parallel testing is possible with current automation tools, it is not possible for a dev team to have access to all existing devices most of the time, this can be a problem for debugging. So we have more and more companies proposing cloud testing services, that means remote access to a large list of real devices and emulators. Confidentiality issues are possible with this system if you cannot install applications in development in a non private cloud, and location can be a problem if the device is abroad. Competition is really hard in this new market. Usually companies creates their own quality tools and/or add overlays to open sources existing ones, new modules for appium, jenkins plugins, promote themselves by doing tutorials, etc. This can be an added value if we understand that if UI automated tests are not efficient with open source tools, it will be the same with commercial tools. A more radical solution can be the outsourcing of the entire test process (Taas model) for applications with reduced changes in specifications and high quality requirements. It gives independency between dev and QA but not always smooth communication.
Bottom line
The trend in UI automation testing in general and especially for mobile apps is to have specialists, for the value added to the product. More and more certifications are wanted for the ISTQB test automation engineer advanced level even if it is new and still not available in many countries. For any dev team, the question is not “do we need to automate UI tests”, but how, depending on our quality expectations, our experience level, and maturity of our processes.
Do you want to know more about Wassa?
Wassa is an innovative digital agency expert in Indoor Location and Computer Vision. Whether you are looking to help your customers to find their way in a building, enhance the user experience of your products, collect data about your customers or analyze the human traffic and behavior in a location, our Innovation Lab brings scientific expertise to design the most adapted solution to your goals.
Find us on:
