UI Testing on iOS with KIF and Fastlane.
This article is a part of How to test User Interface on iOS and Android created to present the approach to Quality Assurance process in projects run by AppUnite company.
Goals that were our motivation to create new testing tools.
- Create an architecture around UI tests, coherent for both iOS and Android platforms.
- Test the application according to the specified QA test scenarios, based on the same pre-prepared data from the server.
- Present the results on a webpage that is public.
We should keep in mind that KIF is not a framework that runs target UI Tests, but Unit Tests. Test Run looks the same, however there is no possibility to access XCUITests classes within test code.
The main goal during my work with UI Tests was to create an architecture, that will allow our QA to modify and create new tests. We decided to prepare simple structures that will describe functionalities for single screen as well as screen groups.
I find Krzysztof’s Kempa article Generic approach to iOS UI Tests in Swift useful in this case. He presents a concept of creating structures called “Page”, which not directly, but does resemble a single screen of the application. Each Page exposes a function that allows to move to different application screens. This concept was in line with my assumptions so I decided to adopt it.
Our current code is based on a Protocol that remaining Page’s implement. Due to the application characteristic I decided to create protocols including functionalities that can be used upon groups of screens (i.e. playing a song is a part of most of the screens and it’s being called by the same interaction each time). So, currently each and every screen that user can access has it`s own structural equivalent. During future improvements of the application it’s important part of developers work to provide description for each screen. Other responsibilities revolve around preparing screens in a way that they include necessary accessibility identifiers and labels.
Here is an example of the structure for home screen:
Thanks to those structures it’s much easier and faster to create new tests. It gives an easy way for the QA to create new tests without looking into code behind screens, controllers and storyboards.
Thanks to autocompletion in Xcode our QA team is able to see what they can test on each screen.
Interacting with objects.
Next thing to consider is the possibility to mock the server responses. As we would like to isolate the server from the application during tests, we decided to mock every server response. For networking we use Moya framework that makes server response mocking a walk in the park. To do that we created a repository with recorded server responses. Later on the repository was added to the iOS and Android project. It allowed us to test both applications on exactly the same data
Our application has many screens where objects can be modifed, and it forced us to create something to resemble database exclusively for tests.
We decided to load earlier prepared json files on start of each test and then each modification was noted in temporary database (which is actually memory cache) while the test was running. It gives us the possibility of testing object editing on screen A and then observing the change on screen.
Running tests on CI servers.
Since we are using fastlane in whole CI process, we decided to run tests with additional lane using scan action. This action generates webpage as a result, which fits into our initial assumptions perfectly. Unfortunately, if we look into the data that is being passed to the results it turns out its not enough. Due to the fact that we want to present a detailed step-by-step passage of the tests, as well as receive information about any probable failure, we decided to generate our own file. In order to do that we extended the base test class, so that it can gather data about steps and failures in individual tests. Then the gathered data is printed as JSON in simulator console. Last step is to intercept simulator logs and create html file.
We can use custom fastlane actions to do this — thanks to scan we are capable of accessing simulator logs and parse them to create results in json file. This file will serve as a data source for results webpage. (in following part I will explain how to create a webpage with such data.)
There is an example how to configure fastlane and gitlab pages to work together.
Our CI is based on GitLab platform. We have couple of mac servers in our office we use for our projects and tests. The most efficient solution for presenting test results seems to be the GitLab Pages (our assumption was for the results to be public). After server configuration the domain is launched, and we can now open every file that will be placed in “public” catalog as Job result. We used Mustache framework to present our json results as a webpage. After using output from our tests it looks like this:
Main advantage of architecture and solutions we prepared was lack of modifications in application code base. The only changes we had to do was to make sure that key screens have proper identification numbers and labels. One more advantage was introducing the possibility to write tests for non-technical members of our team, because the developers ensure description for screens and their functionalities, so QA team can focus on test scenarios.
Currently we are working on creating an application that will gather tests history per build. Thanks to the fact that results are based on json format, we are able to easily import them to newly built web application.
Our plan is also to allow screen capture on each step of the test. It will guarantee that the problems with layouts will be intercepted even if tests fail to do so, as well as allow to compare functionalities between two platforms.
If you would like to know how it looks on Android check this article.
If you like my work hit ❤ button and let me know what you think in comments!