Cross-platform end-to-end tests on mobile application with Detox

Mickaël Morier
Apr 2 · 6 min read
Illustration, a humanoid robot made test on a smartphone
Illustration, a humanoid robot made test on a smartphone

At Doctolib, we have more than 25,000 automated tests of which a large part are end-to-end ones. Our mobile applications are mainly a webview but they are developed with React Native. However, the patient application also has some native screens for the video consultation and some features that require native code.

Since the redesign of mobile applications with React Native in February 2018, we have added Jest unit tests for some utilities, but nothing to actually test native features without manually testing the applications. We strongly believe in the benefits of end-to-end testing at Doctolib, so there has always been a desire to have end-to-end tests on mobile applications, but technically it is not as simple and well-equipped as it is for the Web.

Which testing framework?

A first attempt was made in April 2019 with the Appium but it was a failure. It was too complicated and too limited to set up locally as on our mobile Continuous Integration (CI). So we gave up as soon as the Proof-of-Concept was done.

A year went by and mobile end-to-end testing became a priority for the patient application. Detox seemed like a good candidate and was gaining popularity. It looked easy to write tests and to configure and implement it on a CI pipeline. Therefore I did some quick testing and was rapidly convinced. In a very short time I was able to write my first tests and have them working on the iOS simulator.

Another cool thing about Detox is we can easily launch our application from a push notification or configured with specific permissions (camera, location, etc.). This is a very useful feature in Doctolib cases, for example our telehealth diagnostic screen has different behavior according to camera or microphone permissions.

How to test the application?

The application is mainly a webview but it interacts with native features through events sent with Window.postMessage. As it is impossible to interact with its content, we created a React Native test screen to test the native features directly. This screen only exists in the staging and development builds.

Test screen used by Detox to test native features of the patient application
Test screen used by Detox to test native features of the patient application
Test screen used by Detox to test native features of the patient application

This basic screen is directly available by a button from the home screen of the application. Each button on the test screen triggers a native feature. These are used by Detox to run automated tests but it is also a convenient way for developers and product managers to do manual tests on real devices. Let’s see some examples.

Testing native screens and OS permissions

The video consultation on the mobile application is totally implemented with native code. So with Detox, it is easy to navigate through different screens and check the behavior with different permissions given to the application by the OS. As you can see in the following video, we navigate through a flow of diagnostic screens to check whether your phone is capable of doing a video consultation. This flow is tested with different permissions for camera and microphone.

Execution of Detox telehealth diagnostic tests on iOS simulator

Let’s look at the code of the first test executed in the video. It is pretty simple to write and to understand. It is just a sequence of unit commands to launch the application with specific permissions, to click on buttons and to assert whether certain text is visible or not.

The two first Detox tests on telehealth diagnostic screens

Detox is a cross-platform testing framework, so tests are executed on Android and iOS with the same code. But sometimes Detox does not have the same support on both platforms, or Android emulator and iOS simulator do not have the same capabilities. In that case you have to adapt your test with a condition on the OS.

The first test shows the difference between the capabilities of the simulators. The iOS Simulator does not have a battery state, so we check a message for low battery. On the Android emulator, we have the information so we just check that the message is not present.

The second test is not executed on Android due to a limitation of Detox on this platform. For the moment, it is not possible to change the battery permission on Android.

Testing native features called from the webview

As previously mentioned, several native features are launched from the webview containing the Doctolib website via postMessage. To test these features without piloting the content of the webview, each test button creates a blank webview that injects JavaScript to post the same message as a real message from the Doctolib website. So our webview component reacts to this message and calls the corresponding native feature like in the real application. If you watch the following video, you will see the webview created for each test below the buttons.

Execution of Detox native bridge tests on iOS simulator

Let’s look at the code of some tests executed in the video. Like the previous test code, it taps on buttons and checks if elements are present by testing their text or their id.

Some Detox tests on native features called from the webview

Here again we have some OS differences, so tests contain OS-specific code. Adding a calendar event on Android will open the Calendar application that is why we go back and check if we are returning to the test screen. On iOS it shows a modal over the screen, so we check if the test screen is not visible anymore. This is the only thing we can do because Detox can only pilot your application, not anything outside of it.

If you look at the last test, you will see we can assert text and tapping on buttons of the iOS sharing modal. This is a little hacky because we use the private UI class but it is possible.

Continuous Integration with Bitrise

As you can see in the previous videos, Detox tests are launched by simple shell commands. As a consequence, it was pretty easy to plug it to Bitrise, our mobile CI. For each commit in a pull request or on master and production branches, tests are launched and applications are built. Detox tests are executed for Android and iOS patient apps. They are very stable and fast on the iOS Simulator as shown by the following screenshot.

Detox end-to-end tests results on Bitrise

On the other hand, Detox tests are flaky on Android build due to the Bitrise Android emulator. They are stable when we run them on our respective machines but on Bitrise the emulator is not performing and reactive as well. Despite some adjustments on Bitrise, it happens regularly that a test fails because the emulator does not react to the tests.

This summer, I made an attempt with a Genymotion Cloud emulator instead of the Bitrise one. It was stable but some tests failed because of the physical back button used in some tests which were not understood by Genymotion. Genymotion confirmed to me very recently that there was no problem with the latest version of Detox. So we’ll have to follow up.

What’s next?

These Detox tests are really a first step for Doctolib and many things can be improved. As I mentioned before, the Android emulator of Bitrise is not stable so the test suite is flaky. It might be possible to use Genymotion cloud to fix this. We also can imagine executing all these end-to-end tests on real devices.

For the moment, we only have Detox tests on the patient mobile application. The doctor application has some features which use native code and it will continue to grow, so having Detox tests for this app should improve non-regression and simplify upgrades.

And you, what is your experience about end-to-end tests on mobile applications?

Doctolib

Improving Healthcare for Good

Mickaël Morier

Written by

Full Stack Engineer @Doctolib

Doctolib

Doctolib

Founded in 2013, Doctolib is the fastest growing e-health service in Europe. We provide healthcare professionals with services to improve the efficiency of their organization, transform their patients’ experience, and strengthen cooperation with other practitioners. We help pati

Mickaël Morier

Written by

Full Stack Engineer @Doctolib

Doctolib

Doctolib

Founded in 2013, Doctolib is the fastest growing e-health service in Europe. We provide healthcare professionals with services to improve the efficiency of their organization, transform their patients’ experience, and strengthen cooperation with other practitioners. We help pati

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store