Implementing Detox End-to-End Testing
We dig into end-to-end testing in React Native with Detox, and why we made the switch.
Appium is one of the most popular native app end-to-end testing frameworks, and what we have historically used at CB Insights.
But since we use React Native as our app framework, we started running into performance issues that seriously impacted our usage of it. The issues culminated when we reached a threshold and there were too many UI elements rendered on our app for Appium to handle, resulting in hanging tests.
These issues caused us to start using Detox as our end-to-end mobile testing framework.
What is Detox?
Detox is a gray box end-to-end testing and automation library for mobile apps developed by Wix. We chose Detox because it is:
- Faster and more reliable: A pain point with Appium is the inconsistent communication between the test runner and simulator that could slow down testing. Detox’s gray box design is more efficient and faster since it can easily tell when the app is ready for its next command.
- Relatively OS agnostic
- Well-documented
- Easier for our React Native developers to write tests with: Tests are in JavaScript and run with Jest, which is the same as our unit tests
Updating the Jest Reporter
Detox’s native Jest reporter will print out the entire UI Hierarchy for each failed test. This hierarchy includes every element that was rendered within the app, which can really clog up the logs.
Useful stack trace, right?
To get around this, we created our own Jest reporter to get rid of the UI Hierarchy and to report the failed/passed tests in a manner similar to the native pytest reporter that we switched from.
How did we do this? By overwriting printTestFileFailureMessage to trim the UI hierarchy from the failure messages, and by overwriting onRunComplete to print out a short summary of the test results.
To add it to the reporters, we simply have to edit the Jest config json.
Different Entry File for Android App
While running our Android test suite, we saw the app encountered an error whenever it would try to close after a test passed. After some investigation, we found that Detox seems to have some issues with the react-native-gesture-handler package. Since we do not test this functionality directly in our end-to-end tests, we thought it would be best to try to disable this package while testing.
We solved this problem by creating a custom entry file strictly for testing the Android app, where we did not initialize the package. By passing the new entry file as an arg in our gradlew command, we built the app without the gesture handler erroring out.
Here is our build field in our android configuration:
Disabling Animations
We disabled animations in order to avoid any potential flakiness in our end-to-end tests. We have found that simulators/emulators can become very slow when trying to handle animations and that it’s best to just turn them off if possible.
Disabling React Native animations
React Native should mock animations when the app is built for testing, but we ran into issues where it was not mocking appropriately — so we manually disabled animations by using the sed command below:
Disabling Android Animations
We used the below adb commands to essentially disable animations for the Android emulator.
Accepting Notifications
We found that Detox has issues interacting with system alerts, so instead of having to accept push notifications within our app, we launched the app with this permission already set.
URL Blacklist
When relying on synchronicity, there were some logging API calls that were blocking the tests. Since time to complete is not a priority, we used a blacklist to tell Detox to ignore these calls as blockers.
Hopefully these tips that we used to set up our testing infrastructure are helpful. Check in soon for a future blog post where we will be covering some of the custom API we developed for our testing environment.
