The Night’s Watch: Safeguarding store operations
In Stocks, the Android application that our beloved colleagues in the stores use, there are different features (receptions, breakages, squares, donations…). Each of these features has different flows to be executed, which are the core of the business (Receive a container, add a breakage, square a product, donate a product…).
Many of these flows occur at night. What if we introduce a bug in one of these flows, and our colleagues can’t do their work? That would be a problem. Fortunately, we have a strong culture of using feature flags to disable a change easily. We also have a rollback mechanism just in case of total disaster, but even that will slow down the pace of the stores’ workers as it implies a technical intervention.
Their time is precious at any time, but especially before opening the stores, when they replenish the shelves, clean the aisles, check the expiry dates… In summary, they are getting ready the store for our customers. Any disruption to their work has an impact that must be avoided.
The best case scenario is to have these main flows tested to ensure that we, as engineers, aren’t challenging the stores’ workers’ routine work.
Some months ago, my teammate Luis Cencillo Abad explained how and why we started using Maestro to write UI tests in Android at Mercadona Tech. In his fantastic post, which I strongly recommend, he commented that we used a workaround to mock the backend responses.
In this article, we’re going deep into how to do it in a simple way.
A little bit of context
About what we had
We were already testing the app’s main flows using Maestro tests. These tests ran against our staging environment, where we had the last backend version. So, we had an e2e test suite to check the main flows were working correctly. We should have been happy with it, right? Well, we weren’t.
About the problems
This approach leads to some critical problems:
- Flakiness: The server could be momentarily down when you run the test, or you could experience network problems. The backend version may be bugged, or perhaps a third-party service is broken.
- Speed: Network latency or slow request processing can throw unexpected timeout errors.
- Non-deterministic: In our case, we didn’t have fixtures or fake data for certain endpoints so, as you can imagine, testing some features was quite challenging.
- Difficult test cases: How about empty views or error dialogs? In some cases, they are key to making the user understand what is failing and how to continue with their work. This was impossible to test.
We also noticed that we were using the e2e tests for two different purposes:
- UI testing: Covering the core flows in the application. Performing interactions from the user’s point of view and asserting the result, navigation, and so on.
- Contract testing: As they were real network calls, if the contract between the backend and the app was broken, we’d have detected it in CI/CD, avoiding a crash in production. (If that endpoint was being used in a tested flow, of course).
If you want more information about how we ensured the contracts were fulfilled, we covered it in the Contract Tests: A New Hope post.
Let’s talk about UI testing then, but with solving some problems along the way.
Mocking the network layer
As we said, we were already covered from breaking the contracts, so we no longer need to test against a real backend environment.
First things first, Maestro doesn’t support mocking (if you look for it, you’ll find out that, in the past, Maestro had something experimental, but sadly, it was removed), so we started the research. We wanted something lightweight and easy to manage, and we finally discovered https://www.mocks-server.org/. It was exactly what we wanted.
I’ve prepared the https://github.com/edgarmiro/maestro-tests-with-mocks repository with all the code and a full example. Read the README.md file to run the example. Also, I think you’ll find the maestro and mocks folders quite useful. 😉
I’m going to explain the key parts in this article, though.
Mocks Server
The setup process is quite simple. You can literally have the services running in a few seconds. In our case, we use Docker Compose to start the service. You can find it in the mocks folder:
version: "2"
services:
mocksserver:
image: mocksserver/main
container_name: mocksserver
ports:
- 3100:3100
- 3110:3110
environment:
- MOCKS_LOG=debug
volumes:
- ./src/:/inputI don’t want to extend too much because the documentation is amazing and much better than mine, but, in the src folder, we store some files for the setup:
- collections.json: This is a set of preloaded routes. In the example, you’ll see a list with products or an empty one. There’s collection inheritance, too, so if you have common routes, just add them to the base collection.
[
{
"id": "base",
"routes": [
]
},
{
"id": "with-products",
"from": "base",
"routes": [
"get-products:success"
]
},
{
"id": "without-products",
"from": "base",
"routes": [
"get-products:empty"
]
}
]- routes/products.json: Here, we define the routes (endpoints) and the possible responses we want to mock (variants). For instance, you can see the route-variant get-product:success-1, which returns the data for the product with id 1 or get-product:error, which returns a 400 HTTP error (So useful to test our error scenarios 😉).
[
{
"id": "get-product",
"url": "/api/products/:id/",
"method": "GET",
"variants": [
{
"id": "success-1",
"type": "json",
"delay": 1000,
"options": {
"status": 200,
"body": {
"id": "1",
"name": "Iceberg lettuce",
"image_url": "https://prod-mercadona.imgix.net/images/e047afab489d1c6a9ffc9e7da122ce3f.jpg?fit=crop&h=600&w=600",
"pvp": 0.99
}
}
},
{
"id": "success-2",
"type": "json",
"delay": 500,
"options": {
"status": 200,
"body": {
"id": "2",
"name": "Hacendado Classic Chickpea Hummus Recipe",
"image_url": "https://prod-mercadona.imgix.net/images/321f415fc3e6c039dd1dfa069e9020ef.jpg?fit=crop&h=600&w=600",
"pvp": 1.05
}
}
},
{
"id": "error",
"type": "json",
"options": {
"status": 400,
"body": {}
}
}
]
},
{
"id": "get-products",
"url": "/api/products/",
"method": "GET",
"variants": [
{
"id": "success",
"type": "json",
"delay": 1000,
"options": {
"status": 200,
"body": [
{
"id": "1",
"name": "Iceberg lettuce",
"image_url": "https://prod-mercadona.imgix.net/images/e047afab489d1c6a9ffc9e7da122ce3f.jpg?fit=crop&h=600&w=600",
"pvp": 0.99
},
{
"id": "2",
"name": "Hacendado Classic Chickpea Hummus Recipe",
"image_url": "https://prod-mercadona.imgix.net/images/321f415fc3e6c039dd1dfa069e9020ef.jpg?fit=crop&h=600&w=600",
"pvp": 1.05
}
]
}
},
{
"id": "empty",
"type": "json",
"options": {
"status": 200,
"body": []
}
},
{
"id": "error",
"type": "json",
"options": {
"status": 400,
"body": {}
}
}
]
}
]So, we have the mocking service configured; how can we use the route variants in the tests? Mocks Server has an REST API that interacts with the service while running. This is useful for changing the endpoint responses on the fly. Maestro allows Javascript code execution, so we’ll use it to communicate with our mocking service.
Let’s do it in a test!
Maestro
Here, you can observe the Maestro test, which opens the app and navigates to the product detail screen:
appId: com.example.uitesting.maestro
onFlowStart:
- runScript: mocks/api.js
- evalScript: ${ output.restoreRouteVariants() }
- evalScript: ${ output.selectCollection('with-products') }
---
- launchApp
- assertVisible: "Loading…"
- assertVisible: "Product list"
- evalScript: ${ output.selectRouteVariant('get-product:success-1') }
- tapOn: "Iceberg lettuce"
- assertVisible: "Product detail"
- assertVisible: "Loading…"
- assertVisible: "Iceberg lettuce"
- assertVisible: "0.99 €"As you can see, there’s some previous setup (onFlowStart block). What we’re doing here is:
- Run the api.js script: This script contains three functions that interact with the Mocks Server through the REST API. It’s just the import. As we can’t call the functions directly, we use the output object to communicate between the test and the functions inside this endpoint.
- output.restoreRouteVariants(): We want to start the test from a clean state. We don’t want previously loaded routes, so we restore the environment.
- output.selectCollection(‘with-products’): As we saw in the collections file, this collection loads the get-products:success route variant. This way we don’t have to select it explicitly.
- output.selectRouteVariant(‘get-product:success-1’): We’re asserting the happy case so, we want the data of the product with id 1. We can run the evalscript command whenever we want inside the test to change the responses.
And that’s it. As you can see, we can even assert that the loading state is visible. But how is that possible? Aren’t we mocking the network layer? Shouldn’t it be this immediate? The answer is yes, of course it should. And it is immediate. But I wanted to demonstrate that we can even add a delay to an endpoint just to check this kind of thing.
If you recall the products.json, you’ll see a property named delay just for that purpose.
Demo time!
Try it!
And that’s all folks. I hope this post helps you to test the main flows in your applications with this easy and readable approach in the same way it does for us.
I can’t wait to read your comments about our approach and to know more about your use case. Do you have critical flows? How are you currently testing them? Are you protected against breaking them?
Have a nice day!
