Writing End-to-End Tests for Bluetooth Applications is Difficult
Toward the beginning of my career, I was tasked with creating an Android application that would communicate with a device inside of a vehicle over Bluetooth (BLE specifically). There were many lessons learned, many hours of frustration, but overall, it was a great experience. One thing that stuck out to me throughout the entire process, however, was: “how the heck are you supposed to write tests for something like this?”
Flashing forward to my time with Beam: writing great tests for our code is something we strive for within every team and application. The mobile team that I am a part of is no different. Our app, which just so happens to communicate with our smart toothbrush via Bluetooth, is what finally drove me to figure out just how to test code around Bluetooth communications. We have a lot of logic within the app to respond to events, retrieve certain information, and even issue commands back to the brush. Our testing philosophy very much revolves around the idea that tests should mimic the way our code actually works as closely as possible, so, figuring out a reliable way to recreate the way that Bluetooth operates was critical.
Bluetooth at a High Level
Communicating with devices over Bluetooth is a fairly straight forward process. At a very high-level: one device in the communication, called a peripheral, must be advertising specific pieces of data. The other device, called a central, must then be looking for these advertisements in order to detect the device sending out data. For Beam, the peripheral is the Bluetooth enabled toothbrush, while the central is a smartphone.
Once the central finds the peripheral it is searching for, a connection is then made between the two. The central requests specific services from the peripheral. Services act as a way to group pieces of data to be read from. Inside these services, are characteristics. Characteristics are where specific pieces of data can be found. For example, one such service + characteristic pair from our app would be theBrushService and BrushColorCharacteristic . Our app connects to the BrushService and then it can request information from the BrushColorCharacteristic , to which the brush responds with its color.
There’s a lot more nitty-gritty detail within Bluetooth communications, but the important takeaway is that it’s a near-constant two-way communication between two devices, with the possibility of certain pieces of data being a one-way communication.
The Difficulty of Testing Bluetooth
Testing code that interacts with outside libraries usually follows the general concept of “mock out the interfaces to control when and how said libraries will respond to your code”. The problem with this concept as it relates to Bluetooth communications lies in the fact that the peripheral decides when to update the devices and with what data. Essentially, the problem with testing Bluetooth communications is the necessity to mimic how the peripheral would communicate back to your app.
The biggest piece we had to figure out as a team was how to replicate the flow of:
- App listening for a specific piece of data to come through
- That piece of data coming through in a way that we expect at a time that we can control from the test (ideally without the use of any kind of
sleepor waiting)
Another large hurdle we had to overcome was how to even set up our code to facilitate what is essentially a “fake” Bluetooth layer. We wanted our tests to be able to run anywhere (including simulators) so we needed a way to abstract out the use of the device’s Bluetooth’s APIs completely.
How to Run the Tests
Our mobile app is built with React-Native. For those unfamiliar, react-native is a cross-platform framework for mobile devices that allows developers to write JavaScript that will run on iOS + Android devices natively. One of the caveats to react-native, however, is that when using platform-specific APIs (such as Bluetooth), developers may be forced to write native code in addition to the JavaScript.
When first setting out to create our base for our end to end testing, we needed to find a framework that would work well with react native. With the requirements we had, Appium ended up being our strongest option. The team needed a testing framework that could run on real iOS + Android devices, not just simulators, which is something that Appium supports.
Once Appium is properly configured, designing tests happens fairly easily. There is a myriad of commands that Appium gives developers in regards to communicating with a mobile app. These commands range from tapping on a specific element on the screen to backgrounding the application to all sorts of other helpful functions. One of the specific commands that aided us in mocking out the Bluetooth communications is the ability to push a file to the device during a test (more on this later).
A Solution
The first place our team started was figuring out how to create a fake Bluetooth layer for our code to interact with. It’s easy enough to create an interface and load in the appropriate class based on the current use case. In our example here, that could look something like:
if (production) {
// load the real Bluetooth APIs
} else {
// load our fake Bluetooth layer
}We leverage the use of launch arguments to achieve the above. So if the app is launched with a specific key in its launch arguments (during a test, for example), it will know to load the fake Bluetooth layer. Otherwise, it uses the normal Bluetooth layer.
This was a good starting point, but, we still had a large gap to overcome with how to ensure that our fake Bluetooth layer functioned as close to the real one as possible. The team wanted to have the confidence that even though we were mocking out a large portion of our code, the important things within our app were being tested. We gain the most value from testing what our application does with the results of the Bluetooth communications. Testing Google’s Bluetooth APIs directly doesn’t give us many benefits.
The next step for our team was to create a class that implemented the same interface that our real Bluetooth manager class implements, much like we would with any normal test case. That way, when our app calls into the Bluetooth manager class, even if it’s using the fake one, it will have all of the same necessary properties and methods. This is getting into the crux of what we’re wanting to achieve with our tests: ensure that the data going into and coming out of the Bluetooth layer of code is handled as we would expect, and less about what happens to it inside the Bluetooth layer.
How to “fake” Bluetooth Updates
Another important piece in the Bluetooth communication puzzle is the concept of characteristic updates. Essentially, this means that a device can “subscribe” to a certain characteristic on a peripheral, and whenever the peripheral has new data for that characteristic, it sends out a broadcast, or update. If a device is listening for that update at that time, it can receive the new data from the peripheral.
In Beam’s case, one example of this concept can be seen with the BrushActiveCharacteristic . This characteristic is responsible for letting us know if the brush is currently on. The cool thing here is that the app can subscribe to this characteristic, and the brush will broadcast out to us whenever the Active state of the brush changes, so we know right away when the brush is changed to “on” or “off”.
As handy as this functionality is, it is also one of the pieces of communication that makes testing difficult. Our team had to find a way to reliably control when these updates would occur within the app, to ensure that we were handling them in a way that we expected. Enter: FileMonitor
We landed on the idea that we could use a file watcher type mechanism to watch a specific directory within our app. If a specific file is added to the said directory, we would treat it as if it were a Bluetooth update. As I mentioned previously, we are able to achieve this functionality with the help of Appium.
File Monitor
Creating a FileMonitor class opened the opportunity for our team to essentially have full control of a mechanism that allows Bluetooth updates with data in a format that our application would expect from a real peripheral. This FileMonitor class listens for a specific file to be added to the device during a test that contains data in a specific format. It then parses that data into the appropriate models in a similar manner to when the application is running normally.
An example from our app:
We have our “Nearby Brushes” screen, which displays the nearby Bluetooth brushes:
This screen is hooked into events coming from our Bluetooth layer, and whenever a brush is found, it is loaded into the list for a user to select. During our tests, without our File Monitor, there is no reliable way of making these “brush found” events occur.
Using a combination of our custom File Monitor along with our mockBrushes.json file (which contains a JSON object with an array of brush data), we are able to create some “fake” brushes for use during our tests. Our tests manually move over this mockBrushes.json file to the device, which triggers our FileMonitor to pick up the data from the file, parse it into the expected models, and send the event to our screen, just like in the real Bluetooth communication!
Making use of our File Monitor testing pattern allows us to recreate all of the different scenarios our application would see in the real world from within our testing environment. Things like: brushing events, Bluetooth disconnects, brushes turning on and off, are all now possible to capture within our tests in a way that is identical to how it would behave when communicating with the real Bluetooth APIs.
A Final Note on Testing
Anything and everything is testable. The amount of effort that goes into making something testable is the variable. Teams will always have to make trade-offs on what is and what isn’t important to test. With unlimited time, everything within a codebase should be tested. Obviously, none of us are so lucky to have unlimited time, so we must always make conscious decisions around where to spend our efforts with regards to testing.
Bluetooth communication is one of the core features of our mobile app. It is the thing that directly involves user interaction with both the brush and the app. It’s imperative to us as a team that all of the functionality around these features are as solid as they possibly can be, which means putting the time into figuring out how to test this code is well worth the effort. And since we designed our tests in a way that closely resembles how the app actually functions, we can be much more confident that the code we write is behaving the way we would expect.
