Hardware Testing
Part 1. Integration testing
by Adriana Robles
Before we start talking about how to test a hardware, let’s remember the definition of Hardware. Hardware is a thing built by circuits, cables, plastic, metal and so on, that accomplish a function, like a camera is used to overwatch you home when you are away or a smartwatch that helps you to be connected to your phone main functionalities and track your exercise routine, just to mention a couple of hardware examples.
Now that you are thinking in all those devices that can be setup in your home to create an “smart home” and the endless possibilities to have them communicating between each other and have fabulous functionalities like run a smart routine (sequence of instructions) that arms your home, lock the door and turn on your cameras all done by just saying “Hey Google/Siri/Alexa … I’ll be back”, now imagine all the combinations that a testing team has to analyze and design to test all this and have all those devices working as expected at your home. This is just the beginning, in this article you will have a good base about how this testing is done.
What do you need to know?
As software testing has different levels (you might remember the V model from the ISTQB), Hardware testing also has levels.
As the title suggests I will be talking about the Integration testing level, per my experience in Hardware testing teams, the hardware level is done by the electrical engineers, same as the unit testing is done by the development team.
The first thing is to keep in mind that every time you are interacting with some Hardware by pressing a button, touching the screen or just talking to it by saying “Alexa …” / “Hey Google …” / “Siri …” your action is transformed to a request that calls a service, this service retrieves a response that can be interpreted by your hardware and performs the functionality you were requesting.
Some points to take into consideration when doing Hardware Integration testing (make calls to the services) are:
- Happy paths need to be tested, for example when you send the request to change color light, turn on/off lights, change temperature (smart thermostats), locking your door (smart locks), streaming your home camera and so on.
- Authorization testing, it’s important to test that the service retrieves an error when you do the request with invalid credentials.
- Error scenarios, you need to verify that malformed requests are correctly handled by an error message and code.
Just to clarify the difference between Hardware functional testing and Integration testing, when you do HW functional testing, you will be testing:
- Fiscal interaction with the device
- Validate the device does the functionality by seeing the light turns on or the door lock, for example.
- Login in to the Smart App with different credentials and verify you cannot see or interact with the smart device.
One more thing to highlight is that hardware testing is different from mobile testing, when you do mobile testing you are testing an application or a web site in it’s mobile version, but you are not doing hardware testing, you are just using hardware (cellphones) as a tool to test an application.
Get ready, you will be doing a lot of backend testing.
Let’s do Hardware integration testing!
As in all our testing processes, you need to identify and understand the business requirements, analyze them, and define the test cases. When you define these test cases, you will write them using as testing subject the backend which the Hardware calls, and write test cases to verify the functionality affects the Hardware (test cases where you interact with the device).
Let’s do an exercise; An Smart Light can perform 3 basic functionalities (to mention some), turn on/off, change light intensity and change light color. An example for the backend integration will be:
Taking the same example above a test case to verify this actually provoques something in the device will be:
Now you will be thinking something like “then I have to test the same twice, one calling directly the backend and another by interacting with the device”, the answer is no, you will not have to do same test twice, my recommendation is to do most of the test through the backend because is faster and if the backend responses are ok, you can be 70% sure, your device will do the functionalities, but that other 30% is a basic hardware interaction testing, where you need to make sure the functionality works. For example on the backend you can do malformed requests, send the request with invalid credentials, turn on the light twice and verify the second one sends an error, you can send an invalid color to be set, and in the front you can test the basics, that when you turn on/off the like it works, when you select a color it change to it.
Conclusion
Do hardware testing is different from doing mobile testing, when you do this kind of test be sure you are familiar with backend testing and that if you aim to start doing automation at Hardware testing integration level, I’ll suggest you to start with a service automation testing approach and all the testing that needs interaction with the hardware will be done manually. You can use emulators for the devices, but it is always important to be sure in a sanity test the backend and the real hardware communicates correctly , that the functionalities and requirements are met.
This is just the beginning, for the next article I’ll be talking about automation, reporting and virtual devices.
