Hardware vs Software Product Testing
As you might already know, Somia CX is a client based consultancy, therefore it’s natural for us to get various type of projects. It’s true not only for the scope of the projects, from just evaluating existing product until foundational research, but also from the industry and its product types. In this year quarter alone I’ve personally done 3 different scopes and products: Validating new food-related-type of service, Foundational research in music industry, and the most recent and for me personally the most interesting one: evaluating real functioning prototype in sanitaryware.
The last project is a follow up from series of researches in the last 3 years we’ve done with the client, from exploratory one where we ventured into multiple directions in the same area until real 1:1 functioning prototype where people can actually use the product. I realised there are ways where doing hardware related projects, especially sanitary one, are different. Here are two big findings that we found.
1. Preparing The Prototypes: Not Just Making It Clickable
In software based project, we usually use clickable prototypes where we can simulate real working product, by just using prototyping tools such as Invision or directly from the software creation tools such as Adobe XD or Figma. Depending on the complexity we can create the clickable in 1–3 days. We then let the participants to try the product using scenarios and get their feedbacks.
Just like in software based one, our goal is to evaluate and gather feedback from people who actually try the product. Since the sanitary product we test have water related function, we need to make the prototype to be connected to some water line, easy right? That is also what we initially thought.
- Turning house into testing lab
The first thing we need to do is finding the place where we can install the prototypes, that need to be mounted on the wall, this turns out to be a bit problematic. Since we don’t have a permanent space for hardware setup, we need to find one. It turns out there are limited place where you can install 8 different sanitary product setups without having your cash sucked up by the renovation budget. The solution: using fake walls to mount the prototypes so we don’t have to spend too much money on fixing the actual wall.
- Not just about the prototypes
When you do hardware related testing, there are aspect that could go wrong not from the prototypes itself (well, there are, but more on that later) but from the installation. So you better have on ground team that can help you right away to solve any problems rising. We had a team that help us from installing the prototypes until making sure the test goes smoothly from technical aspects. This team solved many issues that normally we won’t have in a controlled lab or simple clickable prototypes: unstable water pressure, water leaking, water tank sensor not working, water pipe suddenly undone, and many other technical problems. Just like our lovely printer, they can be naughty in the most important time.
2. Hardware Prototypes = Mechanical Bug
We can set boundaries or limitation in software based product so we can focus the evaluation on certain features or journeys. It’s easy to do it via prototyping tools, just create hotspot and link all the page together. You can even simulates micro interaction like hanging navigation or some animation effects. You don’t have to actually code it to make it look ‘real’.
Just like the software based one, the prototype in hardware product also need to look ‘real’ so we can have a proper feedback from the participants, visually & functionally. If by tapping a button water will come out, then prototype need to also do it. But then what set apart the prototypes from real launched product is the internal mechanism of it. Launched product usually has already considered the durability, reliability, maintenance, and assembly process. Meanwhile prototypes is usually still ‘hacked’ together to recreate the functionality, either by creating certain parts with cheaper material, using parts from other products, taping certain parts to hold it in place, or any other means to create the prototype so it can later be iterated efficiently.
This ‘hacked’ together situation means sometimes during the evaluation time, it could fail. During our evaluation time, 3 out of 8 prototypes were faulty. One prototype had problem with its’ water supply part, it leaked everywhere so we had to cut it from the water line. One had problem with its mechanical trigger control since the trigger was taken from another product. One had problem with sensor trigger control. All of this happened the night before the evaluation days. A dedicated engineer from the client tried and managed to save only one with the sensor. It had to be reset using magnet after every time participant use it. With all participants had been booked for the next five days we adapted the the flow of the evaluation to make sure we still can get feedback for the rest of faulty prototypes. We covered those two and only uncover it after participants had tried other prototypes. We used it as a probing tools.
Summary
Testing hardware needs a different sets of preparation and longer time, since it has different complexity. We need to make sure we have all things considered when we do hardware related projects: suitable space, prepared technical aspect of the space & good technical team. And last but not least, an alternate plan when things go south.
