Product management on the Dreem headband
A few learnings collected along the way
At Dreem, the product we work on is wide-ranging. It involves both the headband itself (hardware) and code that makes it work (software). And Dreem’s software can in turn be split up in the following chunks: Mobile (iOS and Android apps); headband firmware (which runs live on the headband); server-side software (e.g. algorithms that crunch the data uploaded from headbands).
All these distinct sides of the product have different styles of development. To most readers, the specificities of software will supposedly be more familiar than those of hardware. (The first difference is that hardware takes longer at every stage, and, requiring significant investment, it is also less accessible to fledgling startups).
The role of the Product team is to make sure we are developing the right product to help our users. However, the reasoning behind product decisions is not always well understood. In our case, building a sleep product implies a number of specificities which we have found interesting to uncover along the way, and which we thought might interest others. And finally, writing down key learnings and concepts is always a worthwhile exercise, as it helps cristallise ideas and clarify one’s thinking.
A word about me: I joined Dreem in June 2016. The Product team, which I am a part of, is in charge on making sure we are building the right product for our users, and oversees a range of different aspects of how Dreem should work. My particular focus is on the headband itself, as a physical object, and the code that runs on it (more on this later). There is a large variety of different topics around the product, and, while not an expert on any of them, my job requires understanding enough about each of them to help make the best decisions and tradeoffs for the product as a whole. What follows is a collection of our high-level learnings, gathered as a team over the last few years.
This post is a general overview of the product development process, focusing on the hardware and the software that runs on it (firmware). It hopefully contains no obscure concepts and jargon, and if you’re thinking of creating a product such as Dreem, work in product development, or are just a curious reader, then we hope what follows will be of interest to you.
Here are some key elements that are characteristic of our product development at Dreem:
- How our hardware product development works
- What the specificities of developing a physical sleep product are
- What role the firmware plays in bringing the best out of our hardware
So without further ado, let’s dive in.
A high-level overview of how we develop hardware
Developing a new hardware product takes a long time (several years in most cases, in our case around 18 months), and involves many different steps along the way, requiring various areas of expertise. So below is an overview of what the entire hardware cycle on a product such as Dreem looks like. (Of course, it never goes exactly as planned, and we inevitably deviate from the initial plan. But the key steps are here, from initial ideation to manufacturing the product at scale. ) And next we’ll detail what acronym stands for, and what each step is about.
Phase 1 — Defining what product to build
Understanding what went right and wrong on the previous round
We’ve got thousands of customers who have bought and used the previous product. Most of them have told us at some point or another something they like or dislike about what they bought. It could be on a variety of subjects, from the critical (“the headband isn’t comfortable”, “the headband does not fit me”) to the more moonshot-like (“could I have a brain machine interface please?”).
An important point to mention here is that, as is often told in product design circles, you shouldn’t always take what users say at face-value: most users won’t know what they want until you show it to them. Or they’ll say they want something when in fact it is not so important, and what they really need is something else.
So we try to take into account not just what users do explicitly tell us, but what we can infer from looking at usage data. Compiling data about how users interact the touchpad, for example, we might ask ourselves: what does this tell us about how easy it is to use it ? How does usability differ on various user segments ? What we have also tried to keep in mind is that for every user that speaks out on a given subject, there are likely many more that think the same but haven’t been through the effort of expressing it.
A word about analysing data. Something we have found to be true is that analytics are best when you know what you are looking for, and want to validate a hypothesis and track something precise over time. The amount of data we have at our disposal being gigantic, there are innumerable ways of slicing and dicing the data, which means we can easily get lost in numbers and overwhelmed. So a useful first step is to get immersed in highly qualitative information: questionnaire answers, support tickets, phone calls, in-person users visits in the office. No quantitative value, just individual user voices. On our falling asleep features, for example, we sensed early on there was a problem. (This was a software feature, but the same holds true for the physical product, too). Usage was lower than expected, but we couldn’t really put our finger on what the cause was. Through accumulating hundreds of verbatim feedbacks, we began to get a sense of what was holding us back, and eventually made corrections on the product for which we quantitatively tracked improvement. It turned out that the problem wasn’t so much the features themselves, but the way they were explained to users and presented within the app. We likely would have continued to make local optimisations to the features, instead of changing the way they were presented, had we not been through user verbatim. So quantitative insights, in general, and real conversations, in particular, have the advantage of allowing for digging deep into a topic. “Why do you think that about X ? OK, and what if X was different in such a way, would you feel better about it? And what do you mean when you say that?” A single question on a form will never be able to capture nuance in a way a conversation can.
Deciding what the main changes will be
This step is about picking the priorities we want to give to the new product. These can be of many kinds: the shape of the product, its size or weight, its look and feel, which sensors to integrate, and which electronics architecture to choose to give it the capacities we’d like.
One particularity of hardware is that in involves probably more “gut-feeling” decisions than, say, software. Because it often requires physical engineering, and expensive production lines, fundamental and irreversible choices have to be made early on.
One simple heuristic is if it can be easily modified later on (i.e. software), we will usually release it out quickly to our user base, monitor it, improve on it, and so on.
Conversely, sometimes the penalty for getting it wrong is higher (both in terms of user experience, financial cost, and work needed to fix it), which is usually the case for hardware. In this case, we will be more conservative with our decision making, all the while never having complete information at our disposal. We sometimes make choices without fully knowing what exact features we will be built on top of our an architecture, or what the user demand for it will be, but at least that allows us to move forward while keeping options open.
A kind of tradeoff we have had a number of times is the following: do we optimise for product performance, user comfort, or cost ? For example, the better the electronics inside the headband, the more we can do with the product, and the better the user experience might be (better sleep data, faster computing power, better features, longer battery autonomy), so we will want to add sophisticated components to the headband. However, by improving the electronics, it is easy to shoot ourselves in the foot by creating an unnaffordable product (adding expensive components to the headband) or an uncomfortable one (electronic components requiring rigid casing which are uncomfortable to sleep on). Another example we were faced with early on was the pulse oximeter (PPG) device: we realised that for it to accurately measure the heart rate for all users, it required a significant red light intensity. However, some users could see that light if it was too bright, and they felt it was preventing them from sleeping. We ended up optimising for user comfort, and picking for an intensity level that was visible to virtually no users, while maximising the accuracy of heart rate detection.
Once we’ve settled on a list of hardware improvements, and we are confident that such a product is feasible (both from an engineering and business model standpoint), we move on to design.
Creating a design that works
Over the past few years, we have had the chance to collaborate on our products with world-class team of industrial designers known as Fuseproject. At the start of a project, we will typically organise a kick-off session, where we’ll discuss what we hope to achieve with this new product. We’ll typically do some soul-searching and look for inspiration, as well as answer any structural questions.
We’ll then explore several design directions, then narrow them down to one or two, and eventually, we are down to a single design direction. (Often, we’ll make rudimentary, non-functional prototypes of these design to compare them.) We’ll have many rounds of back & forth between the design team and ourselves to reach a first version of CAD (Computer-Aided Design).
A particularly important aspect of working with a design team is giving it both sufficient creative autonomy while not going too far astray of technical realities. This is the balance we have been striving for in our collaboration with Fuse: fruitful conversations where designers will push for ideas, while our team brings the technical standpoint as well as deep customer expertise. The initial idea will often come from the product / design team, centred on user experience. But often making a design work technically (picking up EEG signal, achieving a good fit for all head sizes, etc.) can only happen through smart design choices. Signal acquisition (EEG, and pulse oximetry), in particular, is the backbone of everything we do, and requires extremely close collaboration and feedback in both directions between engineering and design.
Why haven’t we built an industrial design (ID) team in-house ? There are several reasons for that. The main is that we currently only have one product, for which the ID phase happens for a few months every year or so, implying a highly intermittent workload. And when we do have design work, it is wide-ranging (visual renderings, materials explorations, sketching). In other words, we don’t often need ID work, but when we do, we need a full creative team on the project, bouncing ideas off each other, instead of just a person or two.
Phase 2 — Getting the details right
Once we have a first product design, we will go back to our lab, and try to build prototypes that actually work. For the Dreem headband, this stage implies getting many things right, such as electronics, signal acquisition, audio, and of course comfort. This stage can take a long time, and often requires going back to the drawing board, and working with our design partner, to find solutions that work in practice.
There are both pure engineering constraints (electronics and embedded software), but also some more subjective aspects. To make sure we get these right, we will get real users to wear the product and to tell us in detail what went well and what didn’t. Was the headband comfortable ? Did you take it off during the night ? How was the audio quality ? All of these answers are extremely valuable to make changes to our prototypes and reach our initial goals.
Having in-house prototyping resources has been important to us, and they are probably worth investing in early on. You never really know if something is going to work until you actually get it in people’s hands, and listen to their thoughts. In particular, a good 3D printer in your workspace can bring a design to physical reality much faster, which ultimately favours decision-making. On touchpad design, for example, we were able to create computer design for various shapes and sizes, 3D print them, try them out extensively, and make a decision, all in the space of a working day. Relying on an external prototyping partner wouldn’t have been that easy, so having a strong expertise in prototyping has been a definite plus.
In our case, one specificity in user testing is the fact that to try a sleep product, you actually need to sleep with the product. It’s not like a smartphone that you grab, play with for 5 minutes, and return having tested it. For some product features, a quick nap can be enough (that’s why we’re fortunate enough to have two dedicated nap rooms in our office in Paris). But for other tests, a real night is necessary, which involves taking a prototype home with you in the evening, along with instructions from the Testing team on what to try out, and then sharing your feedback the next morning. It could be a new headband form factor, but also a new algorithm, a new feature in the app, or anything else.
Phase 3 — Manufacturing the product at scale
Once we have functional prototypes, we will need to ensure our design can be manufactured at scale. There is a difference between a prototype that can be hand-made in our lab by our team, and a product that can be manufactured flawlessly thousands of times. This stage is known as DFM (Design For Manufacturing). In our case, manufacturing partners are involved relatively early in the process, in part because some of our challenges (e.g. dry hair EEG sensors) are new and don’t really have an associated playbook.
Next, as is standard for hardware products, we will clear the relevant manufacturing hurdles by producing successively DVT, PVT, and MP units. This entire process will usually take 3 to 6 months. If you haven’t yet been introduced to the wonderful world of manufacturing, here is what each step roughly represent :
DVT (Design Validation Test): This is about ensuring that the production line can make the product as we intended
PVT (Production Validation Test): This test involves making sure we can manufacture the produce at the production speed we expect to have, with few defective units
MP (Mass Production): After ramping up volume, the product is being built in large quantities, ready to be sold.
One aspect that can make industrialization such a juggling act is the number of external partners involved. To create all components in our product (385 electronic, 83 mechanical), 27 external partners are involved, each of which work with several suppliers. All of these companies need to play their part for the product to be built. If a single partner cannot deliver, then the entire product cannot be made. There is a high degree of unpredictability along the way. We have learned this the hard way, having to explain to customers who had ordered the product several months ago that they would have to wait longer still.
Industrialisation would deserve several more dedicated, in-depth posts written by those that understand its ins and outs. But another aspect to mention is that this stage is sometimes seen by outsiders as automatic, highly automatised conveyor belt production. In fact, as our Industrialisation team will tell you, there is a surprising amount of human input required to explain the manufacturing processes to our partners, make sure they are well understood and well implemented.
It is often the case that we design changes need to be made during each of these steps, as producing the product at scale is different to producing individual units. Once manufacturing hurdles have been successfully cleared, we will ship to product to end users, and another phase kicks in, which involves onboarding our users, making sure we gather feedback, and monitoring usage analytics. Getting actual answers on what went well and what didn’t can take a long time, and results can be ambiguous. But that too will be for another post!
The specificities of developing a sleep product
Sleep, which amounts to about a third of our lives, is unusual from the rest of our activities. Before and after sleeping, you are likely tired and drowsy. You are highly sensitive to bright lights and sounds. Needless to say, everybody sleeps, from babies to the elderly. And of course, no two sleepers are alike.
So all of these constraints lead to some key aspects that we keep in mind when developing Dreem.
Personalisation and measuring impact
When we ship a piece of product to our user base, from a new hardware to a sleep onset feature, we’re often trying to directly impact the user’s sleep. But it is hard to draw conclusions on what is working because of three aspects: 1) sleep does not change over night, but over longer time frames, 2) many factors (product-related or otherwise) can impact somebody’s sleep, 3) no two sleepers are the same.
Behavioural sleep improvements (which we are heavily investing in) require building real habits night after night. This means we need several weeks or months worth of user data to see if we’ve managed to move the needle. In short, if you’re trying to sense if a product makes people sleep well, it takes time for people to get used to the product, and to observe an actual impact on their sleep. So we cannot have a very high frequency of iteration, modifying product features every few days. We instead need to actually give our solution enough time to produce conclusive results (good or bad).
The fact that there no two sleepers are alike means that we need to include multiple use cases by default. This is something we discovered early on when looking at usage patterns. We were expecting to find coherent usage groups (monthly, weekly, power users sleeping with Dreem everyday), perhaps with a few outliers here and there. Instead, we found great individual variability in our data, and had a hard time identifying clear user types. And having across-the-board metrics for all users turns out to be surprisingly difficult. A metric such as the Sleep Score (an integer between 0 and 100 which we introduced to sum up the quality of your night) was the topic of many discussions on our team: should it be an objective measure of your sleep ? should it instead track your own improvement over time? what if you slept well, but were tired — should it reflect that too? what if you were a good sleeper within your age group, but not that great overall — how should that be taken into account?
Adaptability, accessibility, and sensitivity to stimuli
Dealing with a headband means dealing with a wide diversity of head shapes and sizes, hair styles, skin types and sleeping positions (including movements during the night). And because sleep problems affect all kinds of people, it means catering to young and old alike, as well some tech-savvy users and less technophile ones.
Therefore, we try to test our prototypes (both hardware and app) on as as wide a sample of users as possible, to make sure it satisfies the highest number of people. Once again, our nap room is very useful to test all of these aspects.
Without a doubt, one of the most crucial parts of the product user experience for Dreem is comfort. During night time, a person is highly sensitive to physical sensations and it goes without saying that for the user to wear the product product all night long, it needs to be as comfortable as possible. A slight tingle you wouldn’t necessarily notice during the day, can become a real irritation as you are trying to sleep. The way in which you sleep (on the side, on the back, or on the front) could potentially make your experience completely different. The room environment can significantly affect perceived comfort on the head, as we found out comparing tesing results during cold winter nights with those of hot summer nights. And of course, besides comfort, another central challenge is to make sure the headband fits every user, i.e. stays on the head all night long.
A central challenge regarding comfort had to do with achieving sufficient contact between the front band and the forehead to pick up reliable EEG and heart rate biosignals, without applying too much pressure. For a single design, signal quality can also highly dependent on head shape and hair style. We for example discovered that on some head shapes, the pulse oximeter wasn’t close enough to the temple to accurately detect the user’s heart rate.
Another had to do with the sides of the headband, central to comfort, as most people sleep on their side (thereby applying the most pressure to that part of the head). The temple is a sensitive spot, and the headband has to be as soft as possible in that location, while still accommodating wires which require thickness and rigidity.
More generally, aside from the product’s comfort and fit, during night time we are highly sensitive to stimuli of all kinds: auditory, visual, tactile. An LED light which is too bright might be reflected on your ceiling and prevent you from falling asleep. A beeping sound, or the wrong kind of rain can be irritating and keep you awake. If these irritations are of the physical type (e.g. itchy fabric), then they must be eliminated early in the testing process, while software ones (e.g. glitchy touchpad) can be solved later on with a fix (more on this later regarding the use of firmware).
Frictionless daily use
Previous to Dreem, accurately tracking brain activity overnight meant sleeping with a polysomnography device (PSG for short), and that can in no way be called frictionless. Among other things, it requires applying sticky “wet” electrodes, with conductive gel, to various parts of the brain and chest. It requires wearing a physical chest belt to measure breathing rates, and keeping your finger inside a blood oxygen tracker all night long. It needs to be plugged into a PC for the recording to start and end. And you need to have an expert by your side not just to set it up, but also to tell you about your sleep results. So there was undoubtedly room for improving the PSG user experience. Dry electrodes, the touchpad, the Dreem app, plug to upload your data: all of these are the result of wanting the experience to be incomparably better to wearing a polysomnography device.
Our users interact with the product first thing every morning, and last thing every evening. As mentioned above, these are times when you are not at your cognitive peak, and any friction in the user experience can stand in the way of a better sleep. That’s why a super intuitive user experience has been our top product focus from the start. It’s why the only thing you need to do with your headband as you wake up is plug it in, and your sleep data appears on your smartphone an instant later.
Regarding usage, Dreem is also a product that users travel with, and this requires thought, too. What happens if you don’t have a Wi-Fi connection ? How are time zones handled ? What accessories should we include to make charging possible ? All of these questions, and the associated “edge cases” come into play.
Beyond pure product usability, sleep is one of these aspects of health (along with exercise, meditation, or diet) that has a highly behavioral aspect to it. After the initial intention (“I will go to bed early tonight !”, “I will wear my Dreem headband tonight”), it takes months to build a successful sleep routine, and even more so when you integrate a physical product to it. So the initial improvement on the PSG isn’t enough, we have increasingly focused on cracking user psychology for an optimal user experience. Of course, the idea isn’t to get into the dark arts of persuasion to get you to use the headband as much as possible. The idea is to understand what makes people tick so that we can nudge them towards better sleep, night after night.
The firmware as a complement to the hardware
The best thing about the hardware running its own software means the physical product can get better and smarter over time. Similarly to the way Tesla updates its cars to the latest software remotely, we push over-the-air firmware updates too.
Progressive deployment
We will typically roll out new updates progressively, batch by batch, to ensure everything is running smoothly, and react fast if needed. One issue we previously had was that the touchpad was too sensitive: it would pick up unintentional movements as interactions, which could ultimately disturb the user. Looking at detailed analytics, we understood which interactions were causing “false positives”, how many users had been affected by the problem, and when during the night it typically occurred. Based on all of these, we devised and implemented a correction to our touchpad detection algorithm, and tracked the disappearance of the problem over time.
Different versions of the product in the wild
Another of our major projects over the past year has been the Adventurer program, and it wouldn’t be possible without the capabilities of our firmware. We can for example ship different versions of our alarm clock algorithm to different groups of users, which wake people up at different moments of their sleep cycles in the morning, and see how each group responds (a kind of A/B test). This illustrates the way in which, thanks to shipping frequent firmware updates we can have a shorter feedback loop, from scientific theory to actual implementation in people’s bedrooms.
Incremental improvements or entirely new features, but nothing in between
It’s important to keep in mind, however, that while most things can be changed through firmware, it’s not necessarily always a good idea to do so. Key product functions, such as light indicators for example, need to be kept consistent to avoid confusing existing users. Besides, future improvements on a feature may never get a chance to reach users if they tried a first version that did not work. As mentioned before, this is particularly true for some of our Sleep Induction Techniques, for example: if you tried a version of a technique that wasn’t fully optimized and didn’t help you fall asleep, you will be much less likely to try future iterations. And of course, we are dealing with people’s sleep, which is sensitive and has real world impact, and must in no circumstance be negatively impacted by the product. All of this leads us to be conservative in our firmware deployments.
So all in all, firmware updates allow the product to get better over time, but they are best for progressive, cautious iterations, or entirely new features altogether.
Wrap-up
Here was a quick, non-exhaustive overview of what we’ve learned over the years as we’ve scaled Dreem to our users. Let us know if this was useful or interesting to you — we’ve got many more topics we’d like to write about in the future, so if you’ve got a particular preferences or suggestions, we’d be happy to hear them. Thanks for reading !
