FaceTrack tackles the endemic obesity problem in western society. It replaces the bathroom scales, passively tracking a patient’s weight via their smartphone. In this piece I show how I came up with an idea for a ‘radically passive’ user experience, set about validating it, and then evolved the design according to medical and technical nuances I uncovered.
I was brainstorming new side project ideas. I knew I wanted to create a ‘passive’ experience, using one of the sensors on smartphones — now in daily use by 56% of the global population [Source]! I was immediately drawn to the camera — such a firehose of data. I’d also been reading about ‘computer vision’ techniques. Almost naturally then an idea emerged: camera + passive design: what about the front-facing camera? It’s pointed at our faces for several minutes a day… That old saying sprung to mind ‘the eyes are the window to the soul.’ Not just the eyes, maybe the whole face — e.g. a person’s complexion, bags under the eyes and face shape, can all give an immediate indication of stress, diet, rate of aging, fitness/strength, sleep quality, and weight.
Don’t laugh, but weight tracking chimed with a personal experience of seeing the fat under my chin change proportionally to the rest of my body. So this is the avenue I went down: using the front-facing camera to track a person’s weight via measuring dimensions of their face, passively . I imagined an App you install, open once, forget about, and then once a week sends you an email with your weight — your weight tracked from then on with zero effort.
Although my background is in healthcare, I had to check whether this idea, that facial-fat deposits were proportional to body fat. I interviewed friends who were Medical Doctors, one a Resident Physician at Stanford, another at Kaiser Permanente in SF.
a) Target user
Speaking with them was illuminating. They explained that yes, change in facial fat deposits would be proportional to deposits elsewhere, but this varied from person to person immensely. They also suggested I use the tool to tackle the Obesity crisis in America — more than two-thirds (68.8 percent) of US adults are considered to be overweight or obese. There were two possible personas to build for:
- The average healthy person, just interested in looking their best
- People with clinical obesity, that need to lose weight urgently
The Stanford Doctor stressed the need for better patient surveillance. Physicians want to track their patient’s weight, but patients are poorly compliant about measuring it.
b) Actual function
The interviews also highlighted a key product design decision I needed to make. Was I simply measuring weight [i.e. replacing the bathroom scales], or going further and helping them lose weight. I decided to start with the former for the MVP — purely passive measurement. In fact a study by Butryn et al. showed that “more frequent weighing was associated with lower BMI” [link], so weight could arise simply by informing patients of their weight. In the future, we could expand to more pro-active behavioral change techniques based on the results e.g. congratulating improvements, simulating how a person’s face might look if they lost weight [I wonder if beauty is the king of all motivators?!]
a) Computer Vision Measurement
Up to this point I had assumed that it was possible to measure these small changes in face size with the average smartphone’s front-facing camera. To validate this I met with a leading specialist in computer vision techniques from Google’s Android Camera App team. He said it would require experts in the field: ‘tough, but possible.’
He also flagged a technical issue that would intersect with design — how small differences in face shape would be difficult to distinguish from differences in external factors, such as lighting, head pose or facial expression. He suggested that several measurements should be taken each day and scored for ‘confidence — in fact a common computer vision technique. It would re-take photos if it had low confidence in the measurement. If the app collected 2–3 high confidence measurements a day, it could provide a robust result.
b) Using the Camera in the Background
Until this point I had assumed it was possible for an App to use the Camera running in the background — crucial for the passive design. To validate this I spoke with a seasoned Mobile Software Engineer. It turns out that Android does permit background use, but iOS does not for privacy reasons. I wondered if we could overcome this with ‘a trojan horse’ app design e.g. an Alarm clock app that took measurements every time you reached to press the snooze button in the morning! To start however, it was feasible to build for Android only: 51.7% of US smartphone users today [Source].
Ethical Feasibility: Privacy
User privacy concerns are a potential barrier to adoption. Many will be uncomfortable with the camera activating without them knowing. I believe this is surmountable however via:
- Messaging — notice how I used the term ‘measurement’ throughout, instead of ‘taking pictures’ which implies saving.
- Establishing medical credibility in the brand through the real impact on Obesity, or associating with medical providers.
Through interviewing experts I discovered that FaceTrack would be most valuable if designed for obesity sufferers, and that it would be technically feasible on Android. This ‘no-brainer-to-download’ passive user experience, when made available on the universal smartphone, should see rapid and widespread adoption, and have an impact at massive scale.
[You can return to the index of my portfolio here]