Digital Phenotyping — Turning our Smartphones Inward
Insights from the digital fingerprints we leave behind — and how they can lead to better healthcare
I know something’s not right but I can’t figure out what’s going on and why. So early Tuesday morning, I find myself heading to the doctor for a checkup before work.
The clinic is empty except for an elderly woman reading yesterday’s newspaper. The waiting time makes me edgy. I manage to keep myself occupied with a half-finished sudoku left by a previous patient (note-to-self: wash hands after you get home) before the doctor calls me into her office.
Pleasantries exchanged, she asks me how I’m feeling, and I explain my symptoms. Tired. Hard to concentrate. Loss of appetite. Headaches. After a series of medical questions, she checks my blood pressure, pulse, breathing, and ear-nose-throat combo — all normal.
She then syncs my smartphone with hers and within seconds, an alarming picture manifests itself.
Looks like for the past month, I’ve had less than 5 hours of uninterrupted sleep a night. My meeting schedule at work has increased by 20%. The closest thing I got to exercise or fresh air was a 4 minute walk from my home to the bus I take to work. My interactions on social media have plummeted, and I’ve been spending 30% more time at home than usual. Even the average pressure I’ve been applying on my smartphone screen as well as my typing speed have decreased.
And that’s only the beginning…
OK, so there are “a few” kinks to work out before the above becomes a reality, but as our smartphone increasingly becomes an extension of ourselves, new sources of personal health insights will present themselves at rapid speeds.
Now, most people have heard about apps that might help better diagnose diseases — whether they detect melanomas by taking a picture of the skin (we’re a proud investor in SkinVision) or even help uncover Parkinsonian tremors.
To set expectations, this article is not about the latest and coolest diagnostic health apps.
Rather, this article explores the potentials (and risks) of uncovering deep health insights from the passive, background data collected by our smartphones while we go about our day.
Welcome to the world of digital phenotyping!
What is a Digital Phenotype?
“Our phones and computers have become reflections of our personalities, our interests, and our identities. They hold much that is important to us.”
— James Comey, Former FBI Director
To put digital phenotyping in perspective, let’s dive into the interplay between genetics, phenotypes and the digital world.
Our phenotype — our observable traits, including our eye colour, height, gender characteristics, biochemical and physiological composition and behaviour — is a product of our genetics, our environment, the interplay between the two (GxE interactions), and the social influence to which we’re all exposed.
What’s neat is that our phenotype, in turn, produces an extended phenotype, a term first coined by Richard Dawkins in the early 80’s, which describes the impact we produce on our surroundings in order to increase the likelihood of survival (e.g., the transmission of our genes to the next generation).
In nature, spiders build nets, bees build beehives, and earthworms modify their soil’s chemistry, all to maximise survival.
For us humans, I would argue that our transit system (which allows transportation of assets), the skyscrapers we build (designed to increase economic productivity), and sometimes even the goods we purchase are examples of our extended phenotypes.
But unlike other animals, humans’ extended phenotypes can be found both off- and online. Think of an online dating profile, a social media post, an online grocery order or a Medium article (on digital phenotyping?) as examples of an extended online phenotype.
As we use our smartphones to continuously evolve our extended online phenotypes and interact with the digital world, we leave behind digital fingerprints.
These are picked up by our smartphone’s (and wearables’) sensors to create our digital phenotype.
GPS location and movement, battery recharge frequency, voice patterns and speech, length or frequency of texts/calls, number of times visiting a certain app, the way our smartphone is organised, the angle at which we often hold our smartphone or even the frequency of app updates — all these data can make up our digital phenotype.
Together with traditional clinical approaches to characterising diseases (e.g., blood tests, imaging, physical exam), digital phenotypes can help shape an understanding of our overall health, illness and real-world behaviour.
When analysed appropriately, these data could reveal unique patterns that we ourselves might sometimes not be aware of.
Here’s a simple visual that puts it all together:
What Can Digital Phenotyping Reveal?
“iPhone, therefore I am”
— Jason Silva, futurist
With an estimated 6Bn smartphones in the world next year, our relationship with our devices has drastically changed since the advent of the PDA (remember those?).
From the earlier days of the smartphone where we primarily used it as a communication tool, to today’s somewhat counter-intuitive reality where we’ve started to use the smartphone to control our physical environment (e.g., think of ordering an Uber), our relationship with the smartphone is a complex one.
We’re now starting to turn our devices inward and use them as a tool to learn about ourselves. At the heart of this new smartphone usage evolution is digital phenotyping and the insights we can glean from this space.
Imagine if you could:
- Uncover diseases you might have but are not aware of
- List diseases that you might be susceptible to without any blood/genetic tests
- Predict how responsive (or not) you might be to certain medications
- Understand how likely you are to adhere to certain medication
- Predict how your disease will progress over time
No longer do we need to imagine, as while it’s still early days for digital phenotyping, here are some of the conditions that can already be detected:
As you’ll most likely note, most of these have a focus on mental health (at least for now).
Depression, in particular, has seen growing interest, given that more than 300 million people worldwide are afflicted with this disorder.
In a 2016 study, Swedish researchers were able to use digital phenotyping to predict and detect depression and were even able to provide interventional support.
Here’s what they did:
- Knowing that there’s a strong relationship between physical activity and depression, they used the smartphone’s acceleration data to analyse activity and walking time
- GPS information was used every 15 minutes to get the subjects’ location to calculate the distance they had traveled
- WiFi-based location logging was used to measure the time a subject stayed at home
- Total phone usage time was analysed, as were the number of incoming and outgoing calls and text messages, and the total number of contacts these communications were with as a benchmark of social activity
- Number of Calendar events were counted as a proxy for stress
By the end of the study, combining these digital phenotypes led to predicting depression with ~60% accuracy.
Moving away from mental health, in another study, researchers wanted to see whether digital phenotyping could be used to assess how likely older people are to suffer from falls.
In the US, about 3 million people at the age of 65 and older are treated for fall-related injuries. Believe it or not, falls cost the US healthcare system a whopping $50Bn every year, putting falls amongst the top 20 most expensive medical conditions!
Here’s what these researched did:
- Mined a pedestrian activity database that detected six mobility features (e.g., walking, jogging, skipping) and trained a model to recognise these
- Obtained five gait measurements (e.g., stride length) from iPhones strapped on the left and right hip of people walking on an electronic walkway for 30s
- Used transfer learning — a machine learning technique that improves the learning in a new task through the transfer of knowledge from a related task that has already been learned — to come up with a predictive model that classifies older adults as either low or high risk for falls.
Using both gyroscope and accelerometer measurements, the model was ~93% accurate at classifying people as either low risk/high risk for falls.
A steady step forward! (pun intended).
And what about using digital phenotyping together with our extended online phenotype to predict our genetic encoding?
Sounds like science fiction, right?
In a recent proof-of-concept study, scientists were able to show a link between your extended online phenotype and your actual genes! In particular, the genes belonging to our oxytocin regulation system (oxytocin is a hormone that plays a role in social bonding and reproduction).
Stick with me here as I try to explain the genetics…
Oxytocin (the hormone) binds to oxytocin receptors, which come in two genetic variations — one of these, the “pro-social” variant, is linked to lower autistic traits, higher empathy, higher ability for facial recognition and processing of social information.
This 117-subject study showed that people with the “pro-social” genetic variant of the oxytocin receptor had a significantly larger “social network”, as measured by the total number of names and phone numbers saved in their smartphone Contacts, and number of incoming calls.
This, to my knowledge, was the first study that has ever linked digital phenotyping with predicting one’s genetics (but if you come across any others, do let me know!)
With Great Data comes Great Responsibility
“Our mobile phones have become the greatest spy on the planet.”
— John McAfee, tech pioneer
Clearly, digital phenotyping has strong potential in healthcare, but in the words of Voltaire (and Spiderman’s uncle):
“With great power comes great responsibility.”
Currently, most digital phenotyping efforts have been sandboxed in the world of academia, with voluntary participants and for research purposes only.
For digital phenotyping to transition from academia and into clinical practice, privacy and security issues must be addressed.
Questions around data ownership (e.g., who owns your digital phenotype?), and how it’ll be used (e.g., for healthcare insights or for commercial gain) must be explored.
And how do we handle data sharing of your digital phenotype?
An example of accidentally sharing someone’s digital phenotype with another person can be gleaned from Target’s analysis of buying behaviours. By tracking their customers’ shopping habits, Target was able to identify which of their shoppers were most likely expecting a baby (by tracking the purchase of 25 pregnancy-related products).
Clever, right?
Things took a turn when the algorithm flagged a teenage girl living with her parents, and sent her coupons for everything from maternity clothes to diapers.
Her father (who opened her mail) was both surprised and furious, claiming that there must be some mistake!
Unfortunately for him, Target was, well… right on target.
But besides such communications blunders and the obvious dangers of potential data theft, what happens if digital phenotyping is used to:
- Deny people health insurance?
- Influence triaging decisions in emergency situations (e.g., two people arriving with gunshot wounds might get different quality of care based on their digital phenotypes)?
- Prioritise where physicians spend their time (e.g., dedicating more efforts to patients that lead healthier lifestyles)?
In short, as digital phenotyping is a field in its infancy, there’s a new ethical framework that will need to be created, while ensuring the basics are in place: user consent and understanding of how digital phenotypes are being collected and used, and trust that sharing these data will ultimately benefit (and not harm) the user.
Predicting the Future of Digital Phenotyping
With our increasing understanding of the power of digital phenotyping, here are some predictions for this space:
- Beyond Mental Health — while most applications of digital phenotyping today focus on mental health (given the tracking of behavioural data), I predict that we’ll start to see digital phenotyping used to better understand health in other areas. We’re already seeing links to cardiovascular disease and even genetics, so I’m cautiously optimistic about applying this tech south of the neck
- From Patients to Populations — while the focus of today’s research is on individual health, the focus of tomorrow will be on population health. Digital phenotyping could be used to help healthcare systems predict where and what type of services are required to increase the overall health of the population and better manage costs with predictive capabilities (e.g., risk prediction, hospitalisation stays and required capacity)
- Commercialising the Insights — I see multiple healthcare players begin to use and commercialise the insights from digital therapeutics in the future. For example, better categorising patients before clinical trials (e.g., by stress levels), predicting medication adherence or uncovering hidden illnesses might be some commercial applications relevant to pharma, HMOs and payers beyond the existing realm of academia
- Hitting the Inflection — combining digital phenotyping with other data sources (e.g., EMRs, gene sequencing) will have an exponential effect on the insights we can extract. We might uncover new diseases (or multiple subtypes of the same disease), find unique linkages between diseases which are today hidden and be better able to select therapies for patients with higher precision and impact
- The Phenotype Explosion — I’m seeing evidence of digital phenotyping beyond healthcare, a trend I strongly believe will continue. In the financial sector, companies are predicting creditworthiness based on digital phenotypes (see ZestFinance, a company founded by Google’s ex-CIO). And cybersecurity is moving towards using digital phenotyping to replace passwords. If I know the angle at which you hold your smartphone, how much pressure you apply to the screen, how fast you type, and even what fingers you use to interact with your phone, do I really need a password to identify that it’s you (see Pinn)?
It’s said that by 2020, over 44 zettabytes of digital data will exist (that’s 44 trillion gigabytes), of which we’ve currently only analysed about 0.5%.
Imagine the insights we can gain by tapping into this vast data pool with the right tools and approaches.
I’m a firm believer that with the appropriate regulations, digital phenotyping will continue to accelerate our understanding of diseases, our healthcare systems, ourselves, and our ever-evolving environment.
Thank you for reading, and most importantly, for becoming a part of my extended online phenotype.
