Smartphone and Internet Usage Could Provide Insights Into Behaviour and Mental Health
Our increasing usage of smartphones and the Internet (particularly social media/networking) over the last couple of decades has increased what can be termed our ‘digital footprint’ or our ‘digital exhaust’. An individual’s digital footprint is basically the data records they create as a result of their interactions with the Internet and personal digital devices such as smartphones.
All of these interactions sum up to form a unique digital history of the individual, a history that can be mined to infer information about the user. This possibility has of course been utilised, somewhat notoriously, by social media companies who use your online interactions to automatically customise your experience of their systems. From Facebook’s customised advertisements that are based on data-driven inferences about your preferences, to YouTube’s video recommendation system that is informed by your viewing history, data-driven insights about the individual are a cornerstone of the modern web. However, the fact that an individual’s digital footprint could be used to infer their behaviours, preferences and mental states opens up the possibility of using such a process to gain insights into their mental health, insights of clinical value that could be used to anticipate mental ill-health and also inform treatment.
At the intersection of the computing, data and behavioural sciences, this process of learning about an individual’s psychology from their digital footprint has been termed ‘digital phenotyping’, although the simpler term ‘personal sensing’ has also been suggested. ‘Phenotype’ is a scientific term used to denote an organism’s (including humans) naturally observable physical and behavioural characteristics. Thus, from this meaning ‘digital phenotyping’ is the process of measuring or identifying certain (behavioural) characteristics based on an individual’s digital footprint. Whilst this term can apply broadly to data generated from any digital device or Internet interaction, given their ubiquity, our constant usage of them and their broad array of sensors and reasons for use, digital phenotyping is mainly associated with smartphones, hence they will be the focus of this article.
Modern smartphones come equipped with an array of sensors and functionalities. The task of digital phenotyping research is to investigate how data patterns collected from these sensors/functionalities map onto certain psychological states and mental health conditions. Following are some examples of sensors/functionalities and early research that suggests how they are indicators of mental health.
Let us begin with the familiar GPS/geolocation sensor commonly embedded in smartphones, which we often use to guide us with directions from one place to another. Research is exploring the connections between information extracted from geolocation data and depression. For example, some early results suggest that people who are depressed tend to have less spread in the locations they visit. The types of locations one visits could also offer mental health insights, though there has been little research on this idea. One small study found some modest though inconsistent relationships between the time spent in certain locations and mental health; depression and anxiety scores were lower for some people who spent more time in spiritual locations and nondepressed people spent significantly more time at work.
The importance of social connectivity for mental health is well-established. Incoming/outgoing calls and SMSs can serve as measures of social connectivity and by implication could be used to pick up on signs of isolation or mental ill-health. One study found that certain reductions in the number and duration of outgoing calls, as well as number of text messages, were associated with relapses of schizophrenia. On a related note, there has been research into using the Bluetooth feature in smartphones as a measure of sociability and mental health, basically by detecting how many other Bluetooth devices have been in a smartphone’s proximity.
In line with the idea that the language we use provides a window into our minds, text from sources such as patient transcripts and peoples’ social media posts have been analysed with natural language processing using computers to detect mental health conditions. Properties characteristic of language disturbance, such as impoverished vocabulary, semantic incoherence and reduced syntactic complexity can be indicators of mental illness. For example, one research project has used measures of the coherence and complexity of patient transcripts to predict the onset of psychosis. The newsfeeds and forums of services such as Facebook, Twitter and Reddit also provide a rich source of material for mental health detection. For example, in one study the Facebook posts of a patient cohort were analysed to predict, with an accuracy approximately matching screening surveys, depression as recorded in their electronic medical records. It was found that things like hostile language, negative sentiment and rumination or preoccupation with the self, as detected in language usage, were predictors of depression.
Even a person’s tactile interactions with the screen (typing, clicking, scrolling, and scrolling) may provide an indication of their mental health. For example, an agitated state or a manic episode may be preceded by an increased speed of screen reaction, or more mobile phone movement while typing, which can be measured using the accelerometer motion sensor common in smartphones. One study showed that both average delays between keystrokes and auto-correction rates (such as misspellings) correlated positively with a common depression scale. In another study comparing two groups, one with depressive tendencies and one without, the depressive group showed longer periods between pressing and releasing a key. This indicates a slower motor reaction time or psychomotor retardation, which is a feature of depression.
Apart from specific sensors, general phone usage can also provide useful insights into an individual’s psychology. For example, which apps people are using and how often they are being used could offer a way to help gauge mental health. At the simplest level for example, if an individual starts using an app for anxiety management, this is a strong indication that they are experiencing anxiety. Furthermore, there is now a body of unsettled research on negative associations between excessive social media usage and mental ill-health. Finally, by tracking when, how often and in what intervals someone is using their phone, certain inferences might be made. As a simple example, abnormal nocturnal smartphone use could be an indication of insomnia.
As well as passively collecting data from such smartphone sensors and functionalities, smartphones also provide the opportunity to collect in-situ questionnaire responses from users. Known as experience sampling or ecological momentary assessments (EMA), smartphones can be used to periodically push short psychological assessments that prompt users for responses on their thoughts, feelings, behaviours, and environment in the moment. Such information can now be used to investigate connections between passive sensing data and user-reported psychological states. It is also interesting to note that such real-time assessments can provide psychologically valuable secondary information in terms of how users respond to them. For example, the time it takes one to respond to a questionnaire prompt, the time it takes to complete the answers or whether they respond to it at all can provide insights into their current psychological state.
Ethical Considerations
In an era of data surveillance and “digital capitalism”, where big tech companies are storing and mining large volumes of our personal data, particularly for commercial motives and often problematically, due consideration must be given to the ethical and privacy issues surrounding digital phenotyping technology.
As a starting point, it is fair to say that motives behind digital phenotyping research are positive and driven by its beneficent potential to inform mental health care. Of course, given the large volumes of personal data involved and the mental health information that could be inferred from them, it is important that frameworks, guidelines, and technical systems that maximise data security and respect user privacy are implemented.
Some of these challenges come down to proper governance and the responsible handling of the data, at both social and technical levels. Regarding social aspects, practices such as not sharing personal client data without their consent and giving the user control over which sensors they wish to share are two examples. Regarding technical aspects, digital phenotyping systems should implement safe data storage mechanisms and perform minimally sufficient data extractions. For example, suppose that a digital phenotyping system involved capturing a user’s voice from their phone communication, but that rather than requiring an analysis of the person’s words, the system could infer all that it needed from the non-linguistic or acoustic characteristics of the speech, such as pitch or tone. In this way, this acoustic information alone could be sent to a central data repository for analysis before the recorded communication is deleted from the user’s phone. Or perhaps the calculations could occur on the phone itself.
Beyond such practical security and privacy matters however come questions regarding what does it actually mean for an individual’s digital footprint to say something about their mental health and what are the prospects for digital phenotyping to provide actionable insights in clinical practice. Sceptics might consider such digital phenotyping a form of 21st century “digital phrenology”, in the same problematic or controversial camp as predicting an individual’s personality traits or personal orientations using machine learning analyses of facial characteristics. The first point to make in response to this is a distinction between the analysis of bodily features or characteristics versus the analysis of behavioural indicators. Whilst there is no guarantee of perfect correctness, smartphone digital phenotyping is based on two reasonable propositions. Firstly, there is the well-established proposition that there are associations between certain behaviours and certain mental health conditions. Secondly, there is the reasonable proposition that user contexts and behaviours can be inferred from smartphone usage. Ultimately these associations will be captured with rigorous quantitative modelling techniques that use smartphone information inputs to predict psychometric outcome scores or diagnostic category classifications. This being said, it is important to consider the accuracy limitations of such predictive algorithms and their applicability.
Medical testing takes many forms and there is no such thing as a 100% perfect test. Despite this, the analytical accuracy and precision of pathology tests for example are typically very good. In current times, despite providing some false positives and false negatives, tests for COVID-19 are reliable. Compared to such relatively straightforward physical testing however, it may be the case that even the best digital phenotyping systems will not provide diagnostic results with such a high level of reliability, accuracy, and precision. This is naturally in part due to the complexities of the subject matter (i.e. the mind and human behaviour), something which unsatisfactory traditional approaches to mental health diagnosis are no stranger to. But to dismiss the possibility of digital phenotyping because of these relative limitations is to miss its potential utility. At the very least, a client’s smartphone information, irrespective of whether it is driving an algorithmic diagnostic prediction, could help to manually inform their clinician’s own decision making and mental health care provision. Beyond this, if there does arise a digital phenotyping system that can provide diagnostic predictions which have a sufficiently high accuracy or probability of being correct, then even if it might not be sufficiently reliable to be automatically acted upon, it would suffice to help a clinician focus on considering certain possibilities while eliminating others. For even if a test cannot be used to home in on an exact diagnosis, it can at least narrow down the range of possibilities and ultimately inform human expert judgement in a field that needs better diagnostic tools.
The functionality to assess mental health objectively and continuously by analysing the data patterns an individual creates in their daily life offers a radically transformative alternative to traditional methods that often rely on unreliable or inconsistent self-reporting and subjective analyses. Digital phenotyping could very well be a revolutionary digital tool for the future of mental health care.
