Socially insensitive tech…in the home.

‘A place of harmony’ is what people want most from their domestic space — but is smart technology sensitive to the needs of a harmonious home?

It turns out that where home is concerned, there is no place like a harmonious place. People want their domestic space to be one of balance, and, when sharing with others, one of peace and amicability. Smart technology is often marketed as a way of reducing the domestic stressors we face as a household; it makes chores easier to do from anywhere, reduces financial pressure by making homes energy efficient, and reduces energy consumption and wastage. In so doing, smart technology could be a valuable tool to smooth the course of cohabiting, supporting relationships quietly from the side lines as the calm technology learns preferences over time, and eventually fades into the background of the home environment. As yet however, there has been little exploration into the impact of smart technology on the relationship dynamics within the home. Whilst perhaps alleviating domestic time and money pressures, might the introduction of smart technologies upset the domestic balance elsewhere? And how can technology learn and reconcile the various and potentially conflicting needs of a household? In this research I ask; is smart technology sensitive to the needs of a harmonious home?

Helping hand vs heavy hand — ambient assistance creating domestic discord.

In their current guise, smart home devices cause varying levels of domestic unrest. During the research, participants in different living situations revealed the challenges created by the introduction of smart technology into the home; from the struggle to control the technology itself, to undesired changes in their relationship with their home and the people they share it with. These challenges were writ large, causing heightened discomfort when experienced ‘publicly’ in the presence of guests visiting their home (motivation for buying the smart technology depending).

In most households that took part in the research, the enthusiasm, time-investment and capability with smart home devices was one-sided, creating an undercurrent of inequality and tension. Whilst the enthusiastic person seemed relatively unaware of the tension, the less-involved partners revealed feeling a lack of authorship over the space, a lack of mastery within their own home, and, as a result, an eroded sense of belonging.

“It’s set up with his email — so his is the master account if you like. If I want to change the proximity of the sensors for the doorbell, which I have done because the notifications are going crazy on my phone, I have to ask him if he wouldn’t mind doing it, as I don’t know his passwords.” Asia*, living with her husband.

“She doesn’t have the hive app on her phone, she can control the heating using the desktop or the thermostat. But I just use the app. I set it up in the flat. It’s great.” Alexander*, living with his girlfriend.

“We have Amazon Alexa to control the TV and lights. She is all very much [his] domain — I would be perfectly happy with light switches and controls. It’s a nice thing when it works most of the time — it’s frustrating when you want to do something simple and she has a bit of a hissy fit. When things go wrong I have to ask him to come and do it, as I can’t.” Jess*, living with her boyfriend.

Perhaps this developing sense of inequality is unsurprising given that from the start, smart home technology purchase decisions are not prompted by a specific household need or as a solution to a consensually identified problem. Household deliberation over the purchase of smart tech is notably absent — these are solo decisions motivated by personal agendas that will ultimately change the shared space. This is because people perceive smart home technology as innocuous — smart assistants such as Amazon Echo or Google Home, for instance, don’t do anything that your phone doesn’t already do, right? Smart energy or lighting is no different to what you do in your home anyway, it’s just controlled via your phone. It sounds like it’s not going to be a big deal, but smart technology has the power to disrupt the familiar, with one person inadvertently wresting the control of the shared environment and leaving significant others out in the cold. It can create anxiety in their place of sanctuary, and leave them at odds with those they share the space with.

Socially awkward in the corner — When your ‘smart’ assistant is the ‘friend’ you wished you hadn’t invited to the party.

Where voice activated smart technology is concerned, inviting it into the home really is a big deal. By its very nature, voice activated technology changes the feeling of a space. Its presence is felt, even when it is working well. When it’s not going well, the overt interaction and performative nature of conversing with the technology feels particularly intrusive in the home space. What’s worse than being misheard, misunderstood or downright ignored in your own home by your relentlessly extrovert and often facetious smart assistant? Especially when they get on so well with your partner!? Having to invite them to social events with your friends, that’s what!

Unable to gauge the social temperature in the room, interpret the subtext of a situation, and adjust its tone and responses accordingly, the smart assistant is the obnoxious, socially awkward friend at the party. It does not recognise or learn appropriate responses to different people, responding for instance to Granny as it would little Alice, as it would the party host or a casual acquaintance. It is the facetious friend who will only acknowledge specifics, who wilfully misunderstands some guests whilst entirely disregarding others. It’s the ‘Alpha’, big mouthed friend, constantly interrupting and talking over people. Not being able to read the increasingly exasperated tones in the room, it does not alter its behaviour, escalating frustration yet further.

These feelings of frustration are, somewhat ironically, expressed perhaps most overtly in the tone of voice used. To the human ear, the mounting frustration is evident. To the voice activated object, the clipped sentences, increased volume and huffing and puffing go undetected. Meaning that, at the very least, the opportunity to cut an undesired outcome short is missed — along with opportunities to collect data around frequent pain points, and even, rectify them in the moment. If only the smart assistance possessed the sensitivity to detect and respond to the spectral properties of mounting vocal frustration, for example by using a series of defunct command attempts to trigger different rules or responses like relaxing the syntax, making best guesses, or not breaking the flow of conversation for non-critical information.

In my previous article I discussed the three different motivations for introducing smart tech to the home. Identity Makers, Playful Experimenters and Economisers are all equally likely to experience difficulties with their smart tech, however Playful Experimenters and Economisers are more likely to find smart assistants irritating and disruptive in social situations. They felt their difficulties with the technology exposed them as early-adopter-imposters (more on that in the future), or worse, laid their relationship, decision making, and household management open for judgement and private criticism.

As a result, without a private/public mode or settings which have different degrees of intimacy, early adopters tire of unsophisticated voice activated technology. To avoid the resulting discomfort, participants reported either consciously making an effort not to trigger commands when hosting, or removing the technology from the ‘public’ spaces, such as lounges and kitchens, even though these are the spaces they spend most time in as a household.

“She was constantly being activated and interrupting. I’d be like, ‘I’m talking about you, not to you.’” Emma*, lives with her partner .

“I try to do things subtly via the mobile app, rather than use the voice activation when people are around, but they still see that something has happened because the usual controls are not being used, and will invariably ask about it.” Colin*, lives with his girlfriend.

“When we have people around, we use the Bluetooth speaker for music as we entertain downstairs, and keep Alexa upstairs. We moved Alexa upstairs after a while as she was distracting.” Asia*, lives with her husband.

To prevent the technology from getting relegated to the bedroom, smart assistants in particular need to become more socially sensitive; they need to be able to apply context relevant rules, distinguish between different groups, and even different ‘modes’ of ‘us’. Depending on the time of the week and the events of last night, we might be behaving as energised ‘us’, hungover ‘us’… But how do we design sensitive assistants, without vastly increasing the sophistication of its sensors, and therefore the cost? In the absence of biometric voice recognition, we should consider what other data combinations we can use to activate responses to certain guest archetypes. Through design research, we should observe the spectrum of irritants that arise in different contexts within the home. So too should we consider the rules that we need to create so that smart assistants can switch to appropriate social modes (distinguishing between children being present, formal gatherings, etc.), and in so doing, become more sensitive to the social requirements and avoid the potential irritants of different situations.

Mi casa, su casa…but you may need a visitor pass.

“We keep the fridge fully stocked, and the place clean. I’ve always wanted to be a good host — wanted others to be happy to be here, and to enjoy themselves. People know they can help themselves to a beer anytime, do what they want. People are comfortable when they’re round and treat the place like their own.” Max*, lives with his girlfriend and her son.

People want their home to be a place close friends feel welcome to treat as their own and actively create an environment in which they feel comfortable to do so. It’s a demonstration of care and familiarity that goes both ways — ‘my house is your house and you know how to use it’ is a conversation between close friends and family that is easily had, right? Wrong. A shift in rules sets guests up for failure, obfuscating the previously familiar space, and means they are often unable to do the basics. This can leave them feeling incompetent and foolish, the opposite of the intended hospitality.

“When we have people over, they spend ages in the toilet because they can’t work out how to operate the motion sensor lights. They get confused because they see the old light switch on the wall.” Colin*, lives with his girlfriend.

Is this merely a short-term discomfort, experienced only by the technology laggards in our friendship group? Unfortunately not. With switches and dials moving to mobiles, and some smart home gadgets being controlled exclusively from applications, showing guests what to do and how to use the home is one part only, while providing them the ability to use the home is another. Until they have been inducted into how to operate the home, they may as well be granted a visitors’ pass and asked to sign in at the door.

My smart assistant has a big mouth and talks to strangers.

Once vaguely familiar with how to operate the smart assistant, children, family members or guests are potentially granted indiscriminate access to sensitive personal information with the right command. Whether the command is purposefully or unintentionally delivered, neither Amazon Echo, Google Home or their competitors have an identification process. The smart assistant springs to life and does what it’s told, irrespective of appropriateness or user authorisation.

During the research, I heard of numerous people struggling with indiscriminate input and unauthorised access. Adding erroneous items to combined shopping lists or updating the wrong calendars were the two most common errors.

“His son would just add things to the shopping list, we’d find ‘lots of ‘daddies’ items, like ‘daddies biscuits’ added, when we came to check it afterwards. He’s under five.” Claire*, new mum, lives with her partner.

“I’d say, ‘Alexa, add tennis to the calendar’ — and it will update the joint calendar, but I might want it on my calendar — I can probably create a command to distinguish my calendar from the joint calendar — but will my husband also be able to add to my calendar? I have a personal calendar for a reason.” Asia*, living with her husband.

Whilst these examples are both innocent and harmless, imagine the possibilities. Within this research alone, people were attempting to connect their smart assistants with financial spreadsheets, private emails, and even the lock to the front door. All you need is a voice (regional accent permitting) and you can bypass the usual security methods of passwords, and even front door keys. Now anyone can purchase goods or activate paid-for-trials on your account. True, you can activate a password — but then that too can be overheard. The curious child or the cunning home-alone teen need only utter the words “Alexa, start Amazon music trial” or “Ok Google, buy Vodka and cigarettes”, to rack up costs or even start an impromptu house party. Rather than deactivate voice purchases entirely, wouldn’t it be great to grant people within the home certain purchasing authorisation? Allowing children to buy certain things at the right time might be useful, providing a valuable learning and skills development opportunity.

You, me, us; might smart home technology force us to live with the ghosts of relationships past?

Most smart home technology is designed to learn household preferences and routines so that it can deliver an optimum environment — the ultimate goal being that it eventually fades into the background as control becomes personalised and automated. The insensitivities already observed highlight the difficulty that the technology, in its current guise, faces when attempting to reconcile the multiple needs and preferences of individuals sharing the home space. Smart tech also assumes that people adhere rigidly to a routine, when in reality we live complex lives that regularly deviate from routine and as such require adjustments to the home settings to meet their needs. Heating is perhaps the most common example of environmental conflicts in the home, with partners often having different comfort needs and economic perspectives on energy consumption. When together, we found that couples would more likely reach a compromise. When apart and opportunity arose, thermostats were quickly adjusted to suit personal preferences. The installation of smart energy technology in the home did not change this behaviour, with couples overriding settings and activating ‘boosts’ via their mobiles.

Should smart home technology be able to overcome these obstacles of individual authorisation and routine variation, the value would be great. The emergence of smart-er home technology would be upon us. Of course, this value would not come without long-term personal investment — the smart technology is able to collate data, but it still needs to interact with the user to establish and refine personal preference. But what happens when living situations change? With this level of investment, would the ability to isolate and migrate our personal data profile to another home not be essential? For instance, when teens become adults and leave their family home to progress with their lives. What about relationship break ups? You wouldn’t want to leave your preferences behind as you moved out. Similarly, if you are the person left behind, you wouldn’t want to remain entangled in your estranged partner’s domestic patterns, living in the shadow of their heating, lighting, and streaming preferences. Yet more traumatic; what if a loved one in the household passes away? Social media enshrines the curated self of departed loved ones, leaving the bereaved with some elements of their digital presence to contend with. Where Facebook is concerned there is the option to memorialise profiles, an attempt introduced to counteract potential algorithmic insensitivity. Digital technology forces us to remember the dead. ‘This is the vengeance of the dead that haunt us in their absence’, the sociologist Jean Baudrillard warns. Unless smart-er home technology has the capacity to distinguish profiles, there is scope for a more immediate and intimate perception of loss. To avoid living with the sensory echoes of ghosts, smart-er technology must be able to forget.

Reduced capabilities and life situation shifts; when smart home tech augments rather than disrupts.

Whilst the insensitivities of the current generation of technology is drawn out in the research, it also highlights where it comes into its own. For people living life in much the same way as they previously have done, there comes into question the real benefit of switching to smart home systems. Beyond that of a lifestyle choice or statement, or for a potential 3% decrease in energy consumption, the technology fails to meet a real need. For people whose physical capabilities have been reduced however, the technology can bridge a gap.

For instance; pinned to the sofa whilst breast-feeding, a new mum finds great value in being able to control the temperature in her home remotely, albeit from the front room. Furthermore, when temperature is so key to the health and development of a young baby, the credibility and control of smart home devices provides reassurance at a time of great change and potential anxiety.

Adjusting to a new life as an amputee, one participant in our research relies heavily on smart tech for her independence. For her, smart technology is the reason she can continue living in her home. Marketed to a broad church as aspirational technology, smart home gadgets do not carry uncomfortable reminders of new living situations or challenge people’s sense of dignity as other medical equipment, in all its hear-aid beige glory, might do. But more of this in following articles.

“We essentially use our phones as a remote control, I’d be in the lounge watching TV, and was holding a baby that was either trying to be fed or trying to sleep. I have an ambient egg that I use with the Nest to keep an eye on the temperature.” Claire*, new mum, lives with her partner.

“I had an echo dot before my accident. I used it a little but not much. When I had my accident and returned home I started thinking about lights and using Alexa to control them. I didn’t think of smart tech being that useful until I lost my leg to be honest. But I am an early adopter and love technology. Since my accident I have bought all the Philips hue lights and the smart plugs.” Amy*, adapting to life as an amputee.

The adoption of smart technology depends on its use by people in the context of their everyday lives. The insensitivities of the current generation of smart technologies, as touched upon in this research, are already causing some early adopters to tire of it. To develop technology that becomes a choice for people in the future, it is critical to gain a better understanding of peoples’ relationship with their domestic environment, and with the people they share that environment with. We need to apply the practices of ethnographic research in order to create meaningful, sensitive devices that are responsive to the emotional needs of people in the home environment.

Watch out for more of my articles in our sensitive environmental design series, which will focus on Relationship dynamics in the smart home, Monitoring, privacy & surveillance within the smart home, Care & support within the smart home — augmenting human abilities, Quality of space, and Growing up with Smart Technology’

Siobhan is a design researcher specialising in qualitative research. For the last 10 years she’s been going ‘undercover’ to explore how people interact with the products and services they use, in order to inform design decisions. Her work has taken her to East Africa to understand and design for the healthcare and lifestyle challenges faced by patients suffering with hypertension, to a range of different countries in Asia to explore business culture in all its nuances, and to hangout on rooftops and building sites of tradespeople, to understand the complexities and challenges of their working lives.

Siobhan is responsible for designing the sensitive spaces project for Modern Human. This has been a ‘lay of the land’ piece of research, identifying a number of areas of interest for further exploration. If you would like to explore any of the research topics or findings raised within the Sensitive spaces project, please get in touch Siobhan@modernhuman.co

*All names have been changed to protect participants’ identities.

If you liked this article, click the 👏 below so other people will see it here on Medium. Thank you :)