Smart living, not smart devices

PART IV: We still need technology advances

After outlining an ambitious agenda for smart homes, let’s finally look at some of the technology advances needed to realize this agenda.

The rapid evolution of sensor and device technology, the dramatic cost reduction of the hardware, and the amazing development of new artificial intelligence algorithms and user interfaces are providing a great foundation for truly personalized smart living services. Progress towards this agenda requires creative ways to compose these technologies in ways to address consumers’ needs in ways they find useful. As with all innovation, the path from technology advances to innovative uses takes time and experimentation, as well as communication to socialize the advances, leading to broad acceptance.

However, given the current state of technology and the market, substantial technology advances are still required to turn into reality the smart living dreams suggested by our agenda.

Context fusion

True smart living solutions act with a keen understanding of the context of their actions. First, they determine a user’s intent. Next, they decide what context data are relevant for this intent of this user in this situation. Finally, they interpret the context data and reason about how to achieve the intent. Context can encompass many aspects, from personal preferences to history, available sensor data, general knowledge, available devices, risk, and potential alternatives. Today’s artificial intelligence (AI) algorithms interpret data from single sensor streams often almost perfectly, such as in speech recognition or face recognition. However, substantial progress in AI capabilities is still required to interpret combinations of complex situational data. Sometimes, this capability is called broad AI.

Interoperation and autonomy

Without a degree of self-configuration and self-orchestration, it will be difficult to implement smart living solutions, given the great variety of personal preferences, living situations, home configurations, and available devices. “Manual” configuration and programming of the devices for satisfactory function is too complex for most people, and there are not enough engineers around to do the job. The lack of standards presents a huge challenge in coordinating actions across multiple home IoT devices. To overcome this state requires standards in the areas of radio communication, data formats, APIs, functional descriptions, application ontologies and semantic models, identities, security, permissions, and of possibly even more technologies. Once these standards enable interoperability of devices, we can use them to advance the orchestration of devices to accomplish a goal. In the ideal case, devices will recognize each other and collaborate autonomously, in a form of self-orchestration. As a colleague asked several years ago: “when two ’things’ encounter each other in the Internet of Things, how do they know who they are, what they can do, what they are allowed to do, under what conditions?”

At the same time, the locus of control of smart living services needs to be inverted from today’s central control by service providers in the cloud to control by users in the home who selectively subscribe to and authorize individual cloud services. The need for this inversion of control is described in an earlier blog. To enable it, the processing power of devices must increase and become more economical to locally support AI functions such as the context fusion described above. Natural user interfaces and a high degree of adaptive automation will enable this transition.

Local control of smart living functions eases security and privacy concerns. Data can be kept locally, service providers will see only data they absolutely need to provide their services, and strong protections like encryption and firewalls can provide additional security.

Truly human interfaces

User interfaces for smart living solutions must align with the tasks they are intended to perform, rather than with device capabilities. In particular, user interfaces should make full use of all human senses: hearing, seeing, touching, smelling and tasting. While speech control is currently a hot technology, its power for smart living solutions is limited. Gestures or mere presence are more appropriate in many cases, such as with lighting or temperature control. Gas sensors may indicate dangers and trigger alarms or even mitigation measures. Analysis of drain water from clothes washers can be used to reduce detergent consumption. Ideally, the user interface for control should disappear for many functions: the system will observe what we do and learn from our preferences. Naturally, there still needs to be a way for us to intervene and correct.

Often, smart home services need to signal events, confirm commands, or give other indications of their state. Again, this output signal should use the appropriate modalities, from acoustic and speech, to visual and text, and haptic feedback. The right modality depends not only on the activity, but also on the situation. In some cases, an audio message that the laundry has finished may be appropriate. However, when visitors are present or during dinner, a simple haptic signal from a wearable device may be more appropriate.

The way forward

The agenda presented in these blogs is very ambitious, and we’re not going to achieve it in one giant step. Observation of how technology innovation is accepted by consumers shows that we have to approach this agenda in many steps. New solutions take time for consumers to accept — it’s a multi-year journey. Acceptance for home IoT devices has been increasing, but there are still obstacles to overcome.

As with all true innovation, the smart home takes time for socialization among consumers, and for the shaping of the associated technology. When the right value proposition comes along, the market can take off quickly.

Many of the home IoT devices propose solutions to problems that people don’t know they have, or that are not really important to them. Rather than starting by proposing totally novel experiences, we have to solve problems that consumers have today in ways they understand. Simplicity and reliable and transparent performance are more important than flashy appearance. Once consumers value a solution, the experience can be expanded incrementally to provide additional function that may go far beyond the original solution.

An example for this approach is the introduction of the iPhone. In the early 2000s, a great variety of personal information management devices became available, from the Palm Pilots to the Apple Newton, but none gained acceptance beyond enthusiasts. This changed in 2007 when the iPhone came along and created a huge market, not just for Apple, but for the entire smart phone industry and its application and service providers. Steve Jobs announced the iPhone as a phone, a music player and a browser. These are functions that people at the time understood well and used daily. Combining them in one device made sense, in particular as they came with a simple and intuitive user interface. If Steve Jobs had introduced the iPhone as a way to pay for groceries, to look at who’s at their front door, or to get a moisture alarm from their basement, most people would have been puzzled about how this made sense. Today, we use smart phones for these functions and more.

Many manufacturers and service providers are hoping for a market transformation like this with the smart home. We have not yet seen the iPhone equivalent for smart living. As the diversity of potential services is great, its emergence will take time and evolution. Some starting domains are home security, insurance, helping elderly to live at home longer, and entertainment. These are all areas where companies are spending substantial creativity, energy and money, and where personalization and automation using artificial intelligence can make the services more effective and less costly. With time, the best will succeed, and we will enter the world of smart living.