Designing a future where users control their data (and companies still profit)
--
The cloud cripples your data. Instagram has your photos, iMessage your messages and Google your documents. By splitting up our data, we prevent any AI from truly knowing us as individuals. And by giving away all control, we relegate ourselves to mindless drivers of engagement. What have we gained from sharing our lives with Facebook? If data is the new oil, where are our cars?
I believe that users having control over their data will let companies achieve the full potential of AI. By designing technology to align with user values, we can reignite our confidence in technology that amplifies and advances humanity.
AI works by analysing all the data it has about you. Imagine everything laid out on a huge table: your photos, your movements, your favorite songs, your browser history, your relationships, and so on. It does that for every user, and then looks for common patterns to figure out what it means.
Therefore, an AI gets smarter the more data it has. That’s why companies fight so hard over your data. And it’s why Google and Facebook are at the forefront of AI — they already have a ton of data. But since it’s a zero-sum game, even they have just a slice of all the data you generate every day.
Yet, these algorithms will soon influence every part of our world. Google and Facebook already use AI to decide what you see, and by extension what you think, feel and do. Do we really want to base our behaviour on systems we can’t understand and can’t control?
We need to rethink the relation between our data and the services that use it.
First of all, any solution has to benefit both users and companies. Many talented developers are already working to solve the technical challenges of decentral systems. But I think at its core, this is also a design problem. So I would like to show you how on-device data handling enables new concepts that lead to a better AI and user experience.
We start by keeping our most important data with us, on our devices. So if a company wants to access it, they need to ask you for permission — even if you use it in their app. Apps then are just interfaces to view and interact with our data on-device.
That means you stay in control. You don’t have to decide whether you trust a company with your data. And just as importantly, it means that apps can use each other’s data without concerns for privacy. Let’s call it our Circle of Knowledge.
Circle of Knowledge: All apps in the same room
Apps inside the circle have on-device access to each other’s data. Apps outside the circle keep their data separate. It’s a simple mental model, on which we can build an intuitive interface.
When you drag an app out of the circle, it can’t use data from other apps and vice versa. For these apps, you can then set individual permissions that cut off specific apps or sensitive data.
The circle is built on a set of standardized open data types. Apps inside the circle make a simple deal: They get on-device access to all data of these types from others app in the circle. In return, they have to store their own data of these types on-device (and on-device only) and open it up for other apps to use.
The extent of data types could grow over time, starting with the most common data (e.g. status updates, stories, messages, contacts, locations, photos, videos, purchases, notes, documents, websites, reading lists, movies, books, music, podcasts, smart home data, todos, health data, workouts, events and reminders).
Data no longer lives in a sandbox then. Apps can use all data created by the user, regardless of its origin. And you can switch between services without losing data. So …
- YouTube knows that you just watched Coco on Netflix.
- Safari knows what’s left on your Things to-do list for the day.
- Snapchat knows about the conversations you have with your best friend on WhatsApp.
- Amazon knows the jacket you raved about on Instagram.
- Evernote knows about the hundreds of notes you made in Google Keep.
- Spotify knows the song lyrics you looked up with Safari.
- Photos knows your friends are reminiscing about last weekend’s hiking trip on Facebook.
By keeping data on-device, developers can build more personal and modular apps. So when you install a new app, it doesn’t have to start with a blank slate. The app already knows you: what you have done before, how you use other apps and how you want to live your life.
It’s like all apps are in the same room.
That’s where on-device AI comes in. All data from apps inside the circle can be used for system-wide machine learning. Data sets that are today spread across dozens of services, each with their own little AI, can be combined on-device to create a comprehensive view of your behaviour. This becomes possible through technologies like the iPhone’s neural engine and Google’s federated learning, which have made on-device machine learning very capable.
Fueled by on-device data, on-device AI can eliminate the viability of traditional, proprietary data collection. Apps which want to be part of the circle would be required to themself support on-device data and to build on open data types. Over time, this could convince even entrenched companies to give up on collecting all of your data.
When you first open an app that supports on-device access, you get a confirmation that it will be part of the circle. You can toggle it off to keep the app outside the circle. It’s a simple reassurance that everything is under control.
This mental model is more playful and natural to think in than a list of permissions, and it’s reinforced with every new app you install. The user experience leads to users trusting and preferring apps that use on-device data, further pushing companies to move away from legacy data collection.
However, storing data on-device requires us to reconsider how we share it with others and sync it across devices.
You can be in control of your data and still benefit from the convenience of the cloud. Since data in the circle is processed on-device, it doesn’t have to be readable on cloud servers. Instead, we use end-to-end encryption to make sure it’s only accessible by the people you share it with.
Your data is then independent of any specific cloud provider. It doesn’t matter whether it’s stored by Google, Dropbox, a distributed network or a small local provider. They would all be part of the same standard, acting as commodity storage for your encrypted data. You can keep everything in one place, or switch cloud providers for data types you want to keep separate.
This system enables user privacy and control, while being every bit as convenient and powerful as existing cloud services.
Permits: Treat data like money
Sometimes, we do want to share our data with external organizations. Lyft might want location data to coordinate its cars, Facebook could keep around features that rely on data collection, and doctors could request health data.
These cases require your data to be stored and processed on a server you have no control over. Handing over your data is hard to reverse, and can have lasting consequences. So when it happens, you should know exactly what’s at stake.
In many ways, sharing your data is similar to a payment transaction. You wouldn’t buy something without knowing what you get and how much you pay. Likewise, you should understand how data will be processed (and what that means for you) before giving it away.
What if we treat data like money?
Users are familiar with the significance of this interface from Apple Pay and know to treat it different from normal notifications. It expands to an overview in the settings, where you can see all companies you have shared data with, look into details, and revoke permits.
The split between on-device access (through the circle) and external sharing (through permits) is critical for helping users understand and control how their data is handled. Powerful on-device access as the default lessens the need for external sharing, allowing us to shine a spotlight on the choices that truly matter.
This is about more than privacy. Tech companies have lost the trust of their users.
Today, users feel a deliberate absence of clarity about what happens to their data in “the cloud”. It allowed Facebook and others to raise empires without worrying their users. But it has now backfired dramatically, as confusion has grown into helplessness and wariness of all new technology — regardless of the intentions behind it.
Companies need to treat users like adults and design systems that help users understand how their data is handled. Only then can they rebuild the confidence in technology that they need for a future where data is inherent to everything we do.
Profile: A map of yourself
Today, every company has a few pieces of your data. That system might work for making you click on ads, but it can’t power an AI that truly understands you. Imagine what would be possible if your phone knew all about you.
Apps inside the circle use and contribute to a system-wide profile. It uses on-device AI to analyze the open data types and metadata reported by apps in the circle. The profile keeps track of all your interests, which apps can use for their own AI-based features (without any data leaving your device!). It lets users understand and engage with AI through a single interface.
The spatial interface offers a playful way to explore what the AI has learned. Instead of strict categories and rankings, the profile places related interests near each other and highlights common themes. It’s a map of yourself — it grows as you grow, mirroring how you think about your interests.
Tap on an interest to view the data behind it. A calculated interest level shows how each interest is weighted. You can raise it through boosts, or remove the interest altogether.
It’s a good start. But in the long term, the profile could evolve to be about much more than interests.
We need technology that knows us not just as engagement numbers. Instead of distracting us from our goals, it should empower us to live the life we aim for. This requires an AI with deep insights into our life. Insights that can’t be gathered by a single app or service.
Technology should understand our behavior beyond the surface of impulsive taps and swipes.
As AI technology advances, the profile could eventually include your values, opinions, preferences, habits, routines, goals, desires and relationships. Then, the AI wouldn’t just know which content best captures your fleeting attention. It would have a real understanding of what you do, why you do it, and how it can help.
The result would be an OS that always knows your status, and the context of what you are trying to do. Apps can then use these insights to tailor their experience to your current situation.
- Slack knows when you are “in the zone” and doesn’t interrupt.
- Google Maps knows that you spent the morning planning your trip to Amsterdam.
- Mail knows you are trying to be more patient with your coworkers.
- Facebook knows that you are scrambling to finish the presentation for tomorrow.
- Photos knows that you have just gotten over the person you spent this day one year ago with.
- Things knows that you prefer to run errants after lunch.
- App Store knows which apps worked out best for people with a similar profile.
- Siri knows that the phone is placed on the table at a café right now, where you are meeting with your colleague and longtime friend David to go through the notes from yesterday’s design review.
This kind of profile becomes viable through on-device data and on-device AI. It’s a long-term foundation for technology that thrives when it supports your values, that rejects clickbait, made-up news, and sensationalism.
Power over your profile is crucial then. As the role of AI grows, so does the importance of control and transparency. If we want an AI to understand us, we must first make sure that we understand the AI.
Transparency: Siri as the face of AI
You shouldn’t have to stumble through settings to find out how apps use your profile and your data. Transparency is most useful in the immediate context of your actions.
On iOS, Siri is the face of AI. It represents both system-level features and third-party apps, and it’s always just a button press away. Siri should be accountable. It should tell you why AI decisions are made, how your data is handled, and help you change these behaviors.
- Hey Siri, why is Facebook showing ads for standing desks? ➔ Here’s why I think you might be interested in standing desks.
- Hey Siri, which companies have access to this photo? ➔ Google, Inc. has access to your photos. Here is the data permit.
- Hey Siri, why is it cold in here? ➔ You usually don’t come home until 7PM on Mondays. Do you want me to turn up the thermostat?
- Hey Siri, I don’t want any of this to be used by other apps. ➔ Okay, I have removed WhatsApp from the circle of apps that can use each other’s data.
- Hey Siri, I want to eat more Mexican food. ➔ Okay, I have added Mexican food to your interests.
Siri is also available inline as you interact with your data or with content shown through the AI.
These UI metaphors for controlling your data and the AI scale across different contexts, building a cohesive mental model for the circle, data permits and your profile.
Summary
Let’s recap the ideas we went through.
- The cloud cripples your data. AI needs all your data in one place. On-device instead of cloud. You stay in control.
- Circle of Knowledge: All apps in the same room. Apps in the circle can use each other’s data. Open data types. No more sandbox. On-device AI through federated learning. Apps need to contribute to participate.
- Data Permits: Treat data like money. Encrypted storage through cloud providers. Know what’s at stake when sharing data. On-device as the default allows for detailed permits. Transparency leads to trust and confidence.
- Profile: A map of yourself. Interests. System-wide for apps in the circle. Spatial interface. Evolves to learn our values, habits, desires. Apps know the context. Foundation for understanding beyond impulsive engagement.
- Transparency : Siri as the face of AI. Transparency is most useful in context. Siri should be accountable. Through voice and inline.
Cambridge Analytica. GDPR. Time Well Spent. This year, it became clear how important it is for us to be in control of the technology we use. The common thread is a failing of technology to treat us as individuals, to respect our rights and support our values.
This used to be an issue of privacy, but we have moved well beyond that. Today, we have a unique opportunity to connect the dots. Users feel helpless against increasingly imposing technology, while designers and developers are sick of selling out their users. On the other hand, we have decades of research on decentralized technology, powerful computers in our pockets and AI technology that is severely limited by the cloud.
Data is mighty. It can empower us, but it can also be used against us. The path we are currently on leads to a single company owning all of our data. This would crush competition, our freedom and democracy. We need to be deliberate and thoughtful in building the foundation of technology for the next decades.
This will require broad collaboration between companies. Ultimately, the goal would be to create open, cross-platform standards for the ideas outlined here. But the principles behind my designs can take many forms. Together, we can figure out systems that let us control our data, use AI to empower users, and build the future of technology without sacrificing our humanity.
View the original article with more detailed visuals and additional content at thecloudfall.com
