Slowing Down To Speed Up
A cross-discipline collaboration to ethically leverage AI and improve in-person shopping experiences
By Juliette Weiss, Rick Barraza, and Jenny Williams
Think of your last satisfying in-person shopping experience (and if you have to reach back pre-pandemic, that’s ok!). Maybe you were browsing your favorite bookstore, pouring over the staff picks until an intriguing title leaped out. Or you got everything on your grocery list in one go, without having to ask a staff person if they had more in the back.
These experiences — delightful, efficient, fulfilling — are possible because store employees think carefully about what you might need, and then build layouts and displays to help you get what you want. But when it comes to understanding and anticipating shoppers’ needs, in-person retailers have traditionally had less data to leverage than online retailers, who have access to site traffic and other analytics.
AI can help. This week, Microsoft announced the preview of Dynamics 365 Connected Spaces, coming in December. Connected Spaces harnesses data from video cameras to help companies understand their physical spaces better: how people move and interact, and when and where staff might be needed to open a new register or restock shelves. These signals can make spaces more efficient and empower store employees to provide exceptional customer service — all of which improves your shopping experience. Connected Spaces can be applied to manufacturing and warehousing, too, providing support up and down the supply chain.
Because Microsoft is committed to building AI responsibly, the Connected Spaces product team collaborated with Ethics & Society to uphold our responsible AI principles from the start. As an internal product and design agency, Ethics & Society guides technical and experience innovation towards ethical, responsible, and sustainable outcomes. Together, we were able to:
- Deep dive into key considerations around privacy and transparency
- Build a UX that gives store employees more control over the AI system
- Leverage cross-disciplinary expertise to help us achieve better outcomes for individuals and companies
Protecting privacy for shoppers and employees
At Microsoft, we pride ourselves on being faithful stewards of people’s data. We want people to feel respected by our technologies — “optimizing for human dignity,” as we sometimes say. Since Connected Spaces uses cameras to make inferences about the physical environment, we needed to be especially thoughtful in how we would protect the privacy of both shoppers and employees who might be captured in video footage.
First, we looked at the technology itself. Connected Spaces was designed to preserve the anonymity of individuals as much as possible by focusing on movement patterns and locations rather than identifying individual people or analyzing facial characteristics. Additionally, the video stream may be processed automatically on premises or securely on the cloud, based on the customer’s privacy needs and concerns.
Second, we wanted to make sure shoppers and employees can understand in a meaningful way how the system works, what observations are being captured by the cameras, and how these observations might be used. Our user researchers invested significant time and energy in understanding people’s concerns: What questions did people have about the technology? What privacy protections made them feel more comfortable and secure? How could we explain complex technical concepts in ways that people could easily understand?
Since both customers and employees are potentially affected in different ways, we used our research findings to create guidance for store owners to best communicate and disclose these topics to each group. This guidance helps store owners create a disclosure strategy that:
- Raises awareness about the use of Dynamics 365 in their space.
- Helps people understand how the technology works and what privacy protections are in place.
- Adapts to individual people’s communication needs, including considerations of fluency, literacy, and accessibility.
- Empowers employees to both understand and explain the system to others
Empowering retailers through human-AI collaboration
On the retail side, we then thought deeply about the Connected Spaces interface: who would use it and how we could empower them. Building robust, highly accurate AI systems has been a core priority for Connected Spaces. But there will always be situations that an AI can’t predict. Maybe traffic patterns in a store changed suddenly because it started to rain, and shoppers flocked to a covered entrance. Maybe a traffic zone was drawn incorrectly during setup, so the system’s predictions are off.
To accommodate these kinds of mismatches, we explored ways of enriching the user experience to give more control to store managers to optimize Connected Spaces for their unique goals and circumstances. Data scientists, designers, engineers, and store employees collaborated on exploring various interaction patterns around AI collaboration. One key finding was that users needed to have a better understanding of how Connected Spaces works in order to use it effectively.
For example, when a user is telling the system what part of their store they want Connected Spaces to pay attention to, they are prompted to draw a “zone” in the UI. In early iterations, many users drew the zone over a promotional display since that’s what they wanted to track. But the system actually looks at where people are standing — requiring the zone to be drawn on the floor.
By adding an avatar of a potential shopper and a grid on the floor to the UI, users were more easily able to infer where to draw their zone, without having to read tooltips or documentation.
Once a foundational understanding is established, users are better equipped to gain insights from the system status at any given time. More importantly, they can better anticipate where the system might not be meeting their unique expectations. Finally, by providing a tailored set of controls in the interface, users can tweak their particular settings to better align with their business goals.
Leveraging cross-disciplinary expertise
All of this exploration required a wide range of expertise. As Jonah Sterling, Head of Design for Digital Transformation, says, “Design can’t work in a bubble. We need researchers, PMs, data scientists, and legal partners. All disciplines in the product team own the UX and it’s imperative that we work together to get it right.”
To design for Connected Spaces, we started with the model makers and data scientists. As a crew, it was essential to deeply understand how the model works and where it can go wrong. What are the false positives? What are the false negatives? Designers from Ethics & Society worked directly with data scientists to visualize how the different ML models interacted behind the scenes, which surfaced some of the misalignments that could occur.
Misalignments might come from variations in how users set up the system, environmental factors such as a mannequin blocking what the camera can see, or people trying to use the system in ways it wasn’t intended. These learnings have informed improvements across the product, from how we guide users during setup, to which analytics we include in our dashboard reporting, to driving continual improvements to the models themselves.
We also partnered closely with our legal team as legislation around the globe is ever-evolving when it comes to AI. There are myriad examples of AI gone wrong, especially with computer vision and privacy, so we need to take note of these and ensure we are understanding the new challenges and emerging legal developments.
Slowing down to speed up
This kind of deep ethical exploration requires two things: time and collaboration. As Mira Lane, leader of Ethics & Society, says, “Sometimes we have to slow down and be intentional in the moment, so we can run fast later.”
It’s impossible for a single designer to tackle these kinds of complex challenges alone, yet it’s imperative that we prioritize ethical considerations. Whether it’s a large company investing in a horizontal team akin to Ethics & Society or a start-up hiring a single person whose job is to ask the hard questions, there are many ways to activate ethical thinking in product teams. Our blog on Responsible Innovation is a great resource, with concrete ways to reduce harm and strengthen your moral imagination.
As technology continues to evolve, so will the way we approach building it. We’re always learning, and we hope you’ll share your learnings with us, too.
For more information about Dynamics 365 Connected Spaces, please visit our website and blog.
A dedicated group of people came together to bring this effort to life, and we’d like to thank everyone on the Ethics & Society and Connected Spaces team for your engagement!
To stay in the know with Microsoft Design, follow us on Twitter and Instagram, or join our Windows or Office Insider program. And if you are interested in working with us at Microsoft, head over to aka.ms/DesignCareers.
