How to Build Trust through Responsible Bot Design
As we quickly design and deploy tools to slow the spread of COVID-19, assessing the impact of our crisis-driven solutions on our future is critical, particularly as norms around data and technology rapidly shift.
89% of Americans now say they would be willing to share personal health information with employers to help keep themselves and their coworkers safe from COVID. What happens to that data, however, when the threat of infection diminishes? Can we ensure technology helps, rather than harms, the people it serves?
Contact tracing is a well-known, low-tech approach to managing viral spread— it serves a critical public health function, especially while broad vaccination efforts are ongoing. Mostly it’s done manually, which means that when a person tests positive for COVID-19, a trained volunteer or medical personnel calls anyone with whom that person may have come into contact. All of this is very labor intensive.
Our out-of-the-box contact tracing solution in Work.com (Salesforce’s suite of back-to-work and back-to-school tools), though, faced a two-fold challenge:
- Contact tracing efforts often fail because just 19% of Americans say they generally answer their cell phones when an unknown phone number calls.
- Receiving news that you may have been exposed to a potentially deadly virus from anything other than a caring human being could be both strange and unsettling.
In other words, our design challenge became: How might we increase contact tracers’ effectiveness while preserving the most human elements of their work?
Principal Solution Engineers Gerard Iervolino and Chris Gilmore had a solution in mind: Tracey Bot. With messages sent via SMS or another channel, Tracey augments manual contact tracing through automation, assisting contact tracers’ efforts to track down and limit the spread of the virus.
Now, our experiences with bots — computer applications that perform human-like tasks — can vary widely. We’ve all had the experience of getting stuck in a maze of menu options with no hope of speaking to an actual human. It’s frustrating and disempowering, to say the least. As a Principal in the Ethical AI Practice, I knew it would be critical to apply our Ethical Use principles and Ethics by Design methodology to create a humane contact tracing bot experience.
“Working with Yoav and the team truly leveled-up our design process. We learned it’s important to build technology from the perspective that our customers are different from us in ways we cannot always anticipate. ”
– Chris Gilmore, Principal Solution Engineer of Tracey Bot
These are 5 ways to add an ethical lens to a design process, reflected in the decisions we made in building Tracey Bot:
(1) Figure out the right job to be done
Despite the need to quickly deliver a solution, since global contact tracing efforts were already ongoing, we took our time to consider the ripple effects of making a bot like Tracey. The result was a decision to use Tracey only to power manual contact tracing efforts — the solution only sends a message in conjunction with human judgment about a contact tracing list.
What’s more, we carefully allocated certain jobs, but not others, to Tracey Bot. For example, calling someone every day for fourteen days to check on their symptoms and ask them to take their temperature is the perfect job for a bot that excels at repetitive and easily automated tasks. Conversely, walking someone through the complicated emotions that come with possible COVID-19 exposure is really only suited for humans. It’s important to remember that not every problem has a technological solution.
(2) Make sure to be inclusive and design for real people
In the context of the COVID-19 pandemic, we are all doing our best to design solutions for a problem unlike anything we’ve ever experienced. Unfortunately, in the United States, contact tracing efforts have, for the most part, failed to curb the spread of the virus.
There are three realities that have not been working when it comes to manual and proximity contact tracing — most people don’t pick up when a tracer calls, many have not opted into automatic solutions, and not everyone has a smartphone that allows them to participate. However, 90% of text messages are read within three minutes of receipt.
So, in the hopes of making contact tracing inclusive and accessible to everyone, we made certain that Tracey Bot could be used across a variety of channels like SMS, chat, WhatsApp, and more. At the end of the day, we are designing for reality, rather than the ideal.
(3) Be honest and transparent with users
We believe that, as simple as it may seem, a bot needs to identify itself as a bot, and any attempt by a bot to pretend to be a human agent deceptively undermines trust. As a result, our Acceptable Use Policy at Salesforce states that any Einstein bot must immediately disclose this information in the name or in the chat itself, hence the name Tracey Bot.
(4) Design interactions that engender trust
The last thing we would want would be for a user to receive a Tracey notification, only to get stuck in one of those maze-like menus I mentioned before, without the ability to speak to a person. From the first message you receive from Tracey Bot, we hope that the experience feels humane:
“Reply TRACE to this message to step through an automated contact tracing procedure. To be connected with a live contact tracer, reply AGENT.”
This message empowers the user to choose the channel they move through the contact tracing process — continue the bot interaction, speak to a human, or opt-out completely by simply not responding. At any time, the user can send a ‘STOP’ message to opt-out of all communications with Tracey.
Another poor outcome would be receiving a message about possible COVID exposure and having to then wait on hold interminably to speak to someone — or not having an agent available at all. As a result, the guidance we provided in the install guide was clear — only send a message if an agent is going to be available and estimate a wait time.
(5) Humanize the experience
One of our biggest early concerns was ensuring that we could appropriately negotiate the tension that comes with receiving this somewhat scary health news from a non-human bot. As in, “Hi, you may have been exposed to a potentially deadly virus. Press 1 to hear more.” That’s really weird and likely unsettling. We wanted to mitigate that discomfort to the extent we could, while also solving the problem of needing to reach people for public health and safety reasons. So we carefully crafted Tracey’s messaging scripts using templates from the Centers for Disease Control and with guidance from our Conversation Design Principal, Greg Bennett. An example of this was in Tracey’s first message:
“In order to stop COVID-19 from spreading in the community, we follow up with people who have been potentially exposed. We want to work with you to help you get the care that you may need.”
There is power in clear and caring communication.
“Contact tracing is about creating a map of people’s relationships, and that’s very intimate. And it’s coming from a sterile form of communication, a bot. Conversation design can bridge that gap by crafting messaging that is authoritative but not scary and caring but not too friendly. This is similar to the messaging nurses and doctors use in their bedside manner. We want Tracey Bot to have a good bedside manner.”
– Greg Bennett, Conversation Design Principal
Building the Future We Want
When building the interface between employers and employees, or students and schools, designing for relationships is key in creating interactions that foster trust. Truly, trust is more important than ever because the work and school relationships have fundamentally changed since the pandemic began. Employers and schools have to take responsibility for keeping their employees and students safe in the face of new challenges. So while employees are more willing to share sensitive personal health information with an employer, an employer should be explicit with employees about what data is being collected, its purposes, the retention policies, and how the data will be secured. That type of mutual accountability will build relationships that stand the test of time.
It’s imperative we remember that the experiences we craft now are likely to persist. How we choose to build technologies has important implications, since the consequences of our choices today will stretch far beyond this pandemic. If we are building our own future, then ethics should be at the root of all we do. As designers and technology creators, it is not enough to deliver only technological capabilities — we also have an important responsibility to ensure that our solutions are safe and inclusive for all.
Interested in Learning More?
Learn more about Tracey Bot and how to implement it as part of manual contact tracing with Work.com.
If you’re curious about Ethics by Design, take our Responsible Creation of Artificial Intelligence Trailhead module and see how you, too, can make a more ethical product.
You can also learn more about Salesforce’s Office of Ethical and Humane Use or Ethical AI Practice.
Finally, learn more about Salesforce Design at www.salesforce.com/design, follow @SalesforceUX on Twitter, or check out the Salesforce Lightning Design System.
Thank You
Huge thanks to the incredible team with whom I am privileged to work and who collaborated on the development of this piece: Gerard Iervolino, Chris Gilmore, Rob Katz, Madeline Davis, Margaret Seelie, Greg Bennett, Anna Kowalczyk, Sahara Ali, Paula Goldman, Kathy Baxter, and many others too numerous to name here.
Follow me on Twitter.
