How We Used the UX Design Process to Make AI More Approachable
The balance between functionality and usability can be a struggle for enterprise products. As Watson Assistant has grown and progressed over the last decade, the product has become more capable, but the refinement of the end-to-end experience hasn’t always kept pace.
Feedback from our users helped us recognize that we were in need of a more streamlined, user-friendly, user-focused experience that would make implementing and using Watson Assistant not just approachable but delightful. This was particularly important for new users who may have been intimidated by the complex, technical nature of artificial intelligence, machine learning, and natural language processing.
We also recognized that, as designers, we wouldn’t be able to do this on our own. Design thinking and the Agile development process encourage communication and collaboration, and we saw that these approaches were key to delivering an outstanding experience for our users alongside a remarkably capable product.
The first step was to align with our product management counterparts to develop design roadmaps and research plans. Because our product managers are focused on the users first and work regularly to nurture those relationships, we felt like we had an ongoing open line of communication that made getting feedback from our users easier and faster.
Likewise, we partnered with our developers and engineers to completely revamp Watson Assistant’s interaction models. We made a point to work in a more iterative way (we call it The Loop) that encouraged constant feedback and reflection.
A glimpse at our research
Until recently, setting up Watson Assistant typically meant that a team of people from a variety of roles would need to play a part. Subject matter experts were needed to provide the material from which Watson Assistant would retrieve its answers. Designers weighed in on branding, interaction patterns, and the overall look and feel. Engineers would take the content and program it as a dialog tree with choices and dead ends, and front-end developers would plug it all into a website’s HTML, CSS, Javascript, and backend architecture.
As we were planning the recent improvements, we heard from the people on these teams. They were mostly happy with the end product, but getting to the finish line was typically fraught with problems. If changes needed to be made, they’d have to start the process from the beginning. And for the teams that didn’t have the resources or the staffing to dedicate to every role, things were more complicated and time-consuming.
According to our UX Architect Tom Roach, “We needed to find a way to help [users] get to that point radically faster and radically cheaper. They need to be able to build an assistant in a week, not a year.”
We made it a priority to design the product so that any person with the subject matter knowledge could build directly in Watson Assistant. The fact is that the people building Watson Assistant and working on it every day are coders and non-coders alike, and we wanted to make it welcoming to users of all disciplines and skill levels.
“[Users] need to be able to build an assistant in a week, not a year. “
— UX Architect Tom Roach
These people were at the center of our research, our planning, and our designing over the past year plus. From the research, we created personas that represented our stakeholders at every level, and those personas were at the heart of every design decision we made. Every change and every tweak had to be in the best interest of “Tanya”, the subject matter matter expert (and the person building out the content within Watson Assistant), because she had to deliver results for “Cade”, the end user (the person interacting with the chatbot on an organization’s website). Tanya also had to make sure she was able to clearly communicate her successes to the rest of her team: “Paula”, the product manager; “Dinesh”, the data scientist; and “Deb”, the front-end developer.
Our user research lead Ashwini Kamath and the rest of the research team sought to build on the work done by teams in Watson Assistant’s past: “We recognized that these [users] were not working by themselves. They were working in partnership with these other people [on their team]. The subject matter expert is the one that has the knowledge … so we should be enabling them.
“We did a lot of work around understanding what Tanya’s needs were, what her workflow looked like, what did the collaboration between her and her developer look like. I think, at the heart of it, they realized that the functionality was powerful. We just needed to package it in a way that it would be understandable and usable for a wider audience.”
“The subject matter expert is the one that has the knowledge … so we should be enabling them.”
— UX research lead Ashwini Kamath
What’s new
With all of these personas in mind and with a great deal of user research and feedback along the way, we’ve made big design and user experience (UX) changes to Watson Assistant so that it’s more user friendly for all types of users; less complicated to get started and to maintain; and easier to extend, integrate, understand, and improve.
Getting started
We want teams and individuals to feel empowered, enabled, and capable when it comes to creating their new assistant. To help with that, we’ve made it easier for users of all experience levels to design and create an assistant in a way that meets their users’ needs as quickly as possible.
Users are presented with a dashboard screen that’s been redesigned so that’s it’s easy to track the design and deployment process. It’s now much easier to see what you’ve done and what remains, and it follows the order of the steps we recommend to get your assistant up and running.
A new way to build your conversations
Watson Assistant now speaks in actions and steps rather than in a tree-based “if-then” system. The process is now akin to writing dialog or scripting a real-life conversation. It’s less complicated and more straightforward for subject matter experts to understand if they choose to (or have to) create their assistant without relying on the help of a developer.
We did this because we understand how intimidating topics like machine learning and conditional logic can be for both new and experienced subject matter experts. By making the creation flow feel more like a conversation and less like a logic tree, we hope that assistant creators feel more comfortable and capable when creating the actions and steps their users need.
Personalize before you publish
One of the existing features of Watson Assistant that we’ve kept intact is the ability to customize and personalize the look and feel of your assistant without having to write or edit code.
We know that creating and launching an assistant from scratch takes time and money. That’s why we’ve included this template built with the same Carbon design system we use internally at IBM. If you can devote fewer resources to coding and deploying your assistant, you’re free to invest more resources in the things that make it your own like the content and the external channel integrations.
A new, more customizable preview & deployment
We heard from users that they wanted the option of seeing their assistant in action without having to go through the process of submitting pull requests and reviewing a litany of small changes. In addition, we wanted subject matter experts to have the freedom to fine tune their content and conversations alongside their assistant instead of having to navigate a codebase or a complex set of if-then statements.
At least 85% of users we tested had difficulty telling the difference between the old preview page and the preview panel in our new Actions workflow, so to show Watson Assistant as a true preview, we’ve superimposed the web chat portion of the assistant on a webpage instead of a blank screen.
This new preview page takes the assistant you’ve designed and lets you see how it would look and perform. This is where you can easily test and review your conversations, actions, and steps. Once you’re satisfied, you can share your work with others quickly via a public URL. In the coming weeks, we’ll be improving this feature even more: you’ll be able to see what your assistant looks like on your website.
For most users, Watson Assistant is more than just a chatbot on a website. Most implementations are connected to other channels like the phone integration, WhatsApp, and Facebook Messenger. Our users told us that these essential connections weren’t always easy to find, so we added a link directly to the integration catalog if you want to connect (and then preview) additional assistant channels.
Early designs that we tested with our users indicated that when an assistant’s live and draft environments were both on the same page in the workflow, users were unclear how the two differed. This drove us to design two separate pages so that users can see clearly where the live environment lives and where the draft environment lives. We tested this and found that all users were able to find both pages and understood that they were separate. This means that once you’re ready to deploy or make changes, you can do so without fear of breaking your assistant.
In addition, the new publishing model incorporates versioning, so you can review (or deploy) your changes in both draft and live environments with the option to revert if necessary. We’ve added change tracking as well, and there’s an option to save in-progress changes in case you need to take a break and return later.
Understanding your usage
We’ve also redesigned the analytics dashboard in Watson Assistant to make it much easier to find and fix the gaps in your new assistant. In addition to being able to see how your assistant is performing at a glance, you can now dig deeper into specific conversations and actions to see what’s working and, more importantly, where your users are getting lost or frustrated.
The potential complexity of data and analytics can be overwhelming. We sought to make these elements as approachable as possible in order to help subject matter experts make rapid yet effective improvements for their users. A more approachable set of analytics that’s easier to understand and act upon means more opportunities to improve your assistant’s performance.
Help us design what’s next for Watson Assistant
We’ve spent more than a year working on these improvements, and we hope that you’ll both enjoy what we’ve done and help us continue to improve. Give the new Watson Assistant a try and let us know what you think. We’d love your feedback.
Thank you to all of the designers (and I would be remiss if I did not also thank those in product management, content, development, marketing and more!) that worked so hard to make this happen!
Thanks to Will Fanguy for help with the content and design of this post.
