Designing A Machine Learning Compliance Product / Part 4
User Research
Finally…
Getting access to interview users in an enterprise environment is a tall order. It ended up taking me over 4 months from the kickoff meeting till my first interview. I had to give numerous presentations and attend 1:1 meetings with key stakeholders to convey the value of contextual user interviews. This may seem like an unusual request if they’ve never heard of UCD (User centered design).
Subscribing to UCD ensures that user happiness and acceptance of the product is high. Instead of forcing the user to conform to a product workflow, It’s much easier to match the product workflow to the user.
I employed the following tactics to spread the gospel of User Experience Design:
- Schedule multiple meet and greets w/ potential users, project managers, product owners
- Educate project managers all the way to sub C level bosses
- Create educational presentations about user experience research and best practices
- Perform mock user interviews w/ project managers so they go through the experience
Learn, Understand, Design, Learn again
I interviewed 8 users in total spread out over the course of a few weeks. I conducted a ~1hr interviews utilizing ethnographic principles to extract data about the user's responsibilities, frustrations, processes and interactions. This information was instrumental in shaping what has turned out to be an amazing user experience.
User Interview Framework
It’s a good idea to have a well constructed high-level strategic plan on what aspects of the user's responsibilities/process you want to learn about.
I create a Trello card for each interviewee. I can have a great view to track and drive the research progress. This is also a great place for anyone else on the team to review raw interview data.
How you conduct your user interviews is entirely up to you and should be customized to the specific industry and audience you are interviewing.
I often follow a loose improvisational style. You have to follow where the user takes you and select the correct question to ask at the right time. I guarantee you no interview ever goes exactly like the previous one. Users prioritize differently so you have the be flexible but keep on a topic but also allow opportunities to ask the 5 whys. Contextual interviews have often been described as an art unto itself.
User Interview Pillars
There are a few tactics I utilize during an interview:
- Ask Open Ended Questions, I can't stress this enough…
- Why? Why? Why? Why? Why?
- Don’t be afraid to go off script
- Be mindful of time [It’s ok to let them talk but try to stay on topic]
- Listen for emotions/topics repeated multiple times
- ASK OPEN ENDED QUESTIONS (it’s ok not to know exactly what you'll learn. That's why you’re conducting the interview in the first place!)
Interview Analysis
For the first analysis I decided to be completely and unforgivingly transparent. I was going to leave every piece of data extracted from the interview for all to see.
I tend to write outlines and draw workflow diagrams. The outlines are the basis for my information architecture. The users often describe a process and I simply illustrate it. This could take some time as I rewind over and over to make sure I understand. A 2 hour interview can take ~2–3 days just to analyze.
Analysis Focus Areas
There are a few aspects of the interview I make sure to understand. These include emotion, repetition, examples, and overall themes.
Emotion
I deconstruct the interview in front of a 8' high by 12' wide whiteboard wall. My interviews follow a general pattern so each question maps to a outline heading.
Repetition
Users will often return to things that either really make them happy or annoy them to no end.
Examples
Users give examples of things they do like from other applications. I believe it’s an aspect of my job as the user experience designer to distill what technique the references software is doing well and include it into whatever I design.
Themes
A user interview can often be like watching an episode of Game of Thrones. There are 4 or 5 different story lines all going on and the interviewee will often call back as they unpack their mind about what it is that they actually do everyday. Writing all those data points on a giant whiteboard wall helps me uncover commonalities. These end up as epics in user as story mapping cards.
Personas
Armed w/ interview information I finally felt like I had my head around some of the challenges the analysts were facing. I was able to become one of them much in the way method actors become their characters (PS I’m writing a full length article around this topic).
The current software is a rules-based system yet our product would be driven by Artificial Intelligence through the use of techniques like NLP and Deep Learning. This data established our acceptance criteria:
- Analysts would need a channel in which to provide feedback to the data science models
- Actions taken by the model need to differentiate from user actions
- Comments should be tied to a potential violation not the whole alert itself
- Contextual information is needed to provide additional data to assist the analysts when they make a ruling on the legality of the communication.
- Utilize screen space on enterprise issued monitors running 1920 x 1080 resolution
Workflow Breakdown
Understanding the workflow of how a task it completed allows you to understand the factors that will influence the information architecture of a given view. It also gives you insight into the features being used in secondary software. If you can include these features in your designs then there is a greater chance for a high user adoption rate.
Quantitative vs Qualitative
Post the Alpha 1.1 release (actually the 3rd iteration of the design started 4 months earlier) I learned the breakpoint I’d chosen was not in fact the default resolution for the users. They used twin widescreen dell flat panel monitors at 1920 resolution. It was like turning a corner and discovering heaven. Think of all the screen real estate!
You’ve made it this far! Next up we get to the pot of gold at the end of the rainbow, the design composition evolution. Coming soon!
