UIUX Case Study: Mozilla Iris, a QA Testing Tool
--
The Ask
Iris is a brand-new Mozilla project that will be used to automate testing of Firefox. It is scheduled to be released in a 1.0 version in November 2018. It is a project mostly built in Python, with a web interface in React.
This project is in its infancy, and the web interface employs basic functionality. However, it is in need of a design and user experience sensibility to truly maximize its usefulness.
This is a perfect opportunity for someone who would enjoy improving an existing interface, using their creativity, and working with a motivated team.
Context
I recommend you read over my last post to learn more about what Iris is doing behind the scenes and understand the problem we are solving.
Take a look at Iris’ current interface:
My Role
- Research
- Low and High Fidelity Mockups
- Interface Design
- Interaction Design
- Visual Design
Challenges
- I was give 3 months to do research, understand a complex domain and handover informed design solutions.
- Because Iris is new, we have no users.
- Evolution of stakeholder requirements during the design process.
- The need to create the UI for a powerful tool with an approachable design.
The Process
I took a non-linear approach to my design process, understanding the big problems first, exploring solutions through further investigation, then working through smaller problems with a similar process. I wasn’t prescriptive in my processes, but rather approached each problem with the appropriate tools to understand and solve it.
Stating Assumptions
Because of the challenges I stated earlier, I had to work with some limitations. As I began my research, I realized that the most efficient way for me to work was to assume that Iris’ current UI included all the necessary information, that it did not have any unnecessary information, and that it was all organized in a way that made the users job easier. I then made it my job to validate and question those assumptions with the users’ goals and needs as the focus.
Research
In order to provide a solution, I first needed to understand the problem.
Discovery
I jumped into the project mapping out the current user flow, creating a heuristic analysis of the UI and looking at other products that are working on similar problems. I had to get a firm command of the domain of quality assurance testing.
As I mapped out the user flow I wrote down every question I had about the interactions and data presented in the current UI. Finding answers to those questions gave me a groundwork for beginning the next stage of research.
Defining Who
Stakeholders: Firefox, Firefox Users, Product Integrity
Users: Product Integrity Engineers and QA Testers
Defining the Problem
After mapping out the users journey and talking to our product leaders and subject matter experts, I defined two problem statements that Iris is working to solve:
Quality Assurance testers need to run robust tests quickly and easily so that they can gather insights in their test results.
and
Iris stakeholders need to automate the manual testing process so that tests can be completed quickly and prevent regressions in Firefox.
After doing preliminary research and defining the problem, I was able to outline a strategy to work towards a solution.
Generative Research
In the current interface, the user has to click a carrot to expose test details. Our user needed to dig down to find information. There was no hierarchy or sense of importance.
In order to build a user centered interface I needed to know which details would help the user know that they wanted to run a particular test.
So, I created a survey to gather insight from potential users who were not able to talk to me.
I also conducted video interviews with Mozilla QA testers in Romania. They gave me insights like:
“These fields are so similar. I know what they are, but for a new person, it would be confusing.”
“I shouldn’t be required to click into the test to know that it is worth exploring.”
“It would be a dream to see what the problems were and to go ahead with the next things to do.”
“When you have a lot of failures there, that’s when it is the most interesting.”
Providing Solutions Through iterative research, solutions and feedback.
With a foundation of research I now had an understanding of the domain, I knew the problems I was solving and who I was solving them for.
Designing & building prototypes
My building process was iterative and looked something like this:
Iris’ interface is full of information and details. As you can see in the test results here:
So, I pinpointed every piece of information and talked to subject matter experts and pseudo users, asking them the following questions:
- What does this data point represent?
- What its value to our users?
- Is this piece of data critical to our users doing their job?
- Is it in the right location and paired with like data?
- How does it influence our users behavior?
With answers to those questions, I could solve organization and prioritization problems from a logical background.
Then, while developing visual mockups for UI solutions, I pinpointed specific design-centric problems I had uncovered. They included:
- How might we make this information skimmable?
- How might we make the important details stand out?
- Would changes in the logic of the program need to change so we could provide the right solution for the user?
Brainstorm
One of the most pressing needs of out user is to quickly and easily pull out the most important facts and insight from a lot of data. Once we determined that need, I started a parking lot of ideas.
Research Existing Solutions
Because I was a design team of one, I needed to rely on the hard work other designers had already done to solve similar problems. I asked other UX Designers at Mozilla for insight, I searched for and read about UI and Interaction Solutions (especially charts!), and presented my work at Mozilla’s weekly UX Design Critique.
The Outcome
The Iris 2.0 interface is organized into two distinct parts: 1) exploring, selecting and launching tests, and 2) viewing results, finding insights and exploring debug images. A minimal color palette allows us to emphasize important details. We have focused on giving our users the ability to quickly and easily get the most important information they need, while strategically placing other data so that it can be located intuitively.
What’s Next
Our first group of users are currently testing Iris 1.0. They are comparing results from manual tests to those produced by Iris. We are taking advantage of the opportunity to gather qualitative data from about their experience while using Iris.
With the successful launch and use of Iris 2.0, Iris 3.0 will be around before we know it. I know that my work on Iris will leave a groundwork for user centered design in the future.
Takeaways (what I will do differently next time)
- I will present at critique early and often. If a formal critique isn’t available, I’ll create my own!
- I will design small incremental releases and test those smaller components.
- I will emphasize the need for usability testing earlier in my process.
- I will help my stakeholders understand how to interpret lo-fi mockups, so I can use them to quickly communicate ideas.
