The Three Biggest Challenges Designers Often Face in a Deep Domain Industry

You won’t know until you try.

On Aug. 10, 2020, our team officially joined IBM’s Shared Operation Services (SOS) project, which is designed to solve the most common security and compliance audit requirements of IBM Products and Offerings, including IaaS, PaaS, and SaaS. The SOS group helps teams standardize on a common set of security tools, providing cost-effective solution and professional services to all IBM Products and Offerings teams.

The entire design phase lasted six months, and the development work is still ongoing. Compared to previous projects, we faced significant challenges in that deep domain projects like this can be difficult to grasp using merely our shared experiences or common sense gained from daily life.

We leveraged the framework of Enterprise Design Thinking (EDT), appreciating its flexibility as we explored this domain and got to know our users. Starting from scratch, we gradually built an entire working model that almost perfectly fits the needs of the field.

Here’s a look back at what we learned along the way.

Challenge 1: Instilling the principles of Enterprise Design Thinking

IBMers have been talking about EDT for years, yet many product teams still haven’t realized the impact design can have on the entire user experience. Across our organization, teams have their own operating modes — and we found the same to be true for SOS, which had not previously collaborated with a designer or practiced EDT. This became our first challenge, and we addressed it in two ways:

Establish internal empathy

Before you can gain empathy with your users, you first must have it within your own team. This is a dynamic process that requires a certain amount of onboarding — which can become a lengthy process unto itself. To get up and running more rapidly, we clarified the full spectrum of project needs using a core set of activities:

• Invite support from clients and those who have the best understanding of our users’ perspectives.
• Talk with architects who know the architecture framework to understand technical advantages and limits.
• Sync with key stakeholders to document their hopes and fears for the project.

Build an efficient, scientific cooperation model

• Clarify tasks and processes so UX and development teams can use EDT to connect with Agile, developing, planning user stories, and managing tasks in a unified manner. It’s efficient to separate UX and Dev registry management under the same registry in GitHub, while keeping them connected with each other. Review the progress of the entire project together; with transparent task allocation and management, problems can be raised and resolved quickly.
• Enhance the participation of expert users as well as technical experts. In the mid-term we found the architect’s participation in interviews allows for facilitating more in-depth communication on functional architecture issues raised by users.
• Make development the last step in design — and design the first step in development. In order to defend the validity of your designs, consider the difficulty of implementation, speak on the same terms as your development team, and examine your designs from the perspective of implementation.

Challenge 2: Accessing the unique needs of a deep domain

When faced with a project in a field you already know, it usually doesn’t take long to ascertain the needs and develop a strategy. That’s not necessarily the case when you’re met with a problem in a deep domain. To educate ourselves on the various aspects and nuances of this new — to us — field, we took a dual approach:

• Obtain valid learning resources. We asked stakeholders to provide as many background materials as possible, from old product instructions to links for related websites and documents — anything we could get our hands on.
• Build a team knowledge bank. Throughout our learning process, we documented and shared key terms used within this deep domain, along with definitions and examples of how they were used. Other characteristics we captured for each of the terms included words with high repetition rates, industry-specific terms (e.g., “compliance” in the security field), different names for the same concept, and abbreviations of professional terms (e.g., “IKS” for “IBM Cloud Kubernetes Service”).

Three of the 43 terms we learned during the project

Pro tip: Consider the target and involve suitable participants. Always invite the policy and implementation layers — each of which can contribute valuable points — and make sure to end your interactions with actionable follow-ups.

Be flexible

As the discussions progressed and became more in-depth, unfamiliar scenarios emerged and new users rapidly expanded. Sensing that we were getting further away from understanding the problem, we suspended our working session on the third day and switched to a new approach:

• User interviews. Because the knowledge shared by stakeholders was intricate and convolved with unauthenticated information, we arranged interviews between the working sessions to get confirmation from the sponsor users.
• Technical Q&As. We contacted the technician to learn about unfamiliar contexts that were appearing in the working sessions, so we could avoid ignoring key scenes due to our lack of knowledge in this domain.

Day three of our working session — when we opted to change course

Step into the unknown

It’s important to remember that designers hardly ever have any first-person experience — and this is even more true when entering a deep domain. Because our understanding of this field relied entirely on what our sponsor users told us, it became that much more necessary for us to validate what we heard, while discarding any irrelevant information. To maintain our focus, we adjusted our approach by:

• Expanding our sponsor user pool. Our stakeholders put us in touch with the first group of users to interview, then we expanded our pool by asking those users to recommend other people to interview. During this process we: 1) created a sponsor user wiki to accumulate our knowledge and target more interviewees; and 2) locked in our key sponsor as early as possible, which helped us accelerate the research phase and reduce interference.
• Modifying our interview strategy in real-time. Designers must process interview data quickly and adjust the interview strategy and materials quickly. When we found some users’ situations didn’t meet the target scope, we adjusted the list of interviewers; when the design was repeatedly questioned by an interviewee, we improved it before showing it to the next user.

Pro tip: Asking someone what kind of coffee tastes good isn’t as good as serving them coffee and then asking how it can be improved. Likewise, sharing visual designs based on your assumptions — even if you aren’t certain whether the designs are right — can prompt valuable insights from interviewees.

Change your perspective

As we moved into the implementation phase, a new dilemma emerged: An entirely new product containing a significant amount of professional data cannot be implemented entirely with user interview data.

To help solve for this, we summarized the possible data types learned in the interviews and used these as indices for our back-end development colleagues to obtain real cloud platform data. In the process of collecting and sorting back-end data, we uncovered two learnings to help improve efficiency and accuracy:

1. Organize the data according to Title+Example. In addition to knowing what data (“Title”) is available, you also need to know how the data is presented (“Example”). We found it was best to understand all the state types existing within the data. For some string type data, we had to know the character length range.
2. Understand the meaning of the data — and verify it. We quickly organized the data into a structured document, while simultaneously conducting a knowledge alignment with our development team to understand the specific meaning of the data. We then inserted the data into our lo-fi prototype as soon as possible so we could separate key data from nonessential information.

Challenge 3: Overcoming gaps in market knowledge

Without domain experience or domain-appropriate design patterns to reference, and without any data we could use to design around, we needed a new way to gain traction with our stakeholders. So we turned to market and competitive research to help guide us toward implementation.

Returning to our project requirements, we pulled keywords associated with the domain, such as “cloud,” “security,” “asset compliance,” and so on. Using these keywords as a starting point, we identified a batch of competing products in market.

However, because internal IBM products have certain needs that differ from other products in the same field, we also extracted terminology from our design requirements and used those to find related features — all of which we summarized in a comparison grid for discussion with our stakeholders.

Pro tip: Use Capterra to identify, analyze, and compare similar products in market. If you’re unable to locate competing products, try using your product requirements as clues to find competitors.

Sizing up the competition

Tips for Design Management and Operations

Over the past seven months, the SOS design team utilized a series of user research tools and methods to clarify user needs, define product directions, and quickly build designs centered around technology- and user-based needs.

Though we initially met difficulty gaining a toehold due to our designers’ lack of domain-specific experience — as well as our broader team’s differing working styles — we gradually discovered that the diversity of individuals on our team provided a wide range of skills and experience, which ultimately helped us improve the team’s problem-solving capabilities. Overall, we found that frequent, consistent communication and feedback were essential for building chemistry within the team. To reach this point, we embraced the following practices to manage the team and allocate tasks:

Invite outside teams to critique the work

By asking other IBM designers from the global GBS team to weigh in on our in-progress designs, we gained invaluable feedback that helped further polish our work. (Thank you to everyone who participated from our Austin and Shanghai studios! ❤️)

Pushing the work forward

Standardize processes to ensure design quality

We optimized our overall efficiency and design quality by setting clear content delivery standards and file specifications across all our platforms (e.g., InVision, Box, and GitHub) — and by establishing a design and review process that suited the specific needs of the project.

Always improving

Choreograph the team

A clear division of labor within the team is vital. We found these methods to be especially useful:

• Rely on GitHub to track individuals’ respective progress and make sure all tasks are delivered on time.
• Identify problems quickly, communicate in a timely manner, formulate solutions, and synchronize ideas.
• Pave the way for designers’ growth by setting clear standards and encouraging greater responsibility across playback, working session hosting, research, UX/UI, and design strategy.

Key Takeaways

At the beginning of the project, stakeholders talked about creating the minimum viable product (MVP). By the mid-term, they were discussing the minimum detectible effect (MDE). More than a change in terminology, it highlights the larger team’s assimilation of EDT.

Through the efforts of the design team, we used Design Thinking to empower the entire team. Here are some learnings we found effective at each stage of the project:

Early stage

Try to dig as much information as possible out of your key stakeholders to identify opportunities, expectations, and concerns.

Mid-term

1. Discussing the roadmap is a crucial time node. While efficient discussions occur at the decision-making level, they also rely on hands-on technical experts to provide implementation and feasibility ideas as considerations. This work can be divided into two stages: 1) the working session; and 2) the leadership group. This is the watershed of the product trend.
2. If you move forward, don’t forget to look back. Repeatedly return to the initial point of the project, consider the reasons for the deviation of the route, and correct the current route, if necessary.
3. Reflect together and analyze what the product team wants and what the user wants. What is the agreement and gap? Why does this gap exist? Is the gap a blue ocean or a red ocean? How can we find a balance point?

When you design

Take advantage of the design critique. Inviting people with different roles and backgrounds will give you insights you won’t reach on your own.

Finally: Thank you to our brilliant global strategic design team led by Jeff Neely! Your support and coaching makes all of this possible!

Team Members

Design Principal & Director: Ben Landrum
Program Design Lead: Dong Ni (Donnie)
Design Lead: Qiang Yao (Celin)
Researchers & Designers: Guo Chen (Chloe), Ziqiumin Wang (Oscar), An Yan

Authors

Qiang Yao (Celin), Guo Chen (Chloe), Ziqiumin Wang (Oscar)