Exploring Customer Needs with UX at VMware Explore: Radically Innovating VMware’s UX Design Research Methods
Author: Beverly May (VP UX Design)
Original Article published on the VMware OCTO Blog.
I joined VMware a year ago as VP of UX and Design, leading a 250+ person centralized team supporting all of VMware’s core businesses and products. It’s essential as a new leader to observe and assess your team’s design process (or “UX Maturity,” as we call it in UX jargon). One surprise was VMware’s UX research team, which prioritized planning and performing high-visibility research at an unconventional location: our annual industry trade show, Explore. I was curious to understand why a trade show would trump more traditional UX research activities and recruitment methods in priority, but I learned first-hand the wisdom of this approach when I attended VMware Explore Las Vegas in August 2023.
I believe our UX research and design co-creation activities at Explore can offer lessons to any product or UX team in nearly any sector. This blog will frame our Explore presence and UX research in the context of our broader UX team strategy and goals, and then will highlight some of the team’s research innovations at Explore in the hopes that they may inspire others to pursue them as well.
Setting the stage: Establishing UX research foundations within the team
In any enterprise UX team, there is a need for a niche skillset of professionally-trained UX researchers with robust quantitative and qualitative research skills to perform user and product research in close partnership with UX product designers. At some companies, the UX research and UX design roles and responsibilities can be rigidly separated, which generally leads to a slower, more “waterfall” UX process and poorer-quality designs — because the designers have less first-hand understanding of their user’s needs and pain points.
For these reasons, I believe strongly that UX designers must be empowered and enabled to pursue and perform their own research so they can fully “own” and be accountable for their product designs, process, and outcomes- rather than having UX researchers (or product managers) do it all for them. The foundations of user-centered design or UX rest in understanding the user’s needs and goals and addressing those needs through designed solutions; I don’t believe it’s possible for any designer to design effectively without being an active owner and driver of the research that uncovers those insights (side note: this doesn’t mean ALL UX research must be done by designers; professional UX researchers are also critical team members for more sophisticated studies, aiding designers in their research, and for performing cross-product research).
As part of my initial team assessment when I joined, I spoke to every member of the 200+ person UX team. In so doing, I learned that designers’ top two challenges to performing their job effectively were A. a lack of access to real users to perform UX research, and B. a lack of knowledge of how to do UX research effectively — particularly using “lean” or more informal UX research methods. Designers knew their lack of UX research was a problem, but they didn’t have the staffing and time, tools, access to users, or knowledge to solve it.
Consequently, for calendar year 2023, we established five annual goals around UX research, with quarterly team progress tracked through OKRs. The aim was to bolster knowledge, ownership, and accountability for UX research by UX designers. The five research-related UX team goals and initiatives for 2023 include:
- UX designers should learn and adopt a consistent, standardized “8-step” process for UX, which emphasizes industry best practices for how to integrate user research and testing into every aspect of the UX process.
- All products will be evaluated using the UX-Lite usability scorecard for standardized usability testing against industry benchmarks.
- Each lead designer must create and maintain a research and testing plan for their product
- We procured UserZoom platform access and recruitment credits to conduct remote online research and required all designers to take UserZoom training.
- We procured team access to Luma, an online resource of design thinking methods, templates, and training, and offered optional Luma-certified training and library access to all designers in some form.
With all these initiatives in play in 2023, research has become much more ingrained in our overall UX approach and has become an important core capability. UXers are now empowered to be better designers by utilizing standardized usability benchmarking data from UX-Lite, fast recruitment avenues and remote research methods with UserZoom, Luma’s participatory design methods and approaches, and the adoption of the standardized 8-Steps UX process that integrates at every step. We also doubled the UX research team size with a mix of new hires and contractors to aid in this evolution.
Our UX team presence at Explore under the “Design with Us” moniker takes this strategic, research-centric approach to UX and massively amplifies it for much broader business and customer visibility and impact. The foundational UX research initiatives listed above aim at standardizing and upskilling the entire team. Design with Us beautifully complements these with sophisticated marquis showcases of participatory design and research that are customer-centric, playful, and wonderfully inventive.
Design with Us: What is it?
Design with Us (DWU) is the umbrella name for all UX research and co-creation initiatives at our Explore trade show. DWU primarily leverages inventive participatory design activities that are structured like games to solicit active, playful, low-pressure user engagement and feedback. The beauty of DWU for UX practitioners is that many of these initiatives could be deployed in any customer, partner, or stakeholder co-creation setting. What Explore provides is a massive scale with 10,000+ highly engaged attendees. Unlike many company trade shows, Explore attracts both buyers and active users of VMware software, so it is an ideal setting for concentrated user research.
Design with Us preceded my arrival at VMware. DWU began as a small experiment in 2018 at VMware’s VMworld trade show under the name “Design Studio.” The initiative evolved each year, was expanded in 2022, and renamed to “Design with Us,” and then we further invested in and expanded our presence for 2023 to its current form.
For Design with Us at Explore Las Vegas 2023, we increased the number of UX researchers and facilitators to around 40, or nearly 15% of the UX team. Instead of performing product-specific traditional usability testing or basic survey-based feature evaluations, the team pursued cross-product UX research around strategic business themes like AI. This ensured that UXers worked collaboratively across businesses and gleaned more high-value cross-product insights from the user’s perspective.
Broadly, Design with Us 2023 reflected a customer-centric focus on both the choice of research themes (which were based on customer priorities) and how the research was conducted (which leveraged participatory co-creation approaches where users become the designers).
Design with Us at VMware Explore 2023
I was absolutely blown away by the sophistication, creativity, and scale of execution that our UX team achieved at Explore 2023. DWU at Explore Las Vegas spanned all the following UX activities, which were conducted concurrently across multiple trade show sites:
Illustrate your Enterprise: Design Sketching in the Hub: we stationed UX designers with drawing skills to illustrate customer challenges and pain points using pen and paper via customer-journey style narratives. These were then posted on a wall where others could see and vote on them. Each story’s focus was a problem-solution narrative that also challenged the participant to envision how VMware could better address the challenges being presented through improvements to our products, services, or support. This station had a highly visible floor presence with non-stop participant traffic throughout the day. The team conducted 80 sessions at about 30 minutes each and then codified the inputs immediately after for later quantitative data analysis using clustering and trends. The output, a huge wall of ideas in a high-traffic area, was a testament to the power of UX to solve customer challenges using user-centric methods.
Design with Us Pods: The heart of our UX DWU presence was the Pods, which was a concentrated space with a central entry area and five large inner rooms that could each accommodate 10–30 participants in different co-creation and research sessions. Each room focused on a different topic, spanning Pricing, Partnerships, Customer Support, Subscriptions & Commerce, and Multi-Cloud, and leveraged different participatory design approaches. The outcome of each of these was a bounty of new product ideas and features, meaningful data on customer pain points and priorities, and increased customer engagement.
- For example, the “Commerce Café” had 20 participants choose their “meal” of features for their “main dish,” “side dish,” and “dessert,” which helped determine the relative priority and value of each feature for that user profile.
- The Partner pod had 22 participants draw out comic strip-style “stories” of various scenarios they might encounter using stick figures.
- The Customer Support pod had 46 participants prioritize their challenges using the Rose-Thorn-Bud design thinking framework and map out prioritized solutions.
Games for registration and AI research: Running down the middle of the DWU pods area was a series of survey questions on AI that were structured like physical games at a carnival, which was called the “Ball Pit.” 200 participants placed balls in jars to reflect their preferred answer to “multiple choice” questions on topics like their biggest security concerns for AI. The number of balls in each clear jar provided an easy visual tally of customer preferences. Another AI game at the Hub kiosk asked participants to choose their character from a list of nine archetypes (complete with names like the Warrior, Sage, or Bard) to map AI needs to different personas. Another AI survey at the kiosk generated 229 responses on user needs and priorities.
Feature and audience research with the Tanzu Trivia digital game: Two outgoing UXers, dressed up in gold hats and metallic disco shirts to stand out from the crowd. They roamed the main floor, talking to attendees and gaining meaningful product-feature-audience feedback through a fun and easy-to-play iPad trivia game. The participant experience was much more fun than a traditional survey and allowed the pair to rapidly source inputs from 60 customers, as well as recruit 100 new customers for future product research.
Qualitative UX product observational research in Hands-on Labs: We conducted more traditional research as well at Explore, including stationing a few UXers to listen in to large-scale training events at our in-house training forum, Hands-on Labs, which trained 500+ attendees concurrently at any given time on multiple VMware products. UXers quietly observed where and when participants were struggling with either the training or the software itself, including usability considerations, which directly led to insights on areas for both product and training improvement. About 50 participants also submitted suggestions on how to improve Hands-On Labs in a survey.
Additional qualitative interviews, customer feedback sessions, and surveys: In addition to the above, we also joined our Customer Success team to hear first-hand the challenges customers are facing and source potential paths for improving our UX. Sixty-eight customer interviews addressed general customer challenges and opportunities, with an additional seven focused on multi-cloud and AI preparedness.
Post-event quantitative insights and data analysis: Finally, after Explore, our lead researchers mined the data and insights through multiple approaches to present high-level strategic insights to business and product leaders on their product lines and potential new offerings. The beauty of Design with Us was that all customer data and insights were input on-site after each session into Excel and other archival formats for easy post-event summary and analysis.
This lengthy list doesn’t fully capture the magnitude of the Design with Us installations, the ingenuity reflected in the games and co-creation activities, or the enormous efforts of the team to plan and execute this. It’s fair to say that nearly every UX method of user research and co-creation was deployed in some form at VMware Explore Las Vegas 2023.
Most impressively, participants LOVED Design with Us. 98% of participants rated the sessions positively, with nearly a 20% year-year jump in interest and participation. Here are a few examples of participant feedback: “This was the best part of Explore!”; “This was very interesting. A fun, creative way to express concerns and issues we’ve encountered”; “This was so much fun and imaginative. I loved this so much. Thank you.” and “I think it is great that you take this opportunity to listen to people and their challenges.”
What comes next?
We have similarly bold plans for VMware Explore Barcelona in November 2023, with many of the above activities being repeated and some new tracks being added. In Barcelona, you’ll be able to find us right near the main registration area, as well as in the Hub. Post-Barcelona and after Broadcom’s pending acquisition, we hope to continue to iterate and expand Design with Us in future customer forums.
Key takeaways
Design with Us is now the standard-bearer for a new form of engaging and effective participatory design and UX research. Design with Us demonstrates that UX research can be paired with co-creation and design thinking techniques at a very large scale in experiential physical settings and to incredible effect. The UX team gained highly valuable user insights from Design with Us, tangibly demonstrated our ability to foster innovation and be innovative ourselves, delivered valuable strategic product research to business leaders, and deepened our customer loyalty while ensuring everyone had a lot of fun in the process. Customers felt heard, validated, and invested in the participatory design process, and none of it was boring, predictable, excessively sales-focused, overly intrusive, or excessively time-consuming.
Design with Us manifests user-centricity at its core and showcases VMware and the amazing VMware UX Design and Research teams in their best light. I commend the team on doing an incredible job in pushing our creative boundaries! Research and user-centered design can be engaging, collaborative, and fun, and that’s when the innovation magic happens. Design with Us shows us the way!
Author: Beverly May (VP UX Design)
Original Article published on the VMware OCTO Blog.
