Conducting User Research at a Global Conference

How the VMware Design team talked to over 750 users in 4 days at the VMworld Europe Design Studio

Nareg Kirkian
Dec 12, 2019 · 6 min read
Image for post
Image for post
Kevin McBride walked through the designs of VMware Cloud

User testing and research at conferences can be hard when it comes to preparation, finding the right audiences, scheduling times, and logistics.

For many years, VMware product designers have been doing user testing and research during the company’s VMworld global conferences. We find our annual customer conference a valuable opportunity for easy access to a big pool of partners and users.

A little background about VMworld

For many years, VMworld has been one of the most significant virtualization and cloud computing events on a global scale with several exceptional product announcements, exciting hands-on labs, various keynotes, breakout sessions, and vital implications for what the future vision of cloud computing technology looks like on the horizon ahead. The conference happens twice a year, with one in the US and the other in Europe. This year, VMworld Europe had over 14,000 participants and VMworld US had 20,000. For designers, this conference is a must-attend since we are able to meet users, understand their pain points, and deliver solutions with empathy.

VMworld 2019 at Fira Barcelona
VMworld 2019 at Fira Barcelona
VMworld Europe 2019 at Gran Via Fira Barcelona

VMworld Design Studio

In general, VMworld Design Studios help us explore problems, identify user pain points, and get insights from various stakeholders in person.

Just three months ago, my colleague Gabriel talked about how the VMware Design team led the research with over 1,000 users in 4 days at VMworld US 2019.

Last month, during VMworld Europe 2019, the design team from Armenia, Bulgaria, India, and the US gathered in Barcelona to facilitate design studio workshops with partners and users of VMware products. Our most enthusiastic users and IT administrators had a unique chance to get their voices heard, interact in small focus groups, and get introduced to new designs and upcoming product plans.

With a growing attendance this year for the Barcelona event, we talked to more than 750 users over four days. You might be wondering, how’s that possible?

Members of VMware Design Team from Armenia, Bulgaria and the US
Members of VMware Design Team from Armenia, Bulgaria and the US
Members of VMware Design Team from Armenia, Bulgaria, India, and the US.

We conducted various design activities by showing early design concepts and ran hands-on research activities to gather valuable feedback from VMworld participants. Most sessions had at least 3 attendees, though many had up to 18. One of the sessions even had 28 participants!

Most importantly, Design Studio Sessions give the participants an opportunity to provide feedback to the product design teams at VMware. If our customers weren’t able to attend VMworld, they still have the chance to participate in our online Design Studios which are held throughout the year. These are often 1:1 discussions, user interviews, or usability testing sessions. If you are a currently VMware product user, you can sign up for these online sessions that can be found on VMware Design’s Research Page.

The success of these design studios is the result of many months of preparation. More than 25 designers started planning around 27 sessions months ago and worked directly with the product managers and engineering teams. Most of the participants registered via the conference content catalog where they could select the sessions that were relevant to the products they were using for their day-to-day operations.

Session Format and Activities

One of our main objectives is to continuously strive to understand the user’s goals, challenges, and pain points. The design team conducted interactive sessions while also capturing valuable research insights. Some of the different types of research are highlighted below.

Within the scope of design research, interviews are often more exploratory and help to get an in-depth understanding of the experiences and daily lives of the users.

Abhishek Chouksey conducted an interview on the discovery of business applications in vRealize Operations Manager. He wanted to understand how the users are monitoring their applications within their infrastructure and what issues they are facing with monitoring high demand and complex business applications. Abhishek wanted to clarify what matters for them in the end-to-end journey so that the ideal journey can be delivered when it comes to monitoring the services and applications in the infrastructure.

vRealize Operations Manager is a platform that provides unified IT management, bridging physical and virtual infrastructure, meshing vSphere or hypervisor-enabled systems with Amazon Web Services, and helps in capacity planning for the virtual infrastructure.

Abhishek conducting user interviews for the next-gen application-aware operations
Abhishek conducting user interviews for the next-gen application-aware operations
Abhishek conducted user interviews for the next-gen application-aware operations in vRealize Operations

Collaborative design is the most engaging method, which helps in getting the best ideas on the table by involving various users and stakeholders with different backgrounds and perspectives.

Yoana Georgieva gave two tasks to the users to answer questions related to how they are currently managing vCenter and ESXi configurations.

The other collaborative exercise she did was a brainstorming activity where the participants had to come up with ideas, recommendations, and assumptions for what could be a centralized configuration management experience from a single point towards multiple vCenters. Also, they brainstormed how this solution might be designed at a high-level.

vCenter is a centralized management tool for data center server management which helps in deploying virtual machines residing on a distributed virtual data center.

Image for post
Image for post
Yoana conducted the collaborative design session with the vCenter users

We always give our users the ability to amplify their reach and make their voices heard so we can better shape the future of enterprise design at VMware. In this small group setting, while we cannot gain usability insights, we are able to understand pain points and validate end-to-end flows.

Lusine and I aimed to better understand the user journeys when it comes to monitoring their environment in vRealize Operation Manager. How do users currently monitor their environment? How do they currently use dashboards? Then we showcased the modernized dashboards and infrastructure details as part of the monitoring workflow. The participants provided their feedback on how their ideal end-to-end workflow looks like when it comes to troubleshooting both infrastructure and application issues.

Image for post
Image for post
Lusine and I walked through the designs of vRealize Operations— Photo by Bonnie Zhang

Key Takeaways

We always learn from our users by listening to their feedback.

  • Always involve users. We’ve found that they are very collaborative, active, and generous when it comes to giving feedback and sharing their knowledge.
  • Analyze all the data collected and present the insights into company executive leadership, and demonstrate the value of user research.
  • Do a follow-up session with the participants who already signed up for as a prospective Design Studio attendees.

What did our users say about these sessions?

Even though the participants attended many hands-on labs, various keynotes, and breakout sessions at VMworld, for some, the Design Studios are the best thing about their whole VMworld experience. Participants like these sessions as they feel involved in multiple stages of the design process as a collective.

Conclusion

Overall, VMworld Design Studios present us with the opportunity to engage face-to-face with our real users and customers. From these sessions, we gathered invaluable information about the workflows and needs of the users. Moving forward, we are expanding upon the insights collected and defining the next generation of VMware products.

If you are a designer and already planning to get direct feedback from your users, we encourage you to organize user research sessions like the “Design Studios”. It’s an invaluable opportunity, especially in company conferences where users can attend.

See you next year at VMworld 2020 in San Francisco and Barcelona 😀

We’re hiring!

The VMware Design team is looking for talented designers to help us continue transforming enterprise design. Check out our open positions!

Special thanks to the organizers Afifa Tawil, Safa Dada, and Jehad Affoneh. The designers who conducted Design Studios. Bonnie Zhang, Grace Noh, Sanaea Kakalia, and Kevin McBride for reviewing this article.

VMware Design

The VMware Design team | Transforming enterprise user…

Thanks to Bonnie Zhang, Grace Noh, Sanaea Kakalia, and Kevin McBride

Nareg Kirkian

Written by

Sr. Product Designer @VMware (Design Lead). Yerevan Local Leader @interacting. Former life @Workfront_Inc and@tribalworldwide

VMware Design

The VMware Design team | Transforming enterprise user experience and design

Nareg Kirkian

Written by

Sr. Product Designer @VMware (Design Lead). Yerevan Local Leader @interacting. Former life @Workfront_Inc and@tribalworldwide

VMware Design

The VMware Design team | Transforming enterprise user experience and design

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store