Conducting User Research at a Global Conference
How the VMware Design team talked to over 750 users in 4 days at the VMworld Europe Design Studio
User testing and research at conferences can be hard when it comes to preparation, finding the right audiences, scheduling times, and logistics.
For many years, VMware product designers have been doing user testing and research during the company’s VMworld global conferences. We find our annual customer conference a valuable opportunity for easy access to a big pool of partners and users.
A little background about VMworld
For many years, VMworld has been one of the most significant virtualization and cloud computing events on a global scale with several exceptional product announcements, exciting hands-on labs, various keynotes, breakout sessions, and vital implications for what the future vision of cloud computing technology looks like on the horizon ahead. The conference happens twice a year, with one in the US and the other in Europe. This year, VMworld Europe had over 14,000 participants and VMworld US had 20,000. For designers, this conference is a must-attend since we are able to meet users, understand their pain points, and deliver solutions with empathy.
VMworld Design Studio
In general, VMworld Design Studios help us explore problems, identify user pain points, and get insights from various stakeholders in person.
Just three months ago, my colleague Gabriel talked about how the VMware Design team led the research with over 1,000 users in 4 days at VMworld US 2019.
Last month, during VMworld Europe 2019, the design team from Armenia, Bulgaria, India, and the US gathered in Barcelona to facilitate design studio workshops with partners and users of VMware products. Our most enthusiastic users and IT administrators had a unique chance to get their voices heard, interact in small focus groups, and get introduced to new designs and upcoming product plans.
With a growing attendance this year for the Barcelona event, we talked to more than 750 users over four days. You might be wondering, how’s that possible?
We conducted various design activities by showing early design concepts and ran hands-on research activities to gather valuable feedback from VMworld participants. Most sessions had at least 3 attendees, though many had up to 18. One of the sessions even had 28 participants!
Most importantly, Design Studio Sessions give the participants an opportunity to provide feedback to the product design teams at VMware. If our customers weren’t able to attend VMworld, they still have the chance to participate in our online Design Studios which are held throughout the year. These are often 1:1 discussions, user interviews, or usability testing sessions. If you are a currently VMware product user, you can sign up for these online sessions that can be found on VMware Design’s Research Page.
Planning and Recruitment
The success of these design studios is the result of many months of preparation. More than 25 designers started planning around 27 sessions months ago and worked directly with the product managers and engineering teams. Most of the participants registered via the conference content catalog where they could select the sessions that were relevant to the products they were using for their day-to-day operations.
Session Format and Activities
One of our main objectives is to continuously strive to understand the user’s goals, challenges, and pain points. The design team conducted interactive sessions while also capturing valuable research insights. Some of the different types of research are highlighted below.
Interviews
Within the scope of design research, interviews are often more exploratory and help to get an in-depth understanding of the experiences and daily lives of the users.
Abhishek Chouksey conducted an interview on the discovery of business applications in vRealize Operations Manager. He wanted to understand how the users are monitoring their applications within their infrastructure and what issues they are facing with monitoring high demand and complex business applications. Abhishek wanted to clarify what matters for them in the end-to-end journey so that the ideal journey can be delivered when it comes to monitoring the services and applications in the infrastructure.
vRealize Operations Manager is a platform that provides unified IT management, bridging physical and virtual infrastructure, meshing vSphere or hypervisor-enabled systems with Amazon Web Services, and helps in capacity planning for the virtual infrastructure.
Collaborative Designs
Collaborative design is the most engaging method, which helps in getting the best ideas on the table by involving various users and stakeholders with different backgrounds and perspectives.
Yoana Georgieva gave two tasks to the users to answer questions related to how they are currently managing vCenter and ESXi configurations.
The other collaborative exercise she did was a brainstorming activity where the participants had to come up with ideas, recommendations, and assumptions for what could be a centralized configuration management experience from a single point towards multiple vCenters. Also, they brainstormed how this solution might be designed at a high-level.
vCenter is a centralized management tool for data center server management which helps in deploying virtual machines residing on a distributed virtual data center.
Design Walkthroughs with Small Groups
We always give our users the ability to amplify their reach and make their voices heard so we can better shape the future of enterprise design at VMware. In this small group setting, while we cannot gain usability insights, we are able to understand pain points and validate end-to-end flows.
Lusine and I aimed to better understand the user journeys when it comes to monitoring their environment in vRealize Operation Manager. How do users currently monitor their environment? How do they currently use dashboards? Then we showcased the modernized dashboards and infrastructure details as part of the monitoring workflow. The participants provided their feedback on how their ideal end-to-end workflow looks like when it comes to troubleshooting both infrastructure and application issues.
Key Takeaways
We always learn from our users by listening to their feedback.
- Always involve users. We’ve found that they are very collaborative, active, and generous when it comes to giving feedback and sharing their knowledge.
- Analyze all the data collected and present the insights into company executive leadership, and demonstrate the value of user research.
- Do a follow-up session with the participants who already signed up for as a prospective Design Studio attendees.
What did our users say about these sessions?
Even though the participants attended many hands-on labs, various keynotes, and breakout sessions at VMworld, for some, the Design Studios are the best thing about their whole VMworld experience. Participants like these sessions as they feel involved in multiple stages of the design process as a collective.
Conclusion
Overall, VMworld Design Studios present us with the opportunity to engage face-to-face with our real users and customers. From these sessions, we gathered invaluable information about the workflows and needs of the users. Moving forward, we are expanding upon the insights collected and defining the next generation of VMware products.
If you are a designer and already planning to get direct feedback from your users, we encourage you to organize user research sessions like the “Design Studios”. It’s an invaluable opportunity, especially in company conferences where users can attend.
See you next year at VMworld 2020 in San Francisco and Barcelona 😀
We’re hiring!
The VMware Design team is looking for talented designers to help us continue transforming enterprise design. Check out our open positions!
Special thanks to the organizers Afifa Tawil, Safa Dada, and Jehad Affoneh. The designers who conducted Design Studios. Bonnie Zhang, Grace Noh, Sanaea Kakalia, and Kevin McBride for reviewing this article.
