Lessons learned from our virtual research sessions at VMworld

Research at a Virtual Conference

Design Studio is a space for the Designers and Researchers at VMware to directly connect with our users. This program was created over the last few years, to ensure the design team is prioritizing user feedback and providing customers and partners with a space that allows their voice to be heard.

Due to the unexpected circumstances that COVID-19 brought to us this year, VMworld took place virtually for the first time ever. Inevitably, these restrictions had a large impact on the structure of Design Studio research sessions conducted at VMworld.

With minimal notice, we transitioned from moderating physical breakout sessions to sharing feedback in virtual sessions. Although virtual sessions have their benefits, they present challenges around engagement. For example, a Zoom room makes it more difficult for people as they are generally more hesitant to speak up in virtual group settings with strangers.

It also becomes difficult for moderators to pick up on body language and cues. Since we anticipated these issues, we utilized tools that we felt would enhance the user experience as best as possible with the given circumstances. Although there are recruitment advantages to on-site Design Studio from people stumbling upon in person and dropping in, we were able to reach a larger demographic of participants with our virtual sessions. Because VMworld was free and companies were not required to pay for travel, we were able to expand the reach of our sessions to a new audience. There was also a higher percentage of attendance based on sign ups compared to past sessions at VMworld!

Tools to Enable Research

Our early identification of potential challenges in a virtual Design Studio helped us curate a set of tools for designers to set them up for success. We identified tools for easier documentation, analyzing notes as well as data, getting real-time feedback, and scheduling. The tools our team selected included Dovetail App, Miro, Zoom Polling and Microsoft Forms.

Miro

Miro is an online collaborative whiteboard platform that allows participants to express their ideas and suggestions without the pressure of directly speaking up in group sessions. We used Miro to boost engagement throughout sessions because of the interactive and collaborative platform it provided for us.

Dovetail App

Our team also used the Dovetail App to track and analyze our notes and data. Moderators and note takers could easily collaborate and tag notes, helping them formulate insights that they later shared with stakeholders.

Since these tools saved participant feedback in real time, moderators optimized their time that otherwise would have been lost relistening to session recordings.

Polling: Microsoft Forms & Zoom

For polling, we used Microsoft Forms and Zoom Polling. Our moderators were easily able to survey participants during larger sessions to better understand the audience through their responses. Zoom Polling had immediate results that everyone was able to see and analyze which allowed for instant discussion.

On the other hand, scheduling remote sessions at such a large scale was a prominent challenge we faced this year. Due to time constraints, we resorted to using tools we already had access to. Scheduling 90 different calendars is never easy, but we made it work with Acuity Scheduling. Although Acuity works great for sessions throughout the year, it was not the best tool for this scale. In the future, we hope to find a tool that will help us automate scheduling and reduce the manual work.

Chat with Yi Zhang, Product Designer

To reflect on our team’s Design Studio experience at VMworld, I spoke with Yi Zhang, Product Designer, who led a handful of research sessions. I asked him a couple questions around his tooling, user engagement, and takeaways.

Can you tell me how you selected the tools that would be best for your virtual Design Studio sessions?

Dovetail, Microsoft forms, Miro, Figma, Zoom and Zoom polls are the tools I used for all the virtual Design Studio sessions!

A month before VMworld, I used Dovetail to draft my Design Studio sessions and reviewed them with my team. Like how do we lay out the session? What does the structure look like? What are the top questions we would like to ask customers? Those prepared notes helped me and my team practice and make sure we delivered high quality workshops.

At the same time, we created pre-session surveys on Microsoft forms to help us understand our customers better before sessions.

Miro is another important tool that we used for creating interactive slides with customers. We added some slides with prepared components so our customers could easily focus on the workshop itself. By using Zoom, I was able to switch from my Figma demo to other screens.

At the end of every session, we used Zoom polls and got real-time feedback from customers about some high-level questions.

After multiple Design Studio sessions, Dovetail became the single place for all the notes from my team. We also had tons of sticky notes with comments from customers on the Miro board. I was able to summarize the highlights with my team and share with broader stakeholders.

Those tools made virtual Design Studio sessions smooth and interactive.

How did you encourage engagement from participants during virtual sessions?

I learned two things from virtual sessions in order to engage participants.

1. Community. I had both big sessions and small sessions. I had a big session which was about 20+ participants. What I found out was participants were a little silent when I talked to them or interacted with them. So, I changed the strategy. At the beginning of each session, I introduced myself with passion, welcomed all the participants and asked them to introduce themselves. This results in them feeling like they are a part of this community, and they become willing to talk more.
2. Easier consumption of tools. We used Miro for virtual interactive sessions. Even though Miro is easy enough to use, for the first-time user, it still has a learning curve. And we didn’t want participants to waste a lot of time on learning the tool. We noticed that participants had a hard time to understand how to use it for the first session. Then I created a lot of presets on the Miro board. Participants just needed to type and move those sticky notes around. We wanted to make sure the experience is seamless.

Can you talk about the pros and cons of using these tools in the virtual sessions?

Pros:
1. Easy to access for participants.
2. Easy to collaborate and document everything. And that makes summarizing findings and insights easier after all the sessions.

Cons:
1. Participants need additional time and effort to be familiar with tools.
2. Multiple tools are required. But it is easier if you have an idea on how to embed those tools into your sessions.

Any learnings to improve sessions in the future?

It was a great opportunity for me to learn from all the sessions this year. I found participants had different expectations when they came to different sessions. For example, participants were ready for collaboration on Design Studio sessions. But for TAM (Technical Account Manager) Demo Day sessions, participants expected more on seeing new technologies and providing more feedback and comments.

My team and I worked closely during the sessions and we customized the structures by those differences. And we collaborated super efficiently.

All the sessions this year gave us better clarity and direction for virtual sessions. There are a few aspects I would like to improve with my team for future sessions:

1. Better handle on timing and pace. Different customers focus on different goals since they have various workflows. We can come up with different plans for different session structures. For example, some participants might have more questions. How do we make sure we can present things we want and also answer as many as questions for participants? And some participants might have fewer questions, so how do we effectively use extra time?
2. Better engagement. This year we tried to engage participants in multiple ways. We always think about what would be a better way to make participants more engaged? What would be a better way to make the sessions more engaged? How could we inject more energy into our future sessions?

Takeaways

Overall, VMworld was a success! Although it may not have been the experience Design Studio had anticipated, we made the most of the situation presented to us. Transitioning to an online program brought us its fair share of challenges, especially regarding recruitment. Our team focused on other aspects of research by adapting new tools that maximized engagement during sessions. All in all, we were pleased with the feedback we received from our 750+ participants and look forward to implementing changes in the upcoming year to further improve the Design Studio experience for both our team and our users.

You can also read more about Design Studio in person experiences in San Francisco and Barcelona last year.

Are you interested in participating in a Design Studio session? Learn more and sign up here!