In this final post, I want to discuss the outcome of the workshop. In DevDiv, we work hard to promote a learning-first culture so the best way to describe the outcome of the workshop is through the lens of what we learned.
From my perspective, this was the best workshop we’ve ever ran, in terms of engagement, facilitation, outcomes, and overall customer connection and empathy. I find it amazing that I’m writing this, because I honestly was hoping to tread water here. My goal was to have attendees at minimum say that attending the workshop remotely was an “acceptable alternative” to attending in-person. To me, that would have been an enormous success.
If I was being honest, I was biased toward thinking that an in-person, collaborative, workshop was irreplaceable. However, attendees really enjoyed this remote workshop. In fact, our satisfaction scores were on par with our in-person workshop.
When asked how effective it was attending the workshop remotely, most attendees found it to be extremely, if not very, effective.
One attendee remarked that “this has been my favorite workshop so far,” and another said, “I frankly think the remote forum is superior for this type of workshop. With the combination of Teams and Mural, the digital collaboration tools have surpassed analog tools like easels, sticky notes, and working around tables.”
That’s pretty exciting.
I frankly think the remote forum is superior for this type of workshop. — Workshop Attendee
Now, let’s look at our overall learnings.
Learning #1: Attendees Wanted More Time
Arguably, this is a great problem to have. As I mentioned in my first post, one of the biggest concerns I had was that because we were facilitating everything online, attendees would tire more easily or feel that activity time “dragged on” too long. This concern was backed by findings that have suggested online meetings can be exhausting for employees (some have referred to this as “Zoom Fatigue”).
To combat this, we reduced each day of the workshop by 3 hours. We ran each day from 9:30 AM — 2:30 PM with an additional, but optional, office hour from 2:30 PM — 3:30 PM; where groups could continue working with their coach on their MURAL boards.
Consistently, groups took advantage of the office hours, so much so, many reported that they wished we had just put the extra hour as part of the schedule. Additionally, there were many comments around the desire to have longer breaks. It turns out that the transitioning between the General channel and the private group channels in Microsoft Teams created more of a desire to take a quick break to transition and shift mindsets.
Learning #2 Making Things Drives Engagement
From our past in-person workshops, we knew that our activities were popular because they focus groups on making things, in service of a relevant business goal. These two things: working collaboratively on a shared artifact and working together to solve a relevant business problem have been critical drivers in our past success.
Running the workshop remotely didn’t change that. Many participants noted that they enjoyed working on and solving problems together.
If you’re considering running a workshop, design sprint, offsite, or some other form of collaborative gathering, it would serve you well to steer away from theoretical exercises, where attendees work on things that are irrelevant to their day-to-day responsibilities (e.g. “Let’s build a bridge out of balsa wood!”). The closer you can get to attendees feeling that your workshop is a place to acquire career capital, the better.
Learning #3 Emotional Note Taker
Now this was a cool addition to our workshop! A few months ago, our team was fortunate enough to receive a special presentation from Dr. Eva De Lera, a true design-thinker and thought leader in the space of Emotion Heuristics. Our team was inspired by her work and it got us thinking how we might incorporate emotion heuristics into our workshop, to empower our product teams to look for emotions, through expressions, in their customer interviews.
Because all our attendees and customers would be connecting with each other remotely, it seemed like the perfect time to stress the importance of observing non-verbal communication and behavior.
Each of our workshop groups conducted interviews with real customers over Microsoft Teams. We asked our recruiting agency to instruct our customers to set up their webcams for these interviews. Before the interviews we asked workshop groups to assign one person to be the “Emotions Notetaker”. We also included the above illustration in each group’s Microsoft OneNote notebook so they could refer to it alongside their notes while conducting their interviews.
In each interview, the emotions notetaker would focus completely on facial expressions and non-verbal cues. Even though these interviews were happening over webcams, we were delighted to see that attendees still had some eye-opening experiences.
One attendee told me that while she was acting as the emotion notetaker she noticed that, “the customer was saying one thing, but it was clear in his facial expressions that he didn’t really mean it. I think he was hesitant to give us feedback because he didn’t want to tell us something we didn’t want to hear. I think it’s really important in future interviews that I reassure customers that we really want their honest feedback.”
Learning #4 You Have to Make Learning Fun
Research shows that when people are having fun, they’ll be more receptive and relaxed to learn new things. It also turns out, as a facilitator, when I get to have fun and play off things that are happening in front of me, I have a lot more fun too. The energy becomes dynamic and reciprocal.
In short, positive energy and humor both play a huge part in my teaching style. My role as the facilitator is not unlike the role of an entertainer. My job is to create a positive energy, through kindness and fun, so that energy is carried into group activities and — ultimately — the connections our attendees make with our customers.
There’s a lot of work that goes into creating an entertaining space for people to learn, but if you’re in the corporate training business, you have an uphill battle to get people to shed their pre-conceived biases about what corporate training is.
Many of us have attended horrible corporate training. We’ve had the experience of watching lifeless content being presented onscreen, while we passively watch the speaker and take notes — or if we’re being honest — check our phones and wait to check the mandatory training off our list. Those experiences, over time, have made people reluctant to attend or participate in corporate training events. Frankly, I don’t blame them.
I loathe the word training (this is why we call our training a “workshop”). To me, learning — when done right — should be an experience. And if you do it expertly, it should be a memorable one. My goal isn’t to train you like a lap dog, it’s to shift your mindset and open you to new ways of working and connecting with our customers.
If your mindset is to be truly shifted, it requires that you experience something, not be trained into a new set of behaviors.
To that end, I knew that most of the energy I rely on as a presenter, in an in-person classroom, would be absent because I would be stuck behind a screen. I wouldn’t be able to move around a room, engage people directly, or encourage class participation as easily.
To get around these limitations, I incorporated a few things into my presentations:
- Music: Having music playing, as we waited to get started, provided a palatable energy. Throughout the workshop, attendees commented on how they liked the music and I even caught one of our attendees doing a bit of dancing (which caused me to do a little dancing myself, which further delighted attendees). At one point I even took music requests through the Microsoft Teams chat. This reinforced that the Microsoft Teams call was the attendees’ space — a place to feel comfortable, have fun, and connect with each other.
- Video: To emphasize points, I would play Internet-famous clips like Ballmer’s high-energy “Developers! Developers! Developers!” or Kazoo Kid’s “Fun! Fun! Fun!”. This worked to lighten the mood and even poke a little fun at ourselves. In short, it served as a reminder to not take ourselves too seriously.
- Book Giveaways: Simple to execute but incredibly effective. We asked attendees quiz-like questions about the content and, the first person to answer in the chat, won a free book. Everyone loves a thrilling contest-like giveaway.
- Sound Effects: When you’re not in an in-person classroom, you can’t generate those “support cues” like applause when a group has finished presenting or when an attendee has won a book. So instead, I played sound effects like audience applause to simulate those things. I also played the sound of a DJ air horn to celebrate a big moment or to let the group know their time had ran out and I played the Jeopardy! theme song when I asked attendees a quiz question. This turned out to not only be fun, but incredibly effective in creating a high-energy event.
- Attendees’ Names: One of the biggest advantages of presenting in Microsoft Teams, is that you get a complete list of everyone on the call. You should absolutely leverage this. Welcoming people, by name, is effective and intimately personal (e.g. “I see Stacy has joined the call, welcome Stacy! Ryan! Great to have you here!” “We’ve got Jamie joining us now, thanks for attending Jamie!”). It was a lot of fun to see attendees faces light up when they heard their names.
All of these things came together to create an experience that “broke the script” of what attendees were expecting when they joined us for what they probably thought was “another week of boring training”.
One attendee remarked that we, “did a phenomenal job keeping up momentum and energy in the group sessions; there was a feel of dialing into a live radio show — which was actually really effective.”
Learning #5 Foster Conversation During the Leadership Review
At the end of the workshop, we have each group present their findings to our leadership team. We like to create a “Science Fair” atmosphere, where each group presents their Journey Map, walking our senior leaders through what they’ve learned over the past few days.
It’s such an important moment of the workshop, where all our belonging cues come together and prove to new employees that working in DevDiv is about learning and connecting with customers.
We were able to re-create this experience, rather easily, thanks to MURAL. Throughout the week, attendees worked through each activity on their MURAL and when these activities were combined on their entire canvas, it gave them a complete story of connection with our customers.
MURAL turned out to not only be a fantastic online collaboration tool, but a compelling presentation tool as well. Attendees could navigate to each activity and because they had screenshots of customers’ faces from their customer calls, they could directly align their decision making with customer expressions and emotions. All these components came together to make the best leadership review presentations I’ve ever seen come out of our workshops.
More importantly, our senior leaders agreed. In fact, they were so impressed they asked all our workshop groups to return for an encore presentation to the entire engineering leadership team. This has never happened in the history of our workshop!
Additionally, because the entire presentation was happening over Microsoft Teams, we were able to establish a private chat channel with our senior leaders. This created a vibrant “backchannel” conversation where they could ask questions, get additional context, and make suggestions for follow ups, without distracting our presenters. It was neat to be able to have discussions with coaches and senior leaders while our attendees shared their learnings and to track their reactions in real-time.
Learning #6 Our Coaches Made All the Difference
I’ve saved the most important learning for last.
Without any doubt, the success of this workshop belongs to our coaches. You simply cannot pull off an event of this magnitude (in-person or remotely) without the dedication, positive energy, and expertise of coaches.
I’m confident that transitions between Microsoft Teams channels, navigating activities in MURAL, staying on task, or preparing for the leadership review would not have been nearly as successful had we not had the commitment and tireless efforts from members of our UX research team:
Jake Frieberg, Karl Melder, Sarah Kianfar, Deepthi Nalangula, Mania Orand, Charles Harris-White, Annemarie Fulcer, Irina Smoke, Juan Pablo (JP) Carrascal, Jessica Rich, and Daniel Gottlieb take a bow.
And that leads me to my most important learning:
No amount of software can replace the power of human connection.
Our remote workshop wasn’t successful because we had powerful software. That was a key contributor — sure — but the workshop was successful because we had a group of coaches that were deeply committed to empowering all of our product teams to connect and learn from our customers.
It’s that connection: between our attendees, our coaches, and our customers that will always remain the most important part of our workshop.
Regardless of whether we huddle together in a classroom or log into a Microsoft Teams call, it’s our connection to each other and our customers that forges those indelible experiences that will live on long after our workshop is over.