Week 2 — synthesizing research, then shifting gears
Week two of our IxD project on remote collaboration
Insights from Interviews
We found a higher level of satisfaction with remote teams among our participants than we had anticipated. They included a Portfolio Management Analyst for an energy company, a Design Lead for a consulting company, an independent User Experience Researcher, a Magazine Editor, and individuals working in tech, aeronautics, and institutes. Some appreciated the efficiency of virtual meetings as it afforded them more time to work, others found their speed and productivity had decreased.
Communication wise, there was less “water cooler talk,” with conversations feeling more formal, stiff, or transactional, and resulting in less personal knowledge of one another. Some reported using fewer facial expressions or gestures in remote meetings, while others over-compensated by exaggerating them. In order to really connect with folks on their team and beyond, being more proactive than you would in person was mentioned multiple times — whether it’s having to call someone rather than walk to the elevator to remind them of something before a meeting, or schedule a Zoom “coffee break” rather than pop by their desk to see if they’re free. On the bright side, there is less pressure to be performative (e.g. staying late at the office), and more focus on the quality of work.
There were complaints were around live video meetings, too: as one participant said, “talking over someone always feels sh*tty.” One strategy they used was to explicitly hand off to another person, rather than allow more senior or talkative participants to fill space. They had to be creative to have important side conversations during meetings, whether over text, through Zoom chat, or another method. The experience of joining a new company and explicitly discussing teammates’ working styles, as well as company culture and values, seemed heavily dependent on the individual companies’ policies and procedures.
We considered how some issues may arise from considering video meetings as a substitute for in-person meetings, and not a valuable method of communication in their own right; one participant no longer needed to worry about regular commutes from Los Angeles to San Francisco, for example.
Current Landscape: Some examples we gathered
mmhmm app: Allows presenters more flexibility to display themselves alongside presentations and other video elements on remote video calls
Personify: another tool for on-screen presenting with personal flair
ARCall: places your coworkers’ avatars in your space using AR glasses
Spatial: a VR space for remote work collaboration
VSGI Immersive Telepresence: a wall of video that seemed best suited to in-person teams collaborating over a distance
VROOM Telepresence: A two-way robot/VR platform that allows for a remote user to see their office space, and for their coworkers to see their avatar in the space.
Start Beyond: Immersive VR for spatialized learning and growth solutions
Vuforia Chalk: a video platform that allows for pausing and annotation so that remote experts can provide guidance to on-site engineers
Sococo: a platform similar to Gathertown for spatializing the virtual office, school, or other collaborative space.
Early Storyboarding
With these ideas and our research in mind, we set out to storyboard some possible scenarios that could help make synchronous video meetings for office workers valuable and desirable. While there were a few existing VR technologies out there for virtual meetings, we did not want to pursue a VR-based solution. We were concerned not only about hardware for prototyping but also the likelihood and financial feasibility of providing VR headsets to all employees.
Our initial round of rough sketching and storyboarding included imagining how 3D space might translate to online meetings for a “world-cafe”-style discussion, gesture-activated glowing orbs to help quieter participants in meetings speak up, Zoom functionality to encourage icebreaker questions, visualizing each team member’s progress, and imagining parallel conversations in virtual space. We talked about whether it would be beneficial for a computer to track body gestures (what if someone was a doodler or knitter off camera, and wouldn’t want those particular movements to be picked up?), and what functions might be more cumbersome than useful to have off-screen (e.g. to be controlled by gesture) rather than on. The ideas that we brought to class for feedback are below.
Indicators similar to video games that restrict the amount of time garrulous participants could spend in a meeting, provide proactive feedback to the group when someone is about to speak, and gauge energy levels. Our feedback from speaking with our professors was that we would need to consider how the rules of the system would be expressed to participants, how it would determine and express who wanted to talk, and whether participants might want their energy levels publicly versus privately displayed. It would be interesting, though, to have gamified elements, as well as AI altering the flow of meetings. This turned into an interesting discussion of how a virtual moderator might diffuse responsibility (e.g. in requesting breaks).
A physical device that would read and respond to gestures, creating a shared language of colored light and signals on screen for things you might not unmute for or be able to convey in our current Zoom setting. From speaking with Q we realized this was likely already possible via webcam; he shared with us this js5 coding challenge, which accomplished a similar goal using on-screen cartoon overlays. He encouraged us to think further about the interactions that would really necessitate additional gestures.
More flexible ways of creating breakout rooms based on physical spaces, which would allow you to “overhear” or join conversations by moving your focus about the room. There were still questions here around perspective (we came in familiar with Gathertown, but found the overhead perspective a bit disorienting), formation, and appearance (i.e. would it be avatars or live photos?).
Overall, we needed to clarify: what was it about nonverbal communication that we wanted to focus on and improve? Attention, cognitive load, emotion? What were the specific deficiencies of video meetings we felt we could improve? What definition of collaboration was guiding our solution? Dina suggested it might help to narrow down what we were not designing for in order to identify the specific problem we were addressing and solving. So far, our target audience was still as broad as “teams working on a deliverable”.
Working towards a presentation with guest reviewer Cagri Zaman of MIT’s Virtual Experience Design Lab, we focused on the question of: How might we help design teams to pick up nonverbal cues and equalize participation in video conferencing? We expanded upon a scenario in which overlays and a virtual moderator might take the onus off of individuals to take breaks or manage the flow of conversation in meetings.
Our reviewer probed us to consider a more specific goal for video conferencing, and again had questions about the function of the moderator (we’re not sure we clearly communicated that it was virtual), and how much AI would be able to measure behaviors and provide feedback. Additionally, what would be the specific goal for tracking speaking times — how might the team benefit and use this information? There were also concerns that this would become another virtual assistant project (which we had just completed for our first project). In speaking with Dina and Christianne afterwards, they also urged us to consider how publicly you might want your energy levels to display, what might display when someone has their camera off, and the contexts in which we might want to implement the solution.
It was becoming clear to us that while virtual meetings were a prevalent form of remote collaboration, our target may have been too broad and too screen-focused to be the best contender for multi-sensory inputs and outputs. Frustrated but optimistic, we decided to break and come to our next meeting with fresh and open minds towards potential adjustments to the focus of the solution.
Changing direction
As we took a step back, it made us think about all the types of work that, unlike many teams of consultants or designers, for example, do not have two dimensional deliverables. Early on we had talked about architects — who else’s work might we have left out of our initial considerations? We took some time to brainstorm via Miro again under the heading, “Occupations where video conferencing is laughably ineffective”.
As we talked through them, the areas of physical training and therapy stood out as areas that could greatly benefit from expanded sensory inputs and outputs. One of Matt’s friends was a massage therapist who hadn’t yet found a way to run workshops remotely, and we’d all pivoted in our own fitness practices since March. We decided to focus on martial arts as a starting point, since it’s a very form-focused sport, and there was a martial arts studio next to the lobby of one team member’s apartment. We met with Dina and identified important next steps, including identifying the difficulties of remote training specifically (for example, the trainer can’t come and move your hand), whether it’s designed for beginners, intermediate athletes, or experts, and whether the collaboration is with a live instructor.
Exploring existing technology in the fitness space
While we were familiar with biotracking devices for fitness, we were curious about other multi-sensory tools and technologies that had been created for various sports. Dina showed us vGolf, a pair of mixed reality glasses to help golfers improve their stroke on the course.
We also found there are glasses with similarly relevant overlays to help runners and bikers, as well as swimmers track their stats and goals in real time without looking at their phones.
The HomeCourt app adds a smart overlay to your mobile device to enable athletes to do gamified drills with moving targets, and the Zombies Run app adds a location-based audio narrative motivating you to run, if not for your own health, then from the threat of Zombies chasing you.
Through Dance Reality, users will be able to practice dancing with augmented reality as the teacher right from the smart phone. The app shows you where to put your feet using an image of foot prints. When you’re ready, the footprints will begin to move while a voice counts the beat. Although the app is not intended to be a substitute for in-person dance instruction, it will get you started.
There were more expensive physical devices, too: the Mirror allows you to watch your form, connect to a heart-rate monitor, and track progress over time, as well as adjust exercises for any injuries that might require modifications. The Peloton incorporates elements of healthy group fitness competition into your remote ride, and FightCamp incorporates a connected punching bag and pair of gloves to help each at-home boxer track and achieve individualized goals.
Next Steps
At our weekly standing meeting, we decided we were more interested in live fitness than recorded classes, especially after seeing the existing products in the landscape. We brainstormed how we could combine and address issues in remote fitness by having sensors or motion detectors for postures, as well as a mixed reality headset to receive multi-sensory feedback from the instructor. We also wondered about the social element of group fitness — the cost is typically more feasible than one on one lessons, but would you want to “see” other avatars or people in the virtual room, similar to the experience a Peloton provides? We adjusted interview questions towards people teaching and practicing remote fitness, and Matt scheduled times to connect with a barre instructor, yoga instructor, martial arts trainee, and massage therapist for more information.
