Doing Hybrid — Thoughts after CHI2022 and more.
My experience of CHI2022, as a remote participant: it was a huge success, and a massive step forward for the conferences that I see in my research areas. With a few tech problems aside, I was able to attend keynotes, ask questions of authors in papers sessions, take part in workshops, and even host a Special Interest Group (SIG) discussion for in-person attendees. It led me to think a lot about what worked, but the benefit of experiencing it meant that I could also think about what still doesnt quite work yet. CHIWORK’22, although smaller and single track, took further steps towards an awesome hybrid attendance. Meanwhile I experienced some alternative routes, with fully async (DIS) and different treatments for in-person and remote (FAccT).
Full disclosure: I am not involved in any hybrid working groups (e.g. with SIGCHI), nor am I speaking about any plans set from the CHI or CHIIR steering committees etc. I have been lucky to be in a place to discuss hybrid plans, observe discussions of hybrid plans, and experience different versions of hybrid.
The main starting mistakes that I see across conferences are 0) that a one-size-fits-all approach is used for all sessions, and 1) that the number of people/roles involved are drastically underestimated. I then move on to 2) technology thoughts and a little about conference costs. Together, this guidance is summed up with the diagram below.
Some key terms I will use: Roomers to mean in-person in-the-room participants, Zoomers to refer to online participants (regardless of whether the technology is zoom).
Step 0— The type of hybrid
Before we can consider what we really need to make hybrid work, step 0 is to acknowledge that theres a spectrum of ways in which an organiser might approach structuring meeting types and that these need different levels of support:
A) Hybrid Big Broadcast — in this setting, large conferences simply broadcast large talks (e.g. keynotes) to a remote audience in the same way that an in-person audience experiences it. Unlike normal papers sessions, even in-room attendees do not typically ask questions at the mic, but have to submit questions via paper or technology, to be curated by stage hosts. The focus of these is often highly produced, such the presenter and the slides, if not also additional stage actors like sign language interpreters, are shown effectively across screens and online broadcasts. The challenge is to make sure the broadcasted experience is equally good online as it is in person.
B) Hybrid Audience — in this setting the aim is make the in-room and online audience experience the same for e.g. a normal paper session or panels. This means democratizing the zoomer engagement in a way that complements rather than interrupts the roomer audience, and making sure that hosts and presenters can be equally aware of both audiences: whether they can see and hear the speaker effectively, and can ask questions of the speakers. The challenge is to make sure both audiences can take part equally, and be aware of each other.
C) Hybrid Meeting — in this setting (such as workshops and Special Interest Group Discussions), the aim is democratize the experience such that everyone is an equal participant to discussion whether online or offline. Hosts may manage these sessions, but there isnt a divide between speakers and audiences. This means mutual awareness of the alternate-mode participants, and equal management both roomer and zoomer experiences. The challenge is to manage conversations and equal participation across groups, and make both sets of participants equally aware of each other.
D) Hybrid Presenting — in this setting the aim is to make it so that a remote presenter can present to a physical audience with the same experience that an in-person presenter has. This means making sure that the remote presenter can be as aware of how their presentation is being experienced in the physical room, as an in-person presenter would be, and can field questions from in-person and online audiences. The challenge is make sure the remote presenter can get the same experience as in-person presenters.
E) Hybrid Hosting — in this setting, the aim is to allow e.g. remote session chairs or discussion organisers to run sessions. This means making sure that such remote hosts can be aware of presenters in the room, tech problems in the room, audiences in the room, etc, as part of being a host. The challenge is to make sure that remote hosts can be in control of the room in the same way as in-person hosts.
In my opinion, its really critical to comprehend these differences for each part of the event, and they dictate the roles that you need to facilitate both roomers and zoomers. One way a conference can manage its hybrid scope is to exclude certain options (like hybrid hosting or remote presenting), or to merge types if e.g. smaller conferences treat ‘big talk’ broadcasts like normal paper sessions.
Step 1) — The roles involved
Its important here, I think, to focus on roles rather than people. In smaller settings, its possible that some people could manage several roles, but its useful to explicate all of the roles involved that make things work.
Old style in-person papers sessions — Lets start by thinking about our most familiar dynamic (B from A-E in the prev section). The roles being involved here are: host/chair that manage the session, presenters that provide content, room-monitors (e.g. Student Volunteers (SVs)) that run mics around or go on errands to solve emergencies, and AV people that make sure the room is functioning (but dont really focus on if the experience is right). Each of these roles are necessary, and people struggle in roomer meetings when they have to adopt a role they werent expecting (hosts solving AV problems, etc). This is why bigger events such as CHI pay professional service teams for providing AV and solving AV problems. But still if theres not an SV to run and tell someone theres an AV problem, then an audience member has to volunteer because the host cant leave the room. etc.
A) Roomer broadcasts — Beyond a normal set of roles above, roomer broadcasts typically add additional roles, separating out A and V to have someone in charge of effective visual experiences (across slides, camera views of presenters, lighting, and even views of additional aspects like sign language presenters). This is a production role, for the room — beyond making sure the AV is functional, the production role is thinking about the experience in the room that is beyond the control of the host.
For Hybrid broadcasts we need to duplicate some of the roles that are audience focused. It needs a remote production person to consider the best experience for remote attendees, what they need to see, in what balance, etc. It also needs remote monitors — SVs dedicated to attending to the zoomers, who have the same capacity to bring concerns to the hosts. A new common technology gap is cross awareness of zoomer and roomer audiences, and so a recommendation for the two (in person and remote) production roles is to consider how to include audience views in the respective productions.
B) Hybrid Audiences (where we started above with roomer audiences) — Hybrid audience meetings also need to duplicate some key roles. They need zoomer AV people that are making sure e.g. zoom is working effectively — slides are being presented and audio is reaching the remote audiences, and that when needed zoom can be seen and heard in the room. Zoomer monitors are also needed, so that SVs are responsible for quickly raising and solving zoomer problems with the experience — this is critical so that zoomers arent desperately trying to get the attention of in-person hosts, and indeed the roomer host can at any time check across the room with the zoomer monitor who is experiencing the event as a zoomer. Hybrid audiences could also benefit from a zoomer host arranged to work in tandom with the roomer host (as per CHIWORK’22). This is beneficial to help the zoomer host be aware of the zoomer audience (“yes, we have a question online”) — especially when it is hard for a zoomer host to maintain both awarenesses, but may create conflict in hosting responsibilities if both try to take the lead.
This challenge extends to question time — everyone needs to be able to hear both audiences. In this case, we recommend an additional screen in the room showing the online audience, ready with their sound to broadcast to the roomers, and a roomer audience camera that shows the roomers to the zoomers. The roomer microphones used to ask questions also need a camera to show the speaker to the zoomers, as well as the audio of their question. The roomer AV may wish to be ready to spotlight different views of the zoomers.
C) Roomer meetings — These typically have the same main roles as Roomer audience events, except that the roles of audience and host and presenter (if there are any) are all interchangeable — in that they all meet in equal standing (where a host is sometimes just coordinating, but then participating too).
For Hybrid meetings, the challenge is to keep this experience dynamic between zoomers and roomers. If zoomers and roomers are supposed to break out in to groups and talk, this is nearly impossible over a single sound system and in-room zoomer screen. Conversations in the main zoom would be broadcast over the whole room, and zoomers cant hear all the different discussions in the room. Different tables cant involve different zoomers on a big single screen. In this mode, you need a zoomer host, a zoomer sv, and zoomer av people to manage discussions on zoom in the same way that a host, sv, and av team would manage a room. My experience is that the zoomer host then has to manage breakout rooms in zoom, even if there is only one zoomer group, so that it can happen in a place that isnt tied to the audio in the main physical room (that is not added to the breakout zooms). The roomer and zoomer AV people/producers need be ready to dynamically move between whether the roomer and zoomers are separated for discussions or joined for plenaries.
When it comes to plenary periods of hybrid meetings the anyone in the room issue means that we normally have a roaming mic for roomers. To translate this into a hybrid meeting, having a roaming camera (or laptop) and mic means that the online audience can also all look at the person currently presenting to the room. Likewise, if a zoomer is ready to talk, the roomers will need a view of the zoomers and be ready to hear the zoom audio. A personal recommendation is for the zoomer and roomer hosts to have a fast backchannel to talk, so that together they can ‘read the room’ and when its ready to move between discussions. If people are going to be presenting their table work to the rest of the meeting, then using digital tools even in the physical room may help, but otherwise a roaming camera (with the roaming mic) can allow zoomers to see what people are presenting from e.g. posterboards and flipcharts.
d) Hybrid presenting— as a reminder here, the zoomer presenter needs to be aware of both audiences as possible, including what the roomers are experiencing. Instead of translating a roomer presenter to both audiences, we are presenting a zoomer to both audiences, such that they can also ‘read the room’. In this case, in the room, the audience needs a view of both the presenter and their slides (which can typically be achieved by spotlighting), but really the zoomer presenter needs a view of the roomer audience. At CHIWORK, zoomer presenters were hosted by a zoomer host, in which case they will also benefit of this view of the room.
One interesting challenge is to replicate the ability to know what the audience is seeing about them. Can the roomers see my slides correctly? A bonus recommendation is to replicate a roomer view of the projector screen to the zoomers that the zoomer presenter can check.
e) Hybrid hosting — from my speaking to others, the idea that a remote person, alone, may host a physical room includes a lot of expectations: ability to read the roomer room, ability to field roomer questions, ability to check that the AV is working in the room as expected. This is a lot to translate to the zoomer host: a view of the roomer presentation screen (to check), a view of the roomer audience, a view of the roomers asking questions, a way to talk privately to the roomer presenters (!). My experience is that this worked a whole lot better when there are paired roomer and zoomer hosts that can be responsible for these things in their own spaces.
Its a provoking question to ask why its so hard for a zoomer to host a hybrid physical room session, and then ask why we thought roomer hosts would be able to do both.
2) Thoughts about technology and cost.
In most of these cases above, each session would benefit from several video conference connections to make the hybrid meetings effective.
For the zoomers: 0) a view of the roomer presenter/host, 1) a view of their slides, 2) a view of additional stage people (sign language) if relevant, 3) a view of the roomers audience, and 4) a view of the roomer audience members that are asking questions. Optionally for remote presenters 5) a view of the roomer projection area. A zoomer AV has a lot to balance there through spotlighting.
For the roomers: an extra screen (not the main presentation slides) showing: 0) the zoomer audience, 1) the zoomer host, 2) any zoomer audience member asking questions. Additionally 3) the ability to show the zoomer slides on the main projection screen. And optionally 4) a view showing the roomer presenter how they appear to zoomers. A roomer AV has a lot to balance there, especially managing sound in and out of zoom.
This is a non-trivial combination of multiple zoom connections, and audio is a massive challenge. Its not surprising that for a multi-track conference, we are talking about having e.g. 20 parallel rooms involving any combination of hybrid types A-E above. Its not surprising that creating a hybrid conference can dramatically increase the cost of conferences, rather than be a cost-free addition that remote attendees can get for free. This either involves a dramatic rise in SVs with expected expertise in sound systems, zoom settings, etc, or a massive increase in paid AV team members. CHI’22 reported hybrid trebled the on-site AV costs of the conference, but CHIWORK’22 as a single track conference was able to achieve the best dynamic hybrid experience through an awesome team of SVs that were committed to working this out (and of course, the recent SIGCHI VP for conferences as one of the general chairs).
One interesting thing that zoomers get that roomers do not, is the online chat as a backchannel. Zoomers can talk to each other privately about the roomers in a way that the roomers cannot see. Some conferences lock this down and encourage equal chatter through e.g. discord instead, that both roomers and zoomers can observe and take part in, without dozens of roomers joining the zoom call just to engage in the chat. The cost, though, is that typically yet another SV is needed to watch the chat as a source of issues and questions.
The necessary backchannel between roomer and zoomer hosts, and potentially between them and SVs, is really an interesting development for the when considering ‘co-hosting’ roomers and zoomers. In my experience, these have been missing (such that they then happen in full view of the roomers/zoomers), or they are enabled by existing adhoc personal comms, like friends being connected on whatsapp/signal/etc.
In terms of dynamic broadcast of roomer presenters, speakers, audience, and questioners could be achieved by roomer AVs through something like OBS by a zoomer producer, rather than multiple laptop cameras being spotlighted by an online zoomer AV, but they would need a zoom view to check what the experience they have created looks like. This is not so easy for the roomer view of the zoomers. Managing audio is super difficult, if you want a single audio feed going to zoom from the roomers. A single feed from a sounddesk that has presenter and audience mics, as well as presenter laptop audio, to the zoomers needs careful management by a dedicated zoomer producer.
Thats a wrap. I’d welcome any comments and thoughts through @drmaxlwilson on twitter. I have not discussed different social experiences at conferences, and the pros and cons of platforms like gathertown, where you can run into people (which I think is great!) and scheduling sites like hubb that link people to remote parts of hybrid experiences in other platforms (which are organised). I am interested in, but dont have good thoughts about, social events where people can link between roomers and zoomers. But one lesson we did learn at CHIWORK is that some zoomers get their social experience when the conference leaves breakout rooms open during the breaks for zoomers to safely chat without their AV being broadcast over a physical room of roomers.