Hybrid > Physical + Virtual
--
It’s a wrap! The first hybrid ICER conference is over! And it wasn’t too bad.
It started three years ago, at a July 2019 Dagstuhl meeting, when I was cornered by the to-be ICER 2022 Program Chair with an offer I couldn’t refuse: join him to organize the flagship international conference in computing education research as the Site Chair in beautiful Lugano. Of course I could have turned down the offer, but getting computing education researchers from all over the world to my university in Lugano was simply too tempting. Although I have to admit, I was worried about the challenge of organizing a major conference, something I had never done before.
I soon found out that chairing ICER means being involved in two instances of the conference: first as a junior co-chair, and in the subsequent year as the senior co-chair (this applies both to Program Chairs and Site Chairs). So I was going to be involved in the organization of ICER 2021 before being responsible for ICER 2022.
In Fall 2019 I put together a site bid for ICER 2022, which was promptly approved by the ACM SIGCSE board. We were going to follow the traditional ICER conference structure: a single-track physical conference, using roundtable discussions between each paper presentation and Q&A, plus poster sessions, lightning talks, a work-in-progress workshop, and a separately organized doctoral consortium. We expected around 150 participants, and we were going to host the conference at my university, USI Università della Svizzera italiana, in the “mediterranean part” of Switzerland.
Learning the Ropes by Co-Organizing ICER 2021
ICER 2021 was going to be held in Charleston, South Carolina, USA. I was going to observe my senior Site Co-Chair to learn how to organize a real ICER conference.
And then it happened.
For a while we believed that come August 2021 the pandemic would be over, and that we could run ICER physically. But eventually we had to bury that idea. So we had to organize a virtual conference instead. I volunteered to focus on designing the virtual experience. After looking around and talking to a lot of people in similar situations, we eventually settled on using Clowdr as our virtual conference platform. We had an excellent connection to the small team at the Clowdr social enterprise. They were super helpful and even developed new functionality to support our unique ICER roundtable discussions.
With a lot of tweaking and configuration, we managed to put together a virtual conference entirely in Clowdr (including pre-recorded paper presentations, live roundtable discussions and Q&A, virtual coffee breaks, poster sessions, and lightning talks).
The feedback on the ICER 2021 Clowdr experience was generally positive. Several participants called it their best virtual conference experience so far. We were happy to have gained a deep understanding of the platform, which would allow us to stay virtual, or to go hybrid, for ICER 2022.
The Hybrid ICER 2022
Eventually it became clear that ICER 2022 was going to be hybrid. Travel restrictions were lifted, and traveling to conferences became possible again. We were in good shape: Clowdr, now renamed to Midspace, had developed features to support hybrid conferences.
And then it happened.
The week we were going to meet with the Midspace team to agree on a contract, Midspace went out of business. My interpretation of what happened is that their main developer, who was absolutely fantastic and had gone above and beyond in helping us to create a great virtual experience for ICER 2021, had burned out.
With Midspace gone, and little time left to find something comparable (and comparably affordable), we had to switch to our fallback solution: Zoom and Discord. This certainly was not a novel combination per se, but I think few others had used it to run a hybrid conference like ICER before, with a community very keen on having extensive interactions and discussions, and not just a bunch of presentations.
The Hybrid Experience Team
We soon looked into hiring an external company to run the AV for us. However, the prices were out of reach for a small conference like ours. We had to do it ourselves. Luckily, we were able to put together an outstanding Hybrid Experience Team made up entirely of academic volunteers:
- Roberto Minelli, from the USI Software Institute, helped us in planning and putting together the complex technical setup needed for our experience.
- Andrea Gallidabino, from Luce, the Lugano Computing Education research lab at USI, not only served as Web Chair, but he also took on a major role in planning and executing the hybrid experience.
- Jana Seep, from Program Chair Jan Vahrenhold’s group at WWU Münster, helped with planning, and throughout the conference she directed the entire experience.
- Luca Chiodini, also from Luce, helped with the technical planning and setup and was essential for capturing the audio of in-person attendees.
- Carolin Wortmann, also from Jan’s group, was instrumental in helping Jana with the daunting task of controlling the entire show.
Each of those volunteers were absolutely essential for the conference. They worked full time and didn’t get to enjoy the content in a way that traditional student volunteers might.
Main Conference Setup
USI provided us — in addition to the actual conference space — with a professional-grade AV setup, including a control room overlooking the conference space, and all kinds of fancy light and audio equipment, three individually controllable large projectors, and a wired and wireless network setup. On top of that, the USI Faculty of Informatics and the USI Software Institute provided equipment we needed to complement the university’s setup, such as a whole bunch of laptops and speakers to use for various purposes, an Elgato capture card to capture the presenter’s laptop, and a powerful computer for handling all the incoming NDI streams, mixing them in OBS, and streaming them to Zoom.
Here are some key aspects of our conference room setup:
- Physical participants sit around roundtables in the room, see the presenters on stage and their presentations on the big projection screens, and chat on Discord.
- Virtual participants are at home (or locked up in their hotel room due to Covid), watch the Zoom webinar, and chat on Discord.
- Physical presenters stand at the lectern on the podium, using their own laptop, or a laptop we provide, to present their slides. A hybrid experience team member on the podium (Andrea) helps them to connect their laptop. They are filmed by a webcam on the lectern, so that virtual participants can see them live. For those who request to be able to walk around during their presentation, a hybrid experience team member continuously adjusts the webcam to keep them in the picture. Their audio is captured in one of three possible ways (fixed mic at the lectern, wireless lapel mic, or portable hand mic) and fed to the room speaker system as well as into Zoom. Their presentation is captured the same way it would be if they directly projected from a laptop, but it is fed to our gear so we can project it on the main projectors as well as stream it into the Zoom webinar, with the speaker video overlaid at dynamically controlled locations to avoid the inset picture overlapping slide content.
- Virtual presenters sent us their pre-recorded talk, which we play by projecting the video on the room’s projector, feeding the audio into the room speaker system, and streaming both into the Zoom webinar.
- During roundtable discussions, physical participants talk at their tables and come up with questions to be asked. Virtual participants talk at virtual roundtables (video chat rooms within Discord), and submit their questions to the paper’s text channel in Discord.
- During the Q&A, the session chairs, who were all present in person, are on stage (with the physical author), and ask for questions. A hybrid experience team member (Luca) runs around with a wireless microphone to the table asking a question. The program chair, who is present virtually, reads the questions (physical participants hear him coming out of the room speaker system) that were written by virtual participants in the paper’s text channel in Discord.
We record each presentation (virtual and physical) in Zoom and subsequently make it available via a link posted to the paper’s Discord channel. To do this we have to stop the recording at the end of each presentation so we get a separate recording for each talk, which Zoom automatically uploads to the Zoom cloud. This provides a way to watch all talks asynchronously. Treating the recordings of virtual and physical talks the same provides a homogenous way to access them. Moreover, Zoom also provides an automatic transcript of the talk.
Throughout the entire time, two hybrid experience team members (Jana and Carolin) staff the “regia”, the director’s booth overlooking the conference space. They control the audio (mute/unmute, adjust levels), the video (switching between different streams and overlays in OBS), play pre-recorded videos, decide what to project on which of the three projectors (including supporting information, like roundtable instructions, or the list of Zoom Webinar participants and a Google Slide where they placed their pictures, and slide shows during breaks), control lights, play break music, and start and stop the Zoom webinar to capture and share the recordings.
The Discord Setup
Virtual participants had to register in two separate platforms: Zoom and Discord. We created a new Discord server specifically for ICER 2022, and the Hybrid Experience Team developed a Discord bot to provide some extra features. This allowed us to generate invites for each registered conference participant. The team manually approved each participant who followed the invite (and who set their screen name to their full name) by giving them specific roles and thus providing them access to the many Discord channels we used.
- Paper Channels. We created a text channel for each paper. In there, the bot posted the meta-information about the paper, including links to the ACM DL where one could access the PDF. Authors could post their own extra information (from a simple “Welcome, I’m one of the authors! Happy to answer any questions you might have!” to supplementary material like slides). Later, we posted the links to the Zoom recording of their paper presentation.
- Roundtable Channels. We created a bunch of voice/video channels for virtual roundtable discussions. To make them a bit more attractive, we named them after local towns and posted some info and a picture of that location. This way also virtual participants got a bit of a feeling for the beautiful conference location.
- General Channels. We created channels for help, welcome, hybrid introductions (where we asked participants to say hello and to share their most memorable food experience, leading to a stream of very interesting messages providing fun insights into people’s tastes), and leisure activities (where we provided specific recommendations and information about hikes, swimming, and other social activities).
We also provided other channels, which I will get to in a second.
Conference Schedule
We decided to schedule ICER across three days. There is a trade-off: a longer conference has a higher cost (especially on the physical side, including the cost of extra hotel nights), but it allows for a more relaxed schedule with more time for social events and networking. For virtual participants, shorter days may prevent the Zoom fatigue of long hours in front of their computer screen.
In our schedule each research paper talk got a 30-minute slot. We bunched slots into sessions of 2–3 papers. Within each paper slot, we used 18 minutes for the talk, 6 minutes for roundtable discussions, and 6 minutes for Q&A.
Hybrid Breaks
One of the most important parts of a conference are its breaks. Those are the times you get to network, to sketch new research projects, and to start new collaborations.
We wanted the breaks to be hybrid: that is, we wanted to allow physical and virtual participants to interact with each other during breaks. For this we set up five physical lounges, coupled to five Discord rooms. Each physical lounge had a computer with a big display connected to the corresponding discord room:
To enable virtual participants from different time zones to schedule meetings with physical (and other virtual peers) more easily, we numbered each of the six coffee break sessions of the conference. Thus, participants could plan to meet in coffee break #2, instead of meeting at 15:30 CEST, whatever CEST means. The conference website (based on the researchr platform) took care of presenting the session times in each participant’s local time zone.
But most importantly, to encourage physical participants to actually go to the lounges, we tried to make them particularly attractive. First, we put up the most comfy chairs in the entire conference area right in front of those screens. Second, we decorated the space with plants. And third, we provided a selection of Swiss cookies!
Before the conference, our Luce research lab held an internal cookie tasting session to select five particularly tasty kinds of local cookies. We then provisioned each of the five physical lounges with a steady supply of its kind of cookie. For example, you could taste Basler Läckerli in the “Läckerli” lounge. Physical participants were encouraged to taste the cookies but were required to chat with a virtual participant before each tasting. Each Discord channel was populated with information about the cookie available at the corresponding physical station, and virtual participants were told that they could control which cookies their physical peers would get to taste by meeting them via that cookie’s Discord room.
To complement the coffee lounges, and to make the conference space a bit more inviting, we also set up three stations with construction toys: heaps of LEGO DUPLO bricks, Tinkertoys, and wooden KAPLA blocks. We designed two hybrid activities around those heaps.
The “building instructions” activity was aimed at the programming education crowd of ICER’s computing education research community, where teaching programming — and giving unambiguous instructions — is an essential topic. Virtual participants were encouraged to post unambiguous building instructions in the corresponding Discord channel, physical participants were asked to follow those instructions to realize the corresponding physical construction. They would then post a photo of their construction to Discord, so virtual participants could see their plans realized.
The second activity, “guess concept”, was based on the same physical construction toys. The idea was for physical participants to think of a computer science concept, to build a construction representing said concept, and to take a picture of their construction and post in on Discord. Virtual participants then were going to guess what concept that construction might represent.
To get things started, we added a few constructions beforehand. I also asked my son to build a nice KAPLA construction (without constraining him to CS concepts). Although I announced in the opening session that my son gave the ok for participants to destroy the KAPLA tower, nobody dared doing so. So on the second day of the conference, under the heading “the PROBLEM of DECOMPOSITION”, I invited my son to destroy his masterpiece in front of everyone, something he seems to have enjoyed quite a bit. This little intermezzo reminded everyone of the “guess concept” activity, and lead to a flurry of constructions being posted, and guesses being made.
Lightning Talks & Poster Sessions
We had a “lightning talks & posters” track. While it was possible to submit just a poster, or just a lightning talk, many authors submitted both. A lightning talk is a one-slide 3-minute presentation. Lightning talks were bunched up into half-hour slots.
We scheduled the lightning talk slots before a coffee break, which was followed by a poster session. This way, a lightning talk might spark the interest of the audience, who could then talk to the presenter over the break, or discuss the corresponding poster during the subsequent poster session.
For each poster we had a Discord voice/video channel (within which Discord provides an embedded text channel). Authors used the voice/video channel for their poster presentation, and we populated the embedded text channel with the materials, such as the poster PDF. The text channel further allowed all participants to interact with the presenter asynchronously.
The lightning talks were either presented live of as pre-recorded videos played back during the session. Each session began with the sequence of live talks, followed by all pre-recorded talks. There was no Q&A for lightning talks, as synchronous questions could be asked in the break or during the poster session. For each lightning talk we had a Discord channel. If the talk came with a poster, we used the poster’s channel. If the author had submitted only a lightning talk, we created a Discord text channel, with the talk slide and additional material. These channels could be used for further asynchronous discussions.
Keynote
Using this setup, the conference ran smoothly. I started to relax. It was the second conference day, we had gone through all kinds of different sessions without major problems. Our fantastic keynote speaker, Manu Kapur, gave a live in-person presentation, which we streamed into the Zoom webinar. The title of the keynote: “Productive Failure”. What could possibly go wrong?
And then it happened.
The entire university network went down. First participants reported that their Wifi went down. I thought, no problem, we carefully designed our setup so that we went over wires everywhere. Unfortunately, the wired connections did not help. It soon turned out that the whole site was cut off from the Internet. We lost connection with all our remote participants. But thanks to our amazing Hybrid Experience Team, within just a minute they figured out a way to connect the necessary computers over their 4G cell phones and got back to streaming. They asked the keynote speaker to go back a few slides and to restart from there, so that remote participants would not miss anything. He did a wonderful job, and within a couple of minutes everyone was back in the flow, and nobody ever complained. This was some smooth pro-level problem-solving by this team of computing education researchers moonlighting as a live production crew.
Social Events
This was the one aspect of ICER 2022 that was not hybrid. Besides some photos and possibly stories from friends, virtual participants did not get to enjoy the walking tour, the reception at LAC, and the banquet on top of Monte Brè.
There is one more story to tell, though. We had booked a spectacular venue for our banquet: the restaurant on top of Monte San Salvatore, the mountain in the middle of the lake that every visitor of Lugano will have taken a picture of. Everything was ready, and I went on a short vacation a couple of weeks before the conference.
And then it happened.
Sitting at the beach less than three weeks before the conference, I received a message from Elisa Larghi, who not only coordinated the registrations and all the catering, but also organized pretty much everything else besides the technical setup (without her neither would we have had fresh chocolate and fruit at our roundtables each day, nor abundant water bottles for our walks through the hot city streets, and our all-so-important cookie lounges would have been entirely cookie-less). Elisa’s message:
Elisa wouldn’t be Elisa, if she didn’t follow up immediately with: “Brè and Generoso have availability. I have offers for both”. Thanks to her organizational talent, we ended up on top of Monte Brè, with a beautiful view down on the lake, the city, and over to Monte San Salvatore.
Conclusions
To provide this four day ICER 2022 experience, we spent a week setting up the physical and virtual space, a couple of months to design the details of the experience, about a year of sporadic negotiations and planning, and two years of weekly Zoom meetings with the chairs.
Now that ICER 2022 is over, let me conclude with some thoughts that might be helpful for future hybrid conference organizers.
- Hybrid > physical + virtual. A hybrid conference can be more than just a physical conference with some separated virtual “attachment”. But to make hybrid be more than the sum of its parts needs a dedicated effort. Some physical participants did not interact with virtual participants at all. They used the term “hybrid” to refer to virtual participants and probably did not consider themselves to be part of the hybrid experience. This happened despite us encouraging, repeatedly and in many different ways, the point of hybrid interactions. It may just be a learning curve for participants to realize the benefits from hybrid interactions. I guess when a physical participant ends up having to participate virtually (e.g., because they have Covid) they will start to appreciate these benefits.
- Hybrid conferences need hybrid presence from chairs. We ended up having some chairs participate virtually for part of the conference. This meant we had to “eat our own dog food”; we got to appreciate what it meant to participate virtually in this hybrid setup. And virtual participants got to have a chair who was in the same situation as themselves. While this setup happened to us by accident, I believe this should be done by design for all hybrid conferences. And if all chairs want to participate in person, they still can at least follow some of the sessions virtually from a different room.
- Hybrid conferences come at a big cost. If we had had to hire a professional AV team, our conference would have cost dozens of thousands of dollars more. We had four hybrid experience team members busy full time throughout the entire conference. Several of them, plus a fifth member with extensive expertise, were involved over the course of weeks to plan and set up the system and equipment. Also paying for a professional conference platform, instead of building our own on top of Zoom and Discord, would have increased the cost significantly.
- Hybrid conferences improve inclusiveness. While being hybrid comes at a cost, being hybrid, and thus offering the option for virtual participation, is key to being inclusive. Even if only a small number of participants join virtually, the community should bear the cost of going hybrid. That’s what being inclusive means.
- Physical participants benefit from inclusiveness. Our use of Discord, for leaving feedback and doing discussions on papers, lightning talks, and posters provided all authors with a record of the comments about their papers. And the Discord channels for various activities allowed physical participants to coordinate their dinner plans, hiking plans, and even open water swimming outside the conference sessions. Without going hybrid, we would never have added this additional communication feature. Also, using Zoom allowed automatic transcription, something virtual and physical participants can benefit from. And making Zoom recordings of the presentations available did not only benefit asynchronous viewers in different time zones, but also physical participants who wanted to review a presentation.
- Decoration matters. When setting up physical and virtual spaces, it’s important to give them a welcoming touch. Just providing a bare room with chairs, tables, and power plugs, or just providing an empty Discord text channel, is not very inspiring. Arriving in an empty Discord channel won’t get many people to start conversations. Thus, populating all the channels with meaningful, funny, or inspiring initial content is key. The same applies to the physical space. For each and every virtual or physical space, think how to best pre-populate and decorate it to encourage interactions.
- Tying physical and virtual spaces together. This is one of the most important aspects to a successful hybrid conference. It requires some creativity, quite a bit of space, equipment, and possibly other resources. I hope the above description of our coffee breaks and hybrid fun activities provides some inspiration. We got quite a lot of positive feedback about the “cookie lounges”; I would definitely repeat something similar in the future. We found that the “building instructions” activity was less used than the “guess concept” activity. The reason might be that the “building instructions” activity required a virtual participant to initiate a challenge. Many virtual participants might have missed the specific part of the opening session announcing the activity. The “guess concept” activity was more successful, probably because physical participants couldn’t get around noticing the heaps of toys and might have been tempted to follow the nearby instructions. And once the photos appeared in the corresponding Discord channel, everyone was tempted to figure out which concepts the mysterious toy constructions might correspond to.
- Separate audio in concurrent activities. During breaks, and during poster sessions, we live-streamed (via Discord) the various concurrent conversations. Each physical poster stand had a participant laptop connected to the corresponding Discord poster voice/video channel. And each coffee station had a pre-installed computer to interact between physical and virtual participants. We planned the layout of the space to separate the stations as far from each other as possible, to allow the microphones to pick up the voice of people at a station, but not the noise coming from neighboring stations. This did not always work out. Sometimes people cranked up the volume of their station to hear their remote participants, which caused distracting noise for other stations. Or people ended up chatting near a station, which made it hard to hear virtual participants. We have no good solution, except for providing enough space, and maybe some physical separators, between each station. Alternatively, one could provide headsets to physical participants to use when they join a station.
- Treat virtual participants well. On the suggestion of Kristin Stephens-Martinez we set up a Discord break room channel called “Everyone but Lugano”, as a space for virtual participants to meet also without physical participants. Kristin also suggested the creation of a “Hybrid Introductions” Discord channel where all participants could introduce themselves with their most memorable food experience. Furthermore, to encourage interaction at our hybrid coffee break lounges, we deliberately looked for senior participants in the breaks and nudged them to join a hybrid break conversation in a cookie lounge.
- Budgeting a hybrid conference is hard. For me, the most stressful part of the conference probably was that I did not know at all how many virtual vs. physical participants to expect. There were no prior instances to base our estimates on. We had very different registration counts for purely virtual and purely physical prior conference instances. The severity of the pandemic was a constant (a variable?) bother. And the considerable cost of traveling to and staying in a not-exactly-cheap Swiss tourist destination competed with the attractiveness of meeting up in person in the spectacular city of Lugano after a long pandemic break. Throughout most of our planning we estimated to get around 60 physical participants. At some point we even joked it might just be us four chairs. We ended up with 110 participants in Lugano and 45 virtual participants. Booking banquet and reception venues with that financial uncertainty is a risk, and the pandemic-related requirement of some venues to pay the entire event at the time of booking didn’t simplify organization.
Let me close by thanking the whole ICER organizing team, including first and foremost the Hybrid Experience Team (Roberto Minelli, Andrea Gallidabino, Jana Seep, Luca Chiodini, Carolin Wortmann), the person without whom we would have had no space, no food or drinks, no Swiss chocolate or cookies, no goodies, and no social events (Elisa Larghi), the many track and session chairs who were instrumental in realizing the whole event, and my outstanding co-chairs Jan Vahrenhold (ICER 2022 Program Chair), Kathi Fisler (ICER 2023 Program Chair), and Diana Franklin (ICER 2023 Site Chair). Jan deserves extra kudos for doing much more than “just” program-chairing, but helping out with site-chair business wherever he could as well. Thank you all!
