The missing pieces in virtual-ACL

So, ACL 2020, the first all-virtual major NLP event, is now over. It was intense, it was informative, we learned many things (see a nice summary by Vered here), we are exhausted like after previous ACL conferences, if not more. And yet, it didn’t feel like any other conference I attended before.

I really liked having the videos available with the conference, I liked being able to watch talks at x1.8 speed, I liked the possibility of technical discussions around papers in the chat system. It was nice to not eat airplane food and to not be jet-lagged. It was great to be able to send students without papers to “attend ACL” because it was not ~3,000$ per student. But did my students really “attend ACL” in the sense that students last year attended ACL? I certainly didn’t: it did not feel to me like a conference. Crucial aspects were missing, and I would gladly pay 3,000$ instead of 100$ (well, out of grant money…) to get them back. I will try to articulate what they are. I believe many people share these (rather obvious?) views and experiences, but I have not seen them expressed anywhere in writing in this community, and I think it is important to have them expressed in writing. In this sense, you can think of this piece as a late-coming (meta)theme paper”, or maybe, “meta-theme extended-abstract” (and we don’t even need thought experiments — most of us remember what an in-person conference is like).

I am not writing this to complain about the event, I think it was very well organized for a first one, we are still learning the ropes, and it will take a while to perfect the format. It run smoothly, it was professional, we learned things! I also don’t have solutions to the rather-obvious issues I am raising (again, think “theme track award winning piece!”), these issues are genuinely hard problems. But, I hope that documenting the issues will help us try and focus the community on what the missing pieces are, and propose ways for solving them. And, maybe even by just putting these more saliently on participants minds when attending the next event, the participants themselves would get a small push and encouragement to behave differently, and shape the event accordingly, grassroots style. In this sense, you can think of this piece as an IRB-less priming experiment.

Being a blog post and not a real theme paper, though, it is written in a more casual, less edited, more colloquial, style. It may also not be the most well-organized piece I have ever written. I hope you forgive me.

Like I don’t discuss the good things, I also don’t discuss small annoyances and various UI improvements that I think can be made to make things more comfortable. These are easy, and I will address them in different channels. I want this piece to focus on the main missing aspect imo, which is also the single biggest mystery I think we need to nail down going forward.

So, after this introduction, let’s get to the meat:
Yoav’s bitching about what was missing in ACL 2020,
aka:
We got the semantics right but missed on the discourse and pragmatics: let’s change the morphology and syntax to make it better[1].

While the conference was very informative, I think it was clearly lacking in free-form, spontaneous and random semi-social-semi-professional interactions. I have some ideas of how this could be improved for students and early career people, some are hinted here, others I will be happy to discuss in a separate forum. Unfortunately, I have no idea how to improve it for ourselves (= more established researchers, faculty, etc, from early-mid career onwards). I don’t know how a platform that facilitates “random (virtual) encounters” for overly busy people could look like, and I definitely don’t know how such a platform could look like without it being extremely elitist (shoo, young people! it’s old-timers club now!).

The platform is very centered about obtaining knowledge from a source about a topic. It does not facilitate conversation, and indeed, conversation was sorely missing. The Q&A sessions via rocket-chat and question-votings were terrific in some aspects: the interface surfaced good questions that interest many, and removed annoying nitpicky, niche or more-comment-than-question-on-obscure-things questions (well, people could still ask them, we just didn’t waste precious QA time in discussing them until someone was brave enough to say “let’s take it offline”, win-win for all). So this part was great. But, it was also a very bad platform for facilitating more dialog-y discussions, which, while rare at QA sessions, when they happen are often much, much more interesting [2]. Similarly for the tutorial QA session I attended (Interpretability): I think the team of tutors (Ellie, Sebastian, Yonatan) set it up and handled it incredibly well, but yet in several occasions I felt that the answer was not fully covered, and that one or two more rounds of interaction with the askers could have made things so much better. I am not blaming the team in any way for not doing so: it’s the platform’s fault.

There aren’t really many options to casually tell people about your work, nor to learn about theirs (in the broader sense). Sure, when they come to your session, it’s all about you. But few come. And then when you come to other people’s QA session, it’s all about their specific work. There are no “hi, so what are you working on these days?” and no “oh, we’ve done something similar…”, there are also no “so what did you think about … ?”. (These things can occur, and do, sometimes, in the less crowded [read, empty] sessions, but they are really not the norm. And for a good reason: this is not the space for them. But there are also no other spaces to facilitate them).

For each of your papers, you have two very specific hours to tell others about your work in that specific paper, to the specific people who actively came to listen. In these two hours, you usually don’t discuss your other papers or late-breaking projects, you don’t discuss the other person’s work, you don’t just discuss common interests — after all, you only have one hour now to discuss the current work, and other people are coming, let’s stay on topic please! There are many people I would be very interested to hear about their work and current interests, but not in a talk or video (or poster) format. Just listening to them describe (pitch?) it, in a conversation style, where we may also exchange ideas, and maybe drift to related (or less related) topics. It may end up being a short 5-minutes interaction where I learn something new, or realize that ‘ha, X is still working on that dream pipe of theirs like they did last time’. Or it could evolve into an hour-length discussion of cool research ideas, start a new collaboration, etc. But nothing in the current format provides for this.

For example, I really enjoyed Ellie and Tom’s TACL paper, but I don’t have any specific thing to say about it. I also don’t have any specific question to ask either Tom or Ellie, and nothing specific I want to tell them. But bumping into them and exchanging a few words would have been great! And could have developed into some very interesting conversation (or not). Same with many others, but there are just no opportunities for it. (Hi others, I enjoyed many of your works as well, and would have loved to talk to many of you as well!!)

All meetings are very focused, mandating what you should talk about, and to a large extent mandating what is the format [3]. It is either focused on a particular work (in QA sessions), or focused on very broad and moderated discussion of a topic (BoF session led by a senior person), or focused on listening to advice and asking career questions (mentoring sessions). Nothing free form. Nothing organic. No “let’s just hang here for a while and meet people who are passing by”.

In her book very recommended “Because Internet”, the socio-linguist Gretchen McCulloch talks about the importance of hallways. Here’s the relevant page from the book (the highlighted piece on the right is the central ones, the other highlights I made before, and the entire page is recommended for context):

She then goes on to discuss Twitter and other social-media places as hallways. But our virtual conference certainly don’t feel like ones. How do we form effective virtual hallways in our conferences?

Naomi Saphra expressed a similar view and attempted to provide a solution in a series of tweets calling for an “ACL-Town”:

[For those of you who don’t know how these things work (like me a few days ago): it is a 2d world, where you have an avatar and can roam around. When your avatar is sufficiently close to another one (or to a group), you see and hear them in a video/audio interface. So like a zoom-meeting, but very ad-hoc and easy to set up. And also, there are no instructions that mandate you to talk about a paper. One example platform is gather.town (google it). I heard good (and bad) things about it at ICLR and PLDI.]

This sort-of worked, for the ~10–20 people that participated. I’ve heard they had fun listening to the planeries together there, among other things. But it could have been huge, if it got more endorsement from the “establishment”, and was somewhat more integrated into, or at least “formally” be part of, the main event. Otherwise it is really hard to get adoption (“these damn kids with their slang and emojis! this is not a real language! we are here to talk like adults”).

[Anecdote, aka Yoav being mildly creepy: at some point, I logged into this interface, it was empty, so I turned off my video and mic, wrote “afk” next to my avatar’s name, switched to another browser window and… forgot about it. And then two hours later, while being in some zoom session, I started hearing people talking in my headphones. Turns out two students logged in to the platform, happened to stop near my avatar, and started talking. My first instinct was to close this noise, but it took me a while to find that tab, so I followed my second instinct which was to leave the zoom session and eavesdrop on their conversation for a while. It was beautiful, just two people introducing themselves to each other, talking about their (very different from each other’s) research interests, discovering new topics, sharing anecdotes of research and life, forming the beginning of a potential friendship. This lasted for a few minutes and then I left, but it was a lovely reminder of what a conference should feel like. Sorry for eavesdropping, I didn’t want to interfere by notifying you know of my presence.]

My discussion so far was mainly about the professional aspects: how do we enable a more relaxed, free-form, non-centralized discussion on a vaster range of scientific topics with our peers. But there is also the purely social aspect that was missing.

Besides not being conductive of sharing and developing ideas, the pre-mandated formats are also non-conductive for forming friendships and personal connections, which are arguably even more important for the scientific advance in the long run (the reason I can meet someone in an in-person conference and talk to them free-form about research, is that we have established some history together over past conferences). I think this following tweet summarizes this point beautifully (and sadly):

How do we crack the social aspect going forward? how do we facilitate “random” encounters and conversations? how do we motivate people to hang around and talk about research, without mandating a topic for each time slice? how do we facilitate young researchers forming connections with like-minded people? how do we simulate “going out for lunch with an eclectic group of people, who all happen to like NLP”? How do we bump into people, How do we form hallways? Let’s think about it going forward for the next virtual event. These are very hard problems to crack. But I’d argue that they are more important than another MRC-benchmark or large pre-trained model, so maybe let’s focus some efforts on that for a while? (or alternatively we could just solve COVID. That’d also be great.)

Maybe your experience was totally different and it was just me and my buddies who didn’t understand how this thing works. Or maybe you felt the same and have some great ideas for how to make things better. Or maybe you just want a space to hang. Let’s discuss in the comments!

— — — — — —

[1] I usually don’t like to explain metaphors, but will do it this time: the semantics is the “official” reason for a conference: scientists presenting their late-breaking work to other scientists. The discourse and pragmatics are the less official but equally (or more!) important aspects: free form exchange of ideas, forming scientific relations and collaborations, forming personal connections, forming a scientific community and sub-communities, etc. The morphology and syntax are the form, the infrastructure that underlies the interaction and shapes them.

[2] And they are certainly not the appropriate format for a business meeting, but that’s a separate topic.

[3] I did attempt to break the format, turning the QA sessions I visited into Poster sessions, which actually worked quite well, but also came as a surprise to some people, catching them off-guard, and maybe appearing impolite. And I could do it because I am somewhat well established, I don’t think it would have worked for young PhD students to behave similarly. Formats do matter, and set expectations. We should think about how to set them right.