Lauren Kennedy, Monash University; Guillaume Basse, Stanford University; Andrew Gelman, Columbia University; Guido Imbens, Stanford University; Yajuan Si, University of Michigan; Dominik Rothenhausler, Stanford University; and Jann Spiess, Stanford University

The spread of COVID-19 has slowed economies and stopped travel throughout the world. For academics, classes and research collaborations have moved online. But the pandemic has disrupted two other essential aspects of academic life: conferences and seminars, which have for the most part been canceled. Existing collaborative research (except when involving laboratory or field work) can easily move online but cannot replace conferences and seminars, which are key opportunities for developing new collaborations, especially for early-career researchers who are also losing opportunities to promote research contributions, gain valuable feedback, and interact with more senior colleagues outside their advisers’ networks.

An obvious solution is to move conferences and seminars online, much like regular courses, but this has proven challenging for a number of reasons. As a community, we have had relatively little experience organizing online conferences and seminars compared to online courses, which have been around for at least a decade on various platforms.

As an attempt to accelerate our understanding of this new environment, we share our experiences from running a two-day online conference on multilevel regression and poststratification and two ongoing virtual seminar series on causal inference and econometrics, each with hundreds of participants. The move to online meetings presents challenges but also many opportunities that are worth tapping into.

The online conference (conducted by LK, YS, and AG) and the online seminar series (conducted by GI, GB, DR, and JS) were both conducted through Zoom. Some of the advice and reflections here are specific to the technological possibilities of Zoom, but others we believe stand more broadly.

The online conference took a planned workshop with 20 speakers and 50 participants, originally scheduled to be held in a single room on campus, and opened it up to a conference with more than 150 participants. The seminar series were set up as virtual meetings from the outset and met for one hour or 90 minutes, respectively, with new speakers and discussants each week, regularly attracting several hundred attendees.

Technology

The conference was run using a Zoom professional account, which allows for up to 300 attendees and a 24-hour meeting time. The seminar series were run in webinar format, which has a number of additional features, including moderation, live broadcasting, and a high number of view-only attendees. The conference also used Google forms extensively for various organizing tasks.

It is hard to participate in an online conference if you do not have a good internet connection. Even leaving this aside, there were other accessibility challenges. Attendees were allowed to use videos at the start of the online conference, but we eventually had to turn off all video capabilities to reduce the internet demand. Over the two days, streaming the conference took about 5 Gb. The seminars had one video stream, which switched between the moderator, the speaker, and the discussant, along with a back-up plan for the speaker to share slides before the talk and call into the meeting if necessary.

In-person conferences can be a challenge for individuals with visual impairment. Immediate slide sharing and being able to view the content on one’s own screen potentially made the written content easier to view. However, poor color schemes were just as common. In addition, because participants posted questions in the chat, it seemed more difficult to remember to repeat questions into the microphone. One potential solution would be to use the hand raise function or ask participants to post questions in the chat. However, even when encouraged, these were not readily undertaken.

Schedule

The online conference was hastily moved from an in-person schedule to an online schedule, and so it retained the overall outline of a typical conference: session, break, session, lunch, session, break, session.

The online seminars were always intended to be virtual, so the typical structure was adapted to a virtual format, with the seminars interspersed with opportunities for questions, a discussant at the end, and one of the coauthors available in text chat to answer questions immediately.

Accessibility and Cost

All the authors hoped the move to an online conference would create greater diversity and accessibility. In some ways, this worked out. The seminar series would never have been able to schedule such a geographically diverse group of speakers had they been run in person at a single location. The conference, which was moved online after the speakers were invited, saw a surge in the number of participants and geographic diversity, and the registration fee was moved from $50 to $0. We tried to reach broad audiences using an open webpage where anyone could register and advertising on blogs and through email lists.

However, the virtue of being virtual did not solve all accessibility challenges. One immediate challenge was the time zone. The conference was held in US Eastern time (EST), which covers late afternoon to late evening in Europe, but the main organizer (LK) was in Australia, where it was an overnight conference. Likewise, the seminar series were hosted at an accessible time for Europe and North America. Unfortunately, the usual argument for hosting the majority of international conferences in North America and Europe still holds for online conferences—it makes sense to host an event at a time that is most convenient for the majority of attendees.

An advantage of online meetings is their cost is close to zero—no need to pay for airline flights, hotels, and conference rooms—but societies using conferences as a source of income will need to consider the cost structure. One question going forward is whether more conferences and seminars should go online to save the cost and environmental impact of in-person meetings.

Online seminars make it easier to overcome logistical barriers to diversity for both presenters and members of the audience. Yet, in a world where a few big online seminars and conferences replace many small opportunities to present work, they may not resolve—and even increase—the exclusivity of speaking spots. We therefore think inclusiveness along many dimensions, as well as a healthy number of diverse seminars and conferences organized by equally diverse groups of scholars, should not be taken for granted.

Presenting and Moderation

Talking to silence is hard. Talking to silence without any feedback is harder. This can be mitigated by having one or two people (enthusiastic nodders and smilers) to give visual feedback, but this can increase internet demands.

Timing has been difficult. Whoever thought of the flashcards at the front of the audience was a genius; none of the authors has found a good alternative. Options include verbally interrupting (disrupts flow), posting in chat (often missed or seen at the wrong time), or posting text overlay on the videos (likewise). The best option may be to give the time control job to each speaker, who would be told in advance that it is their responsibility to pause when appropriate, answer questions from chat when they see fit, and end on time (or else be stopped abruptly). When speaking remotely, it is not so difficult to keep an eye on the clock and to occasionally check the chat box, if you know this is your job.

But the move to online seminars also offers new opportunities to improve the flow of presentations and the quality of questions. It can be difficult for the presenter to keep track of a chat window, so the online seminars have been relying on a moderator to curate questions and call on the most relevant ones during breaks chosen by the speaker. Although born of a necessity to adapt to an online world, the resulting flow of presentation only interrupted at intended points has produced results superior to in-person seminar presentations in front of small audiences (which are interrupted all too frequently) or huge audiences (where nobody asks questions).

Networking

Face-to-face meetings facilitate social interaction and soft information transfer. Meeting academic idols, speaking face-to-face with colleagues, and having discussions over coffee are valuable to academic endeavors. The online conference tried several methods to encourage and allow for this networking with limited success.

We first tried breakout rooms at the end of each session during the scheduled 15-minute break periods of the conference, with participants randomly partitioned into rooms of three or four people each using the breakout function in Zoom. The technological functionality worked well, but the practice was not successful. The conversations were awkward and many people did not join the rooms (leading to the host reshuffling people). By the third break session, few people were participating.

The conference also used a birds-of-a-feather session at the end of the first day. For this, ideas for session themes were solicited, participants elected their session through a Google form, and then they were allocated to breakout rooms to discuss. This seemed to work much better than the breakout rooms during the break periods. We conjecture this might have been because there was a purpose and a natural conversation leader/organizer.

Following this insight, we used breakout rooms at the end of each session, with each hosted by a speaker from that session, the next day. Participants who wished to join were either randomly allocated or could elect a specific speaker. This consistently led to more participant engagement throughout the remainder of the networking sessions, and we encourage this format for every session—not just those preceding breaks.

One aspect of in-person meetings that has been difficult for us to duplicate is the flexibility with which a meeting participant can come up to individuals or small groups, along with the corresponding ease of breaking off interactions and moving to the next one. If there were some additional feature that allowed participants to see who was in the breakout rooms and move freely between them, this might get closer to this experience.

For one of our online seminars, we implemented additional networking sessions afterward, in which we particularly encouraged graduate students to participate. Whereas the main seminar was set up as a webinar in which only speakers were visible, these side sessions allowed direct exchange between all participants. While hardly feasible for larger groups, this worked reasonably well with the 20–30 people typically signed up for post-talk discussions. It could be worth putting in additional effort to facilitate building connections (e.g., sign-up sheets at scheduled 15-minute group meetings after the seminar), following the successful model of the birds-of-a-feather meetings at the end of the day in the conference.

Openness

There are benefits and costs to making an academic event open. Both the conference and seminars were open to all, but the setups were different. The conference used a standard registration form, from which an email list was made and Zoom link and password were emailed to all registered participants 24 hours ahead of time. This enabled us to allow full participation from all participants during the conference with limited moderation. The seminars, in contrast, were fully open, with public links. There were some challenges with Zoom bombing in earlier seminar sessions, but the meeting was not interrupted and attendees did not see messages sent by the Zoom bombers due to moderation and limits on participation.

Another decision is whether to record sessions. An advantage of online meetings is how easy it is to record. With Zoom, this can be done with the press of a button, and indeed the seminar series were recorded and the sessions posted on YouTube. Recording sessions enhances the accessibility of the session for participants who are unable to attend and produces a product that can be useful for years to come. However, there is a concern that recording could inhibit active participation by less-established researchers, both in presenting and questioning, and so we chose to only post slides for the conference. In addition, there are privacy concerns where participants are working from home.

Overall

COVID has interrupted academic work, including moving seminars and conferences online. We think these online conferences and seminars—often started as a quick replacement to the holes COVID has created in academic life—are here to stay beyond the current crisis.

The growing number of online seminars and conferences do some things better than the in-person events they replace: They are more inclusive, providing wide audiences with high-quality content and reducing travel barriers for speakers and discussants; they allow for innovation in how questions are asked and answered; and they reduce the economic, ecological, and personal strain from traveling. However, they lack when it comes to important networking functions.

We think online conferences and seminars can unlock additional potential once we put more emphasis on these “soft” aspects, which go beyond presentations and discussions to consider the larger role of seminars and conferences in academic life.

Conferences and seminars, large or small, tend to follow the same structure and format when conducted face-to-face. Moving to an online version is not as simple as it sounds and has a number of challenges academia will need to understand in the coming months. There will not be a-one-size-fits-all solution, so experimentation with varying formats and side programs for networking should continue.

For our large seminar series, we are experimenting with different formats, such as varying between one paper in a seminar, three shorter papers on similar topics, or one methodological paper and one applied paper. We also plan to experiment with panels on particular topics, with questions solicited in advance. Having multiple presenters makes it easier to mix senior and junior people without losing the audience. We should be much more proactive in trying out formats that would be difficult to do in live seminars. Not all of these new ideas will work, but we should take advantage of the opportunity to learn by experimentation.

For a small conference that was intended to have a small burden for participation, a more closed conference enhances a community feel and increases participation from early-career researchers. However, for greater dissemination and impact (such as a larger annual meeting or seminar series), a more open format makes sense.

Face-to-face conferences are implicitly structured by their physical constraints. A meeting can have 20 or 100 or 1,000 or 10,000 participants. A workshop that meets around a conference table will feel different than a session held in a traditional classroom, which in turn has a norm of participation that is different from a lecture in an auditorium. The very act of moving from room to room, or choosing among parallel sessions, can stimulate unexpected interactions. Snack and coffee sessions lower the barrier to initiating conversations with strangers.

In contrast, the flatness of the computer screen and the intrusion of everyday life in our homes can make all online interactions feel the same. Therefore, it may be beneficial for organizers to come up with ways to provide a seminar- or conference-like feeling. We can’t fly people to a distant city or trap them in a hotel for two days, but we can prime the pump by demonstrating informal conversation of the sort present at fun and productive academic meetings and creating structures that enable participants to do so as well.

At the same time, we should make use of the advantages of online meetings, most notably in technology and accessibility. For example, technical talks can share code in a way that would be more challenging in a face-to-face conference or seminar. And, perhaps most importantly, online seminars are accessible to people with disabilities or without the financial resources to travel to a distant location.

We hope in the not-so-distant future to return to close, in-person interactions. When this happens, let us make an effort to use online tools to be more inclusive.

Finally, we think it is important to not just experiment, but to share what we have learned from this experimentation. As we all innovate around creating engaging and diverse academic online experiences and learn to overcome technological challenges in their realization, we can stand on the shoulders of a community that tries the same—across continents and disciplines.