Reflections on Successful Meetings with Undergraduate Researchers
Or, how to have a good research meeting with your advisor
This summer I’ve had the good fortune to work with a large group of 13 new undergraduate researchers in my lab. They are talented Michigan students who want to learn how to do world-class research but are just getting started along the path. One aspect of learning to do good research is learning how to do research with others — and in particular, when starting out, how to do research with an advisor. I wanted to touch on one small part of this process: meetings with an advisor. Meetings are a critical part of the knowledge transfer process but not all of them are useful, let alone productive. What makes a good meeting and how should a new undergraduate researcher think about meetings?¹ Further, what kind of behavior should an advisor encourage to help their students grow as researchers?
My goal with this post is to help students recognize how to maximize the effectiveness of their meetings and, overall, my goal as an advisor is always to help students turn into the best researchers they can be. To talk about meetings, I’ll contrast two hypothetical examples of students: One, I’ll call the Pine student (Michigan’s state tree²), and the other the Sapling student. Both students start from the ground level and are often learning their computational craft at the same time as learning how to do good research. The Pine students have made the most of their opportunities and grown in their abilities and flourish. In contrast, the Sapling students certainly put in effort but haven’t seen much growth or much success — yet. To illustrate how these hypothetical students differ in their behavior, I’ll describe several general scenarios for meetings that can help lead one student to grow substantially as a researcher into a towering pine, whereas the other remains just a sapling.
For context, my lab is structured so that undergraduate students typically work in groups. We use a few different tools for communicating about projects. Project meetings happen on an at-least weekly basis and provide low-level guidance on the research, including goal setting, staring at figures, and discussing ideas. Project meetings are scheduled by the students themselves in one or more 30 minute blocks using an appointment calendar; there’s generally enough time for two 30 minute meetings for each group spaced throughout the week. Weekly all-hands lab meetings provide more dissemination and allow students to share details and highlights from that week’s work. Between meetings, we use Slack to communicate for quick updates, questions, and showing off upcoming results to discuss at the next meeting. For some projects, we have also adopted Asana for managing project to-dos and keeping organized on what’s coming down the project pipeline. I also keep my door open for the occasional drop-by question or chat.
In different scenarios below, I outline hypothetical situations of how the Pine and the Sapling might differ in their meeting experiences. Before reading these, I want to be clear that I believe any Sapling student can become a Pine student. These are not permanent distinctions and in fact many of us (including me!) had these Sapling habits at one point. If you, as a student, recognize some part of yourself in a scenario, despair not! Recognizing a bad habit is an important step towards progress and improving that aspect. No student begins their research experience as an amazing PhD-level researcher; we all start somewhere close to ground level and work our way up from there. As an additional aside to the undergraduates in my lab (now and in the future), I am proud to have you as members and want to help you succeed; these scenarios are intended as guides to help you avoid potential pitfalls and shoot up into tall Pines as quickly as possible.
Scenario: a student has just arrived to a meeting they scheduled to discuss their project.
The Sapling student will often arrive and ask what should we talk about or have nothing in particular to discuss. Sometimes this can be due to shyness (and not to being a Sapling student), so I may ask students to start by bringing me up to speed. In this case, a Sapling student may not have thought of how to summarize their progress and have trouble responding without more prompting. The meeting proceeds and we discuss as much as we can in the allotted time, which often runs out before we can wrap up.
The Pine student arrives with an agenda of all the points they want to discuss. Often this list is organized around different topics and prioritized so that if we run out of time, the most pressing issues are addressed first. We go through each with a full understanding of how much we have to talk about for the rest of the meeting.
Reflection: Having students create the meeting agenda provides an important sense of agency that they are driving the project and also leads to more efficient meetings. The Pine student’s effort in drafting an agenda is doubly worthwhile because it allows them to think big-picture about what they did and ask themselves which items need to be discussed first. When the agenda is created between students within the group, their discussions often help create a mental model of the project’s priorities that lead to more effective collaboration.
The Sapling student’s meeting is more reactive to the advisor’s comments and generally leads to less productive meetings where the advisor drives everything or must use the socratic method to discover what has been done. The Sapling student’s meeting could turn into the Pine student’s meeting but it depends on the advisor knowing the agenda already and being able to quickly prioritize things. In projects where I’m very actively involved, I can typically do this, but only from the prospective of what I think has been accomplished. The new researcher is much closer to the day-to-day research and may have noticed or experience things that need to be on the agenda (unusual results, important questions, or new papers) that I won’t necessarily know about yet.
One side-effect of the Sapling meeting style is meeting spill-over. I eventually have to cut the meeting off to start the meeting after, which is often not at an ideal point for deciding on next steps and leaves the student potentially hurt that I couldn’t spend more time. I don’t like cutting off these meetings but I also don’t want to shortchange the next students who are coming in.
For some projects, I’ve borrowed the strategy from my colleague Libby Hemphill and use Asana to drive the meetings. The project is laid out in Asana using an ever-growing number of current tasks and backlog of tasks that will need to be done. Students are assigned (or self-assign) tasks for that week and each meeting’s agenda is determined by what’s currently assigned in Asana. I find this strategy has worked well for some teams, though it does come with the overhead of maintaining a task list and actually getting everyone involved to use Asana (Slack integration with Asana has helped this though).
Finally, sometimes it can be nice to have the occasional informal meeting where there is no set agenda. These meetings might be more brainstorming or have other kinds of discussions not focused on research. However, if I’m collaborating with a new student, having a focused meeting is a much better use of both of our time to help give them the feedback and direction they need to grow into a Pine student.
Suggestions on how to become a Pine:
- Come to a meeting with a prioritized agenda of what you want to discuss. This agenda can even be sent beforehand to everyone involved.
- If you’re in a team, collaboratively come up with the agenda based on what each person is doing.
- Be aware of meeting time and ensure everything you want to discuss gets brought up before the allotted time runs out.
Scenario: the student has been working throughout the week but doesn’t feel like they have enough to discuss and hasn’t yet scheduled a weekly meeting with me.
There are many reasons not to have scheduled a meeting, but sometimes I have seen Sapling students put off meetings out of a sense of shame for not doing enough or being successful enough. Occasionally, a Sapling student may also feel that weekly meetings are not a priority and that it is acceptable to slip to a later week.
The Pine student messages me (on Slack or email) saying that they have been working but currently don’t think they have enough to discuss and would like to know if a meeting will be useful. Further, the Pine students adds what has been blocking them from making progress (e.g., server has crashed; GPUs have been in use) and what steps they have proactively taken to resolve them. Based on the situation, we may still meet to discuss these issues or postpone with the knowledge that things are being done.
Reflection: Communication is the key here, along with an important acknowledgment that there is no shame in having your work not turn out like you thought — the process of research is never certain! Even when students don’t have results, we generally meet to discuss the process of research and how things might have been done differently (without blame) in light of what we learned from their effort. I sometimes find that a Sapling student has been working very hard on a problem that is difficult but either not central to our project’s goals or one that can easily be solved with a bit more experience to get them unblocked. Having a good communication channel can help such Sapling students reprioritize their effort and make progress again.
Suggestions on how to become a Pine:
- Be open and honest with your advisor about your progress and raise attention for things blocking your progress.
- Meet weekly. If you find yourself without much to talk about, discuss why that is the case and aim to resolve it.
Scenario: The meeting has just begun and the student is presenting the first results of the project.
The Sapling student often starts with deep technical details with minimal context, e.g., “So I was trying to run the sklearn code on the data but when I set this parameter to 3, the performance improved.” The next several minutes of the meeting are spent unpacking such statements to determine why they were doing something and what effect it has on the project.
The Pine student arrives and quickly summarizes the goals of the project as a whole and what research question they were trying to pursue within those goals, e.g., “we are trying to understand people’s reactions to memes on Twitter and this week I built a sentiment analysis classifier with sklearn to rate reactions.” This framing immediately sets expectations and helps me ensure that the student’s model of the project’s objectives are aligned with the project’s actual objectives.
Reflection: Advisors are busy and often one research meetings is immediately followed by another on a completely different topic. As a result, any effort the student can put in to helping quickly familiarize the advisor with the project is incredibly useful. The Sapling student isn’t wrong for going into details; this kind of discussion can be incredibly useful. However, the issue is immediately starting out in the weeds without context. Starting by re-iterating the problem allows the student and advisor to quickly get on the same page and make it through more of the meeting by not having to ask clarifying questions.
Suggestions on how to become a Pine:
- Start any discussion of an agenda item with what problem you’re trying to solve. Describe the problem from a high level that everyone understands and then become more specific until you reach the details of what you’re doing.
- If you need to spend all of the meeting on something very detailed, send an email the day before with the context and agenda so that the advisor is prepared to jump right into the details.
Scenario: during the meeting, the student and advisor begin to look at the intermediate data produced over the previous few days.
The Sapling student will pull up data they have just generated and possibly never looked at before. Often, there will be surprises in the data, e.g., blank entries, artifacts from encoding, or unexpected tab/newline insertions. The Sapling student will struggle to explain how these surprises came to be and pinpoint where they might come from during preprocessing. Often, results have been produced from this intermediate data (e.g., a classifier is trained on this data) without having looked at it, which raises additional questions about the validity of those results.
The Pine student will have generated the data before the meeting with enough time to go over the data themselves ask whether anything is useful. If they spotted something unusual, the Pine student corrects the error (or finds an explanation) and regenerates the results. The Pine student can typically point to which programs, scripts, and methods are responsible for producing the data, and can fully replicate the setup they used to produce the data if we want to try something on hand. In the best cases, the Pine student has made things like Jupyter notebooks or bash scripts (pick your favorite language) that are easily sharable within the lab and outside to others.
Reflection: Looking seriously at all the data in your experiments is a critical step for a young researcher. Even as an expert, having a healthy skepticism about your own data and methods is important to building intuition about what phenomena you’re looking at and what your software is actually doing. One of my old advisors, Roberto Navigli, had a sixth sense for finding surprises in data and would call out random line numbers for us to look at, only to find the one line in thousands that had something weird. This skill was terrifying at first as a student (no one wants to show bad data) and a great motivator for me and the other lab members to always double- and triple-check everything. You can only trust your results when you trust the data that led to them.
Suggestions on how to become a Pine:
- Always, always look at your data before the meeting and allow enough time that if you spot something wrong, you can fix it
- Look random parts of your data, not just the first few lines. Be sure there are no surprises
- Meeting time is precious; have your data ready to go in a format easily accessible to all meeting participants. Google docs/sheets work great for this. For large files on servers that everyone has access to, send out the link to the file to everyone before the meeting.
Scenario: the student has been hard at work on a project using a big dataset and wants to present their latest results at the meeting.
The Sapling student will arrive and say they were hoping to present but ended up not having results, which often boils down to a few reasons
- there was a bug in their program and it crashed over night
- the data is too big and the program didn’t scale
- they got the results and generated the plot but realized something was wrong in the plot and didn’t save the results to quickly remake the plot
The Pine student arrives with their results, though not always with the full extent of what they had planned. The Pine student has done the back of the envelope calculation to determine how much time they have before the meeting and generated results for as much of the data as they could, using appropriate subsampling.
Reflection: Many of the projects in my lab involve large datasets, sometimes tens of terabytes of data. Even some smaller datasets have millions of data points that are impractical to study if analyzed naively. This scale can pose a substantial challenge to the budding scientist, who up to this point has not worked at scale in their coursework? The reality is that we generally don’t need to use all of that data to get our initial results. If we have a billion data points, using a random sample of one million will probably give us a fairly close result to the full data, just with larger error bars. Getting small results quickly is important for building intuition on your research questions. Learning how to generate these results quickly — or discover a bug preventing you from scaling — is equally important in developing the craft of software development.
The critical skill here is planning to get results. Running a program is an important step, but the Pine student’s effort in estimating the time is an important earlier step. How long will the process take? Can we use less data now? If we need all the data, can we parallelize this somehow? Good communication with the advisor and peers in the lab can help significantly with planning as well — and possibly on how to strategize to speed things up.
Suggestions on how to become a Pine:
- Run your analysis on the smallest amount of data you can at first (maybe the first 1000 examples) to debug everything and to get a sense of how long it might take. Then try to run as much data as you can before the meeting.
- Never launch a program to produce results on all the data without first checking that it works on a small batch of data first.
Scenario: the student has produced a new figure for us to look at about their project
The Sapling student’s figures often have a few common issues:
- no labels on axes or other interpretable visualizations, or a tiny font
- the data is oddly grouped or organized so that similar things are not visually distinguishable
- the data is heavily skewed but the figure isn’t drawn at log scale for the appropriate axes, making it difficult to read
- the figure was generated in a way that makes it difficult for us to quickly fix these issues
The Pine student’s figures are appropriately labeled and scaled. They have the code for generating the figure handy (e.g., in a Jupyter notebook) so that if we need to change anything, we can do it together. Pine students often have generated more figures than we have time to talk about by doing lots of little analyses, which lets them prioritize the ones we spend time on in our meetings.
Reflection: Figure making is an important art in the process of communicating science. The Sapling student often thinks of making the figure as the goal but the Pine student will realize that the true goal is having someone other than them understand the figure as they themselves do. Learning how to generate high quality, interpretable figures can take time, so being able to quickly iterate is useful and saves time later when we need to generate the figures for the paper.
Suggestions on how to become a Pine:
- Label your axes
- Make sure the code for generating your figure is readily available and easily runnable
- Get the easy figure first (e.g., use Seaborn, Matplotlib, or Gnuplot with their defaults) and then go about the business of customizing. Example figure galleries can help show you possible ways to adjust — with code too.
Scenario: a long technical discussion is wrapping up by discussing the next steps for the project
The Sapling student will ask questions about the discussion but often not take notes; or, if the discussion has included drawing on the whiteboard, no record is taken of any diagrams. When asked if they understand some specific part of what we discussed, the Sapling student says yes (often to avoid admitting they didn’t understand).
The Pine student takes notes in multiple formats. If anything is unclear, the Pine student will ask clarification questions. Often, pictures are taken of the white board discussion and posted to the appropriate Slack channel for the project for everyone to see. Following the advice from my colleague Walter Lasecki, I’ve been encouraging students to use their phone to record the whole audio of the meeting, which is then posted to Slack as well.
Reflection: Notes are critical. I’ve been impressed with the recorded meetings as well. When taking hand-written notes, it becomes a trade-off between writing down what’s being said and being an active participant. Recording the whole audio lets students go back and listen to any part they might have missed or need additional context for understanding.
Moreover, in mixed-gender groups, recording audio eliminates the undesirable trend of having the female researcher end up as the notetaker and not participate as much. The audio recording ensures everyone can be an equal participant without having to worry about writing down what’s being said.
Some students have also taken to following up after each meeting with a meeting summary. These summaries are very useful, as they help drive the discussion at the next meeting and provide a moment of reflection for thinking about what is to be done next. I often have students in my lab send me brief weekly summaries of their work during the week and their plans for the next week. While the former part is useful for finding highlights for recommendation letters, the latter is actually the critical part in nudging students to think bigger picture on the project and what they want to accomplish. Pausing to reflect on what was done during the week is also useful for thinking about the process of research and what could be done different.
Beyond taking notes, being honest about what you do and don’t understand is essential for healthy discussions. There is no shame in admitting you don’t understand some part of the discussion and wanting more clarification. I have found that some students want to preserve the illusion of knowledge and avoid speaking up so they seem competent. Not knowing something can be intimidating — especially at the start of a project when most parts are unknown. One self-actualizing strategy I’ve liked from a student was to always add “yet” to the end of any sentence about not knowing something, e.g., “I don’t know how to train a neural language model yet,” when asking a question. This positive mindset is critical going forward as a researcher; there will be much you won’t know and even more than you’ll never know but it is essential that you have faith in your ability to learn new things as they come up.
After the meeting, if a Pine student realizes they didn’t understand something, they can potentially go back their notes or recording. For technical questions, I encourage students to ask their fellow labmates first for help and advice before reaching out to me. A main motivation for this is building lab knowledge; having to explain something or debug someone else’s code provides a valuable experience to both parties, who inevitably learn more. It also creates a culture of helping one another that lets problems get solved more quickly.
Suggestions on how to become a Pine:
- Record the meeting audio and immediately post it to Slack so you and your teammates can go back over anything later.
- Take pictures of any diagrams on the whiteboard and upload these to Slack too for everyone in the group.
- If you don’t understand something ask for clarification. One useful way to ask is to try to restate what you think is happening and ask if that’s right.
Scenario: the meeting is wrapping up on time and the advisor and student have a few minutes left for discussion on what are the next steps they will be working on until the next meeting.
In general, the Sapling student will leave the meeting without solid understanding of the next steps. Often this vagueness is due to one or more of a few reasons:
- the Sapling student has not taken notes during the meeting and can’t recall the specifics of what needs to be done or how it needs to be done
- the objectives are specified somewhere (e.g., Asana) and the Sapling student has said they understand but upon further reflection realizes that they have more questions
- the Sapling student needs something to make progress (e.g., needs to figure out how to set up a Jupyter notebook) but hasn’t made this clear to the advisor, which blocks the effort on the actual next steps.
The Sapling student may also have a few ideas they have thought of that might be good next directions but won’t bring these up, usually assuming that because there are already several next-steps to be done, their ideas can wait.
The Pine student concludes the meeting by suggesting what they think are the next steps and why. These suggestions include addressing things that are currently blocking the student. Then the advisor and the Pine student discuss the next steps together. The Pine student may also challenge the advisor’s next steps and suggest new directions for the project as a whole.
Reflection: One of the most powerful skills a new researcher can learn is to predict what their advisor would say. Instead of asking what are the next steps, a Pine student suggests what they think the advisor would suggest as the next steps. This role assumption helps students get unstuck on their own later by asking themselves what their advisor would say in the current predicament. In the case of project planning, a discussion of next steps helps the Pine student understand the connection between their current work and the goals of the project. While not everything the Pine student suggests is accepted, the advisor’s role is to help the student understand why certain things are re-prioritized or not done so that the student can learn about the process of science. As students grow in their understanding of the project, I appreciate being challenged to justify why one of my suggestions should be taken — healthy skepticism can lead to fruitful conversations about project goals and potentially new avenues for research. That said, lots of challenging can be exhausting.
Suggestions on how to become a Pine:
- At the end of the meeting, suggest what you think should be the next steps for the project.
More generally, a part of the transition from Sapling to Pine is their reimagining of their work as research rather than homework-like programming assignments. The latter leads to overly narrow questions and a mentality that all work on the project is driven by the advisor, e.g., determining next steps, setting meeting agendas, specifying how to make figures. Initially, this mentality might be necessary for the truly fresh students who is just learning what research means. However, the goal is to have the student take ownership of the research and drive both the research questions and goals in conjunction with the advisor.
In nature, not all seeds, sprouts, and saplings wind up growing into full pines — those that do are the ones that have access to resources (water, light, space, freedom from predators/pests). As an advisor, my responsibility is to provide these as much and best as I can. By preparing for meetings, you can ensure that you, young Sapling, are well positioned to absorb the resources (guidance, feedback, advice) your advisor and labmates can offer.
I hope this post has helped frame some of the opportunities for growth that I’ve seen in my own lab’s students and I welcome any discussion or suggestions from others!
¹ I’ve written this post with the undergraduate researcher in mind, but I suspect it would also apply broadly to new masters students and even the occasional PhD student.
² Specifically, it’s the Eastern white pine.
