Doctoral colloquiums are chicken soup for the young researcher’s soul

Benji Xie
Benji Xie
Apr 4 · 9 min read

In 5–10 yrs, all the technology you built will be obsolete. So what will be your lasting contributions? Ponderings from my first doctoral colloquium…

As a third year PhD candidate who just completed my general exam (please reread “candidate”), it was prime time to attend my first doctoral colloquium (known as a DC). A DC is an opportunity for mid-late stage PhDs to present our work and ideas for your dissertation and receive incredible feedback from caring faculty mentors. So I wandered off to Washington, DC to attend the 2019 iConference, and have my first DC in DC (neat, huh?). My objectives were to get feedback on my dissertation work at my first doctoral colloquium from the very interdisciplinary information school (iSchool) community. And I got everything I asked for and more.

The doctoral colloquium was an all-day event where faculty from iSchools from all over the world mentored PhD students on their dissertation work. The day consisted of student lightning talks, small group discussions on dissertation summaries we previously wrote, CV critiques, and a faculty panel on hiring, faculty life, and everything in between.

In this post, I’ll share my experiences with each part of the DC and also talk about things at a higher level so you too can go through a DC-esque experience and stretch your thinking about research!

Related to my dissertation work on human-AI collaboration in computing education, I was looking for help finding similar work, deepening my theoretical framings related to how people learn and interact with adaptive systems, thinking about how my work does (not) generalize. I was also looking for help in scoping my research contributions, as my work is in the space of human-computer interaction, artificial intelligence in education, and psychometrics.

Lightning talk: Pitch your {self, research}!

The day started with faculty introductions then student lightning talks. Each student had 3 minutes and 2–3 slides to share who they were, what their research was about, and why it mattered.

A total of 25 students participated, with most approaching a dissertation proposal, some near defending, and a few even having recently defended. The doctoral colloquium experience definitely varies according to what stage you are in your PhD. Whereas earlier students (me) have the flexibility to consider feedback that can change their research direction, later students focus on feedback that sharpens how they frame and communicate their work.

For the lightning talk, I talked about my work in human-AI collaboration in computing education. I framed my work as describing how to balance human agency and AI automation within the context of education. AI/adaptive tools can automate tasks which could not be done efficiently or effectively by people alone, but people are better suited to make context-sensitive decisions. Within the context of education, there are complex processes (learning to code, improving assessments) that require expertise and careful consideration of people to understand. So I’m interested in understanding the specific educational domain and stakeholders and their needs to develop adaptive tools which can integrate into their processes. A high level description of my process is depicted below.

The sheer breadth of other students’ research was exceptional! Common themes included studying online and social media communities/identities, understanding subgroups (breastfeeding moms, youth in at-risk communities, black men who have sex with men), and science of science. Some research that caught my attention was Mike DeVito’s work on the temporality of algorithmic literacy, Priya Kumar’s work on how parents enact children’s identities online, Jennifer Pierre’s work on how young people use social media as support, and Firaz Peer’s work on community dashboards as infrastructure for data literacy. Can I do a second PhD?! (but not really)

What I learned makes a great DC lightning talk:

  • have a hook to attract attention (I often try to get the audience to do something.)
  • mention what stage you are in your PhD (especially if you’re nearing the job market!)
  • motivate your research before explaining it (people don’t listen unless it’s relevant to them)
  • keep the slides light on text (← life mantra)
  • close with a conclusion and open questions you have

Breakout session

During the breakout session, I was mentored by Prof. Kentaro Toyama (Michigan iSchool). After reviewing each others’ five page dissertation summaries, we each took turns sharing information about ourselves, our research, and questions we had. Kentaro then provided some insight to blow our minds.

Kentaro encouraged us to frame our background information as according to the following format:

  • What stage are you at in your PhD?
    As mentioned previously, the stage you are in your PhD affects the kind of feedback that can help you.
  • What is your career goal?
    Knowing what your career goal is (e.g. tenure-track research position in CS department) can help direct and frame your research. For example, different research communities accept/require different evidence to demonstrate something is true or works. Knowing what community you want to engage with can affect how you approach a potential evaluation.
  • What is the main intellectual contribution of your work?
    Kentaro put this nicely: “What out of the things that you’re learning in your work transcends the context, and is interesting to a broader information audience?
  • What questions can we help you with?
    While we might feel like we need help with everything, but it’s often more productive to have directed asks.

A huge question that Kentaro was able to answer related to claims of generalization. My work is currently within the domain of computing education, but I am curious about how it may (not?) generalize to educational technology, or even HCI more broadly. Kentaro saw generalization as a rhetorical move. This varies between and within industry and academia, but an example he gave was this: “Here’s something I learned in this this specific context. If I were to take this into this new context, I would be careful to consider these factors.” Showing that your expertise generalizes often involves showing that you know what to look for if you were moving outside of your research context.

During lunch, Prof. Megan Finn (UW iSchool) took over as a faculty mentor. Megan pushed our thinking by asking us five things:

  1. Where are you in your PhD?
  2. What are you doing in your PhD?
  3. What do you need help with?
  4. Argue for the relevance of your work.
  5. What are your career goals?

I found arguing for the relevance of my work to be the most fun challenge. Megan pushed me to start with a concrete example to “set the scene” for people who don’t know my research domain. She went on to push me to surface my tacit assumptions that underly my work and motivate them. Expertise is a one-way road, so having an insightful faculty member from outside of my research community give me feedback was incredibly helpful.

CV Review: Navigation across communal norms

A really neat part of this DC was a review of our Curriculum Vitae (CV), a detailed synopsis of research, teaching, and service work we somebody has done. I have never had my CV reviewed, so I was excited to learn how to think about its design and improve it.

Leading my CV review group Prof. Diane Kelly (Tennessee). Diane stressed the importance of styling our CV such that it is familiar to the target audience (e.g. the institution you’re applying for). Looking at CV of faculty at a department of interest can help inform this process. However, the interdisciplinary nature of iSchools makes it a bit tricky (impossible?) to make your CV familiar to everybody.

Here are some factors that came up when we reviewed each others’ CVs:

  • The order of sections reflect the importance of them. This can vary between a research or teaching position.
  • The right amount of depth and detail is key. You want enough depth to articulate your research contributions, but not so much that it looks like you’re padding your CV!
  • Sections to consider including are a statement of research interests, service, affiliations, teaching experience (especially if you were instructor of record), students supervised.

An obviously important yet very challenging part of the CV is the reference section. The challenge comes in how different communities value contributions differently. Computer science communities are notoriously odd for archiving conference papers. To signal the importance of conference papers, you could ensure “peer-reviewed” or “refereed” is included in the section header, actually explain the difference, or include acceptance rates for each conference paper. Another thing to include is number of pages as that helps people understand the scale of that contribution. And just to further confuse things, differences in citation style inherently frame your research in certain ways. Head spinning yet?

Faculty Panel

The DC concluded with a faculty panel where 7 faculty panelists from many top-tier research iSchools in the United States answered students’ questions on hiring and faculty life. An important caveat to all these responses is that iSchools and their traditions, policies, and processes tend to vary quite a bit. At risk of misrepresenting people and institutions, I’ll leave out all names.

Related to hiring, students asked many questions related to hiring in the very interdisciplinary iSchools. Panelists said that iSchools navigate the interdisciplinary nature of job packets by deferring to the faculty and standards of the research community that applicants claim to be in. This is a bit more challenging when work is interdisciplinary it does not fit into a single community, but iSchools are typically willing to entertain this!

The most important thing for hiring (at a tenure-track R1 university) is “research above all.” Many institutions “hire to tenure,” so faculty want to hire those who can consistently publish (so large time gaps in a CV may need explaining). Multiple panelists agreed that strong letters of recommendations from institutions that are “at or above” the level their home institution was an part of the job packet.

From what panelists said, I can say that a strong job candidate…

  • has a coherent research narrative and trajectory that they can use as a springboard (often hiring for tenure)
  • has a strong, consistent publication track (time gaps in CV may be red flags!)
  • has strong letters of recommendation from institutions that are “at or above” the target institution
  • are collegial (play nicely)

Lots of questions came up about how funding (surprised?). To make ends meet when faculty are between grants, faculty often rely on their department’s bridge funding or do more frugal research (e.g. secondary analysis of preexisting dataset which relates to your research trajectory). They also mentioned how departmental support for student funding was guaranteed in some departments (referred to as a backstop), whereas others make it solely faculty responsibility to fund students.

Post-doctoral positions (“post-docs”) can help extend “the yardstick” of publications without the responsibilities of faculty and before the tenure clock begins. One panelists said the ideal post-doc involves setting up a bunch of publications that could be submitted shortly after starting a tenure-track position (that’s some serious strategizing!). A great advantage is if a post-doc can be included in faculty meetings, mailing lists, and activities; by doing so, they can learn more about faculty life before jumping right in! Panelists recommended caution related “teaching post-docs,” as they may or may not afford research opportunities. A warning/opportunities about post-docs overall is that “institutions can often be Markov” about where candidates come from. So a post-doc at MIT means you are now a candidate coming from MIT!

Regarding transition to faculty life, panelists talked about balancing commitments. While research is key to career advancement, other responsibilities (teaching, service, advising) are critical. As one panelist put it, be “ruthless collegially” when defending your time. Related to non-research work, panelists noted that research impact is not the same as real world impact and may often times be two separate jobs (with career advancement prioritizing research impact). Furthermore, they emphasized how service commitments and asks (e.g. being on committees, reviewing, travel, giving talks) tended to ramp us as faculty advance.

A nice final question related to how faculty coped with the uncertainty and stress of academia. One panelist picked up mindfulness and mediation strategies as the tenure deadline approached. Multiple panelists noted separating work and not work time-wise and even location-wise (e.g. only work in office, never work at home). Honest tracking of time can also help with planning the day (adding a 2–4x multiplier because we’re bad at estimating time). At the end of the day, it’s about balance, and balancing is about having hard and fast transitions between work and not work.

Conclusion

As I reflect on my first doctoral colloquium experience, I can’t help but get excited by the people I interacted with. People define experiences and “workplaces” and I firmly believe academia has some of the best people I have ever met. Over a dozen faculty members volunteered to spend a very full day enriching students they had never met before. And the constructive feedback, careful advice, and candid perspectives of faculty mentors and other students helped me advance my research perspectives, get me more excited about my current and future work, and leave me feeling like these discoveries can make this world a little better off. Onward!

Bits and Behavior

This is the blog for the Code & Cognition lab, directed by professor Andy Ko, Ph.D. at the University of Washington. Here we reflect on what software is, what effects it's having on the world, and our role as public intellectuals in help civilization make sense of code.

Benji Xie

Written by

Benji Xie

Ph.D. candidate at the UW iSchool. There’s a symbiosis between man, machine, and data; I’m all about it. @benjixie

Bits and Behavior

This is the blog for the Code & Cognition lab, directed by professor Andy Ko, Ph.D. at the University of Washington. Here we reflect on what software is, what effects it's having on the world, and our role as public intellectuals in help civilization make sense of code.