Introduction
Every year, class sizes at large universities increase. For example, this Fall, UC Berkeley’s very own introductory course CS 61A exceeded 2,000 students enrolled, making it one of the largest beginner computer science classes in the world. Teaching and facilitating such large classes, though, also has its drawbacks and difficulties. As a group, we interviewed both student teaching assistants (TA’s), graduate student instructors (GSI’s), and professors for college-level classes with over 300 students. We then created an empathy map for each of our interviewees, which we hope will help us define a common need amongst all teachers of such large courses.
Selecting a Domain
The past few weeks have seen a lot of drama in terms of the large class sizes here at UC Berkeley. Particularly, CS 61B, a computer science course with over 500 students this semester, received a lot of backlash regarding lack of support for a supposedly introductory mini project. Students cited impacted office hours and lack of support on Piazza (a collaborative online site dedicated to providing more assistance in large-scale classes).
In light of these events, our group decided that we wanted to investigate the activity of teaching a large class. To set more stringent boundaries, we defined a “large class” as one with over 300 students. We also decided to focus on those who facilitated and taught these classes rather than the students themselves, for amidst all the drama and complaints, we realized we had not seen very many actual responses from instructors themselves.
Understanding that a lot of the course staff for these large classes could actually also be students themselves, we checked with Professor Salehi to make sure that we would be allowed to interview them, given the constraints of the project. We received permission to interview more than one student, as long as he/she was not a friend of the interviewer.
Interview Content
We began by formulating a high level question that could help provide a framework for our eventual actual interview questions. Our high level question was
“What kinds of needs do people who teach or facilitate large college-level courses encounter?”
After this, we created several questions we would ask during the interview which, hopefully, would help us eventually answer our high-level question. The following is a list of questions we asked each interviewee. We aimed to prompt stories out of our interviewees regarding their own experiences teaching such large-scale classes in addition to any problems they could voice on their own. In addition to these questions, we may have branched off and asked additional questions that were relevant to each unique interview.
Demographic Questions:
- Name?
- What class do you teach?
- What is your position?
- How long have you been a [POSITION]?
- How many students are you responsible for?
- How many hours do you teach per week?
- How many hours per week do you spend preparing for your lesson?
General Questions:
- Tell me about your responsibilities/activities as a ____.
- Why do you teach? What is the thing you enjoy the most about teaching?
- What was it like when you first start teaching, as opposed to now?
- Tell me about the most recent class you taught.
- Walk me through the steps you go through to prepare for a lesson / before the semester / before exams / after exams.
- What problems do you think other _____ have encountered?
Interviews
Each group member conducted interviews within the span of one week. These interviews lasted around 15 minutes each. Detailed interview notes can be found here. We then completed empathy maps, which helped us formulate need statements for each interviewee.
Vishnu’s Needs
- Vishnu needs know his fellow course staff better, especially in such a large class where there is a lot of staff.
- Vishnu needs more support during impacted times of the semester, like near exams.
- Vishnu needs to be able to catch his mistakes or avoid them altogether, as this detracts from his class time.
Professor Ibnayov’s* Needs
- Professor Ibnayov needs smaller class size to better focus on teaching.
- Professor Ibnayov needs more administrative support to ease the management workload.
- Professor Ibnayov needs better ways to enhance student cooperation.
*Name changed per request of the interviewee
Everyone has a bucket, and it represents their own feelings of confidence and self-esteem. Some people have bucket that’s more full, some people have buckets that are more empty. And people that have a bucket that’s empty, sometimes think that the way they can fill their bucket, is from taking from other people’s buckets. So they fill their own bucket of self-esteem by taking away other people’s confidence and self-esteem. And it’s unhealthy — you can’t fill your bucket by taking away someone else’s bucket. Really the way you fill your bucket is you try to fill other people’s buckets, and your bucket gets filled along the way.
Yulun’s Needs
- Yulun needs a way to bundle questions from students who come to office hours.
- Yulun needs to know what exactly was covered in the lecture.
- Yulun needs to know effective methods to help students solve homework themselves.
Karina’s Needs
- Karina needs a sounding board or a way to get help cope with receiving feedback.
- Karina needs education about how to educate or some tool that will help create lecture information in different formats.
- Karina needs away to make friends on campus.
Needfinding Statement
After considering each empathy map and their interviewees’ needs, we discovered a general consensus amongst all our subjects that communication between the instructor/facilitator and student could be improved. We specifically chose the word “communication” to convey an open-ended-ness of what exactly was being communicated. In Professor Ibnayov’s case, it would be communication of a moral code (as seen in the bucket analogy). This is a more abstract idea to be communicated. In the other interviewees’ cases, the communication problems were more practical, ranging from relaying mistakes clearly (Vishnu), relaying repeated questions effectively (Yulun), and simply trying to figure out whether or not students had actually understood the material (Karina).
How may we streamline communication between instructors/facilitators and students?
Prototyping
After deciding on the above HMW statement, we each created a prototype of a different solution that could address our needfinding question. Our prototypes all fall into one of four categories: most likely to work, most likely to delight, most rational, and dark horse.
“Most Likely to Work” Prototype: Thumbs-Up, the Real-Time Student-Teacher Feedback Application
Our research revealed that, despite many tools like Piazza that supposedly enable better communication, instructors still often have difficulty deciphering their students’ comprehension level, particularly in real-time scenarios like lectures and discussions. This was based on an empathy map from our interviewee Karina, who often wondered whether or not she should move forward with the lesson because she was unsure of her students’ level of comprehension. In this case, Karina would use the prototype to have a better understanding of her students’ comprehension level during her discussions so that she does not need to worry whether or not she is moving too quickly.
Thumbs-Up, a Figma prototype, is a mobile application for both students and instructors to provide feedback and questions during class time. With it, students may use a toggle to indicate their general understanding throughout a class as well as ask questions. The instructor, on the other hand, can use either the smartphone screen or haptics to determine students’ averaged understanding as well as any questions that have arisen. In this way, there are two users, both of which are accounted for in the prototype.
A hi-fi prototype and demonstration can be found at this Figma Link. Sidenote: some screens are horizontal, and some are vertical, which will require the prototype settings to be toggled appropriately.
Specifically, this prototype aims to answer the question “What would the back-and-forth communication between students and instructors be like?” We answered this question by not only prototyping screens for both the student and the instructor, but also by animating various scenarios and use cases for the application.
“Most Likely to Delight” Prototype: Bundle, Streamlining your Office Hours
Our needfinding interviews in large college-level courses uncovered the potential to streamline office hours held by TAs and GSI. They reported that students often come to their office hours with the same questions. This need is not met by other tools used like Piazza or CampusWire since students like the personal and direct exchange with TAs and GSIs they now and value. Moreover, questions in large-scale communication tools tend to get lost, stay unanswered, and cannot be easily searched for common questions. In order to make more out of the scarce time of the TAs and GSIs there is a need to streamline the questions they receive during their office hours. This helps not only the instructors, but also the students since they can learn from each other and realize that they are not the only one struggling with the material.
Bundle, a communication and organization tool for office hours, helps students and instructors to tackle this problem. In a first step, students can select office hour slots at their favorite instructor. Second, they can submit a brief questions with some additional explanations and categorize them with tags. They can either create new ones or select common tags used before. After they submit their questions the instructor can see their questions in an overview screen. In this overview screen the instructor can clustered by topics which students have which questions. Similar questions will be grouped together by the app so that the instructor can help the students at the same time.
Here is a link to the Figma prototype.
As a GSI, Yulun would use the overview screen automatically generated by the app. In preparation for his office hours he would take a look at the upcoming questions and prepare for them. This tackles his need of combining repeatedly asked questions during his office hour by aggregating similar questions into the same time slots. Since he does not need to do the grouping of questions by himself the app would delight him particularly since he would not have to put in extra work but save him a lot of time and would not get asked the same questions over and over again.
“Most Rational” Prototype: Course-Commune, Enhance Communication, Collaboration and Students’ Confidence
According to our interviewees, the biggest concerns in large courses are lack of communication, collaboration and students’ lack of confidence due to overly competitive environment, which led to professors’ difficulty in keeping the whole class at the same pace, huge workload for TA’s to answer students’ questions online respectively, and many students’ eventually giving up. Therefore, based on the empathy maps, a better incentivization mechanism should be introduced to encourage students fight together, instead of fighting each other.
Inspired by interviewee Professor Ibyanov’s grading metric — taking students’ help online into account when grading their performance, Course-Commune, a paper prototype, is designed to be an online study community that actively reward students who help out each other by uploading resources, answering others’ questions and giving professor reference for students’ participation and collaboration. The more help you give, the more points you receive, which could transfer into performance credits in your final score.
Here is a video of the paper prototype:
“Dark Horse” Prototype: WalkOut
This prototype is trying to solve the communication gap between students and TAs. From the feedback we heard from TA’s, many times TA’s have a hard time engaging students understanding. Students sometimes also don’t communicate clearly if they understand a specific concept. This app tries to bypass this fear of shame where students don’t communicate their understanding by using an anonymous medium of communication that allows students to share their understanding level with the TA. On the TA side, the app shows the overall to cumulative level of understanding of the class so the teacher know whether they need to fast forward or slow down. If a student fails to understand the material well, the app suggests the student to “WalkOut.”
Here is a link to the Figma prototype.
Based on Karina’s empty map, she is worried that she isn’t good enough to teach the discussion section and concerned whether she should move forward or slow down how fast she covers material. She’s also bothered by students’ feedback that sometimes can be harsh.
In this case both the students and the TA will use the app to share and receive their perspective on the pace in which the discussion is moving. Students will rank their understanding as the discussion moves on, and the TA will get a general feel for how the class is doing. For both groups, this anonymous way of communicating will eliminate the personal aspect of the input, detaching a face from the feedback.
Collaborators: Helen Li, Daniel Haim, Zirou Ma, Christoph Wilhelm (Group 10)