Analyzing UW Math Faculty Course Evaluations
You’ve probably filled out a end-of-term course evaluation at some time in your student life. It’s the survey that asks you to rate your professor’s coherence, oral and visual presentation, helpfulness, and all that. After you fill it out, it disappears into a black hole, never to be seen again.
Most students don’t know that the results of this survey are published on Mathsoc. You can look at summaries of classes from previous terms. There is no search functionality, but the data is there.
In this article, I will analyze this dataset and answer common questions about professors and teaching quality. Who are the best and worst profs in UW Math? Do profs get lazy and stop caring about classes after they get tenure? Do classes generally get better in upper year? All of these questions can be answered with data science!
First look at the dataset
First, I downloaded all the PDF data files for the terms from Winter 2012 to Spring 2016 (14 terms total), and converted them to TXT. Then, I wrote a Python script to parse the TXT files and extract all the relevant numbers.
I looked at the following questions from the survey:
- Evaluate the organization and coherence of the lectures (Excellent / Good / Satisfactory / Unsatisfactory / Very Poor)
- Evaluate the instructor’s treatment of students’ questions(Excellent / Good / Satisfactory / Unsatisfactory / Very Poor)
- Evaluate the effectiveness of the instructor’s visual presentation (Excellent / Good / Satisfactory / Unsatisfactory / Very Poor)
- Evaluate the effectiveness of the instructor’s oral presentation (Excellent / Good / Satisfactory / Unsatisfactory / Very Poor)
- Evaluate the overall effectiveness of the instructor as a teacher (Excellent / Good / Satisfactory / Unsatisfactory / Very Poor)
I then converted it to a 5-point system where 5 = Excellent, 4 = Good, 3 = Satisfactory, 2 = Unsatisfactory, 1 = Very Poor. Some questions had answer options of (Too much / Somewhat too much / Okay / Somewhat too little / Too little), and I ignored these since they don’t easily convert to a 5-point system.
The dataset contains some very small classes, so to reduce variance, I only looked at classes with at least 15 students.
Most courses are okay: the average response to the “overall effectiveness” question is somewhere between “Good” and “Excellent”, with only a small number of classes scoring “Satisfactory” or below.
The answers to the 5 questions (coherence, visual, oral, questions, overall) are highly correlated. Oral presentation is slightly more important than visual presentation for overall effectiveness.
Since the responses are so highly correlated, I’ll just look at the “overall effectiveness” question from now on.
Who are the best / worst / most prolific profs?
Most profs teach less than one class per term, on average. (In this analysis, I’m counting a class twice if you teach two sections of Math 135, but only once if you teach a crosslisted class like CS486/686.)
Quite a large number of lecturers are PhD students who only teach once or twice. There are also some profs who regularly teach multiple classes every term. The 5 most prolific (most sections taught) instructors are:
- Mark Petrick (30)
- Lori Case (26)
- Bradley Lushman (26)
- Mukto Akash (25)
- Keith Freeland (24)
For the list of best and worst profs, I only consider profs who taught at least 3 classes. There are 284 profs who taught at least 3 classes from Winter 2012 to Spring 2016. I will list the most frequent classes taught by each prof in brackets.
The 10 best profs in the Math faculty are:
- Kathryn Hare (MATH147, MATH148, PMATH450)
- Michael Eden (MATH137)
- Siu-Hang Li (ACTSC232, ACTSC433, ACTSC611, ACTSC622)
- Eric Blais (CS365, CS489, CS860)
- Yu-Ru Liu (MATH135, MATH138, PMATH348)
- Serge D’Alessio (MATH137)
- Dan Wolczuk (MATH136, MATH137, MATH138, MATH235, MATH237)
- Edward Dupont (CS115, CS116, MATH136, MATH237)
- David McKinnon (MATH127, MATH147, PMATH346, PMATH347)
- Ian Goldberg (CS458, CS135)
The 10 worst profs in the Math faculty are:
- Iakov Nekrich (CS240, CS341)
- Jonathan Buss (CS245)
- Steven Gindi (MATH106, MATH235)
- Daniela Maftuleac (CS245, CS115)
- Edward Chan (CS348)
- Kamyar Moshksar (STAT230, MATH135, AMATH332)
- David Toman (CS245, CS338)
- Maite Dupuis (MATH137, MATH127)
- Eugene Zima (CS350)
- Yongqiang Zhao (MATH136, MATH106)
Interestingly, 3 of the 10 worst instructors have CS245 as their most frequently taught class (often regarded as the worst required CS course), but when CS245 is taught by anybody other than these 3, it generally receives positive reviews.
I also looked at the relationship between score versus class size, as well as score versus experience of instructor, but found no relationship.
Here’s a sorted list of all 284 profs, from highest to lowest ratings.
Relationship between salary and teaching quality
Next, I investigated the correlation between teaching quality and salary. Do better instructors actually get raises, as they often claim? Or do profs stop caring about their lectures after they get tenure?
The salary for all university employees are publicly disclosed, so I matched up all the professors from the evaluations with their salaries. Not all instructors were listed in the salary disclosure: only faculty with salaries above 100k are included, so the following analysis only includes the names I could match up.
First, I plotted a scatterplot of average class rating versus salary:
There is no statistically significant relationship here. (Correlation coefficient is 0.02, t-test finds no significance)
The salary disclosure page also lists the titles of all university employees. Most instructors have title either Assistant Professor, Associate Professor, Professor, or Lecturer. Basically, the normal progression is Assistant Professor -> Associate Professor -> Professor, and a Lecturer is someone hired to teach classes and doesn’t do research. Professors and Associate Professors are tenured (permanent positions) but Assistant Professors are not.
As expected, there is a significant salary increase when going from Assistant Professor -> Associate Professor -> Professor. The salary of a Lecturer is somewhere in between Assistant and Associate Professors.
How does title correlate to teaching quality?
There is no significant difference in teaching quality between research faculty (Assistant/Associate/Professor), whether they are tenured or not. However, Lecturers, who focus on teaching and don’t have research responsibilities, teach better than research faculty. (The t-test for difference between means gives p < 0.05 for Lecturer vs each of the other 3 classes, but no significant difference within the 3 professor classes).
Course evaluations by major and grade level
How do course evaluations differ for different majors? Most classes in the Math faculty fall under one of six program groupings: ACTSCI, AMATH, CO, CS, MATH, PMATH, and STAT.
Pure Math majors are the happiest with their classes, and Actuarial Science majors are the least happy. I don’t blame them — I’d also get depressed if I had to constantly calculate the probability of dying in the next 30 years.
Next, it’s often claimed that first year classes are the worst, and classes get better in upper years as you choose the ones that interest you. Is this really true? I separated the courses by course code into 1xx, 2xx, 3xx, 4xx, and Grad (containing 6xx and above).
The evidence doesn’t support this; all undergraduate classes are about equally good, regardless of level. Grad students, however, do enjoy their classes more than undergrads.
That’s it for now, comment if you have more ideas on things to analyze!