Teaching Insights: Improving Classroom Instruction through Insights for Faculty
One of the largest data products that the Data Science & Analytics team has built to date is our Teaching Insights tool, essentially a dashboard for faculty. This dashboard has insights about how their current students are doing, former students, and analytics about their courses over time.
My favorite parts of the dashboard
Does an assignment prepare a student to succeed?
The chart below allows an instructor to analyze the assignments that they give in a course, including quizzes and exams. The x-axis is the average student score on that assignment and the y-axis is how the students’ scores correlated to their final course grade. In other words, did this assignment prepare the student to pass the course or was it a good assessment of the student’s skills in this course?
In this same chart, we used colors to help instructors interpret the data shown. The hover of the chart shows the assignment name and more details about the assignment.
Competing against yourself for faster assignment grading
Replicating an Orange Theory model, we show the faculty member their grading speed in the last 30 days, 30–60 days, and overall. This is calculated as the difference in time between an assignment due date (or assignment submission date if late) and when the grade was entered. Ideally, the instructor is improving their speed over time in order to provide timely feedback to students. We chose to only compare against the faculty member themselves instead of their department peers or the college overall due to the different nature of assignments and time required to grade each one.
Metrics for current students
The student summary table shows a faculty member a listing of all of their students in a class with metrics about that student. Particularly helpful is seeing in one place the last time a student viewed any page for this class in the LMS, how many days early a student typically submits their assignments, number of missed, late, or failed assignments, and any flags in our advising management system.
When are students the busiest?
You know how students are always talking about how busy they are? Well, as an instructor is planning out their semester, we show them for past semesters what weeks had higher percentages of late assignment submissions, along with the number of assignments due that week. So looking at the example chart below, it seems that week 6 had a high percentage of late assignments. As an instructor, I could evaluate what is due that week and maybe either spread out some of those assignments, or make them less difficult for students to complete in a timely manner.
Are all types of students able to succeed in my class?
We give instructors the ability to disaggregate their final course grade scores by gender, ethnicity, race, age, and first generation to college status. This allows faculty to evaluate if some groups might struggle more with their class. Once they identify an issue of uneven success rates, then it typically leads to conversations about how to improve this.
How do I influence our college goals?
Our college has five strategic goal metrics that we track at an institution level. We’ve included these metrics on this data product for the instructor to see how they influence these goal metrics by filtering to their former students. We also show a comparison to their department overall for that metric.
I’ll also point out in the screenshot above that each metric has a link to view our data dictionary definition for that metric, for clarity and transparency in how it’s defined and calculated.
What are my students doing next semester?
To help improve retention of our students, we added a chart to show faculty what percentage of their students have registered for the next term, with the ability to drill to detail on the specific students who have not yet registered.
How We Built This
This tool is built in Oracle APEX, which is not traditionally known as a data visualization or dashboarding tool. We chose this tool because it has visualization capabilities and it does not have individual licensing costs. APEX is included with an Oracle database license, which we already had. This allowed us to develop this tool for our 1,500 faculty members without high costs.
When we were first designing this data product, we pulled together an advisory committee consisting of faculty representation from each school at our college, ADs, and Deans. We developed iteratively, each month showing them new features and getting their feedback. I remember after one meeting when I heard two faculty members saying to each other as they left the room that what we showed was cool but they weren’t sure how’d they’d use it. That was a wake up call and forced us to really think through how we could make this a tool they would want to use on a regular basis and they’d quickly see the value of it.
This tool was first released almost two years ago and we’ve seen an uptick in usage. What’s really helped get the word out to all faculty is by having a faculty advocate who can speak to how he uses it in his classes.
