The Role of Curriculum Pacing in Learning Analytics
Introduction
Using learner data to make teaching decisions is becoming more and more common. When students are using digital technology for education, it becomes very easy to gather vast amounts of data about their learning. Using this data, educators can make more informed decisions about what to teach in their classrooms, who needs more help, and who can advance quickly.
A 2018 survey in the United States found that 95% of the K-12 teachers use a combination of academic data and non-academic data to understand their students’ performance.
34% of the surveyed teachers reported that there was too much data for them to look at. This is why you need designers, learning scientists and data analytics experts to carefully carve out data representations that are meaningful and actionable to the end-users.
There are so many sources of educational data these days, and it is very easy to get lost in the sea of information.
Over the years in my learning analytics journey, I and my team have made hundreds of data reports for different school districts in the US. Our human-centered learning analytics approach has let us discover some wonderful insights about how data can start making a real impact on the learning journeys of the students.
Human-Centered Learning Analytics
If Learning Analytics has to have an impact on student learning, the data needs to be understood well by the education stakeholders and decision makers.If you show the result of a complex regression model to a classroom teacher, they will probably not know what to make of it (unless they’ve been trained for doing so). But if you show them the expected scores of the students in the final tests, they will know what to do with that data.
As you analyze data of more and more students, you have to present it in a form that is easy to interpret and act upon. Remember, you want your data to be acted upon. Otherwise you almost wasted your time doing the analysis.
Although, when you are dealing with data from thousands of students, you start wondering about the data presentation. How can you present the big student data in a way that educators and administrators understand? How can you speak their language? How do you know what they care about, and make your learning analytics findings more intelligible? Because once your findings are understood, they can be acted upon — sometimes immediately. If you can show that a certain school needs help in a certain knowledge area, someone can talk with the principal or the curriculum coordinator.
To make our learning analytics findings self-evident and actionable to education stakeholders, we should come up with innovative data presentations, build a strong narrative, and focus on simple but effective things.
For example, show the hot spots in the data that can trigger action, show the trend in the data, highlight anomalies or odd-one-outs, contrast the data with the ideal situation, etc. For the data analyst, a bar chart can be like ducks and bunnies, but for a teacher, it can be an actionable piece of information! And our focus is always on the actionability of the data. If the end-users can make use of the data, we have done our job well. If not, we have to start all over again.
Curriculum Pacing
When we started looking at digital curriculum data with some district administrators, we found that one of the things they really cared about was pacing calendars. The pacing calendars are curriculum sequences laid out on the calendars that tell teachers how they can teach the curriculum over time so that they can finish teaching all of the topics by the end of the school year. Here is a sample of Math Grade 3 pacing calendar from Houston ISD:
These pacing calendar guides can help the teachers stay on track with their teaching — a thing very difficult when you have a really dynamic classroom. So the district administrators that we were working with asked whether we could show them the pacing of their district.
Curriculum Pacing Data Visualization
(In 2018, we won the best-short-paper award at the Intelligent Tutoring System for our research study Curriculum Pacing: A New Approach to Discover Instructional Practices in Classrooms)
We decided to build a data visualization that can show us the learning journey of multiple students at the same time. We made many different attempts to portray learning journeys of students over time. Some plots had too much information and were hard to read, while others just looked plain ugly. After many different trials and errors, we, we ended up putting time on the X-axis, and curriculum on the Y axis, and made a heatmap to show the student’s progress.
Ta da! We finally had a great visual representation of a student’s learning trajectory over time! We called our plot a pacing plot. In the pacing plot, the X-axis captures time. As you go from left to right, the time moves forward. And the Y-axis represents the curriculum. As you go from bottom to top, the curriculum advances. You can see that the student on the left finished up their course in about 40 weeks. You can also see some holidays, some weeks where multiple chapters from the curriculum were looked at, and some attempts around the 30th week to catch up with older material. Let us see some more examples:
Above, the plot on the left shows that the student had an ‘accelerating’ pace and finished the digital curriculum quite quickly. The plot on the right shows a very interesting pattern. We see that the student referred to the older material many times during the year. We call these patterns ‘icicles’ because they look like — well — icicles! This is not the only format of the pacing plot. There are several design factors of curriculum pacing plots that can be altered to make different versions:
- Data: single student, classroom, school, district
- X-axis: normalized weeks, calendar time
- Y-axis: lesson number, content arranged by avg. week used
- Plot type: heatmap, scatter plot
- Fills: usage, score, percentile
When you have pacing plots of thousands of students, it is likely that you have different sets of students following different pacing patterns. In our research study, we clustered different pacing patterns of the students using the K-Means algorithm and showed the similar patterns together by overlaying them on each other. Below are some interesting clusters of the students that we found:
You can make a pacing plot for each school, and overlay the ideal pacing pattern on top of it. Or you can compare it with last year’s pacing pattern. And there are many more things you can do! When we put these pacing plots or clusters in front of the district administrators, not only do they like them, they also know what to do with them. That makes the data visualization valuable The ultimate goal of using these visualizations is to identify where students and teachers need help.
Actionable Intelligence from Educational Data
We want our learning analytics to impact students’ learning. When we show data to education stakeholders, we want it to be actionable — so that teachers can quickly help their students do the best. And our team always keeps working towards this end. Almost every day — except when we are updating our software.
Data @ Playpower Labs
We are a dedicated team of Learning Analytics experts at Playpower Labs. We offer data analytics toolkits, explainable adaptivity engines, and data science services to educational organizations. If you want to use your educational data for decision making and product improvement, please send me a message on LinkedIn or an email at nirmal@playpowerlabs.com!
