Sound the Alarm! The Danger of Academic Early Alerts

Every day countless students are flagged as ‘at risk’ by academic early alert systems. These tools notify faculty and/or advisers of students who are in danger of failing or dropping a course, enabling them to proactively target students for interventions and outreach. The adoption of these systems in higher education has been rapid and enthusiastic. A recent survey by Educause found more than a third of higher education institutions have institution-wide deployments of early alert systems, with another third reporting targeted or initial deployments.

But despite widespread excitement around early alert systems in higher education, their impact on student success remains unclear. To better understand the lack of consistently positive outcomes observed with these tools and appreciate their limitations, I want to share a story about fire alarms.

My sons recently experienced perhaps the greatest moment of their young lives. After my in-laws successfully bid at a school silent auction, a fire truck came to our house and the firefighters gave my sons a personal tour of the fire engine — even letting them wake the neighborhood with wailing sirens and blasts of the deafening horn. Then to top it all off, they got to ride in the fire truck to school and get dropped off in front of all their friends.

Pure childhood bliss.

But before driving the boys to school, the firefighters spent some time to discuss fire safety, asking whether we had working fire alarms and a plan to escape the house safely in the event of a fire. And while we did have working alarms, my wife and I realized that we hadn’t previously discussed how to respond to a house fire as a family. So that evening we spent time with our boys selecting an outside meeting location, planning escape routes from each bedroom, and discussing important things to know, like touching a doorknob to see if it’s hot before opening and crawling on your belly if there’s smoke.

I share this story because it serves as an excellent analogy for understanding the limits of early alert tools in higher education.

Just as a fire alarm has limited value without an escape plan and knowledge of how to exit a house safely, early alert tools implemented without evidence-informed communication and intervention strategies risk disaster. Alerts may lead to student outreach that is ignored, ineffective, or even harmful. For example, students may be sent demotivating or unactionable messages, educators may have limited awareness of effective learning and studying strategies to help students improve, and faculty may resist or question the value of early alert tools because they haven’t been consulted during the adoption process or convinced of their efficacy. Consequently, it isn’t surprising to see leading early alert providers acknowledge institutional implementations resulting in “an almost double-digit negative impact” and institutions disappointed with observed outcomes.

So what can institutions do to better realize the promise of early alert tools in education?

In addition to the importance of strong executive leadership, professional advising support, and faculty engagement, there is growing recognition that the effectiveness of learning analytic tools like early alerts hinge on thoughtfully linking data-derived insights to research in the learning sciences. Educational data should not only be meaningful, it needs to be interpreted and acted on in ways consistent with current learning theory. To ensure positive student outcomes, the information surfaced by early alert tools should be used in conjunction with evidence-informed intervention strategies and research-based curricular improvements.

For example, consider the task of providing intervention feedback to students flagged by an early alert system. Recent reviews of the educational feedback literature highlight several key principles of effective feedback provision. These include:

• Establishing trust with students early in a semester and explicitly fostering student feedback literacy through class discussions and multi-stage assessment design
• Ensuring intervention communications are encouraging, timely, and supportive of student self-efficacy
• Delivering feedback that clarifies a student’s current status in relation to clear course learning goals, while providing specific suggestions for improvement
• Prioritizing student agency in the learning process and establishing an expectation that students follow-up on received feedback
• Avoiding comparisons among students, academic threats, or gratuitous praise

In reviewing these principles, it’s clear they have important implications not only for how early alert communications should be crafted, e.g., their tone, timing, and content, but also for the instructional design of the courses in which they are deployed.

Early outreach with a struggling student is unlikely to improve her prospects if communication doesn’t reflect research on how to maximize student receptivity and course assessments lack the means, motivation, and opportunity for students to apply received feedback. Academic early alerts can be valuable tools in pursuit of improved student outcomes, but should be viewed as only a single component of a much broader evidence-informed student success strategy.

Unfortunately, educators are rarely supported with guidance on how to provide effective intervention feedback, nor is the influence of course design on the effectiveness of early alert systems a common consideration. We’re installing fire alarms in classrooms but leaving educators and students ill-equipped to respond when they detect smoke. It isn’t surprising, therefore, to find early alert implementations resulting in wildly varying impacts on students outcomes.

Ultimately, the success of any early alert system will depend on supplementing it’s predictive insights with evidence-informed strategies from the learning sciences. We must do a better job of translating learning research for educators and student success professionals, as well as providing them with the resources and tools needed to ensure that students benefit from the warnings that early alert systems provide.

With the rapidly expanding use of early alerts in higher education, we can’t forget to ask:

“What good is an alarm if you don’t know how to respond effectively?”