Not So Fast: The Hidden Value of Delaying Educational Feedback
I recently read a meta-analysis on the effects of feedback in computer-based learning (Van der Kleij, Feskens, & Eggen, 2015) and one of the authors’ primary conclusions was that their analysis supported the broadly accepted claim that delaying feedback has a negative impact on learning outcomes. This belief that educational feedback is best if provided immediately and frequently is widespread and repeatedly espoused in learning design and educational research circles. This is understandable given that it mirrors the recommendations of most major literature reviews on the topic (e.g., Kulik & Kulik, 1988; Mory, 2004). It’s also is a principle widely employed in modern adaptive homework and intelligent tutoring systems, which often provide instantaneous feedback to students after answer submission. Surveys of student preferences, furthermore, consistently find that students overwhelmingly prefer immediate to delayed feedback (Miller, 2009; Mullet, Butler, Verdin, Borris, & Marsh, 2014).
But while there is little question that learners ought to receive feedback on their learning efforts, should it always be immediate? Is there value in delaying or decreasing educational feedback?
Evidence continues to mount regarding the value of delaying feedback to learners in many situations. In particular, studies reveal that immediate feedback often produces inferior learning outcomes when the measure of learning is long-term retention/transfer rather than near-term improvement in performance.
Immediate trial-by-trial feedback can be viewed as a form of massed practice that may function as a cognitive crutch, boosting student performance during skill acquisition and supporting the illusion of comprehension, but ultimately leading to transitory learning outcomes and shallower understanding.
For example, Hattie (2009) notes that studies of feedback showing the largest effects were those providing students copious cues and reinforcement. But this finding isn’t surprising given these studies typically evaluate ‘learning’ by how quickly student performance increases. In contrast, delaying and reducing feedback, which inevitably reduces learner performance by increasing errors, fomenting additional struggle, and slowing knowledge acquisition, may engender superior long-term learning outcomes(Soderstrom & Bjork, 2013).
So the challenge for learning designers is to think critically about when delaying learner feedback may be superior to immediate feedback and avoid defaulting to immediate feedback in most learning situations. In particular, designers must be careful not to be misled into mistaking quick gains in student performance — buttressed by immediate and frequent feedback — for enduring and flexible learning.
Does the Evidence Really Favor Immediate Feedback?
What I’ve suggested may sound heretical given the general research consensus on the importance of immediate feedback, but the superiority of delayed over immediate feedback in certain situations has been observed by researchers for decades (e.g., More, 1969; Kulhavy & Anderson, 1972; Sturges, 1978).
In perhaps the seminal meta-analysis on feedback timing (Kulik & Kulik, 1988), the authors found that delaying feedback produced beneficial outcomes in all but one of the eight studies assessing knowledge retention at lengths approaching a week. However, this finding was dismissed by the authors and subsequent researchers (e.g., Mory, 2004) because most of the studies that found superior effects for delayed feedback were conducted in laboratory rather than in classroom settings. But this explanation is deeply unsatisfactory, as several authors have noted (e.g.,Butler, Karpicke, & Roediger, 2007; Metcalfe, Kornell, & Finn, 2009), because it overlooks the plausible explanation that students in classroom settings are less likely to adequately process delayed feedback compared with students in more controlled laboratory settings. A student receiving feedback on her test several days later, for instance, may not even look at the feedback, whereas students receiving trial-by-trial feedback are essentially compelled to process feedback after each question.
Furthermore, feedback research purporting to establish the superiority of immediate feedback is deeply marred by serious methodological and conceptual issues (see Mullet et al., 2014). Specifically, studies typically commonly evaluate learning outcomes with post-tests occurring immediately or within a short period of time (measuring gains in performance rather than long-term retention), confound timing of feedback with feedback type (e.g., knowledge of results vs. elaborated feedback), and, as noted earlier, fail to ensure students adequately process feedback across different timing conditions. The historical dismissal of the value of delayed feedback, therefore, appears to be largely unjustified.
All of these issues plague the meta-analysis mentioned earlier by Van der Kleij and colleagues (2015) denigrating the value of delayed computer-based feedback. It is particularly mind boggling to see the authors suggest that delayed feedback is inferior to immediate feedback on the basis of studies that don’t include feedback timing as a measured variable. The two studies in the meta-analysis (Ifenthaler, 2010; Murphy, 2007) purporting to show the largest negative effects for delayed feedback, for instance, employ delayed feedback in all experimental conditions and the reported effect sizes were solely a function of the different types of feedback provided to participants. This is like concluding that coffee has a negative effect on memory after having two groups drink coffee, teaching one group a novel studying technique, and then concluding that coffee is harmful when it’s found that the group using the new memory technique did worse on a subsequent memory test!
How Delaying Feedback Can Improve Learning
Despite the dearth of quality of research on feedback timing, are there good reasons for thinking that delaying feedback is educationally valuable? Evidence continues to mount supporting the superiority of delayed over immediate feedback in a variety of learning situations (e.g., Butler & Roediger, 2008; Guzman-Munoz & Johson, 2007; Metcalfe, Kornell, & Finn, 2009; Mullet et al., 2014). The most interesting questions, however, are when and why delaying feedback may engender better learning outcomes.
Although research is still preliminary, delayed feedback may have immense learning value for several reasons:
- Delayed feedback can function as spaced practice. Delaying feedback for questions that students already know can strengthen learner knowledge and support long-term retention of material (Butler et al., 2007; Pashler et al., 2007). In fact, studies have found that the optimal delay for providing students feedback on their performance mirrors the ideal lag times for restudy in studies of distributed practice (Smith & Kimball, 2010). And even in situations where learners do not know the correct answers, delayed feedback can still enhance learning (Kornell, 2014).
- Delayed feedback can facilitate increased metacognitive activity. Several studies report increased student ability to detect errors and transfer knowledge when receiving delayed feedback (Clariana et al., 2000; Sanders, 2005; Schooler & Anderson, 1990). Supporting performance accuracy may sometimes interfere with greater student mindfulness during learning. Letting students struggle, work to identify errors, and even pursue incorrect paths (at least for awhile) may support greater understanding in the long run. The delay of feedback can also facilitate “productive failure” (Kapur & Bielaczyc, 2012).
- Delaying feedback can reduce cognitive load. For learners possessing moderate prior knowledge in a domain or tasked with solving complex problems, immediate feedback can be redundant and interfere with important metacognitive activities (Fyfe & Rittle-Johnson, 2016; Shute, 2007).
- Delaying feedback can create desirable difficulties. The finding that frequent and immediate feedback boosts short-term performance but produces inferior learning long-term is well-supported in motor-learning research (see Schmidt, 1991). Similar findings regarding slower progress during training, but increased long-term learning have also been found in studies of cognitive learning as well (Schooler & Anderson, 1990). Effective education needs to make learning appropriately troublesome for students and engage them in productive struggle. In many cases this means letting learners ponder, reflect, and try to resolve errors on their own without presenting a worked example, hint, or solution the moment they give up or answer incorrectly.
There is no doubt that immediate feedback can be effective in a variety of learning situations — particularly when learners completely lack the basic knowledge necessary to answer a question or solve a problem. There is nothing educationally desirable about letting learners struggle without any hope of success or progress.
However, there are many learning situations where delaying or decreasing feedback may lead to deeper processing and more effortful retrieval of information — activities that support enduring and flexible learning. As designers of learning experiences we must resist the urge to always provide learners with immediate feedback and find ways to reduce their cognitive struggle. It’s likely the most effective learning experiences involve a mix of immediate and delayed feedback, with the timing of feedback influenced by problem difficulty, learning outcome, desired retention length, learner prior knowledge, and practical considerations about student ability/willingness to process received feedback.
So when it comes to providing students feedback, don’t be so fast. Successful learning looks more like a marathon than a sprint.
References
Butler, A. C., Karpicke, J. D., & Roediger, H. L. (2007). The effect of type and timing of feedback on learning from multiple-choice tests. Journal of Experimental Psychology. Applied, 13(4), 273–281.
Butler, A. C., & Roediger, H. L. (2008). Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Memory & Cognition, 36(3), 604–616. doi:10.3758/MC.36.3.604
Clariana, R. B., Wagner, D., & Roher Murphy, L. C. (2000). Applying a connectionist description of feedback timing. Educational Technology Research and Development, 48, 5–22.
Fyfe, E. R., Rittle-Johnson, B. (2016) Feedback both helps and hinders learning: The causal role of prior knowledge. Journal of Educational Psychology, 108(1), 82–97.
Guzman-Munoz, F., & Johnson, A. (2007). Error Feedback and the Acquisition of Geographical Representations. Applied Cognitive Psychology, 21.
Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge.
Ifenthaler, D. (2010). Bridging the gap between expert-novice differences: The model-based feedback approach. Journal of Research on Technology in Education, 43, 103–117.
Kapur, M., & Bielaczyc, K. (2012). Designing for productive failure. Journal of the Learning Sciences, 21(1), 45–83.
Kornell, N. (2014). Attempting to answer a meaningful question enhances subsequent learning even when feedback is delayed. Journal of Experimental Psychology. Learning, Memory, and Cognition, 40(1), 106–14.
Kulhavy, R. W., & Anderson, R. C. (1972). Delay-retention effect with multiple-choice tests. Journal of Educational Psychology, 63, 505–512.
Kulik, J. A, & Kulik, C. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79–97.
Metcalfe, J., Kornell, N., & Finn, B. (2009). Delayed versus immediate feedback in children’s and adults’ vocabulary learning. Memory & Cognition, 37(8), 1077– 1087.
Miller, T. (2009). Formative computer-based assessments: the potentials and pitfalls of two formative computer-based assessments used in professional learning programs. Dissertation Abstracts International, 70, 4,, AATNR48227. (9780494482278).
More, A. J. (1969). Delay of feedback and the acquisition and retention of verbal materials in the classroom. Journal of Educational Psychology, 60, 339–342.
Mory, E. H. (2004). Feedback research review. In D. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 745–783). Mahwah, NJ: Erlbaum.
Mullet, H. G., Butler, A. C., Verdin, B., von Borries, R., & Marsh, E. J. (2014). Delaying feedback promotes transfer of knowledge despite student preferences to receive feedback immediately. Journal of Applied Research in Memory and Cognition, 3(3), 222–229.
Murphy, P. (2007). Reading comprehension exercises online: The effects of feedback, proficiency and interaction. Language Learning and Technology, 11, 107–129.
Pashler, H., Rohrer, D., Cepeda, N. J., & Carpenter, S. K. (2007). Enhancing learning and retarding forgetting: Choices and consequences. Psychonomic Bulletin & Review, 14, 187–193.
Sanders, M. (2005). The Effect Of Immediate Feedback And After Action Reviews (AARS) On Learning, Retention And Transfer. Masters Thesis. Retrieved from: http://stars.library.ucf.edu/cgi/viewcontent.cgi?article=1384&context=etd
Schmidt, R. A. (1991). Frequent augmented feedback can degrade learning: Evidence and interpretations. In Tutorials in motor neuroscience. Edited by J. Requin and G. E. Stelmach, 59–75. Dordrecht, The Netherlands: Kluwer.
Schooler, L., & Anderson, J. (1990). The disruptive potential of immediate feedback. In Proceedings of the Cognitive Science Society. Hillsdale, NJ: Erlbaum.
Shute, V. (2007). Focus on Formative Feedback. Review of educational research. Princeton, NJ.
Smith, T. A., & Kimball, D. R. (2010). Learning from feedback: Spacing and the delay-retention effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(1), 80–95.
Soderstrom, N. & Bjork, R. (2013). Learning versus performance. In Dana Dunn (ed.) Oxford Bibliographies Online: Psychology. New York: Oxford University Press.
Sturges, P. T. (1978). Delay of informative feedback in computer-assisted testing. Journal of Educational Psychology, 70, 378–387.
Van der Kleij, F. M., Feskens, R. C. W., & Eggen, T. J. H. M. (2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research, 85(4), 475–511.
