Digital Math has a Guessing Crisis
In The Marshmallow Challenge teams of five students are given 20 spaghetti sticks and one marshmallow and are challenged to build the tallest tower they can. If you can pick teams, try to add as many kindergartners as you can: kindergartners typically outperformed business students and CEOs in both creativity and height. According to Autodesk Fellow Tom Wujec, kindergartners build multiple prototypes along the way, failing multiple times, but learning from each failure.
Kindergartners are good at prototyping because they are not afraid of failure. From my own research, I would add that they also seek immediate rewards (“Yay, it stuck!”) rather than future rewards (“If I plan for 10 minutes, I could make something great.”) Whether from a lack of fear of the future or enjoyment of the now, young students are not afraid to fail fast. Which leads to a different type of problem: they are also not afraid to wildly guess.
Over two years, I put eye-tracking glasses and MOXO sensors on more than 164 children in Title 1 schools and at the Boy’s & Girl’s Club as they used digital learning tools. After initial observations, I prototyped how to build a digital tool that can teach math independent of teacher input. I solved creating the right reward framework (see the article 7 Ways to Create a Feeling of Learning) but ran into a much harder problem: children cared about getting a problem right, but had no problem guessing until they got that problem right.
After the initial observational research, we created a prototype to teach children Roman numerals, a new concept not taught in school.
Inevitably, children would quickly identify V as 5 and VII as 7 but when given IV, they would guess 6. They would be told they were wrong and then guess another random number, like 10. Feedback would tell them why the answer was 4, but they would skip past that, ready for the next question.
Soon the same question would come up, “What is IV?” They would either guess 6 again or randomly guess — they were confused and frustrated. We saw this behavior across all of our studies: when children do not know how to do something, they don’t sit back and say — what’s going on here?” Instead, they plunge into wild guessing. Research at Carnegie Mellon has found the same. When children start guessing they don’t stop. Students would answer these same types of tricky questions 15 or more times, getting them wrong over and over again. There were buttons for help and feedback offered along the way, but the children did not opt into this alternate learning strategy.
Rather than combat this wild guessing, I built the learning software to account for guessing. When the student got something wrong, like IV = 6, they would be asked that type of question until they got it right. Eventually — and that’s a long eventually — they would start to see the connection that IV and 4 are related, but it was through many, many exposures (15+).
After this work, I proposed to my client we could make a different type of lesson, one that specifically tried to teach children that wildly guessing was not a good strategy. We could create a program that showed children that understanding how to solve a problem is more important than getting the answer right.
Unfortunately, teaching children how not to guess was not a high-priority item for my client, who was under pressure to make sure they taught all of the standards. Executive functioning or emotional regulation were nice to have but not essential to the platform. So I was left with this idea: What if online lessons didn’t just teach math but spent more time on how to learn math? Below I share how I might go about tackling this prickly issue of wild guessing.
If someone were to make a first prototype on teaching “How to Go Beyond Guessing,” here are some lessons I learned:
1. Children avoid confusion but brush off failure
I gave math tests to middle and high school students to see how I could get them to persevere. They wore emotion sensors (MOXO) while they were taking the tests. The sensor data (anxiety) would go higher and higher as problems appeared to which they did not know the answers. Eventually, the students would give up and just start guessing without thinking, and their skin conductance would substantially decrease (calm down).
While failure is acceptable, thinking hard about something you don’t fully know is hard and anxiety-provoking. Building tools that can help children stay, if not flourish, in the unknown could allow them to wrestle with hard ideas.
How might we encourage children to embrace and respond to confusion?
2. Students are not given a chance to self-reflect on their current understanding.
One of my clients, the LEGO Group, wanted to make the most difficult LEGO sets (Technic) easier for children to build. I watched children try to build and eventually give up on these difficult models. A main problem was that specific instructions for “Check to make sure your model works like it should” was not included in the instructions. As adults, we would naturally check as we build, but children have not yet developed the skill to naturally reflect. Consequentially, by the time they realized there was a mistake, it was too late, and they would have to start over. We showed children different options for improving the sets, and the top requested feature was a page that instructs them to confirm the Lego model was working as intended.
In digital tools, children are also not prompted to ask “Do I know this?” My gut reaction to this problem is to add moments of self-reflection into the learning process. I would want to prototype the best way to communicate this process to children. They’re so used to adults telling them what they know or don’t know, so creative solutions would be needed.
How might we create moments of reflection in digital learning?
3. Admitting you don’t know is failure.
Our education culture has taught kids to fight hard to show they “knew it” and hide their mistakes (read John Holt’s “How Children Fail” for a profound dive into this topic). Consequentially, most children I interview work hard to create the veneer that they know everything: “This class is boring;” “I need harder lessons;” “I already knew all that;” “That questions was unfair.” I have yet to interview a child who tells me, “You know, I didn’t know what was going on in that lesson,” despite most children failing consistently.
For children, saying you don’t know something is like saying you’re a baby, that you can’t handle it, and that you need help. Their goal is the opposite, to show that they are more adult than you give them credit for. How can we make acknowledging ignorance seem more adult-like?
How might we make acknowledging ignorance feel like a strength?
4. Children don’t understand chance, so punishing guessing won’t work.
In our Roman numeral study, the program had strong punishment for getting a problem wrong: you’d have to start over. Even with that rule in place, children consistently resorted to guessing. They are like eternal optimists who don’t understand chance: “If I guess it, I’ll probably get it right!” So modifying rewards or emphasizing the importance of “getting it right” will likely not lead to children taking a step back. In Prodigy Math where children need to get answers right to defeat the bad guy, I seldom saw a child spend time on a problem — the importance just did not register.
How might we make guessing a non-option instead of a bad option?
5. Children will view themselves as capable of everything.
I offered children the chance to choose a difficulty. A large majority (especially boys) always chose the hardest difficulty each round — bring it on! Even after they got eight problems wrong in a row, they would just stick with the hardest difficulty. I’d nudge them to the normal difficulty, and they would take offense. This over confidence is a double-edged sword. On the one hand, they are up for a challenge. On the other hand, they struggle to self-reflect that they are in over their heads.
How might we help children identify when they are struggling?
With these foundational insights in place, I want to make some disclaimers. I don’t know what will help children move away from guessing — I have not prototyped any solutions (yet). I don’t know if this defaulting to guessing applies for under third grade (where goals become less important) or after fifth grade (where failure becomes more influential). I also don’t know how much of an issue guessing is for children from wealthy families. In another study, I observed some children from affluent families start crying if they did not know an answer — their environment had instilled a strong fear of failure in them.
In summary, there’s more unknown than known. But I do believe a program that could train children to reflect on what they know and take the correspondingly correct answer would fundamentally shift the multiple-choice landscape digital education takes today. And I want to be part of that shift.
