AI Is A Prodigy. Still Needs To Improve On Math Word Problems
Do NLP Models Cheat at Math Word Problems? Microsoft Research Says Even SOTA Models Rely on Shallow Heuristics
I am very intrigued by this paper.
Here is a child prodigy.
Can draw the most beautiful images at will. Can beat the best at games. Can find patterns and people in a blink of an eye. Can translate languages at the speed of speech. Is great at arithmetic and matrix multiplication.
But……
But cannot solve Math Word Problems designed for a 9-year olds?
Why would that happen?
I consider solving Math Word Problems a skill that elevates the problem-solving competency of a child. It integrates math, language, and lego-like integrative skills.
It is not everyone’s cup of warm beverage.
The paper concludes that the current language models cheat — meaning, they predict the answer by replying to simple heuristics that associate the arithmetic operators to certain words.
Well, I would argue there is nothing wrong with that approach. I have done that too for advanced concepts in certain subjects — but I have failed more than I have succeeded — because it only gets you so far.
There is a certain grounding that comes in nutting it out. The language models will get there soon enough. But for now, I find it intriguing.
