Time to put an end to BERTology (or, ML/DL is not even relevant to NLU)

Walid Saba, PhD
Oct 18, 2020 · 12 min read

Background

There are 3 technical (read: theoretical, scientific) reasons why the data-driven/quantitative/statistical/machine learning approaches (that I will collectively refer to as BERTology) are utterly hopeless and futile efforts, at least when it comes to language understanding. This is a big claim, I understand, especially given…