We Are Just Scaling AI And Not Coming With Novel Ideas
I know many people will have a hard disagreement with the title itself and would start calling me out even before reading the article. I know we are coming up with a lot of small things, but rarely do we see a truly new idea. The problem with AI is that most people entering the field do not know what was invented in the 90s, and many things we see now are just scaled-up or slightly modified versions of the same old things.
For so many years even I believed in many wrong things. Everything is moving at such a rapid pace with new models and strategies coming every few weeks, it is becoming quite tough to keep track of everything. But if you look closely you will see only a little has changed except the scale of compute and data. Somehow we are still working with decade-old ideas. One example I like to give about not coming up with new ideas is the exorbitant use of XgBoost or other tree-based models, most financial models are still running on these, not on deep learning-based models.
So, without further ado, let’s take a deep dive into why it is so tough to come up with novel ideas.
Table Of Content
- Scarcity Of New Ideas
- We Don’t Know What To Pursue Anymore
- The Scaling Hypothesis
- Misconceptions: Rehashing Of Old Ideas
