Member-only story
10 Years Of AI Progress & What’s Next?
The field of AI as we see it today was vastly different 10 years ago. Most of the progress in AI was limited to a few research labs and products. Things have drastically changed since then, and now everyone is aware of the presence of AI in their lives.
A few major techniques hadn’t been invented, and thus, we were struggling really hard to train bigger models. Bigger models of 2014 are minuscule for 2024.
Topics Covered:
- Why 2014 Was Such An Important Year?
- What They Got Right About AI?
- What’s Next?
- OpenAI’s New o3 Just changed The AI Game
- What Nobody Talks About
Why 2014 Was Such An Important Year?
Let’s take a little history tour and understand what was happening in AI around 10 years ago.
Before 2014, VGG 16, Alexnet, or some of their variants were the reigning champions. Training a 15–20 layer network was close to impossible. A few key techniques were yet to be invented. For training, even a 16-layer network they had to freeze most layers and train only 3–4 layers at a time and successively train the full network. A lot of weight initialization methods were missing and many other training techniques that we take…