Are We Hitting The Scaling Limits Of AI?

This past week, we heard a lot of noise about LLMs hitting a wall. Is this true? Is a new AI winter upon us? Or is it just a hiatus? The matter of fact is that people need to truly know what is happening with scaling laws. It is not hard to find AI experts having completely opposing views on the future of AI. This reminds me of Kenneth Stanley’s book, “Why Greatness Can’t Be Planned” which primarily argues that no one knows what it takes to make a breakthrough in a certain field and that’s exactly the same thing happening lately with the AI.

Last week we saw many big labs and researchers showcasing their disappointment with diminishing returns on AI as well as others hyping it even more.

So, let’s see what is happening actually and how are things looking for the upcoming years.

Topics Covered

• The Scaling Hypothesis
• Other Side Of The Scaling Hypothesis
• What’s Happening In Reality?
• Economics Of AI
• My Take On AGI And Scaling

Photo by Google DeepMind on Unsplash

The Scaling Hypothesis

So, what is the Scaling hypothesis?

The scaling hypothesis in artificial intelligence claims that a model’s cognitive ability scales with increased compute…