AIGuys Digest | Dec 2024
š Welcome to the AIGuys Digest Newsletter, where we cover State-of-the-Art AI breakthroughs and all the major AI newsš. Donāt forget to check my new book on AI, it covers a lot of AI optimizations and hands-on code:
Ultimate Neural Network Programming with Python
š Inside this Issue:
- š¤ Latest Breakthroughs: This month itās all about whatās LLMs Reasoning capabilities, VLMs, LLMs for time series, and whether OpenAIās new o1 is worth $200.
- š AI Monthly News: Discover how these stories revolutionize industries and impact everyday life.
- š Editorās Special: This covers the interesting talks, lectures, and articles we came across recently.
Letās embark on this journey of discovery together! šš¤š
Follow me on Twitter and LinkedIn at RealAIGuys and AIGuysEditor.
Latest Breakthroughs
The age-old question regarding LLMs: Do large language models (LLMs) solve reasoning tasks by learning robust generalizable algorithms, or do they memorize training data?
To investigate this question, recently a paper used arithmetic reasoning as a representative task. Using causal analysis, they identified a subset of the model (a circuit) that explains most of the modelās behavior for basic arithmetic logic and examined its functionality. Now we finally have the answer to how LLMs solve maths and reasoning tasks.
LLMs Canāt Learn Maths & Reasoning, Finally Proved!
If you take a look at the industrial data you would see that in many places we are still using classical Machine Learning algorithms. There is a good reason to use classical ML and AI algorithms over new Deep learning-based methods in industrial settings; the amount and quality of proprietary data. Most banks still use some variant of XGBoost for tabular data. We have seen crazy progress in Deep Learning models, but there are still many fields where growth has been barely linear. One such field where we have seen limited growth is time series forecasting. But now things have changed and we finally have some transformer-based models for Time series prediction.
LLMs For Time Series Forecasting !!!
The real world is not just language, most of our intelligence is not even part of language, but more of in visual positioning of ourselves in the world. lately, we have seen that LLMs are not improving much with pretraining, there are some clever techniques like what OpenAIās o1 implemented, but the base modelsā performance has already plateaued. But why? Simply, we have fed almost the entire text data to LLMs, they donāt have much to learn from text. So, the next logical step is to feed these big foundational models the visual data. And thatās exactly what we are going to talk about.
Visual Reasoning for LLMs (VLMs)
OpenAI has released the new o1 and o1-pro, and they are making a lot of noise just like always, but this time, the reason is something else. It is the $200 price tag that is making the most noise instead of how good the model really is. A $200/month is not a small amount by any means, this is a significant salary for a lot of people in low-income countries.
If the path to AGI goes through the pocket of the rich, Iām positive that itāll create an even bigger difference between the rich and the poor, instead of solving the world problems of inequality and climate change. So, letās take a deep dive and try to understand whatās new in this and is it even worth paying $200 a month for this newly released model.
Is OpenAIās New o1-pro Worth $200/month?
AI Monthly News
Research Advancements:
OpenAIās Reasoning Models: OpenAI introduced its latest reasoning models, o3 and o3-mini, which excel in complex problem-solving tasks, including coding, mathematics, and scientific challenges. These models represent a substantial leap in AI capabilities, particularly in logical reasoning and analytical tasks.
DeepSeekās AI Model: Chinese AI firm DeepSeek, a subsidiary of High-Flyer, launched DeepSeek-V3, a large language model with 671 billion parameters. Developed with optimized resource utilization, it matches or surpasses models like GPT-4o and Claude 3.5 Sonnet, highlighting Chinaās rapid progress in AI research despite hardware constraints.
Industry Developments:
Nvidiaās Acquisition of Run:ai: Nvidia completed its $700 million acquisition of Israeli AI firm Run:ai after receiving antitrust clearance from the European Commission. Run:ai plans to open-source its software to extend its availability beyond Nvidia GPUs, aiming to support the broader AI ecosystem.
Salesforceās Agentforce 2.0: Salesforce unveiled Agentforce 2.0, an advanced AI agent program enhancing reasoning, integration, and customization features. The full release is expected in February 2025, with positive reactions from Wall Street analysts.
OpenAIās For-Profit Transition: OpenAI announced plans to restructure into a for-profit public benefit corporation to attract more investment, acknowledging the need for substantial capital in pursuing artificial general intelligence. This move has sparked discussions about the implications for AI development and commercialization.
Geopolitical Movements:
Russia-China AI Collaboration: Russian President Vladimir Putin directed the government and Sberbank to collaborate with China in AI research and development, aiming to bolster Russiaās position in AI amid Western sanctions limiting access to crucial technology.
Regulatory Discussions:
Call for AI Regulation in the UK: The UK AI industry body, UKAI, advocated for the establishment of a dedicated AI regulator to provide oversight similar to the Financial Conduct Authority, emphasizing the need for unified and efficient regulation amid growing concerns about AI technologies.
Editorās Special
- Byte Latent Transformer: Patches Scale Better Than Tokens (Paper Explained) Click here
- Inside OpenAIās Turbulent Year Click here
- The Potential for AI in Science and Mathematics ā Terence Tao Click here
š¤ Join the Conversation: Your thoughts and insights are valuable to us. Share your perspectives, and letās build a community where knowledge and ideas flow freely. Follow us on Twitter and LinkedIn at RealAIGuys and AIGuysEditor.
Thank you for being part of the AIGuys community. Together, weāre not just observing the AI revolution; weāre part of it. Until next time, keep pushing the boundaries of whatās possible. šš
Your AIGuys Digest Team