AIGuys Digest | May 2025
🌟 Welcome to the AIGuys Digest Newsletter, where we cover State-of-the-Art AI breakthroughs and all the major AI news🚀. Don’t forget to check out my new book on AI. It covers a lot of AI optimizations and hands-on code:
Ultimate Neural Network Programming with Python
🔍 Inside this Issue:
- 🤖 Latest Breakthroughs: This month, it is all about LLM Leaderboard Illusion, (How) Do LLMs Reason and Plan?, and Google I/O In A Nutshell.
- 🌐 AI Monthly News: Discover how these stories revolutionize industries and impact everyday life. Google I/O 2025, New models from OpenAI and Anthropic, and the UAE government to provide city-wide access to GPT.
- 📚 Editor’s Special: This covers the interesting talks, lectures, and articles we came across recently.
Let’s embark on this journey of discovery together! 🚀🤖🌟
Follow me on Twitter and LinkedIn at RealAIGuys and AIGuysEditor.
Latest Breakthroughs
I stopped believing in the AI benchmarks a long time ago. I have read multiple papers that claimed things that were not true. And the best example of this is the “Sparks of AGI paper”. I kept wondering, how come we are improving on so many benchmarks so fast? And yet, in my personal usage, my productivity barely went up. So, finally, we have a paper that shows exactly what scam is going on in the LLM evaluation space.
I’m sure with each new model, things do get better, but how much is what we have to doubt, especially after seeing the leaderboard illusion paper?
Access to Chatbot Arena data yields substantial benefits; even limited additional data can result in relative performance gains of up to 112% on the arena distribution.
LLM Leaderboard Illusion
Large Reasoning Models (LRM) have been all the rage for the last few months. The age of LLMs is over, now it is LRMs’ time. Be it Gemini 2.5, Claude thinking mode, or GPT o-series models, all of them have moved towards reasoning models.
Fundamentally, all of them are still LLMs only, but suddenly, these models feel much better and smarter. Their ability to reason and plan seems to increase manyfold. Every week we are crushing different benchmarks, but as responsible researchers and AI enthusiasts, we must ask, how much of this development is real and how much of it is just a hype, a marketing gimmick.
(How) Do LLMs Reason and Plan?
Google is not new to innovations, but the latest Google I/O was something else. This time, Google outdid itself and every other big lab. With so many crazy AI innovations, Google is truly flying high and trying to recapture the AI market it lost to other key players in the market. So, let’s see what is crazy about the latest Google I/O and what different types of AI and applications are being released in the coming weeks and months.
With Google’s Veo3 (text-2-video) breaking the internet, some crazy other models are making quite some waves in the AI space.
Google I/O In A Nutshell
AI Monthly News
Google DeepMind’s AlphaEvolve & Gemini 2.5 Pro / Veo 3 / A.I. Mode
On May 14, Google DeepMind introduced AlphaEvolve, a Gemini-powered evolutionary coding agent designed to autonomously generate and refine algorithms using LLM-guided mutations and selection.
Mechanism
- It accepts an evaluation function (e.g., performance metrics) and an initial algorithm. At each iteration, AlphaEvolve uses the Gemini LLM to propose code variants, then evaluates and selects top-performing candidates (Turn0search19).
- Unlike domain-specific predecessors (AlphaFold, AlphaTensor), AlphaEvolve targets general-purpose algorithmic discovery, enabling it to operate across a broad spectrum of scientific and engineering tasks.
Gemini 2.5 Pro, Veo 3 & A.I. Mode (May 20, 2025)
On May 20 during Google I/O 2025, Google announced a suite of updates across Gemini, video generation, and search:
Gemini 2.5 Pro
- Positioned at the top of the LMArena benchmark, surpassing competitors in reasoning, coding, and long-context comprehension — poised to outperform nearly all publicly known models.
- Enhancements included improved multimodal capabilities that allow seamless transitions between text, code, and visual inputs.
Veo 3
- Unveiled as a state-of-the-art video-generation model capable of synthesizing high-quality videos from text or image prompts, complete with synchronized audio (dialogue, sound effects).
- Google expanded Veo 3’s rollout to 71 additional countries on May 24, underscoring its ambition to democratize video creation tools.
- Netizens weighed in on Veo 3’s potential to revolutionize filmmaking, yet expressed concerns over job displacement for creative professionals.
A.I. Mode for Search
- Integrated Gemini LLM into Google Search as “A.I. Mode,” delivering agentic search experiences that can handle multi-step tasks (e.g., booking travel, comparing products) by reasoning over search results and executing actions.
- This represents a strategic pivot from traditional query–result paradigms toward interactive, LLM-based search agents.
Gemini 2.5 Pro Upgrades
- In addition to new benchmarks, Google highlighted enhancements in reliability, latency reduction, and developer accessibility — Gemini 2.5 Pro being accessible via Google Cloud’s GenAI API.
- Developers can now fine-tune Gemini 2.5 Pro on proprietary data, accelerating domain-specific applications.
Updates on Claude and ChatGPT
Both Anthropic and Claude have released their latest model in the last one month.
GPT-4.1: A specialized model built for coding tasks and precise instruction-following, outperforming GPT-4o on web development and debugging tasks while being faster than its “o-series” reasoning cousin.
GPT-4.1 mini: Replaced GPT-4o mini for all users, offering an improved fallback model that balances speed and intelligence for casual and free-tier users
Anthropic also launched Claude Opus 4 and Claude Sonnet 4, marking a pivotal step in AI coding agents and long-horizon reasoning.
- Coding Proficiency: Claude Opus 4 achieved a 72.5% score on the SWE-bench benchmark — dramatically outperforming OpenAI’s GPT-4.1 at 54.6% in software engineering tasks (e.g., code refactoring, architecture planning). It excelled at multi-step coding for hours at a time, autonomously engaging in long-running tasks for up to seven hours during Rakuten tests.
- Hybrid Reasoning Modes: Both Opus 4 and Sonnet 4 support “fast mode” (near-instant responses) and “deep thought mode” (extended chain-of-thought summaries), enabling richer reasoning for complex problem domains.
- Agentic Use Cases: Claude Opus 4 powers advanced AI agents capable of orchestrating multi-channel marketing campaigns or performing strategic research across large datasets such as patent databases and academic papers.
Read More: Click here
UAE–OpenAI Partnership: Free ChatGPT Plus in Dubai (Late May 2025)
Recently, the UAE government announced that Dubai residents would soon receive complimentary ChatGPT Plus subscriptions, tied to the broader Stargate UAE project:
Details & Infrastructure
- Partnership includes establishing a one‐gigawatt AI data center in Abu Dhabi; phase one (200 megawatts) expected online by mid–2026 (Turn4news12; Turn4news13).
- This initiative positions the UAE as the first nation to grant free access to a premium AI chatbot service at scale, aiming to bolster its status as a global AI hub (Turn4news13).
Strategic Implications
- By providing free ChatGPT Plus, the government seeks to accelerate AI literacy, support digital transformation, and attract tech investment.
- Analysts note potential network effects: broader population access may yield increased data for localized model fine-tuning (Turn4news12).
News Article: Click here
Editor’s Special
- Yann LeCun: We Won’t Reach AGI By Scaling Up LLMS: Click here
- Accelerating Scientific Discovery with AI — lecture by Sir Demis Hassabis: Click here
- (How) Do LLMs Reason/Plan? (Talk given at Microsoft Research; 4/11/25): Click here
🤝 Join the Conversation: Your thoughts and insights are valuable to us. Share your perspectives, and let’s build a community where knowledge and ideas flow freely. Follow us on Twitter and LinkedIn at RealAIGuys and AIGuysEditor.
Thank you for being part of the AIGuys community. Together, we’re not just observing the AI revolution; we’re part of it. Until next time, keep pushing the boundaries of what’s possible. 🚀🌟
Your AIGuys Digest Team
