Homepage
Open in app
Sign in
Get started
Autonomous Agents — #AI
Notes on Artificial Intelligence and Machine Learning
Follow
Understanding Math Behind Chinchilla Laws
Understanding Math Behind Chinchilla Laws
Optimizing LLM Performance through Compute-Efficiency
Freedom Preetham
Jun 14
Part 2 — An Advanced Thesis: Learning from Joint Distributions
Part 2 — An Advanced Thesis: Learning from Joint Distributions
In continuation on the discussions from Part 1, where I surmised that we truly do not need big data for training today’s model, I present…
Freedom Preetham
Jun 13
Part 1 — How Many Cat Pictures? Does AI Really Need Big Data?
Part 1 — How Many Cat Pictures? Does AI Really Need Big Data?
In the realm of artificial intelligence, there has been a longstanding belief that big data is essential for effective learning and model…
Freedom Preetham
Jun 13
Advanced Attention Mechanisms for Long Sequence Transformers
Advanced Attention Mechanisms for Long Sequence Transformers
In processing long sequences, Transformers face challenges such as attention dilution and increased noise. As the sequence length grows…
Freedom Preetham
May 28
Math Behind Positional Embeddings in Transformer Models
Math Behind Positional Embeddings in Transformer Models
Positional embeddings are a fundamental component in transformer models, providing critical positional information to the model. This blog…
Freedom Preetham
May 28
Comprehensive Breakdown of Selective Structured State Space Model — Mamba (S5).
Comprehensive Breakdown of Selective Structured State Space Model — Mamba (S5).
Foundation models often use the Transformer architecture, which faces inefficiencies with long sequences. Mamba AI improves this by…
Freedom Preetham
May 3
Part 3 — Randomized Algo and Spectral Decomposition for High-Dimensional Fractional Laplacians
Part 3 — Randomized Algo and Spectral Decomposition for High-Dimensional Fractional Laplacians
In the ambit of mathematical and computational sciences, solving ultra high-dimensional partial differential equations (PDEs) has always…
Freedom Preetham
Apr 23
RAG Does Not Reduce Hallucinations in LLMs — Math Deep Dive
RAG Does Not Reduce Hallucinations in LLMs — Math Deep Dive
Too much marketing cool-aid has been spent on stating that RAG avoids or reduces hallucinations in LLMs. This is not true at all.
Freedom Preetham
Feb 16
A Math Deep Dive on Gating in Neural Architectures
A Math Deep Dive on Gating in Neural Architectures
Freedom Preetham
Feb 7
Unpredictable Latent Errors in AI can be Catastrophic — Mathematical Explanation
Unpredictable Latent Errors in AI can be Catastrophic — Mathematical Explanation
Is the future of Artificial Intelligence dystopian or utopian? This question has always been a subject of debate, with major camps…
Freedom Preetham
Dec 17, 2023
Enhancing LLM’s Reasoning Through JEPA— A Comprehensive Mathematical Deep Dive
Enhancing LLM’s Reasoning Through JEPA— A Comprehensive Mathematical Deep Dive
Freedom Preetham
Dec 15, 2023
LLMs, Transformers, GPTs — Here is One Ring to Rule Them All
LLMs, Transformers, GPTs — Here is One Ring to Rule Them All
If LLMs were a RR Tolkien’s narration, then this blog is one ring to rule them all :) I have compiled a list of in-depth blogs I have…
Freedom Preetham
Dec 6, 2023
Part 9 — Memory-Augmented Transformer Networks: A Mathematical Insight
Part 9 — Memory-Augmented Transformer Networks: A Mathematical Insight
In the realm of sequential data processing, traditional Transformer architectures excel in handling short-term dependencies but falter in…
Freedom Preetham
Dec 6, 2023
Part 8 — Mathematical Explanation of Why It’s Hard for LLMs to Memorize
Part 8 — Mathematical Explanation of Why It’s Hard for LLMs to Memorize
From the beginning of this blog series we have seen how the development of transformer models like GPT-4 represents a paradigm shift in…
Freedom Preetham
Dec 5, 2023
Part 7 — Strategies for Enhancing LLM Safety: Mathematical and Ethical Frameworks
Part 7 — Strategies for Enhancing LLM Safety: Mathematical and Ethical Frameworks
The quest to enhance the safety of Large Language Models (LLMs) is a sophisticated interplay of technical innovation, ethical…
Freedom Preetham
Dec 2, 2023
Part 6 — Adversarial Attacks on LLM. A Mathematical and Strategic Analysis
Part 6 — Adversarial Attacks on LLM. A Mathematical and Strategic Analysis
Adversarial attacks on Large Language Models (LLMs) represent a sophisticated area of concern in AI safety, requiring an intricate blend…
Freedom Preetham
Dec 1, 2023
Deep Dive into Rank Collapse in LLMs
Deep Dive into Rank Collapse in LLMs
Transformers, central to advancements in machine learning, leverage the self-attention mechanism for tasks across various domains…
Freedom Preetham
Nov 29, 2023
Simplifying Transformer Blocks — A Detailed Mathematical Explanation
Simplifying Transformer Blocks — A Detailed Mathematical Explanation
Large language models (LLMs) can expand their capabilities through various scaling strategies. The more straightforward approach involves…
Freedom Preetham
Nov 28, 2023
Part 5 — In-Depth Analysis of Red Teaming in LLMs: A Mathematical and Empirical Approach
Part 5 — In-Depth Analysis of Red Teaming in LLMs: A Mathematical and Empirical Approach
The field of Large Language Models (LLMs) is rapidly advancing, necessitating robust red teaming strategies to ensure their safety and…
Freedom Preetham
Nov 26, 2023
Part 4 — Enhancing Safety in LLMs: A Rigorous Examination of Jailbreaking
Part 4 — Enhancing Safety in LLMs: A Rigorous Examination of Jailbreaking
The concept of jailbreaking Large Language Models (LLMs) such as GPT-4 represents a formidable challenge within the domain of artificial…
Freedom Preetham
Nov 26, 2023
Q* Algorithm is NOT Q-Learning
Q* Algorithm is NOT Q-Learning
Freedom Preetham
Nov 25, 2023
Part 3 — Mathematically Assessing Closed-LLMs for Generalization
Part 3 — Mathematically Assessing Closed-LLMs for Generalization
In the realm of closed Large Language Models (LLMs) such as OpenAI or Anthropic, the true test of intelligence and versatility lies in…
Freedom Preetham
Nov 24, 2023
Part 2 — LLMs Beyond Memorization
Part 2 — LLMs Beyond Memorization
Large Language Models (LLMs) represent a significant leap in artificial intelligence, transcending the notion of mere memorization models…
Freedom Preetham
Nov 23, 2023
Part 1 — Are LLMs Just a Memory Trick?
Part 1 — Are LLMs Just a Memory Trick?
It has become fashionable for critics to accuse Large Language Models (LLMs) as mere memorization devices, arguing that their extensive…
Freedom Preetham
Nov 23, 2023
Part 3 — AGI; An Advanced Mathematical Perspective
Part 3 — AGI; An Advanced Mathematical Perspective
Following up on Part 2, which explored the nature of AGI and its comparison with human cognition, this blog delves into the ‘how’ of AGI…
Freedom Preetham
Nov 23, 2023
About Autonomous Agents
Latest Stories
Archive
About Medium
Terms
Privacy
Teams