A comprehensive guide on inferencing in LLMs — Part 2

Same Core, Different Gears: How GPT, LLaMA, Mistral, Claude, and Gemini Trade Scale, Speed, and Context

We’ve done the groundwork — Part 1 covered the mechanics: transformer attention, softmax and temperature, autoregressive decoding, and why prefill is compute-bound while decode is memory-bound. Here, we compare architectures through that lens: what GPT, LLaMA, Mistral, Claude, and Gemini change in attention (GQA/MQA/SWA), normalization, activations, and context handling — and how those choices translate into time-to-first-token, tokens/sec, and VRAM footprint. If Part 1 taught you how a token appears, Part 2 teaches you why some models make it appear faster, longer, or cheaper.

Architecture-Specific Inference Details

Not all LLMs are identical under the hood. Different model families (GPT, LLaMA, etc.) have architectural variations that can affect inference efficiency, capacity, and capabilities. Here we outline a few notable differences in popular architectures and how they relate to inference:

✍️ Author’s Note

If you are enjoying this piece, follow me for more deep dives into latest technological trends.

👉 Liked the article? Smash those claps (50 if you’re feeling generous!)
☕ Appreciate the effort? Support my work on Buy Me A Coffee link
🔗 Let’s connect on LinkedIn — I love meeting curious…