Unless otherwise noted, all diagrams and images are by the author (images generated using MidJourney with a paid license by the author)

Understanding Latent Dirichlet Allocation (LDA) — A Data Scientist’s Guide (Part 2)

LDA Convergence Explained with a Dog Pedigree Model

“What if my a priori understanding of dog breed group distribution is inaccurate? Is my LDA model doomed?”

My wife asked.

Welcome back to part 2 of the series, where I share my journey of explaining LDA to my wife. In the previous blog post, we discussed how LDA works and how it can be understood as a dog pedigree model.

This time around, let's dive into the iterative fitting process of LDA!

Part 1 (link):

• How does LDA work?
• How to explain LDA to a non-technical person?

Part 2 (We are here now!):

• How does LDA improve iteratively?
• How does LDA converge?
• Bonus: Get your LDA cheatsheet here!

Part 3:

• When to use LDA & when not to?
• How can we use it in Python?
• What are the alternatives & variants to LDAs (excluding LLMs)?