😲 Quantifying Surprise — A Data Scientist’s Intro To Information Theory — Part 1/5: Foundations

Gain intuition into Information Theory and master its applications in Machine Learning and Data Analysis. Python code provided. 🐍

Surprise! Generated using Gemini.

During the telecommunication boom, Claude Shannon, in his seminal 1948 paper¹, posed a question that would revolutionise technology:

How can we quantify communication?

Shannon’s findings remain fundamental to expressing information quantification, storage, and communication. These insights made major contributions to the creation of technologies ranging from signal processing, data compression (e.g., Zip files and compact discs) to the Internet and artificial intelligence. More broadly, his work has significantly impacted diverse fields such as neurobiology, statistical physics and computer science (e.g, cybersecurity, cloud computing, and machine learning).

[Shannon’s paper is the]

Magna Carta of the Information Age

— Scientific American

This is the first article in a series that explores information quantification — an essential tool for data…