The Meaning Behind Logistic Classification, from Physics

In classification problems, why do we use the logistic and softmax functions? Thermal physics may have an answer.

Logistic regression is ubiquitous, but what’s the intuition behind why it works? (Image by Author)

Logistic regression is perhaps the most popular and well-known machine learning model. It solves the binary classification problem — for predicting whether a data point belongs to a category.

The key ingredient is the the logistic function, where an input is converted into a probability, written in the form:

But why does the exponential make an appearance? What is the intuition behind it, beyond just converting a real number into a probability?

It turns out, thermal physics has an answer. But before digging into insights from physics, let’s understand the mathematics first.

Mathematical Quandary

Before tackling the “why” behind the logistic function, let’s understand its properties first.

Time to get the math out of the way! (Photo by Michal Matlon on Unsplash)

It is helpful to understand the logistic function in a more generalized form — one that is applicable for multiple categories — so that the…