A Subtle Discourse on Linear Discriminant Analysis
One of the several approaches to the classification problem involves dividing the input space into subregions, pertaining to each of the possible classes. The boundaries dividing these regions can be modeled as smooth or rough, depending on the prediction function used for classification.
The function which is responsible for modeling these boundaries is called a discriminant function in LDA.
The ideation of LDA can be attributed to the Bayes Theorem for Classification, which is used to predict the probability of an observation belonging to a class based on prior probability and the density function of the observation.
Where f(x) is the density function and π_k is the prior probability of class k
The density functions are generally assumed to be standard distributions, such as a Gaussian.
where μ_k and σ_k are the mean and variance for the kth class.
By plugging the second equation in the first and taking the log of P_r, we get the discriminant function
An LDA model classifies an observation x to a class k, such that the value corresponding to δ_k is the largest across all classes.
