Bayesian Inference for the Mean of a Gaussian Distribution with Known Variance
Assuming that the variance is known, consider the problem of estimating the mean μ from a given set of N observations x=x1,…,xN. The likelihood function, which is the probability that observed data occurs when μ is given, can be regarded as a function of μ, and is as follows.
Looking at this likelihood function, it takes the form of an exponent in quadratic form for μ. Therefore, if we choose a Gaussian distribution for the posterior distribution p(μ), it will be the conjugate prior distribution of this likelihood function. This is because the posterior distribution when using this is in the form of two exponents of the quadratic function for μ, which is also a Gaussian distribution. Therefore, we take the prior distribution as follows.
The posterior distribution is as follows.
By a transformation that completes the square of the exponential part, the posterior distribution can be expressed as follows
The following equation holds for μN and σN2.
μML is the maximum likelihood estimated solution of μ and is the sample mean.
First, the mean of the posterior distribution in Eq(5) is the value taken between the mean μ0 of the prior distribution and the maximum likelihood estimated solution μML. If the number of observed data points is N=0, it is the mean of the prior distribution. On the other hand, for N→∞, the mean of the posterior distribution is the maximum likelihood estimated solution. Let us consider the variance in the same way. If the number of data becomes N→∞, the variance σN2 approaches zero and the posterior distribution becomes an infinitely sharp density around the maximum likelihood estimate.
You can draw the Gaussian distribution with known variance by this code. The data points were generated with a Gaussian distribution with mean 0.75 and variance 0.1, and a prior distribution with mean 0 was chosen. For both the posterior distribution and the likelihood function, the variance was set to its true value.
