An introduction to sampling from distributions

In this post, we will discuss how to sample data vectors from a distribution.

You can also find a video introduction sampling here and a video on inverse sampling here

Inverse Sampling

Consider the case of sampling data points from a normal distribution centred around 0 with a standard deviation of 1.

Image for post
Image for post
A normal distribution with mean 0 and standard deviation 1

If you are familiar with libraries such as numpy , then you will recall that this can be trivially done by using functions like numpy.random.normal() . But for now, let us assume we do not have access to such a library function. How might one go about achieving this task?

Inversion Sampling

The inversion sampling method is based on the following property :

Image for post
Image for post

To prove this theorem, we start out by writing down the CDF(Cumulative Density Function) corresponding to the inverse function.

Image for post
Image for post

Given the theorem, we now have an algorithm that generates samples from a given pdf(probability density function) f.

Image for post
Image for post

In the next post, we will look into some approximate method of sampling which does not require computing the inverse CDF.

Reference

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store