How To Create Awesome Noise That Is Actually Real
if you’ve ever looked at a digital image, you couldn’t fail to notice this grainy agitated noise differentiating two adjacent pixels viewing on practically the same object. Why on God’s green earth would two pixels capturing clear sky, for example, won’t have the same gray level value?
Well, of course, there could be a lot of reasons — It could be the camera sensor, the optics of the camera, the texture of the captured object or it could just the fact that I am a lousy photographer? Well no, I am not going to try and explain all the possible effects, I am not crazy. Instead, this post I about one particular noise.
I am talking about Shot noise or photon noise.
So, why should I care?
Shot noise is here to stay. It is associated with the particle nature of light and therefore won’t go anywhere (you need light to see your images don’t you?).
Imagine that you are a doorman and you want to know how many people go through the door within one hour. Let’s say that in one hour, you counted 65 people that went through the door. I dare you to count again!
Whenever counting independent events you are going to have random fluctuation in your measurement. Meaning that in the next hour of count you will probably get a slightly different value than 65. The amazing thing is that this would occur even if the rate of people arriving would remain the same.
So now, let’s replace the people with tiny spheres, called photons (for simplicity let’s assume that they represent light) and the door with camera’s aperture and we got shot noise in image sensors.
How do you count light?
Image sensors measure scene irradiance by counting the number of discrete photons incident on the sensor over a given time interval. In digital sensors, the photoelectric effect is used to convert photons into electrons, whereas film-based sensors rely on photo-sensitive chemical reactions. In both cases, the independence of random individual photon arrivals leads to shot noise, a signal-dependent form of uncertainty that is a property of the underlying signal itself.
Signal shot noise is fundamentally connected to the way photons spatially and temporally arrive on an image sensor. For example, the image below shows a Monte Carlo simulation where 200 photons are randomly interacting with a 20 × 20 pixel region. As can be seen, the number of photon interactions varies from zero to four interactions per pixel. The standard deviation for the number of interactions per pixel is called photon shot noise.
This difference within photons distribution is mainly because of random statistical fluctuations. i.e. when collecting photons from an unvarying source over a set amount of time — say ten random 30 seconds intervals over the course of twenty minutes — it is highly unlikely that you will collect 20 photons every single time. Instead, the first interval may give you 20 photons, the next interval 23 photons, the next interval 18 photons, and so on.
So, how do these fluctuations of photons arriving at the image sensor affect our images? Noise can distort the visual detail of your photo, making it something that photographers try to avoid. A simulated set of images is shown in the image below where the average number of particles, or photons, per pixel is specified. This showed that for low numbers of photons noise dominates but as the number of photons increases image structure become more visible.
With a solid understanding of the cause of shot noise let us now examine the relations between photons and image gray level values.
Full-Well Capacity and Signal-To-Noise (SNR) In image sensors
With machine vision applications, some of the most important specifications beyond resolution and frame speed to determine whether the camera meet the measurement requirements, are full well capacity, Signal to Noise Ratio (SNR), and dynamic range (DR) specifications.
Using the full-well capacity and SNR we can deduce the Gray Level intensity for each pixel as a function of the number of photo-electrons interacting with it. The relation between photons and photo-electrons is called “quantum efficiency” (Q.E.).
What this means is that not all incident photons on the pixel area contribute to its value. Some of them just don’t do anything, for example, imagine 100 photons hit a single pixel with a Q.E equals 0.5. Meaning only 50 electrons would get produced.
The number of electrons which can be contained in a pixel is referred to as the full well capacity. Let’s examine the specification of a real camera the Adimec S-25A70/CXP which utilized OnSemi Vita25k image sensor. OnSemi specifies the maximum full well figure in the sensor datasheet as 22kel. The full-well capacity corresponds to the highest gray level value possible derived from the sensor resolution. e.g. for 8-bit sensor, it would be 2⁸-1 = 255 gray level values. We can easily compute the gray level value for each pixel as follows:
Sometimes the full-well is not specified in the specification sheet. We can extract the full-well capacity from the SNR. The SNR in a specification sheet is determined by taking the maximum signal (Full well) divided by the shot noise of the maximum signal, or SQRT(Full Well).
Let’s Generate Some Noise
Now for the fun part, say we want to create a generator of shot noise that will mimic these fluctuations of photons arriving to the image sensor. First, we need to figure out how shot noise behaves or more accurately how does shot noise distributes. Luckily for us, photon counting is a classic Poisson process. Why do you ask? I found a pretty thorough walk-through about The Poisson Distribution and Poisson Process. Check it out.
For the time being let’s go over the implementation and see how it corresponds to the true measured shot noise.
Here’s one way to implement it in MATLAB (the source code can be found in my Shot-Noise-Generator GitHub repository). First, we need to define our sensor’s resolution and Full Well. Then, for each pixel we convert it’s gray level value to a number of photo-electrons hit to sensor. Next, we generate random samples from a Poisson distribution to mimic the fluctuation of photon-electrons arriving at the image sensor and finally convert back to the new gray level value.
% Define Full Well and sensor Resolutionfw = 20000;bit_level = 8;% Generate Noisy pixel valuenumber_of_photo_electrons= graylevel_2_photoelectrons(pixel_value,fw,bit_level);new_photo_electrons_number=poisson_random_gen(photo_electrons_number);noisy_pixel_value = photoelectrons_2_graylevel(new_photo_electrons_number,fw,bit_level);
Here’s a couple of noisy images where we only applied shot noise on the right side for comparison. pretty noisy huh?
Comparing to an actual device
Let us run a comparison between an actual shot noise captured on a real camera and our shot noise generator. In the Figure below we can see 2 plots. In Blue, the pixels averaged gray level values is shown between 50 to 255 in X-axis, on the Y-axis we can view the pixels shot noise. i.e. the standard deviation of the pixels gray level values.
In Orange, we can see the simulated shot noise according to a specific averaged gray level value. We can see that the two plots align validating our shot-noise generator.
It seems that as the number of photon-electrons increases, the standard deviation of the shot noise increases as the square root of the signal. Actually, it would be accurate to say that the signal to noise ratio (SNR) increases as the image intensity increases.
Generating Poisson random values
It turned out that there is a common way to generate Poisson process which Wikipedia attributes the following algorithm to Donald Knuth:
algorithm poisson random number (Knuth):
init:
Let L ← exp(−λ), k ← 0 and p ← 1.
do:
k ← k + 1.
Generate uniform random number u in [0,1] and let p ← p × u.
while p > L.
return k − 1.The idea behind this algorithm is simple. The time between photons arrivals in a Poisson process is exponentially distributed. So, we just need to count how many arrivals there are in an interval by simulating the times between arrivals and adding them up until the time sum spills over the interval. You will end up with a number of photons arrived in the time interval.
Although it is fairly simple, it still has its problems. The expected run time of the loop is proportional to the parameter λ. Where λ is the averaged photons incident on the sensor. This algorithm is commonly used despite its terrible performance for large arguments.
There are other algorithms more suitable for real-time applications but generally speaking most of them are basic accept-reject methods.
So, back to the issue at hand. We want to create a generator that, for a specific time interval (for our case it is our camera exposure time) will count the number of occurring events. These events are photons incidents. Any time you have events which occur individually at random moments, but which tend to occur at an average rate when viewed as a group, you have a Poisson process.
For example, if per exposure time of 3msec on average we know that there are 1000 photons incidents. Those photons incidents are scattered randomly throughout the exposure time, but there are more or less 1000 per exposure time.
If 1000 such photons incidents happen every exposure time, it means that, on average, one photon incident happens every 3msec/1000 =3µsec. So, let’s define a variable λ=1/3 µsec and call it rate parameter. The rate parameter λ is a measure of frequency: the average rate of events (in this case, photons incidents) per unit of time (in this case, milliseconds).
Knowing this, we can ask questions like, what is the probability that a photon incident will happen within the next second? There’s a well-known function to answer such questions. It’s called the cumulative distribution function for the exponential distribution, and it looks like this:
Basically, the more time passes, the more likely a photon incident will occur. Suppose the Poisson distribution we want to simulate is:
Then the corresponding exponential distribution for events time interval and its CDF (cumulative distribution function) would be
Inverse transforming approach is applied here: we generate uniform distributed variable u which falls in [0, 1] and transform it according to:
then a is a random variable follows an exponential distribution, representing time interval. Replacing 1-u with u, and sum these time intervals will give us:
That is what the fifth line of Knuth’s algorithm is doing. Finally, we just need to count how many time intervals totally contained in a unit time i.e. our exposure time, that is the Poisson number of the number of photons incidents.
This approach was described by Donald Knuth in the Art of computer programming and it is a way to sample method which allows one to sample from a distribution which is difficult or impossible to simulate by an inverse transformation. Instead, draws are taken from an instrumental density and accepted with a carefully chosen probability. The resulting draw is a draw from the target density.
To visualize what we just did look at the image below; We Simply chose a random point on the y-axis between 0 and 1, distributed uniformly, and locate the corresponding time value on the x-axis. For example, if we choose the point 0.5 from the top of the graph, the time until our next photons incident would be approx. 2.1µsec.
Final Thoughts
Thank you for reading.
if you’re interested in the source code it can be found in my Shot-Noise-Generator GitHub repository.
As always, if you have any questions or comments feel free to leave your feedback below or you can always reach me on LinkedIn. Till then, see you in the next post! 😄
