Random Walks And Diffusion
Why does milk diffuse in coffee? Or why does perfume spread out in the ambient air? We are usually told that the reason is that because entropy always increases and diffusion means more disorder and so an increase in entropy, so that’s why it should happen. But what really causes diffusion on the microscopic level and hence causes entropy to increase?
Diffusion is described using a partial differential equation called the diffusion equation it describes how the density of the quantity of interest changes as time passes, but we can derive the diffusion equation and explain why diffusion happens using the statistical properties of the particles and the concept of random walks.
What’s a random walk
A random walk is basically a process in which each step taken is random, in which each direction is equally likely to be taken (or it can be biased sometimes), it can be done by flipping a coin and if it lands heads you take a step forwards, if it lands tails you take a step backwards (that’s a random walk in only 1 dimension, but it can be generalized to higher dimensions).
Despite the fact that this process is undirected that is every direction is equally likely to be taken, it gives rise to the diffusion process, and we’re going to prove this mathematically.
If we consider some number of random walkers that move N number of steps, then the distribution of the positions of these particles is a bell curve that is centered around zero, zero is the most likely position because it’s more likely that the random walker took an equal number of steps in the positive and negative direction while it’s unlikely for it to have moved all N steps forwards (flipping a coin and getting N H’s in a row).
As the number of steps increases, there’ll be more ways for the particle to land at particular spots that are further away from the origin and the probability of these positions will increase, so the distribution will get wider, this is how diffusion happens, the change in probability or the flattening out of the Gaussian distribution will tell us how the concentration of particles will change with time.
The standard deviation(basically, how much the bell curve is spread out) of the graph will be the square root of N (number of random steps taken). Since forward and backward steps are equally likely at all times, the expected average finishing position must be back at the origin. Standard deviation measures how far away from the origin, on average, we can expect to land, regardless of direction.
Now, what is the equation for the probability distribution of the positions of the particles? Or after N steps, what’s the probability of landing at a specific number n?
That will be the portion of all possible paths that lead to this position divided by all possible paths that the particle can take. we need to first find all possible paths that lead to n, we define n_f and n_b to be the number of forward and backwards steps which can be obtained from
solving these equations gives
so all possible paths that lead to n are
this gives all possible paths that lead to n, the factorials in the denominator account for the redundancies in counting, so if HHTTHH was one such possible way to reach n swabbing the first two H’s wouldn’t make a difference, so you need to account for all these redundancies that result from swapping similar letters and that is equivalent to their factorial.
We now find the total number of possible paths, since at each step you have two possibilities, and you have N steps, then it’ll be 2^N possible paths.
And then using Stirling’s formula to approximate factorials we get (after lots of mathematical manipulation ):
And generalizing it for all positions x
and this in fact is the equation for a Gaussian distribution, where the standard deviation here is the square root of N.
Now we need to write the probability distribution for gas particles, We can think of N as time instead of the number of steps taken, N is proportional to time so write the equation like this
So what we see in here is that the term 1\sqrt(t) is the term that will bring the distribution down, while the term 1\t in the exponent of e will make the distribution wider as time progresses, and that’s exactly what gives rise to the diffusion process! That’s how random walks cause diffusion (as for the D term it’s called the diffusivity constant, we’ll see more about it below).It’s important to note that this equation represents the probability distribution at time t of a particle that started initially at zero .
And that also explains why entropy increases as the particles spread out , when they spread they’ll have more ways to occupy space and arrange themselves in it so more disorder , the flattening out of the bell curve means that probability decreases and you become less certain where the next particle will land , less certainty means higher entropy .
Here is how the distribution function that we just derived looks like as time passes.
Diffusion Equation
Let’s see now how we can describe diffusion using the diffusion equation , the random motion of particles gives rise to the diffusion process and we can use these random walks to derive the equation, we consider here diffusion in only one dimension along the x-axis. We divide the space into small cells and consider the change in density in each cell.
Now let’s consider the middle cell, how does it’s density change as time passes? well there’ll be some particles that’ll leave it and others that’ll enter it from neighboring cells the net change will be
(The fraction 1/2 is here because half of the particles will go left and the other half will go right according to our random walk scheme)
Now we do some manipulation to make the equation look as a difference of differences
Difference in differences is equivalent to
(The subscribes indicate what we are considering the change in density with respect to time in the term before the equal sign and with respect to position for the term after the equal sign)
We then want to consider the limit as delta t →0 and delta x → 0, we divide and multiply the equation by delta t and delta x²
if we introduce a constant
Then taking the limit as delta t →0 and delta x → 0, in such a way that D stays a constant, we finally arrive at the diffusion equation
The equation we derived earlier for the probability distribution
which does satisfy the diffusion equation( though we should replace P by rho), but we want to find the total density at any time and that equation describes only particles that started at the origin we could have had some initial distribution for the particles, the full solution is
This looks scary but what it says is that the density equals the amount of particles times the probability distribution of the particles , integrated from positive infinity to negative infinity( we integrate because now we want the density and not the probability function)
One final note, in here we only focused on diffusion in one dimension, but everything we’ve said can be easily generalized to 2D and 3D and the equations would be very similar.
