Chaos
Chaos Theory meets Python Programming
I recently read the amazing book Chaos: Making a New Science by James Gleick and it introduced me to the science and beauty of Chaos. It was a revelation to understand concepts like Butterfly Effect, Strange Attractors, Mandelbrot set, etc. Also, it was great fun to recreate the wonderful visuals presented in the book using Python and below I cover some of them.
Butterfly Effect
Butterfly effect is the most popular concept of chaos theory and perhaps the most misunderstood also. The general idea most people have about the butterfly effect is that it signifies small factors that cause huge changes later on (like a butterfly flaps its wings in Peking and you get a tornado in New York). But what the idea actually talks about is the sensitive dependence on initial conditions in large complex systems, which means that even a small change in initial conditions could lead to huge changes later. A butterfly’s flap might cause a tornado or might even prevent one. The system is too complex and entangled that we can never know what would have happened even if we make a small change.
There are simple mechanical systems like a water wheel or double pendulum that exhibit this behavior. Think of the above image as the different parameters we are tracking about the system (say a double pendulum’s position, speed, etc.). What the animation actually shows is that the system never repeats the same state. The final graph resembles the wings of a butterfly and I would like to think that the name of the effect comes from this visual rather than the other popular idea of the tornado. Even though it might look like the curves are intersecting, if you look closely at the 3D graph, you can see that they never intersect. It is very easy to code up the Lorenz attractor because, at the core, it is just a set of 3 differential equations, which describe the rate of change of 3 quantities with respect to time.
Bifurcation
Bifurcation was the most interesting concept for me in chaos theory. It deals with population growth. Let’s say we have a butterfly population of x and a rate of growth of r. x is a number between 0 and 1 indicating the ratio of the hypothetical maximum possible population. We need to model the population growth and we know that if the population grows uncontrollably, the butterflies will starve to death. So the model has to be self-regularizing (in data science terms). The logistic map works perfectly in this scenario.
The equation shows how the population of butterflies next year depends on the population this year. Multiplication with the term in brackets ensures that the population doesn’t grow uncontrollably. If we use the above equation and start with a particular value of r, say 2, the population fluctuates and finally settles down at a stable value after many years.
What if we increase the value of r? The stable population also increases. What if we increase r further, say 3? You will find that the population never converges to a single number. Instead, it fluctuates between 2 numbers. Visually the graph splits into two as there are two stable populations. If the increase r further, the graph splits again. This is the onset of chaos!
The bifurcation diagram can be easily created in Python using the below code.
Fractals
The bifurcation diagram has an interesting property. If you zoom into the cluttered regions in the graph, you will see the patterns are repeating.
Fractals are patterns that are self-similar across different scales. There are many naturally occurring fractals like snowflakes, trees, blood circulatory systems, lighting, etc. Gleick argues in the book that since a self-repeating pattern needs only less information to be encoded, we find it commonly in complex organs of living things, like human lungs, as DNA needs to encode very little information to tell the body how to create that organ.
Mandelbrot Set
Mandelbrot set is the set of numbers for which the iterative function x² + c, where c is a constant, doesn’t grow to infinity. Let’s understand this by taking an example where x = 2 and c=-1. Applying the function iteratively, we get x1= 2² - 1= 3, x2 = 3² - 1 = 8, x3 = 8²- 1 = 63, and it keeps on growing to infinity and hence it cannot be a part of our set. Let’s take x=1, we get x1= 1 - 1= 0, x2 = 0 - 1 = -1, x3 = 1 - 1 = 0. It fluctuates close to 0 and never grows to infinity and hence becomes part of the set. If we write a code to find all complex numbers which are part of the set and plot them, we get the beautiful visual below.
Relationship between Mandelbrot set and Bifurcation diagram
There is a hidden relationship between the Mandelbrot set and the bifurcation diagram. If we plot the values to which the numbers in the Mandelbrot set converge on the z-axis, we actually get the bifurcation diagram! Amazing, isn’t it?
Final Thoughts
The science and beauty of chaos theory have been a revelation for me, thanks to Gleick’s book. It changes the way you look at the world and you start seeing fractals and non-linear systems in your daily life. I am also intrigued by the implications it has on the field of machine learning which tries to predict the future state of systems, and here we find that the state of chaotic systems can never be predicted, at least not beyond a time duration!
