The Schrödinger equation can be derived from classical mechanics, general relativity, and duality
Richard Feynman, in his famous lectures, wrote:
We do not intend to have you think we have derived the Schrödinger equation but only wish to show you one way of thinking about it. When Schrödinger first wrote it down, he gave a kind of derivation based on some heuristic arguments and some brilliant intuitive guesses. Some of the arguments he used were even false, but that does not matter; the only important thing is that the ultimate equation gives a correct description of nature. [Feynman, Lectures on Physics, III-16.1]
A new paper, just published by Nature, indicates that a derivation may be possible:
Quantum Mechanics can be understood through stochastic optimization on spacetimes
The main contribution of this paper is to explain where the imaginary structure comes from in quantum mechanics. It is…
The authors, Jussi Lindgren and Jukka Liukkonen, wanted to figure out a path to those quantum operators that did not use postulates. It turns out that they follow a map that I’ve used before — although they don’t use the term — called duality, and a map that I don’t know anything about: general relativity. In this post I walk along their trail, and try to explain what I think is going on.
Here is the Cliff Notes version of the paper. The setup is:
- A particle, like a photon or an electron, moving nearly at the speed of light.
- A variable influence on the motion at every location in space-time — like a persistent background noise.
The paper starts solidly in classical mechanics. The particle paths are those that “minimize the action”, which means that the change in kinetic energy equals the change in potential energy .
Minimizing action is a way of talking about the path of Newton’s apple. The increase in the kinetic energy (how much damage the apple will do to Newton) matches the decrease in gravitational energy as it gets ever closer to Newton. It gives you exactly that relationship in exactly the right proportions, and that’s how we calculate the trajectories of particles, apples, and space-ships.
Next, the authors select models for the two non-classical features of the setup:
- The particle is moving at nearly the speed of light, so they use the tools of general relativity to coordinate the lengths of the measuring rod and the ticks of the clock.
- The persistent noise is a fluctuation in space and time, so they use the tools of duality in stochastic optimal control to coordinate the statistics of this background noise with the dynamics of the particle.
Finally, going step by careful step, they arrive at the famous Schrödinger wave equation, whose solutions are probability waves.
Let’s see how this trail of ideas might apply to a well-known puzzle. Take a look at the famous single-photon two slit experiment by Hamamatsu in 1981:
At minute 6:30, the experiment starts firing single photons from emitter to screen. Each impact registers as a black dot — energy deposited on the screen. By 8:00 you can see that the dots are filling in an interference pattern.
So, what have we seen?
- Each photon goes along its own path and hits the screen in exactly one spot.
- A single photon arrives in exactly one place. No waves.
- As more and more dots appear, and they fill in the famous interference pattern.
That’s what Thomas Young saw in 1801. He couldn’t see the dots, but 180 years later Hamamatsu could.
The interference bands are consistent with each slit being a source of coherent standing waves. See, for example, the nice explanation from BC Open Textbooks:
Explain the phenomena of interference. Define constructive interference for a double slit and destructive interference…
The double slit experiments reveals two things.
- The interference pattern appears only when we fire an enormous number of photons through the slits.
- The large numbers of photons passing through each slit are organized into two coherent standing waves.
Here’s how the authors’ derivation applies to this experiment. First, we discuss the particle point of view.
Each particle starts out with a position and a velocity. Along the way the particle is influenced by the noise. This influence can be approximated by a sequence of “bumps”. Each bump occurs at a random time and each bump changes the position in a random way. After each bump the action is minimized— until the next bump. Finally, at the end of the trip, the particle bangs into the screen and leaves a dot.
Now, here is the wave point of view. The waves don’t move in the double slit experiment: they are standing waves. The dark regions are where lots of particles deposit their energy and the light regions are where few photons do, like sand on the surface of a vibrating drum.
To predict the shape of the light and dark regions you can solve the Schrödinger equation for the setup of the experiment. The conditions of the setup are a source of particles, the properties of the particles, two slits with their particular dimensions and separation, and the screen in back. The probability distribution of energy predicted by the wave function will be what we see on the screen.
All aspects of the experiment, the source, the slits, and the screen, all of these conditions, need to be worked into the equations (this is not easy). If you decide to measure which slit each particle goes through, well, then that measurement apparatus needs to be included too. The equations will be different, the wave function will be different, and the pattern in the back will be different.
Personally, I find this to be quite reasonable. The particle paths are really there, from start to finish. And the wave function describes the distribution of the paths.
Frankly, I have always been puzzled by the statement that the waves in the two slit experiment “cancel”. What is being canceled? If each particle has energy then where does the energy go when the waves cancel? It’s easier to think in terms of probability. The dots are where the probability is.
Finally, I am relieved that multiple universes are not required to explain this experiment. Waves don’t collapse into particles. Particles just bump along, zigzagging through life like we all do. The wave patterns emerge from the collective behavior of zillions of particles. Describing those patterns is what a wave function does.
This paper is a major breakthrough for my understanding of the double slit experiment, and perhaps for your understanding, too. But maybe the authors have been too hasty? I have certainly seen some pushing back on the setup.
A persistently variable field is not controversial, but waves require rotation. In the authors’ model, the rotation comes from a persistently variable field in space-time.
So let’s think about that. Suppose the particles are photons. Photons do not experience time. The photon is a flash. It transfers energy from one part of the universe to another, along a line that is as straight as straight can be, instantaneously. So how can it bump?
Well, suppose that the path of the photon is composed of a long (straight) line of little bits. Each interaction of a quantum of space with the quantum of photon energy obeys Heisenberg’s uncertainty principle. This applies between position and momentum, and also between energy and time. If energy and momentum are conserved, then position and time must be uncertain. So perhaps the uncertainty principle applied to space-time could be the source of the Brownian motion proposed by the authors.
This is gets back to the very interesting question posed by the Physics Detective: what is waving? He thinks that perhaps it is space that waves.
The part of this trail that is familiar to me is duality in stochastic optimal control. This is used in Quantitative Finance used to price option contracts. The optimal control part is the Black-Scholes method, which prescribes how a trader can hedge an option when the price of the underlying security moves as if it were driven by Brownian motion. By the arbitrage principle, at any point in time the portfolio of the trader will be equal to the price of the option.
The mysterious part is that you can use duality to calculate the option price. The dual solution is a probability distribution of prices — called the equivalent martingale measure. You integrate the option formula using this probability distribution, and that will be the price of the option.
It took me a long time to understand this. I was stuck in the control part, which is very difficult to solve. To understand equivalent martingale measures, I wrote my own paper in a more familiar context. The options pricing geeks who reviewed my paper said I was just regurgitating something already well known, but it was a revelation to me and to many others as I gleaned from comments to me over many years.
Now, Lindgren and Liukkonen have supplied a very nice revelation for how the Schrödinger equation could be derived, a plausible explanation of the double slit experiment, and an exciting possibility for a quantum theory of space-time. Well done Jussi and Jukka!
PS: Jussi has started a blog on duality: