Gradient Ascent
You find yourself lost in the woods in a mountain range surround by thick fog, and you decide your best option is to find your way to the highest point around to signal a flare.
It turns out the optimal algorithm for doing so is called Gradient Ascent.
Gradient is a fancy word for slope, and ascent means going upwards.
The algorithm is simple: look as far as you can in every direction and find the direction that gets you the highest. Go in that direction. When you reach the highest point you could see from the last spot, re-evaluate. If every direction you look is down from your current position, you’ve reached the local maximum, which is a fancy term for the highest point in the surrounding area.
Gradient Ascent has applications in biology (fitness landscapes)[see 19:00] , finance, and big data optimization.
There are three ways to beat the Gradient Ascent algorithm.
Firstly, have richer data. Collect experience from past mountain climbers to find better algorithms for finding reaching the summit. Maybe particular rock formations or tree markings hold secrets to reaching the summit faster than simply walking up the steepest hill.
Secondly, look further. Using binoculars or fog lights might reveal higher points than what can be seen with the naked eye.
Thirdly, be willing to head temporarily downhill to reach a higher peak. If you find yourself at the top of a foothill, but you’re in a mountain range, despite not being able to see the next peak, you might get an itching feeling that you’re not as high as you could be. Take the risk, and walk down far enough to start over again. Don’t change the algorithm, just the initial conditions.
The truth is, most people live life like they are dropped randomly in a fog-covered mountain range. But imagine that altitude represents wealth, or social credibility, opportunity or self-actualization.
Most people get placed randomly, look around in the immediate vicinity for the best route, rinse and repeat. They don’t go deep into past experience. They don’t expand their view before moving. They don’t risk going downhill in order to reach higher peaks.
If you want to reach something higher than your local foothill, best do your research, broaden your vision, and risk looking for higher mountains.
PS: Another implication is that where you start initially has a dramatic impact on where you end up. If you only ever follow the algorithm to perfection, your end result is all but predetermined. Have the boldness to carve your own path, create your own rules, search deeper and wider, and reach for higher heights.
PPS: The fundamental flaw with this approach is that the landscape is always changing, the fog has variable visibility, and your legs will give out on you eventually. What if the mountains were constantly shifting? A “dynamic fitness landscape” suggests the most adaptable survive. Best to plan your algorithm not upon the highest mountain you can see, but which area is going to rise the highest just as you arrive.
