
a little bit. How big of a step we can take ? Well, that’s the bad news. Our derivative only guarantees that the function will decrease if take infinitely small step. We can’t do that. Generally, you want to control how big…in opposite direction, negative, to be sure that our function will decrease, at least a little bit. How big of a step we can take ? Well, that’s the bad news. Our derivative only guarantees that the function will decrease if take infinitely small step. We can’t do that. Generally, you want to control how big of step you make with some kind of hyper-parameter. This hyper-parameter is called learning rate and I’ll talk about it later. Let’s now see what happens if we start at a point x = -2. The derivat…