Andrew Tch
Aug 22, 2017 · 1 min read

We train NN to understand what is correct and what is not in general )

The point of passing small amount is for the network to learn slowly, do not overfit and perform better in overall.

I will illustrate this with an example of finding max value of a function:

Source

If the learning rate is too high, the NN may end jumping between two first local maximums, never hitting the third — true — maximum. The slower the learning rate is, the bigger is the chance not to miss something.

)

    Andrew Tch

    Written by

    Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
    Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
    Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade