The outline of Gradient Descent

What is Gradient Descent?

A blindfolded person trying to climb mountains, Source: [1]
Top-down view of the hill, Source: [2]

How it works

  • “b” describes the next position of our climber,
  • “a” represents his current position.
  • The minus sign refers to the minimization part of gradient descent.
  • The “gamma” in the middle is a waiting factor and
  • The gradient term ( Δf(a) ) is simply the direction of the steepest descent.

Importance of the Learning Rate

Big and small Learning Rates, Source: [2]
The procedure followed, Source: [4]

How to make sure that it works properly?

Difference between good and bad learning rates, Source: [2]

Conclusion

References

  1. https://leadingindia.ai/
  2. https://www.analyticsvidhya.com/blog/2017/03/introduction-to-gradient-descent-algorithm-along-its-variants/
  3. https://towardsdatascience.com/gradient-descent-in-a-nutshell-eaf8c18212f0
  4. https://www.jeremyjordan.me/gradient-descent/
  5. https://medium.com/diogo-menezes-borges/what-is-gradient-descent-235a6c8d26b
  6. https://blog.paperspace.com/intro-to-optimization-in-deep-learning-gradient-descent/

--

--

--

Department of Computer Science and Engineering, Mody University, Lakshmangarh.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Simple Image Classification using Convolutional Neural Network — Deep Learning in python.

Web API’s and Natural Language Processing

Machine learning: A beginner’s guide to teaching artificial intelligence

Hidden Markov Models with Viterbi Algorithm…

Forward Kinematics Calculation for Robotic Arm

Introducing TFServe: Simple and easy HTTP server for tensorflow model inference

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sunil kumar Jangir

Sunil kumar Jangir

Department of Computer Science and Engineering, Mody University, Lakshmangarh.

More from Medium

Logistic Regression for Machine Learning

Machine Learning Series | part 1

Support Vector Machines — An Introduction

Understanding bias and variance trade-off