What in god’s name is Gradient Descent?

Gradient Descent is a very generic optimization algorithm capable of finding optimal solutions to a wide range of problems. The general idea (…) is to tweak parameters iteratively in order to minimize a cost function.


What is a Cost Function?

Our linear model (red) generalising to all the data available (blue dots). Wikipedia
Resultado de imagem para linear regression
Difference or distance between the predicted value (red) and the actual value (blue). Source
Source

Gradient Descent: Minimising the Cost Function

Source
Cost Function vs Value of Weight (parameter θ). Source
Cost Function vs Value of Weight (parameter θ). Learning Rate = 0.10. Source
Cost Function vs Value of Weight (parameter θ). Learning Rate = 1. Source
Cost Function vs Value of Weight (parameter θ). Learning Rate = 1.6. Source
Resultado de imagem para gradient descent
Source
Hands-on Machine Learning with Scikit-Learn & TensorFlow

The types of Gradient Descent


You the mighty reader applauding!

The Making Of… a Data Scientist

Welcome to “The Making of… a Data Scientist”. This is my personal blog with all I’ve been learning so far about this wonderful field! Hope you can get something useful for your path as well!

Diogo Menezes Borges

Written by

Data Science enthusiast and wannabe

The Making Of… a Data Scientist

Welcome to “The Making of… a Data Scientist”. This is my personal blog with all I’ve been learning so far about this wonderful field! Hope you can get something useful for your path as well!