Machine learning with linear regression and gradient decent part 3

Now we are here finally we talk about gradient decent we have gone through normal linear regression learnt what we need to know bout cost function and now we get to see the fun side of machine learning ,i started out this tutorial thinking it was gonna be a quick 5 min read ,boy was i wrong machine learning on its own is kinda hard to get your mind wrapped around and if you can understand it by just reading all this am pretty sure your one of the smartest people in the room ,so lets get on with it

Gradient decent

Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point. If instead one takes steps proportional to the positive of the gradient, one approaches a local maximum of that function; the procedure is then known as gradient ascent.

Well imagine your on a mountain and their is a pot of gold hidden somewhere down the end of the mountain and you are tasked to find it but here is the hinch your first blindfolded and turned 360 times and at the end of that am sure you really wont know where you are

,well thats sad now you have to kinda make a trip down lane by checking every step you make if your actually going down or not ,i know hard to beleive right but bear with me

here is a sulface plot to help illustrate our idea if you notice their is kinda an arrow pointing in the downwards direction its kinda obvious to know that we are aiming for that blue region thats where our j(cost function) would be the lowest

Am gonna jump a lot of complicated terms ,explanation and formulas and give you the formula we are using to find our m and c variables with graident decent ,but if you wanna know that complex part i advice you reading tailor series and this link.

M = m’ - a*(the differential of our cost function)*x

C = c’ - a*(the differential of our cost function)

M = the present value of m

m’ = previous value of m

a =learning rate

the differential of our cost function =2/n E(y-y’) if you dont understand first order differentiation you might wanna read that here or google it

lets look at this a little bit shall we you know when i mentioned iterative and talked about checking where i am every time to know if i am going in the right direction we need some kinda memory dont we so we kinda need to iterate through and store our previous position on the slope thats our m’ and c’ ,our learning rate lets just say its how fast we move down the slope

So our final equation is something like this

M= m’ -(a(2N(E((y -(mx + c))* x)))

C = c’ -(a(2N(E((y-(my+ c)))))

Thats it thats gradient decent in all its glory thats how we are gonna find our values for m and c programmatically by turning this equations to code

Our next tutorial we are gonna do all this in code js and python to be exact catch you then☝🏾.