What most machine learning models want to maximize or minimize
Limits, the base of derivatived
How most machine learning cost functions find their optimum value
We already know what’s a function, but to know what happens when we approach one of the values where a function has no domain or any other value, here come the limits.
With limits, we will be able to analyze de continuity of a function, therefore, the derivability of it.
The function that tends to a limit
The function f approaches the limit l near a, if we can make f(x) as close as we like to l by requiring that x be sufficiently close to, but unequal to, a.
The best way to understand this definition is by plotting some examples:
The limits that we want to check are at values marked by grey dotted lines.
- In the first graphic, we plotted y = x, in this case, the function tends to y= 1.5 when we approach x = 1.5.
- In the second-one, we plotted y = x², in this case, the function tends to y= 1.5² when we approach x = 1.5.
- In the third case, we defined the function to change at x = 1.5, at this value y tends to 1.5² and 1.5 at the same time, so the left and right limits are not the same.
Limits
Ok, we can say that a function tends to a value at some point, but we need a more strict definition to be able to get useful results, for example, ¿what’s the distance that we should check?.
Definition
The function f approaches the limit l near a means: for every ε > 0 there is some δ > 0 such that, for all x, if 0 < |x- a| < δ, then |f(x) — l| < ε.
The mathematical notation for a limit is the following:
It says that the limit of f(x) when x approaches a is l.
We can express the side of the limit using this notation, in the + case, we check the approach from the right(bigger values than a). In the - case, we check the approach from the left(lower values than a).
We will need to introduce some lemmas, but be quiet, lately we will work out some examples so you can view it in action!
Some lemmas and theorems
- A function can not have two distinct limits for the same a, if l approaches l at a and f approaches m at a, then m = l.
- If x is close to x0, and y is close to y0, then x + y will be close to x0 + y0.
- If x is close to x0, and y is close to y0, xy will be close to x0y0
- If x is close to x0, and y is close to y0, 1/y will be close to 1/y0.
- If lim f(x) = l and Iim g(x) = m, then lim(f+g)(x) = l+m.
- If lim f(x) = l and Iim g(x) = m, then lim(f · g)(x) = l · m.
- If lim f(x) = l and Iim g(x) = m, then lim(1/ g)(x) = l/m.
Example
We will evaluate the following function:
We want to analyze the limit at the value of 3, to do that we will check f(3) and the side limits.
As you can see, this function has no solution for x = 3, let’s check the limits of it, to calculate the side limits we’ve used am epsilon of 0,01.
As the delta value decreases, left and right limits tend to infinity, but in the opposite sign, so we have 2 distinct limits for a value that has no solution. This case has some special properties that will be explained in future posts.
Summary
This is the second post of the calculus series, in it we introduced what are limits and some properties of them. Limits will allow us to define derivates and integrals. Techniques used for optimizing functions to find the minimum error, and here is where calculus gets involved.
This is the twenty-second post of my particular #100daysofML, I will be publishing the advances of this challenge at GitHub, Twitter, and Medium (Adrià Serra).
