LOGISTIC REGRESSION THE BAYESIAN WAY

Mrunaldhar Bathula
Nov 4 · 4 min read

Logistic Regression is an algorithm that is used as a classifier i.e that it determines the probability of occurrence of an event given that data had already occurred …..but wait………. it is a common thing and known by most of the people who work on DATA SCIENCE or MACHINE LEARNING you may question me that what is new in this post ………….answer for your question is that I am going to explain logistic regression in both intuitive and mathematical ways so that you can appreciate the functionality and heart of the algorithm.

For beginners who don’t know much about logistic regression…………….. logistic regression is an algorithm that outputs the probability of occurrence of our upcoming event given the data of events we have had already occurred, This algorithm performs by fitting an sigmoid function on the data we have and predicts the probability of occurrence of events given that data had already occurred

Firstly you may ask me what is need of logistic regression ???

The answer is if you have an event in which you have to predict the probability of occurrence of making a good tea given that data such as amount of sugar used and amount of tea powder used and amount of milk used,The logistic regression is one of the best way out ,On visualizing our data our data will look some thing like this as shown in FIGURE1 . In figure 1 Y axis shows us data about weather the tea we prepare is good or not versus a parameter (amount of sugar added in making tea)

FIGURE 1

The main aim of regression is to fit a curve onto the data such that the fitted curve will be able to predict the value or probability of occurrence of an event

If we try to fit a straight line onto that data i.e linear regression it looks something like this as shown in FIGURE 2

from the curve we can see for values of X>some-value the value of Y is greater than 1 and for some values X<(value of X-intercept) the value of Y<1 ,from the above data we can conclude that fitting a linear model for the above classifier data is in inaccurate since probability of an event must lie in the range of [0,1] since we had defined Y as a probability ,so it must lie in range of [0,1] hence we can conclude that fitting this type of a straight line to our data is a bad idea

So friends our constraint is Y must not exceed 1 or less than 0 hence the function Y must have Y=0 and Y=1 as the asymptodes of our function Y=f(X)

you may think that graph given in Figure 3 is perfect for our need

FIGURE 3

Hurray !! we finally arrived……..but friends its not over since the at turning points the function is continuous but not differentiable so its impossible to find the optimum of the function(while figuring out the parameters that determine the exact shape of the curve) so we have to make corners smooth so that the function is both continuous and differntiable and there is one more concern with the graph as in figure3 the probability of the occurrence of that event is suddenly decreasing to zero but on observation from data probability gradually decreases and rate of change of probability decreases as we move towards left from the center of the graph and for the next half of the graph the probability gradually increases to one but not suddenly it must gradually increasse so we can see that there is a need for an smooth graph for which first derivate increases for the first half and decreases for the second half

So from the above discussion we can conclude that there is an need for an “S” shaped function which has asymptodes at Y=0 and Y=1

Figure 4

you may question me that “You have told us only about an “s” shaped function but why you considered this function shown in figure4 “

(or)

you may ask me why sigmoid!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! not other functions ???

The answer for the above question is in my next story ……stay tuned

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade