• The name logistic regression emerges from logistic distribution function.

eZ
1  eZ

• Mathematically, logistic regression attempts to estimate conditional probability of an event

(or class probability).

• It is a symmetrical distribution (density function)

• F(Z) is S-shaped curve

eZ
f (t ) 
 1 e 
Z 2

eZ
F (Z ) 
1  eZ
Logistic Function(Sigmoidal function)

1 ez
P (Y  1)   ( z )  
1  ez 1  e z
z   0  1 x1   2 x2  ...   n xn

• The logistic regression model is given by:

e (  0  1 X i ) i
i   e(  0  1 X i )

1  e (  0  1 X i ) 1 i

  
ln i    0  1 X i
1 i 
Function with linear properties (Link Function)

• The logit function is the logarithmic transformation of the logistic function. It is defined as
the natural logarithm of odds.

• Logit of a variable  (with value between 0 and 1) is given by:


Logit ( )  ln( )   0  1 x
1 

Likelihood function for Binary Logistic Function

• Probability density function for binary logistic regression is given by:

 n
ln L(  )   yi (  0  1 xi ) 
i 1
n

 ln( 1  exp( 
i 1
0  1 xi ))

Estimation of LR parameters

 ln( L(  0, 1)) n n exp(    x )

  yi   0 1 i 0
 0 i 1 i 11  exp(  0  1xi )

 ln( L(  0, 1 )) n n x exp(    x )
  xi yi   i 0 1 i 0
1 i 1 i 11  exp(  0  1xi )
The above system of equations are solved iteratively to estimate 0 and 1