1.

5 Measuring the goodness of fit

Special case:
Assume in the full model

i , i = 1, 2 , K , n

are n parameters, one per observation. Thus,

u i = b ' ( i ) ,
are also n parameters, one per observation.
Then, the log-likelihood function is

[ y i i b ( i )]

l =
+ c ( y i , ) .
a ( )
i =1

Then,

y i b ' ( i ) y i i
l
=
=
=0
i
a ( )
a ( )
.
i = y i

Suppose we want to use a simplest (null) model to fit the data in

which

1 = 2 = L = n = 1 = 2 = L = n =
Then, in the null model,

n
y i b ' ( )
l
=
=
(
)

a
i =1

y
i =1

= y
1

yi

a ( ) = 0
i =1

Let

1
1
1
1
i = (b ' ) ( i ) = (b ' ) ( yi ), = (b ' ) ( ) = (b ' ) ( y ) .

Denote

l ( 1 , 2 , K , n ) = l ( y1 , y 2 , K , y n )

( )]

y ii b i
=
+ c ( y i , )
a ( )

i =1

and

( )]

y i b

+ c ( y i , )
l ( ) =
.
i =1
a ( )
n

Then,

2(log - likelihood (original model) log - likelihood (null model))

= 2[l ( y1 , y2 , K , yn ) l ( )]
=

[
y ( ) + b( ) b( )]

a( )
n

i =1

D( y1 , y2 , K yn , )
a( )

Note that
n

[ (

) ( ) ( )]

D ( y1 , y 2 , K , y n , ) = 2 y i i + b b i
i =1

is called the deviance and can be used to measure the goodness of fit
with the null model.

Example 1 (continue):

Y ~ N , 2 . Then,
2

 = , b ( ) =

2
2

2
2
n

[ (

) ( ) ( )]

D ( y1 , y 2 , K , y n , ) = 2 y i i + b b i
i =1

= 2 [ y i ( i ) + b ( ) b ( i )]
i =1
n

= 2
i =1

y2
y i2
yi (yi y ) + 2 2

(y

i =1

y)

Example 2 (continue):

Y ~ P ( ) (Poisson). Then,

= log ( ), b ( ) = e =
n

[ (

) ( ) ( )]

D ( y1 , y 2 , K , y n , ) = 2 y i i + b b i
i =1

= 2 [ y i (log ( i ) log ( )) + i ]
i =1
n

= 2 [ y i (log ( y i ) log ( y )) + y y i ]
i =1

y
= 2 y i log i
i =1
y
n

( y i y )

Example 3 (continue):

Y ~

B (m , p )
(Binomial distribution, in frequency), 0 Y 1 . Then,
m

 p

1
1
= log
, b( ) = log 1 + e = log
= log

1
1
1
1

p
p

= log

[(

) ( ) ( )]

D( y1 , y2 , K , yn , ) = 2 yi i + b b i
i =1

1
1

log
+ log
= 2 yi log i log

1
1
1
1

i =1

i
i

n
y
1
1
y

log
+ log
= 2 yi log i log
y
y
y
y
1
1
1
1

i =1

i
i

n
1 yi
y

= 2 yi log i + (1 yi ) log
y
y
1

i =1

Example 4 (continue):

Y ~ G ( , v ) (Gamma distribution). Then,

1
, b ( ) = log ( ) = log

[ (

) ( ) ( )]

D ( y1 , y 2 , K , y n , ) = 2 y i i + b b i
i =1

n
1
1 1
1

= 2 y i
+ log + log

i =1

i
i

n
1 1
1
1
= 2 y i
+ log + log
y
i =1
y
yi
yi
n
y

y
= 2 i 1 log i
i =1 y

y
n
y y
y
log i
= 2 i
y
i =1
y

Example 5 (continue):

Y ~ IG ( , 2 ) (Inverse Gaussian distribution). Then,

1
1
(
)
=

b
,
2

2 2

[ (

) ( ) ( )]

D ( y 1 , y 2 , K , y n , ) = 2 y i i + b b i
i =1

1
1
= 2 y i
+
2
2 2
i =1
2 i

1
1

+

1
1
= 2 y i
+
2
2y2
i =1
2 yi

1
1
+

yi
y

y
1
1
= 2 i2 +

2 yi
y
i =1 2 y
n

( y i y )2

i =1

yi y 2

General case:
The simple model (the null model) we commonly use to fit the data

involves

parameters,

( ), where

~i = g 1 xi

g ( i ) = xi , i = 1, 2, K , n.

Let

is the iterated reweighted least square

estimate. Then, the deviance function

D( y1 , y2 ,K, yn , ~1 , ~2 ,K, ~n )
= 2a( )[l ( y1 , y2 ,K, yn ) l (~1 , ~2 ,K, ~n )]
n

[(

) ( ) ( )]

~
~
= 2 yi i i + b i b i
i =1

where

i = (b ' ) ( i ) = (b ' ) ( yi ), i = (b ' ) (~i ) .

1

The deviance function can be used to assess the discrepancy of a fit

with g ( i ) = xi . Intuitively, large deviance implies that the

( )

1
~
distance between the data yi and the fitted value i = g xi

is large. That is, the fit with g ( i ) = xi might not be sensible.

The forms of the deviances for different distribution are given below:
n

Normal distribution:

(y
i =1

2
~ i )

y
2 y i log ~i ( y i ~ i )
Poisson distribution: i =1
i

Binomial distribution (in frequency):

y
1 yi
2 yi log ~i + (1 yi )log
~

1
i =1
i
i

Gamma distribution:

2
i =1

y i ~ i
~
i
n

Inverse Gaussian distribution:

i =1

y
log ~i

( y i ~ i )2
y i ~ i2