1

6.8 Lack of Fit Test: Multiple Linear Regression

Suppose we have the following data:
y11 , y12 , , y1n1 : n1 repeated observations at x11 , x12 , , x1 p 1

y21 , y22 , , y2 n2 : n2 repeated observations at x21 , x22 , , x2 p 1

ym1 , ym 2 , , ymnm : nm repeated observations at xm1 , xm 2 , , xmp1

m

Note: n j n .
j 1

Objective: test

H 0 : E y ji 0 1 x j1 p 1 x jp 1 .

The true model (model m): Y ji u j ji

The estimate of u j is

f ( x j ) ji .

nj

u j y j

y
i 1

ji

nj

The reduced model (model p) : y ji 0 1 x j1 p 1 x jp 1 ji :

The fitted value of y ji in reduced model is
y ji b0 b1 x j1 b p 1 x jp 1 y j .
y ji y j

u j y j

(model p)

y ij

(model m)

(data)

RSS model p RSS ( model m)

nj

RSS ( model m)

nj

RSS (model m) y ji u j y ji y j
m

j 1 i 1

j 1 i 1

nj

RSS (model p ) y ji y ji
m

j 1 i 1
nj

y ji b0 b1 x j1 b p 1 x jp1
m

j 1 i 1

nj

eij2
j 1 i 1

residual sum of square in multiple linear regression

m

nj

nj

RSS model m RSS model p (u j y ji ) 2 ( y j y j ) 2

j 1 i 1

n j ( y j y j ) 2
j 1

j 1 i 1

ANOVA table:
Source
RSS ( model p )

df

SS

Lack of fit

m-p y t y b t X t y y t y b t X t y RSS model m

Pure error

n-m

SS (b0 )

SS (b0 , b1 , b p )

RSS model m
ny 2

SS (b1 , , bP1 | b0 ) p-1

b t X t Y ny 2
n

Total

y t y y i2
i 1

Note:
nj

RSS (model m) yij y j is called pure

m

j 1 i 1

error sum of squares.

RSS model p RSS model m n j ( y j y j ) 2 is called

j 1

lack of fit sum of

squares.

Note:

Fundamental Equation:
m

nj

nj

( y ji y j ) 2 ( y ji y j ) 2 n j ( y j y j ) 2 .
j 1 i 1

j 1 i 1

j 1

That is
Residual sum of squares (model p) = Pure error sum of squares
+ Lack of fit sum of squares

To test

H 0 : E y ji 0 1 x j1 p 1 x jp 1
m

RSS model p RSS model m

f

and reject

RSS model m

n (y
j 1

m p

nm

H 0 : E y ji 0 1 x j1 p 1 x jp 1

Example:

y j ) 2

m p
m

nj

( y
j 1 i 1

ji

y j )2

nm

as

f f m p , n m ,

For the data below,

13

12

11

13

11

13

X1

-1

-1

-1

-1

X2

-1

-1

-1

-1

Test for lack of fit at 0.05 .

[solution:]
n 10, p 3, m 5 ,
1 1
1 1
X

1 0

1
,y

8
13
.

13

Then,

b X X
t

10.5
X y 0.25 .
2.25
t

Thus,
RSS ( model 3) y t y b t X t y 12 .

To calculate the pure error sum of squares RSS (model 5) ,

j1

, x j2

2
2
(-1,-1): y1 8, ( y1i y1 ) (8 8) 0 .
i 1

3
11 12 13
12, ( y 2i y 2 ) 2 (11 12) 2 (12 12) 2 (13 12) 2 2
(-1,1): y 2
3
i 1
3
987
8, ( y 3i y 3 ) 2 (9 8) 2 (8 8) 2 (7 8) 2 2
(1,-1): y3
3
i 1
1

2
2
(1,1): y 4 13, ( y 4i y 4 ) (13 13) 0
i 1

(0,0):

(y

5i

y5 )

y51 y52 2 11 13 2

i 1

2.

Therefore, the pure error sum of squares is

RSS (model 5) 0 2 2 0 2 6 .

RSS model 3 RSS model 5

RSS model 5

10 5

H 0 : E y ji 0 1 x j1 2 x j 2 .

12 6
5 3 2 2.5 5.79 f
2 , 5 , 0.05 not
6
5

reject