Analysis of Variance and Design g of Experiments Experimentsp -I

MODULE II
LECTURE - 7

GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE

Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur

H 0 : k = k0 , k = 1, 1 2,.., 2 r < p when C Case 2 Test 2: T t of f a subset b t of f parameters t h d 2 and

r +1 , r + 2 ,..., p are unknown

In case 1, the test of hypothesis was developed when all s s are considered in the sense that we test for each

i = i0 , i = 1, 2,..., p. Now consider another situation, in which the interest is to test only a subset of 1 , 2 ,..., p , i.e., not all but
only a few parameters. This type of test of hypothesis can be used e.g., in the following situation. Suppose five levels of voltage are applied to check the rotations per minute (rpm) of a fan at 160 volts, 180 volts, 200 volts, 220 volts and 240 volts. It can be realized in practice that when the voltage is low, then the rpm at 160, 180 and 200 volts can be observed easily. At 220 and 240 volts, the fan rotates at the full speed and there is not much difference in the rotations per minute at these voltages. So the interest of the experimenter lies in testing the hypothesis related to only first three effects, viz., , 1 for 160 volts, 2 for 180 volts and 3 for 200 volts. The null hypothesis in this case can be written as:
H 0 : 1 = 10 , 20 = 20 , 3 = 30

when 4 , 5 and 2 are unknown. Note that under case 1, the null hypothesis will be
H 0 : 1 = 10 , 20 = 20 , 3 = 30 , 4 = 40 , 5 = 50 .

3
Let 1 , 2 ,..., p be the p parameters. We can divide them into two parts such that out of 1 , 2 ,..., r , r +1 ,..., p , and we are interested in testing the hypothesis of a subset of it.
0 Suppose, we want to test the null hypothesis H 0 : k = k , k = 1, 2,.., r < p when r +1 , r + 2 ,..., p and 2 are unknown.

The alternative hypothesis under consideration is H 1 : k k0 for at least one k = 1, 2,.., r < p. In order to develop a test for such a hypothesis, the linear model

Y = X +
under the usual assumptions can be rewritten as follows:
(1) Partition X = ( X 1 X 2 ), = (2)

where (1) = ( 1 , 2 ,..., r ), (2) = ( r +1 , r + 2 ,..., p ) with the orders of matrices as X 1 : n r , X 2 : n ( p r ), (1) : r 1 and ( 2 ) : ( p r ) 1. Th model The d l can be b rewritten itt as

Y = X + (1) = ( X1 X 2 ) + (2) = X 1 (1) + X 2 (2) + .

4
The null hypothesis of interest is now
2 0 (2) and H 0 : (1) = (1) = ( 10 , 20 ,..., r0 ) where h d are unknown. k

The complete parametric space is

= {( , 2 ); < i < , 2 > 0, i = 1, 2,..., p}

and sample space under H 0 is
0 = {( (1) , (2) , 2 ); < i < , 2 > 0, i = r + 1, r + 2,..., p}.

The likelihood function is

1 exp 2 ( y X )( y X ) . L( y | , 2 ) = 2 2 2 1
n 2

Th maximum The i value l of f lik likelihood lih d function f ti under d is i obtained bt i d b by substituting b tit ti th the maximum i lik likelihood lih d estimates of and 2 , i.e.,

= ( X X )1 X y 2 =
as

1 )( y X ) (y X n
n

1 2 1 )( y X ) exp 2 ( y X Max L( y | , ) = 2 2 2
2

2 n n exp . = ' 2 2 ( y X ) ( y X )

5
Now we find the maximum value of likelihood function under H 0 . The model under H 0 becomes
0 Y = X 1 (1) + X 2 2 + . The Th lik likelihood lih d function f ti under d H 0 is i

1 2 1 0 0 exp 2 ( y X 1 (1) L( y | , ) = X 2 (2) )( y X 1 (1) X 2 (2) ) 2 2 2

2

1 2 1 exp 2 ( y * X 2 (2) )( y * X 2 (2) ) = 2 2 2

(0) . Note that ( 2) and 2 are the unknown parameters. This likelihood function looks like as if it is where y* = y X 1 (1)

written for y* ~ N ( X 2 (2) , 2 ). This helps is writing the maximum likelihood estimators of ( 2) and 2 directly as = ( X ' X ) 1 X ' y * (2) ( ) 2 2 2 )( y * X 2 = ( y * X 2 (2) 2 (2) ). 1 n

X . Since X X is a positive definite matrix, so X 2' X 2 is also positive Note that X 2' X 2 is a principal minor of X
definite. Thus ( X 2' X 2 ) 1 exists and is unique. Thus the maximum value of likelihood function under H 0 is obtained as
2 , ) = 1 exp 1 ( y * X Max L ( y* | 2 (2) ) ( y * X 2 (2) ) 2 2 2 2 2 n

2 n n = exp . 2 ( y * X 2 2 (2) ) '( y * X 2 (2) )

0 The likelihood ratio test statistic for H 0 : ((1)) = (1) is ( )

max L( y | , 2 ) max L( y | , 2 )
n 2

)( y X ) (y X = )( y * X ( y * X 2 (2) 2 (2)

)( y * X ( y * X 2 (2) 2 (2) ) ( y X ) ( y X ) + ( y X ) ( y X ) = )( y X ) y X ( n )( y * X 2 ( y * X 2 (2) 2 (2) ) ( y X ) ( y X ) = 1 + )( y X ) y X ( n q1 2 = 1 + q2

-n 2

)( y X ). )( y * X q2 = ( y X where q1 = ( y * X 2 (2) 2 (2) ) ( y X ) ( y X ) and

Now we simplify p y q1 and q2 Consider
' 1 ' ' 1 ' )( y * X ( y * X 2 (2) 2 (2) ) = ( y * X 2 ( X 2 X 2 ) X 2 y*) ( y * X 2 ( X 2 X 2 ) X 2 y*) ' 1 ' = y *' I X 2 ( X 2 X 2 ) X 2 y*

I X ( X X ) 1 X ( y X 0 X ) + X 0 = 2 2 2 2 1 (1) 2 (2) 2 (2) ( y X 1 (1) X 2 (2) ) + X 2 (2) 0 1 0 X2) X2 = ( y X 1 (1) X 2 (2) ) I X 2 ( X 2 ( y X 1 (1) X 2 (2) ).

7
' ' 1 ' The other terms becomes zero using the result X 2 I X 2 ( X 2 X 2 ) X 2 = 0. Consider )( y X ) = ( y X ( X ' X ) 1 X ' y )( y X ( X ' X ) 1 X ' y ) (y X
1 = y I X ( X X ) X ' y

0 0 ) + X 0 + X ) I X ( X ' X ) 1 X ( y X 0 X ( y X 1 (1) = X 2 (2) 1 (1) 2 (2) 1 (1) 2 (2) ) + X 1 (1) + X 2 (2) ) 0 1 0 = ( y X 1 (1) X 2 (2) ) ' I X ( X X ) X ( y X 1 (1) X 2 (2) )
0 I X ( X X )1 X = 0. Note that under H 0 , the term X 1 (1) + X 2 (2) and other terms become zero using the result X ' 0 can be expressed as ( X 1 X 2 )( (1) (2) ) ' . Thus

)( y * X q1 = ( y * X 2 (2) 2 (2) ) ( y X ) ( y X )
1 X 2 ) 1 X 2 = y *' I X 2 ( X 2 y*y ' I X ( X X ) X y 1 1 0 0 ' ' 0 0 = ( y X 1 (1) X 2 (2) ) I X 2 (X 2 X 2 ) X 2 ( y X 1 (1) X 2 (2) ) ( y X 1 (1) X 2 (2) ) ' I X ( X X ) X ( y X 1 (1) X 2 (2) )
0 ' ' 0 1 1 = ( y X 1 (1) X 2 (2) ) X ( X X ) X X 2 ( X 2 X 2 ) X 2 ( y X 1 (1) X 2 (2) )

)( y X ) q2 = ( y X

= y [ I X ( X X ) X ] y
'

0 0 0 0 = ( y X 1 (1) X 2 (2) ) + ( X 1 (1) + X 2 (2) ) [ I X ( X X ) X ] ( y X 1 (1) X 2 (2) ) + ( X 1 (1) + X 2 (2) ) 1 0 0 = ( y X 1 (1) X 2 (2) ) ' I X ( X X ) X ( y X 1 (1) X 2 (2) ).

Other terms become zero. Note that in simplifying the terms q1 and q2, we tried to write them in the quadratic form with
0 same variable ( y X 1 (1) X 2 (2) ).

8
Using the same argument as in the case 1, we can say that since is a monotonic decreasing function of lik lih d ratio likelihood ti t test t rejects j t H 0 whenever h
q1 , so the q2

q1 >C q2
where C is a constant to be determined by the size of the test. The likelihood ratio test statistic can also be obtained through least squares method as follows:
0 (q1 + q2 ) : Minimum value of ( y X )( y X ) when H 0 : (1) = (1) holds true.

(q1 + q2 ) : q2 q1

Sum of squares due to H 0

: Sum of squares due to error. : Sum of squares due to the deviation from H 0 or sum of squares due to (1) adjusted for (2)

0 = 0 then If (1)

) '( y X q1 = ( y X 2 (2) 2 (2) ) ( y X ) '( y X ) ' X ' y) ( yy X y) = ( yy (2) 2 X y =

Reduction sum of squares or sum of squares due to

' X ' y. (2) 2

sum of squares due to (2) ignoring (1)

Now we have the following theorem based on the Theorems 6 and 7. Th Theorem 10 Let
0 Z = Y X 1 (1) X 2 (2)

Q1 = Z AZ Q2 = Z BZ
where
' ' A = X ( X X )1 X ' X 2 ( X 2 X 2 ) 1 X 2

B = I X ( X X ) 1 X '.
Then
Q1
2

and

Q2
2

are independently distributed. distributed Further

Q1
2

~ 2 ( q ) and

Q2
2

~ 2 ( n p ). )

Thus under H 0 ,
Q1 / r n p Q1 follow a F-distribution F ( r , n p ). = Q2 / ( n p ) r Q2

Hence the constant C in

is

C = F1 ( r, n p )
where F1 ( r, n p) denotes the upper 100 % points on F-distribution with r and (n - p) degrees of freedom.

10

The analysis of variance table for this null hypothesis is as follows:

0 ANOVA for testing H 0 : (1) = (1)

Source of variation
Due to (1)

Degrees of freedom
r

Sum of squares
q1

Mean squares
q1 r

F - value
n p q1 r q2

Error (n p)

C = F1 ( p, n p )
q2
H0 : = 0

q2 (n p )

Total

n -(p q)

q1 + q2