Experimental Design and the

Analysis of Variance

Comparing t > 2 Groups - Numeric Responses

Extension of Methods used to Compare 2 Groups
Independent Samples and Paired Data Designs
Normal and non-normal data distributions
Data
Design

Normal

Nonnormal

Independent
Samples
(CRD)
Paired Data
(RBD)

F-Test
1-Way
ANOVA
F-Test
2-Way
ANOVA

KruskalWallis Test
Friedmans
Test

Completely Randomized Design (CRD)

Controlled Experiments - Subjects assigned at random to
one of the t treatments to be compared
Observational Studies - Subjects are sampled from t
existing groups
Statistical model yij is measurement from the jth subject
from group i:
yij i ij i ij

where is the overall mean, i is the effect

of treatment i , ij is a random error, and i is
the population mean for group i

1-Way ANOVA for Normal Data (CRD)

For each group obtain the mean, standard deviation, and
sample size:

y i.

yij
j

ni

si

2
(
y

y
)
ij i.
j

ni 1

Obtain the overall mean and sample size

N n1 ... nt

n1 y1. ... nt y t .
y ..

yij

Analysis of Variance - Sums of Squares

Total Variation

TSS i 1 j 1 ( yij y .. ) 2
k

ni

dfTotal N 1

Between Group (Sample) Variation

SST i 1 j 1 ( y i. y .. ) i 1 ni ( y i. y .. ) 2
t

ni

dfT t 1

Within Group (Sample) Variation

SSE i 1 j 1 ( yij y i. ) i 1 (ni 1) si2
t

ni

TSS SST SSE

dfTotal dfT df E

df E N t

Analysis of Variance Table and F-Test

Source of
Variation
Treatments
Error
Total

Sum of Squares
SST
SSE
TSS

Degrres of
Freedom
t-1
N-t
N-1

Mean Square
MST=SST/(t-1)
MSE=SSE/(N-t)

F
F=MST/MSE

Assumption: All distributions normal with common variance

H0: No differences among Group Means (t =0)
HA: Group means are not all equal (Not all i are 0)

MST
T .S . : Fobs
MSE
R.R. : Fobs F ,t 1, N t
P val : P ( F Fobs )

(Table 9)

Expected Mean Squares

Model: yij = +i + ij with ij ~ N(0,2), i = 0:
E ( MSE ) 2
t

E ( MST ) 2

2
n

i i
i 1

t 1
t

E ( MST )

E ( MSE )

2
n

i i

i 1

t 1

2
n

i i
i 1
2

(t 1)
E ( MST )
When H 0 : 1 t 0 is true,
1
E ( MSE )
E ( MST )
otherwise ( H a is true),
1
E ( MSE )

Expected Mean Squares

3 Factors effect magnitude of F-statistic (for fixed t)
True group effects (1,,t)
Group sample sizes (n1,,nt)
Within group variance (2)

Fobs = MST/MSE
When H0 is true (1==t=0), E(MST)/E(MSE)=1
Marginal Effects of each factor (all other factors fixed)
As spread in (1,,t) E(MST)/E(MSE)
As (n1,,nt) E(MST)/E(MSE) (when H0 false)
As 2 E(MST)/E(MSE) (when H0 false)

0.09

0.09

0.08

0.08

0.07

0.07

0.06

0.06

0.05

0.05

0.04

0.04

E ( MST )
E ( MSE )

0.03

0.02

0.01

0
0

20

40

60

80

100

120

140

160

180

200

A)=100, 1=-20, 2=0, 3=20, = 20

n
4
8
12
20

0.09

0.08

0.07

A
9
17
25
41

B
129
257
385
641

C
1.5
2
2.5
3.5

0.03

0.02

0.01

0
0

20

40

60

80

100

120

140

160

180

200

B)=100, 1=-20, 2=0, 3=20, = 5

D
9
17
25
41

0.09

0.08

0.06
0.07

0.05
0.06

0.04
0.05

0.03
0.04

0.02

0.03

0.01

0.02

0.01

20

40

60

80

100

120

140

160

180
0
0

C)=100, 1=-5, 2=0, 3=5, = 20

20

40

60

80

100

120

140

160

180

D)=100, 1=-5, 2=0, 3=5, = 5

200

Example - Seasonal Diet Patterns in Ravens

Treatments - t = 4 seasons of year (3 replicates each)

Winter: November, December, January

Spring: February, March, April
Summer: May, June, July
Fall: August, September, October

Response (Y) - Vegetation (percent of total pellet weight)

Transformation (For approximate normality):

Y ' arcsin

100

Source: K.A. Engel and L.S. Young (1989). Spatial and Temporal Patterns in the Diet of Common
Ravens in Southwestern Idaho, The Condor, 91:372-378

Seasonal Diet Patterns in Ravens - Data/Means

Y
j=1
j=2
j=3

Winter(i=1)
94.3
90.3
83.0

Fall(i=2)
80.7
90.5
91.8

Summer(i=3)
80.5
74.3
32.4

Fall (i=4)
67.8
91.8
89.3

Y'
j=1
j=2
j=3

Winter(i=1)
1.329721
1.254080
1.145808

Fall(i=2)
1.115957
1.257474
1.280374

Summer(i=3)
1.113428
1.039152
0.605545

Fall (i=4)
0.967390
1.280374
1.237554

1.329721 1.254080 1.145808

y1.
1.24203
3
1.115957 1.257474 1.280374
y 2.
1.217935
3
1.113428 1.039152 0.605545
y 3.
0.919375
3
0.967390 1.280374 1.237554
y 4.
1.16773
3
1.329721 ... 1.237554
y ..
1.135572
12

Seasonal Diet Patterns in Ravens - Data/Means

Plot of Transformed Data by Season
1.500000

1.400000

1.300000

Transformed % Vegetation

1.200000

1.100000

1.000000

0.900000

0.800000

0.700000

0.600000

0.500000
0

3
Season

Seasonal Diet Patterns in Ravens - ANOVA

Total Variation : (dfTotal 12 - 1 11)
TSS (1.329721 1.135572) 2 ... (1.27554 1.135572) 2 0.438425
Between Group Variation : (dfT 4 - 1 3)

SST 3 (1.24203 1.135572) 2 ... (1.161773 1.135572) 2 0.197387

Within Group Variation : (dfE 12 - 4 8)
SSE (1.329721 1.243203) 2 ... (1.237554 1.161773) 2 0.241038
ANOVA
Source of Variation
Between Groups
Within Groups

SS
0.197387
0.241038

df
3
8

Total

0.438425

11

MS
F
P-value
0.065796 2.183752 0.167768
0.03013

F crit
4.06618

Do not conclude that seasons differ with respect to vegetation intake

Seasonal Diet Patterns in Ravens - Spreadsheet

Month
NOV
DEC
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT

Season
1
1
1
2
2
2
3
3
3
4
4
4

Y'
1.329721
1.254080
1.145808
1.115957
1.257474
1.280374
1.113428
1.039152
0.605545
0.967390
1.280374
1.237554

Total SS
(Y-Overall Mean)2

Season MeanOverall Mean

1.243203
1.135572
1.243203
1.135572
1.243203
1.135572
1.217935
1.135572
1.217935
1.135572
1.217935
1.135572
0.919375
1.135572
0.919375
1.135572
0.919375
1.135572
1.161773
1.135572
1.161773
1.135572
1.161773
1.135572
Sum

TSS
0.037694
0.014044
0.000105
0.000385
0.014860
0.020968
0.000490
0.009297
0.280928
0.028285
0.020968
0.010400
0.438425

Between Season SS
(Group Mean-Overall Mean)2

SST
0.011584
0.011584
0.011584
0.006784
0.006784
0.006784
0.046741
0.046741
0.046741
0.000687
0.000687
0.000687
0.197387

SSE
0.007485
0.000118
0.009486
0.010400
0.001563
0.003899
0.037657
0.014346
0.098489
0.037785
0.014066
0.005743
0.241038

Within Season SS
(Y-Group Mean)2

CRD with Non-Normal Data

Kruskal-Wallis Test
Extension of Wilcoxon Rank-Sum Test to k > 2 Groups
Procedure:
Rank the observations across groups from smallest (1) to largest
( N = n1+...+nk ), adjusting for ties
Compute the rank sums for each group: T1,...,Tk . Note that
T1+...+Tk = N(N+1)/2

Kruskal-Wallis Test
H0: The k population distributions are identical (1=...=k)
HA: Not all k distributions are identical (Not all i are equal)
2

12
k Ti
T .S . : H
3( N 1)

i 1
N ( N 1)
ni
R.R. : H

2
, k 1

P val : P ( H )
2

An adjustment to H is suggested when there are many ties in the

data. Formula is given on page 344 of O&L.

Example - Seasonal Diet Patterns in Ravens

Month
NOV
DEC
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT

Season
1
1
1
2
2
2
3
3
3
4
4
4

Y'
1.329721
1.254080
1.145808
1.115957
1.257474
1.280374
1.113428
1.039152
0.605545
0.967390
1.280374
1.237554

H 0 : No seasonal difference

Rank
12
8
6
5
9
10.5
4
3
1
2
10.5
7

T1 = 12+8+6 = 26
T2 = 5+9+10.5 = 24.5
T3 = 4+3+1 = 8
T4 = 2+10.5+7 = 19.5

H a : Seasonal Differences

(26) 2 (24.5) 2 (8) 2 (19.5) 2

12
T .S . : H

3(12 1) 44.12 39 5.12

12(12 1) 3
3
3
3
R.R.( 0.05) : H .205, 41 7.815
P value : P ( 2 H 5.12) .1632

Post-hoc Comparisons of Treatments

If differences in group means are determined from the Ftest, researchers want to compare pairs of groups. Three
popular methods include:
Fishers LSD - Upon rejecting the null hypothesis of no
differences in group means, LSD method is equivalent to doing
pairwise comparisons among all pairs of groups as in Chapter 6.
Tukeys Method - Specifically compares all t(t-1)/2 pairs of
groups. Utilizes a special table (Table 11, p. 701).
Bonferronis Method - Adjusts individual comparison error rates so
that all conclusions will be correct at desired
confidence/significance level. Any number of comparisons can be
made. Very general approach can be applied to any inferential
problem

Fishers Least Significant Difference Procedure

Protected Version is to only apply method after
significant result in overall F-test
For each pair of groups, compute the least significant
difference (LSD) that the sample means need to differ
by to conclude the population means are not equal

LSDij t / 2

1 1
MSE

n n
j
i

with df N t

Conclude i j if y i. y j . LSDij

Fisher' s Confidence Interval : y i. y j . LSDij

Tukeys W Procedure
More conservative than Fishers LSD (minimum
significant difference and confidence interval width are
higher).
Derived so that the probability that at least one false
difference is detected is (experimentwise error rate)
Wij q (t , )

MSE
n

q given in Table 11, p. 701 with N-t

Conclude i j if y i. y j . Wij

Tukey' s Confidence Interval : y i. y j . Wij

When the sample sizes are unequal, use n

t
1
1

n1
nt

Bonferronis Method (Most General)

Wish to make C comparisons of pairs of groups with simultaneous
confidence intervals or 2-sided tests
When all pair of treatments are to be compared, C = t(t-1)/2
Want the overall confidence level for all intervals to be correct to
be 95% or the overall type I error rate for all tests to be 0.05
For confidence intervals, construct (1-(0.05/C))100% CIs for the
difference in each pair of group means (wider than 95% CIs)
Conduct each test at =0.05/C significance level (rejection region
cut-offs more extreme than when =0.05)
Critical t-values are given in table on class website, we will use
notation: t/2,C, where C=#Comparisons, = df

Bonferronis Method (Most General)

1 1
Bij t / 2,C ,v MSE

n n
j
i
(t given on class website with v N-t )
Conclude i j if y i. y j . Bij

Bonferroni' s Confidence Interval : y i. y j . Bij

Example - Seasonal Diet Patterns in Ravens

Note: No differences were found, these calculations are only
for demonstration purposes
MSE 0.03013 ni 3 t.025,8 2.306 q.05,t 4,df E 8 4.53 t.025,C 6,df E 8 3.479
1 1
LSDij 2.306 (0.03013) 0.3268
3 3
1
Wij 4.53 (0.03013) 0.4540
3
1 1
Bij 3.479 (0.03013) 0.4930
3 3

Comparison(i vs j) Group i Mean Group j MeanDifference

1 vs 2
1.243203
1.217935
0.025267
1 vs 3
1.243203
0.919375
0.323828
1 vs 4
1.243203
1.161773
0.081430
2 vs 3
1.217935
0.919375
0.298560
2 vs 4
1.217935
1.161773
0.056162
3 vs 4
0.919375
1.161773
-0.242398

Randomized Block Design (RBD)

t > 2 Treatments (groups) to be compared
b Blocks of homogeneous units are sampled. Blocks can
be individual subjects. Blocks are made up of t subunits
Subunits within a block receive one treatment. When
subjects are blocks, receive treatments in random order.
Outcome when Treatment i is assigned to Block j is
labeled Yij
Effect of Trt i is labeled i
Effect of Block j is labeled j
Random error term is labeled ij
Efficiency gain from removing block-to-block variability
from experimental error

Randomized Complete Block Designs

Model:

Yij i j ij i j ij
t

i 1

0 E ( ij ) 0

V ( ij )

Test for differences among treatment effects:

H0: 1t 0
HA: Not all i = 0

(1t )
(Not all i are equal)

Typically not interested in measuring block effects (although

sometimes wish to estimate their variance in the population of
blocks). Using Block designs increases efficiency in making
inferences on treatment effects

RBD - ANOVA F-Test (Normal Data)

Data Structure: (t Treatments, b Subjects)
Mean for Treatment i:

y i.

Mean for Subject (Block) j:

Overall Mean:

y. j

y ..

Overall sample size: N = bt

ANOVA:Treatment, Block, and Error Sums of Squares

TSS i 1 j 1 yij y ..
t

SSB t y
SSE y

df Total bt 1

SST b i 1 y i . y ..

df T t 1

df B b 1

j 1

.j

ij

..

y i. y . j y ..

TSS SST SSB

df E (b 1)(t 1)

RBD - ANOVA F-Test (Normal Data)

ANOVA Table:
Source
Treatments
Blocks
Error
Total

SS
SST
SSB
SSE
TSS

df
t-1
b-1
(b-1)(t-1)
bt-1

H0: 1t 0
HA: Not all i = 0

T .S . : Fobs
R.R. : Fobs

MS
MST = SST/(t-1)
MSB = SSB/(b-1)
MSE = SSE/[(b-1)(t-1)]

(1t )
(Not all i are equal)

MST

MSE
F ,t 1,( b 1)( t 1)

P val : P ( F Fobs )

F
F = MST/MSE

Pairwise Comparison of Treatment Means

Tukeys Method- q in Studentized Range Table with = (b-1)(t-1)
MSE
Wij q (t , v)
b
Conclude i j if y i. y j . Wij

Tukey' s Confidence Interval : y i. y j . Wij

Bonferronis Method - t-values from table on class

website with = (b-1)(t-1) and C=t(t-1)/2
Bij t / 2,C ,v

2 MSE
b

Conclude i j if y i. y j . Bij

Bonferroni' s Confidence Interval : y i. y j . Bij

Expected Mean Squares / Relative Efficiency

Expected Mean Squares: As with CRD, the Expected Mean
Squares for Treatment and Error are functions of the sample
sizes (b, the number of blocks), the true treatment effects ( 1,
,t) and the variance of the random error terms (2)
By assigning all treatments to units within blocks, error
variance is (much) smaller for RBD than CRD (which
combines block variation&random error into error term)
Relative Efficiency of RBD to CRD (how many times as
many replicates would be needed for CRD to have as precise
of estimates of treatment means as RBD does):

MSECR (b 1) MSB b(t 1) MSE

RE ( RCB, CR )

MSERCB
(bt 1) MSE

Example - Caffeine and Endurance

Treatments: t=4 Doses of Caffeine: 0, 5, 9, 13 mg

Blocks: b=9 Well-conditioned cyclists
Response: yij=Minutes to exhaustion for cyclist j @ dose i
Data:

Dose \ Subject
0
5
9
13

1
36.05
42.47
51.50
37.55

2
52.47
85.15
65.00
59.30

3
56.55
63.20
73.10
79.12

4
45.20
52.10
64.40
58.33

5
35.25
66.20
57.45
70.54

6
66.38
73.25
76.49
69.47

7
40.57
44.50
40.55
46.48

8
57.15
57.17
66.47
66.35

9
28.34
35.05
33.17
36.20

Plot of Y versus Subject by Dose

90.00

80.00

70.00

Time to Exhaustion

60.00

50.00

0 mg
5 mg
9mg

40.00

13 mg

30.00

20.00

10.00

0.00
0

5
Cyclist

10

Example - Caffeine and Endurance

Subject\Dose
1
2
3
4
5
6
7
8
9
Dose Mean
Dose Dev
Squared Dev
TSS

0mg
36.05
52.47
56.55
45.20
35.25
66.38
40.57
57.15
28.34
46.44
-8.80
77.38

5mg
42.47
85.15
63.20
52.10
66.20
73.25
44.50
57.17
35.05
57.68
2.44
5.95

9mg
51.50
65.00
73.10
64.40
57.45
76.49
40.55
66.47
33.17
58.68
3.44
11.86

13mg
37.55
59.30
79.12
58.33
70.54
69.47
46.48
66.35
36.20
58.15
2.91
8.48

Subj MeanSubj Dev Sqr Dev

41.89
-13.34
178.07
65.48
10.24
104.93
67.99
12.76
162.71
55.01
-0.23
0.05
57.36
2.12
4.51
71.40
16.16
261.17
43.03
-12.21
149.12
61.79
6.55
42.88
33.19
-22.05
486.06
55.24
1389.50
103.68

7752.773

TSS (36.05 55.24) 2 (36.20 55.24) 2 7752.773 dfTotal 4(9) 1 35

SSB 4(41.89 55.24)

(33.19 55.24) 4(1389.50) 5558.00

SST 9 (46.44 55.24) 2 (58.15 55.24) 2 9(103.68) 933.12 dfT 4 1 3

2

df B 9 1 8

SSE (36.05 41.89 46.44 55.24) 2 (36.20 33.19 58.15 55.24) 2

TSS SST SSB 7752.773 933.12 5558 1261.653 df E (4 1)(9 1) 24

Example - Caffeine and Endurance

Source
Dose
Cyclist
Error
Total

df
3
8
24
35

SS
933.12
5558.00
1261.65
7752.77

MS
311.04
694.75
52.57

H 0 : No Caffeine Dose Effect (1 4 0)

H A : Differences Exist Among Doses
MST 311.04
T .S . : Fobs

5.92
MSE 52.57
R.R.( 0.05) : Fobs F.05,3, 24 3.01
P value : P ( F 5.92) .0036 (From EXCEL)
Conclude that true means are not all equal

F
5.92

Example - Caffeine and Endurance

Tukey' s W : q.05, 4, 24

1
3.90 W 3.90 52.57 9.43
9

Bonferroni' s B : t.05 / 2, 6, 24
Doses
5mg vs 0mg
9mg vs 0mg
13mg vs 0mg
9mg vs 5mg
13mg vs 5mg
13mg vs 9mg

2
2.875 B 2.875 52.57 9.83
9

High Mean
57.6767
58.6811
58.1489
58.6811
58.1489
58.1489

Low Mean Difference Conclude

46.4400
11.2367

46.4400
12.2411

46.4400
11.7089

57.6767
1.0044
NSD
57.6767
0.4722
NSD
58.6811
-0.5322
NSD

Example - Caffeine and Endurance

Relative Efficiency of Randomized Block to Completely Randomized Design :
t 4 b 9 MSB 694.75 MSE 52.57
RE ( RCB, CR )

(b 1) MSB b(t 1) MSE 8(694.75) 9(3)(52.57) 6977.39

3.79
(bt 1) MSE
(9(4) 1)(52.57)
1839.95

Would have needed 3.79 times as many cyclists per dose to have the
same precision on the estimates of mean endurance time.
9(3.79) 35 cyclists per dose
4(35) = 140 total cyclists

RBD -- Non-Normal Data

Friedmans Test
When data are non-normal, test is based on ranks
Procedure to obtain test statistic:
Rank the k treatments within each block (1=smallest,
k=largest) adjusting for ties
Compute rank sums for treatments (Ti) across blocks
H0: The k populations are identical (1=...=k)
HA: Differences exist among the k group means
12
k
2
T .S . : Fr
T
3b(k 1)

i 1 i
bk (k 1)
R.R. : Fr 2 ,k 1
P val : P( 2 Fr )

Example - Caffeine and Endurance

Subject\Dose
1
2
3
4
5
6
7
8
9

0mg
36.05
52.47
56.55
45.2
35.25
66.38
40.57
57.15
28.34

5mg
42.47
85.15
63.2
52.1
66.2
73.25
44.5
57.17
35.05

9mg
51.5
65
73.1
64.4
57.45
76.49
40.55
66.47
33.17

13mg
37.55
59.3
79.12
58.33
70.54
69.47
46.48
66.35
36.2

Ranks

Total

0mg
1
1
1
1
1
1
2
1
1
10

5mg
3
4
2
2
3
3
3
2
3
25

9mg
4
3
3
4
2
4
1
4
2
27

13mg
2
2
4
3
4
2
4
3
4
28

H 0 : No Dose Differences
H a : Dose Differences Exist

12
26856
2
2
T .S . : Fr
(10) (28) 3(9)(4 1)
135 14.2
9(4)(4 1)
180
R.R.( 0.05) : Fr .205, 41 7.815
P - value : P ( 2 14.2) .0026 (From EXCEL)
Conclude Means (Medians) are not all equal

Latin Square Design

Design used to compare t treatments when there are
two sources of extraneous variation (types of blocks),
each observed at t levels
Best suited for analyses when t 10
Classic Example: Car Tire Comparison
Treatments: 4 Brands of tires (A,B,C,D)
Extraneous Source 1: Car (1,2,3,4)
Extraneous Source 2: Position (Driver Front, Passenger
Front, Driver Rear, Passenger Rear)
Car\Position
1
2
3
4

DF
A
B
C
D

PF
B
C
D
A

DR
C
D
A
B

PR
D
A
B
C

Latin Square Design - Model

Model (t treatments, rows, columns, N=t2) :

yijk k i k ijk

Overall Mean

y ...
^

k Effect of Treatment k k y ..k y ...

i

Effect due to row i

i y i.. y ...
^

Effect due to Column j j y . j . y ...

ijk

Random Error Term

Latin Square Design - ANOVA & F-Test

t

Total Sum of Squares : TSS yijk y ...

i 1 j 1

df t 2 1

Treatment Sum of Squares SST t y ..k y ...

k 1

Row Sum of Squares SSR t y i.. y ...

i 1

dfT t 1

Column Sum of Squares SSC t y . j . y ...

j 1

df R t 1

df C t 1

Error Sum of Squares SSE TSS SST SSR SSC

H0: 1 = = t = 0

df E (t 2 1) 3(t 1) (t 1)(t 2)

Ha: Not all k = 0

TS: Fobs = MST/MSE = (SST/(t-1))/(SSE/((t-1)(t-2)))

RR: Fobs F, t-1, (t-1)(t-2)

Pairwise Comparison of Treatment Means

Tukeys Method- q in Studentized Range Table with = (t-1)(t-2)
MSE
Wij q (t , v)
t
Conclude i j if y i. y j . Wij

Tukey' s Confidence Interval : y i. y j . Wij

Bonferronis Method - t-values from table on class

website with = (t-1)(t-2) and C=t(t-1)/2
Bij t / 2,C ,v

2 MSE
t

Conclude i j if y i. y j . Bij

Bonferroni' s Confidence Interval : y i. y j . Bij

Expected Mean Squares / Relative Efficiency

Expected Mean Squares: As with CRD, the Expected Mean
Squares for Treatment and Error are functions of the sample
sizes (t, the number of blocks), the true treatment effects (1,
,t) and the variance of the random error terms (2)
By assigning all treatments to units within blocks, error
variance is (much) smaller for LS than CRD (which combines
block variation&random error into error term)
Relative Efficiency of LS to CRD (how many times as many
replicates would be needed for CRD to have as precise of
estimates of treatment means as LS does):

MSECR MSR MSC (t 1) MSE

RE ( LS , CR )

MSELS
(t 1) MSE

2-Way ANOVA
2 nominal or ordinal factors are believed to
be related to a quantitative response
Additive Effects: The effects of the levels of
each factor do not depend on the levels of
the other factor.
Interaction: The effects of levels of each
factor depend on the levels of the other
factor
Notation: ij is the mean response when
factor A is at level i and Factor B at j

2-Way ANOVA - Model

yijk i j ij ijk

i 1,..., a

j 1,..., b k 1,..., r

yijk Measurement on k th unit receiving Factors A at level i, B at level j

Overall Mean
i Effect of i th level of factor A
j Effect of j th level of factor B
ij Interaction effect when i th level of A and j th level of B are combined
ijk Random Error Terms

Model depends on whether all levels of interest for a factor are

included in experiment:
Fixed Effects: All levels of factors A and B included
Random Effects: Subset of levels included for factors A and B
Mixed Effects: One factor has all levels, other factor a subset

Fixed Effects Model

Factor A: Effects are fixed constants and sum to 0
Factor B: Effects are fixed constants and sum to 0
Interaction: Effects are fixed constants and sum to 0
over all levels of factor B, for each level of factor A,
and vice versa
Error Terms: Random Variables that are assumed to be
independent and normally distributed with mean 0,
variance 2
a

i 0,
i 1

j 0
j 1

ij 0 j
i 1

~
N
0
,

ij
ijk

b

j 1

Example - Thalidomide for AIDS

Response: 28-day weight gain in AIDS patients

Factor A: Drug: Thalidomide/Placebo
Factor B: TB Status of Patient: TB+/TBSubjects: 32 patients (16 TB+ and 16 TB-).
Random assignment of 8 from each group to
each drug). Data:

Thalidomide/TB+: 9,6,4.5,2,2.5,3,1,1.5
Thalidomide/TB-: 2.5,3.5,4,1,0.5,4,1.5,2
Placebo/TB+: 0,1,-1,-2,-3,-3,0.5,-2.5
Placebo/TB-: -0.5,0,2.5,0.5,-1.5,0,1,3.5

ANOVA Approach
Total Variation (TSS) is partitioned into 4
components:
Factor A: Variation in means among levels of A
Factor B: Variation in means among levels of B
Interaction: Variation in means among combinations
of levels of A and B that are not due to A or B alone
Error: Variation among subjects within the same
combinations of levels of A and B (Within SS)

Analysis of Variance
a

Total Variation : TSS yijk y ...

i 1 j 1 k 1

df Total abr 1

Factor A Sum of Squares : SSA br y i.. y ...

i 1
b

Factor B Sum of Squares : SSB ar y . j . y ...

j 1

df A a 1

df B b 1

Interaction Sum of Squares : SSAB r y ij . y i.. y . j . y ...

i 1 j 1

Error Sum of Squares : SSE yijk y ij .

i 1 j 1 k 1

TSS = SSA + SSB + SSAB + SSE

dfTotal = dfA + dfB + dfAB + dfE

df E ab( r 1)

df AB (a 1)(b 1)

ANOVA Approach - Fixed Effects

Procedure:
First test for interaction effects
If interaction test not significant, test for Factor A and B effects
Test for Interaction :
H 0 : 11 ... ab 0

Test for Factor A

H 0 : 1 ... a 0

Test for Factor B

H 0 : 1 ... b 0

H a : Not all ij 0

H a : Not all i 0

H a : Not all j 0

MSAB
MSE
F ,( a 1)(b 1),ab ( r 1)

TS : FAB
RR : FAB

MSA
MSB
TS : FB
MSE
MSE
RR : FA F ,( a 1),ab ( r 1) RR : FB F ( b 1),ab ( r 1)

TS : FA

Example - Thalidomide for AIDS

Individual Patients

Group Means

tb

7.5

Negative

Positive

3.000

2.000

5.0

2.5

0.0

-2.5

1.000

0.000

-1.000

Placebo

Placebo

Thalidomide

Thalidomide

drug

drug

Report
WTGAIN
GROUP
TB+/Thalidomide
TB-/Thalidomide
TB+/Placebo
TB-/Placebo
Total

Mean
3.688
2.375
-1.250
.688
1.375

N
8
8
8
8
32

Std. Deviation
2.6984
1.3562
1.6036
1.6243
2.6027

Example - Thalidomide for AIDS

Tests of Between-Subjects Effects
Dependent Variable: WTGAIN
Type III Sum
Source
of Squares
df
Mean Square
Corrected Model
109.688a
3
36.563
Intercept
60.500
1
60.500
DRUG
87.781
1
87.781
TB
.781
1
.781
DRUG * TB
21.125
1
21.125
Error
100.313
28
3.583
Total
270.500
32
Corrected Total
210.000
31
a. R Squared = .522 (Adjusted R Squared = .471)

F
10.206
16.887
24.502
.218
5.897

Sig.
.000
.000
.000
.644
.022

There is a significant Drug*TB interaction (FDT=5.897, P=.022)

The Drug effect depends on TB status (and vice versa)

Comparing Main Effects (No Interaction)

Tukeys Method- q in Studentized Range Table with = ab(r-1)
MSE
Wij q (a, v)
br

MSE
Wij q (b, v)
ar

Conclude : i j if y i.. y j .. WijA

i j if y .i. y . j . W ijB

Tukey' s CI : ( i j ) : y i.. y j .. W ijA

( i j ) : y .i. y . j . WijB

Bonferronis Method - t-values in Bonferroni table with =ab (r-1)

B ijA t / 2,a ( a 1) / 2,v

2 MSE
br

B ijB t / 2,b (b 1) / 2,v

Conclude : i j if y i.. y j .. B ijA

2 MSE
ar

i j if y .i. y . j . B ijB

Bonferroni' s CI : (i - j ) : y i.. y j .. B ijA

( i - j ) : y .i. y . j . B ijB

Comparing Main Effects (Interaction)

Tukeys Method- q in Studentized Range Table with = ab(r-1)
MSE
Wij q (a, v)
r
A

Within k th level of Factor B, Conclude : i j if y ik . y jk . W ijA

Tukey' s CI : ( i j ) : y ik . y jk . WijA Similar for Factor B in A

Bonferronis Method - t-values in Bonferroni table with =ab (r-1)

B ij t / 2,a ( a 1) / 2,v
A

2 MSE
r

Within k th level of B, Conclude : i j if y ik . y jk . B ijA

Bonferroni' s CI : (i - j ) : y ik . y jk . B ijA

Miscellaneous Topics
2-Factor ANOVA can be conducted in a Randomized
Block Design, where each block is made up of ab
experimental units. Analysis is direct extension of
RBD with 1-factor ANOVA
Factorial Experiments can be conducted with any
number of factors. Higher order interactions can be
formed (for instance, the AB interaction effects may
differ for various levels of factor C).
When experiments are not balanced, calculations are
immensely messier and you must use statistical
software packages for calculations

Mixed Effects Models

Assume:
Factor A Fixed (All levels of interest in study)

Factor B Random (Sample of levels used in study)

j ~ N(0,b2)

(Independent)

AB Interaction terms Random

)ij ~ N(0ab2(Independent)

Analysis of Variance is computed exactly as in

Fixed Effects case (Sums of Squares, dfs, MSs)
Error terms for tests change (See next slide).

ANOVA Approach Mixed Effects

Procedure:
First test for interaction effects
If interaction test not significant, test for Factor A and B effects
Test for Interaction :

Test for Factor A

Test for Factor B

2
H 0 : ab
0

H 0 : 1 ... a 0

H 0 : b2 0

2
H a : ab
0

H a : Not all i 0

H a : b2 0

MSAB
MSE
F ,( a 1)(b 1),ab ( r 1)

TS : FAB
RR : FAB

MSA
MSB
TS : FB
MSAB
MSAB
RR : FA F ,( a 1),( a 1)(b 1) RR : FB F ,(b 1),( a 1)(b 1)

TS : FA

Comparing Main Effects for A (No Interaction)

Tukeys Method- q in Studentized Range Table with = (a-1)(b-1)
MSAB
W ij q (a, v)
br
A

Conclude : i j if y i.. y j .. W ijA

Tukey' s CI : ( i j ) : y i.. y j .. WijA

Bonferronis Method - t-values in Bonferroni table with = (a-1)(b-1)

B ijA t / 2,a ( a 1) / 2,v

2 MSAB
br

Conclude : i j if y i.. y j .. B ijA

Bonferroni' s CI : (i - j ) : y i.. y j .. B ijA

Random Effects Models

Assume:
Factor A Random (Sample of levels used in study)
i ~ N(0,a2)

(Independent)

Factor B Random (Sample of levels used in study)

j ~ N(0,b2)

(Independent)

AB Interaction terms Random

)ij ~ N(0ab2(Independent)

Analysis of Variance is computed exactly as in

Fixed Effects case (Sums of Squares, dfs, MSs)
Error terms for tests change (See next slide).

ANOVA Approach Mixed Effects

Procedure:
First test for interaction effects
If interaction test not significant, test for Factor A and B effects
Test for Interaction :
2
H 0 : ab
0
2
H a : ab
0

MSAB
MSE
F ,( a 1)(b 1),ab ( r 1)

TS : FAB
RR : FAB

Test for Factor A

Test for Factor B

H 0 : a2 0

H 0 : b2 0

H a : a2 0

H a : b2 0

MSA
MSB
TS : FB
MSAB
MSAB
RR : FA F ,( a 1),( a 1)(b 1) RR : FB F ( b 1),( a 1)(b 1)

TS : FA

Nested Designs
Designs where levels of one factor are nested (as
opposed to crossed) wrt other factor
Examples Include:

Classrooms nested within schools

Litters nested within Feed Varieties
Hair swatches nested within shampoo types
Swamps of varying sizes (e.g. large, medium, small)
Restaurants nested within national chains

Nested Design - Model

Yijk i j (i ) ijk

i 1,...a j 1,..., bi k 1,..., r

where :
Yijk Response for k th rep of Factor A at i th level, B at jth level within A

Overall Mean
i Effect of i th level of A (Fixed or Random)
j (i ) Effect of jth level of B within i th level of A (Fixed or Random)
ijk Random error term for k th rep when A is at i, B is at j(i)

Nested Design - ANOVA

Total Variation :
bi

TSS Yijk Y ...

i 1 j 1 k 1

dfTotal r bi 1

i 1

Factor A :

SSA r bi Y i.. Y ...

i 1

df A a 1

Factor B Nested Within A

bi

SSB ( A) r Y ij . Y i..
i 1 j 1

df B ( A) bi a
i 1

Error :
a

bi

SSE Yijk Y ij .
i 1 j 1 k 1

df E (r 1) bi
i 1

Factors A and B Fixed

a

i 1

bi

j 1

j (i )

0 i 1,..., a ijk ~ N 0, 2

Tests for Differences Among Factor A Effects

H 0 : 1 ... a 0 H A : Not all i 0
MSA
Test Statistic : FA
P - value : P F FA
MSE
Rejection Region : FA F ,a 1,( r 1) b
i
Tests for Differences Among Factor B Effects
H 0 : j ( i ) 0 i, j

H A : Not all j ( i ) 0

MSB( A)
P - value : P F FB ( A)
MSE
Rejection Region : FB ( A) F , b a ,( r 1) b
i
i

Test Statistic : FB ( A)

Comparing Main Effects for A

Tukeys Method- q in Studentized Range Table with = (r-1)bi
MSE 1
1
W ij q (a, v)

2 rbi rb j
A

Conclude : i j if y i.. y j .. W ijA

Tukey' s CI : ( i j ) : y i.. y j .. WijA

Bonferronis Method - t-values in Bonferroni table with = (r-1)bi

B ij t / 2,a ( a 1) / 2,v
A

MSE

rb rb
j
i

Conclude : i j if y i.. y j .. B ijA

Bonferroni' s CI : (i - j ) : y i.. y j .. B ijA

Comparing Effects for Factor B Within A

Tukeys Method- q in Studentized Range Table with = (r-1)bi
B

W ij ( k )

MSE
q (bk , v)
r

Conclude : i ( k ) j ( k ) if y ki. y kj . WijB( k )

Tukey' s CI : ( i ( k ) j ( k ) ) : y ki. y kj . W ijB( k )

Bonferronis Method - t-values in Bonferroni table with = (r-1)bi

B ij ( k ) t / 2,bk ( bk 1) / 2,v
B

2
MSE
r

Conclude : i ( k ) j ( k ) if y ki. y kj . B ijB( k )

Bonferroni' s CI : ( i ( k ) - j ( k ) ) : y ki. y kj . B ijB( k )

Factor A Fixed and B Random

2
2

~
N

0
,

~
N

0
,

i
j (i )
b
ijk
a

i 1

Tests for Differences Among Factor A Effects

H 0 : 1 ... a 0 H A : Not all i 0
MSA
Test Statistic : FA
P - value : P F FA
MSB( A)
Rejection Region : FA F ,a 1, b a
i
Tests for Differences Among Factor B Effects
H 0 : b2 0 i, j

H A : b2 0

MSB( A)
P - value : P F FB ( A)
MSE
Rejection Region : FB ( A) F , b a ,( r 1) b
i
i

Test Statistic : FB ( A)

Comparing Main Effects for A (B Random)

Tukeys Method- q in Studentized Range Table with = bi-a
MSB( A) 1
1
W ij q (a, v)

2
rbi rb j
A

Conclude : i j if y i.. y j .. W ijA

Tukey' s CI : ( i j ) : y i.. y j .. WijA

Bonferronis Method - t-values in Bonferroni table with = bi-a

B ij t / 2,a ( a 1) / 2,v
A

MSB( A)

rb rb
j
i

Conclude : i j if y i.. y j .. B ijA

Bonferroni' s CI : (i - j ) : y i.. y j .. B ijA

Factors A and B Random

i ~ N 0, a2 j (i ) ~ N 0, b2 ijk ~ N 0, 2
Tests for Differences Among Factor A Effects
H 0 : a2 0 H A : a2 0
MSA
Test Statistic : FA
P - value : P F FA
MSB( A)
Rejection Region : FA F ,a 1, b a
i
Tests for Differences Among Factor B Effects
H 0 : b2 0 i, j

H A : b2 0

MSB( A)
P - value : P F FB ( A)
MSE
Rejection Region : FB ( A) F , b a ,( r 1) b
i
i

Test Statistic : FB ( A)

Elements of Split-Plot Designs

Split-Plot Experiment: Factorial design with at least 2
factors, where experimental units wrt factors differ in
size or observational points.
Whole plot: Largest experimental unit
Whole Plot Factor: Factor that has levels assigned to
whole plots. Can be extended to 2 or more factors
Subplot: Experimental units that the whole plot is split
into (where observations are made)
Subplot Factor: Factor that has levels assigned to
subplots
Blocks: Aggregates of whole plots that receive all levels
of whole plot factor

Split Plot Design

Note: Within each block we would assign at random the 3

levels of A to the whole plots and the 4 levels of B to the
subplots within whole plots

Examples
Agriculture: Varieties of a crop or gas may need to be
grown in large areas, while varieties of fertilizer or
varying growth periods may be observed in subsets of
the area.
Engineering: May need long heating periods for a
process and may be able to compare several
formulations of a by-product within each level of the
heating factor.
Behavioral Sciences: Many studies involve repeated
measurements on the same subjects and are analyzed as
a split-plot (See Repeated Measures lecture)

Design Structure
Blocks: b groups of experimental units to be exposed
to all combinations of whole plot and subplot factors
Whole plots: a experimental units to which the whole
plot factor levels will be assigned to at random within
blocks
Subplots: c subunits within whole plots to which the
subplot factor levels will be assigned to at random.
Fully balanced experiment will have n=abc
observations

Data Elements (Fixed Factors, Random Blocks)

Yijk: Observation from wpt i, block j, and spt k
: Overall mean level
i : Effect of ith level of whole plot factor (Fixed)
bj: Effect of jth block (Random)

(ab )ij : Random error corresponding to whole plot elements

in block j where wpt i is applied
k: Effect of kth level of subplot factor (Fixed)
)ik: Interaction btwn wpt i and spt k
(bc )jk: Interaction btwn block j and spt k (often set to 0)
ijk: Random Error= (bc )jk+ (abc)ijk
Note that if block/spt interaction is assumed to be 0,
represents the block/spt within wpt interaction

Model and Common Assumptions

Yijk = + i + b j + (ab )ij + k + ( )ik + ijk
a

i 1

b j ~ NID (0, b2 )
2
( ab) ij ~ NID (0, ab
)
c

k 1

( )
i 1

ik

( ) ik 0
k 1

ijk ~ NID (0. 2 )

COV (b j , ( ab) ij ) COV (b j , ijk ) COV ((ab) ij , ijk ) 0

Tests for Fixed Effects

Whole Plot Trt Effects : H 0 : 1 a 0
Test Statistic : FWP

MSWP
MS BLOCK *WP

PWP P ( F FWP | F ~ Fa 1.( a 1)( b 1) )

Subplot Trt Effects : H 0 : 1 c 0
Test Statistic : FSP

MS SP
MS ERROR

PSP P ( F FSP | F ~ Fc 1.a ( b 1)( c 1) )

WP SP Interaction : H 0 : ( ) ik 0 i , k
Test Statistic : FWP SP

MSWP SP

MS ERROR

PWP SP P ( F FWP SP | F ~ F( a 1)( c 1).a ( b 1)( c 1) )

Comparing Factor Levels

Whole Plot Factor Levels :
95% CI for i i ' :

i ..

Y i '.. t

2MS BLOCK WP
bc

Sub Plot Factor Levels :

2MS ERROR
ab
Sub Plot Effects Within same whole plot (Interaction) :
95% CI for ( k k ' ) :

..k

Y ..k ' t

95% CI for k k ' ( ) ik ( ) ik ' :

i .k

Y i .k ' t

2MS ERROR
b

Whole Plot Effects within same sub plot (Interaction) :

Y i '.k t

i .k

2 MS BLOCK WP (c 1) MS ERROR
bc

(c 1) MS ERROR MS BLOCK WP 2
(c 1) MS ERROR 2 MS BLOCK WP 2
a (b 1)(c 1)

(a 1)(b 1)

(df given below)

Repeated Measures Designs

a Treatments/Conditions to compare
N subjects to be included in study (each subject
will receive only one treatment)
n subjects receive trt i: an = N

t time periods of data will be obtained

Effects of trt, time and trtxtime interaction of
primary interest.
Between Subject Factor: Treatment
Within Subject Factors: Time, TrtxTime

Model
Yijk i b j (i ) k ( ) ik ijk

overall mean
i effect of trt i

i 1

b j (i ) effect of j th subject in trt i b j (i ) ~ NID 0, b2

k effect of k time period

th

k 1

( ) ik interaction between trt i and time k

ijk random error term ijk ~ NID 0, 2

( )
i 1

ik

( ) ik 0
k 1

Note the random error term is actually the interaction between

subjects (within treatments) and time

Tests for Fixed Effects

Treatment Effects : H 0 : 1 a 0
Test Statistic : FTRTS

MSTRTS
MS SUBJECTS (TRTS )

PTRTS P ( F FTRTS | F ~ Fa 1, a ( n 1) )
Time Effects : H 0 : 1 t 0
Test Statistic : FTIME

MSTIME
MS ERROR

PTIME P ( F FTIME | F ~ Ft 1,a ( n 1)(t 1) )

Treatment/ Time Interaction : H 0 : ( ) ik 0 i, k
Test Statistic : FTRT TIME

MSTRT TIME
MS ERROR

PTRT TIME P ( F FTRT TIME | F ~ F( a 1)( t 1),a ( n 1)(t 1) )

Comparing Factor Levels

Comparing Treatment Levels :
95% CI for i i ' :

i ..

Y i '.. t

2MS SUBJECTS (TRTS )

nt

Comparing Time Levels :

2MS ERROR
..k
an
Comparing Treatment Levels Within Time Levels :

95% CI for k k ' :

i.k

Y i '.k t

Y ..k ' t

2 MS SUBJECTS (TRTS ) (t 1) MS ERROR

nt

with approximate df :
^

(t 1)MS

ERROR MS SUBJECT (TRT )

(t 1) MS ERROR 2 MS SUBJECT (TRT )

a(n 1)
a(n 1)(t 1)