Analysis of Variance

Analysis of Variance
Analysis of variance is a technique that
partitions the total sum of squares of
deviations of the observations about their
mean into portions associated with
independent variables in the experiment
and a portion associated with error
 Analysis of Variance

A factor refers to a categorical quantity under

examination in an experiment as a possible
cause of variation in the response variable.

Levels refer to the categories, measurements,

or strata of a factor of interest in the
experiment.
One way Analysis of Variance
 F-Test

1. Tests the Equality of 2 or more (p)

Population Means

2. Variables
⚫ One Nominal Independent Variable
⚫ One Continuous Dependent Variable
 Assumptions

1. Randomness & Independence of Errors

2. Normality
• Populations (for each condition) are Normally
Distributed
3. Homogeneity of Variance
• Populations (for each condition) have Equal
Variances
 Hypotheses

H0: µ1 = µ2 = µ3 = ... = µp
➢ All population means are equal
➢ No Treatment effect

H1: Not all µj are equal (µ1≠µ2 ≠ ... ≠ µp)

➢ At least 1 population mean is different
➢ Treatment Effect
 Hypotheses
f(X)
H0: µ1 = µ2 = µ3 = ... = µp
⚫ All population means are
equal X
⚫ No treatment effect
1 = 2 = 3

H1: Not all µj are equal f(X)

• At least 1 population mean
is different
• Treatment Effect X
1 =  2  3
 Total variation

Variation due to Variation due to

treatment
Sum of Squares Among
Sum of Squares Between
Sum of Squares Treatment
(SST)
Among Groups Variation
random sampling
Sum of Squares Within
Sum of Squares Error
(SSE)
Within Groups Variation
 Total Variation

SS (Total ) = (Y11 − Y ) + (Y21 − Y ) +  + (Yij − Y )

2 2 2
Y

Response, Y

Y

Group 1 Group 2 Group 3

 Treatment Variation

SST = n1 (Y1 − Y ) + n2 (Y2 − Y ) +  + n p (Y p − Y )

2 2 2

Response, Y
Y3
Y
Y2
Y1

Group 1 Group 2 Group 3

 Random (Error) Variation
SSE = (Y11 − Y1 ) + (Y21 − Y2 ) + + (Ypj − Yp )
2 2 2

Response, Y

Y3
Y2
Y1

Group 1 Group 2 Group 3

 F-Test - Test Statistic
1. Test Statistic is F = MST / MSE
SST / ( p − 1)
=
SSE / ( n − p )

MST is Mean Square for Treatment

MSE is Mean Square for Error
2. Degrees of Freedom
ν1 = p -1
ν2 = n - p
⚫ p = # Populations, Groups, or Levels

⚫ n = Total Sample Size

 One-Way ANOVA Summary Table
Degrees
Source of Sum of Mean Sum of
of F - Ratio
Variation Squares Squares
Freedom
MSTr =
Treatment p-1 SSTr MSTr/MSE
SSTr/(p-1)
MSE =
Error n-p SSE
SSE/(n-p)
SS(Total) =
Total n-1
SSTr+ SSE
 Critical Value

If means are equal,

F = MST / MSE ≈1.
Only reject large F ! Reject H 0

Do Not 
Reject H0

0 F
Fa ( p - 1, n - p )

Always One-Tail!
 Example
As a vet epidemiologist you
want to see if 3 food Food1 Food2 Food3
supplements have different 25.40 23.40 20.00
mean milk yields. You assign 26.31 21.80 22.20
15 cows, 5 per food 24.10 23.50 19.75
supplement. 23.74 22.75 20.60
Question: At the 0.05 level, is 25.10 21.60 20.40
there a difference in mean
yields?
 Solution
H0: 1 = 2 = 3
Test Statistic:
Ha: Not All Equal
F = MST/MSE
 = 0.05 = 23.5820/0.9211 = 25.6
1 = 2 2 = 12
Critical Value: Decision:
Reject at α = 0.05

Conclusion:
 = 0.05
There is evidence that population
means are different
0 3.89 F
 Summary Table - Solution

Source of Degrees of Sum of Mean F

Variation Freedom Squares Square
(Variance)
Food 3-1=2 47.1640 23.5820 25.60

Error 15 - 3 = 12 11.0532 .9211

Total 15 - 1 = 14 58.2172
Two way Analysis of Variance
 Two-way ANOVA

1. Experimental Units (Subjects) Are Assigned

Randomly within Blocks
⚫ Blocks are Assumed Homogeneous

2. One Factor or Independent Variable of Interest

⚫ 2 or More Treatment Levels or Classifications

3. One Blocking Factor

 Two-way ANOVA
Factor Levels: Treatments
A, B, C, D
Experimental Treatments are randomly
Units assigned within blocks
Block 1 A C D B
Block 2 C D B A
Block 3 B A D C
. . . .
. . . . .
.
. . . . .
Block b D C A B
 Two-way ANOVA

1.Tests the Equality of 2 or More (p) Population

Means

2.Variables
⚫ One Nominal Independent Variable
⚫ One Nominal Blocking Variable
⚫ One Continuous Dependent Variable
 Two-way ANOVA Assumptions
1.Normality
⚫ Probability Distribution of each Block-
Treatment combination is Normal

2.Homogeneity of Variance
⚫ Probability Distributions of all Block-Treatment
combinations have Equal Variances
Two-way ANOVA Hypotheses

H0: µ1 = µ2 = µ3 = ... = µp
➢ All population means are equal
➢ No Treatment effect

H1: Not all µj are equal (µ1≠µ2 ≠ ... ≠ µp)

➢ At least 1 population mean is different
➢ Treatment Effect
 Two-way ANOVA Hypotheses

H0: 1 = 2 = ... = p f(X)

⚫ All population means

are equal
X
⚫ No treatment effect 1 = 2 = 3
Ha: Not all j are equal
⚫ At least one population
f(X)
mean is different
⚫ Treatment effect
X
⚫ 1  2  ...  p 1 =  2  3
The F Ratio for Two-way ANOVA

SS=SSE+SSB+SST
MST SST / ( p − 1)
F= =
MSE SSE / ( n − 1 − p + 1 − b + 1)
SST / ( p − 1)
=
SSE / ( n − p − b + 1)
Two-way ANOVA F-Test Statistic
1.Test Statistic
⚫ F = MST / MSE
• MST Is Mean Square for Treatment
• MSE Is Mean Square for Error

2.Degrees of Freedom
⚫ ν1 = p -1
⚫ ν 2 = n – b – p +1
• p = # Treatments, b = # Blocks
n = Total Sample Size
Two-way ANOVA F-Test Critical Value

If means are equal,

F = MST / MSE ≈ 1.
Only reject large F ! Reject H0

Do Not α
Reject H0

0 F
Fa ( p - 1, n - p )

Always One-Tail!
 Example
You wish to determine which of four brands of tires has the longest tread
life. You randomly assign one of each brand (A, B, C, and D) to a tire
location on each of 5 cars. At the 0.05 level, is there a difference in mean
tread life?

Tire Location
Left
Block Front Right Front Left Rear Right Rear
Car 1 A: 42,000 C: 58,000 B: 38,000 D: 44,000
Car 2 B: 40,000 D: 48,000 A: 39,000 C: 50,000
Car 3 C: 48,000 D: 39,000 B: 36,000 A: 39,000
Car 4 A: 41,000 B: 38,000 D: 42,000 C: 43,000
Car 5 D: 51,000 A: 44,000 C: 52,000 B: 35,000
 Two-way ANOVA Solution
H0: µ1 = µ2 = µ3= µ4
Ha: Not all are equal Test Statistic: F = 11.9933
α = 0.05
ν1 = 3 ν2 = 12 Decision: Reject at α = 0.05

Critical Value: Conclusion:

There is evidence that the
population means are different.
α = 0.05

F
0 3.49