ANALYSIS OF VARIANCE (ANOVA)

Analysis of Variance, in its simplest form, can be thought of as an extension of the

two-sample t-test to more than two samples. (Further methods in Chapter 8 of Business
Statistics)
As an example: Using the 2-sample t-test we have tested to see whether there was any
difference between the size of invoices in a company's Leeds and Bradford stores. Using
Analysis of variance we can, simultaneously, investigate invoices from as many towns as
we wish, assuming that sufficient data is available.

Problem: Why can't we just carry out repeated t-tests on pairs of the variables?

If many independent tests are carried out pairwise then the probability of being correct
for the combined results is greatly reduced. For example, if we compare the average
marks of two students at the end of a semester to see if their mean scores are significantly
different we would have, at a 5% level, 0.95 probability of being correct. Comparing more
students:

Students Pairwise tests P( all correct) P(at least one incorrect)

2 1 0.95 0.05
3 3 0.953 = 0.875 0.125
n
n {n(n-1)}/2 0.95 1 - 0.95n
10 45 0.9545 = 0.10 0.90

Obviously this situation is not acceptable

Solution: We need therefore to use methods of analysis which will allow the variation
between all n means to be tested simultaneously giving an overall probability of 0.95 of
being correct at the 5% level. This type of analysis is referred to as Analysis of Variance
or 'ANOVA' in short. In general, it:
• measures the overall variation within a variable;
• finds the variation between its group means;
• combines these to calculate a single test statistic;
• uses this to carry out a hypothesis test in the usual manner.

In general, Analysis of Variance, ANOVA, compares the variation between groups and
the variation within samples by analysing their variances.

One-way ANOVA: Is there any difference between the average sales at various
departmental stores within a company?

Two-way ANOVA: Is there any difference between the average sales at various stores
within a company and/or the types of department? The overall variation is split 'two ways'.

1
One-way ANOVA
Total variation
(SST)

Variation due to difference Residual (error) variation not

between the groups, i.e. due to difference between the
between the group means. group means.
(SSG) (SSE)

Two-way ANOVA
Total variation
(SST)

Variation due to difference Residual (error) variation not

between the groups, i.e. due to difference between the
between the group means main group means.
(SSG) (SSE1)

Variation due to difference Genuine residual (error)

between the block means, variation not due to
i.e. second group means difference between either
(SSBl) set of group means (SSE)

where SST = Total Sum of Squares; SSG = Treatment Sum of Squares between the
groups; SSBl = Blocks Sum of Squares; SSE = Sum of Squares of Errors.
(At this stage just think of 'sums of squares' as being a measure of variation.)

The method of measuring this variation is variance, which is standard deviation squared.
Total variance = between groups variance + variance due to the errors

It follows that: Total sum of = Sum of squares between + Sum of squares due
squares (SST) the groups (SSG) to the errors (SSE)

If we find any two of the three sums of squares then the other can be found by difference.

In practice we calculate SST and SSG and then find SSE by difference.

Since the method is much easier to understand with a numerical example, it will be
explained in stages using theory and a numerical example simultaneously.

2
Example 1

One important factor in selecting software for word processing and database management
systems is the time required to learn how to use a particular system. In order to evaluate
three database management systems, a firm devised a test to see how many training hours
were needed for five of its word processing operators to become proficient in each of three
systems.
System A 16 19 14 13 18 hours
System B 16 17 13 12 17 hours
System C 24 22 19 18 22 hours

Using a 5% significance level, is there any difference between the training time needed for
the three systems?

In this case the 'groups' are the three database management systems. These account for
some, but not all, of the total variance. Some, however, is not explained by the difference
between them. The residual variance is referred to as that due to the ‘errors’.

Total variance = between systems variance + variance due to the errors.

It follows that: Total sum = Sum of squares + Sum of squares

of squares between systems of errors
(SST) (SSSys) (SSE)

In practice we transpose this equation and use: SSE = SST - SSSys

Calculation of Sums of Squares

The 'square' for each case is (x - x )2 where x is the value for that case and x is the mean.
The 'total sum of squares' is therefore ∑ (x − x ) . The classical method for calculating this
2

sum is to tabulate the values; subtract the mean from each value; square the results; and
finally sum the squares. The use of a statistical calculator is preferable!

In the lecture on summary statistics we saw that the standard deviation is calculated by:
∑ (x − x )
2
sn = so ∑ (x − x )2 = ns 2n with s from the calculator using [xσn]
n
∑ (x − x )
2
or s n −1 = so ∑ (x − x )2 = (n − 1)s 2n −1 with s from the calculator using [xσn-1]
n −1
Both methods estimate exactly the same value for the total sum of squares.

TotalSS Input all the data individually and output the values for n , x and σn from the
calculator in SD mode. Use these values to calculate σ2n and nσ2n .
n x σn σn2 σn2 .
nσ
15 17.33 3.419 11.69 175.3 = SS Total

3
SSSys Calculate n and x for each of the management systems separately:
n x
System A 5 16
System B 5 15
System C 5 21

Input as frequency data, (n = 5), in your calculator and output n, x and σn .

n x σn σn2 σ n2 .
nσ
SS for Systems 15 17.33 2.625 6.889 103.3 = SSSys

SSE is found by difference SSE = SST - SSSys = 175.3 - 103.3 = 72.0

Using the ANOVA table

Below is the general format of an analysis of variance table. If you find it helpful then
make use of it, otherwise just work with the numbers, as on the next page.

General ANOVA Table (for k groups, total sample size N)

Source S.S d.f. M.S.S. F

SSG MSG
Between groups SSG k-1 = MSG =F
k −1 MSE

SSE
Errors SSE (N-1) - (k-1) = MSE
N−k

Total SST N-1

Method
• Fill in the total sum of squares, SST, and the between groups sum of squares, SSG,
after calculation; find the sum of squares due to the errors, SSE, by difference;
• the degrees of freedom, d.f., for the total and the groups are one less than the total
number of values and the number of groups respectively; find the error degree of
freedom by difference;
• the mean sums of squares, M.S.S., is found in each case by dividing the sum of
squares, SS, by the corresponding degrees of freedom.
• The test statistic, F, is the ratio of the mean sum of squares due to the differences
between the group means and that due to the errors.

4
In this example: (k = 3 systems, N = 15 values)

Source S.S. d.f. M.S.S. F

Between
103.3 3-1=2 103.3/2 = 51.65 51.65/6.00 = 8.61
systems
Errors 72.0 14 - 2 = 12 72.0/12 = 6.00

Total 175.3 15 - 1 = 14

The hypothesis test

The methodology for this hypothesis test is similar to that described last week.

The null hypothesis, H0, is that all the group means are equal. H0: µ1 = µ2 = µ3 = µ4 etc.

The alternative hypothesis, H1, is that at least two of the group means are different.

The significance level is as stated or 5% by default.

The critical value is from the F-tables, Fα (ν1, ν 2 ) , with the two degrees of freedom from
the groups, ν1, and the errors, ν2.

The test statistic is the F-value calculated from the sample in the ANOVA table.

The conclusion is reached by comparing the test statistic with the critical value and
rejecting the null hypothesis if the test statistic is the larger of the two.

Example 1 (cont.)

H0: µA = µB = µC H1: At least two of the means are different.

Critical value: F0.05 (2,12) = 3.89 (Deg. of free. from 'between systems' and 'errors'.)
Test statistic: 8.61
Conclusion: T.S. > C.V. so reject H0. There is a difference between the mean learning
times for at least two of the three database management systems.

Where does any difference lie?

We can calculate a critical difference, CD, which depends on the MSE, the sample sizes
and the significance level, such that any difference between means which exceeds the CD
is significant and any less than it is not.
1 1 
The critical difference formula is: CD = t MSE  + 
 n1 n 2 
t has the error degrees of freedom and one tail. MSE from the ANOVA table.

5
Example1 (cont.)

1 1  1 1
CD = t MSE +  ⇒ 1.78 6.00 +  = 2.76
 n1 n 2  5 5

From the samples x A = 16, x B = 15, x C = 21.

System C takes significantly longer to learn than Systems A and B which are similar.

Two-way ANOVA
In the above example it might have been reasonable to suggest that the five Operators
might have different learning speeds and were therefore responsible for some of the
variation in the time needed to master the three Systems. By extending the analysis from
one-way ANOVA to two-way ANOVA we can find our whether Operator variability is a
significant factor or whether the differences found previously were just due to the Systems.

Example 2 Operators
1 2 3 4 5
System A 16 19 14 13 18
System B 16 17 13 12 17
System C 24 22 19 18 22

Again we ask the same question: using a 5% level, is there any difference between the
training time for the three systems? We can use the Operator variation just to explain some
of the unexplained error thereby reducing it, 'blocked' design, or we can consider it in a
similar manner to the System variation in the last example in order to see if there is a
difference between the Operators.

In the first case the 'groups' are the three database management systems and the 'blocks'
being used to reduce the error are the different operators who themselves may differ in
speed of learning. In the second we have a second set of groups - the Operators.

Total variance = between systems variance + between operators variance

+ variance of errors.

So Total sum = Sum of squares + Sum of squares + Sum of squares

of squares between systems between operators of errors
(SST) (SSSys) (SSOps) (SSE)

In 2-way ANOVA we find SST, SSSys, SSOps and then find SSE by difference.
SSE = SST - SSSys - SSOps

We already have SST and SSSys from 1-way ANOVA but still need to find SSOps.

6
 Operators
1 2 3 4 5 Means
System A 16 19 14 13 18 16.00
System B 16 17 13 12 17 15.00
System C 24 22 19 18 22 21.00
Means 18.67 19.33 15.33 14.33 19.00 17.33

From example 1: SST = 175.3 and SSSys = 103.3

SSOps Inputting the Operator means as frequency data (n = 3) gives:

n = 15, x = 17.33, σ n = 2.078 and nσn2 = 64.7 = SSOps

Two-way ANOVA table, including both Systems and Operators:

Source S.S. d.f. M.S.S. F
Between Systems 103.3 3-1=2 103.3/2 = 51.65 51.65/0.91 = 56.76
Between Operators 64.7 5-1=4 64.7/4 = 16.18 16.18/0.91 = 17.78
Errors 7.3 14 - 6 = 8 7.3/8 = 0.91
Total 175.3 15 - 1 = 14

Hypothesis test (1) for Systems

H0: µ A = µB = µC H1: At least two of them are different.
Critical value: F0.05 (2,8) = 4.46
Test Statistic: 56.76 (Notice how the test statistic has increased with the use of the more
powerful two-way ANOVA)
Conclusion: T.S. > C.V. so reject H0. There is a difference between at least two of the
mean times needed for training on the different systems.
Using CD of 1.12 (see overhead): C takes significantly longer to learn than A and/or B.

Hypothesis test (2) for Operators

H0: µ 1 = µ2 = µ3 = µ4 = µ5 H1: At least two of them are different.
Critical value: F0.05 (4,8) = 3.84
Test Statistic: 17.78
Conclusion: T.S. > C.V. so reject H0. There is a difference between at least two of the
Operators in the mean time needed for learning the systems.

Using CD of 1.45 calculated as previously (see overhead): Operators 3 and 4 are

significantly quicker learners than Operators 1, 2 and 5.

7
Table 4 PERCENTAGE POINTS OF THE F DISTRIBUTION α = 5%

Numerator degrees of freedom

ν1
ν2 1 2 3 4 5 6 7 8 9

1 161.40 199.50 215.70 224.60 230.20 234.00 236.80 238.90 240.50

2 18.51 19.00 19.16 19.25 19.30 19.33 19.35 19.37 19.38
3 10.13 9.55 9.28 9.12 9.01 8.94 8.89 8.85 8081
4 7.71 6.94 6.56 6.39 6.26 6.16 6.09 6.04 6.00
5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 4.77
6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 4.10
7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 3.68
8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 3.39
9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 3.18
10 4.96 4.10 3.71 3.48 3.33 3.22 3.14 3.07 3.02
Denominator degrees of freedom

11 4.84 3.98 3.59 3.36 3.20 3.09 3.01 2.95 2.90

12 4.75 3.89 3.49 3.26 3.11 3.00 2.91 2.85 2.80
13 4.67 3.81 3.41 3.18 3.03 2.92 2.83 2.77 2.71
14 4.60 3.74 3.34 3.11 2.96 2.85 2.76 2.70 2.65
15 4.54 3.68 3.29 3.06 2.90 2.79 2.71 2.64 2.59
16 4.49 3.63 3.24 3.01 2.85 2.74 2.66 2.59 2.54
17 4.45 3.59 3.20 2.96 2.81 2.70 2.61 2.55 2.49
18 4.41 3.55 3.16 2.93 2.77 2.66 2.58 2.51 2.46
19 4.38 3.52 3.13 2.90 2.74 2.63 2.54 2.48 2.42
20 4.35 3.49 3.10 2.87 2.71 2.60 2.51 2.45 2.39
21 4.32 3.47 3.07 2.84 2.68 2.57 2.49 2.42 2.37
22 4.30 3.44 3.05 2.82 2.66 2.55 2.46 2.40 2.34
23 4.28 3.42 3.03 2.80 2.64 2.53 2.44 2.37 2.32
24 4.26 3.40 3.01 2.78 2.62 2.51 2.42 2.36 2.30
25 4.24 3.39 2.99 2.76 2.60 2.49 2.40 2.34 2.28
26 4.23 3.37 2.98 2.74 2.59 2.47 2.39 2.32 2.27
27 4.21 3.35 2.96 2.73 2.57 2.46 2.37 2.31 2.25
28 4.20 3.34 2.95 2.71 2.56 2.45 2.36 2.29 2.24
29 4.18 3.33 2.93 2.70 2.55 2.43 2.35 2.28 2.22
30 4.17 3.32 2.92 2.69 2.53 2.42 2.33 2.27 2.21
40 4.08 3.23 2.84 2.61 2.45 2.34 2.25 2.18 2.12
60 4.00 3.15 2.76 2.53 2.37 2.25 2.17 2.10 2.04
120 3.92 3.07 2.68 2.45 2.29 2.17 2.09 2.02 1.96
∞ 3.84 3.00 2.60 2.37 2.21 2.10 2.01 1.94 1.88