Lecture 9: Confidence Intervals, Residuals

Formal Analysis Formal Analysis

The Analysis of Variance –

ANOVA The F Ratio and P-value do
not tell you which treatment
The F Ratio and P-value
means are different.
only indicate if statistically
significant differences exist.
1 2

Confidence Interval Confidence Interval

Gives a range of values for ∗

1 1
the likely size of the
difference between two
population means. 95% confidence with df = df for
Error
3 4

Confidence Interval Confidence Interval

1 1
df = 18, 95% Confidence 143.1 163.0 2.101 120.05
10 10
t*=2.101
19.9 2.101 4.9
MSError = 120.05 19.9 10.3
30.2 to 9.6

5 6

1
 Lecture 9: Confidence Intervals, Residuals

Interpretation Generalization
Men with Stage 1 hypertension
We are 95% confident that on the 50 mmol Na/day diet will
the difference in mean have an average blood pressure
from 9.6 to 30.2 mmHg lower
blood pressures for the two then men with Stage 1
diets is between –30.2 hypertension on the 200 mmol
mmHg and –9.6 mmHg. Na/day diet.
7 8

JMP Oneway Analysis of Systolic BP (mmHg) By Diet

Oneway Anova
t Test

The t Test part of the output

200 mmol/day-050 mmol/day
Assuming equal variances
Difference 19.9000 t Ratio 4.061224

has for the Std Err Dif 4.9000 DF

Upper CL Dif 30.1945 Prob > |t|
Lower CL Dif 9.6055 Prob > t
18
0.0007*
0.0004*

Dif = 200 mmol – 50 mmol. Confidence 0.95 Prob < t 0.9996

-20 -10 0 10 20

Upper CL Dif: 30.1945

Lower CL Dif: 9.6055
9 10

Residuals JMP

We should check the From the menu on the

residuals to see if they output go to Save –
satisfy the Fisher Residuals.
Conditions.
11 12

2
 Lecture 9: Confidence Intervals, Residuals

Equal Standard Deviation Oneway Analysis of Systolic BP

(mmHg) centered by Diet By Diet
25
20
15
10
5

Plot the residuals versus

0
-5
-10

diet. -15
-20
-25
050 mmol/day 200 mmol/day

Compute means and Means and Std Deviations

Diet

standard deviations. Level Number Mean Std Dev

050 mmol/day 10 5.684e-15 11.7516
200 mmol/day 10 0 10.0995
13 14

Fisher Conditions Normal Distribution

Residuals add to zero.
Residuals have the same standard Analyze the distribution of
deviations.
 Although not exactly the same, the residuals to see if they
two standard deviations are very close
to being equal. could have come from a
 Only worry if one standard deviation normal distribution.
is 2, 3 or more times as big as another.
15 16

Distributions
Systolic BP (mmHg) centered by Diet
1.64

1.28 0.9 Fisher Conditions

0.8
0.67
0.7
0.6
0.0 0.5
0.4
0.3

Histogram is slightly skewed

-0.67
0.2
-1.28 0.1
-1.64

right.
10 Box plot is slightly skewed right.
Most points come close to the
8

2
diagonal (normal model) line.
-30 -20 -10 0 10 20 30
17 18

3
 Lecture 9: Confidence Intervals, Residuals
Normal Distribution?
There is some doubt as to
whether the residuals come
from a normal distribution.
19
Consequence
The stated confidence level for
the confidence interval (95%)
may not be the true
confidence.
The true confidence may be
lower.
21
Comment
The true P-value and true
confidence level may be
slightly different from what
is reported.
23
Consequence
The reported P-value may not be
the true P-value.
However, the P-value (0.0007) is
so small even if it is not exactly
correct, we would still conclude
that there is a difference in diets.
20
Comment
The analysis of variance is still
valid and the conclusion that
there is a statistically
significant difference in mean
blood pressure for the two
diets is not in doubt.
22
Comment
Remember the analysis of
residuals is a secondary analysis
and usually doesn’t make a
difference unless the conditions
are severely violated and the
analysis is marginal (a P-value
close to 0.05).
24