Condence Regions

Condence regions are multivariate extensions of univariate

condence intervals.
Recall the denition of a 100(1 )% CI for a parameter
: for X f(x|), , the interval (t
1
(X), t
2
(X)) is a
100(1 )% CI for if
Pr[t
1
(X) t
2
(X)] = 1 .
If represents a univariate mean , a 100(1 )% CI for
is given by [

X t
n1,

2
_
s
2
/n,

X +t
n1,

2
_
s
2
/n].
Similarly, for
p1
, the region R(X) is a 100(1 )%
condence region for if
Pr[R(X) will cover the true ] = 1 .
279
Condence Regions (contd)
For the mean vector
p1
, we know that before the sample
is selected,
Pr
_
n(

X )

S
1
(

X )
(n 1)p
(n p)
F
p,np
()
_
= 1 ,
meaning that

X is within statistical distance
[(n 1)pF
p,np
()/(n p)]
1/2
from with probability 1 .
Once a sample is obtained and x, S are computed, the set of
values
n( x )

S
1
( x )
(n 1)p
(n p)
F
p,np
()
denes an ellipsoidal region R(X) that is likely to cover .
280
Condence Regions (contd)
To decide whether a hypothesized value
0
is contained in
the condence region, we evaluate
n( x
0
)

S
1
( x
0
)
and compare it to the scaled F value above. If the squared
distance from x to
0
is larger than [(n1)pF
p,np
()/(np)],

0
is not in the condence region.
This is exactly equivalent to testing H
o
: =
0
versus H
1
:
=
0
using Hotellings T
2
statistic.
Thus, the 100(1 )% condence region is composed of
all values
0
for which the T
2
test would NOT reject
H
o
: =
0
versus H
1
: =
0
at level .
281
Condence Regions (contd)
What can we say about the shape of the condence region?
It is a p-dimensional ellipsoid centered at the sample mean
vector

X.
Recall that if (
i
, e
i
) are an eigenvalue-eigenvector pair of S,
then letting
(n 1)pF
p,np
()/(n p) = c
2
,
the ith axis of the condence ellipse has half length
c

i
n
=
_
(n 1)pF
p,np
()/(n p)

i
n
along the e
i
direction.
282
Condence Regions (contd)
Thus, beginning from the center of the ellipse at x, the
axes of the condence ellipse are

(n 1)p
n(n p)
F
p,np
().
Since the second term is constant for all axes, the ratios
of the
i
will reect the relative elongations.
Larger dierences in the sample variances across the p
measurements (due to real causes or to dierences in
the scale of the measurements), will create larger ratios
of eigenvalues (correlations are also involved)
283
Example: Microwave Ovens
Recall the microwave oven radiation data in Tables 4.1 and
4.5, where two radiation measurements, x
1
and x
2
, were
obtained from n = 42 ovens. Here, the x
j
denotes the trans-
formed (by Box-Cox) radiation measurements, using a power
= 0.25.
Sample statistics for those data are:
x =
_
0.564
0.603
_
, S =
_
0.014 0.012
0.012 0.015
_
, S
1
=
_
203.02 163.39
163.39 200.23
_
.
Eigenvalue and eigenvector pairs for S are

1
= 0.026 e

1
= [0.704, 0.710]

2
= 0.002 e

2
= [0.71, 0.704]
284
Example: Microwave Ovens (contd)
The 95% CR for is given by all values
1
,
2
that satisfy:
42[0.564
1
0.603
2
]

_
203.02 163.39
163.39 200.23
_ _
0.564
1
0.603
2
_
6.62,
where
2(41)
40
F
2,40
(0.05) =
2(41)
40
3.23 = 6.62.
Is
0
= [0.562 0.589]

a plausible value for ? To check,

plug
0
into the expression above and see if it satises the
inequality. In this case, we get 1.30 which is less than 6.62,
and conclude that
0
is plausible at the 95% level.
285
Example: Microwave Ovens (contd)
286
Example: Microwave Ovens (contd)
The joint condence ellipsoid is centered at x = [0.564 0.603]

and the half lengths of the two axes are

0.026

2(41)
42(40)
3.23 = 0.064,

0.002

2(41)
42(40)
3.23 = 0.018.
The axis are in the direction of the two eigenvectors when x
is taken as the origin.
The ratio

0.026

0.002
= 3.6
indicates that the major axis is 3.6 times longer than the
minor axis.
287
Simultaneous Condence Statements
Often we are interested in drawing inference about each
j
.
One possibility is to construct ordinary condence intervals
x
j
t
n1
(

2
)

s
jj
n
,
for each
j
. One problem is that the combined set of
individual intervals result in a simultaneous condence level
that is less than the nominal 1 .
There are various ways of constructing a collection of
individual condence intervals so that the joint condence
level for the family of parameters remains at 1
Intuitively, CIs that protect against erosion of the condence
level will be wider than the individual (1 ) 100% CIs.
288
Simultaneous Condence Statements (contd)
Suppose that we have p variables. The population mean of
the rst variable
1
can be written as
a

1
= [1 0 ... 0],
and in general,
j
= a

j
where a

j
is the p1 row vector with
a one in the jth position and zeros in all other positions.
Given a sample x
1
, x
2
, ..., x
n
of p-dimensional vectors, an
estimator of
j
is a

j
x, with an estimated variance of a

j
Sa
j
/n.
Then, an ordinary (1) 100% CI for
j
can be written as
a

j
x t
n1
(/2)

_
a

j
Sa
j
n
.
289
Simultaneous Condence Statements (contd)
An alternative way to interpret the ordinary (1 ) 100%
condence interval is as follows: the CI is the set of values
of a

for which
|t| =

n(a

j
x a

j
)
_
a

j
Sa
j

t
n1
(/2),
or, equivalently
t
2
=
n(a

j
x a

j
)
2
a

j
Sa
j
=
n(a

j
( x ))
2
a

j
Sa
j
t
2
n1
(/2).
290
Simultaneous Condence Statements (contd)
Intuitively, if we wish to construct a set of tests for many
dierent vectors a and have condence level 1 that all
intervals will cover the true a

, we will need a larger critical

value on the right-hand side of the inequality.
What is the maximum value that the statistic t
2
can reach
for some vector a?
max
a
t
2
= max
a
n(a

( x ))
2
a

Sa
= n( x )

S
1
( x ) = T
2
,
using the maximization lemma (2.50) on page 80 of your
textbook (you checked this on an assignment).
The maximum T
2
is achieved when a is proportional to
S
1
( x ).
291
Simultaneous Condence Statements (contd)
Let X
1
, ..., X
n
be a sample from N
p
(, ). Then simultane-
ously for all a, the intervals given by
a

_
p(n 1)
(n p)
F
p,np
()
a

Sa
n
will cover a

with probability of at least 1 .

Proof: recall that
T
2
= n( x )

S
1
( x ) c
2
=
n(a

( x ))
2
a

Sa
c
2
for every a.
292
Simultaneous Condence Statements (contd)
Equivalently:
a

x c
_
a

Sa/n a

x +c
_
a

Sa/n for all a.

Choosing
c
2
= p(n 1)F
p,np
()/(n p)
results in intervals that contain a

with probability no smaller

than
1 = Pr(T
2
c
2
).
293
Simultaneous Condence Statements (contd)
The intervals we just dened are called T
2
because their
length is determined by the sampling distribution of T
2
.
For a the vector with zeros everywhere and 1 in the jth
position, the T
2
interval is
x
j

p(n 1)
(n p)
F
p,np
()

s
jj
n

j
x
j
+

p(n 1)
(n p)
F
p,np
()

s
jj
n
.
Note that for a the vector with zeros everywhere except 1 in
the jth position and -1 in the kth position, the interval would
correspond to
j

k
. In this case,
a

x = x
j
x
k
, and a

Sa = s
jj
2s
jk
+s
kk
.
294
Example: Microwave Ovens (contd)
Before we had obtained a simultaneous 95% condence
ellipsoid for
1
and
2
, the means of the fourth root of
radiation with door closed and door open.
We now compute 95% T
2
intervals for the two means.
First note that

p(n 1)
n(n p)
F
p,np
(0.05) =

2(41)
42(40)
3.23 = 0.397.
is common to both intervals.
295
Example: Microwave Ovens (contd)
For
1
,
2
:
x
1
0.397

s
11
0.564 (0.397 0.12) 0.564 0.0476
x
2
0.397

s
22
0.603 (0.397 0.121) 0.603 0.048.
For the dierence between doors closed and open:
x
1
x
2
0.397
_
s
11
2s
12
+s
22
0.039 (0.397 0.0748)
[0.069, 0.009],
suggesting that closing the door signicantly reduces the
(fourth root) radiation emitted by the ovens.
The T
2
intervals are shadows or projections of the condence
ellipse onto the component axes.
296
Example: Microwave Ovens (contd)
The T
2
intervals are shadows or projections of the condence
ellipse onto the component axes.
297
Comparison of simultaneous and ordinary t
intervals
The ordinary one-at-a-time t intervals each have coverage
probability 1 , but the joint coverage probability of p in-
tervals is not known.
In the special case where the covariance matrix is diagonal,
the joint coverage probability of p ordinary t intervals is
(1 )
p
.
Clearly, to guarantee 1 joint coverage probability, the t
intervals need to be made wider.
How much wider depends on p, n and .
298
Comparison of condence intervals (contd)
The multipliers of (s
jj
/n)
1/2
in the simultaneous intervals
and in the t intervals are, respectively

p(n 1)
(n p)
F
p,np
(), and t
n1
(/2).
For example, for = 0.05, n = 15 and p = 4, the
simultaneous intervals are
(4.14 2.145)
2.145
100% = 93%
wider.
299
Comparison of condence intervals (contd)
An one-at-a-time t interval is the correct choice if we are
interested in only one of the components of .
While simultaneous T
2
intervals have the correct joint
coverage probability, they tend to be too conservative if
we are only interested in the p components of (as opposed
to all possible linear combinations of the components).
Note that for p = 2, the two T
2
intervals dene a rectangle
that contains the ellipse (with 95% coverage probability) and
more.
Thus, the rectangle formed by the two T
2
intervals has more
than 1 coverage probability.
300
The Bonferroni method for multiple
comparisons
The Bonferroni method is useful when we wish to make
a small number m of comparisons for linear combinations
a

1
, ..., a

m
.
Let C
i
denote a condence statement about a

i
such that
Pr(C
i
true) = 1
i
. Then
Pr(all C
i
true) = 1 Pr( at least one C
i
false)
1

i
Pr(C
i
false)
= 1

i
(1 Pr(C
i
true)
= 1 (
1
+
2
+... +
m
).
301
The Bonferroni method for multiple
comparisons
Consider, for example, m individual t intervals for
1
, ...,
m
,
with
i
= /m. From the Bonferroni inequality, we have
that:
Pr
_

X
i
t
n1
(

2m
)
_
s
ii
n
contains
i
, for all i
_
1
m

i=1

m
= 1 .
In general, to make condence statements about p means,
we divide the signicance level by the number of intervals
we want to construct p.
Microwave ovens: see T
2
and Bonferroni intervals in next
gure.
302
T
2
and Bonferroni condence intervals
303