NTHU MATH 2820, 2008

Lecture Notes
Ch8, p.73

Interval Estimation
What is interval estimation?
Question 6.10

Is it satisfactory to report only an estimated value of ? Note that 1. A point estimate, though it will represent our best guess for the true value of the parameter, may be close to that true value but (for continuous case) will virtually never actually equal it. 2. Some measure of how close the point estimate is likely to be the true value is required. One way to do this is to report both the estimate and its estimated standard error. The following questions arise naturally: 1. A point estimator only gives a value. Wound it be better that we can give customer a range of possible value? This amounts to replacing the point estimate, a single number, by an entire interval of plausible values. 2. Is there an estimation method that can combine together the two types of information, i.e., estimated value and standard error?
Ch8, p.74

Definition 6.21 (interval estimator, coverage probability, interval estimate, confidence interval, and confidence level, TBp. 217 & 279)

For a random vector X = (X1 , . . . , Xn ), an interval estimator of a parameter with coverage probability 1 is a random interval (L (X), U (X)), where 1. L (X), U (X) are functions of data, 2. L (X) < U (X), and, 3. P ( (L (X), U (X))) = 1 . If X = x is observed, the interval (L (x), U (x)) is called an interval estimate. The term 100 (1 )% condence interval is used to denote either an interval estimator with coverage probability 1 or an interval estimate. The 100(1 )% is referred to as condence level. Note. The is usually assigned a small value, e.g. 0.1, 0.05, or 0.01.
made by Shao-Wei Cheng (NTHU, Taiwan)

NTHU MATH 2820, 2008

Lecture Notes
Ch8, p.75

Question 6.11

How to interpret the 100(1 )% in a 100(1 )% interval estimate (L(x), U (x))? For example, for a 95% interval estimate, say (23.5, 47.8), can we say that
Repeated construction of 95% confidence intervals

P ( (23.5, 47.8)) = 0.95?

methods of constructing interval estimators Recall. methods of estimating standard error (1) exact distribution (2) asymptotic method (3) bootstrap The three methods can be applied to forming condence intervals.
Definition 6.22 (pivotal quantity)

A function of data X1 , . . . , Xn and parameter , denoted by Q(X, ) = Q(X1 , . . . , Xn , ), is called a pivotal quantity if the distribution of Q(X, ) is irrelevant to all parameters. (c.f. ancillary statistics)
Ch8, p.76

Example 6.38 (some pivotal quantities, TBp. 279-281)

1. Let X1 , . . . , Xn be i.i.d. from Exponential distribution E(), then

n

i=1

Xi (n, 1).

Because the pdf of (n, 1) is irrelevant to , n Xi is a pivotal i=1 quantity. 2. Let X1 , . . . , Xn be i.i.d. from Uniform distribution U (0, ). Then, the pdf of X(n) Q= is nq n1 , 0 q 1. Because the pdf is irrelevant to , (n) is a pivotal quantity. 3. Let X1 , . . . , Xn be i.i.d. from Normal distribution N (, 2 ), where is a parameter and the value of is known. Then, n(X ) N (0, 1). Because the pdf of N(0, 1) is irrelevant to , n(X) is a pivotal quan tity.
made by Shao-Wei Cheng (NTHU, Taiwan)
X

NTHU MATH 2820, 2008

2

Lecture Notes
Ch8, p.77

4. Let X1 , . . . , Xn be i.i.d. from Normal distribution N(, ), where both and are unknown. Then, n(X ) tn1 , S n (Xi X)2 where S 2 = i=1n1 . Because the pdf of tn1 is irrelevant to and , is a pivotal quantity. 5. Let X1 , . . . , Xn be i.i.d. from Normal distribution N(, 2 ), where both and are unknown. Then, (n 1)S 2 2 . n1 2 2 Because the pdf of 2 is irrelevant to and , (n1)S is a pivotal 2 n1 quantity.
Theorem 6.21 (exact distribution method for constructing confidence intervals)
2 n(X) S

Let Q(X, ) be a pivotal quantity, and A be a set such that P (Q(X, ) A) = 1 , where A does not depend on parameters. Then, the set { : Q(x, ) A} is a 100(1 )% condence interval (or condence set) of .
Ch8, p.78

Example 6.39 (cont. Ex. 6.38, exact confidence intervals, TBp. 279-281)

1. Let a and b satisfy P ((n, 1) a) = /2 and P ((n, 1) b) = /2, respectively, and A = (a, b), then
n

1 =P

a<
i=1

Xi < b

=P

b a << nX nX

Hence, (a/nX, b/nX) is a 100(1 )% condent interval of . a 1 2. Let a and b satisfy 0 nq n1 dq = and b nq n1 dq = , re2 2 1 1 n spectively (whose solutions are a = ( 2 ) and b = (1 ) n ). 2 Let A = (a, b), then X(n) 1 = P a< <b X(n) X(n) b a 1 << = P < < =P . X(n) X(n) b a Hence, (X(n)/b, X(n)/a) is is a 100(1)% condent interval of .
made by Shao-Wei Cheng (NTHU, Taiwan)

NTHU MATH 2820, 2008

Lecture Notes
Ch8, p.79

3. Let z() denote the point beyond which the N(0, 1) distribution has probability . Note that z() = z(1 ). Then, n(X ) 1 = P z(/2) < < z(/2) = P X z(/2) < < X + z(/2) . n n Hence, X z(/2) n forms a 100(1)% condent interval of . 4. Let tn () denote the point beyond which the tn distribution has probability . Note that tn () = tn (1 ). Then, n(X ) < tn1 (/2) S S S = P X tn1 (/2) < < X + tn1 (/2) . n n S Hence, X tn1 (/2) n forms a 100(1 )% condent interval of . 1 = P tn1 (/2) <
Ch8, p.80

5. Let 2 () denote the point beyond which the 2 distribun n tion has probability . Then, (n 1)S 2 2 < 2 (/2) 1 = P n1(1 /2) < n1 2 (n 1)S 2 (n 1)S 2 2 = P < < 2 . 2 (/2) n1(1 /2) n1 Hence,
2 (n1)S 2 , 2(n1)S 2 n1 (/2) n1(1/2) 2

forms a 100(1)% condent

interval of . Question: How to nd a 100(1 )% condent interval of ?

Note. The construction of condence intervals based on exact distribution requires detailed knowledge of the sampling distribution as well as some cleverness. Question: Are there other methods that can oer a more general rule?
made by Shao-Wei Cheng (NTHU, Taiwan)

NTHU MATH 2820, 2008

Lecture Notes
Ch8, p.81

Definition 6.23 (asymptotically pivotal quantity)

For data X = (X1 , . . . , Xn ), a function of X and parameter , denoted by Qn (X, ) = Q(X1, . . . , Xn , ), is called asymptotically pivotal quantity if the limiting distribution of Qn (X, ) as n , is irrelevant to all parameters.
Question 6.12

What theorems/properties you have learned can help us to identify an asymptotically pivotal quantity?
Theorem 6.22 (asymptotic method for constructing confidence intervals, TBp. 281)

The MLE satises

D nI(0)( 0 ) N (0, 1),

where 0 is the true value of parameter. It can be further argued that

D nI()( 0) N (0, 1).

Ch8, p.82

Therefore, 1P z(/2)

nI()( 0) z(/2) .

and an asymptotic 100(1 ) condence interval for 0 is 1 z(/2) . nI()

estimate(critical value)(estimated standard error)

Note.

Many condence intervals have a form like the asymptotic condence interval of MLE, in which the center of condence interval is the estimated value of , the length of condence interval depends on: 1. condence level, 2. sample size n, and 3. a quantity that is related to the (variance of) distribution assigned in statistical modeling step. Question : We prefer to have a C.I. with longer length or shorter length?
made by Shao-Wei Cheng (NTHU, Taiwan)

NTHU MATH 2820, 2008

Lecture Notes
Ch8, p.83

Example 6.40 (cont. Ex. 6.19, asymptotic confidence interval for Poisson mean, TBp. 282)

Let X1 , . . . , Xn be i.i.d. from Poisson distribution P (), then the MLE 1 of is = X and the Fisher information of P () is I() = . Hence an asymptotic 100(1 )% condence interval for is X z(/2) X/n,

since the sampling distribution of X is approximately Normal. Data: Asbestos bers on lters. = 24.9, s = /n = 1.04 . An asymptotic 95% condence interval for is (1.96s ) = (22.9, 26.9).
Note. (TBp.283)

For i.i.d. case, the asymptotic variance of MLE is 1/(nI(0)) = 1/E[l 0(0 )]2 = 1/E[l 00(0)], where 0 is the true value of parameter, and l is the log-likelihood of all data.
Ch8, p.84

For non-i.i.d. case, 1 . nI(0) However, under some regular conditions, it can be shown that for non-i.i.d. case, the asymptotic variance of MLE is not of the form the MLE is asymptotically normal, and its asymptotic variance still equals 1 or E[l 0(0)]2 1 , E[l 00(0)]

where l is the log function of joint pdf/pmf of all data.

Example 6.41 (cont. Ex. 6.15, asymptotic C.I. for Hardy-Weinberg Equilibrium, TBp. 283)

Note that for (X1 , . . . , Xr ) Multinomial(n, p1 , . . . , pr ), the counts X1 , . . . , Xr are not independent. Let us return to the example of Hardy-Weinberg equilibrium, in which the data (X1 , X2 , X3 ) Multinomial(n, p1 , p2 , p3 ), where p1 = (1 )2 , p2 = 2(1 ), and p3 = 2 , for 0 < < 1.
made by Shao-Wei Cheng (NTHU, Taiwan)