22 views

Uploaded by thrphys1940

ecometrics

ecometrics

© All Rights Reserved

- Notes 2
- 7. Concepts in Probability, Statistics and Stochastic Modelling
- 5. 2009 Unbiased Estimate & Confidence Interval
- Efficiency of Estimator
- 13. Maths - Ijamss - An Exact Expression for the Mean Square Error of Ratio Estimator
- Adapt Formula Sheet
- B. J. Winer-Statistical Principles in Experimental Design-NY c._ McGraw-Hill Book Company (1962)
- Constant Expected Return
- Family Estimators
- Comparing Measures of Sample Skewness and Kurtosis
- 850B8d01 JS and GK Paper in BIJ Almost Unbiased Modified Linear Regression Estimators for Estimation of Population Mean
- 2015 10 Exam c Syllabus
- Good Practices in Free-Energy Calculations
- Study of Some Improved Ratio Type Estimators Under Second Order Approximation
- constantexpectedreturn.pdf
- A Kernel Method for Estimating Finite Population Distribution Functions Using Auxiliary Information
- NAM_S10.pdf
- Multiauxiliarity Information
- sampling_distributions_and_point_estimation_of_parameters.pdf
- Constant Expected Return

You are on page 1of 27

14. Properties of Estimators

Marcus Chambers

Department of Economics

University of Essex

07/09 February 2012

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

2/27

Outline

1

Introduction

2

Unbiased Estimators

3

Efcient Estimators

4

Linear Estimators and Mean Square Error

5

An Example

Reference: R. L. Thomas, Using Statistics in Economics,

McGraw-Hill, 2005, sections 11.1 and 11.2.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Introduction 3/27

To provide some background, consider a population of

values for a random variable X.

The variable X will have a probability distribution, which

may be known or unkown (typically the latter).

Suppose this distribution can be characterised by an

unknown parameter .

The parameter could represent the mean () or variance

(

2

) of the distribution, or it could represent the regression

slope parameter ().

Whatever it represents, the parameter needs to be

estimated using sample information.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Introduction 4/27

Suppose we have a random sample of n observations

taken from the population of X:

X

1

, X

2

, . . . , X

n

,

where X

i

denotes the ith observation.

Let the estimator of be denoted Q, which will be some

function of the observations i.e.

Q = Q(X

1

, X

2

, . . . , X

n

).

For example, if were the population mean , then our

estimator would be the sample mean

X =

X

1

+ X

2

+. . . + X

n

n

.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Introduction 5/27

Whatever the estimator Q, it will have a sampling

distribution.

This is because the value of Q changes with each different

possible sample, and the sampling distribution represents

the distribution of Q across all possible samples.

We can therefore talk about quantities such as the mean

and variance of the estimator Q i.e.

E(Q) and E[Q E(Q)]

2

.

In the regression model we have two estimators, a and b,

of and , the intercept and slope parameters.

Both a and b will have their own sampling distributions, as

well as a joint sampling distribution.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Introduction 6/27

The small sample properties of estimators are determined

by the sampling distribution for estimators obtained using a

given sample size n.

Such properties hold even when n may be small.

This contrasts with large sample, or asymptotic, properties

which are obtained as the sample size n gets larger and

larger i.e. as n .

You have already seen the Central Limit Theorem, which is

an example of an asymptotic property for the sample

mean.

It is often written in the form

Z

n

=

n(

X )

d

N(0, 1) as n .

We shall not be concerned with asymptotic properties

here, only small sample properties.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Unbiased Estimators 7/27

A desirable small sample property for an estimator to

possess is unbiasedness:

Denition

An estimator Q is said to be an unbiased estimator of if, and

only if, E(Q) = .

Put another way, the mean of the sampling distribution of

the estimator Q is equal to the true population parameter .

Or, if we were to take many samples, the average of all Qs

obtained would be equal to .

This is illustrated on the next slide:

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Unbiased Estimators 8/27

The distribution of Q is centred at , meaning that there is

no systematic tendency to either over-estimate or

under-estimate .

This does not mean that in any given sample the estimator

will equal (it almost certainly wont)!

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Unbiased Estimators 9/27

We have already seen that the sample mean is an

unbiased estimator of the population mean: E(

X) = .

Note that the property of unbiasedness does not depend

on the sample size; it is therefore a small sample property.

If an estimator does not satisfy the unbiasedness property

it is said to be biased and so there is a systematic

tendency to error in estimating .

The bias of an estimator Q is dened as

bias(Q) = E(Q) .

If Q tends to over-estimate then E(Q) > and

bias(Q) > 0.

Alternatively, if Q tends to under-estimate then E(Q) <

and bias(Q) < 0.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Unbiased Estimators 10/27

An example of a biased estimator is

v

2

=

(X

i

X)

2

n

,

which is an estimator of

2

, the population variance.

It can be shown that

E(v

2

) =

n 1

n

2

<

2

,

and hence bias(v

2

) < 0.

This motivates the unbiased estimator, s

2

, of

2

, in which n

in the denominator is replaced by n 1:

s

2

=

(X

i

X)

2

n 1

;

this results in E(s

2

) =

2

.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Unbiased Estimators 11/27

The negative bias of v

2

is depicted above.

But is unbiasedness all that we require of an estimator?

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Efcient Estimators 12/27

In practice we are faced with using a single sample to

determine our estimator Q.

Although unbiasedness is a good property for Q to

possess, we need to consider other aspects of the

sampling distribution as well, such as the variance.

Consider two unbiased estimators, Q

1

and Q

2

, so that

E(Q

1

) = and E(Q

2

) = .

Suppose, however, that the variance of Q

1

is larger than

the variance of Q

2

, so that V(Q

1

) > V(Q

2

).

Which estimator would we prefer?

The diagram on the next slide will help answer this

question. . .

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Efcient Estimators 13/27

Although both estimators are unbiased, there is a higher

probability of being far away from using the estimator Q

1

than with Q

2

i.e.

Pr(Q

1

>

) > Pr(Q

2

>

).

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Efcient Estimators 14/27

We would therefore prefer to use Q

2

whose distribution is

more condensed around than the distribution of Q

1

.

Another desirable property for an estimator to possess is

efciency:

Denition

An estimator Q is said to be an efcient estimator of if, and

only if:

(i) it is unbiased, so that E(Q) = ; and

(ii) no other unbiased estimator of has a smaller variance.

The two important properties for efciency, therefore, are

unbiasedness and smallest (or minimum) variance.

Efcient estimators are sometimes also called best

unbiased estimators.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Linear Estimators and Mean Square Error 15/27

It can be difcult, in practice, to show that an unbiased

estimator has the smallest variance among all estimators.

It is often easier to restrict attention to linear estimators,

that is, ones which are a linear combination of the sample

observations.

A linear estimator is therefore of the form

Q = a

1

X

1

+ a

2

X

2

+. . . + a

n

X

n

,

where a

1

, . . . , a

n

are a set of constants (or weights).

An example of a linear estimator is the sample mean:

X =

X

1

+. . . + X

n

n

=

1

n

X

1

+. . . +

1

n

X

n

,

so the weights are a

1

= 1/n, . . . , a

n

= 1/n.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Linear Estimators and Mean Square Error 16/27

Denition

An estimator, Q, is said to be a best linear unbiased estimator

(BLUE) of if, and only if:

(i) it is a linear estimator, that is, Q =

i

a

i

X

i

, where the a

i

are

constants;

(ii) it is unbiased, so that E(Q) = ; and

(iii) no other linear unbiased estimator has a smaller variance.

Note that a BLUE estimator may not be the best possible,

because there may exist an efcient nonlinear estimator

with a smaller variance.

In terms of estimating the population mean , it turns out

that the sample mean

X is both BLUE and efcient.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Linear Estimators and Mean Square Error 17/27

The concept of efciency is useful when comparing

unbiased estimators, because we always prefer the one

with the smaller variance.

But suppose we are faced with the following problem.

There are two estimators, Q

1

and Q

2

, of a parameter .

Q

2

is unbiased and efcient while Q

1

has a small (positive)

bias but also a smaller variance than Q

2

.

We have E(Q

1

) > , E(Q

2

) = , V(Q

1

) < V(Q

2

).

Can we compare these estimators in a meaningful way?

Consider the diagram on the next slide:

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Linear Estimators and Mean Square Error 18/27

The distribution of Q

2

is centred at but has large

dispersion while the distribution of Q

1

lies to the right of

but with a smaller variance.

There is a trade-off here between bias and variance.

Can we combine both in a single measure?

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Linear Estimators and Mean Square Error 19/27

We shall consider the error of the estimator, given by Q.

The mean square error (MSE) of Q is given by

MSE(Q) = E(Q )

2

.

It can be shown that

MSE(Q) = V(Q) + [bias(Q)]

2

,

i.e. the MSE of Q is equal to the variance plus the square

of the bias.

One way of choosing the preferred estimator would be to

choose the one that has the smallest MSE, which places

equal weight on variance and squared bias.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Linear Estimators and Mean Square Error 20/27

A more general method uses weights to reect the relative

importance placed on variance and squared bias.

We could then choose the estimator as the one which

minimises

M(Q) = V(Q) + (1 )[bias(Q)]

2

,

where 0 < < 1 denotes the weight.

The larger is , the more importance is placed on variance,

while the smaller is , the more importance is placed on

(squared) bias.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Linear Estimators and Mean Square Error 21/27

It is usually a good idea, when estimating a parameter, to

use all the sample information that is available.

In general, the more information used, the more efcient

the estimator will be.

For example, it wouldnt make sense to ignore 50% of a

sample when estimating the population mean the more

information (observations) the better the estimator.

An estimator that uses all the sample information is said to

be sufcient.

However, if an estimator uses all the observations

inappropriately, it will not be efcient despite being

sufcient.

The point is that an estimator cannot be efcient unless it

is sufcient.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

An Example 22/27

Example (Thomas, p.327). A variable X is normally

distributed with mean and variance

2

. Three estimators

of are proposed:

m =

X 10, m =

X +

5

n

, m

n 1

n 2

X,

where

X is the sample mean and n the sample size.

(a) Explain why all three estimators will have sampling

distributions that are normal in shape.

(b) Recalling that E(

all the proposed estimators are biased and hence

determine the bias in each case. If n = 10 and = 8,

which estimator has the smallest absolute bias?

(c) Recalling that V(

X) =

2

/n, use Theorem 1.1 to nd

the variance of the sampling distribution for each

estimator. Hence determine which estimator has the

largest variance.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

An Example 23/27

Solution (a). First, note that m, m and m

functions of

X.

From the central limit theorem, we know that

X has a

normal distribution.

As all linear functions of normally distributed variables are

themselves normally distributed, it follows that the

sampling distributions of m, m and m

normal.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

An Example 24/27

Solution (b). Recall, from Theorem 1.1, that

E(a + bX) = a + bE(X),

where a and b are constants.

We begin by nding the expected values of each estimator:

E( m) = E(

X 10) = E(

X) 10 = 10;

E( m) = E

X +

5

n

= E(

X) +

5

n

= +

5

n

;

E(m

) = E

n 1

n 2

n 1

n 2

E(

X) =

n 1

n 2

.

Thus all the estimators are biased.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

An Example 25/27

The biases are

bias( m) = E( m) = 10 = 10;

bias( m) = E( m) = +

5

n

=

5

n

;

bias(m

) = E(m

) =

n 1

n 2

1

n 2

.

When n = 10 and = 8,

bias( m) = 10;

bias( m) =

5

10

= 0.5;

bias(m

) =

1

10 2

8 = 1.

Hence m has the smallest bias in absolute terms.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

An Example 26/27

Solution (c). Recall, from Theorem 1.1, that

V(a + bX) = b

2

V(X),

where a and b are constants.

The variances are

V( m) = V(

X) =

2

n

;

V( m) = V(

X) =

2

n

;

V(m

) =

n 1

n 2

2

V(

X) =

n 1

n 2

2

n

.

Thus, since (n 1)/(n 2) > 1, it follows that m

has the

largest variance.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

Summary 27/27

Summary

Small sample properties of estimators.

Next week:

the Classical two-variable regression model.

EC114 Introduction to Quantitative Economics 14. Properties of Estimators

- Notes 2Uploaded bySandeep Singh
- 7. Concepts in Probability, Statistics and Stochastic ModellingUploaded byAbiodun Gbenga
- 5. 2009 Unbiased Estimate & Confidence IntervalUploaded byfabremil7472
- Efficiency of EstimatorUploaded byAnonymous 0U9j6BLllB
- 13. Maths - Ijamss - An Exact Expression for the Mean Square Error of Ratio EstimatorUploaded byiaset123
- Adapt Formula SheetUploaded byUngoliant101
- B. J. Winer-Statistical Principles in Experimental Design-NY c._ McGraw-Hill Book Company (1962)Uploaded byniki098
- Constant Expected ReturnUploaded bybeckybaik
- Family EstimatorsUploaded byAnonymous 0U9j6BLllB
- Comparing Measures of Sample Skewness and KurtosisUploaded byElizabeth Collins
- 850B8d01 JS and GK Paper in BIJ Almost Unbiased Modified Linear Regression Estimators for Estimation of Population MeanUploaded byJambulingam Subramani
- 2015 10 Exam c SyllabusUploaded byphulam146
- Good Practices in Free-Energy CalculationsUploaded byMilton Paredes Avalos
- Study of Some Improved Ratio Type Estimators Under Second Order ApproximationUploaded byAnonymous 0U9j6BLllB
- constantexpectedreturn.pdfUploaded byCristian Núñez Clausen
- A Kernel Method for Estimating Finite Population Distribution Functions Using Auxiliary InformationUploaded byArsene Hidiroglou
- NAM_S10.pdfUploaded byvoikien01
- Multiauxiliarity InformationUploaded byAnonymous 0U9j6BLllB
- sampling_distributions_and_point_estimation_of_parameters.pdfUploaded bySivagami Saminathan
- Constant Expected ReturnUploaded byCristian Núñez Clausen
- 01.estimation.pdfUploaded byShafiullaShaik
- 12Uploaded byAndresAmaya
- SAEstimation_Part2Uploaded byJuan Tomas Sayago
- BayesUploaded bysammittal
- CT3 Question paperUploaded byaishwaryadoshi
- Edu 2015 06 Exam c SyllabusUploaded byAlex Halikias
- EstimationUploaded byyashbhardwaj
- A New Biased EstimatorUploaded bym91e
- v102n08p481Uploaded byvanpato
- 1999JA900026.pdfUploaded byAnkush

- National AccountingUploaded bythrphys1940
- Introduction to MacroeconomicsUploaded bythrphys1940
- Constrained Maximisation IV Lagrange FunctionUploaded bythrphys1940
- Constrained Maximisation IV- ExamplesUploaded bythrphys1940
- Firm Theory and DualityUploaded bythrphys1940
- Constrained Maximisation IIUploaded bythrphys1940
- Constrained Maximisation IUploaded bythrphys1940
- Total DifferentationUploaded bythrphys1940
- Optimization With More Than One Variable IIUploaded bythrphys1940
- Optimization With More Than One VariableUploaded bythrphys1940
- Partial DifferentationUploaded bythrphys1940
- Methods of Economic Nalysis IIUploaded bythrphys1940
- Methods of Economic AnalysisUploaded bythrphys1940
- Quadratic EquationsUploaded bythrphys1940
- Linear Equarions - Economic ApplicationsUploaded bythrphys1940
- Linear Equations - SupplementUploaded bythrphys1940
- Linear EquationsUploaded bythrphys1940
- Further Inference TopicsUploaded bythrphys1940
- Further Probability TopicsUploaded bythrphys1940
- Hypothesis Testing IIUploaded bythrphys1940
- Hypothesis TestingUploaded bythrphys1940
- Estimation and Testing of Population ParametersUploaded bythrphys1940
- Basic Statistical InferenceUploaded bythrphys1940
- Continuous Probability DistributionUploaded bythrphys1940
- Discrete Probability DistributionUploaded bythrphys1940
- Introduction to ProbabilityUploaded bythrphys1940
- Introduction to StatisticsUploaded bythrphys1940
- Further Regression Topics IIUploaded bythrphys1940

- phases eclipses and tidesUploaded byapi-169639475
- Formal Report Distillation of ALcoholic BeveragesUploaded byPatricia Moran
- Ansi Nema c29.11 WatermarkedUploaded byhenry
- 5_1_00_TablesUploaded byshunmugathason
- Numerical Bank for IIT(MAINS)/NEET/AIIMS along with XIIth Preparation (Current Electricity)Uploaded byAnonymous cI0VgvT
- (a Treatise on Natural Philosophy 2 (Kelvin Tait 1912)Uploaded byLurzizare
- Highly active Pd and Pd–Au nanoparticles supported on functionalized graphene nanoplatelets for enhanced formic acid oxidationUploaded byshankar8572
- 6.Basic Workshop PracticeUploaded byVikram Rao
- Surface TensionUploaded bySneha Goyal
- Triggering in Trigatron Spark Gaps - A Fundamental StudyUploaded bysantosh_babar_26
- MP4-4Uploaded byHassan El Shimy
- Chapter_09 ESP Design ExamplesUploaded bydewidar1234
- How Does a Quadrotor Fly ?Uploaded byabdulla_alazzawi
- An Economic Analogy with Maxwell Equations in Fractional SpaceUploaded byAnonymous 0U9j6BLllB
- Aggregate Objective Functions and Pareto Frontiers - Messac, Sukam and MelachrinoudisUploaded byMichelle Duarte Marciano
- Radiance TutorialUploaded byMayank Singh
- CoreLineUploaded byElmedin Osmanovic
- Effects of seasonal variability on FT-NIR prediction of dry matter content for whole Hass avocado fruitUploaded byChu Bagunu
- Bayes' Theorem and Quantum RetrodictionUploaded bynakulsahadeva
- taper roller bearingUploaded bySatish Pathak
- HairUploaded byLala
- LaMotte 1766-KIT pH TDS Salt Temperature Tracer PockeTester InstructionsUploaded byPromagEnviro.com
- Numerical MethodsUploaded byVictor Kevin Contreras
- [Renata Dmowska, Barry Saltzman] Seismological StrUploaded byAchmad Faisal J-rocks Stars
- EHP-QuickStartGuideUploaded byLiladhar Ganesh Dhoble
- 115Uploaded bykabuby86
- Low Angle XRDUploaded byKoushik Ponnuru
- 471-526 exam 3 Fall 2015Uploaded byok9275
- 2018-04-01 BookUploaded byDr-Rehab Yousef
- Lab Report#1 Pcs130 magnetic fieldUploaded byAbdulrahman Jijawi