4/27/2016

https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

PublishedonSTAT414/415(https://onlinecourses.science.psu.edu/stat414)
Home>MethodofMoments

MethodofMoments
Inshort,themethodofmomentsinvolvesequatingsamplemomentswiththeoretical
moments.So,let'sstartbymakingsurewerecallthedefinitionsoftheoreticalmoments,
aswellaslearnthedefinitionsofsamplemoments.

Definitions.
(1)E(X
1,2,...

isthekth(theoretical)momentofthedistribution(abouttheorigin),fork=

(2)E [(X ) ]isthekth(theoretical)momentofthedistribution(aboutthemean),

fork=1,2,...
k

(3)M
(4)M

1
k

=
n

1
=
n

k
i

isthekthsamplemoment,fork=1,2,...

i=1

k
(X i X )

isthekthsamplemomentaboutthemean,fork=1,2,...

i=1

OneFormoftheMethod
Thebasicideabehindthisformofthemethodisto:
(1)EquatethefirstsamplemomentabouttheoriginM1

1
=
n

firsttheoreticalmomentE(X).
(2)EquatethesecondsamplemomentabouttheoriginM2
secondtheoreticalmomentE(X2).

Xi = X

tothe

i=1

=
n

2
i

tothe

i=1

(3)Continueequatingsamplemomentsabouttheorigin,Mk,withthe
correspondingtheoreticalmomentsE(Xk),k=3,4,...untilyouhaveasmany
equationsasyouhaveparameters.
(4)Solvefortheparameters.
Theresultingvaluesarecalledmethodofmomentsestimators.Itseemsreasonable
thatthismethodwouldprovidegoodestimates,sincetheempiricaldistributionconverges
https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

1/6

4/27/2016

https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

insomesensetotheprobabilitydistribution.Therefore,thecorrespondingmoments
shouldbeaboutequal.

(ATrivial)Example
LetX1,X2,...,XnbeBernoullirandomvariableswithparameterp.Whatisthemethodof
momentsestimatorofp?

Solution.Here,thefirsttheoreticalmomentabouttheoriginis:
E(Xi)=p
Wehavejustoneparameterforwhichwearetryingtoderivethemethodof
momentsestimator.Therefore,weneedjustoneequation.Equatingthefirst
theoreticalmomentabouttheoriginwiththecorrespondingsamplemoment,we
get:
p =

1
n

Xi
i=1

Now,wejusthavetosolveforp.Whoops!Inthiscase,theequationisalready
solvedforp.Ourworkisdone!Wejustneedtoputahat(^)ontheparameter
tomakeitclearthatitisanestimator.Wecanalsosubscripttheestimatorwith
an"MM"toindicatethattheestimatoristhemethodofmomentsestimator:
^
=
p
MM

1
n

Xi
i=1

So,inthiscase,themethodofmomentsestimatoristhesameasthemaximum
likelihoodestimator,namely,thesampleproportion.

Example
LetX1,X2,...,Xnbenormalrandomvariableswithmeanandvariance2.Whatarethe
methodofmomentsestimatorsofthemeanandvariance2?

Solution.Thefirstandsecondtheoreticalmomentsabouttheoriginare:
E(Xi)=andE(Xi2)=2+2
(Incidentally,incaseit'snotobvious,thatsecondmomentcanbederivedfrom
manipulatingtheshortcutformulaforthevariance.)Inthiscase,wehavetwo
parametersforwhichwearetryingtoderivemethodofmomentsestimators.
Therefore,weneedtwoequationshere.Equatingthefirsttheoreticalmoment
abouttheoriginwiththecorrespondingsamplemoment,weget:

https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

2/6

4/27/2016

https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193
n

1
E(X) = =
n

Xi
i=1

And,equatingthesecondtheoreticalmomentabouttheoriginwiththe
correspondingsamplemoment,weget:
E(X

) =

=
n

2
i

i=1

Now,thefirstequationtellsusthatthemethodofmomentsestimatorforthe
meanisthesamplemean:
^

1
MM

=
n

Xi = X
i=1

And,substitutingthesamplemeaninforinthesecondequationandsolving
for2,wegetthatthemethodofmomentsestimatorforthevariance2is:
2

MM

=
n

2
i

1
n

i=1

2
i

i=1

whichcanberewrittenas:
2

MM

1
=
n

2
(X i X )
i=1

Again,forthisexample,themethodofmomentsestimatorsarethesameasthe
maximumlikelihoodestimators.

Insomecases,ratherthanusingthesamplemomentsabouttheorigin,itiseasiertouse
thesamplemomentsaboutthemean.Doingso,providesuswithanalternativeformof
themethodofmoments.

AnotherFormoftheMethod
Thebasicideabehindthisformofthemethodisto:

(1)EquatethefirstsamplemomentabouttheoriginM1

firsttheoreticalmomentE(X).
(2)EquatethesecondsamplemomentaboutthemeanM2

Xi = X

tothe

i=1

1
n

2
(X i X )
i=1

tothesecondtheoreticalmomentaboutthemeanE[(X ) ].
2

(3)ContinueequatingsamplemomentsaboutthemeanMk withthe
correspondingtheoreticalmomentsaboutthemeanE[(X )k ],k=3,4,...
https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

3/6

4/27/2016

https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

untilyouhaveasmanyequationsasyouhaveparameters.
(4)Solvefortheparameters.
Again,theresultingvaluesarecalledmethodofmomentsestimators.

Example
LetX1,X2,...,Xnbegammarandomvariableswithparametersand,sothatthe
probabilitydensityfunctionis:
1

f ( xi ) =

x/

()

forx>0.Therefore,thelikelihoodfunction:
L(, ) = (

()

) ( x1 x2 xn )

exp [

xi ]

isdifficulttodifferentiatebecauseofthegammafunction().So,ratherthanfindingthe
maximumlikelihoodestimators,whatarethemethodofmomentsestimatorsofand?

Solution.Thefirsttheoreticalmomentabouttheoriginis:
E(Xi)=
Andthesecondtheoreticalmomentaboutthemeanis:
Var(Xi)=E(Xi)2=2
Again,sincewehavetwoparametersforwhichwearetryingtoderivemethodof
momentsestimators,weneedtwoequations.Equatingthefirsttheoreticalmoment
abouttheoriginwiththecorrespondingsamplemoment,weget:
1
E(X) = =
n

Xi = X
i=1

And,equatingthesecondtheoreticalmomentaboutthemeanwiththecorresponding
samplemoment,weget:

V ar(X) =

1
=
n

2
(X i X )
i=1

Now,wejusthavetosolveforthetwoparametersand.Let'sstartbysolvingfor
inthefirstequation(E(X)).Doingso,weget:
https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

4/6

4/27/2016

https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

X
=

Now,substituting

X
=

intothesecondequation(Var(X)),weget:

X
= (

= X =

1
n

2
(X i X )
i=1

Now,solvingforinthatlastequation,andputtingonitshat,wegetthatthemethodof
momentestimatorforis:

^
MM =

2
(X i X )

nX

i=1

And,substitutingthatvalueofbackintotheequationwehavefor,andputtingonits
hat,wegetthatthemethodofmomentestimatorforis:

^MM =

X
=

^
MM

2
(1/nX ) (X i X )
i=1

nX
=

2
(X i X )
i=1

(AnotherTrivial)Example
Let'sreturntotheexampleinwhichX1,X2,...,Xnarenormalrandomvariableswith
meanandvariance2.Whatarethemethodofmomentsestimatorsofthemeanand
variance2?

Solution.Thefirsttheoreticalmomentabouttheoriginis:
E(Xi)=
And,thesecondtheoreticalmomentaboutthemeanis:
Var(Xi)=E(Xi)2=2
Again,sincewehavetwoparametersforwhichwearetryingtoderivemethodof
momentsestimators,weneedtwoequations.Equatingthefirsttheoreticalmoment
abouttheoriginwiththecorrespondingsamplemoment,weget:

https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

5/6

4/27/2016

https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193
n

Xi

E(X) = =
n

i=1

And,equatingthesecondtheoreticalmomentaboutthemeanwiththecorresponding
samplemoment,weget:

1
=
n

2
(X i X )
i=1

Now,wejusthavetosolveforthetwoparameters.Oh!Well,inthiscase,the
equationsarealreadysolvedforand2.Ourworkisdone!Wejustneedtoputahat
(^)ontheparameterstomakeitclearthattheyareestimators.Doingso,wegetthatthe
methodofmomentsestimatorofis:

= X
MM

(whichweknow,fromourpreviouswork,isunbiased).Themethodofmoments
estimatorof2is:
2

=
MM

1
n

2
(X i X )
i=1

(whichweknow,fromourpreviouswork,isbiased).Thisexample,inconjunctionwith
thesecondexample,illustrateshowthetwodifferentformsofthemethodcanrequire
varyingamountsofworkdependingonthesituation.

SourceURL:https://onlinecourses.science.psu.edu/stat414/node/193

https://onlinecourses.science.psu.edu/stat414/print/book/export/html/193

6/6