ECO519.

Spring2009(FirstHalf)
AndriyNorets(anorets@princeton.edu)
OfficeHours:Thursday3:005:00pm,313FisherHall

The first half of the course will be devoted to the Bayesian approach to econometrics. If class participants express
interest,wewillalsocoverestimationofdynamicdiscretechoicemodels.Thegradeforthefirsthalfwillbebasedon
problemsetsandaproject.Detailedinstructionsontheprojectwillbeprovided.Theproblemsetsandtheprojectwill
requiretheuseofaprogramminglanguage,e.g.,Matlab,R,Fortran,orC.ThewebsiteforthecourseisonBlackboard.
Atentativelistoftopicswithcorrespondingreadingsisgivenbelow.Thislistwillbeupdatedaswegooverthematerial.
Readingsmarkedwithasterisk*arenotrequired.

Topics

1.FoundationsofBayesianapproach

probabilityasadegreeofbeliefandconditioning;
likelihoodprinciple(GCh1,3.5;BergerandWolpert,Ch13);
DeFinettitheorem(HeathandSudderth(1976)*,Scherwish(1995)*);
modeling:ingredients,completemodel,estimation,crediblesets,modelcomparison(GCh2,Geweke(2007));
priordistributions:conjugate,noninformative,hierarchical(GCh3.1,3.2;Berger(1985),Ch3*;Bernardo
(1979)*);
someasymptoticresults(GCh.3.4).

2.Posteriorsimulation

Basicsimulationmethods:pseudorandomnumbers,inversecdfmethod,acceptanceandimportancesampling
(GCh4,GilksandWild (1992)*);
IntrotoMCMC,Gibbssampling,MetropolisHastings,andHybridMCMCalgorithms(GCh4);
Examples:linearmodels,modelswithlatentvariables,timeseriesmodels,SUR,mixturesofdistributions(GCh
5,6,7);
Software:BACC,WinBUGS,Matlab,R;
Practicalissues:implementationchecks,convergencetesting,numericalaccuracy,improvingmixing,variance
reduction,(GCh4.4,4.7,8.1);
Computingmarginallikelihood.

3.Markovchainsongeneralstatespace(GCh4,Tierney(1994),RobertsandRosenthal(2004)*);
4.Bayesiannonparametrics;Dirichletprocessprior

Introduction(GoshandRamamoorthi(2003),ch3;Fergueson(1973),Antoniak(1973),Sethuraman(1994));
Applications(EscobarandWest(1998);Hirano(1998);Conley,Hansen,McCulloch,Rossi(2007))

5.Estimationofdynamicdiscretechoicemodels

Rust(1994)inHandbookofEconometrics,Volume4;surveybyAguirregabiriaandMira(2008);
Bayesianinference(Norets(2007))

Books:
Thesetwobooksareverybasic,theyincludeexamplesinMatlab(Koop)andWinBUGS(Lancaster),providealink
betweenundergraduateeconometricsandBayesianapproach:

Koop,2003,Bayesianeconometrics
Lancaster,2004,AnIntroductiontomodernBayesianeconometrics

ThisisanappliedBayesianstatisticsbookwithlotsofpracticalsuggestions.Howeveritisgearedtowardsbio
statisticiansratherthaneconometricians:

Gelman,Carlin,Stern,Rubin,2004,Bayesiandataanalysis.

Themaintextforthecourse,moreadvancedlevel,masteringthematerialinthisbookwouldallowyoutostartdoing
researchinthefieldofBayesianeconometrics:

Geweke,2005,ContemporaryBayesianeconometricsandstatistics.

Foundations:

BernardoandSmith,1994,BayesianTheory.Anintroductiontosubjectiveprobabilityandhowitisrelatedto
Bayesianapproachtostatistics.
Berger,1985,StatisticaldecisiontheoryandBayesianAnalysis.Acomprehensivebookondecisiontheoryand
theBayesianapproachinpreMCMCera.
BergerandWolpert,1988,TheLikelihoodPrinciple.

AmathematicallyrigorousbookonBayesianandfrequentiststatisticaltheory:

Scherwish,1995,Theoryofstatistics.

TheoryforMarkovchains

Nummelin,1984.GeneralirreducibleMarkovchainsandnonnegativeoperators.CambridgeUniv.Press.
MeynandTweedie,1993.Markovchainsandstochasticstability.SpringerVerlag,London.Availableonline:
http://probability.ca/MT/

Bayesiannonparametrics

GhoshandRamamoorthi(2003).Bayesiannonparametrics.Springer.Availableinelectronicformfrom
Princetonlibrary.


Papers:

Heath,D.L.andSudderth,W.D.(1976).DeFinetti'stheoremforexchangeablerandomvariables.Amer.Statist.
30,333345.
Geweke,2007."BayesianModelComparisonandValidation",AmericanEconomicReview97(2),pp.6064.
Availableathttp://www.biz.uiowa.edu/faculty/jgeweke/papers/paper95/ms.pdf
Bernardo1979.ReferenceposteriordistributionsforBayesianinference(withdiscussion).J.Roy.Statist.Soc.,
Ser.B41,113147.
GilksandWild, 1992. AdaptiveRejectionSamplingforGibbsSampling.AppliedStatistics,Vol.41,No.2.,pp.337
348.

ThispaperwasthefirsttoshowhowtouseageneraltheoryofMarkovchains(fromamonographbyNummelin(1984))
forestablishingconvergenceofMCMCalgorithms:

Tierney,1994,MarkovChainsforExploringPosteriorDistributions,TheAnnalsofStatistics,Vol.22,No.4.(Dec.,
1994),pp.17011728.

StableURL:http://links.jstor.org/sici?sici=00905364%28199412%2922%3A4%3C1701%3AMCFEPD%3E2.0.CO%3B26
AmorerecentpaperonMCMCandMarkovchaintheory;itcontainsafewselfcontainedproofs,moreinlinewitha
newerbookonMarkovchainsbyMeynandTweedie(1993):

G.O.RobertsandJ.S.Rosenthal,GeneralstatespaceMarkovchainsandMCMCalgorithms.(ProbabilitySurveys
1:2071,2004.)Availableat:http://probability.ca/jeff/research.html

Bayesiannonparametrics:

Fergueson(1973),"ABayesianAnalysisofSomeNonparametricProblems,"TheAnnalsofStatistics,1,209230.
Antoniak(1974),"MixturesofDirichletProcessesWithApplicationstoBayesianNonparametricProblems,"The
AnnalsofStatistics,2,11521174.
Sethuraman(1994),AconstructivedefinitionofDirichletpriors,StatisticaSinica4,639650.
EscobarandWest(1998),ComputingnonparametrichierarchicalmodelsinPracticalnonparametricand
semiparametricBayesianstatistics,ed.Dey,Muller,Sinha.
Conley,Hansen,McCulloch,Rossi(2007),AsemiparametricBayesianapproachtotheinstrumentalvariable
problem.Manuscript,ChicagoGSB.
Hirano(1998),PhDdissertation,availableatKeisukeHiranoswebsite.


Software
WinBUGS
http://www.mrcbsu.cam.ac.uk/bugs/winbugs/contents.shtml
Youonlyprovideadescriptionofyourmodel,anMCMCalgorithmiscreatedautomaticallyforyou.Itcansave
youenormousamountoftime(ifitworksforyourmodel.)Itoftenwontworkandwontprovideasensible
descriptionoftheproblem.Youmightbeabletofixsomeproblemsbychangingdefaultalgorithmsitusesfor
samplinginparticularcircumstances,e.g.,replacingslicesamplingbyMHalgorithmworkedforme.
WinBUGScanalsobecalledfromR:http://www.stat.columbia.edu/~gelman/bugsR/,seealsoLancasterstext.
Demonstration.

BACC

http://www2.cirano.qc.ca/~bacc/
ProvidesposteriorsimulatorsformodelsdescribedinGeweke(2005).
ItcanbecalledfromMatlabandGauss,seeGCh5.
Ccodeisavailable.IuseBACCCcodeforworkingwithmatricesandgeneratingrandomvariables.
ItcanbecompiledunderWindowsbyVisualC++(oneof)thebestintegrateddevelopmentenvironment
available.AfreeversionofVisualC++isavailablefromMicrosoft,youcannotusethisfreeversiontocreate
windowsinterfacesbutyoucancompileconsoleapplications,includingBACC.)
TheCcodecanalsobecompiledunderLinux;so,youcanuseUnixserverstorunlongsimulations
Demonstration

Matlab
someexamplesontheKoopsbookwebsite,http://www.wiley.com/legacy/wileychi/koopbayesian/
checkBlackboardforbasicroutinesforMCMCoutputanalysis
EconometricsToolboxbyJamesP.LeSage:http://www.spatialeconometrics.com/routinesforgeneratingRV,
MCMCoutputanalysisandotherusefulcode.
R

AlistofpackagesforBayesiancomputinginR:http://cran.rproject.org/web/views/Bayesian.html
CODAOutputanalysisanddiagnosticsforMCMC: http://cran.r-project.org/web/packages/coda/index.html