Fixed Effects FixedEffects

EITM2011
Christopher Berry ChristopherBerry
SimplestExample:DiffinDiff
Themostbasicdesign,thedifferenceindifferences
(DD)method,isalsousedinmanyrandomizedexperimentsin
hi h t t t d t ll t d whichpretreatmentdataarecollected.
Pre Post
ATE
Treatment Y
1
Y
2
Control Y
3
Y
4
ATE
(Y
2
Y
1
) (Y
4
Y
3
)
Theaveragetreatmenteffect (ATE)isthedifferencein
differences differences.
Innonexperimentalsetting,reliesonsomesortofexogenous
variationthatalterstreatmentforonegroupmorethan
another
Outcome
Average
Treatment
Effect
Y
2
Treatment Group
Effect
Y
4
Y
1
ControlGroup
Y
3
Time
Treatment
Happens
Outcome
Measuredeffect
without pre
Y
2
Treatment Group
withoutpre
measurement
Y
4
Y
1
ControlGroup
Y
3
Time
Treatment
Happens
Example: Berry & Gersen on Election Timing Example:Berry&GersenonElectionTiming
Table5:DifferenceinDifferences
Before After
Treatment 60,517 60,845
Control 62,516 63,433
Difference 1,999 2,588
Differenceindifferences 589 $
Note:Treatment denotesdi stri ctsthatswi tchedto
oncycl eel ecti ons;Control thosethatdi dnot.Before
denotesthel astyearofpretreatmentsal ary,1987;
After i stheaverageacrossposttreatmentyears,
1999to2008.Al l val uesareadjustedto2008dol l ars.
ComparetoAnzia(2011),whosecrosssectionalstudyusesonlythe
After dataandconcludesthattheposttreatmentdifferenceisthe
causaleffectofthetreatment.Effectively,sheisestimatingY
2
Y
4
.
Whatassumptionstoyouneedinordertoclaimthat(Y
2
Y
4
)is
equivalenttoATE?
Remember Why Were Doing This RememberWhyWe reDoingThis
TheFixedEffectsdesignisageneralizationoftheDiffinDiff
approach.Beforegoingintothedetails,letsrememberthe
motivation.
Wewouldliketoestimatetheeffectofthebinarytreatment,D,on
theoutcomeY.Theproblemisthatwedonteverobservethesame
unit under both treatment and control rather we observe one unitunderbothtreatmentandcontrol,ratherweobserveone
groupofunitsundertreatmentandanotherundercontrol.Using
OLS,wewouldestimate:
Y
i
= +D
i
+
i i

i

i
Theestimateof isjustthedifferencebetweenthemeanY forD =
1(thetreatmentgroup)andthemeanY forD =0(thecontrol
group) group)
0 0
1 1
Y


+ =
+ + =
Y
Thus,theOLSestimateis
( )
0 1 0 1
Y + = Y
UsingRegressiontoEstimateTreatment
ff ( i h d i ) Effects(withRandomAssignment)
If the treatment is randomly assigned then D is uncorrelated Ifthetreatmentisrandomlyassigned,then D isuncorrelated
with.
D i l t d ith if d l if D isuncorrelatedwith ifandonlyif
0 1
=
( ) = + =
0 1 0 1
Y Y
SowhenD
i
isuncorrelatedwith
i
thestandardmethod(OLS)
givesanunbiasedestimateof,whichistheaverage g g
treatmenteffect
ButsupposethereisaconcernthatD
i
isendogenous,correlated pp
i
g ,
with
i
RegressionwithoutRandomAssignment
(Si l C ) (SimpleCase)
Ifwedonothaverandomization,thereisnoguaranteethatD isuncorrelated
with (D may be endogenous) with (D maybeendogenous)
ThustheOLSestimateisstill
( ) Y Y
IfD iscorrelatedwith,then
( )
0 1 0 1
Y + = Y
0 1

,
Hence doesnotestimate, butsomeotherquantitythatdepends
onthecorrelationofD and.(Wewilldefinethisquantityshortly.)
0 1

0 1
Y Y
Takeaway:IfD iscorrelatedwith,thenOLSgivesabiasedestimateof
Rescued by the CIA? RescuedbytheCIA?
Howtogetcausalestimatesviaregressionwithoutrandomassignment?
SupposetheCIAholdsgivenavectorofobservedcovariates,X
i
.
Decomposetherandompartofthepotentialoutcome,
i
,intoa
linearfunctionofX
i
andv
i
:
v + = X
where isavectorofpopulationcoefficients.Since isdefinedbythe
regressionof
i
onX
i
theresidualv
i
and X
i
areuncorrelatedby
construction. Moreover, by virtue of the CIA, the residual in the
i i i
v + = X
construction.Moreover,byvirtueoftheCIA,theresidualinthe
regression:
isthereforeuncorrelatedwithD
i
andX
i
,and isthecausaleffectof
i i i i
v D Y + + + = X
i i
,
interest.
Thekeyassumption(CIA)isthatobservablecharacteristicsX
i
arethe
onlyreasonwhy
i
andD
i
arecorrelated(selectiononobservables;
unconfoundedness) A key question about any regression study is unconfoundedness).Akeyquestionaboutanyregressionstudyis
whetherthisassumptionisplausible.
Omitted Variables Bias (OVB) OmittedVariablesBias(OVB)
SupposeweleaveoutX
i
(forexample,becausewecannot
observe it). observeit).
Thentheestimateof hasbias:
( )
XD
E + =

where
XD
isthevectorofcoefficientsfromtheauxiliary
regressionsoftheelementsofX
i
onD
i
.
The bias is proportional to the correlation between X and D ThebiasisproportionaltothecorrelationbetweenX andD
(
XD
) andtotheeffectofX onY().
Toparaphrase:Shortequalslongplustheeffectofthe
omitted times the regression of omitted on included omittedtimestheregressionofomittedonincluded.
Bottomline:ifthereisanomittedvariablecorrelatedwith
bothtreatmentandoutcome, willNOTgivethecausal
estimateweareinterestedin.
What Does OVB Mean? WhatDoesOVBMean?
TheOVBformulaisoneofthemostimportant e O o u a s o e o t e ost po ta t
thingstoknowaboutregression
IfyouclaimnoOVBforyourstudy,youre y y y y
effectivelysayingthattheregressionyouhaveis
theregressionyouwant
Inotherwords,youdependontheCIAfora
causalinterpretationofyourregressionestimates
k l ti b bl f d d aka,selectiononobservables,unconfoundedness
Ifyoudonttrulybelievetheassumption,then
you shouldnt truly believe your regression youshouldn ttrulybelieveyourregression
How Does Panel Data Help Us? HowDoesPanelDataHelpUs?
Supposewehavemultipleobservationsonthesame
units(e.g.,individuals,states)overtime.Thenwecan ( g , , )
writethemotivatingequationas:
Y = D + X + Z + Y
it
=D
it
+X
it
+Z
i
+
it
wherei designatesunitsandt designatestimeperiods. g g p
Asbefore,D
it
istreatmentstatus,whichcannowvary
withinunitsovertime.
X is a vector of covariates that vary over time within X
it
isavectorofcovariatesthatvaryovertimewithin
units
Z
i
isavectorofcovariatesthatvaryacrossunitsbutnot
overtime;
it
istheerrorterm
FixedEffects
h b f h h d h Nowtherearesomeattributesoftheunitsthatdontchangeover
time,Z
i
.TheorytellsusthattheZ
i
sshouldbeinthemodel
(excludingthemleadstoOVB).
But suppose we cannot observe/measure all of the relevant Z s We Butsupposewecannotobserve/measurealloftherelevantZ
i
s.We
canstillremovetheZ
i
s(observableandunobservable)by
estimatingthefollowingequation,whichremovesunitlevel
averagesfrombothsides:
( ) ( ) ( )
i it i i i it i it i it
Z Z X X D D Y Y + + + =
NotethattheZ
i
sdropoutoftheequation becausetheydonotvary
i l l d ll h h h ll overtime. Importantly, notonlydoalltheZ
i
sthatourtheorytells
ustobeconcernedaboutdropout,butinfactanyZi thatwecould
everimagine,whetherobservableorunobsevable,dropsouttoo.
Note that deviating from unitlevel averages is equivalent to
X D Y + + + =
Notethatdeviatingfromunitlevelaveragesisequivalentto
estimatingafixedeffect,ordummyvariable,forearchindividual
unit,whichwedenote
i
.
it it it i it
X D Y + + +
Whyarethesetwoequationsequivalent?
SomeDetails
h h h diff i f d i i i Whyarethetwoapproachesdifferencingfrommeansandestimatingunit
specificdummiesequivalent?Remembertheregressionanatomyformula.
(MostlyHarmless, sec.3.1.2and5.1)
Anysetofregressioncoefficientscanbeestimatedin2steps.Togetthe
multivariatecoefficientononesetofvariables,firstregressthemallontheother
includedvariables.ThenregressYontheresidualsfromthefirststep.The
residualsfromaregressiononafullsetofunitleveldummiesinapanelare
deviationsfromunitlevelmeans.
C ll i i i h h fi d ff d Conceptually,itisimportanttonotethatthetermfixed effectdoesnot
actuallymeanthat
i
isbeingtreatedasnonrandom;itmeansthat
i
is
allowedtobearbitrarilycorrelatedwithD
it
andX
it
.(Forthisreason,
Wooldridgeadvocatesusing,c
i
,todenotethefixedeffectratherthanaGreek
l tt h H b bl i ht b t I f ll M tl H l i i letter,suchas.Hesprobablyright,butIfollowMostlyHarmlessinusing,
asthatisourtextfortheweek.SeeWooldridge,sec.10.2)
Arelatedapproachtofixedeffectsiffirstdifferencing.AgaintheZiswilldrop
outbecausetheydontvaryovertime.Withonlytwotime periods,FEandFD y y y p
areequivalent.Withmorethantwoperiodstheywillbedifferentand
estimatesubstantivelydifferentrelationships.Usetheregressionanatomy
forumlatothinkabouthowtheyaredifferent.TheFDmodelis:
( ) ( )
) 1 ( ) 1 ( ) 1 ( ) 1 (
+ + =
t i it t i it t i it t i it
X X D D Y Y
ThinkingaboutTiming
Source:Nichols2007
Interpretation Interpretation
Conditioningon
i
,thefixedeffects,controlsforall
unitspecific factorswhether observable or unit specificfactors whetherobservableor
unobservablethatareconstantovertime.
Thuswehaveremovedapotentiallylargesourceof
OVB The only omitted variables that threaten a FE OVB.TheonlyomittedvariablesthatthreatenaFE
designarethosethatchangewithinunitsovertime.
Wecanruleoutall timeinvariantunitlevelfactorsasa
f OVB h h b bl sourceofOVBeventhoughwemaynoteverbeableto
observeormeasurethem.
Wenowinterprettheestimatedeffect,,astheeffect p , ,
ofawithinunitchangeintreatment.Forthisreason,
theFEestimatorisalsocalledthewithin estimator.
In effect each unit serves as its own control group Ineffect,eachunitservesasitsowncontrolgroup.
Assumptions Assumptions
AversionoftheCIA,specifiedinchangesratherthanlevels
Alltheselectiononunobservables isduetotimeinvariant
factors
Putdifferently,notimevarying OVB,ornotimevaryingOV
thatiscorrelatedwithchangesintreatmentandchangesin
outcome.Theomittedvariabledoesntchangeovertime.
Statednontechnically:Exceptforthechangeinpolicy,the
twogroupsshouldnototherwisehavehaddifferent
changesovertime.
Alsoneedafunctionalformassumption:effectofthe p
treatmentislinearandadditive(needthisfortheZ
i
todrop
out).
Summary of Approach SummaryofApproach
Useacrosstimeorcohortvariationtocontrolforunobservedbut
fixedomittedvariables
Thisdesigncanbeextendedtotwowayandnwayfixedeffectsjust
astheDDcanbeextendedtothediffindiffindiffetc.
Cannotbeusedtocomparelevelsoftheoutcomebetween
t t t d t l j t h t d treatmentandcontrolgroups,justchangesortrends
Eachuniteffectivelyservesasitsowncontrolgroup
IfthevariableyoucareabouttheoreticallyisinZ
i
,i.e.,time
invariant then FE wont help you But if what you care about is invariant,thenFEwon thelpyou.Butifwhatyoucareaboutis
timevarying,D
it
,thenFEsweepsoutabunchofstuffyoudont
careaboutinordertogiveyouamorecredibleestimateofthe
thingyoudocareabout.
Thekeyidentifyingassumptionisthatthecounterfactualtrendin
treatmentandcontrolgroupsisthesame
Notimevaryingomittedvariables
W l h h ff f h i ddi i Wealsomustassumethattheeffectofthetreatmentisadditive
andconstant
Panel vs. Repeated CrossSections Panelvs.RepeatedCross Sections
Inmanycases,youcanusethesetechniqueswithrepeatedcrosssections
(repeatedsamplesfromthesamepopulation)insteadofpaneldata ( p p p p ) p
(repeatedobservationsofthesame individuals)
Sincethediffindiffestimatorisdefinedintermsofsamplemeans,itcanbe
computedequallywellusingpanelorrepeatedcrosssectiondata(RCS)
You must have grouped data or group indicators in the data Youmusthavegroupeddataorgroupindicatorsinthedata
GroupingdatatoobtainapseudopanelisanapplicationofIV
Thetreatmentvariablecanoftenbedefinedastheproportionofthe
groupreceivingthetreatment(whichmaybe100%,asinachangeinstate
law with individuals grouped by state) lawwithindividualsgroupedbystate)
RCSanalysisrequirestheassumptionthatunobservableindividualeffects
aredrawnfromthesamepopulationdistributionacrossperiodsbefore
andafterthetreatment.Otherwisethereisthepossibilityof
l b compositionalbias.
Essentially,assumethetruecausalrelationshipistableovertime.This
assumptionisnotrequiredforpanelanalysis.
AdvantagesofRCSarethatattritionandnonresponse issuesareless g p
severeandsamplesizesareusuallybigger.OftenRCSdatasetsgofarther
backintimebecausethissortofdatahasbeencollectedlongerthantrue
paneldata.
Fixedvs.RandomEffects
OneofthedesirablefeaturesoftheFEdesignisthatitallowsforthe
unitspecificeffecttobecorrelatedwiththeXs.Thusitexplicitly
accountsforoneformofendogeneity,thatresultingfromtime
invariant omitted variables invariantomittedvariables.
Analternativeapproachistouserandomeffects.TheREmodel
assumesthattheunitspecificeffectisuncorrelatedwiththeXs.
Therefore,thereisnoharminignoringit(noOVB)anditjustbecomes , g g ( ) j
partoftheresidual.
WhentheassumptionsoftheREmodelaresatisfied,itwillbemore
efficientthanFE.However,whentheassumptionsdonthold,itwillbe
bi d B t t FE ill b bi d b t i ffi i t biased.Bycontrast,FEwillbeunbiasedbutinefficient.
ThisleadstoaHausmantypetestforREvs.FE.IftheREassumptions
arevalid,theFEandREcoefficientsshouldbethesame.Testthat
hypothesis. hypothesis.
Bottomline:fromacausalinferenceperspective,REisnotparticularly
useful,sinceitrequiresassuming noOVB.
REiswayoverusedinpoliticalscienceandHausman testrarelyreported y p y p
ThechoiceofREvs.FEisnotamatterofpersonalpreferenceorstyle
DotheHausman test!Inmyownwork,IhaveneverseenacasewhereRE
isnotresoundinglyrejected
FEvs.LaggedDependentVariable
Usingalaggeddependentvariable(LDV)isanalternativetoFE,witha
similaridentificationframework(seeMostlyHarmless,sec.5.3)
LDV assumes essentially the OVB arises not from a timeinvariant LDVassumes,essentially,theOVBarisesnotfromatime invariant
unitlevelfactor,butfromtimevaryingpretreatmenttrends.
MostfamousexampleistheAshenfelter dip
Generallyspeaking,youdonotwanttousebothFEandLDVatthe y p g, y
sametime.Doingsorequiresdynamicpaneldatamodels(e.g.,
ArellanoBond)whichareverycomplexandrequirestrong
assumptions
Rather estimate FE and LDV models separately to check to robustness Rather,estimateFEandLDVmodelsseparatelytochecktorobustness
ofyourresults.Youwanttoseesimilarresultsusingdifferent
identifyingassumptions.
FE and LDV have a useful bracketing property FEandLDVhaveauseful bracketing property
IfthetruemodelisLDVbutyouuseFE,estimatesofapositivetreatment
effectwillbetoobig
IfthetruemodelisFEbutyouuseLDV,estimatesofapositivetreatment
ff t ill b t ll effectwillbetoosmall
ThereforeyoucanthinkofFEandLDVasboundingthecausaleffectof
interest(withsomeassumptions)
TestableImplicationsofAssumptions
Keyidentifyingassumptionisthattrendswouldbethe
samefortreatmentandcontrolgroupsintheabsenceof
the treatment thetreatment
Wecanttestthisdirectly,butwecantestfordifferencesin
trendsbeforethetreatmentisgiven
Thi t t b i l h i l Thistestcanbesimple:graphorsimplycomparemean
outcomesbytreatmentandcontrolovertime,lookingfor
theeffecttostartafterthetreatment,notbefore
It b d f ll i th K i k & W k Itcanbedonemoreformally,asintheKuziemko &Werker
paper,byincludingleadsandlags
Anotherapproachistoaddaunitspecifictimetrendsto
th d l Th ti t d b li l themodel.Thetimetrendscanbelinear,ormorecomplex,
aswithGentzkows 4
th
orderpolynomial
Worryifaddingsuchtrendscausesbigchangesinyourresults
Foranyoftheseapproaches,youneedenoughpre
treatmentdatatoestablishacleartrend
PotentialProblems
Forobviousreasons,youcannotestimatetimeinvariantcovariates
intheFEmodel.Ifthevariablewhoseeffectyoucareaboutdoesnt
changeovertime,FEisnotthedesignforyou.
Note,thislimitationdoesnotjustifytheuseofRE(unlessyouhave
otherreasonsforbelievingthenoOVB assumption).Rather,ittells
youthattheremaynotbeenoughvariationinyourdatatoestimate
theeffectyoucareabout.Findanotherdesign. y g
ThekeytoFEisthecommontrendsassumption.Inadditiontothe
sortsoftestsdescribedabove.Thinksubstantivelyaboutthe
problemandaskwhetherotherthingchangesimultaneouslywith
the treatment If so you need to find a way to control for them thetreatment.Ifso,youneedtofindawaytocontrolforthem.
FEestimatesarenotoriouslysusceptibletoattenuationbiasdueto
measurementerror
Watch for miscoding of withinunit changes Watchformiscodingofwithin unitchanges
Notappropriateincaseswherethecompositionofthetreatment
andcontrolgroupschangesasaresultofthetreatment
Example:generouswelfarebenefitsattractrecipientstoastate p g p
Evaluating a DD/FD/FE Paper EvaluatingaDD/FD/FEPaper
Isthisreallyanaturalexperiment;isthepolicychangeuncorrelatedwith
thechangeinoutcomeotherthanthroughthechangeintreatment?
Are there other things that change as a result of the policy change? Could they Arethereotherthingsthatchangeasaresultofthepolicychange?Couldthey
affecttheoutcome?
Inessence,thecontrolgroup(counterfactual)foreachunitisitself
underadifferenttreatmentstatus(e.g.,beforethepolicychange).Isthisa
good counterfactual? No general answer depends on the application goodcounterfactual?Nogeneralanswer;dependsontheapplication.
Aretherepreexistingtimetrendswithinunitsthatcouldaccountforthe
observedeffectsofthechangeintreatment?
Isthechangeintreatmentinducedbythechangeinpolicybigenoughto g y g p y g g
producemeasurable,quantitativelyimportantchangesinoutcomes?
Doestheanalysisidentifyshorttermorlongtermeffects?
Whichdoyouwant,fortheoreticalreasonsorpolicyrelevance?
R l t d t th h i b t FD d FE d d d l th f th l RelatedtothechoicebetweenFDandFE,anddependsonlengthofthepanel
andtheoreticalunderstandingofspeedofadjustment
Doestheanalysisaccountforserialcorrelation(orothertypesofnon
independence)intheresiduals?SeeMostlyHarmless,chap.8.
EXAMPLE PAPER
FixedEffects
EXAMPLEPAPER
HowMuchIsaSeatonthe
SecurityCouncilWorth?
Kuziemko andWerker
Overview Overview
Thebigquestionisinthetitle.Thenarrowerquestion
addressedinthepaperis:
Whatisthecausaleffectofsecuritycouncilmembership
onUSforeignaid? g
Istheevidenceconsistentwithvotebuying?
Whycantwejustregresssecuritycouncilmembership
on US Aid? onUSAid?
Of15seats,5arereservedforpermanentmembers
Remaining10seatsarereservedforcountriesserving2
year terms elected from regional blocs Countries yearterms,electedfromregionalblocs.Countries
campaignfornominationbytheirregion.
Neithercategoryofseatisrandomlyassigned.Sowehave
the usual OVB issues theusualOVBissues.
Empirical Strategy EmpiricalStrategy
UseFEmodeltostudydifferenceinUSaidduringyears y g y
whenacountryisonthecouncilvs.yearswhenthe
countryisoffthecouncil
I t t it il b hi ith d f Interactsecuritycouncilmembershipwithadummyfor
importantyears(lotsofNYtimescoverage).Ifvote
buyingisthemotive,weshouldexpectbiggeraid y g p gg
effectsinyearswhencouncilisinvolvedinimportant
globalissues.
Use an event time specification to confirm that Useaneventtimespecificationtoconfirmthat
changesinaiddidnotstartbeforejoiningthecouncil
anddonotcontinueafterleavingthecouncil.
ModelSpecification p
MainModel
WhereX
it
includestimevaryingcountrylevel covariates,W
rt
isaquartic
regionaltimetrend,
t
and
i
areyearandcountryFE,respectively.
EventtimeModel
WhereT1istheyearbeforecountryi iselectedtothecouncil, T0isyearof
election,T1T2isthetermofoffice,andT3T4aretwoyearsaftertermends.
Results
EventTimeGraph
Fixed Effects FixedEffects
EITM2011
Christopher Berry ChristopherBerry
Concluding Thoughts ConcludingThoughts
EITM2011
Christopher Berry ChristopherBerry
WhyWorryaboutResearchDesign?
Withobservationaldata,theassociationbetweenthe
outcomeandtheexplanatoryvariableofinterestislikelyto
be misleading because it partly reflects omitted factors that bemisleadingbecauseitpartlyreflectsomittedfactorsthat
arerelatedtobothvariables.
Ifthesefactorscouldbemeasuredandheldconstantina
i th itt d i bl bi ld b li i t d regression,theomittedvariablesbiaswouldbeeliminated.
But,theorytypicallydoesnotspecifyallofthevariables
thatshouldbeheldconstantwhileestimatingarelationship
d it i diffi lt t ll th l t i bl anditisdifficulttomeasurealltherelevantvariables
accuratelyeveniftheyarespecified.
Inotherwords,thereisalmostalwayssomeomitted
i bl bi i th t th i l t l variablesbias,meaningthatthereisalmostalwayssome
doubtaboutthecausalinterpretationofyourregression
Togetplausiblyunbiasedestimates,youneedagood
h d i researchdesign.
ElementsofResearchDesign
Key Element Example KeyElement Example
Atheorythatleadstotestablehypotheses Ifvotersdiscriminateagainstwomen,thewomen
whoareelectedmustbebetteronaveragethan
theirmalecounterparts
d h d ff f b Anestimand
Often acomparativestatic/partialderivative
Inthepotentialoutcomesframework,acausaleffectof
somekind(ATE,ATT,ITT,etc.)
Thedifference inperformancebetweenwomen
andmen,allelseequal
D t Di t i t l l f d l j t di Data Districtlevelfederalprojectspending
Bills sponsoredandcoponsored
Anestimator
Generated froma statisticalmodel
Involves an implicit definition of the counterfactual or
District fixedeffectsmodel

f
representswithindistrictdifferenceinoutlays
or bills during years with woman legislator vs Involves animplicitdefinitionofthecounterfactual,or
controlgroup
orbillsduringyearswithwomanlegislatorvs.
yearswithman
AssumptionsforIdentification
Whatassumptionsarenecessaryforyoutoclaimthat
your estimator is a valid empirical estimate of the
Outlays/bills arevalidproxiesforlegislator
performance
Nothing else within district is changing when sex yourestimator isavalidempiricalestimateofthe
estimand?
UsuallysomeversionoftheCIA
Assumptionsshouldalwaysbeexplicitlystated
Nothingelsewithin districtischangingwhensex
ofrepresentativechanges
Note:weakerthanCIA,becausedistricttime
invariantunobservables maydiffer
Tests/validationofassumptions Includelineartimetrends / p
Assumptionsareusuallynotdirectlytestable(orthey
wouldntbeassumptions)
Butoftenanimplicationoftheassumptioncanbetested
Orrobustnesstochangingassumptionscanbeshown
Shownochange intrendbeforewomanelected
Effectsdontholdforwidows
Effectsholdusingregressiondiscontinuitydesign
GoodDesign PlausibleAssumptions
Assumptionsofsomekindarealwaysnecessaryfor
identificationinobservationalstudies
Theplausibilityoftheresultsdependsontheplausibility
oftheassumptions(e.g.,CIA)
If you dont believe your assumptions then you shouldnt Ifyoudon tbelieveyourassumptions,thenyoushouldn t
believeyourresults
Yougettopickyourdataandyourmodel,butyoudont
get to pick your assumptions gettopickyourassumptions
Thedataandmodelimplythenecessaryassumptions
Ifyouareuncomfortablewithyourassumptions,you
shouldconsiderchangingsomeotherelementofyour
researchdesign
Changingdata,model,orevenestimand mayallowmore g g , , y
plausibleassumptions
Identifyingassumptionsshouldalwaysbestatedexplicitly
SummaryofDesignsfromThisWeek
Design KeyIdentifyingAssumption(Stated Informally)
Regression Conditionalindependence:Conditionalonthecovariatesincluded
in the model the treatment is as good as randomly assigned No inthemodel,thetreatmentisasgoodasrandomlyassigned.No
omittedvariablebias.
Matching Seeabove.
IV E l i t i ti Th i t t ff t th t t t b t h IV Exclusionrestriction:The instrumentaffectsthetreatment,buthas
noeffectonthedependentvariableotherwise.
RD Continuity:Nothing elsechangesdiscontinuouslyatthecutoff.The
units just above and just below are the same but for the treatment unitsjustaboveandjustbelowarethesamebutforthetreatment.
FixedEffects Exceptforthechangeintreatment,thetwogroupsshouldnot
otherwisehavehaddifferentchangesovertime.NoOVBfromtime
varyingomitted variables. y g
Notheoremcantellyouwhetheranyoftheseassumptionsisplausibleinyourcase.
Theplausibilityoftheassumptionmustbeassessedbasedonyoursubstantive
k l d f th bl t d i knowledgeoftheproblemyouarestudying.
WhatToDoIfYoureStuckwithOLS
Supposedatalimitationspreventyoufromhavingagreatresearch
design.Whentheusualtechniquesrequireidentifyingassumptions
(e g CIA) that are not tenable in a particular case ask what can be (e.g.,CIA)thatarenottenableinaparticularcase,askwhatcanbe
learnedfromthedata(ifanything)withweaker(orno)
assumptions.Kosuketaughttwousefultoolsforthisearlierinthe
week.
Manskis partialidentificationtechniquesfocusonboundsofa
treatmenteffect,ratherthanpointestimates,basedonveryweak
(orno)assumptions.Whatcanwelearnfromthedatawehave
without making assumptions we dont believe? withoutmakingassumptionswedon tbelieve?
Rosenbaumssensitivityanalysisattemptstoquantifyjusthow
strongtheCIAisinpractice.
The key idea is to revisit on implication of the CIA: controlling for ThekeyideaistorevisitonimplicationoftheCIA:controllingfor
covariates,thetreatmentandcontrolgroupsareequallylikelytohave
receivedthetreatment.Calltheratioofprobabilitiesgamma=1
Thenask,howdifferentwouldtheiroddsoftreatmenthavetobein
order to change the conclusions of the study Big gammamore ordertochangetheconclusionsofthestudy.Biggammamore
threatfromOVB
ConcludingThoughts
Causalinferenceisfundamentallybasedonclearthinking
andsubstantiveknowledgeoftheselectionprocess,not
fancy econometrics fancyeconometrics
Researchdesignistheartofcombiningdata,statistics,and
assumptions toproducecrediblecausalinference
Al t t id tif i ti li itl d k Alwaysstateyouridentifyingassumptionsexplicitlyandask
whetheryoucoulddefendthemwithastraightfacebefore
askepticalaudience
C t t d t ti th b t f Concentrateondemonstratingtherobustnessofyour
resultstoalternativeidentificationstrategiesratherthan
findingtheonetruemodel
N t ll ti bl i il bl d t J t Notallquestionsareanswerablegivenavailabledata.Just
acceptthat.
Pleasefillouttheonlinesurveywithasmuchfeedbackas
ibl possible
Goforthandconquer!
ADifferentView
InthecurrentissueofPoliticalAnalysis,oneofthe
editorsoffersthefollowingviewoffixedeffects: g
Theunitspecificinterceptsaresimplypropertiesofthe
variouspoliticalandeconomicsystemsthatwe
currentlycannotorhavenotmeasured.InthelargeN
panels used in labor economics, it is likely the case that panelsusedinlaboreconomics,itislikelythecasethat
theunexplainedpersontopersonvariationis
idiosyncraticandlikelybothnotworthandimpossible
to model But in comparative political economy we tomodel.Butincomparativepoliticaleconomy,we
oughttobeabletomodelwhynationsdifferandto
maketheunmodeled unitheterogeneitysmallenough
tobeignorable.Tomymind,thisshouldbeourgoal.