Newsom 1

USP 656 Multilevel Regression

Winter 2006
Jkj..khh.kh.khk.jhb.jkk..klhhlhllklhlllkklhl !i.e. 0 mg 5 mg "n# $0 mg%. &owever i' the rese"r(her
w"nts to m"ke in'eren(es be)on# the *"rti(ul"r v"lues o' the in#e*en#ent v"ri"ble use# in the stu#)
" r"n#om e''e(ts mo#el is use#. + (ommon e,"m*le woul# be the use o' *ubli( "rt works
re*resenting low mo#er"te "n# high "bstr"(tness !e.g. st"tue o' " w"r hero vs. " *ivoting geometri(
#esign%. -he rese"r(her woul# like to ,(v.'v,(.v.(,v.,(v re'erre# to "s /Mo#el 001 or /v"ri"n(e
(om*onent mo#els.1 +n"l)ses using both 'i,e# "n# r"n#om e''e(ts "re ("lle# /mi,e# mo#els.1
Fixed and Random Coefficients in Multilevel Regression
-he r"n#om vs. 'i,e# #istin(tion 'or v"ri"bles "n# e''e(ts is im*ort"nt in multilevel regression. 0n
multilevel regression mo#els both level2$ "n# level22 predictors "re "ssume# to be 'i,e#. &owever
level2$ intercepts and slopes "re t)*i("ll) "ssume# to v"r) r"n#oml) "(ross grou*s. 3e("use o' the
"ssum*tions "bout their error #istributions we ("ll their v"ri"n(es oo=var(U0j) "n# 11=var(U1j),
/r"n#om (oe''i(ients.1
-his me"ns th"t we ("n think "bout 0j "n# 1j "s "kin to the r"n#om variables 0 #es(ribe#
"bove. 0nste"# o' "ttem*ting to gener"li.e be)on# the *"rti(ul"r v"lues o' the in#e*en#ent v"ri"ble
we "re "ttem*ting to gener"li.e be)on# the *"rti(ul"r grou*s in the stu#). 4or inst"n(e we m") h"ve
50 (om*"nies but we wish to gener"li.e to " l"rger universe o' (om*"nies when we e,"mine the
me"ns !inter(e*ts% or the ,2) rel"tionshi* !slo*es%.
0n &5M v"ri"n(es 'or the inter(e*ts "n# slo*es "re estim"te# b) #e'"ult. -h"t is when )ou
'irst (onstru(t " mo#el U0j "n# U1j "re estim"te# b) #e'"ult. &owever the rese"r(her h"s the (hoi(e
o' setting U0j "n# U1j to be .ero. 3) setting them to .ero we "re testing " mo#el in whi(h we "ssume
0j "n# 1j #o not v"r) r"n#oml) "(ross grou*s. 0n '"(t their v"ri"n(e is "ssume# to be .ero so the)
"re "ssume# to be (onst"nt or /nonv"r)ing1 "(ross grou*s. 4or e,"m*le 'i,e# nonv"r)ing inter(e*ts
woul# im*l) the grou* "ver"ge 'or the #e*en#ent v"ri"ble is "ssume# to be e6u"l in e"(h grou*.
+lthough we t)*i("ll) re'er to this (onstr"int "s /'i,ing the inter(e*ts1 or /'i,ing the slo*es1 the term is
somewh"t loosel) "**lie# be("use we "re re"ll) "ssuming the) "re 'i,e# and nonvarying.
Intercept Model and Random Effects ANOVA
0n the sim*lest &5M mo#el there "re no *re#i(tors. We h"ve one level2$ e6u"tion "n# one level22
e6u"tion7
0 ij j ij
Y R = +
where Y is the #e*en#ent v"ri"ble 0j is the inter(e*t "n# Rij is the resi#u"l or error. 0j ("n be
thought o' "s the me"n o' e"(h grou*. -he level22 e6u"tion "lso h"s no *re#i(tors in it8s sim*lest
'orm7
0 00 0 j j
U = +
where 0j is the #e*en#ent v"ri"ble 00 is the level22 inter(e*t "n# U0j is the level22 error. 0n this
e6u"tion 00 re*resents the gr"n# me"n or the me"n o' the inter(e*ts. U0j re*resents the #evi"tion o'
e"(h me"n 'rom the gr"n# me"n. When the "ver"ge #evi"tion is l"rge there "re l"rge grou*
#i''eren(es.
0t ("n be shown th"t this is re"ll) the +N9:+ mo#el. +N9:+ e,"mines the #evi"tions o' grou* me"ns
'rom the gr"n# me"n. 3e("use we "ssume th"t the grou* me"ns re*resente# b) 0j "n# thus their
#evi"tions "re r"n#oml) v"r)ing this mo#el is e6uiv"lent to the random effects +N9:+ mo#el. We
("n rewrite our two &5M e6u"tions "s " single e6u"tion i' we *lug in the level22 terms into level2$
e6u"tion.
00 0 ij j ij
Y U R = + +
0n this e6u"tion 00 re*resents the gr"n# me"n the U0j term re*resents the #evi"tions 'rom the me"n
!or me"n #i''eren(es or tre"tment e''e(t i' )ou woul# like% "n# Rij re*resents the within2grou* error or
Newsom 2
USP 656 Multilevel Regression
Winter 2006
v"ri"tion. 0n more m"them"ti("ll) oriente# +N9:+ te,tbooks )ou will see "n +N9:+ mo#el written
something like7
ij j ij
Y e = + +
0' the mo#el is "n r"n#om e''e(ts +N9:+ j is "ssume# to h"ve " r"n#om #istribution. 0n multilevel
regression be("use we "ssume U0j to v"r) r"n#oml) the sim*le &5M mo#el with no level2$ or level22
*re#i(tors is e6uiv"lent to the r"n#om e''e(ts +N9:+ mo#el.
When we "## *re#i(tors to the level2$ e6u"tion the) "re (ov"ri"tes "n# the mo#el be(omes "
r"n#om e''e(ts +N;9:+ in whi(h the me"ns "re "#juste# 'or the (ov"ri"te.
Newsom 3
USP 656 Multilevel Regression
Winter 2006
Summary a!le
Random vs"
Fixed
#efinition Example $se in Multilevel
Regression
Varia!les Random variable7 !$% is "ssume# to be me"sure# with
me"surement error. -he s(ores "re " 'un(tion o' " true
s(ore "n# r"n#om error< !2% the v"lues (ome 'rom "n#
"re inten#e# to gener"li.e to " mu(h l"rger *o*ul"tion
o' *ossible v"lues with " (ert"in *rob"bilit) #istribution
!e.g. norm"l #istribution%< !=% the number o' v"lues in
the stu#) is sm"ll rel"tive to the v"lues o' the v"ri"ble "s
it "**e"rs in the *o*ul"tion it is #r"wn 'rom. Fixed
variable7 !$% "ssume# to be me"sure# without
me"surement error< !2% #esire# gener"li."tion to
*o*ul"tion or other stu#ies is to the s"me v"lues< !=%
the v"ri"ble use# in the stu#) (ont"ins "ll or most o' the
v"ri"ble8s v"lues in the *o*ul"tion.
0t is im*ort"nt to #istinguish between " v"ri"ble th"t is
varying "n# " v"ri"ble th"t is random. + 'i,e# v"ri"ble
("n h"ve #i''erent v"lues it is not ne(ess"ril) inv"ri"nt
!e6u"l% "(ross grou*s.
Random variable:
*hotogr"*hs
re*resenting in#ivi#u"ls
with #i''ering levels o'
"ttr"(tiveness
m"ni*ul"te# in "n
e,*eriment (ensus
tr"(ks
Fixed variable:
gen#er r"(e or
intervention vs. (ontrol
grou*.
Pre#i(tor
v"ri"bles in M5R
gener"ll)
"ssume# to be
'i,e#
Effects Random effect7 !$% #i''erent st"tisti("l mo#el o'
regression or +N9:+ mo#el whi(h "ssumes th"t "n
in#e*en#ent v"ri"ble is r"n#om< !2% gener"ll) use# i'
the levels o' the in#e*en#ent v"ri"ble "re thought to be
" sm"ll subset o' the *ossible v"lues whi(h one wishes
to gener"li.e to< !=% will *rob"bl) *ro#u(e l"rger
st"n#"r# errors !less *ower'ul%. Fixed effect7 !$%
st"tisti("l mo#el t)*i("ll) use# in regression "n# +N9:+
"ssuming in#e*en#ent v"ri"ble is 'i,e#< !2%
gener"li."tion o' the results "**l) to simil"r v"lues o'
in#e*en#ent v"ri"ble in the *o*ul"tion or in other
stu#ies< !=% will *rob"bl) *ro#u(e sm"ller st"n#"r#
errors !more *ower'ul%.
Random effect7
r"n#om e''e(ts +N9:+
r"n#om e''e(ts
regression
Fixed effect7 'i,e#
e''e(ts +N9:+ 'i,e#
e''e(ts regression
0nter(e*t onl)
mo#els in M5R
"re e6uiv"lent to
r"n#om e''e(ts
+N9:+ or
+N;9:+.
Coefficients Random coefficient: term "**lies onl) to M5R
"n"l)ses in whi(h inter(e*ts slo*es "n# v"ri"n(es ("n
be "ssume# to be r"n#om. M5R "n"l)ses most
t)*i("ll) "ssume r"n#om (oe''i(ients. 9ne ("n
(on(e*tu"li.e the (oe''i(ients obt"ine# 'rom the level2$
regressions "s " t)*e o' r"n#om v"ri"ble whi(h (omes
'rom "n# gener"li.es to " #istribution o' *ossible v"lues.
>rou*s "re (on(eive# o' "s " subset o' the *ossible
grou*s.
Fixed coefficient: " (oe''i(ient ("n be 'i,e# to be non2
v"r)ing !inv"ri"nt% "(ross grou*s b) setting its between
grou* v"ri"n(e to .ero.
R"n#om (oe''i(ients must be v"ri"ble "(ross grou*s.
;on(e*tu"ll) 'i,e# (oe''i(ients m") be inv"ri"nt or
v"r)ing "(ross grou*s.
Random coefficient7
the level22 *re#i(tor
"ver"ge in(ome is
use# to *re#i(t s(hool
*er'orm"n(e in e"(h
s(hool. 0nter(e*t
v"lues 'or s(hool
*er'orm"n(e "re
"ssume# to be "
s"m*le o' the inter(e*ts
'rom " l"rger *o*ul"tion
o' s(hools.
Fixed coefficient7
slo*es or inter(e*ts
(onstr"ine# to be e6u"l
over #i''erent s(hools.
3oth use# in
M5R. Slo*es "n#
inter(e*t v"lues
("n be
(onsi#ere# to be
'i,e# or r"n#om
#e*en#ing on
rese"r(hers
"ssum*tions "n#
how the mo#el is
s*e(i'ie#. -he
v"ri"n(e o' the
slo*es or
inter(e*ts "re
(onsi#ere#
r"n#om
(oe''i(ients.