' $

Simultaneous Estimation

General Example

Xi ∼ f (·|θi ), i = 1, ..., n Xi ∼ binomial(N, pi )

θi ∼ π(·|ψ) pi ∼ beta(α, β)

π(θi |Xi , ψ) = pi |Xi , α, β ∼

f (Xi |θi )π(θi |ψ) beta(Xi + α, N − Xi + β)
Z
f (Xi |θi )π(θi |ψ)dθi

Context For example, n students all take a multiple-choice test

& %
consisting of N questions, and pi is a measure of student i’s ability.

ActSc 613 | Lecture 23 © 2015-17 by M. Zhu, PhD 1 of 2

' $

Empirical Bayes
• do not specify ψ, e.g., choose (α, β) = (1, 1) or ( 21 , 12 )
• can integrate out θi to obtain the marginal distributuion of Xi :
Z
m(xi |ψ) = f (xi |θi )π(θi |ψ)dθi

iid
• X1 , ..., Xn ∼ m(·|ψ) all contain some information about ψ
• can obtain an estimate, say ψ,b using all of X1 , ..., Xn ; and use
b
the “plug-in” posterior, π(θi |Xi , ψ)
• e.g., obtain (b b using all X1 , ..., Xn , and use
α, β)
beta(Xi + α b for posterior inference
b, N − Xi + β)

Remark Alternatively, can put another prior on ψ. Called

& %
hierarchical Bayes. Can keep on doing it but eventually must stop.

ActSc 613 | Lecture 23 © 2015-17 by M. Zhu, PhD 2 of 2