ROBUST STATISTICAL FUSION OF IMAGE LABELS

Landman et al IEEE Transactions on Medical Imaging VOL. 31, NO. 2, FEBRUARY 2012

Labeling in MR image analysis

Labeling Problem Identifying class membership of voxels Currently no true answer
Manual Voxel-by-voxel Labeling Considered as a gold standard Exceptionally time consuming and resource intensive Difference in interpretation between raters Validating automatic or semi-automatic methods The study of structures for which no automated method exists

Manual Labeling
The objective in manual labeling To produce more accurate and reproducible results for each rater
The approach Estimate true labels (true segmentation) Simultaneously estimate rater reliability as well STAPLE algorithm (Warfield et al. 2004, Rohfling et al.

2003)
Simultaneous Truth And Performance Level Estimation Used in multi-atlas segmentation (Heckmann et al. 2006, Lotijonen

et al. 2010, Langerak et al. 2010) Later extended to be applicable to continuous vector images (Warfield et al. 2009, Commonwick et al. 2009)

Limitation of STAPLE algorithm

STAPLE requires that all raters delineate all voxels

within a given region Raters are often requested to label datasets more than once in order to establish a measure of intra-rater reliability. Raters are often divided into a class of experts and novices Label inversion problem due to highly inaccurate raters

Label inversion problem

STAPLER algorithm
Algorithm Estimate a maximum a posteriori of both rater reliability and true labels in the Expectation Maximization framework
Evaluation Random rater simulation Boundary random rater simulation Simulations to characterize the occurrence of label inversion problem Note Minimally trained raters and large number of participants

STAPLER algorithm
How to estimate true labels? Majority voting :
How to estimate rater performance? Rater performance is not considered to be perfect but to be fuzzy. | : [ 0 , 1 , 2 , , (1 )] Confusion matrix (hidden variable in EM framework)

1 1 2 0.9 0.1

2 0.2 0.8

EM algorithm
Expectation Maximization Parameter estimation in probabilistic models with incomplete data Computes iteratively the Maximum Likelihood estimation with the assumption of hidden variable
Toy Example: Coin flipping with two different coins, A and B

= { , } : 10 trials of flipping. = , , , , , , , , , , , = 1 , 2 , , 5 = {, }

EM algorithm
Assume that we know which coin is flipped, for example:
Coin
B A A B A
Results H T T T H H T H T H (5H 5T) H H H H T H H H H H (9H 1T) H H H H T H T H H H (8H 2T) H T T T T H H T H T (4H 6T) H T H H T T H H H H (7H 3T)

How to estimate = , maximizing the likelihood of

(| , )?

A: 24 Head 6 Tail
B: 9 Head 11 Tail

EM algorithm
=
=
24 24+6 9 9+11

= 0.8
= 0.45

Maximum Likelihood Estimation

arg max (| , )

=
23

30 24 1 24
6 5

6 5

23 24 1

24 6 1

24 24 6 24 30 = 0

EM algorithm
What if we dont know which coin is used? = {1 , 2 , , 5 } is called hidden variable or latent factor Maximizing log with respect to Efficient iterative process and guarantees to converge
Repeat E-step and M-step until the algorithm converges E-step: , [log (, |)]
M-step: arg max , [log (, |)]

EM algorithm becomes very slow as the number of

variables increases.

EM algorithm
Begin with initial parameters , = 0.6, 0.5
Compute the probability of the event for A and B
Event
#1 (5H 5T) #2 (9H 1T) #3 (8H 2T) #4 (4H 6T) #5 (7H 3T)

(, = , )
0.2007 (105 0.65 0.45 ) 0.04 0.1209 0.1115 0.2150

(, = , )
0.2495 (105 0.55 0.55 ) 0.0098 0.0439 0.2051 0.1172

( = |, ) 0.2/(0.2+0.25)=0.45

EM algorithm
Compute expectation (E-step)
Event #1 (5H 5T) #2 (9H 1T) #3 (8H 2T) #4 (4H 6T) #5 (7H 3T) ( = |, ) ( = |, ) ( , = ) ( , = ) 0.45 0.80 0.73 0.35 0.65 0.55 0.2 0.27 0.65 0.35 2.2H 2.2T 7.2H 0.8T 5.9H 1.5T 1.4H 2.1T 4.5H 1.9T 2.8H 2.8T 1.8H 0.2T 2.1H 0.5T 2.6H 3.9T 2.5H 1.1T

Estimate new parameter (M-step)

1 = 21.3 21.3+8.6 1 = 0.71, = 11.7 11.7+8.4

= 0.58

EM algorithm
1.
2. 3. 4. 5.

Guess initial model parameters Compute the conditional probability for every possible hidden variables Estimate new model parameters Iterate until the parameters converge After 10 iterations
10 10 = 0.8, = 0.52

Label Fusion
An image of N = + voxels : the number of training voxels : the number of undetermined voxels
, , are the set of all voxels = {0,2,3, , L 1} is all possible labels = {1 , 2 , } is a collection of raters represents the r-th observation of voxel i by rater j represents the hidden true segmentation

STAPLE algorithm
Estimate true segmentation in a probabilistic framework : [ 0 , 1 , 2 , , (1 )] :
Probability distribution function of the true label
() ( matrix) represents the probability that voxel i has true

label s on the k-th iteration

Hidden variable that control the p.d.f

( |) ( matrix) represents the probability that rater j

observes label s when the true label is s on the k-th iteration (rater performance)

STAPLE algorithm
EM algorithm to estimate the hidden true segmentation
E-step: the calculation of the conditional probability of

the true segmentation

()

= ,

( =)

( |) ( |)

( =)

M-step: the calculation of the rater performance

parameters
+1 ( |) =
: =

()

()

STAPLE algorithm
Example
Rater Label 1 Label 2 A 1 2 1 0.9 0.1 2 0.2 0.8 A 1 2 B 1 2 1 0.7 0.3 2 0.4 0.6 B 1 1 C 1 2 1 0.5 0.5 C 2 1 2 0.5 0.5

1 1 = 2 2 =

0.90.70.5 0.90.70.5+0.10.30.5 0.80.30.5 0.80.30.5+0.20.60.5

= 0.9545 = 0.6667

.9545 0.9545 + 0.3333 .3333 1 2 1 = 0.9545 + 0.3333 1 1 1 =

STAPLER algorithm
New E-step

= ,

( =)

( |)

( =)

where = +1 is a global prior

= +1 =

New M-step
+1 =

= () ()

Results
Data A high-resolution MPRAGE 149 x 81 x 39 voxels 0.82 x 0.82 x 1.5 mm resolution Expert labeled the cerebellum from each dataset with 12 divisions of the cerebellar hemispheres
Simulation Voxel-wise random raters Boundary random raters Evaluation Jaccard Index Dice coefficient

Results
Simulated Label Set Constructed by 10 or 25 Monte Carlo iterations

Result
Voxel-wise random simulated labels

Result
Boundary-random simulated labels

Result
Empirical simulation results 38 undergraduate raters Raters labeled between 10 and 100 slices for the axial set

Discussion
STAPLER extends the applicability of the STAPLE

technique to common research situations with missing, partial, and repeated data. STAPLER facilitates use of training data and reliability priors to improve accuracy. STAPLER enables parallel manual labeling and reduces detrimental impacts STAPLER can readily be augmented by introducing spatially adaptive, unconditional label probabilities.

EM algorithm
Log likelihood function defined as = log (|) log is strictly concave and also strictly increasing because 1 = 2 < 0
Definition of convex function
+ (1 ) + (1 )()

Compute an updated estimate such that,

= ln ln (| )

Introduce a hidden variable

=

, (|)

EM algorithm
Rewrite the equation = ln , ln = , ln , , ln ,
, ln , ,

(| )

Thus,
+

+1 = arg max arg max

arg max

, ln ,

, ln , =