Medical Image Registration

Yujun Guo
Dept.of CS
Kent State University

Outline
Why registration
Registration basics
Rigid registration
Non-rigid registration
Applications

Modalities in Medical Image

Computed Tomography (CT), Magnetic Resonance (MR)
imaging, Ultrasound, and X-ray give anatomic information.
Positron Emission Tomography (PET) and Single Photon
Emission CT (SPECT) give functional information.

Registration
Monomodality:

A series of same modality images (CT/CT, MR/MR,

Mammogram pairs,).
Images may be acquired weeks or months apart;
taken from different viewpoints.
Aligning images in order to detect subtle changes in
intensity or shape

Multimodality:

Complementary anatomic and functional information

from multiple modalities can be obtained for the
precise diagnosis and treatment.
Examples:PET and SPECT (low resolution, functional
information) need MR or CT (high resolution,
anatomical information) to get structure
information.

Registration Problem Definition

q = (912,632)
p = (825,856)
Pixel location in first image

q = T(p;a)
Homologous pixel location in
second image

Pixel location mapping function

Example Mapping Function

q = (912,632)
p = (825,856)

Pixel scaling and

translation

Image Registration
Define a transform T that will map
one image onto another image of the
same object such that some image
quality criterion is maximized.
A mapping between two images both
spatially and with respect to intensity
I2 = g (T(I1))

Registration Scheme

Components
Feature Space
Search Space or transformation
Similarity Metric
Search Strategy

Feature Space
Geometric landmarks:
Points
Edges
Contours
Surfaces, etc.
Intensities:
Raw pixel values
Feature-based
Intensity-based

23 35
24 56

Image transformations
Input
image

Transformation
x' m 0 m1 m 2
y ' m 3 m 4 m5

w' m 6 m 7 m8

Mrigid

Original
shape

cos
sin
0

sin
cos
0

tx
ty
1

Rigid transformation

ut
Out p
e
imag

x
y

w

m 0 m1 m 2
Maffine m 3 m 4 m5
0
0
1

Affine transformation

Rigid
Non-rigid

Similarity Metric
Absolute difference
SSD (Sum of Squared Difference)
Correlation Coefficient
Mutual Information / Normalized
Mutual Information

Search Strategy
Powells direction set method
Downhill simplex method
Dynamic programming
Relaxation matching
Hierarchical techniques

Multi-modality Brain image

registration
Intensity-based
3D/3D Rigid transformation, DOF=6 (3
translations, 3 rotations)
Maximization of Normalized Mutual
Information
Simplex Downhill
Multi-resolution
Dataset: Vanderbilt University
http://www.vuse.vanderbilt.edu/~image/registration/results.html

Mutual Information as Similarity

Measure
Mutual information is applied to measure the

statistic dependence between the image intensities of

corresponding voxels in both images, which is assumed to
be maximal if the images are geometrically aligned.

I ( A, B ) PAB ( a, b) log
a

PAB ( a, b)
PA (a ) PB (b)

H ( A) H ( B ) H ( A, B )
H ( A) H ( A | B )
H ( B ) H ( B | A)

Normalized Mutual Information

Extension of Mutual Information
Maes et. al.:

NMI ( A, B ) H ( A, B ) MI ( A, B )
2 MI ( A)
NMI ( A, B )
H ( A) H ( B )

Studholme et. Al.:

NMI ( A, B)

H ( A) H ( B)
H ( A, B)

Compensate for the sensitivity of MI to

changes in image overlap

Geometry Transformation
Image Coordinate transform:
The features (dimension, voxel size, slice spacing, gantry
tilt, orientation) of images, which are acquired from
different modalities, are not the same.
From voxel units (column, row, slice spacing) to
millimeter units with its origin in the center of the image
volume.
T ( x, y , z )

x'

y'
z'

1

a01
a11

a02
a12

a
20
0

a21
0

a22
0

a00

a10

x

y
z

1

Target Image & Template Image

j

Target Image Grid

Template Image Grid

x
Target Image
Physical Coordinates

Space
Transform

x
Template Image
Physical Coordinates

Images from the same patient

MRI-T2

PET
Target Image ?
Template Image ?

128 x 128 pixels

256 x 256 pixels

Images provided as part of the project: Retrospective Image Registration Evaluation,
NIH, Project No. 8R01EB002124-03, Principal Investigator, J. Michael Fitzpatrick, Vanderbilt University, Nashville, TN.

Interpolation
Nearest Neighbor
Tri-linear Interpolation
Partial-Volume Interpolation
Higher order partial-volume
interpolation

Evaluating similarity measure for

each transformation
y

Transform

x
Template
Image

x
Target Image

Optimization
Powells Direction Set method
Downhill Simplex method

Multi-resolution
Why Multi-resolution
Methods for detecting optimality can not guarantee that
a global optimal value will be found.
Time to evaluate the registration criterion is proportional
to the number of voxels.

The result at coarser level is used as the starting

point for the finer level.
Currently multi-resolution approaches:
Sub-sampling
Averaging
Wavelet

Registration Result (I)

A typical superposition
of CT-MR images.
Left : before
registration Right:
after registration.

Rigid transformation (II)

A typical superposition
of MR-PET images.
Left : before
registration Right:
after registration.

Mammography
Breast cancer is the second leading cause
of death among women in USA.
Detected in its early stage, breast cancer
is most treatable.
Mammography is the main tool for
detection and diagnosis of breast
malignances.
It reduces breast cancer mortality by 25%
to 30% for women in the 50 to 70 age
group

Mammogram Registration
Temporal/bilateral mammograms
vary
Breast compression
Breast position
Imaging Technique
Change in Breast

Mammogram registration
techniques
Whole breast area vs. regional
Nipple location
Control-point location
Rigid & non-rigid registration

Non-rigid Mammogram Registration

Intensity-based
Elastic transformation
Multi-resolution
Demons algorithm (Thirion, 1996)

Demons
Transform

Scene (Target)

Model (Template)

Demons (Cont.)
Transform

Scene

Forces

Model

Demons (Cont.)
Intensity

Current
Estimation

Space
Gradient
Desired
Displacement
Scene

Demons
From Optical Flow
Scene: f, Model: g
Assumption: The intensity of a
moving object is constant with time
(1)
(2)

Description of the Approach

1. Select demon points.
2. Compute the force u on the model
at each of the selected demons
3. Determine a global transformation
based on the computed u and apply
it to the model
4. If the model images is now
registered to the scene image, stop.
Else, go to Step 2.

Registration Components
Image Intensities
Non-rigid transformation, one
displacement vector for each pixel
Bilinear interpolation
Absolute difference as similarity
metric
Multi-resolution
Dataset: MIAS,DDSM

Demons Results (I) Synthetic Images

Level=
2

Level=
4

Level=
3
Level=
5

Demons Result (II) MIAS

Original
images

Before

After rigid registration

After non-rigid registration

Ongoing registration topics

Trade-off of computation and
accuracy
Evaluation of registration results
Visualization of registration

Applications: Change Detection

Images taken at different times
Following registration, the differences
between the images may be
indicative of change
Deciding if the change is really there
may be quite difficult

Other Applications
Multi-subject registration to develop
organ variation atlases.
Used as the basis for detecting
abnormal variations

Object recognition - alignment of

object model instance and image of
unknown object (segmentation)

References
Maes F,Collignon A, et al. Multimodality image
registration by maximization of mutual
information. IEEE Trans. Med. Imaging. 1997,
V16,pp187-198
L.G.Brown, A survey of image registration
techniques, ACM Computing Surveys, vol. 24, no.
4, pp. 325376, 1992.
Jean-Philippe Thirion, Non-Rigid Matching Using
Demons, IEEE Conference on Computer Vision
and Pattern Recognition,1996