IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO.
1, JANUARY 2008
Hyperspectral Change Detection in the Presence
of Diurnal and Seasonal Variations
Michael T. Eismann, Senior Member, IEEE, Joseph Meola, Member, IEEE,
and Russell C. Hardie, Senior Member, IEEE
Abstract—Hyperspectral change detection has been shown to
be a promising approach for detecting subtle targets in complex
backgrounds. Reported change-detection methods are typically
based on linear predictors that assume a space-invariant affine
transformation between image pairs. Unfortunately, several physi-
cal mechanisms can lead to a significant space variance in the spec-
tral change associated with background clutter. This may include
shadowing and other illumination variations, as well as seasonal
impacts on the spectral nature of the vegetation. If not properly
addressed, this can lead to poor change-detection performance.
This paper explores the space-varying nature of such changes
through empirical measurements and investigates spectrally seg-
mented linear predictors to accommodate these effects. Several
specific algorithms are developed and applied to change imagery
captured under controlled conditions, and the impacts on clutter
suppression and change detection are quantified and compared.
The results indicate that such techniques can provide markedly
improved performance when the environmental conditions associ-
ated with the image pairs are substantially different.
Index Terms—Change detection, hyperspectral, image analysis,
spectral clustering, target detection.
I. I NTRODUCTION
H YPERSPECTRAL remote sensing is an advancing tech-
nology with a wide range of applications, including the
automated search for specific objects of interest (i.e., targets)
based on differences between their surface material proper-
ties relative to the surrounding background. Examples include
civil and combat search-and-rescue operations, border surveil-
lance, and military target detection. Hyperspectral sensing and
processing technology has matured to the point where this
automated target-detection concept can be performed in real
time on-board airborne platforms [1]. However, the false-alarm
rate can be higher than desired in many applications, especially
when the background clutter is highly structured and does not
conform well to local normal [2], normal mixture [3], linear
mixing [4], and subspace [5] models that are often used to
represent it in the detection algorithms.
Manuscript received April 23, 2007; revised June 20, 2007.
M. T. Eismann and J. Meola are with the Sensors Directorate, Air Force
Research Laboratory, Wright-Patterson AFB, OH 45433-7700 USA (e-mail:
michael.eismann@wpafb.af.mil).
R. C. Hardie is with the Department of Electrical and Computer Engineering
and the Electro-Optics Program, University of Dayton, Dayton, OH 45459-
0226 USA (e-mail: rhardie@udayton.edu).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TGRS.2007.907973
0196-2892/$25.00 © 2007 IEEE
237
In some applications, reference imagery of an area to be
searched exists from prior missions and can be used to detect
subtle changes due to insertion, deletion, or movement of
targets between observations. Since the illumination and envi-
ronmental parameters generally change between observations,
simple subtractive change-detection methods are inadequate.
Several change-detection methods developed for panchromatic
and multispectral sensors employ linear [6] and nonlinear [7]
prediction algorithms that deal with illumination and other
background changes by transforming the reference image to the
current observation conditions such that it can be subtracted off
to suppress the stationary structured clutter. Margalit et al. [8]
applied a local normal model and developed a local linear
predictor based on the joint second-order statistics of two or
more images for this purpose.
There has been a substantial research in detecting land cover
changes in remotely sensed panchromatic and multispectral
imagery. Thorough reviews of this work can be found in [9] and
[10]. Common methods described operate on band ratios, veg-
etation indexes, or other extracted features which make them
less susceptible to variations in illumination across and between
the images. Several techniques exploit spatial and contextual
information in the change-detection process [11], [12] and use
multitemporal segmentation [13] to determine how land cover
regions change in time. Although the focus of this paper is
on the detection of small targets in a cluttered background, as
opposed to changes in the spatial distribution of the background
(i.e., land cover) itself, many of the techniques developed as part
of this prior research are relevant to this problem.
Stocker and Schaum [14], [15] have applied global linear pre-
diction to perform the detection of small targets in hyperspectral
imagery through change detection. Their chronochrome (CC)
method [14] is based on the joint second-order statistics be-
tween the two images, whereas their covariance equalization
(CE) method [15] does not require the estimation of the cross-
covariance matrix, thus reducing the sensitivity to misregistra-
tion between images in estimating the affine transformation.
Change detection is performed in both cases by applying stan-
dard anomaly or matched filter detectors to the normalized
difference or change residual image. Substantial improvements
in change-detection performance relative to single image de-
tection have been demonstrated for the cases shown by virtue
of the reduction of the structured clutter.
The CC and CE methods are related to other reported
change predictors. CC is closely related to the image regression
method described in [9], whereas CE has also been described
as a whitening/dewhitening transform [16]. Recent work has
238 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 1, JANUARY 2008

extended this method to a family of related predictive trans-

forms based on different assumptions about the hyperspectral
image statistics [17] and refined methods for parameter estima-
tion [18].
The underlying assumption of the CC and CE methods for
hyperspectral change detection is that the affine transformation
capturing the nontarget changes (i.e., illumination, environ-
ment, and other background changes) is space invariant. There Fig. 1. Basic change-detection block diagram.
are, clearly, cases where this assumption would not be valid,
but the impact of spatially varying statistics as a function of A. Observation Model
time of day and season has not been systematically evaluated.
The two hyperspectral images can be represented by K × N
Carlotto [19] applied image segmentation or clustering to detect
data matrices X (test) and Y (reference) of the form
man-made changes, and this approach may be an effective
avenue to deal with a space-varying change. The particular X = [ x1 x2 ··· xN ] (1)
implementation reported performs mean-based clustering of a
reference image and then detects changes in a target image Y = [ y1 y2 ··· yN ] (2)
using a class-conditional Mahalanobis distance detector based
on the class map from reference image clustering. We take a where xi and yi are the corresponding K × 1 spectral vectors
different approach to this segmented concept by developing var- for the ith spatial position, K is the number of spectral bands,
ious class-conditional affine transformations as space-varying and N is the number of spatial locations. By assuming a linear
predictors of the structured background clutter. space-invariant observation model [22], the spectral observa-
In this paper, we present the results of a systematic hyper- tions are related to an inherent material reflectance ρi by
spectral measurement campaign that was conducted in order
to develop a better quantitative understanding of the impacts xi = Tx ρi + dx (3)
of space-varying background changes on the performance of yi = Ty ρi + dy (4)
global affine predictors used for change detection. Furthermore,
various segmented predictors are developed and assessed with where the diagonal matrices Tx and Ty include all the multi-
regard to their ability to mitigate such space-varying effects. plicative elements (product of sensor gain, atmospheric trans-
These results expand on preliminary approaches and observa- mission, and downwelling radiance), the offset vectors dx and
tions reported in [20] and [21]. Section II provides a review dy include the additive elements (sensor dark level plus product
of the baseline CC and CE methods and extends them to of gain and path radiance), and the parameters of the affine
a segmented form. Section III describes the data collection transformations are assumed to change between observations.
campaign that was performed to understand the nature of For notational convenience, we will drop the spatial position
space-varying background changes in hyperspectral imagery. index on the vectors.
Section IV reports on an assessment using these data of global It is readily shown that the transformation from the reference
predictor performance in light of space-varying change, as well domain to the test domain is the affine transformation
as the efficacy of segmented approaches in mitigating these
effects. Finally, the conclusion and suggestions for future work x = Txy y + dxy (5)
are provided in Section V.
where
II. C HANGE -D ETECTION P ROCESSING Txy = Tx T−1 (6)
y
Consider the observation of two hyperspectral images: 1) a dxy = dx − Txy dy . (7)
test image, possibly including targets of interest that are the
object of detection; and 2) a reference image, captured at some The objective of change-detection processing is to make
other time when the targets are not present. Thus, the reference accurate estimates of the transformation parameters (T̂xy and
image provides a representation of the background clutter in d̂xy ) such that the test image can be predicted from the refer-
which the targets are deployed. By properly normalizing the ence image according to
reference image to the domain of the test image, the background
clutter might be suppressed through differencing such that the x̂ = T̂xy y + d̂xy . (8)
targets can be detected more readily, as shown in Fig. 1. We
define background to include all natural or man-made objects Then, a change residual image
present at the same locations in both the reference and test
images, while target only refers to an object present at a δ = x − x̂ = x − (T̂xy y + d̂xy ) (9)
particular location in one image but not the other. That is, a
target is a change in the form of the insertion, deletion, or can be formed, for which the stationary background clutter
movement of a physical object, typically man-made, between should be attenuated, theoretically leaving only target changes
observations. and sensor noise.
EISMANN et al.: HYPERSPECTRAL CHANGE DETECTION 239

B. Global Linear Prediction where

The CE method [15] for change detection works by estimat- 1 1
p(x|q) =
ing the transformation parameters in (8) based on the second- (2π)K/2 |Cx|q |1/2
order statistics of each image: the mean vectors mx and my and  
1 T −1
the covariance matrices Cx and Cy . If the covariance matrices × exp − [x − mx|q ] Cx|q [x − mx|q ] . (18)
are diagonalized [23] in the form 2

A normal mixture model is fully represented by the set of

Cx = Vx Dx VxT (10)
parameters {P (q), mx|q , Cx|q , q = 1, 2, . . . , Q}. When these
Cy = Vy Dy VyT (11) parameters are known or estimates of these parameters exist,
each image spectrum x can be assigned into a class using a
the estimated transformation parameters become distance metric from the respective classes. Two alternatives
considered here include the Euclidean distance
−1/2
T̂(CE)
xy = C1/2
x Cy
−1/2 T
d(q) = xi − mx|q  (19)
= Vx D1/2 T
x Vx Vy Dy Vy (12)
and the Mahalanobis distance
d̂(CE)
xy = mx − T̂(CE)
xy my . (13)
d(q) = (xi − mx|q )T C−1
x|q (xi − mx|q ). (20)
In (10)–(12), Vx and Vy are the eigenvector matrices, and Dx
and Dy are the diagonalized covariance matrices corresponding The assignment process is classification, with these two al-
to the test and reference images, respectively. ternatives referred to as linear (or mean-based) and quadratic
Rather than just using the second-order statistics of the two classification, respectively.
images, CC processing [14] makes use of the cross-covariance Normal mixture modeling includes not only the classification
Cxy to estimate the transformation matrix of the image spectra but also the concurrent estimation of the
model parameters. This process is typically called clustering,
T̂(CC)
xy = Cxy C−1
y . (14) and a variety of methods are described in the literature. In this
paper, two iterative clustering methods are employed: a mean-
The offset estimate has the same form as (13). One fundamental based approach commonly known as Linde–Buzo–Gray (LBG)
difference between the methods is that CC requires spatially clustering or vector quantization [25] and a quadratic approach
registered imagery in order to estimate the cross-covariance based on stochastic expectation maximization (SEM) [26], [27].
matrix, whereas CE does not. Both methods transform the The latter approach is much more computationally complex
second-order image statistics according to since it must estimate and invert class-conditional covariance
matrices during every iteration. We restrict the minimum size
mx̂ = T̂xy my + d̂xy (15) of the classes to be sufficiently large so as to avoid excessive
target contamination and avoid any rank deficiency or matrix
Cx̂ = T̂xy Cy T̂T
xy . (16)
singularities in computing the transformations.

C. Normal Mixture Model D. Segmented Linear Prediction

Space-varying changes in illumination or other environmen-
tal influences will cause the transformation parameters compos-
ing the observation models shown in (3) and (4) to depend, in
some unknown manner, on spatial position. One way to deal
with such space-varying changes is to represent the images with
a normal mixture model [24] and allow for the transformation
parameters to differ between spectral clusters. The normal
mixture model describes each spectrum x in an image as a
member of one of Q classes defined by a class index q where
q = 1, 2, . . . , Q. Each spectrum has a prior probability P (q)
belonging to each respective class, and the class-conditional
probability density function p(x|q) is normally distributed and
completely described by a mean vector mx|q and covariance
matrix Cx|q . The probability density function p(x) is then
given by


Q Since the segmented predictor allows the flexibility to trans-
p(x) = P (q)p(x|q) (17)
q=1
Given a normal mixture model, the data are classified into
disjoint subsets Ωq according to the methods discussed in the
previous section, where the index q represents the class index.
The specific mapping of image position to the class index will
be referred to in this paper as the class map. By using the
same class map for both the reference and test images, class-
conditional transformation parameters T̂xy |q and d̂xy |q can be
estimated individually for the scene spectra corresponding to
each respective class, and a segmented linear predictor can be
formed using the transformation parameters corresponding to
the class to which a given pixel is assigned, or
x̂|q = (T̂xy |q)y + (d̂xy |q) (21)
for q : y ∈ Ωq . These steps are performed within the predictor
block in Fig. 1 and are conceptually shown in Fig. 2.
form image regions differently, it provides additional degrees
of freedom to deal with space-varying changes. As a simple
240 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 1, JANUARY 2008
Fig. 2. Conceptual depiction of segmented change detection.
example, an image region that is in shadow during the reference
observation and fully illuminated in the test observation would
be expected to compose a different cluster than a fully illumi-
nated region in both and, thus, be transformed with a different
predictor.
One challenge of using the segmented linear predictor is
the determination of the classification map. The approach em-
ployed by Carlotto [19] clusters the reference image. It then
uses the resulting class map to compute the class-conditional
mean and covariance matrices for the test image and detects
targets using a class-conditional Mahalanobis distance metric
based on these parameters. Classification in this manner is
based on similarity in material types and illumination during
the reference observation and not necessarily similarity in
the transformation parameters between observations, which is
really being sought.
To explore this segmentation issue further, we employ sev-
eral different segmentation variations to quantitatively assess
their efficacy for mitigating space-varying background changes.
First, both LBG and SEM clustering are used to evaluate
whether the extra computational expense of quadratic clustering
provides an overall benefit. Next, class-conditional transfor-
mation parameters are derived from both reference and test
image clustering in order to assess whether either approach
provides an advantage. Finally, two additional variations are
pursued in an attempt to better capture cross-scene statistical
behavior.
We refer to one such variation as minimum-squared-error
(mse) class map optimization. With this method, a classifica-
tion map is derived using the baseline approach of LBG or
SEM clustering based on the test image, and the correspond-
ing class-conditional transformation parameters are estimated
based on the class statistics for each cluster. Next, a set of class-
conditional predictors is computed for each pixel spectrum in
the image of the form
x̂|q = (T̂xy |q)y + (d̂xy |q) (22)
for q = 1, 2, . . . , Q. Then, the predictor is selected, which
corresponds to the mse of the difference vector
q = arg min x − (x̂|k)2 (23)
k
for k ∈ {1, 2, . . . , Q}, and the classification map is updated
accordingly. Finally, the sequence of steps involving class-
conditional parameter and predictor estimation, squared error
minimization, and classification map updating is repeated until
it converges. As will be shown later, convergence generally
occurs with only a few iterations. The methods are later referred
to as optimized LBG and optimized SEM.
Another variation attempts to perform a joint clustering
of both images and to derive the class-conditional transfor-
mation parameters based on the joint classification map. In
this variation, a 2K × N joint data matrix is formed by sim-
ply concatenating the corresponding spectral vectors from the
reference and test images on a pixel-by-pixel basis, and the
resulting concatenated image is clustered by either the LBG or
SEM method. The resulting classification map is referred to as
the joint classification map.
E. Change Detection
If we define b as a random vector process representing the
structured background clutter in the test image, n as a random
vector process representing sensor noise, and s as a determin-
istic target signature, then the change-detection problem can be
represented by
H0 : x = b + n
H1 : x = s + n. (24)
Expressing (24) in terms of the change residual yields the
following equivalent hypotheses:
H0 : δ = (b − x̂) + n
H1 : δ = (s − x̂) + n. (25)
The quantity (b − x̂) represents the suppressed background
clutter, (s − b) is the true target change signature, and x̂ is
the background clutter prediction from (8) or (21) based on the
reference image. The change-detection problem reduces to a
traditional problem of detecting either a known or unknown
additive target in a normally distributed noise if we assume
perfect clutter suppression (x̂ = b) and normal statistics for
sensor noise. Therefore, a meaningful metric for the quality
of the predictors is the relative degree to which the clutter is
suppressed in the change residual δ compared with the clutter
in the test image x (i.e., no change analysis). This can be
quantified on a band-by-band basis by the metric
 
Cx (k, k)
Sk = 10 log (26)
Cδ (k, k)
where Cx (k, k) and Cδ (k, k) are the kth diagonal elements of
the covariance matrices for x and δ. Total clutter suppression is
quantified by
 
trace(Dx )
Stotal = 10 log (27)
trace(Dδ )
where Dx and Dδ are the diagonalized covariance matrices for
x and δ.
EISMANN et al.: HYPERSPECTRAL CHANGE DETECTION 241

If the target signature is unknown, the optimal decision

statistic for the change-detection problem under the given as-
sumptions is the Mahalanobis distance

r = (δ − mn )T C−1
n (δ − mn ). (28)

In practice, the mean vector mn and covariance matrix Cn

are estimated from the sample statistics of the change residual
image either globally or over a local region around the test spec-
trum. We refer to the former case as global anomaly detection
(AD) and the latter as spatially adaptive AD, commonly known
as the Reed–Xiaoli (RX) algorithm [2]. If a specific change
with a known target spectrum is sought, then a matched filter
or other signature-based approach theoretically provides an
improved performance, assuming that the signature is properly
normalized to the measurement domain [28], [29]. This paper
is limited to an examination of anomaly change detection and
will leave these issues to future work.

III. D ATA C OLLECTION

In order to examine a space-varying change in hyperspectral
imagery, we assembled an imaging spectrometer instrument to
capture spatially registered data sets at a fixed viewing geome-
try, with controlled changes in scene content, solar illumination
angles, weather conditions, time of day, and seasonal effects.
The collection instrument employed was a Hyperspec VS-25
Imaging Spectrograph produced by Headwall Photonics. As
implemented for this test, the instrument produced data over
the 400–900-nm spectral range with nominally 8-nm spectral
resolution at 3.6-nm spectral sampling. It was scanned in
the azimuthal direction with a vertically oriented slit using a
precision motor-controlled mount. The instrument was located
in a fifth-floor laboratory and viewed a scene roughly 150 m
away, with about 4-cm spatial sampling and 70-ms frame
integration time. Typical images were 800 × 1024 spatial pixels
in size. The spatial-spectral distortion was compensated by a
thorough laboratory alignment and characterization to less than
0.1%, and the spectrometer was calibrated to absolute spectral
radiance using an integrating sphere. The fixed mounting and
scan motor precision ensure precise spatial registration between
all images collected. Details on the instrument characterization
and the conduct of the tests can be found in [30].
Data collection was conducted from late August 2005 to
mid-May 2006 in order to capture a large seasonal variation.
Ideally, one would like to separately test illumination and sea-
sonal vegetation changes, but this was not completely possible
because of the change in solar illumination angles with season.
The impact of illumination changes was isolated by collecting
numerous data sets with different illumination conditions over
a short period of time in which no significant vegetative alter-
ations occur within the scene. We attempted to isolate seasonal
changes over a nine-month duration by capturing data at times
of day for which the solar azimuth (SA) angles were similar,
thus attempting to keep the directionality of shadows in the
scene the same. However, the length of the shadows varies
considerably due to the uncontrollable change in solar zenith
angles.
Fig. 3. Visible images of change pair for clutter suppression analysis.
(a) Reference (August 25, 2005). (b) Test (November 2, 2005).
Several of the results in this paper are based on the represen-
tative image pairs depicted by the photographs in Figs. 3 and 4.
The scene is composed of a set of four 5 ft × 5 ft panels
tilted roughly orthogonal to the sensor line-of-sight direction
and placed along a treeline at the edge of an open grass field.
The panels are intended to represent man-made background
clutter, with surfaces of flame-sprayed aluminum, black paint,
tan paint, and green paint. Because of the substantial change
in observation time between the reference and test image
(August 25, 2005, to November 2, 2005), the scene shown
in Fig. 2 exhibits significant space-varying changes due to
illumination, shadowing, senescence of the tree leaves, and
other seasonal changes. This image pair does not include any
target changes, but serves as a test set for clutter suppression
analysis. The change pair shown in Fig. 3 includes two green
tarp bundles that are placed in the test image (circled in the
figure) and absent in the reference image. These are target
changes, which correspond to a total of 105 image pixels, that
will be the object of change-detection analysis.
IV. R ESULTS
The presentation of the results is organized in three parts.
First, the impacts of illumination and seasonal changes across
242 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 1, JANUARY 2008
Fig. 4. Visible images of change pair for change-detection analysis.
(a) Reference (November 2, 2005). (b) Test (October 14, 2005).
the data collection are quantitatively assessed with regard
to the ability to suppress background clutter in the change
residual image using global affine predictors. Next, the clutter
suppression capabilities of the various segmented predictors
are compared to those of the global predictors. Finally, the
various methods are compared with regard to change-detection
performance.
Prior to processing, all data are transformed into the leading
10-D principal component subspace [31] of the reference image
using the transformation matrix Vx from (10). For these data,
essentially, all of the structured background clutter power is
contained in this leading subspace, and this transformation
reduces the dimensionality of the data to extract statistically vi-
able covariance matrix estimates. The classification, prediction,
and detection steps shown in Fig. 2 are all performed in the
10-D subspace. Fig. 5 shows a scatter plot of the first two
principal components based on a coarse manual segmentation
corresponding to the trees (dark gray), grass (black), and the
shadow area and panel region (light gray) for the change pair
shown in Fig. 3. It is fairly clear that a single affine transfor-
mation will not make these data coincide, even in these two
dimensions. Note that the panel positions or orientations do
not change between observations; therefore, they are consid-
ered stationary man-made clutter in this test, as opposed to
Fig. 5. Scatter plots of leading principal components of change pair for clutter
suppression analysis. (a) Reference (August 25, 2005). (b) Test (November 2,
2005).
targets. There are no target changes between these particular
observations.
A. Clutter Suppression Using Global Linear Prediction
The impacts of illumination changes on clutter suppression
are quantified by the example shown in Fig. 6 for a data set
captured from May 8–22, 2005, with images at hourly intervals
from 8:00 A . M . to 7:00 P. M . Fig. 6(a) shows the measured
solar elevation (SE, degrees up from horizon) and SA (degrees
clockwise from north) angles for the times at which the images
were captured. The image mean (integrated over the spectral
bands) is shown in Fig. 6(b) to illustrate how the illumination
characteristics vary across the image set. The illumination
peaks at about 3:00 P. M ., whereas the minimum occurs at
8:00 A . M ., with a mean illumination of about half that of the
peak. This is roughly consistent with the solar zenith angle, but
is also influenced by the SA in terms of the reduction of the
shadows in the late afternoon.
Clutter suppression using the global predictors is assessed
as a function of time of day using a similar image in May to
the 11:00 A . M . image as a reference. The results, shown in
EISMANN et al.: HYPERSPECTRAL CHANGE DETECTION
Fig. 6. Clutter suppression analysis versus diurnal change. (a) Solar illumina-
tion angles versus time of day. (b) Image mean versus time of day. (c) Clutter
suppression versus time of day for 11:00 A . M . reference.
Fig. 6(c), indicated that CC and CE perform relatively well
at background suppression when the solar position is most
similar to that of the reference image. Suppression continues
243
to decrease as the solar position varies from hour to hour
due apparently to nonuniform illumination conditions. More
prominent shadows manifest themselves early and late in the
day, adding to the complexity of linear prediction and reduced
clutter suppression. CE performs nearly as well as CC, which
is the optimal global linear predictor in a least-squared error
sense.
Over the course of the seasonal data set, the SE angle varies
greatly, as shown in Fig. 7(a). The maximum SE for the test
image is near 60◦ , whereas the minimum in December is below
30◦ . Likewise, tree leaves senescence and grass conditions alter
significantly. The same experimental approach as the illumina-
tion change case is used to examine the seasonal variation in
clutter suppression. Fig. 7(b) shows the change in image mean
and the normalized differential vegetation index (NDVI) over
this image set. The NDVI [30] is shown as an indicator of the
relative health of the vegetation, which impacts on the image
mean through the change in the mean reflectance of the scene,
and was computed on the original full-dimensionality data. In
this case, the mean and NDVI start out low in the early fall
due to the dry vegetation conditions, increase in the fall as the
vegetative health improves, decline in the winter as the trees
become bare and illumination decreases, and rise again in the
spring as the tree leaves reappear. The overall decrease in the
image mean during the winter is also caused by a reduced solar
illumination, whereas the more erratic behavior in the image
mean and NDVI may be correlated with the small differences
in SA angle. That is, it may be due to differences in shadowing
between collects.
By using the August 25 image as the reference, the global
predictor works well for clutter suppression early on and gener-
ally depreciates as the seasons change, reaching a minimum in
late December and early January. After this point, suppression
generally improves again as the tree leaves return, and the
scene is apparently more strongly correlated with the early
fall reference. The suppression results for the whole scene
are also attributed to the seasonal illumination conditions to a
certain extent. Lower solar angles result in less scene radiance
and lower scene variance in some cases. The low elevation
also results in drastic shadows present within the scene, which
are difficult to predict and suppress. As the SE increases, the
suppression improves. It is also possible that the senescence
and loss of tree leaves in the late fall and winter months may
have an appreciable effect on these results, but it is difficult
to separate this impact from the illumination effects in the
data. For the seasonal change data, it is also apparent that
CC produces markedly better results than CE. Further insights
into the specific behavior of various scene components can be
found in [20] and [30].
B. Clutter Suppression Using Segmented Linear Prediction
The comparison of segmented and global linear predictors
with respect to clutter suppression is based on the image pair
shown in Fig. 3, which includes no target changes. The first
result is based on the use of SEM clustering on the clutter
suppression change pair. For this analysis, SEM clustering
was first performed on the test image using six classes and
244
Fig. 7. Clutter suppression analysis versus seasonal change. (a) Solar illumi-
nation angles versus day of year. (b) Image mean and NDVI versus day of year.
(c) Clutter suppression versus day of year for August 25 reference.
20 iterations. The segmented predictor was estimated using
this baseline clustering approach, as well as employing three
optimization iterations, as described in Section II.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 1, JANUARY 2008
Fig. 8. Clutter suppression results for global CC, SEM segmented (SEM), and
optimized SEM segmented (SEM Opt) predictors.
Fig. 8 quantifies the resulting clutter suppression as a func-
tion of principal component for the baseline and optimized
SEM segmented predictors as compared to the global CC
predictor. The basic CC approach using the cross-covariance
matrix to form the transformation parameters was employed
for both global and segmented predictors as it appears to
be the superior method for the spatially registered data from
this collection campaign. The segmented predictors provide a
considerable improvement in clutter suppression relative to the
global method, especially for the leading principal components
for which the background clutter is most structured. The op-
timization also provides a noticeable improvement relative to
the baseline segmented approach. In this case, the optimization
converges in about two iterations.
Fig. 9 shows the nature of the clutter suppression in the form
of scatter plots of the leading two principal components of the
change residual image for the three predictors. Reducing clutter
variance is hoped to increase target detectability. The improve-
ment in clutter suppression between the global and baseline
segmented methods appears to be driven significantly by the
accommodation of the mean differences between the various
SEM classes, whereas the optimization appears to reduce the
variance of each class as well.
The analysis described earlier was repeated using LBG clus-
tering, and the resulting clutter suppression is compared to the
SEM and global CC results in Fig. 10. Three optimization
iterations were performed for both the SEM and LBG cases,
with the result showing that LBG slightly outperforms SEM
from the perspective of clutter suppression. In the LBG case,
the optimization effectively converges in a single iteration. Con-
trary to the SEM results, optimization has only a slight effect
on the LBG results, presumably because compensation of the
mean differences between classes in the segmented predictor
has such a significant impact on clutter suppression.
The ability of the segmented predictors to more effectively
cancel structured clutter can be depicted by looking at the
residual image bands. Fig. 11 compares the first principal com-
ponent of the change residual image for the global, optimized
EISMANN et al.: HYPERSPECTRAL CHANGE DETECTION
Fig. 9. Scatter plots of leading principal components of change residual image
for different predictors. (a) Global CC. (b) SEM segmented. (c) Optimized SEM
segmented.
SEM, and optimized LBG predictors. Note that the result of
the global predictor still maintains much of the background
structure in the two observations. This structure is greatly sup-
pressed in the change residual images for segmented predictors,
245
Fig. 10. Comparison of clutter suppression using global CC and optimized
SEM (SEM Opt) and LBG (LBG Opt) segmented predictors.
although some visible structure remains in the region of the
panels, especially in the LBG result. The images are auto-
scaled and do not capture the large absolute difference in clutter
variance.
C. Change Detection Using Segmented Linear Prediction
Because the change processing will have an impact on the
target change signal as well as the background clutter, it is
important to extend the analysis beyond clutter suppression to
change target detectability. This was performed by using the
change pair shown in Fig. 3, for which an actual target change
existed, and then comparing the receiver operating characteris-
tic (ROC) performance for a detector operating on the change
residual image. An extensive analysis was performed for all
predictor and detector variations described in Section II, and
a specific subset of the results is summarized here to portray
the primary observations. To aid the reader in keeping track of
the variations, Table I outlines the combinations investigated
in terms of predictor type, clustering source, and detector type.
Most of the results are based on a simple global AD, whereas
some spatially adaptive detector (RX) results are also pro-
vided. Note that target detection, like change prediction, is only
performed in the leading 10-D principal component subspace
of the original hyperspectral data.
The first result, shown in Fig. 12, gives the ROC perfor-
mance for a global AD applied to the test image (AD), the
change residual image for the global predictor (CC/AD), and
the change residual image for the SEM and LBG segmented
predictors (SEM/AD and LBG/AD). ROC performance was
measured on a per pixel basis, with a truth mask for the target
change pixels, and compares the fraction of detected pixels
within the truth mask (probability of detection) to fraction of
detected pixels outside the truth mask (probability of false
alarm) for the full range of detection thresholds. Two cases are
shown: one based on initial clustering on the reference image
(no targets present) and the other based on initial clustering
on the test image (with targets present). In both cases, the
246 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 1, JANUARY 2008
Fig. 11. First principal component of change residual images for different
predictors. (a) Global CC. (b) Optimized SEM segmented. (c) Optimized LBG
segmented.
SEM/AD and LBG/AD exhibit improved performance over
both AD and CC/AD. This performance advantage was much
greater when the initial clustering based on the reference image
was used. This could possibly be due to target contamination
problems when using the test image for initial clustering,
but the reason for the performance difference requires further
exploration.
TABLE I
PREDICTOR AND DETECTOR COMBINATIONS INVESTIGATED WITH
RESPECT TO CHANGE-DETECTION PERFORMANCE
Fig. 12. Detection performance results using single image AD and global
change detector based on CC predictor (CC/AD), SEM segmented predictor
(SEM/AD), and LBG segmented predictor (LBG/AD). (a) Initial clustering on
reference image. (b) Initial clustering on test image.
Fig. 13 is a typical result showing the effect of optimization
and the number of classes on the change-detection performance
based on SEM clustering. Contrary to the clutter suppres-
sion results, it shows that optimization results in a reduced
detection performance, apparently because it suppresses the
EISMANN et al.: HYPERSPECTRAL CHANGE DETECTION
Fig. 13. Effect of the parameter setting on change detection using SEM-based
segmented predictor. (a) Optimization iterations. (b) Number of classes.
target-to-background contrast as well. It also indicates, for this
data set, that there appears to be little benefit to using more
than four classes in the segmentation. Results based on LBG
clustering were generally consistent with these observations.
The different variations of the segmented SEM predictor
are compared in Fig. 14. The predictor based on the refer-
ence classification map provides a better detection performance
than that based on the test classification map. It is unclear
whether this is due to contamination of the clustering due
to the presence of the targets or whether this result is spe-
cific to the nature of this image pair. The joint clustering
approach provides almost a commensurate performance as the
reference image clustering in the region of high detection
probability and produces significantly fewer false alarms at
lower detection probability. This suggests that using the joint
method might be the preferred way of capturing the joint image
statistics in the segmented predictor. This may be important
in ensuring a robust predictor over various scene statistics,
247
Fig. 14. Change-detection comparison of SEM segmented predictor variants
based on the clustering source.
which is an issue that requires further study with a broader
test set.
The final results, shown in Fig. 15, compare the change-
detection performance of using an RX detector with a single
pixel target window, 45 × 45 pixel background window, and
a 15 × 15 pixel guard band. Fig. 15(a) compares the single
frame detection performance using AD and RX relative to
the change-detection performance using CC/RX and SEM/RX
(clustered on reference) predictors. It shows that the single
frame RX detector achieves, for this case, a commensurate
detection performance to the change detectors. The RX single
frame and change detectors also provide a better performance
than any of the global change detectors previously shown. This
result might be due to the specific deployment of the two targets
in an open region of the image, where one would expect such
a locally adaptive method to excel. Therefore, this should not
be taken as a general observation on the relative merits of these
methods.
Fig. 15(b) compares the relative performance of CC/AD,
CC/RX, and SEM/RX for the case where the reference and test
images were switched such that the change-detection problem
became one of detecting the removal of a target. In this case,
single frame detection is not possible since the targets are not
present in the new test image. Again, the change-detection per-
formance using the RX detector is similar using the segmented
and global predictors, with the latter approach being a bit better.
This suggests that the combination of segmented predictors and
spatially adaptive change detectors requires further study with
a broader data set since the results do not correlate well with
those shown previously for the global detector. Specific analy-
sis for target changes deployed amid the structured clutter is
needed.
V. C ONCLUSION
The research summarized in this paper provides several
insights into the nature of diurnal and seasonal change in
hyperspectral imagery. It also compares the capabilities of
248 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 1, JANUARY 2008
Fig. 15. Change-detection performance comparison using single-frame AD,
single-frame adaptive detector (RX), CC predictor with RX change detector
(CC/RX), and SEM segmented predictor with RX change detector (SEM/RX).
(a) Clustering on reference image and detection on test image. (b) Clustering
on test image and detection on reference image.
various predictive methods to suppress stationary background
clutter and change detectors to find subtle target changes in the
presence of such space-varying background change. Shadowing
changes due to solar angle variations are probably the most
prominent source of space-varying change in the collected
data, but changes in vegetative state over long time periods
also seriously limit global linear predictors commonly used in
change-detection algorithms. Segmented predictors, especially
those based on quadratic clustering of the joint data, demon-
strated a substantial improvement in both clutter suppression
and change detection when coupled with a global AD. When
investigating specific variations of these methods, observations
drawn from their impact on clutter suppression did not always
correlate with those concerning change detection. The use of
a spatially adaptive detector along with background change
predictors greatly increased change-detection performance for
both target insertions and removals, but the relative advantage
of segmented prediction was not as significant in this case. We
suspect that this is due to the particular characteristics of the
data and recommend a broader study with a greater diversity
in target deployment characteristics to more fully investigate
the optimal combination and relative merits of segmented pre-
diction and spatially adaptive detection for this problem. Such a
study is also needed to establish the robustness of the promising
methods explored in this paper.
There are several other areas where future work is warranted
to address limitations of the change-detection methods. The
ability of the segmented predictor to effectively deal with mixed
pixels on segment edges, especially partially shadowed pixels,
and more complex urbanized background environments re-
quires further quantitative assessment. Also, further investiga-
tion in the utility of preprocessing for illumination suppression,
the impact of target contamination in the reference imagery, and
the extension to multitemporal hyperspectral image sequences
would be of value. It would also be interesting to combine the
statistical approaches presented here with physically motivated
methods that make use of radiative transfer codes such as
MODTRAN for normalizing the illumination across the image.
For example, the relative spectral illumination between fully
illuminated and shadowed pixels could be estimated and nor-
malized by the model, and any residual errors might be compen-
sated by the statistical segmented predictors. Finally, the effects
of misregistration that would typically exist in remotely sensed
hyperspectral imagery were intentionally circumvented in this
paper by the experiment design in order to isolate impacts
due to diurnal and seasonal changes. The robustness of the
developed methods in the presence of such misregistration,
including real-time implementation, requires further study.
R EFERENCES
[1] B. Stevenson et al., “Design and performance of the civil air patrol
ARCHER hyperspectral processing system,” Proc. SPIE, vol. 5806,
pp. 731–742, Mar. 2005.
[2] I. S. Reed and X. Yu, “Adaptive multi-band CFAR detection of an optical
pattern with unknown spectral distribution,” IEEE Trans. Acoust., Speech,
Signal Process., vol. 38, no. 10, pp. 1760–1770, Oct. 1990.
[3] D. W. J. Stein et al., “Anomaly detection from hyperspectral imagery,”
IEEE Signal Process. Mag., vol. 19, no. 1, pp. 58–69, Jan. 2002.
[4] N. Keshava and J. F. Mustard, “Spectral unmixing,” IEEE Signal Process.
Mag., vol. 19, no. 1, pp. 44–57, Jan. 2002.
[5] J. Harsanyi and C. I Chang, “Hyperspectral image classification and
dimensionality reduction: An orthogonal subspace projection approach,”
IEEE Trans. Geosci. Remote Sens., vol. 32, no. 4, pp. 779–785, Jul. 1994.
[6] B. G. Lee, V. T. Tom, and M. J. Carlotto, “A signal-symbol approach
to change detection,” in Proc. Nat. Conf. Artif. Intell., Aug. 1986,
pp. 1138–1143.
[7] M. J. Carlotto, “Nonlinear background estimation and change detec-
tion for wide area search,” Opt. Eng., vol. 39, no. 5, pp. 1223–1229,
May 2000.
[8] A. Margalit, I. S. Reed, and R. M. Gagliadri, “Adaptive optical target
detection using correlated images,” IEEE Trans. Aerosp. Electron. Syst.,
vol. AES-21, no. 3, pp. 394–405, May 1985.
[9] A. Singh, “Digital change detection techniques using remotely-sensed
data,” Int. J. Remote Sens., vol. 10, no. 6, pp. 989–1003, 1989.
[10] R. J. Radke et al., “Image change detection algorithms: A system-
atic survey,” IEEE Trans. Image Process., vol. 14, no. 3, pp. 294–307,
Mar. 2005.
[11] J. R. G. Townshend and C. O. Justice, “Spatial variability of images and
the monitoring of changes in the normalized difference vegetation index,”
Int. J. Remote Sens., vol. 16, no. 12, pp. 2187–2195, 1995.
EISMANN et al.: HYPERSPECTRAL CHANGE DETECTION
[12] R. Cossu, S. Chaudhuri, and L. Bruzzone, “A context-sensitive Bayesian
technique for the partially supervised classification of multitemporal
images,” IEEE Geosci. Remote Sens. Lett., vol. 2, no. 3, pp. 352–356,
Jul. 2005.
[13] L. Bruzzone and S. B. Serpico, “An iterative technique for the detection
of land-cover transitions in multitemporal remote-sensing images,” IEEE
Trans. Geosci. Remote Sens., vol. 35, no. 4, pp. 858–867, Jul. 1997.
[14] A. Schaum and A. Stocker, “Long-interval chronochrome target detec-
tion,” presented at Int. Symp. Spectral Sens. Res., 1997.
[15] A. Schaum and A. Stocker, “Hyperspectral change detection and super-
vised matched filtering based on covariance equalization,” Proc. SPIE,
vol. 5425, pp. 77–90, 2004.
[16] R. Mayer, F. Bucholtz, and D. Scribner, “Object detection by using
“whitening/dewhitening” to transform target signatures in multitemporal
hyperspectral and multispectral imagery,” IEEE Trans. Geosci. Remote
Sens., vol. 41, no. 5, pp. 1136–1142, May 2003.
[17] R. Mayer et al., “A family of spectral target signature transforms: Rela-
tionship to the past, new transforms, and sensitivity tests,” IEEE Geosci.
Remote Sens. Lett., vol. 1, no. 1, pp. 26–30, Jan. 2004.
[18] R. Mayer et al., “A metric of background candidate assessment for spec-
tral target signature transforms,” IEEE Geosci. Remote Sens. Lett., vol. 2,
no. 2, pp. 113–117, Apr. 2005.
[19] M. J. Carlotto, “A cluster-based approach for detecting man-made objects
and changes in imagery,” IEEE Trans. Geosci. Remote Sens., vol. 43,
no. 2, pp. 374–387, Feb. 2005.
[20] J. Meola, M. T. Eismann, K. J. Barnard, and R. C. Hardie, “Analysis
of hyperspectral change detection as affected by vegetation and illu-
mination variations,” Proc. SPIE, vol. 6565, pp. 65651S-1–65651S-12,
Apr. 2007.
[21] M. T. Eismann and J. Meola, “Use of spectral clustering to enhance clutter
suppression for hyperspectral change detection,” Proc. SPIE, vol. 6565,
pp. 65651T-1–65651T-12, Apr. 2007.
[22] J. R. Schott, Remote Sensing: The Image Chain Approach. Oxford, U.K.:
Oxford Univ. Press, 1997.
[23] K. Hoffman and R. Kunze, Linear Algebra, 2nd ed. Upper Saddle River,
NJ: Prentice-Hall, 1971.
[24] M. T. Eismann and D. W. J. Stein, “Stochastic mixture modeling,” in
Hyperspectral Data Exploitation: Theory and Applications, C. Chang, Ed.
New York: Wiley, 2007.
[25] Y. Linde et al., “An algorithm for vector quantization,” IEEE Trans.
Commun., vol. COM-28, no. 1, pp. 84–95, Jan. 1980.
[26] P. Masson and W. Pieczynski, “SEM algorithm and unsupervised statisti-
cal segmentation of satellite images,” IEEE Trans. Geosci. Remote Sens.,
vol. 31, no. 3, pp. 618–633, May 1993.
[27] T. K. Moon, “The expectation-maximization algorithm,” IEEE Signal
Process. Mag., vol. 31, no. 6, pp. 47–60, Nov. 1996.
[28] D. Manolakis and G. Shaw, “Detection algorithms for hyperspectral imag-
ing applications,” IEEE Signal Process. Mag., vol. 19, no. 1, pp. 29–43,
Jan. 2002.
[29] G. Healey and D. Slater, “Models and methods for automated material
identification in hyperspectral imagery acquired under unknown illumi-
nation and atmospheric conditions,” IEEE Trans. Geosci. Remote Sens.,
vol. 37, no. 6, pp. 2706–2717, Nov. 1999.
[30] J. Meola, “Analysis of hyperspectral change and target detection
as affected by vegetation and illumination variations,” M.S. thesis,
Univ. Dayton, Dayton, OH, 2006.
[31] R. A. Schowengerdt, Remote Sensing: Models and Methods for Image
Processing. San Diego, CA: Academic, 1997.
[32] J. Rouse et al., “Monitoring vegetation systems in the Great Plains with
ERTS,” in Proc. 3rd ERTS Symp., 1973, vol. 1, pp. 309–317.
249
Michael T. Eismann (M’03–SM’07) received the
B.S. degree in physics from Thomas More College,
Crestview Hills, KY, in 1985, the M.S. degree in
electrical engineering from the Georgia Institute of
Technology, Atlanta, in 1987, and the Ph.D. de-
gree in electrooptics from the University of Dayton,
Dayton, OH, in 2004.
He is currently the Electro-Optical Sensor Tech-
nology Division Technical Advisor with the Sensors
Directorate of the Air Force Research Laboratory
(AFRL), Wright-Patterson Air Force Base, OH. He
is responsible for overseeing the development of active and passive electroop-
tical and infrared sensor technology, and transition into operational airborne
targeting and reconnaissance systems. His current research interest includes
development of hyperspectral imaging sensors and image processing. Prior
to joining AFRL in 1996, he was employed by the Environmental Research
Institute of Michigan (ERIM). He joined ERIM in 1991, where he was involved
in research concerning active and passive optical and infrared targeting and
reconnaissance, optical information processing, and holographic optics.
Joseph Meola (M’02) received the B.S. and M.S.
degrees in electrical engineering from the Univer-
sity of Dayton, Dayton, OH, in 2004 and 2006,
respectively.
He is a Research Engineer with the Electro-
Optical Targeting Branch, Sensors Directorate of the
Air Force Research Laboratory (AFRL), Wright-
Patterson Air Force Base, OH. Prior to joining
AFRL, he worked as a Research Assistant with the
University of Dayton while obtaining the M.S. de-
gree in electrical engineering. His current research
involves image processing, target-detection algorithms, and development and
characterization of hyperspectral imaging sensors. In particular, his current
efforts involve change detection work using hyperspectral imagery. His current
research work at AFRL has evolved from his thesis work in the area of seasonal
and diurnal hyperspectral change detection.
Russell C. Hardie (S’92–M’92–SM’99) received
the B.S. degree (magna cum laude) in engineering
science from Loyola College, Baltimore, MD, in
1988 and the M.S. and Ph.D. degrees in electrical en-
gineering from the University of Delaware, Newark,
in 1990 and 1992, respectively.
He is currently a Full Professor with the De-
partment of Electrical and Computer Engineering,
University of Dayton, Dayton, OH, and holds a
joint appointment with the Electro-Optics Program.
He was a Senior Scientist with the Earth Satellite
Corporation, MA, prior to his appointment at the University of Dayton in
1993. His research interests include a wide variety of topics in the area of
digital signal and image processing such as image enhancement and restoration,
pattern recognition, and medical image processing.
Dr. Hardie was a corecipient of the Rudolf Kingslake Medal and Prize
from SPIE in 1998 for his work on multiframe image resolution enhancement
algorithms and several University of Dayton teaching awards. He has served
on the organization committee of the IEEE–EURASIP Workshop on Nonlinear
Signal and Image Processing.