Target Classification Performance as a Function of

Measurement Uncertainty
Seung Ho Doo1, Graeme Smith,1 and Chris Baker1
1
Electrical and Computer Engineering
The Ohio State University
Columbus, OH, USA
{doo.3, smith.8347, baker.1891}@osu.edu

AbstractIn this paper, we demonstrate target classification

using the proposed features in previously reported research
under measurement uncertainty conditions. The MSTAR dataset
is widely used real target measurements in automatic target
recognition society. Extremely high classification results of the
dataset, which are over 90% correct classification, have been
reported from some literatures. However, this high classification
results could be acquired not only by the classification system,
but also the cleanness of the dataset. Therefore, in this paper,
more realistic target classification scenarios including target
aspect angle estimation error, strong white Gaussian noise, and
different combination of test and training targets are applied for
classification and its corresponding results are examined. The
proposed target feature extraction techniques show the
robustness of the measurement uncertainties and excellent
classification results.
Index Terms Target Feature Extraction, Automatic Target
Recognition, SAR

I. INTRODUCTION
In this paper, target classification performance of the
MSTAR dataset is examined as a function of measurement
uncertainty using specially designed features that both cope
with and exploit signature variability. Performance is reported
as a function of measurement uncertainty conditions [1].
The MSTAR dataset is one of the most widely used real
target SAR images in Automatic Target Recognition (ATR)
community [2-4]. Various kinds of target classification
techniques have been developed for a MSTAR target ATR
system, and significantly high classification performance,
which is over 90%, is reached by some publications [2,5]. This
high classification performance could be acquired not only by
the high performance of the target classification systems, but
also the idealness of the MSTAR dataset. In other words, the
Signal to Noise Ratio (SNR) of the MSTAR dataset is high,
over 30 dB, and the target alignment also is nearly perfect.
However, in real target classification situations, more difficult
conditions can occur such as low SNR, target aspect angle
estimation failure, and inconsistent elevation angles between
test and training targets.
In [1], we proposed reliable target feature extraction
techniques which can cope with the variability contained in
target measurements. The proposed features include dominant
scattering area size, dominant scattering area, and scatterer
positional stability information. The proposed features are

c
978-1-4673-7297-8/15/$31.00 2015
IEEE

briefly explained in the next section. The proposed techniques

successfully cope with the variability contained in real target
measurements and shows excellent results over 0.95 correct
classification ratio.
In this paper, we examine the target classification
performance using the proposed feature extraction techniques
under measurement uncertainties: 1) errors in the target aspect
angle estimation, 2) complex Additive White Gaussian Noise
(AWGN), and 3) different test and training target variants. The
proposed techniques show excellent target classification
performance under existence of these high measurement
uncertainties.
II. TARGET FEATURE EXTRACTION
Dominant scattering points in SAR images are widely
adopted as a target feature in ATR research, and the proposed
target feature extraction techniques also exploit the feature
extracted from the dominant scattering area in SAR images
[6,7]. In the proposed techniques, 210 of the strongest
scattering pixels are selected in a SAR image since it results in
the highest target classification performance. To extract the
proposed features, binary images are generated by setting the
selected 210 strong points as 1 and 0 for the others. This binary
image is termed Dominant Scattering Area (DSA) image.
DSA contains strong scattering points which have been
reported as relatively more stable than weak scattering for
target classification. By calculating the distance between the
two farthest pixels in vertical and horizontal axes, length and
width information of the selected pixels in DSA can be
measured and it is termed Dominant Scattering Area Size
(DSAS) in this paper.
We also proposed a feature which exploits angular stability
information from a SAR image. Since the integration angle of
the MSTAR SAR images is 3, two half integration angle
range, 1.5, SAR images can be created from one 3 integration
SAR image. Each half integration angle image contains
scattering information of only the half integration angle, the
comparing of the two half integration angle SAR images can
show the scattering difference between the two angle extents.
The angular stability information is generated as follows.
An Inverse Fast Fourier transform (IFFT) converts a SAR
image to a series of range profiles. Then, de-weighting is
applied to eliminate the windowing effect, in this case, a Taylor
window. The set of range profiles splits into two aspect angle

587

TABLE I. OPERATING PARAMETERS OF MSTAR DATASET

Bandwidth

0.591 GHz

Center Frequency

9.59 GHz

Azimuth Angle Range

Full 360

Down-Range Resolution

0.3047 m

Cross-Range Resolution

0.3047 m

Target Range

Approximately 5 Km

Elevation Angles

15, 17, and more

Angular Spacing

Approximately 0.03

Data type

SAR image

ranges, [
- 1.5,
] and [
,
+ 1.5], where

is the center

aspect angel of a SAR image. Each set of range profiles are
transformed back to a SAR image by taking FFT and Taylor
window. These two SAR images contain the same target but
cover two different aspect angle range, [
- 1.5,
] and [
,

+ 1.5], respectively. These SAR images have same downrange and half cross-range resolution compared to the original
SAR image. Scattering stability information is calculated by
subtracting one SAR image from the other and taking absolute
value. A high value in the new image represents that the point
in a SAR image has high angular variability. In other words,
the point has different scattering intensity in the two aspect
angle ranges. Since the different aspect of target information is
extracted as scatterer angular variability information, we expect
this feature contains different information of a target
measurement. The binary image created by this process is
termed Scatterer Positional Stability Information (SPSI).
III. TARGET CLASSIFICATION
Ten MSTAR target are adopted for classification in this
paper: T62, T72, BTR60, BTR70, BMP2, 2S1, ZSU23,
BRDM2, D7, and ZIL131. The important operating parameters
of the MSTAR dataset are listed in Table I. Among the various
elevation angle data, 15 and 17 data are selected in this paper
since the largest amount of SAR image sets exist in these
angles.
The assumptions for the proposed feature extraction process
are 1) the aspect angles of test targets can be estimated with
some known degree of error, 2) it is not possible to estimate the
direction of the front and the back of a target in a SAR image,
3) the aspect angle information of training targets are known to
some fixed degree, 4) the potential target sizes are similar to
each other, and 5) target are already detected.
In real target classification situations, the aspect angle of a
test target is not known. Hence, it is assumed that the aspect
angles of test targets are estimated by adding a uniformly
distributed random error signals to the true aspect angle. The
estimated test target aspect angle is,







588

(1)

where

 is a true target aspect angle and N is a uniformly

distributed random variable in [-n/2,n/2] range. As we change
the value n, the aspect angel errors can be controlled. All the
SAR images of test targets are rotated by

 so that the
reference directions of all targets are always the same.
Training target SAR images are also rotated by their true
aspect angle,

 since the properties of training data are
known.
The potential target aspect angle ranges in training data can
be narrowed by using the estimated aspect angle of a test target,


 . Since it is already assumed that the front and back
directions of a target cannot be recognized in a SAR image,
two cases should be considered:

 and

 + 180 can be
the front direction of a target. To consider the aspect angle
estimation errors and the stability of dominant scatterers, 20
aspect angle range, which is widely accepted target aspect
angle estimation error, is chosen for narrowing potential target
aspect angel range [8].
IV. CLASSIFICATION RESULTS
To classify the 10 MSTAR targets, all the three proposed
target features, DSA, DSAS, and SPSI, are adopted. To show
the performance of the proposed features, a simple classifier,
K-nearest neighbor is adopted with 5 as K value. This simple
classifier can reduce the effect of the finely tuned parameters
only for specific target data, in this case, the MSTAR dataset.
Three classification scenarios are considered such as 1) target
aspect angle estimation without error, 2) target aspect angle
error with error, 3) target classification with noise added SAR
images.
A. Target Aspect Angle Estimation without Error
10 target classification result using the three proposed
features, DSA, DSAS, and SPSI, are presented in Table I.
Similar classification result is already shown in [1], but we
rewrite the results here for classification performance
comparison with realistic target classification scenarios. All
SAR images of 15 and 17 depression angle data are used as
test and training data, respectively. Under the perfect target
aspect angle estimation, excellent classification result, 0.957, is
acquired.
B. Target Aspect Angle Estimation with Error
To add a degree of uncertainty, target aspect angle estimation
errors are considered. The error can be made by adjusting the
range of the random variable, N, in equation (1). As we change
the error angle range n from 0 to 20, target classification
performance is investigated. Fig. 1 shows the target
classification result for the 10 MSTAR targets. As the angle
estimation error increases, the target classification performance
degrades. Even under the existence of the large estimation error,
22.5, the correct target classification ratio is almost 0.9 which
is excellent results under strong angle estimation error.
C. Complex Gaussian Noise Added SAR Images
To make more realistic target classification environments, a
complex Gaussian noise is added to all the SAR images for the

2015 IEEE 5th Asia-Pacific Conference on Synthetic Aperture Radar(APSAR)

TABLE II. TARGET CLASSIFICATION RESULTS BY THREE PROPOSED FEATURES (DSA, DSAS, AND SPSI)
T72
BTR60
BMP2
2S1
ZSU23
BRDM2
BTR60
D7
T72
ZIL131
Average
Rate

T72
0.980
0.000
0.020
0.011
0.007
0.018
0.005
0.000
0.007
0.000

BTR60
0.000
0.980
0.005
0.004
0.000
0.011
0.021
0.000
0.007
0.000

BMP2
0.020
0.000
0.974
0.007
0.004
0.011
0.010
0.004
0.004
0.000

2S1
0.000
0.000
0.000
0.945
0.004
0.000
0.000
0.004
0.007
0.007

ZSU23
0.000
0.000
0.000
0.011
0.971
0.000
0.000
0.004
0.015
0.004

BRDM2
0.000
0.015
0.000
0.004
0.004
0.905
0.005
0.000
0.004
0.000

BTR60
0.000
0.005
0.000
0.000
0.000
0.026
0.949
0.000
0.004
0.007

D7
0.000
0.000
0.000
0.000
0.007
0.004
0.000
0.985
0.004
0.000

T72
0.000
0.000
0.000
0.000
0.004
0.011
0.000
0.004
0.901
0.004

ZIL131
0.000
0.000
0.000
0.018
0.000
0.015
0.010
0.000
0.048
0.978

0.957

TABLE II. 9 COMBINATIONS OF BMP2 AND T72 VARIANTS

Scenario #1
#2
#3
#4
#5
#6
#7
#8
#9

Fig. 1. Target classification result with varying aspect angle estimation error
Fig. 2. For example, in the case of a tank target, the direction of a turret can
vary, so the test and training targets have different

both test and training data. As the SNR changes, the mean of
the correct target classification ratio is examined. The SNR of
target measurements is calculated as follows.

 









(2)





where 
 is a variance operator. Since it is not possible to
perfectly delete the noise from the MSTAR SAR images, it is
assumed the variance of image in equation (2) is calculated
directly from the MSTAR SAR images.
The SNR changes from 50 dB to -10 dB, and the 10 target
classification results are examined. Fig. 2 shows the target
classification ratio with varying SNR. As can be seen, the
classification ratio with noise contaminated SAR image is over
0.9 under the strong noise, -4 dB SNR. This shows the
robustness of the proposed features under strong noise
classification environments.
D. Different Test and Training Data
In real target classification, it is not practical to assume that
the test and training targets are exactly identical to each other.

BMP2 (Test/Training)
9566 / C21
9566 / C21
9566 / C21
9563 / 9566
9563 / 9566
9563 / 9566
9566 / C21
9566 / C21
9566 / C21

T72 (Test/Training)
132 / S7
132 / 812
812/ S7
132 / S7
132 / 812
812/ S7
132 / S7
132 / 812
812/ S7

directions of the turrets. Hence, in this section, different

variants of targets are used to show the performance of the
proposed features for more realistic classification situations.
In the MSTAR dataset, 3 variants of BMP2 and T72 targets
exist, respectively: BMP2 variants (9563, 9566, C21) and T72
variants (132, 812, S7). In this section, non-identical variants
are used for test and training data to demonstrate the robustness
of the proposed features to minor target changes. In the case of
BMP2, it is reported that all the three variants are basically the
same, but only minor variations exist [9]. Relatively larger
variations exist in T72 variants than BMP2. As the 8 other
targets are remained the same, different BMP2 and T72
variants are used for 10 target classification, and their
properties are as follows: 132 (side skirts and no rear fuel
drum), 812 (side skirts and 2 rear fuel drums), and S7 (no side
skirt and no rear fuel drum). 9 different combinations of the
variants are applied for classification and Table II presents the
combinations.
To demonstrate the effect of different test and training
targets, the confusion matrix of one case, #9, is shown in Table
III. As can be seen, the classification ratios of the two targets,
T72 and BMP2, decreased from 0.98 to 0.75 and 0.98 to 0.89
due to the difference of the difference of the test and training
target. However, the proposed features still recognize T72 and
BMP2 with high correct classification performance which
successfully copes with the small variation of targets. This
results shows the proposed features can extract unique
information of each target and it is significantly important in
real target classification. The average correct classification

2015 IEEE 5th Asia-Pacific Conference on Synthetic Aperture Radar(APSAR)

589

TABLE III. TARGET CLASSIFICATION RESULTS OF DIFFERENT TEST AND TRAINING TARGETS

T72 (S7)
BTR60
BMP2
(C21)
2S1
ZSU23
BRDM2
BTR60
D7
T72
ZIL131
Average
Rate

T72
(812)
0.754
0.000

0.016
0.985

BMP2
(9563)
0.136
0.000

0.056

0.020

0.842

0.056

0.010

0.010

0.005

0.000

0.000

0.000

0.007
0.004
0.018
0.005
0.004
0.011
0.000

0.007
0.000
0.011
0.021
0.000
0.004
0.000

0.007
0.004
0.007
0.000
0.004
0.004
0.000

0.938
0.000
0.000
0.000
0.000
0.007
0.007

0.011
0.978
0.000
0.000
0.004
0.015
0.000

0.004
0.004
0.901
0.005
0.000
0.004
0.000

0.007
0.000
0.026
0.964
0.000
0.004
0.007

0.000
0.007
0.004
0.000
0.985
0.004
0.000

0.000
0.004
0.015
0.000
0.004
0.901
0.007

0.018
0.000
0.018
0.005
0.000
0.048
0.978

BTR60

2S1

ZSU23

BRDM2

BTR60

D7

T72

ZIL131

0.010
0.000

0.021
0.000

0.026
0.010

0.010
0.005

0.000
0.000

0.026
0.000

0.000
0.000

0.923

0.9565 to 0.923, but it still excellent classification performance

for 10 real targets. These results show the reliability and
applicability of the proposed features for real target
classification.
ACKNOWLEDGMENT
The Temasek laboratory, Singapore, sponsored this
research.
REFERENCES

Fig. 3. 10 target classification result with varying SNR

ratio of 10 targets with 9 scenarios is 0.895, which is still

excellent for 10 real target classification.
V. CONCLUSIONS
In this paper, we have examined the reliable target feature
extraction technique with realistic classification scenarios. To
make realistic classification environments, uncertainties of
target measurements are considered such as aspect angle errors,
complex AWGN, and different combination of test and training
targets, and their effects on target classification results are
investigated. In the case of the aspect angle error, the three
proposed features show excellent correct classification
performance which is over 0.9 under existence of 22.5 aspect
angle error. Complex AWGN is also added to the original SAR
images as an uncertainty of target measurements, and their
effects on target classification performance are examined. The
proposed features show high robustness to AWGN as it
correctly classify targets the 10 MSTAR targets under the
strong AWGN. As more realistic classification scenario, 9
different test and training target combinations are adopted and
its classification performance is compared with the same test
and training target case. Due to the difference between the test
and training targets, classification results decreased from

590

[1] S. Doo, G. Smith, and C. Baker, Reliable Target Feature

Extraction and Classification using Potential Target
Information, IEEE International Radar Conference 2015.
[2] X. Liao, P. Runkle, and L. Carin, Identification of ground
targets from sequential high-range-resolution radar signatures,
IEEE Transactions on Aerospace and Electronic System, vol.
38, no. 4, pp. 12301242, 2002.
[3] L. M. Novak, G. J. Owirka, and A. L. Weaver, Automatic
target recognition using enhanced resolution SAR data, IEEE
Transactions on Aerospace and Electronic System, vol. 35, no.
1, pp. 157175, 1999.
[4] J. Gudnason, J. Cui, and M. Brookes, HRR automatic target
recognition from superresolution scattering canter features,
IEEE Transactions on Aerospace and Electronic System, vol.
45, no. 4, pp. 15121524, 2009.
[5] R. Schumacher and J. Schiller, Non-cooperative target
identification of battlefield targets-classification results based on
SAR images, IEEE Radar Conference 2005.
[6] M. C
etin and W. C. Karl, Feature-enhanced synthetic aperture
radar image formation based on nonquadratic regularization,
IEEE Transactions on Image Processing, vol. 10, no. 4, pp. 623
631, 2001.
[7] L. C. Potter and R. L. Moses, Attributed scattering centers for
SAR ATR, IEEE Transactions on Image Processing, vol. 6, no.
1, pp. 7991, 1997.
[8] C. Yuan, and D. Casasent. "MSTAR 10-Class classification and
confuser and clutter rejection using SVRDM." Defense and
Security Symposium. International Society for Optics and
Photonics, 2006.
[9] T. D. Ross, et al. "Standard SAR ATR evaluation experiments
using the MSTAR public release data set." Aerospace/Defense
Sensing and Controls. International Society for Optics and
Photonics, 1998.

2015 IEEE 5th Asia-Pacific Conference on Synthetic Aperture Radar(APSAR)