Beruflich Dokumente
Kultur Dokumente
3, MARCH 2010
AbstractProcessing blurred images is a key problem in many provide a degraded version of the true scene. An important class
image applications. Existing methods to obtain blur invariants of degradations we are faced with in practice is image blurring,
which are invariant with respect to centrally symmetric blur are which can be caused by diffraction, lens aberration, wrong
based on geometric moments or complex moments. In this paper,
we propose a new method to construct a set of blur invariants focus, and atmospheric turbulence. In pattern recognition, two
using the orthogonal Legendre moments. Some important proper- options have been widely explored either through a two steps
ties of Legendre moments for the blurred image are presented and approach by restoring the image and then applying recognition
proved. The performance of the proposed descriptors is evaluated methods, or by designing a direct one-step solution, free of
with various point-spread functions and different image noises. blurring effects. In the former case, the point spread function
The comparison of the present approach with previous methods
in terms of pattern recognition accuracy is also provided. The (PSF), most often unknown in real applications, should be
experimental results show that the proposed descriptors are more estimated [1][5]. In the latter case, finding a set of invariants
robust to noise and have better discriminative power than the that are not affected by blurring is the key problem and the
methods based on geometric or complex moments. subject of this paper.
Index TermsBlur invariants, blurred image, Legendre mo- The pioneering work in this field was performed by Flusser
ments, pattern recognition, symmetric blur. and Suk [6] who derived invariants to convolution with an arbi-
trary centrosymmetric PSF. These invariants have been success-
fully used in template matching of satellite images [6], in pat-
I. INTRODUCTION
tern recognition [7][10], in blurred digit and character recog-
nition [11], [12], in normalizing blurred images into canonical
MAGE processing is a very active area that has impacts
I in many domains from remote sensing, robotics, traffic
surveillance, to medicine. Automatic target recognition and
forms [13], [14], and in focus/defocus quantitative measurement
[15]. More recently, Flusser and Zitova introduced the com-
bined blur-rotation invariants [16] and reported their successful
tracking, character recognition, 3-D scene analysis and recon- application to satellite image registration [17] and camera mo-
struction are only a few objectives to deal with. Since the real tion estimation [18]. Suk and Flusser further proposed a set of
sensing systems are usually imperfect and the environmental combined invariants which are invariant to affine transform and
conditions are changing over time, the acquired images often to blur [19]. The extension of blur invariants to -dimensions
has also been investigated [20], [21]. All the existing methods
to derive the blur invariants are based on geometric moments
Manuscript received May 16, 2009; revised October 14, 2009. First published
or complex moments. However, both geometric moments and
November 20, 2009; current version published February 18, 2010. This work
was supported in part by the National Natural Science Foundation of China complex moments contain redundant information and are sen-
under Grant 60873048, in part by the National Basic Research Program of China sitive to noise especially when high-order moments are con-
under Grant 2010CB732503, in part by the Program for Changjiang Scholars cerned. This is due to the fact that the kernel polynomials are
and Innovative Research Team in University, and in part by the Natural Science
Foundation of Jiangsu Province under Grant BK2008279. The associate editor not orthogonal.
coordinating the review of this manuscript and approving it for publication was Teague has suggested the use of orthogonal moments to re-
Dr. Ercan E. Kuruoglu. cover the image from moments [22]. It was shown that the or-
H. Zhang, H. Shu, and L. Luo are with the Laboratory of Image Science
and Technology, Department of Computer Science and Engineering, Southeast thogonal moments are better than other types of moments in
University and also with the Centre de Recherche en Information Biomdi- terms of information redundancy, and are more robust to noise
cale Sino-Franais, 210096 Nanjing, China (e-mail: wtian@seu.edu.cn; shu. [23]. As noted by Teh and Chin [23], the moment invariants are
list@seu.edu.cn; luo.list@seu.edu.cn).
G. N. Han is with the Institute de Recherche en Mathmatiques et Ap- considered reliable features in pattern recognition if they are in-
plication, Universit Louis Pasteur, 67984 Strasbourg, France (e-mail: sensitive to the presence of image noise. Consequently, it could
guoniu@math.u-strasbg.fr).
be expected that the use of orthogonal moments in the construc-
G. Coatrieux is with the Institut TELECOM; TELECOM Bretagne; INSERM
U650 Latim, Brest, F-29238, France (e-mail: gouenou.coatrieux@telecom-bre- tion of blur invariant provides better recognition results. To the
tagne.eu). authors knowledge, no orthogonal moments have been used to
J. L. Coatrieux is with the INSERM U642, 35000 Rennes, France,
construct the blur invariants.
Laboratoire Traitement du Signal et de lImage, Universit de Rennes I,
35000 Rennes, France, and also with the Centre de Recherche en Infor- In this paper, we propose a new method to derive a set of blur
mation Biomdicale Sino-Franais (CRIBs), 35000 Rennes, France(e-mail: invariants based on orthogonal Legendre moments (for a recent
jean-louis.coatrieux@univ-rennes1.fr). survey on moments, refer to [24][27]). The organization of this
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org. paper is as follows: in Section II, we review the theory of blur
Digital Object Identifier 10.1109/TIP.2009.2036702 invariants of geometric moments and the definition of Legendre
1057-7149/$26.00 2010 IEEE
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
ZHANG et al.: BLURRED IMAGE RECOGNITION BY LEGENDRE MOMENT INVARIANTS 597
moments. In Section III, we establish a relationship between Lemma 1: The centroid of the blurred image is related
the Legendre moments of the blurred image and those of the to the centroid of the original image and that of the PSF
original image and the PSF. Based on this relationship, a set of as
blur invariants using Legendre moments is provided. The exper-
imental results for evaluating the performance of the proposed (7)
descriptors are given in Section IV. Finally, some concluding
remarks are provided.
In particular, if is centrally symmetric, then
. In such a case, we have , .
II. BLUR INVARIANTS AND LEGENDRE MOMENTS The proof of Lemma 1 can be found in [9].
This section first reviews the theory of blur invariants of geo-
B. Legendre Moments
metric moments proposed by Flusser and Suk [6], [7], and then
presents some basic definitions of Legendre moments. The 2-D th order Legendre moment of image function
is defined as [28]
A. Blur Invariants of Geometric Moments
The 2-D geometric moment of order , with image in- (8)
tensity function , is defined as
where is the -order orthonormal Legendre polyno-
(1) mials given by
(9)
where, without loss of generality, we assume that the image
function is defined on the square .
with
The corresponding central moment of image is de-
fined as even (10)
odd
(2) The corresponding central moments are defined as
III. METHOD
Let be a blurred version of the original image .
The blurring is classically described by the convolution In this section, we first establish a relationship between the
Legendre moments of the blurred image and those of the orig-
(4) inal image and the PSF. We then derive a set of blur moment
invariants.
where is the PSF of the imaging system, and denotes
linear convolution. A. Legendre Moments of the Blurred Image
In this paper, we assume that the PSF, , is a centrally The 2-D normalized Legendre moments of blurred image,
symmetric image function and the imaging system is energy- , are defined by (12), shown at the bottom of the next
preserving, that is, page.
In the rest of this subsection, we discuss how to express the
(5) Legendre moments of blurred image defined by (12) in terms of
Legendre moments of the original image and the PSF.
(6) Making the notation
and where the superscript indi-
As noted by Flusser [7], the assumption of centrally symmetry is cates the vector transposition, we have
not a significant limitation of practical utilization of the method.
Most real sensors and imaging systems have PSFs with certain (13)
degrees of symmetry. In many cases they have even higher sym-
metry than central, such as axial or radial symmetry. Thus, the where , with , is a
central symmetry assumption is general enough to describe al- lower triangular matrix whose elements are given
most all practical situations. by (10).
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
598 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010
(14) (20)
where , with , is the inverse
matrix of . The elements of are given by [29] The following theorem reveals the relationship between the
Legendre moments of the blurred image and those of the orig-
even
inal image and the PSF.
Theorem 1: Let be the original image function and
odd. the PSF be an arbitrary image function, and be
(15) a blurred version of , then the relations
By expanding (14), we obtain
(16)
Similarly (21)
(17) and
(22)
(18)
hold for every and .
Proof: Substituting (19) and (20) into (12), we have (23),
Substitution of (16) and (17) into (18) yields shown at the bottom of the next page. The above equation can
be rewritten as
(24)
(12)
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
ZHANG et al.: BLURRED IMAGE RECOGNITION BY LEGENDRE MOMENT INVARIANTS 599
(23)
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
600 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010
Fig. 2. Some examples of the blurred image: (a) averaging blur with additive
zero-mean Gaussian noise, STD = 10; (b) motion blur with additive zero-mean
Gaussian noise, STD = 20; (c) out-of-focus blur with additive salt-and-pepper
noise, density = 0:01; (d) Gaussian blur with additive salt-and-pepper noise,
density = 0:02.
Fig. 5. Relative error for out-of-focus blur with salt-and-pepper noise shown
in Fig. 2(c). Horizontal axis: noise density; vertical axis: relative error between
the corrupted image and original image.
Fig. 3. Relative error for averaging blur with Gaussian noise shown in Fig. 2(a).
Horizontal axis: standard deviation of noise; vertical axis: relative error between
the corrupted image and original image.
Fig. 6. Relative error for Gaussian blur adding salt-and-pepper noise shown in
Fig. 2(d). Horizontal axis: noise density; vertical axis: relative error between the
corrupted image and original image.
(27)
Fig. 4. Relative error for motion blur with Gaussian noise shown in Fig. 2(b).
Horizontal axis: standard deviation of noise; vertical axis: relative error between
the corrupted image and original image. where is Euclidean norm in space. In the following
experiments, moment invariants of order up to are used.
The next experiment was carried out to verify the perfor-
parameters such as the size for averaging blur, the radius for mance of the invariants to both image blur and noise. The orig-
out-of-focus and the depth for motion filter were chosen equal to inal cat image was blurred by a 9 9 averaging mask and a
the size of blur mask in all the experiments. We first checked that zero-mean Gaussian noise with standard deviation (STD) from 1
the eighteen Legendre moment invariants of order up to seven to 50 was added. Some examples of the blurred image with addi-
(listed in Appendix B) were exactly equal to those of the orig- tive Gaussian noise or salt-and-pepper noise are shown in Fig. 2.
inal image whatever the blurring mode (the corresponding nu- Plots in Fig. 3 compare the relative error defined by (27) for
merical values are omitted here). Flussers method based on geometric moment invariants (GMI)
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
ZHANG et al.: BLURRED IMAGE RECOGNITION BY LEGENDRE MOMENT INVARIANTS 601
Fig. 8. Eight objects selected from the Coil-100 image database of Columbia University.
Fig. 9. Some examples of the blurred images corrupted by various types of noise.
TABLE I TABLE II
RECOGNITION RATES OBTAINED RESPECTIVELY WITH GMI, CMI AND LMI RECOGNITION RATES OF THE GMI, CMI AND LMI IN OBJECT
FOR ALPHANUMERIC CHARACTER IN FIG. 7 RECOGNITION (FIG. 9)
where eighteen blur invariants derived from central moments blurred by a 11 11 motion filter, and the same Gaussian noise
are used [7], the complex moment invariants (CMI) reported in was added. The results (Fig. 4) again indicate the better behavior
[16] and the present Legendre moment invariants (LMI) up to of the proposed method. Similarly, the original cat image was
order seven by averaging blur with different Gaussian noises. It degraded on one hand by out-of-focus blur (13 pixel-radius of
can be seen from the figure that the proposed descriptors per- the PSF support) and by adding a salt-and-pepper noise with
form better than the GMI and CMI. Then, the cat image was noise densities varying from 0.004 to 0.2 (see Fig. 5) and, on an-
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
602 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010
Fig. 10. The comb. The extent of out-of-focus blur increases from Image1 to Image8.
other hand, by Gaussian blur (the PSF was a Gaussian function but the decrease in recognition rate is more significant when the
with 15 pixel-radius of support) with the same salt-and-pepper noise level is increased. This is also true for the LMI. The CMI
noise (see Fig. 6). It can be also seen that a better robustness is do not perform well in these experiments due to their additional
achieved whatever the PSF or the additive noises. invariance to rotation. The worse numerical stability is a tax on
the combined invariance. The orthogonality of LMI explains the
B. Classification Results difference in performance with GMI.
This experiment was carried out to compare the discrimina- We also compared the computational load of the GMI, CMI
tion power of the GMI, CMI and LMI. A set of alphanumeric and LMI in these two experiments. The programs were imple-
characters whose size is 50 50 pixels (Fig. 7) is used for the mented in MATLAB 6.5 on a PC P4 2.4 GHZ, 512M RAM.
recognition task. The reason for choosing such a character set is It can be seen from Tables I and II that the GMI and the LMI
that the elements in subset {0, o}, {2, Z}, {7, T}, and {9, q} can computations are much faster than the CMI ones. This is due to
be easily misclassified due to their similarity. The testing set is the fact that the computation of the complex moments requires
generated by adding averaging blur, out-of-focus blur, Gaussian a mapping transformation which is time consuming.
blur and motion blur with mask of sizes 3 3, 4 4, 5 5,
6 6, 7 7, 8 8, 9 9, 10 10, 11 11, 12 12 pixels, re- C. Real Image Analysis
spectively. The parameter of Gaussian blur was chosen equal In the last experiment, we tested the performance of the in-
to 1 or 2, and the parameter of motion blur set to 0 or 1, variants on images degraded by real out-of-focus blur. A se-
forming a set of 480 images. Note that the original images as quence of eight pictures of a comb lying on a black ground was
well as the blurred images are mapped onto the area of orthog- taken by a digital camera (Panasonic DMC-FZ50). The images
onality, and the actual size of the blurred images in this exper- differ from each other by the level of out-of-focus blur. The pic-
iment is 80 80. This is followed by adding a white Gaussian ture was captured 8 times from the same position but with dif-
noise with different standard deviations, salt-and-pepper noise ferent focus depth, manually set. All the test images are depicted
with different noise densities and multiplicative noise with dif- in Fig. 10. The values of GMI, CMI, and LMI were computed
ferent noise densities. The Euclidean distance is used here as for each image. Table III depicts the values of , where de-
the classification measure. Table I shows the classification rates notes the mean of eight real images and the standard deviation.
using the different moment invariants. One can observe from From this table, it can be seen that the minimal value of the LMI
this table that the recognition results are quite good for the dif- is 3.42% and the maximum value of the LMI is 6.15%, which
ferent methods in the noise-free case. The classification rates are lower than those obtained with GMI (resp. 4.91%, 12.43%)
remain high for low and moderate noise levels but decrease sig- and the CMI (resp. 7.47%, 7.54%).
nificantly when the noise level goes up. However, if the GMI
behaves better than the CMI, the LMI approach is the only one
V. CONCLUSION AND PERSPECTIVES
providing a rate close to or over 90% whatever the noise nature
and its level. In this paper, we have proposed a new approach to derive a
In the next example, eight objects were selected from the set of blur invariants using the orthogonal Legendre moments.
Coil-100 image database of Columbia University as an original The relationship between the Legendre moments of the blurred
image set (see Fig. 8). The actual size of the blurred images in image and those of the original image and the PSF has been
this experiment is 160 160. Fig. 9 shows some examples of established, and using this relationship, a set of blur invariants
the blurred and corrupted images. The recognition results are based on Legendre moments has been derived. The experiments
displayed in Table II. They lead to the same conclusions re- conducted so far in very distinct situations demonstrated that the
garding the performance of the respective moment invariants proposed descriptors are more robust to noise and have better
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
ZHANG et al.: BLURRED IMAGE RECOGNITION BY LEGENDRE MOMENT INVARIANTS 603
TABLE III
GMI CMI AND LMI VALUES OF THE REAL IMAGES IN FIG. 10
(A6)
(A1)
(A2)
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
604 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010
shown at the bottom of the page. (A7) can be further written as Letting , we have
(A9), shown at the bottom of the page, where the convention
if is used in the above equation.
Since the matrix is the inverse of , we have
; thus, (A9) becomes
(A11)
(A10) (A13)
(A7)
(A8)
(A9)
(A12)
(A14)
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
ZHANG et al.: BLURRED IMAGE RECOGNITION BY LEGENDRE MOMENT INVARIANTS 605
(A17)
(A15)
(A16)
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
606 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010
Using (21), we have (A18), shown at the bottom of the page, Similarly
where
s t
L(s;tg) 0 Ls;t
(f )
= Li(f;j)
i =0 j =0
i +j <s+t
s0i t0j
(A19) 2 Ls(h;t) A(s; t; i0 ; j 0 ; s0 ; t0 ): (A20)
s =0 t =0
(A18)
(A21)
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
ZHANG et al.: BLURRED IMAGE RECOGNITION BY LEGENDRE MOMENT INVARIANTS 607
Substituting (A18) and (A20) into (A16) and using (A17), equation, we obtain (A25), shown at the bottom of the page,
we obtain (A21), shown at the bottom of the previous where
page. Define (A22) and (A23), shown at the bottom
of the page. it can be easily verified from (A19) that
; thus, we
have . Using this relationship, (A21) can be
rewritten as (A24), shown at the bottom of the page. Changing
the order of summation and shifting the indices in the above
(A22)
(A23)
(A24)
(A25)
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
608 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010
Thus
(A29)
The proof of Theorem 3 has been completed.
(A26)
Using (A19), we have (A27), shown at the bottom of the page. APPENDIX B
where and are respectively given by LIST OF LEGENDRE MOMENT INVARIANTS
(A1) and (A2). UP TO THE SEVENTH ORDER
Using Lemma 2, we have The expressions given below provide to the interested readers
all the elements to replicate our method and to apply it to other
(A28) examples.
(A27)
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
ZHANG et al.: BLURRED IMAGE RECOGNITION BY LEGENDRE MOMENT INVARIANTS 609
Third order
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
610 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.
ZHANG et al.: BLURRED IMAGE RECOGNITION BY LEGENDRE MOMENT INVARIANTS 611
Limin Luo (M90SM97) received the Ph.D. Jean Louis Coatrieux (M84SM89F94) re-
degree in 1986 from the University of Rennes, ceived the Ph.D. degree and the State Doctorate in
France. Sciences in 1973 and 1983, respectively, from the
He is currently a Professor in the Department University of Rennes 1, Rennes, France.
of Computer Science and Engineering, Southeast Since 1986, he has been Director of Research at the
University, China. His current research interests National Institute for Health and Medical Research
include medical imaging, image analysis, com- (INSERM), France, and since 1993, he has been a
puter-assisted systems for diagnosis and therapy Professor at the New Jersey Institute of Technology.
in medicine, and computer vision. He is also a Professor at Telecom Bretagne, Brest,
Dr. Luo is an associate editor of the IEEE France. He was Head of the Laboratoire Traitement
Engineering in Medicine and Biology Magzine du Signal et de lImage, INSERM, until 2003. His ex-
and Innovation et Technologies en Biologie et Medicine (ITBM). perience is related to 3-D images, signal processing, pattern recognition, com-
putational modeling, and complex systems with applications in integrative bio-
medicine. He has published more than 300 papers in journals and conferences
and edited many books in these areas.
Dr. Coatrieux served as the Editor-in-Chief of the IEEE TRANSACTIONS ON
BIOMEDICAL ENGINEERING (19962000) and is on the boards of several jour-
nals. He has received several awards from the IEEE (among which include the
EMBS Service Award, 1999, the Third Millennium Award, 2000, and the EMBS
Career Achievement Award, 2006), and he received the Doctor Honoris Causa
from the Southeast University, China.
Authorized licensd use limted to: IE Xplore. Downlade on May 13,20 at 1:4750 UTC from IE Xplore. Restricon aply.