12 DECEMBER 1999
1589
PAPER
A New Class of Multichannel Image Processing Filters:
Vector MedianRational Hybrid Filters
Lazhar KHRIJI
, Nonmembers
SUMMARY A new class of nonlinear lters called Vector
Median Rational Hybrid Filters (VMRHF) for multispectral im
age processing is introduced and applied to color image ltering
problems. These lters are based on Rational Functions (RF).
The VMRHF lter is a twostage lter, which exploits the fea
tures of the vector median lter and those of the rational opera
tor. The lter output is a result of vector rational function oper
ating on the output of three subfunctions. Two vector median
(VMF) sublters and one center weighted vector median lter
(CWVMF) are proposed to be used here due to their desirable
properties, such as, edge and details preservation and accurate
chromaticity estimation. Experimental results show that the new
VMRHF outperforms a number of widely known nonlinear lters
for multispectral image processing such as the Vector Median il
ter (VMF) and Distance Directional Filters (DDf) with respect
to all criteria used.
key words: rational functions, vector rational lters, vector
median lters, vector medianrational hybrid lters
1. Introduction
Multichannel image processing is investigated in this
paper using a vector approach[10]. This has been
proved to be more appropriate for vectorvalued signals
compared to traditional componentwise approaches.
The suitability of such a technique is often credited
to the inherent correlation that exists between the im
age channels [10]. In the vector approach, each image
element is considered as an mdimensional vector (m
is the number of image channels). Color image pix
els are often represented by threecomponent vectors,
m = 3, whose characteristics, i.e., magnitude and di
rection are examined. The vectors direction signies
its chromaticity; while, its magnitude is a measure of
its brightness. A number of vector processing lters
have been proposed in the literature for the purpose
of color image processing [2], [11]. One class of lters
considers the distance in the vector space between the
image vectors; typical representative of this class is the
vector median lter (VMF) [1]. A second class of l
ters, called vector directional lters (VDF) [16], op
erates by considering the vectors direction. VMFs are
derived as maximum likelihood estimators for an expo
nential distribution [1]; while, VDFs are spherical esti
mators, when the underlying distribution is a spheri
Manuscript received November 19, 1998.
Manuscript revised June 30, 1999.
i=1
a
1i
x
j
+
n
i=1
n
j=1
a
2ij
x
i
x
j
+
b
0
+
n
i=1
b
1i
x
i
+
n
i=1
n
j=1
b
2ij
x
i
x
j
+
, (1)
where x
1
, x
2
, . . . , x
n
are the scalar inputs to the lter
and y is the lter output, a
0
, b
0
, a
ij
and b
ij
are lter
parameters.
The representation described in Eq. (1) is unique
up to common factors in the numerator and denomi
nator polynomials. The RF (Rational Function) must
clearly have a nite order to be useful in solving prac
tical problems. Like polynomial functions, a rational
function is a universal approximator [9]. Moreover, it is
able to achieve substantially higher accuracy with lower
complexity and possesses better extrapolation capabil
ities than polynomial functions.
Straight forward application of the rational func
tions to multichannel image processing would be based
on processing the image channels separately. This how
ever, fails to utilize the inherent correlation that is usu
ally present in multichannel images. Consequently, vec
tor processing of multichannel images is desirable [10].
The generalization of the scalar rational lter deni
tion to vector and scalar signals alike is given by the
following denition.
Denition 2.1: Let x
1
, x
2
, . . . , x
n
be the n input vec
tors to the lter, where x
i
=
x
1
i
, x
2
i
, . . . , x
m
i
T
and
x
k
i
0, 1, . . . , M, M is an integer. The VRF output
is given by
V RF = RF
x
1
, x
2
, . . . , x
n
=
P(x
1
, x
2
, . . . , x
n
)
Q(x
1
, x
2
, . . . , x
n
)
=
rf
1
, rf
2
, . . . , rf
m
T
(2)
where P is a vectorvalued polynomial and Q is a scalar
polynomial. Both are functions of the input vectors.
The ith component of the VRF output is written as
rf
i
=
P
i
(x
1
, x
2
, . . . , x
n
)
Q(x
1
, x
2
, . . . , x
n
)
0, 1, . . . , M (3)
where
P
i
(x
1
, x
2
, . . . , x
n
) = a
0
+
n
k=1
a
k
x
i
k
(4)
+
n
k1=1
n
k2=1
a
k1k2
x
i
k1
x
i
k2
+
(5)
and
Q(x
1
, x
2
, . . . , x
n
) = b
0
+
n
j=1
n
k=1
b
jk
x
j
x
k

p
(6)
.
p
is the L
p
norm, and the square braket notation
used in Eq. (3) above, [] refers to the integer part of
,
+
. b
0
> 0, b
ij
are constant, and a
i1,i2,...,ir
used in Eq. (4) is a function of the input vectors:
a
i1,i2,...,ir
= f(x
1
, x
2
, . . . , x
n
). (7)
When the vector dimension is 1, the VRF reduces to a
special case of the scalar RF
3. Vector MedianRational Hybrid Filters
(VMRHF)
When extending the medianrational hybrid operation
to vectorvalued signals, we place some requirements on
the resulting vector medianrational hybrid operation:
The operation should have properties similar to
those of the scalar case.
It should have robust data smoothing ability for
dierent i.i.d. noise distributions (Gaussian, im
pulsive, mixed Gaussianimpulsive), while retain
ing sharp edges in the signal.
It reduces to the scalar lter if the vector dimension
is 1.
3.1 Vector MedianRational Hybrid Filters
Let f (x) : Z
l
Z
m
, represent a multichannel signal
and W Z
l
be a window of nite size n (lter length).
l represents the signal dimensions and m the number of
signal channels. The pixels in W will be denoted as x
i
,
i = 1, 2, . . . , n and f (x
i
) will be denoted as f
i
. f
i
are m
dimensional (m 2) vectors in the vector space dened
by the m signal channels. The VMRHF is dened next.
Denition 3.1: the output vector y(f
i
) of the VM
RHF is the result of a vector rational function tak
ing into account three input subfunctions which form
an input functions set
1
,
2
,
3
, where the central
one (
2
) is xed as a center weighted vector median
sublter
y(f
i
) =
2
(f
i
) +
3
j=1
j
(f
i
)
h +k[[
1
(f
i
)
3
(f
i
)[[
2
(8)
where [[.[[
2
is the L
2
vector norm, = [
1
,
2
,
3
] char
acterizes the constant vector coecient of the input
subfunctions. In this approach, we have chosen a very
simple prototype lter coecients which satises the
condition:
3
i=1
i
= 0. In our study, = [1, 2, 1]
T
.
h and k are some positive constants. The parameter k
is used to control the amount of the nonlinear eect.
The sublters
1
and
3
are chosen so that an accept
able compromise is achieved between noise reduction,
edge and chromaticity preservation. It is easy to ob
serve that this VMRHF diers from a linear lowpass
KHRIJI and GABBOUJ: A NEW CLASS OF MULTICHANNEL IMAGE PROCESSING FILTERS
1591
lter mainly for the scaling, which is introduced on the
1
and
3
terms. Indeed, such terms are divided by a
factor proportional to the output of an edgesensing
term characterized by the Euclidean distance of the
vector dierence (
1
3
). The weight of the vector
medianoperation output term is accordingly modied,
in order to keep the gain constant. The behavior of
the proposed VMRHF structure for dierent positive
values of parameter k is described next.
k 0, the form of the lter is given as a linear
lowpass combination of the three nonlinear sub
functions:
y(f
i
) = c
1
1
(f
i
) +c
2
2
(f
i
) +c
3
3
(f
i
), (9)
where coecients c
1
, c
2
, and c
3
are some constants.
k , the output of the lter is identical to the
central sublter output and the vector rational
function has no eect:
y(f
i
) =
2
(f
i
). (10)
For intermediate values of k, the [[
1
(f
i
)
3
(f
i
)[[
2
term perceives the presence of a detail and accord
ingly reduces the smoothing eect of the operator.
Therefore, the VMRHF operates as a linear lowpass
lter between three nonlinear suboperators, the coef
cients of which are modulated by the edgesensitive
component.
4. The Proposed Filter Structures
Vector MedianRational Hybrid Filters (VMRHFs) are
promising detail preserving ltering structures [8] since
it was shown that every sublter is able to preserve
signal details within their subwindows. VMRHFs are
grouped into two classes: unidirectional VMRHFs and
bidirectional VMRHFs. The structures for a 33 win
dow unidirectional VMRHF and a bidirectional VM
RHF are shown in Figs. 1 (a) and (b), respectively.
Only the points indicated in black in each window mask
are used in the corresponding operation.
Unidirectional VMRHFs are designed to preserve
image details along the vertical, horizontal and the two
diagonal directions. Therefore, the samples of the same
value neighborhood must be located along those direc
tions in order to preserve the center sample by unidirec
tional VMRHFs. On the other hand, bidirectional VM
RHFs can preserve details within the two corresponding
directions in one operation.
The central sublter is a center weighted vector
median lter characterized by its high detail preser
vation capability. One of the following three sets of
weights can be used depending the noise properties and
the image details [4]. Mask M1 emphasizes details in
the horizontal and vertical directions, while M2 the two
diagonal directions. On the other hand, mask M3 seeks
Fig. 1 Structures of VMRHF. (a) The unidirectional structure,
(b) the bidirectional structure.
details in all of these directions simultaneously.
M1
0 1 0
1 3 1
0 1 0
M2
1 0 1
0 3 0
1 0 1
M3
1 1 1
1 3 1
1 1 1
5. Experimental Results
VMRHF have been evaluated, and their performance
has been compared against those of some widely known
vector nonlinear lters: the vector median lter (VMF),
the distance directional lter (DDF), the generalized
vector directional lter (GVDF) [13] and the marginal
medianrational hybrid lter (mMRHF), using RGB
color images.
The noise attenuation properties of the dierent
lters are examined by utilizing two color images: (1)
part of Lena image (256 256), see Fig. 2 (a); and (2)
the roze image (240 150), see Fig. 2 (b). The test im
ages have been contaminated using various noise source
models in order to assess the performance of the lters
under dierent scenarios:
Gaussian noise, ^(0,
2
).
Impulsive noise: each image channel is corrupted
independently using salt and pepper noise. We
1592
IEICE TRANS. INF. & SYST., VOL.E82D, NO.12 DECEMBER 1999
Fig. 2 Test color images. (a) Lena image, (b) roze image.
assume that both salt and pepper are equally likely
to occur.
Mixed Gaussianimpulsive noise: the impulsive
noise is xed (salt and pepper 2% in each image
channel), while the variance of the Gaussian noise
is varied.
The original image, as well as its noisy versions,
are represented in the RGB color space. This color
coordinate system is considered to be objective, since
it is based on the physical measurements of the color
attributes. The lters operate on the images in the
RGB color space.
A number of dierent objective measures can be
utilized for quantitative comparison of the performance
of the dierent lters. These criteria provide some mea
sure of closeness between two digital images by exploit
ing the dierences in the statistical distributions of the
pixel values [3]. The most widely used measures are
the mean absolute error (MAE), and the mean square
error (MSE) dened as:
MAE =
1
MN
M
i=1
N
j=1
[[y
i, j
d
i, j
[[
1
(11)
MSE =
1
MN
M
i=1
N
j=1
[[y
i, j
d
i, j
[[
2
2
(12)
where M, N are the image dimensions, y
i, j
is the vec
tor value of pixel (i, j) of the ltered image, d
i, j
is the
corresponding pixel in the original noise free image, and
[[.[[
1
, [[.[[
2
are the L
1
 and L
2
vector norm, respectively.
Notwithstanding the RGB is the most popular
color space used conventionally to store, process, dis
play and analyze color images, the human perception
of color cannot be described using the RGB model [14].
Consequently, measures such as the mean square er
ror dened in the RGB color space is not appropriate
to quantify the perceptual error between images. It
is therefore important to use color spaces which are
closely related to the human perceptual characteris
tics and suitable for dening appropriate measures of
perceptual errors between color vectors. A number of
such color spaces are used in areas such as multimedia,
video communications (e.g., high denition television),
motion picture production, the printing industry, and
graphic arts. Among these, perceptually uniform color
spaces are the most appropriate to dene simple yet
precise measures of perceptual errors. The Commis
sion Internationale de lEclairage (CIE) standardized
two color spaces, L
and L
, as perceptually
uniform[5].
Conversion from RGB to L
space, the
L
and b
i=1
N
j=1
[[E
Lab
[[
M
i=1
N
j=1
[[E
Lab
[[
(13)
where E
Lab
is the perceptual color error between two
color vectors and dened as the Euclidean distance be
tween them, given by
E
Lab
= [(L
)
2
+ (a
)
2
+ (b
)
2
]
1
2
, (14)
where L
, a
, and b
,
a
, and b
components, respectively. E
Lab
is the mag
nitude of the original image pixel vector in the L
Lab
= [(L
)
2
+ (a
)
2
+ (b
)
2
]
1
2
.
The results obtained are shown in the form of plots in
Figs. 35 for the three noise models: Gaussian, impul
sive, and Gaussian mixed with impulsive, respectively.
The simulation results of the two structures of the VM
RHF are very close to each other (slight dierences), we
KHRIJI and GABBOUJ: A NEW CLASS OF MULTICHANNEL IMAGE PROCESSING FILTERS
1593
Fig. 3 Comparative results for the images in Fig. 2 contaminated by Gaussian noise.
(a) Lena image, (b) roze image.
Fig. 4 Comparative results for the images in Fig. 2 contaminated by impulsive noise
(salt and pepper). (a) Lena image, (b) roze image.
Fig. 5 Comparative results for the images in Fig. 2 contaminated by mixed noise (salt
and pepper 2% in each component, and Gaussian with zero mean and variable variance).
(a) Lena image, (b) roze image.
1594
IEICE TRANS. INF. & SYST., VOL.E82D, NO.12 DECEMBER 1999
Fig. 6 Results for the Lena image (Fig. 2 (a)). (a) Contami
nated image by mixed noise (impulsive 2% in each channel and
Gaussian with zero mean and variance 50). Images (b), (c) and
(d) are the processed images by the DDF, VMF and VMRHF,
respectively.
hence reported only those provided by the bidirectional
structure given by Fig. 1 (b).
As can be veried from the plots, the VMRH l
ters provide better results than those obtained by any
other lter under consideration. Recall that VMRH
lter uses no information about the type and the de
gree of noise corruption. Moreover, consistent results
have been obtained when using a variety of other color
images and the same evaluation procedure.
The ltered images are presented for visual assess
ment, since in many cases they are, ultimately, the best
subjective measure of the eciency of image processing
techniques. Figures 6 (a) and 7 (a) show the corrupted
images by mixed noise (impulsive 2% in each channel
and Gaussian ^(0, 50)) of Lena and roze, respectively.
Figures 6 (b)(d) represent the ltered images by DDF,
VMF and VMRHF, respectively for Lena image. Fig
ures 7 (b)(d) show the same representation procedure
for the roze image. All the lters considered operate
using a square 3 3 processing window.
Vector processing, i.e. VMRHF, produced bet
ter results compared to marginal (componentwise) l
tering, i.e. marginal MRHF. The marginal median
rational hybrid lter fails to take into account the de
pendence between the components (i.e. the interchannel
correlation).
The new VMRH lter outperforms the GVDF
which uses a priori knowledge about the actual noise
characteristics to optimize its performance.
Fig. 7 Results for the roze image [Fig. 2 (b)]. (a) Contami
nated image by mixed noise (impulsive 2% in each channel and
Gaussian with zero mean and variance 50). Images (b), (c) and
(d) are the processed images by the DDF, VMF and VMRHF,
respectively.
Fig. 8 (a) Part (128128) of the color Lenna image corrupted
with additive (4% in each channel) impulsive noise. (b)(f) re
sults using the DDF, the GVDF, the VMF, the marginal MRHF
and the VMRHF, respectively.
An additional sample processing results are pre
sented in Figs. 8 (a)(f). Figure 8 (a) shows a part
(128 128) of color Lenna image corrupted with ad
KHRIJI and GABBOUJ: A NEW CLASS OF MULTICHANNEL IMAGE PROCESSING FILTERS
1595
ditive (4% in each channel) impulsive noise. Fig
ures 8 (b)(f) present the results using the DDF, the
GVDF, the VMF, the marginal MRHF and the VM
RHF respectively. A comparison of the images clearly
favors the proposed VMRHF over its counterparts
(VMF, GVDF, DDF, and Marginal MRHF). The pro
posed VMRHF can eectively remove impulses, smooth
out nominal noise and keep edges, details and color uni
formity unchanged as we can see from the related error
measures summarized in the plots.
Furthermore, it is worth mentioning that the pro
posed lter has comparable computational complexity
to those used in the comparison, particularly the VMF.
The vector rational operation does not introduces sig
nicant additional computational cost. In the absence
of any fancy or fast algorithms, the number of com
parators used in the median lter with a window of
size n is N
c
=
n(n1)
2
. According to Figs. 1 (a)(b), the
rst stage of the VMRHF requires 41 comparators: 10
comparators for
1
(n
1
= 5), 10 comparators for
3
(n
3
= 5) and 21 comparators for
2
(n
2
= 7). The
second stage requires a small lookup table for the de
nominator, one multiplication, three additions and one
division per output sample.
6. Conclusions
MedianRational Hybrid Filters are extended in this pa
per to vectorvalued signals and applied to multidimen
sional image processing. The vector medianrational
hybrid lter is a vector rational operation over three
sublters in which the middle one is a center weighted
vector median lter. These new lters exhibit very de
sirable ltering properties and utilize in an eective way
the performance of the vector rational function lters
and the features of vector median lters. Simulation
results and subjective evaluation of the ltered images
indicate that the VMRHFs outperform all other lters
under consideration. Vector processing has also proved
to produce better results than marginal (component
wise) processing. Moreover, as it can be seen from the
processed images, the VMRHF preserve the chromatic
ity component of the processed color images.
References
[1] J. Astola, P. Haavisto, and Y. Neuvo, Vector median l
ter, Proc. IEEE, vol.78, pp.678689, April 1990.
[2] J. Astola and P. Kuosmanen, Fundamentals of Nonlinear
Digital Filtering, CRC Press, Boca Raton, New York, 1997.
[3] A.M. Eskicioglo, P.S. Fisher, and S. Chen, Image quality
measures and their performance, IEEE Trans. Commun.,
vol.43, pp.29592965, 1995.
[4] M. Gabbouj, P. Haavisto, and Y. Neuvo, Recent advances
in median ltering, Communications, Control and Signal
Processing, ed. E. Arikan, vol.II, pp.10801094, Ankara,
Turkey, Elsevier Science Publishers B.V., 1990.
[5] A.K. Jain, Fundamentals of Digital Image Processing,
PrenticeHall, New Jersey, 1989.
[6] L. Khriji, F.A. Cheikh, and M. Gabbouj, High resolution
digital resampling using vector rational lters, J. Optical
Engineering, Special Issue on Sampled Imaging Systems,
vol.38, no.5, pp.893901, May 1999.
[7] L. Khriji and M. Gabbouj, Medianrational hybrid l
ters for image restoration, IEE Electronics Letters, vol.34,
no.10, pp.977979, May 1998.
[8] L. Khriji and M. Gabbouj, Medianrational hybrid lters,
Proc. IEEE Int. Conf. Image Proc., ICIP98, Chicago, Illi
nois, Oct. 47, 1998.
[9] H. Leung and S. Haykin, Detection and estimation using
an adaptive rational function lter, IEEE Trans. Signal
Processing, vol.42, no.12, pp.33663376, Dec. 1994.
[10] R. Machuca and K. Phillips, Applications of vector elds
to image processing, IEEE Trans. Pattern Anal. & Mach.
Intell., vol.PAMI5, pp.316329, May 1983.
[11] I. Pitas and A.N. Venetsanopoulos, Nonlinear Digital Fil
ters  Principles and Applications, Kluwer Academic Pub
lishers, 1990.
[12] K.N. Plataniotis, D. Androutsos, and A.N. Venetsanopoulos,
Adaptive multichannel lters for colour image processing,
Signal Processing: Image Communication, vol.11, pp.171
177, 1998.
[13] K.N. Plataniotis, D. Androutsos, and A.N. Venetsanopoulos,
Colour image processing using Fuzzy vector directional l
ters, Proc. IEEE Workshop on Nonlinear Signal and Image
Processing, NSIP95, pp.535538, 1995.
[14] W.K. Pratt, Digital Image Processing, John Wiley & Sons,
New York, NY, 1991.
[15] G. Ramponi, The rational Filter for Image Smoothing,
IEEE Signal Processing Letters, vol.3, March 1996.
[16] P.E. Trahanias, D. Karakos, and A.N. Venetsanopoulos,
Directional processing of color images: Theory and ex
perimental results, IEEE Trans. Image Proc., vol.5, no.6,
pp.868880, June 1996.
Lazhar Khriji received his BS degree in electronics in 1990
from the Faculty of Sciences of Tunis, Tunisia, and his MS in au
tomation and control in 1992 from the Ecole Normale Seperieure
de lEnseignement Technique, Tunisia, and the Ph.D. degree in
1999 from the Faculty of Sciences of Tunis, Tunisia. He is cur
rently Assistant Professor at the Electrical Engineering Depart
ment of the Ecole National dIngenieurs de Monastir, Tunisia.
His research interests include nonlinear signal and image pro
cessing and analysis, multichannel signal processing and adaptive
nonlinear ltering.
1596
IEICE TRANS. INF. & SYST., VOL.E82D, NO.12 DECEMBER 1999
Moncef Gabbouj received his BS
degree in electrical engineering in 1985
from Oklahoma State University, Still
water, and his MS and PhD degrees in
electrical engineering from Purdue Uni
versity, West Lafayette, Indiana, in 1986
and 1989, respectively. Dr. Gabbouj is
currently a professor in the Signal Pro
cessing Laboratory of Tampere University
of Technology, Tampere, Finland. From
1995 to 1998 he was a professor with the
Department of Information Technology of Pori School of Tech
nology and Economics, Pori, and during 1997 and 1998 he was on
sabbatical leave with the Academy of Finland. From 1994 to 1995
he was an associate professor with the Signal Processing Labo
ratory of Tampere University of Technology, Tampere, Finland.
From 1990 to 1993 he held was a senior research scientist with
the Research Institute for Information Technology, Tampere, Fin
land. His research interests include nonlinear signal and image
processing and analysis, contentbased analysis and retrieval and
mathematical morphology. Dr. Gabbouj is the ViceChairman of
the IEEEEURASIP NSIP (Nonlinear Signal and Image Process
ing) Board. He is currently the Technical Committee Chairman
of the EC COST 211quat. He served as associate editor of the
IEEE Transactions on Image Processing, and was guest editor of
the European journal Signal Processing, special issue on nonlin
ear digital signal processing (August 1994). He is the past chair
of the IEEE Circuits and Systems Society, Technical Commit
tee on Digital Signal Processing, and the IEEE SP/CAS Finland
Chapter. He was also the DSP track chair of the 1996 IEEE IS
CAS and the program chair of NORSIG96, and is the technical
program chair of EUSIPCO 2000. Dr. Gabbouj is the Director of
the International University Program in Information Technology
and member of the Council of the Department of Information
Technology at Tampere University of Technology. He is also the
Secretary of the International Advisory Board of Tampere Inter
national Center of Signal Processing, TICSP. He is a member of
Eta Kappa Nu, Phi Kappa Phi, IEEE SP and CAS societies. Dr.
Gabbouj was corecipient of the Myril B. Reed Best Paper Award
from the 32nd Midwest Symposium on Circuits and Systems and
corecipient of the NORSIG 94 Best Paper Award from the 1994
Nordic Signal Processing Symposium. He is coauthor of over
150 publications.