IET Biometrics

Research Article

Face–iris multi-modal biometric system using ISSN 2047-4938

Received on 4th November 2017
Revised 16th February 2018
multi-resolution Log-Gabor filter with spectral Accepted on 20th February 2018
E-First on 19th March 2018
regression kernel discriminant analysis doi: 10.1049/iet-bmt.2017.0251
www.ietdl.org

Basma Ammour1 , Toufik Bouden2, Larbi Boubchir3

1Department of Electronics, University of Mohammed Seddik Benyahia, BP 98, Jijel, Algeria
2Department of Automatic, University of Mohammed Seddik Benyahia, BP 98, Jijel, Algeria
3LIASD Research Lab, University of Paris 8, 2 rue Liberté, 93526 Saint-Denis, France

E-mail: basma_ammour@yahoo.com

Abstract: A multi-modal biometric system is used to verify or identify a person by exploiting information of more than one
biometric modality. It combines the strengths of the unimodal biometric system to solve their limitations. This study proposes
schemes of multi-modal biometric system based on texture information extracted from face and two iris (left and right) using
hybrid level of fusion. Feature extraction is the key step to get a robust recognition system. Multi-resolution two-dimensional
Log-Gabor filter combined with spectral regression kernel discriminant analysis is exploited to extract features from both face
and iris modalities. These features are used in the fusion and the classification process. The proposed schemes were tested
using CASIA Iris Distance database in the verification mode. The experimental results show that the proposed multi-modal
biometric system yields attractive performances of up to 0.24% in terms of equal error rate and outperforms the recent similar
existing state-of-the-art methods.

1 Introduction
Biometrics technology is based on different automated methods
which measuring and examining the physiological or behavioural
characteristics of individuals in verification or identification mode.
These features point extracted from captured image should be
unique and invariant over time. It is stored in a vector called
template. This template compared with the templates stored in the
biometric database [1]. Many physiological (such as fingers, hands,
irises, faces, ears, and voices) and behavioural (such as gaits,
odours, feet, and signatures) body parts have been used as
modalities for the biometric systems [2]. These systems are used in
different places such as airports, buildings, military security
checkpoints, ATM machines, and are also being used to implement
national identity [3].
Conventional unimodal biometric systems use single biometric
modality in the recognition process. They suffer from different
problems such as lack of uniqueness, restricted degrees of freedom,
non-universality, intra-class variation, noisy data, vulnerable to
spoofing, and unacceptable error recognition rates. To overcome
these limitations, the solution is the use of the multi-modal
biometric system. The multi-modal biometric system is obtained by
fusion of extracted information from multiple biometric modalities
[3, 4]. Information fusion in the biometric system can occur in four
levels: sensor-level, feature-level, score-level, and decision-level
[5, 6]. The multi-modal biometric system is constructed according
to the choice of the biometric modalities and the level of fusion. In
[7], they combined the face, fingerprint, and hand geometry
modalities. In [8], they combined the iris and the face modalities.
In this paper, face and iris modalities are selected and used for the
construction of a multi-modal biometric system.
The face modality is the most natural way in the identification
of a person, whereas iris recognition is one of the most accurate
biometrics. There is the advantage of using one sensor for
capturing the iris and face, where the iris is extracted from the
captured face image. Furthermore, merging the face and iris can be
achieved the high universality [9].
This paper proposes multi-modal biometric schemes based on
face and iris modalities. Multi-resolution 2D Log-Gabor filter is
exploited to extract features. Spectral regression kernel
discriminant analysis (SRKDA) is then applied on the extracted
feature set in order to reduce its high dimensionality. The face and
iris are combined by using hybrid level of information fusion
(score-level fusion, feature-level fusion, and decision-level fusion
at the same time) for constructing a high reliable biometric system
with best performance. The proposed schemes are evaluated on the
CASIA Iris Distance database in verification mode. This proposed
multi-modal biometric system yields an attractive performance
compared to the unimodal biometric system and outperforms some
recent state-of-the-art systems.
The rest of this paper is organised as follows. Section 2 presents
the related works on the multi-modal biometric systems based on
the face and iris modalities. The proposed face–iris multi-modal
biometric system is described in Section 3, while the experimental
results carried out to validate this system are reported in Section 4.
Finally, a conclusion is drawn in Section 5.
2 Related works
Several multi-modal biometric systems are proposed over past few
decades. Each proposed system used specific fusion level, feature
extraction method, and matching techniques among them: in [10],
Rattani and Tistarelli proposed to fuse the face and iris modalities
by extracted features with the scale-invariant feature transform
(SIFT) followed by a concatenation operation of feature vectors.
After that, Euclidean distance is used for the matching process.
They evaluated their approach on chimeric database constructed
from CASIA iris V 3 database for iris and the Equinox database for
the face. They achieved the best performance with equal error rate
(EER) = 0.04%. Son and Lee in [11] proposed to extract features
with multi-level two-dimensional (2D) Daubechies wavelet
transform for two modalities, in which the mean and standard
deviation are calculated in each subregion. The feature vectors are
reduced using direct linear discriminant analysis (LDA) and then
concatenated. Experimental results carried out on chimeric
database combined (ORL and IIS face database) for the face and
eyes images were acquired with CCD camera for iris. They
reported recognition rate of 99.12 and 99.7% in (ORL face and iris
database) and (IIS face and iris database), respectively. Huo et al.
in [12] developed face–iris multi-modal system based on feature-
level fusion. They used 2D Gabor filter bank for features
extraction, these features are transformed by histogram statistics

IET Biom., 2018, Vol. 7 Iss. 5, pp. 482-489 482

© The Institution of Engineering and Technology 2018

Fig. 1  Diagram of the proposed multi-modal biometric system
Fig. 2  Face image pre-processing
(a) Face image, (b) Histogram equalisation, (c) Image resizing
into an energy-orientation variance histogram. The feature-level
fusion is performed by principal components analysis (PCA) and
support vector machine. In their experiments on (ORL face and
CASIA V1 iris database) and (PIE-illum face and CASIA V4-
Lamp iris database), they report the best performance with
minimum total error rate of 0% in the two databases. Zhang et al.
in [13] extracted features from the face and iris modalities by PCA
and Daugman's algorithm, respectively. Score-level fusion
performed using both sum and product rules, while the score
normalisation is performed using min–max rule. They conducted
experiments on near infrared (NIR) images database, and they
reported the best performance with genuine acceptance rate (GAR)  Biometric system can be divided into three main modules, pre-
= 99.750% at false acceptance rate (FAR) = 0.010%. Wang et al. in
[14] proposed face–iris multi-modal biometric system based on the
score-level fusion with weighted sum rule, using Fisher
discriminant analysis and neural network with radial basis function
for the classification process. On the other hand, features are
extracted as in [13]. Experimental results performed on chimeric
database constructed from NLPR database for iris (good quality)
and ORL, MIT, Yale databases for the face. They reported good
performance with EER of 0%. Zhang et al. in [15] presented a new
fusion technique of face and iris samples. It used canonical
correlation analysis to perform a mapping between the face and iris
data sets for low-quality images. Sum rule level fusion and min–
max normalisation are used. Experiments performed on images
captured with NIR light. They achieved the best performance on
low-quality image database. Morizet and Gilles in [16] proposed
the face–iris multi-modal biometric system, based on Eigenface
algorithm and Daugman's approach by using three-level wavelet
packets, respectively. The score normalisation of iris and face are
modelled by Gaussian distributions and are normalised using Z-
score. Then the two scores are fused using weighted sum rule. The
proposed system evaluated on a chimeric database (Face FERET
database and CASIA Iris V3 database). They reported promising
results with EER of 0.010%. Eskandari and Toygar in [3] extracted
features by local binary patterns (LBP) and Daugman's algorithm
from the face and iris, respectively. After that, they used the
particle swarm optimisation (PSO) method to select features, and
combined the two modalities by weighted sum rule in score-level
fusion. Chimeric databases are used to measure the performance of
IET Biom., 2018, Vol. 7 Iss. 5, pp. 482-489
© The Institution of Engineering and Technology 2018
this proposed system on (FERET face and UBIRIS iris database)
and (BANCA face and UBIRIS iris database). The proposed
system reported good performance with EER of 2.5 and 0.5% for
(FERET face and UBIRIS iris database) and (BANCA face and
UBIRIS iris database), respectively. In [17], the authors proposed
face–iris multi-modal biometric system based on score-level fusion
and feature-level fusion. Iris features are extracted with 1D Log-
Gabor filter, while facial features are extracted with local and
global features like PCA and LBP. Feature selection is performed
with PSO and backtracking search algorithm (BSA). Experiments
are performed on CASIA Iris Distance database, Print Attack face
database, Replay Attack face database, and IIIT-Delhi Contact
Lens iris database. They achieved best recognition rates. In [18],
Khiari-Hili et al. combined the face and iris in score-level fusion
by employed occlusion-based metric as quality indicator. Iris
biometric system is based on Daugman's algorithm, while face
biometric system is implemented with a bio-inspired approach
combined difference of Gaussians (DoG) filtering with uniform
LBP (LBPU2). The performance of the proposed system examined
on IV2 multi-modal database. They reported EER of 0.30 and
1.24% in controlled environments and uncontrolled illumination,
respectively. Miao et al. in [19] extracted features with Gabor filter
and LBP from the face, while the iris features are extracted with
ordinary filters (OMs), both face and iris modalities are combined
with bin-based classifier at score-level fusion. Experiments
examined on CASIA Iris Distance database. The proposed system
achieved the best performance with EER of 0.39%. Eskandari and
Sharifi in [20] extracted features by applying 1D Log-Gabor filter
for each modality. The authors proposed different schemes, such as
score-level fusion, feature-level fusion, and hybrid-level fusion.
Experiments performed on CASIA Iris Distance database. This
proposed system reported EER of 0.27% and achieved at FAR = 
0.01%, the GAR = 98.93%. In [5], face and iris modalities are
fused based on feature-level fusion, score-level fusion, and
decision-level fusion. In this proposed system, facial features are
extracted with 1D Log-Gabor filter and high-dimensional LBP
(HD-LBP), while iris features are only extracted with 1D Log-
Gabor filter. They reported on CASIA Iris Distance database the
best performance with GAR = 93.91% at FAR = 0.01%. The next
section describes our proposed face–iris multi-modal biometric
system.
3 Proposed multi-modal biometric system
processing, feature extraction, and matching module. In this work,
a multi-modal biometric system is proposed, and it is based on the
face and iris modalities as illustrated in Fig. 1.
3.1 Pre-processing
The image pre-processing step aims to process the face and iris
images in order to enhance their quality and also to extract the
region of interests (ROIs). In this work, face images from CASIA
Iris Distance database are already detected. In order to obtain the
ROI of the face, the face image is cropped using the center
positions of the left and right eyes which are detected by Viola and
Jones algorithm. This algorithm is implemented on Matlab and
becomes famous due to its available open-source implementation.
The algorithm has four stages: Haar feature selection, creating an
integral image, Adaboost training, and cascading classifiers [21].
After face detection step, histogram equalisation is performed, it
usually increases the global contrast of the images, especially when
the usable data of the image is represented by close contrast values.
Through this adjustment, the intensities can be better distributed on
the histogram. Fig. 2 shows an example of the pre-processing
applied on a face image.
For iris images, eyes are detected using Viola Jones algorithm.
It is based on the principle that they are darker than other part in
the face image. It searches small patches which are approximately
as large as an eye and are darker than their neighbourhoods [21].
After eyes are detected, the left and right eyes are separated and iris
segmentation is performed by Daugman's algorithm. It used Hough
483
transform and Canny edge detector to determinate the radius and
the centre of the iris and the pupil of each eye image. The Hough
transform is a pattern recognition technique developed in 1962 by
Paul Hough. This technique allows us to recognise lines (straight),
circles, or any form present in an image. The iris region can be
approximated by two circles, one for the iris/sclerotic boundary
and another within the first to the iris/pupil boundary. After
extracting the contours and determination, the centre of the iris and
the pupil of the eye image, both eyelids (upper and lower), are
isolated by applying the radon transform on the horizontal contours
[22, 23]. Fig. 3 shows different steps constituting iris image pre-
processing: iris detection, segmentation, and normalisation.
3.2 Feature extraction and reduction
3.2.1 Multi-resolution 2D Log-Gabor filter: Gabor filter has been
used in several applications such as image enhancement, contour
detection, feature extraction, and image denoising in pattern
recognition. The input image must be transformed into a feature
representation that is easier for a classification process. Features
formed from the response of Gabor filters may form a good set of
features because it can locally represent frequency information.
Limitations of Gabor filter are the presence of DC component and
bandwidth limited to one octave. This leads to use Log-Gabor
filter. One-dimensional Log-Gabor filter has not DC component
and allows using large bandwidth but only in one dimension. Its
function has the frequency response given by this equation:
− log( f / f 0) 2
G( f ) = exp (1)
2 log(σ/ f 0) 2
where f 0 and σ are the parameters of the filter. f 0 is the centre
frequency of the filter, σ affects the bandwidth of the filter. It is
useful to maintain the same shape while the frequency parameter is
varied. To do this, the ratio σ/ f 0 should remain constant. Two-
dimensional Log-Gabor filter is used to capture two-dimensional
characteristics patterns. Owing to its added dimension, the filter is
not only designed for a particular frequency, but also is designed
for a particular orientation. The orientation component is a
Gaussian distance function according to the angle in polar
coordinates. This filter is defined by the following equation:
− log( f / f 0) 2 −(θ − θ0)2
G( f , θ) = exp 2 exp (2)
2 log(σ f / f 0) 2σθ2
where f 0 is the centre frequency, σ f the width parameter for the
frequency, θ0 the centre orientation, and σθ the width parameter of
the orientation. This filter is applied to the image by using the
convolution operation between the image and the filter. Multi-
resolution 2D Log-Gabor filter G( f s, θ0) is 2D Log-Gabor filter
used in different scales (s) and orientations (o) [24, 25].
One-dimensional Log-Gabor filter is used for features
extraction of both face and iris set in [17]. In this work, 2D multi-
resolution Log-Gabor filter is used to select the best parameters. It
is the first use of 2D Log-Gabor filter for both face and iris features
extraction in multi-modal biometric system.
3.2.2 Spectral regression kernel discriminant analysis: LDA
is used to find a projection matrix, which is optimised to separate
different classes of objects. When the features are non-linear
distributed, kernel discriminant analysis (KDA) can give better
performances than LDA. KDA introduces a kernel function, which
corresponds to the non-linear mapping. It needs to compute an
eigen decomposition of the kernel matrix. It is computationally
very expensive when the features training data set is large. To
avoid this problem, one uses SRKDA, which needs only to solve a
set of regularised regression [26]. SRKDA that is proposed by Cai
et al. [26] is a powerful technique for dimensionality reduction for
multi-resolution 2D Log-Gabor features in this work. The SRKDA
algorithm is described in [26].
484
3.3 Matching
The template extracted by multi-resolution 2D Log-Gabor filter
combined with SRKDA in the training process is compared by the
test template generated with the same feature extraction method.
Different distances can be used such as Manhattan, Mahalanobis,
chi-square distance etc. In our study, the Euclidean distance has
been considered.
3.4 Fusion process
In this work, face, left iris, and right iris modalities are combined to
construct the multi-modal biometric system. Information fusion
can be performed through different fusion schemes: feature-level
fusion, score-level fusion, and hybrid-level fusion. The goal of this
study is to choose the best fusion scheme. The combination of
features from different modalities is realised by concatenating the
feature vectors [9, 10]. In score-level fusion, different techniques
are used such as sum rule, max rule, min rule, weighted sum rule of
scores given by different matchers. The quantity nim represents the
normalised score for matcher m (m = 1, 2, …, M where M is the
number of matchers) applied to user i (i = 1, 2, …, I, where I is the
number of individuals in the database). The fused score for user i is
denoted as f i [25] and given by:
• Sum rule:
M
fi = ∑ nim, ∀i (3)
m=1
• Maximum rule (max rule):
f i = max (ni1, ni2, …, niM ), ∀i (4)
• Minimum rule (min rule):
f i = min (ni1, ni2, …, niM ), ∀i (5)
• Weighted sum rule fusion:
f i = w1ni1 + w2ni2 + w3ni3 + ⋯ + wiM (6)
From the equation, wi is the weight varied over the range 0–1.
In the case of multi-modal system based on two modalities, the
equation of fusion is given by:
f i = w1ni1 + w2ni2 (7)
where w1 + w2 = 1.
In the proposed multi-modal biometric system, the weight of
iris is greater than the face because the iris is more accurate than
the face [8]. This proposed system uses feature-level fusion of face
with left iris. After that, it uses score-level fusion of face, and left
and right iris. At the last, the OR rule is used in the decision-level
fusion. The proposed fusion process for multi-modal biometric
system is shown in Fig. 4.
4 Experiments
In this work, CASIA Iris Distance database is used to evaluate the
performance of the proposed face–iris multi-modal biometric
system. It is including clearly both dual-eye iris and face pattern.
This database contains 142 subjects and a total number of 2567
images. These face images were acquired at-a-distance of ∼3 m
from the camera [28]. Face images were captured by a high-
resolution infrared camera of 2352 × 1728 pixels. Fig. 5 shows
some sample images from CASIA Iris Distance database.
Recall the typology of errors and performance measures for
biometric system. FAR and false rejection rate (FRR) depend on
the threshold used in the decision-making process.
When the system operates in authentication mode, it uses what
is called a receiver operating characteristic (ROC) curve. This
curve shows the FRR according to FAR. FRR is the genuine
acceptance rate (GAR) in which GAR = 1 − FRR. Biometric
IET Biom., 2018, Vol. 7 Iss. 5, pp. 482-489
© The Institution of Engineering and Technology 2018
Fig. 3  Iris image pre-processing
(a) Eyes detection, (b) Segmentation, (c) Isolate both eyelids upper and lower, (d) Normalisation

Fig. 4  Block diagram of the proposed face–iris multi-modal biometric system based on hybrid level fusion

Fig. 5  Some images from CASIA Iris Distance database

Table 1 Performance evaluation of face biometric system calculate the similarity scores between these images with the
s o σ/ f 0 EER, % Thr FAR, % GAR, % training images in the database.
3 4 0.65 8.96 0.101 0.82 82.25
Intra-class variation calculates the similarities between the
testing images with training images of individuals in the same
3 4 0.85 8.27 0.111 0.77 82.25
class. In each class, 45 tests are calculated which give 45 × 90 in
4 5 0.65 8.38 0.113 0.84 82.50 total. So, 4050 intra-class comparisons are made. Inter-class
4 5 0.85 6.86 0.118 0.71 84.00 variation calculates the similarities between the testing images of
5 8 0.65 5.54 0.126 0.62 85.75 individual in each class with the training images of individuals in
5 8 0.85 3.43 0.132 0.73 90.75 other classes. So, in total (89 + 88 + 97 + ⋯ + 1) = 4005 inter-
The best results are written in bold in Tables 1–4
class comparisons are made.

4.1.1 Face: Experimental results on CASIA Iris Distance database

systems are often compared on a particular point on the curve of
errors. The point of EER which corresponds to the FAR = FRR.
Moreover, this curve tends to follow more the shape of the mark,
more the system is efficient and accurate [8, 29].
4.1 Experiments on unimodal biometric system
In the unimodal biometric system, 90 subjects are considered
where each subject has a set of images including the whole face
and dual-eye iris patterns [19]. We have selected ten images for
each subject in the database. Five images are used for training,
while the rest of the images are used for the test. In the evaluation
of verification process, for each test image, the system will
have realised to evaluate the performance of the face unimodal
recognition system. After face pre-processing step, feature
extraction is performed using multi-resolution 2D Log-Gabor filter
algorithm with SRKDA. The face images used in this experiment
have a size of 1728 × 2352 pixels, and they are redimensioned to
173 × 210 pixels. Each face image convolved with 2D Log-Gabor
filter after selecting parameters of scales (s), orientations (o), and
ratio σ/ f 0. When s = 5 and o = 8, the dimension of the obtained
feature vectors is 40 × 173 × 210 = 1, 453, 200. This vector is very
large. Since the high correlation between adjacent pixels in an
image, the information redundancy of Gaborette face images is
reduced using down-sampling by factor of four in lines and
columns, respectively. Which means the size of feature vector is

IET Biom., 2018, Vol. 7 Iss. 5, pp. 482-489 485

© The Institution of Engineering and Technology 2018
Fig. 6  ROC curve of the face unimodal biometric system Fig. 8  ROC curve of the right iris unimodal biometric system

Table 2 Performance evaluation of iris biometric system Table 3 Performance evaluation of face–iris multi-modal
s o σ/ f 0 EER, % Thr FAR, % GAR, % biometric system using feature-level fusion
left iris 4 5 0.65 4.02 0.291 0.33 94.50 Feature-level fusion EER, % Thr FAR, % GAR, %
4 5 0.85 3.71 0.313 0.32 95.32 left iris and face 0.85 0.494 0.34 98.25
5 8 0.65 2.16 0.311 0.32 96.00 right iris and face 1.56 0.413 0.59 97.00
5 8 0.85 1.43 0.331 0.60 97.50
right iris 4 5 0.65 4.40 0.275 0.70 88.25
Fig. 6 shows the ROC curve of the face unimodal biometric
4 5 0.85 3.95 0.298 0.60 89.75
system used the best parameters of multi-resolution 2D Log-Gabor
5 8 0.65 2.34 0.309 0.47 96.00 filter.
5 8 0.85 2.19 0.301 0.42 90.00
4.1.2 Iris: The evaluation of the unimodal biometric system using
iris is performed on the left iris and right iris extracted from the
face image of CASIA Iris Distance database. Irises are localised,
segmented, and normalised as shown in Fig. 3 based on Daugman's
algorithm. The image size of the normalised iris is 24 × 240 pixels.
In this experiment, features are extracted using multi-resolution
2D Log-Gabor filter, after choosing parameters of scale (s),
orientation (o), and ratio. Two-dimensional Log-Gabor filters are
convolved with iris image to obtain a vector of size 40 × 24 × 240 
= 230,400. This vector is reduced using SRKDA to C − 1 = 89
pixels. For iris evaluation results, the following parameters of 2D
log Gabor filter have been chosen in this experiment: (s = 4, o = 5,
σ/ f 0 = 0.65), (s = 4, o = 5, σ/ f 0 = 0.85), (s = 5, o = 8, σ/ f 0 = 0.65),
(s = 5, o = 8, σ/ f 0 = 0.85). The best EER performance is obtained
when we used 40 2D Log-Gabor filter with (s = 5, o = 8, and
σ/ f 0 = 0.85) for the left and right iris system. For the left iris, it
gives results with EER = 1.43% for threshold Thr = 0.331 and
shows that for an FAR = 0.60%, the GAR = 97.50%. For the right
iris, it gives results with EER = 2.19% for threshold Thr = 0.301
and shows that for an FAR = 0.42%, the GAR = 96.00%.
Fig. 7  ROC curve of the left iris unimodal biometric system Table 2 presents the performance evaluation of iris system using
different parameters of multi-resolution 2D Log-Gabor filter (scale,
1,453,200/(4 × 4) = 90,825. This vector is normalised to zero mean orientation, and ratio σ/ f 0). Figs. 7 and 8 show the ROC curves of
and unit variance, then reduced using SRKDA technique. The the left iris and right iris unimodal biometric systems, respectively,
kernel SRKDA's function is a Gaussian function and its standard using the best parameters of multi-resolution 2D Log-Gabor filter.
deviation is 0.02. The size of obtained feature vector is C − 1 = 89,
where C = 90 is the number of classes in the database. 4.2 Experiments on multi-modal biometric system
Choosing the best parameters of 2D Log-Gabor filter bank such
as scale (s), orientation (o), and ratio σ/ f 0 that gives best In the proposed face–iris multi-modal biometric system, the
performance of the system. In this study, we used (s = 3, o = 4, features are extracted from the face and two irises images using the
σ/ f 0 = 0.65), (s = 3, o = 4, σ/ f 0 = 0.85), (s = 4, o = 5, σ/ f 0 = 0.65), best parameters for 2D Log-Gabor filter bank chosen in the
(s = 4, o = 5, σ/ f 0 = 0.85), (s = 5, o = 8, σ/ f 0 = 0.65), and (s = 5, o  previous subsections and a hybrid fusion. Feature-level fusion and
= 8, σ/ f 0 = 0.85). score-level fusion of the face and two irises are studding before
construction of the proposed multi-modal biometric system, in
Table 1 shows the performance evaluation of the face unimodal order to get best evaluation results of the biometric system based
biometric system using different parameters of multi-resolution 2D on feature-level fusion, and on score-level fusion. Feature-level
Log-Gabor filter. From the table, the best EER result is 3.43% for a fusion, in which feature vectors are concatenated. The left iris
threshold of 0.132, the FAR = 0.73% at GAR = 90.75%. It is combined with face gives results of EER = 0.85% with threshold
obtained for (s = 5, o = 8, and ratio σ/ f 0 = 0.85). Thr = 0.494 and at FAR = 0.34%, the GAR = 98.25%. The right iris
combined with face gives results of EER = 1.56% with threshold

486 IET Biom., 2018, Vol. 7 Iss. 5, pp. 482-489

© The Institution of Engineering and Technology 2018
Fig. 9  ROC curves of unimodal and proposed multi-modal biometric
systems using feature-level fusion of the left iris–face Fig. 11  ROC curves of unimodal systems and proposed face–iris multi-
modal biometric system
Table 4 Performance evaluation of multi-modal biometric
system based on the face and two irises using score-level Table 6 Comparison of the performance of unimodal and
fusion multi-modal biometric systems
Score-level fusion EER, % Thr FAR, % GAR, % Biometric system EER, % FAR, % GAR, %
min rule 0.51 0.297 0.128 97.25 face 3.43 0.73 90.75
max rule 1.46 0.331 0.49 97.00 left iris 1.43 0.60 97.50
sum rule 0.32 0.326 0.12 98.75 right iris 2.19 0.42 96.00
weighted sum rule 1.49 0.310 0.17 98.75 left iris and face feature fusion 0.85 0.34 98.25
score-level fusion using sum rule 3.32 0.12 98.75
proposed system 0.24 0.06 99.50

multi-modal biometric systems using score-level fusion with sum

rule.
The proposed multi-modal biometric system used features
concatenation of the left iris and face, sum rule for score-level
fusion of face, left iris, and right iris. On the other hand, OR rule is
used for decision-level fusion of obtained decision from the left
iris, feature-level fusion of face and left iris, and score-level fusion
of the left iris, right iris, and face as shown in Fig. 4. It achieves the
best EER result (EER = 0.24% with threshold of 0.493, at FAR = 
0.06%, the GAR = 99.5%) as shown in Table 5. By analysing the
ROC curves shown in Fig. 11, we notice that this proposed multi-
modal biometric system outperforms the unimodal biometric
systems.

4.3 Comparison of unimodal and multi-modal biometric

Fig. 10  ROC curves of unimodal and proposed multi-modal biometric
using score-level fusion with sum rule
Table 5 Performance evaluation of the proposed face–iris
multi-modal biometric system
EER, % Thr FAR, % GAR, %
proposed method 0.24 0.493 0.06 99.5
Thr = 0.413 and at FAR = 0.59%, the GAR = 97.00%. We can
notice that the multi-modal biometric system based on the left iris
and face provides the best result compared with multi-modal
biometric system based on the right iris and face. Table 3 presents
the evaluation results of multi-modal system using feature-level
fusion. Fig. 9 shows the ROC curves of unimodal and multi-modal
biometric system based on left iris and face modalities.
Score-level fusion of the left iris, right iris, and face is achieved
using sum rule, min rule, max rule, and weighted sum rule.
According to Table 4 of evaluation results, sum rule gives a better
result with EER = 0.32% for Thr = 0.326 and at FAR = 0.12% the
GAR = 98.75%. Fig. 10 shows the ROC curves of unimodal and
IET Biom., 2018, Vol. 7 Iss. 5, pp. 482-489
© The Institution of Engineering and Technology 2018
systems
In this section, comparison study is made between unimodal and
multi-modal biometric systems in terms of obtained performance
(EER, FAR, FRR). From Table 6, it is clear that the multi-modal
biometric systems based on feature-level fusion, score-level fusion,
and the proposed hybrid-level fusion provides the best performance
of EER equal to 0.85, 0.32, and 0.24%, respectively, outperform
the unimodal biometric systems using only the face modality, left
iris, and right iris which achieve the EER of 3.43, 1.43, and 2.19%,
respectively. The obtained results of multi-modal biometric
systems based on feature-level fusion (at FAR = 0.34%, the GAR = 
98.5%), score-level fusion (at FAR = 0.12%, the GAR = 98.75%),
and the proposed hybrid-level fusion (at FAR = 0.24%, the GAR = 
99.5%) achieve best results compared to unimodal biometric
system based on the face (at FAR = 0.73%, the GAR = 90.75%),
left iris (at FAR = 0.60%, the GAR = 97.50%), and based on the
right iris (at FAR = 0.42%, the GAR = 96.00%) in terms of FAR
and GAR. In the other hand, the proposed multi-modal biometric
system provides higher performance against all unimodal biometric
systems and multi-modal biometric systems based on feature-level
fusion and score-level fusion.
Table 7 presents a comparative study of the proposed multi-
modal biometric system with some similar and recent existing
487
Table 7 Comparison of the proposed face–iris multi-modal biometric method with some recent and similar state-of-the-art
methods
Authors Feature extraction Feature Fusion Database and number of considered Evaluation results
reduction level images (best EER)
Wang et al. [14] face: PCA and FDA / score-level NLPR for iris good quality. ORL, MIT, and Yale EER = 0%
(2003) fusion for face. 90 subjects with 5 face and 5 iris
images for each subject
iris: 2D Gabor — — — —
Rattani and Tistarelli SIFT for face and iris spacial feature-level CASIA V3 for iris, Equinox for face, 57 EER ≃ 0.05%
[10] (2009) sampling fusion subjects with 10 samples for each subject. 1
sample for training and 9 samples for test
Eskandari and face: LBP PSO score-level (FERET and UBIRIS): 170 subjects with 4 EER = 2.5%,EER = 
Toygar [3] (2013) fusion samples for each subject. 2 samples for 0.5%, in (FERET and
training and 2 samples for test. (BANCA and UBIRIS), (BANCA and
UBIRIS): 40 subjects with 8 samples for each UBIRIS,) respectively
subject. 3 samples for training and 5 samples
for test
iris: 1D Log-Gabor — — — —
Eskandari and iris: 1D Log-Gabor PSO, BSA feature-level CASIA-Iris-Distance database. 90 subjects PSO: EER = 3.78%,
Toygar [17] (2015) face: five local and fusion and with 10 samples for each subject. 5 samples FAR = 0.01% at GAR = 
global features score-level for training and 5 samples for test 94.44%
fusion
Khiari-Hili et al. [18] face: DoG combined / score-level IV 2 multimodal database. 315 subjects in EER = 0.63%, EER = 
(2016) with LBPU2 fusion which 52 subjects used for training and the 0.96% in quality
remainder used for test variation and multi-
session, respectively
iris: Daugman's — — — —
Miao et al. [19] face: Gabor filter and AdaBoost score-level CASIA Iris Distance database. 142 subjects. EER = 0.39%
(2016) LBP algorithm fusion 928 images from the first 50 subjects are
used for training and the rest for test
iris: ordinary filters — — — —
(OMs)
Sharifi and 1D Log-Gabor for face BSA hybrid-level CASIA Iris Distance database. 90 subjects EER = 0.27% ± 0.41,
Eskandari [20] and iris fusion with 10 samples for each subject. 5 samples FAR = 0.01% at GAR = 
(2016) for training and 5 samples for test 98.93% ± 1.11
Eskandari and face: 1D Log-Gabor BSA and hybrid-level CASIA Iris Distance database. 90 subjects FAR = 0.01%. at GAR = 
Sharifi [5] (2017) and HD-LBP. Iris: 1D LDA fusion with 10 samples for each subject. 5 samples 93.91%
log-Gabor for training and 5 samples for test
proposed method multi-resolution 2D SRKDA hybrid-level CASIA Iris Distance database. 90 subjects EER = 0.24%, FAR = 
Log-Gabor for face fusion with 10 samples for each subject. 5 samples 0.06% at GAR = 99.5%
and iris for training and 5 samples for test

face–iris multi-modal biometric systems such as in [3, 5, 9, 13, 17,
19, 20] that is operated in verification mode. Most of the proposed
system in the literature were examined on virtual database such as
in [3, 9, 13]. Recently, there are some proposed systems evaluated
on real database like IV2 multi-modal database and CASIA Iris
Distance database. From the obtained results, it can be noticed that
our proposed multi-modal biometric system achieves best
performance in terms of (FAR and GAR) and has lower EER
compared to some recent proposed works such as in [5, 17, 19, 20],
that operated in verification mode and evaluated on CASIA Iris
Distance database in which used the same number of images in the
evaluation process.
5 Conclusion
A face–iris multi-modal biometric system was proposed in this
paper based on 2D Log-Gabor filter bank features extraction from
the face and two iris images. SRKDA technique reduces the high
dimensionality of the extracted feature vectors. The combination of
face and two iris modalities is achieved in different level of fusion
to investigate the performance improvement of each level. On the
other hand, this paper proposes a design of a multi-modal biometric
system using a hybrid-level fusion (i.e. feature-level fusion, score-
level fusion, and decision-level fusion at the same time), to exploit
the advantages of these fusion levels. Experimental results have
demonstrated that the proposed multi-modal system using the
hybrid fusion achieves an attractive improvement (up to 0.24% in
terms of EER) compared to the unimodal and multi-modal systems
investigated in this work. It also outperforms the state-of-the-art
multi-modal systems considered in this study. Our future work is to
focus on feature extraction by studying other multi-scale
representations capable to further capture the relevant and
discriminative information allowing improving the performance of
the proposed face–iris multi-modal biometric system.
6 References
[1] Ghayoumi, M.: ‘A review of multimodal biometric systems: fusion methods
and their applications’. Proc. Int. Conf. Computer and Information Science,
Las Vegas, NV, USA, July 2015
[2] Sim, H.M., Asmuni, H., Hassan, R., et al.: ‘Multimodal biometrics: weighted
score level fusion based on non-ideal iris and face images’, Expert Syst. Appl.,
2014, 41, (11), pp. 5390–5404
[3] Eskandari, M., Toygar, Ö.: ‘Score level fusion for face–iris multimodal
biometric system’, in Gelenbe, E., Lent, R., (Eds.): ‘Information sciences and
systems’ (Springer International Publishing, Switzerland, 2013), pp. 199–208
[4] Deshmukh, S., Patwardhan, M., Mahajan, A.: ‘Survey on real-time facial
expression recognition techniques’, IET Biometrics, 2016, 5, (3), pp. 155–163
[5] Eskandari, M., Sharifi, O.: ‘Optimum scheme selection for face-iris
biometric’, IET Biometrics, 2017, 6, (5), pp. 334–341
[6] Connaughton, R., Bowyer, K.W., Flynn, P.J.: ‘Fusion of face and iris
biometrics’, in Burge, M., Bowyer, K. (Eds.): ‘Handbook of iris recognition’
(Springer, London, 2013), pp. 219–237
[7] Ross, A., Jain, A.: ‘Information fusion in biometrics’, Pattern Recognit. Lett.,
2003, 24, (13), pp. 2115–2125
[8] Liau, H.F., Isa, D.: ‘Feature selection for support vector machine-based face–
iris multimodal biometric system’, Expert Syst. Appl., 2011, 38, (9), pp.
11105–11111
[9] Eskandari, M., Toygar, Ö.: ‘Fusion of face and iris biometrics using local and
global feature extraction methods’, Signal, Image Video Process., 2014, 8, (6),
pp. 995–1006

488 IET Biom., 2018, Vol. 7 Iss. 5, pp. 482-489

© The Institution of Engineering and Technology 2018
[10] Rattani, A., Tistarelli, M.: ‘Robust multi-modal and multi-unit feature level
fusion of face and iris biometrics’, in Tistarelli, M., Nixon, M.S. (Eds.):
‘Advances in biometrics’ (Springer, Berlin, Heidelberg, 2009), pp. 960–969
[11] Son, B., Lee, Y.: ‘Biometric authentication system using reduced joint feature
vector of iris and face’, in Kanade, T., Jain, A., Ratha, N.K. (Eds.): ‘Audio-
and video-based biometric person authentication’ (Springer, Berlin,
Heidelberg, 2005), pp. 513–522
[12] Huo, G., Liu, Y., Zhu, X., et al.: ‘Face–iris multimodal biometric scheme
based on feature level fusion’, J. Electron. Imaging, 2015, 24, (6), pp.
063020–063020
[13] Zhang, Z., Wang, R., Pan, K., et al.: ‘Fusion of near infrared face and iris
biometrics’, in Lee, S.W., Li, S.Z. (Eds.): ‘Advances in biometrics’ (Springer,
Berlin, Heidelberg, 2007), pp. 172–180
[14] Wang, Y., Tan, T., Jain, A.K.: ‘Combining face and iris biometrics for identity
verification’, in Kittler, J., Nixon, M.S. (Eds.): ‘Audio- and video-based
biometric person authentication’ (Springer, Berlin, Heidelberg, 2003), pp.
805–813
[15] Zhang, X., Sun, Z., Tan, T.: ‘Hierarchical fusion of face and iris for personal
identification’. Proc. Int. Conf. Pattern Recognition, Istanbul, Turkey, August
2010, pp. 217–220
[16] Morizet, N., Gilles, J.: ‘A new adaptive combination approach to score level
fusion for face and iris biometrics combining wavelets and statistical
moments’, in Bebis, G., et al. (Eds.): ‘Advances in visual computing’
(Springer, Berlin, Heidelberg, 2008), pp. 661–671
[17] Eskandari, M., Toygar, Ö.: ‘Selection of optimized features and weights on
face-iris fusion using distance images’, J. Comput. Vis. Image Underst., 2015,
137, pp. 63–75
[18] Khiari-Hili, N., Montagne, C., Lelandais, S., et al.: ‘Quality dependent
multimodal fusion of face and iris biometrics’. Proc. Int. Conf. Image
IET Biom., 2018, Vol. 7 Iss. 5, pp. 482-489
© The Institution of Engineering and Technology 2018
Processing Theory Tools and Applications (IPTA), Oulu, Finland, 2016, pp.
1–6
[19] Miao, D., Zhang, M., Sun, Z., et al.: ‘Bin-based classifier fusion of iris and
face biometrics’, Neurocomputing, 2017, 224, pp. 105–118
[20] Sharifi, O., Eskandari, M.: ‘Optimal face-iris multimodal fusion scheme’,
Symmetry, 2016, 8, (6), p. 48
[21] Murphy, T.M., Broussard, R., Schultz, R., et al.: ‘Face detection with a Viola-
Jones based hybrid network’, IET Biometrics, 2017, 6, (3), pp. 200–210
[22] Wang, Q., Zhu, B., Liu, Y., et al.: ‘Iris-face fusion and security analysis based
on fisher discriminant’, Int. J. Smart Sens. Intell. Syst., 2015, 8, (1), pp. 387–
407
[23] Wang, Z., Wang, E., Wang, S., et al.: ‘Multimodal biometric system using
face-iris fusion feature’, J. Comput., 2011, 6, (5), pp. 931–938
[24] Yingzi, D.: ‘Using 2D Log-Gabor spatial filters for iris recognition’. Proc. Int.
Conf. Biometric Technology for Human Identification, Florida, USA, April
2006
[25] Bounneche, M.D., Boubchir, L., Bouridane, A., et al.: ‘Multi-spectral
palmprint recognition based on oriented multiscale Log-Gabor filters’,
Neurocomputing., 2016, 205, (12), pp. 274–286
[26] Cai, D., He, X., Han, J.: ‘Speed up kernel discriminant analysis’, Int. J. Very
Large Data Bases, 2011, 20, (1), pp. 21–33
[27] Elnasir, S., Shamsuddin, S.M., Farokhi, S.: ‘Accurate palm vein recognition
based on wavelet scattering and spectral regression kernel discriminant
analysis’, J. Electron. Imaging, 2015, 24, (1), pp. 13031–13031
[28] ‘Biometrics ideal test’. 2013. Available at http://biometrics.idealtest.org/
dbDetailForUser.doid=4
[29] Jain, A.K., Duin, R.P.W., Mao, J.: ‘Statistical pattern recognition: a review’,
IEEE Trans. Pattern Anal. Mach. Intell., 2000, 22, (1), pp. 4–37
489