ILLUMINATION-ROBUST FACE RECOGNITION USING RETINA MODELING
Ngoc-Son Vu1,2 , Alice Caplier2
1
Vesalis Sarl, 63000 Clermont-Ferrand, France
2
GIPSA-lab, BP 46 - 38402 St Martin d’Heres - France
ngoc-son.vu, alice.caplier@gipsa-lab.grenoble-inp.fr
ABSTRACT
Illumination variations that might occur on face images de-
grade the performance of face recognition systems. In this pa-
per, we propose a novel method of illumination normalization
based on retina modeling by combining two adaptive nonlin-
ear functions and a Difference of Gaussians filter. The pro-
posed algorithm is evaluated on the Yale B database and the
Feret illumination database using two face recognition meth-
ods: PCA based and Local Binary Pattern based (LBP). Ex-
perimental results show that the proposed method achieves
very high recognition rates even for the most challenging il-
lumination conditions. Our algorithm has also a low compu-
tational complexity.
Index Terms— Face recognition, illumination invariant,
retinal processing
1. INTRODUCTION
Illumination variations that occur on face images degrade the
performance of face recognition systems dramatically. Meth-
ods developed in order to overcome the illumination prob-
lem can be divided into three categories: illumination invari-
ant feature extraction, illumination modeling and illumination
variation removal.
The first approach seeks illumination invariant features
to represent face images. Examples of such features include
edge maps, image intensity derivation and images convolved
with 2D Gabor-like filters [1, 2]. However, Adini’s empirical
study [3] shows that none of these representations is suffi-
cient by itself to overcome illumination variations because of
changes in the illumination direction. Methods in the second
category use multiple images of each person under various
lighting conditions to learn an appropriate model of illumina-
tion variations [4, 5, 6, 7, 8]. Generally, methods belonging to
this category can achieve good recognition results. However,
they require many images captured under different lighting
conditions for each subject. Such methods are not adaptable
to many pratical applications such as video surveillance. The
third approach transforms image to canonical form in which
the illumination variations are erased. Examples of such tech-
niques are Histogram Equalization (HE), Gamma Correction
978-1-4244-5654-3/09/$26.00 ©2009 IEEE 3289
(GC) or recent methods based on retinex theory.
In retinex theory, an image I(x, y) is modeled as the prod-
uct of the reflectance R and the illumination L. The prob-
lem of obtaining R from an input image I can be solved by
estimating L. Several methods have been presented to esti-
mate L, such as Single Scale Retinex (SSR) [9], Multi Scale
Retinex (MSR) [9] and self-quotient image (SQI) [10]. How-
ever, these methods still can not exactly estimate L, so large
illumination variations are not completely removed.
In this paper, instead of estimating L from I, we propose a
new method of illumination normalization which mimics the
performance of the human retina. The retina pre filtering has
been employed in many applications, such as moving con-
tours enhancement [11]. This paper presents an improvement
of the retina filter to normalize illumination. The motivation
is that the natural ability of human retina enables us to see ob-
jects in different lighting conditions. The proposed algorithm
combines two adaptive nonlinear functions and a Difference
of Gaussians filter. This can be related to the performance
of two layers of the retina: the photoreceptors and the outer
plexiform layer.
The rest of the paper is structured as follows. In Section
2, the model of retinal processing is briefly presented. In Sec-
tion 3, the proposed method is described in detailv. Experi-
mental results are presented in Section 4. Finally a conclusion
is given in Section 5.
2. MODEL OF RETINAL PROCESSING
Basically, the retina is made of three layers: the photore-
ceptors layer with cones and rods; the outer plexiform layer
(OPL) with horizontal, bipolar and amacrine cells; and the in-
ner plexiform layer (IPL) with ganglion cells. The goal here
is not to precisely model the dynamics of retinal processing,
such as is done, for example, in [12]. We aim to identify
which processing acts on the retinal signal for illumination
normalization. This section demonstrates that bipolar cells
remove not only illumination variations and noise but also en-
hance the image edges.
ICIP 2009
2.1. Light adaptation filter 3. PROPOSED ALGORITHM

Rods and cones have quite different properties: rods have the
ability to see at night, under conditions of very low illumina-
tion; cones have the ability to deal with bright signals. But
both photoreceptors are sensible to light variations and play a
crucial rule as light adaptation filters. To exploit and mimic
this property, an adaptive nonlinear function could be applied
on the input signal. In [12], the nonlinear function takes in-
spiration from the Naka-Rushton equation:
X ficient light adaptation filter. Thus, we apply two consecutive
Y = (1)
X + X0
where X represents the input light intensity, X0 is the adap-
tation factor, and Y is the adapted signal. Figure 1 illustrates
As mentioned above, a model with a nonlinear operation and
a DoG filter can be used for variation illumination removal.
In our model, two consecutive nonlinear operations are used
for a more efficient light adaptation filter and a truncation is
used to enhance the global image contrast.
3.1. Two adaptive nonlinear operations
As verified in [14], duplex nonlinear operations act as an ef-
adaptive nonlinear functions in this step.
The adaptation factor (X0 in equation (1)) of the first non-
linear function is computed for each pixel by performing a
low pass filter on the input image [14]:

Iin
F1 (p) = Iin (p) ∗ G1 + (2)
2
where p is the current pixel; F1 (p) is the adaptation factor at
pixel p; Iin is the intensity of the input image; ∗ denotes the
convolution operation; Iin is the mean value of the input; and
G1 is a 2D Gaussian low pass filter with standard deviation
σ1 :
2 2
1 − x 2σ+y2
G1 (x, y) = e 1 (3)
2πσ12

Fig. 1. Naka-Rushton function with different adaptation fac- The input image is then processed according to the Naka-
tors X0. Rushton equation (1) using the adaptation factor F1 :
Iin (p)
the Naka-Rushton function for different values of X0. If X0 Ila1 (p) = (Iin (max) + F1 (p)) (4)
Iin (p) + F1 (p)
is small, the output has increased sensitivity. If X0 is large,
there is not much change in sensitivity. In [12], X0 varies The term Iin (max) + F1 (p) is a normalization factor where
for each pixel. It is the average light intensity in the neigh- Iin (max) is the maximal value of the image intensity.
borhood of the current pixel. The result of Naka-Rushton The second nonlinear function works similarly, the light
function in an image is the enhancement of the local dynamic adapted image Ila2 is obtained by:
range in dark regions while bright regions remain almost un-
Ila1 (p)
changed. Ila2 (p) = (Ila1 (max) + F2 (p)) (5)
Ila1 (p) + F2 (p)
with
2.2. OPL filter 2 2
Ila1 1 − x 2σ+y2
F2 (p) = Ila1 (p) ∗ G2 + , G2 (x, y) = e 2 (6)
Photoreceptors perform not only as a light adaptation filter but 2 2πσ22
also as a low pass filter. Horizontal cells perform the second
low pass filter. In OPL, bipolar cells calculate the difference An avantage of this light adaptation filter is that the im-
between photoreceptor and horizontal cell responses. Then, age Ila2 does not change with different low pass filter sizes
bipolar cells act as a band pass filter: they remove high fre- [14]. By default in this paper, σ1 and σ2 are set to 1 and 3
quency noise and low frequency illumination. respectively.
To model the processes of OPL, two Gaussian low pass
filters with different standard deviations corresponding to the 3.2. DoG filter
effects of photoreceptors and horizontal cells are used [12]. The image Ila2 is then transmitted to bipolar cells and pro-
Finally, bipolar cells act like a Difference of Gaussians filter cessed by using a Difference of Gaussians (DoG) filter:
(DoG). Note that a DoG filter enhances the image edges [12,
13]. Ibip = DoG ∗ Ila2 (7)

3290
 paper mainly deals with the illumination problem, we only
choose 64 frontal images captured under 64 different lighting
conditions for each of 10 subjects. Images in the database
are divided into 5 subsets according to the angle of the light
source. The 5 subsets are subset 1 (0 ◦ to 12 ◦ ), subset 2 (13 ◦
to 25 ◦ ), subset 3 (26 ◦ to 50 ◦ ), subset 4 (51 ◦ to 77 ◦ ), and
(a) Iin (b) Ila2 (c) IP h (d) IH (e) Ibip (f) Ipp subset 5 (above 78 ◦ ). The effect of the proposed illumination

Fig. 2. Steps of the proposed algorithm.

where DoG is given by:

2 2 2 2
1 − x +y
2 1 − x +y
2
DoG = 2 e 2σP h − 2 e 2σH (8)
2πσP h 2πσH
the terms σP h and σH correspond to the standard deviations Fig. 3. Examples of face images in YaleB database normal-
of the low pass filters modeling photoreceptors and horizontal ized by the proposed method.
cells. In the implementation of this paper, they are set to 0.5
and 4 respectively. normalization method is illustrated in figure 3 and it is clear
A zero-mean normalization is used in the next step to that not only the illumination variations are removed but the
rescale the dynamic range of the image. The substraction of image contours are also enhanced.
the mean μIbip is not necessary because it is near to 0. As our goal here is to evaluate the performance of the
Ibip (p) − μIbip proposed illumination normalization method, a very common
Ibip (p)
Inor (p) = = (9) face recognition algorithm, the Principal Components Anal-
σIbip 2 )
E(Ibip ysis (PCA) based algorithm [16], is used for computing the
recognition rates. Only ideal images with the most neutral
A drawback of the DoG filter is an inherent reduction in light source for all 10 subjects are used as the training set and
overall image contrast [13]. So, in the final step, to enhance all images in turn are used as the test set. The proposed algo-
the image contrast, the extreme values are removed by a trun- rithm is also compared with other common methods.
cation with a threshold T h which is set to 5 in the implemen-
tation of this paper.
Table 1. Recognition rates for different methods on Yale B
max(T h, |Inor (p)|) if Inor (p) ≥ 0
Ipp (p) = { (10) database
−max(T h, |Inor (p)|) otherwise Methods Direction light (subset)
Figure 2 illustrates an example of the effects of the pro- 1&2 3 4 5
posed processing. IP h and IH correspond to the photorecep- Without preprocessing 99.5 65.0 27.9 12.1
tors and horizontal cells reponses respectively. Cone-cast [5] 100 100 100 -
9PL [6] 100 100 97.2 -
Harmonic image [7] 100 99.7 96.9 -
4. EXPERIMENTAL RESULTS
Gradient angle [8] 100 100 98.6 -
Local normalization [18] 100 100 96.4 -
We evaluate the performance of the proposed illumination
Proposed method 100 100 100 100
normalization algorithm for the purpose of face recognition.
The considered databases include the Yale B database [5] and
the Feret illumination database [15]. In the implementation, It is clear from Table 1 that the proposed illumination
all face images are cropped manually. In order to show that normalization method improves the performance of the PCA
the proposed illumination normalization method improves the based algorithm remarkably with excellent recognition rates,
performance of various face recognition algorithms, two algo- even for the most difficult subset. Although the method of
rithms (PCA [16] and LBP [17]) are used. Georghiades et al. [5] gives recognition rates of 100%, it re-
quires seven images for each person to obtain the shape and
albedo of a face, it also requires a high computational effort
4.1. Results on Yale B database
due to a large number of extreme rays that make up the illu-
The Yale B database containing 5760 images taken from 10 mination cones. Lee et al. [6] use nine images captured under
subjects under 576 viewing conditions (9 poses × 64 illumi- different lighting directions for training the model. Chen et
nation conditions) is commonly used to evaluate the perfor- al. [8] use a probability distribution for the image gradient
mance of illumination invariant face recognition. Since this as a function of the surfaces geometry and reflectance. Note

3291
that authors of these methods did not test with subset 5. In
comparison with other methods, our algorithm is more robust
to illumination variations and is of lower complexity. Only
one image per subject is required for the training set and the
techniques used are simple. The most computational calcula-
tion in our algorithm is the convolution operation with Gaus-
sian kernel. Suppose that the size of a normalized image is
m × n. By replacing one 2D Gaussian calculation by two in-
dependent 1D Gaussian ones, the computational complexity
of algorithm is O(mnw) where w is the size of 1D Gaussian
kernel and w = 6σ in this paper. As the method is of low
complexity, it can be applied as a preprocessing technique in
real-time applications such as video surveillance.
4.2. Results on Feret illumination database
In the Feret database, one of the most famous databases for
the evaluation of face recognition algorithms, all frontal face
images are divided into five categories: fa, fb, fc, dup1 and
dup2 (see [15] for details). Fa images and fc images were
taken under different illumination conditions. As we are only
concerned with the illumination problem, 1196 fa images are
used as gallery and 194 fc images are used as test set.
Recently, the Local Binary Pattern (LBP) algorithm is re-
ported to be a good representation for face images which is
invariant to monotone change in lighting conditions. The
LBP based algorithm has also been successfully applied in
face recognition without preprocessing [17, 19]. However, as
Table 2. Recognition rates for different methods on FERET
fc probe sets
Methods fc
Best result of [15] 82
LBP [17] 51
Weighted LBP [17] 79
LGBPHS [19] 97
LBP + proposed preprocessing 98
shown in Table 2, the LBP based algorithm recognition rate
is significantly enhanced when the preprocessing method is
used. The 98% recognition rate obtained is higher than that
of existing variations of LBP method: weighted LBP [17] and
Local Gabor Binary Pattern Histogram Sequence (LGBPHS)
[19]. Note that to achieve the rate of 97%, the LGBPHS
method has to compute 40 images convolved with Gabor fil-
ters which is of higher complexity than using our preprocess-
ing method and the LBP based recognition method.
5. CONCLUSION
In this paper, we present a novel method for removing varia-
tion illumination. The efficiency of the method is estimated
with respect to the performance of different face recogni-
tion algorithms. The proposed model takes inspiration from
3292
the properties of the retina which enable eyes to see objects
in different illumination conditions. The model has princi-
pal components: two nonlinear functions and a Difference of
Gaussians filter. In the experiments, the very high recognition
rates are achieved with different face recognition algorithms
associated with the proposed preprocessing technique.
6. REFERENCES
[1] X. Yongsheng Gao and Maylor K.H. Leung, “Face recognition using
line edge map,” IEEE Trans. PAMI, vol. 24, pp. 764–779, 2002.
[2] C. Liu and H. Wechsler, “Gabor feature based classification using the
enhanced fisher linear discriminant model for face recognition,” IEEE
Trans. On Image Processing, vol. 11, pp. 467–476, 2002.
[3] Y. Adini, Y. Moses, and S. Ullman, “Face recognition: The problem
of compensating for changes in illumination directions,” IEEE Trans.
PAMI, vol. 19, pp. 721–732, 1997.
[4] A.U. Batur and M.H. Hayes III, “Linear subspaces for illumination
robust face recognition,” CVPR, vol. 2, pp. 296–301, 2001.
[5] A.S. Georghiades and P.N. Belhumeur, “From few to many: illumina-
tion cone models for face recognition under variable lighting and pose,”
IEEE Trans. PAMI, vol. 23, pp. 643–660, 2001.
[6] J.C. Lee, J. Ho, and D. Kriegman, “Nine points of light: acquiring
subspaces for face recognition under variable lighting,” CVPR, vol. 1,
pp. 519–526, 2001.
[7] L. Zhang and D. Samaras, “Face recognition under variable lighting
using harmonic image exemplars,” CVPR, vol. 1, pp. 19–25, 2003.
[8] H.F. Chen, P.N. Belhumeur, and D.W. Jacobs, “In search of illumina-
tion invariants recognition under variable lighting and pose,” CVPR,
vol. 1, pp. 254–261, 2000.
[9] D.J. Jobson, Z. Rahman, and G.A. Woodell, “A multiscale retinex for
ridging the gap between color images and the human observation of
scenes,” IEEE Trans. On Image Processing, vol. 6, pp. 965–976, 1997.
[10] H. Wang, S. Li, and Y. Wang, “Generalized quotient image,” CVPR,
vol. 2, pp. 498–505, 2004.
[11] A. Benoit and A. Caplier, “Head nods analysis: Interpreation of non
verbal communication gestures,” ICIP, vol. 3, pp. 425–428, 2005.
[12] A. Benoit, The human visual system as a complete solution for image
processing, Ph.D. thesis, INPG, Grenoble, France, 2007.
[13] W. Davidson and M. Abramowitz, “Molecular expressions microscopy
primer: Digital image processing - difference of gaussians edge en-
hancement algorithm,” Olympus America Inc., and Florida State Uni-
versity.
[14] L. Meylan, D. Alleysson, and S. Susstrunk, “Model of retinal local
adaptation for the tone mapping of color filter array images,” J. Opt.
Soc. Amer., vol. 24, pp. 2807–2816, 2007.
[15] J. Phillips, H.Moon, and S.A. Rizvi et al., “The feret evaluation
methodology for face-recognition algorithms,” IEEE Trans. PAMI, vol.
22, pp. 1090–1104, 2000.
[16] M. Turk and A. Pentland, “Eigenfaces for recognition,” J. Cognitive
Neuroscience 3, pp. 71–86, 1991.
[17] T. Ahonen, A. Hadid, and M. Pietikainen, “Face recognition with local
binary patterns,” ECCV, pp. 469–481, 2004.
[18] X. Xie and K.M. Lam, “An efficient illumination normalization method
for face recognition,” Pattern Recognition Letters 27, pp. 609–617,
2006.
[19] W. Zhang, S. Shan, W. Gao, X. Chen, and H. Zhang, “Local gabor
binary pattern histogram sequence (lgbphs): A novel non-statistical
model for face representaion and recognition,” ICCV, vol. 1, pp. 786–
791, 2005.