Pose-invariant gender classification based

on 3D face reconstruction and synthesis

from single 2D image
A. Moeini and H. Moeini
A novel method is proposed for pose-invariant gender classication
based on three-dimensional (3D) face reconstruction from only 2D
frontal images. A 3D face model is reconstructed from only a single
2D frontal image. Then, for each two-class of gender in the database,
a feature library matrix (FLM) is created from yaw face poses by rotating the 3D reconstructed models and extracting features in the rotated
face. Each FLM is subsequently rendered based on the yaw angles of
face poses. Then, an array of the FLM is selected based on the estimated yaw angles for each class of gender. Finally, the selected
arrays from FLMs are compared with target image features by
support vector machine classication. Promising results are acquired
to handle pose in gender classication on the available compared
with the state-of-the-art methods.

Introduction: Pose-invariant gender classication is one of the most difcult and challenging tasks in computer vision. Generally, gender classication encompasses two main steps: feature extraction and
classication. The method used in feature extraction should give adequate features, otherwise it will be misclassied. Hence, there are
many methods to extract features that robustly recognise gender under
the restriction of a one two-dimensional (2D) single training sample
for each two-class. The common gender classication approaches are
mostly categorised into two separate types: geometric-based and featurebased methods. Geometric features contain information about the
location and shape of facial features. Geometric features are more sensitive to noise and tracking errors than appearance features.
The feature-based approaches [13] utilise image lters; either on the
whole faces to construct global features, or several particular face
regions to construct local features, to extract the feature changes in the
face image. Popular approaches are Gabor wavelets [4], the local binary
patterns (LBP) operator [2], the interlaced derivative patterns (IDP) [3],
the local derivative pattern [5] and so on. Recently, Shan proposed
gender recognition on real-world faces [3]. In this method, LBP was
employed to describe faces, and Adaboost was used to select the discriminative LBP features. Finally, gender classication results were acquired
by applying the support vector machine (SVM) with the boosted LBP features. Moreover, Shobeirinejad and Gao [3] presented the IDP method as
a novel feature extraction, which is a derivative-based technique to extract
discriminative facial features for gender classication. This method operates on a neighbourhood around a pixel and concatenates the extracted
regional feature distributions as a feature vector.
In this Letter, a new approach is proposed for handling face poses in
gender classication based on 3D reconstruction and synthesising of
the face. Accordingly, a 3D model was initially reconstructed from 2D
frontal face images with facial expression. To reconstruct a 3D model
from each human frontal face with facial expression, a facial expression
generic elastic model (FE-GEM) was used. Then, each 3D reconstructed
face in the gallery was synthesised to all possible views and a feature
library matrix (FLM) was generated based on the yaw angles of face
poses for each two-class (male and female). On the other hand, automatic
head pose estimation by the eye-based normalisation approach was used
to extract the yaw angles of face poses. Therefore, for each two-class, an
array of FLM was selected based on the yaw angles of face poses which
were estimated from automatic head pose estimation. Finally, gender
classication was performed by the SVM [6] between the FLMs selected
arrays and the target images features.
Constructing a 3D database: To reconstruct the 3D face model from a
single image, the FE-GEM method [7] is used. Moeini et al. [7] proposed a method for 3D face modelling that is robust to facial expression.
In this method, three FE-GEMs were employed in the GEM framework
to generate three models for the input image. Then, these three generated
models were combined based on the similarity distance around the lips
and mouth of input images with the corresponding distance in facial
expression generic models. Finally, the desired model was generated
for input images from a mixture of the combined models. To construct
the 3D database for each gender, one 2D frontal image was registered
from each person with an arbitrary facial expression. Therefore, 2D

images for constructing the 3D database included one frontal image

with any facial expression.
Pose-invariant gender classication: In this Section, the feature extraction method is proposed from a 3D database by LBP [2] for
pose-invariant gender classication. Then, the method for
pose-invariant gender classication is represented.
Visual illustration of the proposed method for extracting the feature
by LBP is shown in Fig. 1. On the basis of the proposed method, the
process can be summarised as follows:
1. Input: Two sets of the 3D faces database based on two classes of
gender (male and female).
2. For each two-class, 3D faces were synthesised based on face pose and
possible views in all poses of the face were extracted with steps of 5
(s = 5) degrees on either side at the yaw direction in the pose matrix Pi,Y.
In fact, arrays of each Pi,Y matrix for each seven-class were arranged
based on the yaw angle of the face pose from the corresponding class,
where i indicates the number of class (i = 1 male and i = 2 female) and Y
is the number of arrays in the yaw dimension of the Pi,Y matrix.
3. The feature vectors were extracted based on Fig. 2 (an example of
feature extraction for each image is shown in Fig. 2) from each array
of pose matrix Pi,Y and by the LBP method. Then, for each specic
pose yaw, an array of FLM Ai,Y was created which was a feature set
of created views by synthesising 3D reconstructed models. However,
for each two-class, a FLM Ai,Y was created based on the yaw angle of
the face pose.
4. Output: FLMs Ai,Y = [A1,Y, A2,Y], where i is the number of classes in
the database.
texture pose matrix P i,Y

yaw angle (Y )

90

+90

LBP feature extraction

feature library matrix Ai,Y
90

+90

yaw angle (Y )

Fig. 1 Visual illustration of creating FLM from present 3D database

In this Figure, examples of creating A2,Y for female gender are given
LBP/LPQ
high-dimensional feature

PCA

low-dimensional
feature

Fig. 2 Visual illustration of feature extraction by LBP/LPQ from face images

using PCA

To extract high-dimensional features, a global histogram of LBP for

non-rigid parts of the face and a local histogram of LBP for rigid
parts of the face were applied, according to Fig. 2. Then, a lowdimensional feature was generated by principal component analysis
(PCA) from the high-dimensional feature. Hence, to separate the
images, this method can help to highlight the rigid part of the face
and ignore the non-rigid part of face in feature extraction. Moreover,
this LBP feature extraction with this image separation can resolve the
problem of face recognition to change facial expressions in the proposed
expression-invariant gender classication system.
The visual illustration of the proposed pose-invariant gender classication is shown in Fig. 3. It operated in two stages: ofine and online
stages. In the ofine stage, a frontal face image of each person was registered from an available 2D database. Then, for each registered image,
3D face reconstruction was performed by the FE-GEM to produce a
3D face database based on two-classes of gender. Then, feature extraction was performed and a FLM was created for each gender.

ELECTRONICS LETTERS 14th May 2015 Vol. 51 No. 10 pp. 760762

train images
offline
3D face
database
3D face
reconstruction

yaw
pose matrix P i,Y

feature library matrix

A i,Y

A i,Y = yaw

gender classification
result

test images
LBP feature extraction

SVM

pre-processing and face detection

head pose estimation

online

Fig. 3 Visual illustration of facial expression recognition from static face

image and 3D face synthesis
training images
90

45

testing images
0
+45

+90

Fig. 4 Samples of face images with seven facial expressions at ve face poses
from Radboud face database

In the online stage, pre-processing and face detection are performed

for each test image by CLM landmark localisation [8]. Then, the yaw
angles of the face are estimated by the eye-based normalisation head
yaw estimation method. Then, for each two-class, an array (Ai,Y = yaw)
of the FLM is selected based on yaw estimated angles. Finally,
gender classication is performed by the SVM [6] (linear) between
the target image features and selected FLMs Ai,Y = yaw = [A1,Y = yaw, A2,
Y = yaw].

Table 1: Gender classication rate (%) on Radboud face database

for males
Method
90

45

Pose (deg)
0
+45 +90

Mean

95.3
92.9
90.4
88.1

98.1 99.8 97.4 96.8

95.2 98.5 94 90.2
93.8 97.4 92.4 88.3
91.2 95 90.9 86.4

97.4
94.1
92.4
90.3

Downsampling + proposed method 80.3 87.4 91.9 89.0 78.1

85.3

Proposed method
IDP + proposed method
Gabor + proposed method
LPQ + proposed method

Table 2: Gender classication rate (%) on Radboud face database

for females
Method
Proposed method
IDP + proposed method
Gabor + proposed method

90
96.1
94.8
91.1

Pose (deg)
45
0
+45
99.1 100 99.4
96.9 100 95.9
94.6 98.8 95

+90 Mean
96.9 98.3
93.6 96.2
89.7 93.8

LPQ + proposed method

90.4 93 96.8 92.1 89.8
Downsampling + proposed method 79.5 86.4 92 87.1 78

92.4
84.6

Experiments: For experiments, a subset of the Radboud database with

seven different facial expressions at ve different poses (90, 45,
0, +45 and +90) were used for testing images. The Radboud faces
database [9] include 20 Caucasian male adults, 19 Caucasian female
adults, 18 Moroccan male adults, 6 Caucasian female children and 4
Caucasian male children and face images of each person were captured
with three different gaze directions. The seven expressions include

neutral, sad, angry, fearful, disgusted, happy and surprised. In this

experiment, a subset of 57 subjects were utilised which include 20
Caucasian males, 19 Caucasian females and 18 Moroccan males with
seven facial expressions and three different gaze directions. Examples
of facial expressions and different gaze directions in the Radboud
faces database at different face poses are shown in Fig. 4. Therefore,
there are 798 [38 (male person) 3 (gaze direction) 7 (expression)]
and 399 [19 (female person) 3 7] testing images for male and
females from 57 people in each face pose, respectively. Hence, the
total images for seven facial expressions are 1197 (171 7) in each
face pose.
To evaluate the generalisation performance to novel subjects, a
10-fold cross-validation testing scheme was adopted in the present
experiments for person-independent expression-invariant gender classication. For comparison, several popular methods in feature extraction
were used in the proposed method instead of the LBP. These methods
include Gabor [4], LPQ [10], IDP [3] and downsampling [10]. The proposed gender classication method seems to outperform the other
implemented approaches. These measurement results are shown in
Tables 1 and 2. Tables 1 and 2 show the gender classication rates of
the proposed methods in comparison with other methods on the
Radboud face database.
Conclusion: In this Letter, a combined approach is presented for
pose-invariant gender classication from only 2D frontal images in
the gallery. The proposed method was tested on the available image
in order to perform pose-invariant gender classication. Experimental
results were compared with classical as well as state-of-the-art
methods and it was demonstrated that the proposed method was better
than similar approaches.
The Institution of Engineering and Technology 2015
26 February 2015
doi: 10.1049/el.2015.0520
One or more of the Figures in this Letter are available in colour online.
A. Moeini (Department of Electrical Engineering, Amirkabir University
of Technology, Tehran, Iran)
E-mail: alimoeini@aut.ac.ir
H. Moeini (Department of Electrical Engineering, Semnan University,
Semnan, Iran)
References
1 Wu, M., Zhou, J., and Sun, J.: Multi-scale ICA texture pattern for
gender recognition, Electron. Lett., 2012, 48, pp. 629631
2 Park, Y.K., and Kim, J.K.: Fast adaptive smoothing based on LBP for
robust face recognition, Electron. Lett., 2007, 43, pp. 13501351
3 Shobeirinejad, A., and Gao, Y.: Gender classication using interlaced
derivative patterns. Proc. of ICPR, Istanbul, Turkey, August 2010,
pp. 15091512
4 Qin, H.F., Qin, L., Xue, L., and Yu, C.B.: Gabor-based weighted
region covariance matrix for face recognition, Electron. Lett., 2012,
48, pp. 992993
5 Zhang, B., Gao, Y., Zhao, S., and Liu, J.: Local derivative pattern
versus local binary pattern: face recognition with high-order local
pattern descriptor, IEEE Trans. Image Process., 2010, 19, pp. 533544
6 Hsu, C.-W., Chang, C.-C., and Lin, C.-J.: A practical guide to support
vector classication. Technical Report, Taipei, 2003
7 Moeini, A., Faez, K., and Moeini, H.: Expression-invariant 3D face
reconstruction from a single image by facial expression generic
elastic models, J. Electron. Imaging, 2014, 23, pp. 59
8 Saragih, J., Lucey, S., and Cohn, J.: Deformable model tting by regularized landmark mean-shift, Int. J. Comput. Vis., 2011, 91,
pp. 200215
9 Langner, O., Dotsch, R., Bijlstra, G., and Wigboldus, D.H.J.:
Presentation and validation of the Radboud faces database, Cogn.
Emotion, 2010, 24, pp. 13771388
10 Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., and Ma, Y.: Robust
face recognition via sparse representation, IEEE Trans. Pattern Anal.
Mach. Intell., 2009, 31, pp. 210227

ELECTRONICS LETTERS 14th May 2015 Vol. 51 No. 10 pp. 760762