Face Detection

M. Tech Project Report - Stage 2 Submitted in partial fulllment of the requirements for the degree of Master of Technology by Laxman Singh Sayana Roll No: 07305085 under the guidance of Prof. M. R. Bhujade

Department of Computer Science and Engineering Indian Institute of Technology, Bombay Powai, Mumbai-400076 January 12, 2009

Acknowledgements
I would like to thank my project guide Prof. M. R. Bhujade whose constant guidance, suggestions and encouragement helped me throughout the MTP rst and second stage work. Laxman Singh Sayana

Contents
1 Introduction 1.1 The goal of face detection . . . . . . . . . . . . . . . . . . . . 1.2 Scope of the project . . . . . . . . . . . . . . . . . . . . . . . . 2 Previous Work: 2.1 Challenges in Face Detection . . . . . . . . . . . . . . . . . . . 2.2 Dierent Approaches for face detection . . . . . . . . . . . . . 3 Second Stage Work 3.1 Assumptions . . . . . . . . . . . . . . . . . . . . 3.2 Face detection Algorithm . . . . . . . . . . . . . 3.3 Pre-processing skin color information collection 3.4 Skin color region detection algorithm . . . . . . 3.5 Binary skin detection . . . . . . . . . . . . . . . 3.5.1 Removal of small regions . . . . . . . . 3.5.2 Removal of non face candidates . . . . . 3.6 Template Matching . . . . . . . . . . . . . . . . 3.7 Results . . . . . . . . . . . . . . . . . . . . . . . 4 Future work 1 1 2 3 3 5

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

6 . 6 . 7 . 7 . 7 . 9 . 9 . 9 . 14 . 15 16

List of Figures
3.1 3.2 3.3 3.4 3.5 3.6 3.7 Skin color detection(Training image1) . . Skin color detection(Training image2) . . Binary image(Training image1) . . . . . Binary image with small region removal . Binary image (Training image2) . . . . . Binary image with small region removal . Face template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 10 10 11 11 12 14

Abstract Detection of faces in an image is a problem that has gained importance in the last decade.Identifying faces is quite simple for human because it comes naturally but it is not so simple teaching a computer to detect faces in an image.The objective of this project is to come up with an algorithm that is able to detect faces and identify their location in a given color image. Face detection and recognition is one of the challenging research areas in the eld of computer vision. Face detection is a necessary rst-step in face recognition systems, with the purpose of localizing and extracting the face region from the background. The application of such a system are numerous, from automated security systems, census, human-computer interfaces etc. In this report, we are presenting a method to detect skin color from still images.Then try to remove all non face region.

Chapter 1 Introduction
Face detection is the process of nding all possible faces in a given image.More precisely, we can say face detector has to determine the locations and sizes of all possible human faces in given image. It is a more complex case than face localization in which the number of faces is already known. On the other hand, face detection is the essential rst step towards many advanced computer vision, biometrics recognition and multimedia applications, such as face tracking, face recognition, and video surveillance.Face detection is one of the most popular topics of research in computer vision eld. In recent years, face recognition has attracted much attention and its research has rapidly expanded by not only engineers but also neuroscientists, since it has many potential applications in computer vision communication and automatic access control system.However, face detection is not straightforward because it has lots of variations of image appearance,such as pose variation (front,non-front), occlusion,image orientation,illuminating condition and facial expression.

1.1

The goal of face detection

The goal of face detection is to determine whether or not there are any faces in image are present or not and if present,return the image location and extent of each face. The face detection problem can be dened as follows: Given as input an arbitrary image, which could be a digitized video signal or a scanned photograph, determine whether or not there are any human faces in the image, and if there are, return an encoding of their location. The encoding in this system is to teach face in a bounding box dened by the 1

image coordinates of the corners.

1.2

Scope of the project

Face detection is process of nding the faces in an given image. In chapter 2 we will discuss about the various factors that can aect the face detection like size, facial expression, lightning condition, image background etc and also discuss dierent approaches for detecting faces. In chapter 3 we are describing skin color detection algorithm and also discuss how to remove non face region chapter 4 describes the future work which include the implementation of Template matching algorithm using SVM as classier.

Chapter 2 Previous Work:

2.1 Challenges in Face Detection

Due to scale, rotation, pose and illumination variation, face detection involves many research challenges. How to detect dierent scales of faces, how to be robust to illumination variation, how to achieve high detection rate with low false detection rates are only few of all issues a face detection algorithm needs to consider. For face detection following problems need to be considered[4, 6, 3]: Size: A face detector should be able to detect faces in dierent sizes.This we can achieved by scaling the input image. And small faces are more dicult to detect than the large face. Expressions: The appearance of a face changes considerably for different facial expressions and, thus, makes the face detection more dicult.The simplest type of variability of images of a face can be expressed independently of the face itself, by rotating, translating, scaling and mirroring its image.Also changes in the overall brightness and contrast of the image and occlusion by other objects. Pose variation: The images of face vary due to relative camera-face pose(frontal,45 degree, prole), and some facial features such as an eye or the nose may become partially or wholly occluded.Another source of variation is the distance of the face from the camera, changes in which can result in perspective distortion. Lighting and Texture Variation: Now we will describe how the variation caused by the object and its environment, specically the

objects surface properties and the light sources.Changes in the light source in particular can change a faces appearance. Background Variation: When an object has predictable shape, it is possible to extract a window which contains only pixels within the object, and to ignore the background.However, for prole faces, the border of the face itself is the most important feature, and its shape varies from person to person.Thus the boundary is not predictable, so the background cannot be simply masked o and ignored. Presence or absence of structural components: Facial features such as beards, mustaches and glasses may or may not be present.And also there may be variability among these components including shape, color and size. Shape Variation: Shape variation includes facial expressions,whether the mouth and eyes are open or closed,and the shape of the individuals face.The appearance of faces are directly aected by persons facial expression. Occlusion: Faces may be partially occluded by other objects.In an image with a group of people some faces may partially occlude other faces.Partial occlusions of faces can be caused by objects within the environment(e.g, poles and people), objects worn by the person (glasses, scarf, mask), Other body parts of person (hands) and shadows. Image orientation: Faces can appear in dierent orientation the image plane depending on the the angle of the camera and the face. images directly vary for dierent rotations about the cameras optical axis. Imaging conditions: When the images is formed,factors such as lighting (spectra,source distribution and intensity) and camera characteristics (sensor response, lenses) aect the appearance of a face.

2.2

Dierent Approaches for face detection

Here we will discuss existing techniques to detect faces from a single intensity or color image. Single image detection methods can be classify into four categories(M.tech rst stage work): Knowledge-based methods: Knowledge-based methods use human-coded rules to model facial features, such as two symmetric eyes, a nose in the middle and a mouth underneath the nose.These methods are designed mainly for face localization[4]. Feature invariant approaches: These algorithm aim to nd structural features that exist even when the pose, viewpoint, or lighting conditions vary, and then use these to locate faces.Skin color, edges and shapes falls into this category.These methods are designed mainly for face localization[4]. Template matching methods: Template matching methods calculate the correlation between a test image and pre-selected facial templates.Several standard patterns of face are stored to describe the face as whole or the facial features separately.These methods have been used for both face localization and detection[4]. Appearance-based methods: The last category, appearance-based, adopts machine learning techniques to extract discriminative features from a pre-labeled training set. The Eigenface method is the most fundamental method in this category. Recently proposed face detection algorithms such as support vector machines, neural networks, statistical classiers and AdaBoostbased face detection also belong to this class. These methods are designed mainly for face detection[4].

Chapter 3 Second Stage Work

The goal of our project is to detect multiple faces in an image. Existing face detection algorithms include template matching, the use of neural networks[2], featured-based approach in which features of the face are extracted and veried, model based approach which involves the shape and/or color of the face candidates. These all methods have been discussed in rst stage work.Therefore we combined these techniques to formulate a face detection algorithm that is able to give a high success rate in a relatively short time. Our method focuses on the use of a skin color model along with its statistical information[4, 1, 5, 2, 7]. Aiming towards a test image containing two or more faces, a statistical approach is fairly reliable.

3.1

Assumptions

The following assumptions are made to the input image in order to simplify the algorithm and improve eciency: 1. The input is a color image 2. There are multiple faces with frontal view and upright orientation 3. The size of faces within the image should approximately be the same 4. Little deviation in brightness for all the faces within the image 5. Faces have to be greater than a certain size in the image so that facial features can be detected

3.2

Face detection Algorithm

Our face detection algorithm consists of four stages: pre-processing, skin color region detection,binary skin detection(removal of non face region) and template matching.

3.3

Pre-processing skin color information collection

Using human skin colors is a relatively fast and reliable way of locating human faces in an image.The color distribution of skin colors of dierent people was found to be clustered in a small area of the chromatic color space. Although skin colors of dierent people appear to vary over a wide range, they dier much less in color than in brightness. In other words, skin colors of dierent people are very close, but they dier mainly in intensities. With this, we could proceed to develop a skin-color model in the chromatic color space[4, 1, 5, 2, 7]. According to various research it is found that skin colors have predominantly red coloring in comparison to the green and blue components. In examining the skin colors, it is interesting to note that the blue component varies across the various tones. This supports the theory that the blue component does not contribute much to the color of esh. The red-to-green ratio seems to vary; however, most of the colors lie within the range of 2:1 to 3:1 for red-to-green component. Also, it is worth noting that all the skin colors have red and green that are above a minimum threshold value, which we have approximately determined to be 40 on a scale of 0 to 255.

3.4

Skin color region detection algorithm

The skin color detection algorithm is a simple color threshold and comparison method. Based on some analytical assessments of average skin colors, the program analyzes the Red and Green components of each pixel against two assessed average esh color values on a 2D coordinate plane. By taking the Dot Product of each pixels Red and Green values with that of the two assessed esh color values, the algorithm determines whether a pixel is or is not a esh color. In addition, the program checks that each pixels Red and Green values are above a pre-determined threshold[5, 2, 7].

The Algorithm is described below: Load Image le into a bueredImage. for each pixel(x,y) in the image: IF(pixel.Red < 40 ORpixel.Green < 40) Set output(x,y)= BLACK. ELSE IF (DOT_PRODUCT(pixel, skin_color1) <= 0.995) OR (DOT_PRODUCT(pixel, skin_color2) <= 0.995) Set output(x,y)= BLACK. ELSE Set output(x, y) = pixel (x, y).

Figure 3.1: Skin color detection(Training image1)

3.5

Binary skin detection

Based on the skin color thresholding, a resulting black and white mask is obtained with all the faces in addition to some artifacts (body parts, background). This mask is then rened through binary morphological operations to reduce the background contribution and remove holes within faces.

3.5.1

Removal of small regions

After skin color detection next step is to remove all the binary blobs found on face and non face region.Face blobs are dense and consistently had sizes in the hundreds of pixels.Thus we rst try to remove all connected components smaller than 50 pixels. The face blobs will not aect by this step[1, 5, 2, 7].

3.5.2

Removal of non face candidates

Now next step is to removal of large connected components and that our face blobs cover a much smaller area. Naturally, this led to the idea of removing connected components with a large number of pixels.However we have to be some what careful.The large connected components detections, though 9

Figure 3.2: Skin color detection(Training image2)

Figure 3.3: Binary image(Training image1)

10

Figure 3.4: Binary image with small region removal

Figure 3.5: Binary image (Training image2)

11

Figure 3.6: Binary image with small region removal dense enough to yield large connected areas, are still more sparse than the face blobs. Secondly, due to the presence of red clothing, there may be chance that two or more face blobs are often seen connected to one another. To determine a good blob size threshold above which we can eliminate, we average the window sizes hypothesized by our Maximal Rejection step. This tends to be a good estimate of the typical face size for our image. The threshold is taken to be the square of three times this window length, or nine times the typical face area. After doing the analysis of the face sizes in all the training images, it was found that the area of the minimum face size in all of the training images was around 800 pixels. So we can safely assume that all the regions having their area <= 600 pixels are non-face regions. As shown in all the regions having area <= 600 pixels are removed which essentially gets rid of small isolated regions (which is basically background noise or some isolated regions in the clothes). But still there are other non-face regions, which are still present. After doing some analysis of the training images, it was found that the area of the minimum sized face in each of the training images is dierent depending on how the picture was taken (whether it is taken closely or not). So dierent threshold ranges are set for dierent training images based on the average size or area of face in the training image. The average size or

12

area for a training image is calculated by taking the average of the areas of all the remaining regions. Here are the thresholds that could be used for various training images based on the average size or area of the face in a particular training image: For average area < 2400 pixels and > 600 pixels: threshold = 780 pixels For average area < 4500 pixels and >= 2400 pixels: threshold = 1750 pixels For average area < 6500 pixels and >= 4500 pixels: threshold = 2000 pixels For average area >= 6500 pixels: threshold = 2500 pixels

13

Figure 3.7: Face template

3.6

Template Matching

The basic idea of face template matching is to nd regions which are similar to the template. Hence, generating a template which looks like a face is important. The template must be such that it rejects the non-face regions in the image. The template must be such that it rejects the non-face regions in the image. The template that was used for matching is shown in gure. The size of the actual template is 50x50 pixels. Larger templates yield low correlation for smaller faces, while the smaller template is able to yield high correlation for both the small and large faces. In an eort to achieve optimal results, multi-resolution template matching should be used. Every pixel in the mask image was considered as the center of a candidate face. Multiple size images were then extracted using that pixel as the center of such an image and correlation was performed with the template. The size of the extracted image, which matches the size of the underlying face, was expected to give the maximum correlation. However, in extracting larger sizes, parts of nearby faces are included in the image. This tends to aect the correlation and yields higher values for larger sizes irrespective of the underlying content in the image. We tried to implement template matching method for removing non face region in an image but not able to complete it.Therefore this will be done in next stage.

14

3.7

Results

Following are the results which we have nd so far: Training image1: No of faces :2 Detected skin region:3 Non face region detection:1 Training image2: No of faces:11 Detected skin region:18 Non face region detection:7

15

Chapter 4 Future work

In this report we have describe human skin color detection algorithm which gives the possible face candidate for face detection.Then we describe how the skin color detection algorithm is working how did we set the threshold value.Then we describe how to remove small region which are not making a face candidate. Next stage work will be detecting actual face candidate using template matching and also using better classier which will exactly discriminate the face and non face region.

16

Bibliography
[1] D. Chai and K.N. Ngan. Locating facial region of a head-and-shoulders color image,proc. third intl conf. automatic face and gesture recognition,, 1998. [2] Seok-wun Ha jing Zhang, Yang Liu. A novel approach of face detection based on skin color segmentation and pca,ieee conf. the 9th international conference for young computer scientists, 2008. [3] u.j.monich Lutzgoldmann. Components and their topology for robust face detection in the presence of partial occlusions, september, 2007. [4] IEEE-David J. Kriegman Senior Member IEEE Ming-Hsuan Yang, Member and IEEE Narendra Ahuja, Fellow. Detecting faces in images: A survey,, january 2002. [5] Divya S. Vidyadharan Praseeda Lekshmi V, Dr. SasiKumar. Face detection and localization of facial features in still and video images,ieee conf. rst international conference on emerging trends in engineering and technology, 2008. [6] H. Rowley. Neural network-based face detection,, may. 1999. [7] Tong Li Wang Zhanjie. A face detection system based skin color and neural network,ieee conf. 2008 international conference on computer science and software engineering, 2008.

17