ATAL BIHARI VAJPYEE

INDIAN INSTITUTE OF INFORMATION

TECHNOLOGY
GWALIOR

Report on
Drowsiness Detection

Submitted to: Submitted by:

Dr. Santosh Singh Rathore 2018IMG-041 Rachit Goel

Software Engineering Course Coordinator 2018IMG-063 Shivesh Tiwari

2018IMG-064 Siddharth Verma
2018IMG-067 Utkarsh Singh
Introduction
The driver drowsiness and distraction warning system which detects the distraction and
real-time drivers eye closeness situation, to determine the drowsiness by a defined criterion.
This could be a suitable methodology to prevent traffic accidents caused due to distraction from
the road.
There are three main categories of real-time drowsiness detectors:
Vehicle-based, Signal-based and Facial feature-based.

Vehicle-based methods try to infer drowsiness from vehicle situation and monitor the variations
of steering wheel angle, acceleration, lateral position, etc.
Signal based methods infer drowsiness from psychophysiological parameters. The most
important physiological signals that used and investigated are ElectroEncephaloGram (EEG),
ElectroOculoGram (EOG), activities of autonomous nervous system from ElectroCardioGram
(ECG), Skin Temperature (ST), Galvanic Skin Response (GSR) and also neuromuscular
activities as ElectroMyoGram (EMG). This approach needs to consider invasive captors, which
can affect driving negatively.
However, these above approaches detect driver drowsiness slowly.
Facial feature-based methods can evaluate the target in real-time without invasive instruments.
They are inexpensive and more accessible than other methods.
Many engineers have proposed different methods for face detection with using different
landmark points on face. Eye and mouth are considered as a dominant sign of fatigue. They
used strategies involving: convolution neural networks , examine the eye by oblique view using
oblique face detector, percentage of eyelid closure (PERCLOS).Datasets used are: The ULG
Multimodality Drowsiness Database (DROZY), two video datasets of drivers with various
facial characteristics, like with and without glass and sunglass or different ethnicity for
designing algorithms for yawning detection. Kinect camera is used for drivers monitoring and
identify driving tasks in a real vehicle. They achieved to 80.7% accuracy with FFNN network
for their classification.
Approach Discussion
The first step towards drowsiness detection is estimation of the head position. When a driver is
driving, there are basically two types of movement of his head: Rotational and Translational. For
estimation a reference is also taken which is the looking ahead head and the variation from this
reference is calculated. The Head position estimation can be understood in the following way:
Consider a point on head of the driver and its coordinates as (U,V,W), This point undergoes
Rotational movement R which is 3X3 matrix and Translational Movement t (3X1 matrix), So the
final location (X,Y,Z) of this point is calculated using these values. If the final location is out of
range from the reference an alert is sent, otherwise it moves to the next step.
The next step involves crop of the eye region. Rotating head to left, right, down, and up. The
introduced algorithm by Viola and Jonas is used for face detection and Dlib’s facial landmark
detector. Finally, the offset value to the shape of the driver’s head is normalized and then
compared with a specific range.

Another challenge in cropping the eye from the face is lighting conditions. Histogram equalizer
to equalize eye contrast. This approach makes the methodology more accuracy for eye closeness
detection. After eye equalizing and detection, the cropped eye is sent to the network.

Two datasets have been used . The first one is the ZJU dataset, and the second one is a mixture
dataset of ZJU and the created database. The mixed datasets have more number of images and
The strength point of this dataset is that it considers the different situation of the eyes. The data
are collected from different distances, rotation and angles, which increases the driver’s freedom
of action that is a more real situation. Data with and without glasses is also present.

In this approach only one eye is considered because face is a symmetric shape. For this goal,
the algorithm computes distances from the most right and the most left point of the right and
left eyes, and compares them to each other and select the bigger distance for cropping. After
this the images is sent to the Neural Networks for detection. Three neural networks namely FD-
DNN, VGG16, VGG19 are used and there results are compared.

When the images passes through the neural network, the network estimates the probability of
closeness of eyes. If more than 50% for more than 20 images, a danger is considered and alert
is sent
Finding

Table1: Comparison of accuracy of our extended dataset and ZJU dataset with different methods.

Table2 : RESULT OF THREE PROPOSED NETWORK ON ZJU DATASET

Advantages
1. The authors extended ZJU dataset to become more robust than the available datasets
with more positions. Also, a model-based estimation is proposed to detect risky head
turns for both distraction estimation and reducing the false close eye detection. Then,
landmark points are applied to access ROI and use them as the input to three different
networks to achieve better results.

2. The experimental results indicated that the proposed networks are faster than the
available models and expended dataset advantage is considering oblique view which
make system works in more different situations

3. In today’s world there are more vehicles on the roads than ever before at all hours of
the night and day. This increase in road usage is a factor in the rate of traffic accidents
on roads. Road safety is a topic discussed almost every day in the papers and on our
airwaves in this country and worldwide, unfortunately though this is usually because of
the lack of safety and precaution taken by road users. In addition, especially in India,
hundreds of thousands travel over long distances to work, day in, and day out. This long
daily start very early in the morning and end late in the evening can fatigue the driver.
The critical points at which driver fatigue related collisions happen are between 2am
and 6am and mid-afternoon between 2pm and 4pm when our body clock is at its lowest
point. Males aged 18 to 30 are in the high-risk category. If a driver persists in fighting
sleep while driving, the impairment level is the same as driving while over the drink
drive limit. Eventually a driver will drift in and out of consciousness and experience
"micro sleeps" which can last for up to 10 seconds. In this time a driver has no control
of the vehicle. Drivers can experience such a micro sleep with their eyes wide open.So
we can understand that there is serious need for some mechanism to deal with this
problem and this driver alert system will provide a accurate and effective results. Use
of this device will certainly help in reducing the number of accidents and Providing
better experience to the driver.

Critical Analysis
The approach discussed in this paper is facial feature based and involves deep learning. The
procedure starts by capturing the head position from the camera. Then ROI is identified and the
final image after processing is sent to the neural network to check. There is a point to be noted
is that Eye closeness detection is a challenging task since there is ambient factors that effect on
it, as lighting condition, image resolution, driver position and, different shape of eyes, different
threshold for closure of an eye. To find a dataset containing of images of eyes with such
variation is difficult. But in this approach the authors have used a extended dataset which is
combined with existing dataset to create a more comprehensive database to train the model.
They have also compared different neural network techniques. From this analysis we can see
which neural network provides best results on a given dataset.
Another important method used in this approach is the symmetry of the face. As face is
symmetrical, we can use image of only on eye to compare. This approach have reduced the
computation time and chances of false detection.

For future works, the authors intend to focus on yawning analysis for fatigue detection by
landmark points of lips. Also, a different level of drowsiness can be investigated without
limitations of being in two situations because the boundaries among different levels of
drowsiness are so narrow, and it would be more challenging task. It is expected from the authors
that they will bring something new in future.

The drowsiness plays a vital role in safe driving and therefore, this paper proposed a dataset for
driver drowsiness detection and studied several networks to achieve better accuracy
and less time needed for drowsiness detection based on eye states.

Referrals
1.Deep learning based Driver Distraction and Drowsiness Detection By Maryam Hashemi,
Alireza Mirrashid, Aliasghar Beheshti Shirazi (https://dblp.org/rec/html/journals/corr/abs-
2001-05137)

Individual Contribution
The Research paper has been thoroughly read and analysed by each member
Rachit Goel Introduction

Shivesh Tiwari Critical Analysis

Siddharth Verma Advantages

Utkarsh Singh Approach Discussion