BLUE EYES TECHNOLOGY

B. Tech. Seminar Report

BY

MUNISH BANSAL (06429)

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING

NATIONAL INSTITUTE OF TECHNOLOGY
HAMIRPUR-177005, HP (INDIA)

November, 2009
 BLUE EYES TECHNOLOGY

A Seminar Report

Submitted in partial fulfillment of the

Requirement for the award of the degree

Of

Bachelor of Technology

In

ELECTRONICS & COMMUNICATION ENGINEERING

BY

MUNISH BANSAL (06429)

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING

NATIONAL INSTITUTE OF TECHNOLOGY
HAMIRPUR-177005, HP (INDIA)

November, 2009
Abstract - Human error is still one of the I. INTRODUCTION
most frequent causes of catastrophes and
ecological disasters. The main reason is The Blue eyes system provides
that the monitoring systems concern only technical means for monitoring and
the state of the processes whereas human recording the operator’s basic
contribution to the overall performance physiological parameters. The most
of the system is left unsupervised. Since important parameter is saccadic activity,
the control instruments are automated to which enables the system to monitor
a large extent, a human – operator the status of the operator’s visual
becomes a passive observer of the attention along with head acceleration,
supervised system, which results in which accompanies large displacement
weariness and vigilance drop. This, he of the visual axis (saccades larger
may not notice important changes of than15 degrees). Complex industrial
indications causing financial or ecological environment can create a danger of
consequences and a threat to human life. exposing the operator to toxic substances,
It therefore is crucial to assure that the which can affect his cardiac, circulatory
operator’s conscious brain is involved in and pulmonary systems. Thus, on the
an active system supervising over the grounds of plethysmographic signal
whole work time period. taken from the forehead skin surface,
It is possible to measure indirectly the system computes heart beat rate and
the level of the operator’s conscious brain blood oxygenation.
involvement using eye motility analysis.
Although there are capable sensors The The Blue eyes system checks
available on the market, a complex above parameters against abnormal (e.g.
solution enabling transformation, analysis a low level of blood oxygenation or a
and reasoning based on measured signals high pulse rate) or undesirable (e.g. a
still does not exist. In large control rooms, longer period of lowered visual
wiring the operator to the central system attention) values and triggers user-
is a serious limitation of his mobility defined alarms when necessary.
and disables his operation. Utilization
of wireless technology becomes essential. Quite often in an emergency
situation operators speak to themselves Bluetooth connection, buffers incoming
expressing their surprise or stating sensor data, performs on-line data
verbally the problem. Therefore, the analysis, records the conclusions for
operator’s voice, physiological further exploration and provides
parameters and an overall view of the visualization interface.
operating room are recorded. This helps
to reconstruct the course of operators’
work and provides data for long-term
analysis.
BlueEyes consists of a mobile
measuring device and a central analytical
system. The mobile device is integrated
with Bluetooth module providing wireless
interface between sensors worn by the
operator and the central unit. ID cards
Figure1. Overall system diagram
assigned to each of the operators and
adequate user profiles on the central unit
side provide necessary data The task of the mobile Data
personalization so different people can Acquisition Unit are to maintain Bluetooth
use a single mobile device (called connection, to get information from the
hereafter DAU – Data Acquisition sensor and sending it over the wireless
Unit). The overall system diagram is connection ,to deliver the alarm messages
shown in Figure 1. The tasks of the sent from the Central System Unit to the
mobile Data Acquisition Unit are to operator and handle personalized ID cards.
maintain Bluetooth connections, to get Central System Unit maintains the other side
information from the sensor and sending of the Bluetooth connection, buffers
it over the wireless connection, to deliver incoming sensor data, performs on-line data
the alarm messages sent from the Central analysis, records the conclusion for further
System Unit to the operator and handle exploration and provides visualization
personalized ID cards. Central System interface.
Unit maintains the other side of the
 II. PERFORMANCE Blue eyes system monitors the status
REQUIREMENTS of the operator’s visual attention through
measurement of saccadic activity. The

The portable nature of the system checks parameters like heart beat

mobile unit results in a number of rate and blood oxygenation against

performance requirements. As the device is abnormal and triggers user defined alarms.

intended to run on batteries, low power BlueEyes system consists of a

consumption is the most important mobile measuring device and a central

constraint. Moreover, it is necessary to analytical system. The mobile device is

assure proper timing while receiving and integrated with Bluetooth module providing

transmitting sensor signals. To make the wireless interface between sensors worn by

operation comfortable the device should be the operator and the central unit. ID cards

lightweight and electrically safe. Finally the assigned to each of the operators and

use of standard and inexpensive IC’s will adequate user profiles on the central unit

keep the price of the device at relatively low side provide necessary data personalization

level. so The system consists of

> Mobile measuring device (DAU)
The priority of the central unit
is to provide real-time buffering and >Central System Unit (CSU)

incoming sensor signals and semi-real-time

processing of the data, which requires
speed-optimizes filtering and reasoning
algorithms. Moreover, the design should
assure the possibility of distributing the
processing among two or more central unit
nodes (e.g. to offload the database system
related tasks to a dedicated server).

Figure2. System overview

III. SYSTEM
OVERVIEWS
 IV. THE HARDWARE  Jazz Multisensor interface
 Beeper and LED indicators ,6 AA
4.1. DATA ACQUISITION UNIT:- batteries and voltage level monitor
Data Acquisition Unit is a mobile
part of the Blue eyes system. Its main task
is to fetch the physiological data from the
sensor and to send it to the central system to
be processed. To accomplish the task the
device must manage wireless Bluetooth
connections (connection establishment,
authentication and termination). Personal
ID cards and PIN codes provide
operator's authorization.Communication Figure3. DAU components

with the operator is carried on using a

simple 5-key keyboard, a small LCD 4.2. CENTRAL SYSTEM UNIT:-

display and a beeper. When an exceptional Central System Unit hardware is

situation is detected the device uses them the second peer of the wireless

to notify the operator. Voice data is connection. The box contains a Bluetooth

transferred using a small headset, module (based on ROK101008) and a PCM

interfaced to the DAU with standard codec for voice data transmission.The

minijack plugs. The Data Acquisition Unit module is interfaced to a PC using a

comprises several hardware modules: parallel, serial and USB cable. The audio
data is accessible through standard mini-jack
 Atmel 89C52 microcontroller -
sockets To program operator's personal ID
system core
cards we developed a simple
 Bluetooth module (based on
programming device. The programmer is
ROK101008)
interfaced to a PC using serial and PS/2
 HD44780 - small LCD display
(power source) ports. Inside, there is
 24C16 - I2C EEPROM (on a
Atmel 89C2051 microcontroller, which
removable ID card)
handles UART transmission and I2C
 MC145483 – 13bit PCM codec EEPROM (ID card) programming.
 consumer thread safe queues. Any number
of consumers can register to receive the
data supplied by a producer. Every single
consumer can register at any number of
producers, receiving therefore different
types of data. Naturally, every consumer
may be a producer for other consumers.
Connection Manager is
responsible for managing the wireless
communicationbetween the mobile Data
Figure4. CSU components Acquisition Units and the central system.
Data Analysismodule performs the
V. THE SOFTWARE analysis of the raw sensor data in order
toobtain information about the operator’s
Blue Eyes software's main task is physiological condition.
to look after working operators' Visualization module provides a
physiological condition. To assure instant user interface for the supervisors. It
reaction on the operators' condition enables them to watch each of the working
change the software performs real time operator’s physiological condition along
buffering of the incoming data, real-time with a preview of selected video source
physiological data analysis and alarm and related sound stream. All the
triggering. The Blue Eyes software incoming alarm messages are instantly
comprises several functional modules signaled to the supervisor. The Visualization
System core facilitates the transfers flow module can be set in an offline mode,
between other system modules (e.g. where all the data is fetched from the
transfers raw data from the database. Watching all the recorded
ConnectionManager to data analyzers, physiological parameters, alarms, video
processed data from the data analyzers to and audio data the supervisor is able
GUI controls, other data analyzers, toreconstruct the course of the selected
data logger etc.). The System Core operator’s duty.
fundamental are single-producer-multi-
 state. An important element of
incorporating emotion into computing is for
productivity for a computer user. A study
(Dryer & Horowitz, 1997) has shown that
people with personalities that are similar
or complement each other collaborate
well. Dryer (1999) has also shown that
people view their computer as having a
personality. For these reasons, it is
important to develop computers which can
work well with its user.

VII. TYPES OF MOTIONAL

SENSORS:
Figure5. Software analysis diagram

For Hand:
 Emotion Mouse
VI. EMOTION COMPUTING  Sentic Mouse
For Eyes:
 Expression Glasses
Rosalind Picard (1997) describes  Magic Pointing
 Eye Tracking
why emotions are important to the For Voice:
computing community. There are two  Artificial Inteligence
aspects of affective computing: giving the  Speech Recognition
computer the ability to detect emotions and
giving the computer the ability to express
VIII. EMOTION MOUSE
emotions. Not only are emotions crucial for
rational decision making as Picard describes,
One proposed, non—invasive
but emotion detection is an important step to
method for gaining user information through
an adaptive computer system. An adaptive,
touch is via a computer input device, the
smart computer system has been driving
mouse. This then allows the user to relate
our efforts to detect a person’s emotional
the cardiac rhythm, the body temperature,
electrical conductivity of the skin and other IX. SENTIC MOUSE
physiological attributes with the mood. This
It is a modified computer mouse that
has led to the creation of the “Emotion
includes a directional pressure sensor for
Mouse”. The device can measure heart
aiding in recognition of emotional valence
rate,temperature, galvanic skin response
(liking/attraction vs. disliking/avoidance)
and minute bodily movements and matches
them with six emotional states: happiness,
surprise, anger, fear, sadness and disgust.
The mouse includes a set of sensors,
including infrared detectors and
temperature-sensitive chips. These
components, User researchers’ stress, will Figure7. Senetic mouse
also be crafted into other commonly used
items such as the office chair, the steering X. EXPRESSION GLASSES
wheel, the keyboard and the phone
handle. Integrating the system into the A wearable device which allows any
steering wheel, for instance, could allow an viewer to visualize the confusion and
alert to be sounded when a driver becomes interest levels of the wearer. Other recent
drowsy. developments in related technology is the
attempt to learn the needs of the user just by
following the interaction between the user
and the computer in order to know what
he/she is interested in at any given
moment. For example, by remembering
the type of websites that the user links to
according to the mood and time of the day,
the computer could search on related sites
and suggest the results the user.

Figure6. Emotion mouse

 target.Two specific MAGIC pointing
techniques, one conservative and one
liberal, were designed, analyzed, and
implemented with an eye tracker we
developed. They were then tested in a pilot
study. This early stage exploration
showed that the MAGIC pointing
techniques might offer many advantages,
Figure8. Expression Glass including reduced physical effort and
fatigue as compared to traditional manual

XI. MAGIC POINTING pointing, greater accuracy and naturalness

than traditional gaze pointing, and
possibly faster speed than manual
This work explores a new direction
pointing. In our view, there are two
in utilizing eye gaze for computer input.
fundamental shortcomings to the
Gaze tracking has long been considered
existing gaze pointing techniques,
as an alternative or potentially superior
regardless of the maturity of eye tracking
pointing method for computer input.
technology.
We believe that many fundamental
limitations exist with traditional gaze
XII. EYE TRACKING
pointing. In particular, it is unnatural to
overload a perceptual channel such as vision
Since the goal of this work is to
with a motor control task. We
explore MAGIC pointing as a user interface
therefore propose an alternative
technique, we started out by purchasing a
approach, dubbed MAGIC (Manual And
commercial eye tracker (ASL Model 5000)
Gaze Input Cascaded) pointing. With such
after a market survey. In comparison to the
an approach, pointing appears to the user to
system reported in early studies this system
be a manual task, used for fine
is much more compact and reliable.
manipulation and selection. However, a
However, we felt that it was still not robust
large portion of the cursor movement is
enough for a variety of people with
eliminated by warping the cursor to the
different eye characteristics, such as
eye gaze area, which encompasses the
pupil brightness and correction glasses. We representing those processes via machines
hence chose to develop and use our own (like computers, robots, etc). AI is
eye tracking system. Available behavior of a machine, which, if
commercial systems, such as those performed by a human being, would be
made by ISCAN Incorporated, LC T called intelligent. It makes machines smarter
echnologies, and Applied Science and more useful, and is less expensive than
Laboratories (ASL), rely on a single natural intelligence. Natural language
light source that is positioned either processing (NLP) refers to artificial
off the camera axis in the case of the intelligence methods of communicating with
ISCANETL-400 systems, or on-axis in the a computer in a natural language like
case of the LCT and the ASL E504 systems. English. The main objective of a NLP
program is to understand input and initiate
action. The input words are scanned and
matched against internally stored known
words. Identification of a key word causes
some action to be taken. In this way, one can
communicate with the computer in one’s
language. No special commands or
computer language are required. There is
no need to enter programs in a special
language for creating software.

XIII. SPEECH RECOGNITION

Figure10. Geometric Facial data extraction
The user speaks to the computer

XII. ARTIFICIAL INTELLIGENT through a microphone, which, in used; a

simple system may contain a minimum of

Artificial intelligence (AI) three filters. The more the number of filters

involves two basic ideas. First, it used, the higher the probability of accurate

involves studying the thought processes of recognition. Presently, switched capacitor

human beings. Second, it deals with digital filters are used because these can be
custom-built in integrated circuit form. a number of large retailers have
These are smaller and cheaper than active implemented surveillance systems that
filters using operational amplifiers. The record and interpret customer movements,
filter output is then fed to the ADC to using software from Almaden's BlueEyes
translate the analogue signal into digital research project. BlueEyes is developing
word. The ADC samples the filter outputs ways for computers to anticipate users'
many times a second. Each sample wants by gathering video data on eye
represents different amplitude of the movement and facial expression. Your gaze
signal .Evenly spaced vertical lines might rest on a Web site heading, for
represent the amplitude of the audio filter example, and that would prompt your
output at the instant of sampling. Each computer to find similar links and to call
value is then converted to a binary number them up in a new window. But the first
proportional to the amplitude of the sample. practical use for the research turns out to
A central processor unit (CPU) controls the be snooping on shoppers.
input circuits that are fed by the ADCS. A Another application would be in the
large RAM (random access memory) stores automobile industry. By simply touching
all the digital values in a buffer area. This acomputer input device such as a mouse, the
digital information, representing the spoken computer system is designed tobe able to
word, is now accessed by the CPU to determine a person's emotional state. for
process it further. The normal speech has a cars, it could be useful tohelp with critical
frequency range of 200 Hz to 7 kHz. decisions like: "I know you want to get into
Recognizing a telephone call is more the fast lane, but I'm afraid I can't do
difficult as it has bandwidth limitation of that.Yourtoo upset right now" and therefore
300 Hz to3.3 kHz. assist in driving safely.
Current interfaces between
computers and humans can present
XIV. APPLICATION OF BLUE- information vividly, but have no sense of
EYE TECHNOLOGY whether that information is ever viewed or
understood. In contrast, new real-time
Engineers at IBM's ffice:smarttags" computer vision techniques for perceiving
Research Center in San Jose, CA, report that people allows us to create "Face-
responsive Displays" and "Perceptive
Environments", which can sense
respond to users that are viewing them.
Using stereo-vision techniques, we are able
to detect, track, and identify users robustly
and in real time. This information can
make spoken language interface more
robust, by selecting the acoustic information
from a visually-localized
Environments can become aware of how
many people are present, what activity is
occuring, and therefore what display or
messaging modalities are most appropriate
to use in the current situation. The results
of our research will allow the interface
between computers and human users to
become more natural and intuitive.
The familiar and useful come from
things we recognize. Many of our favorite
things' appearance communicate their use;
they show the change in their value though
patina. As technologists we are now poised
to imagine a world where computing objects
communicate with us in-situ; where we are.
We use our looks, feelings, and actions to
give the computer the experience it needs to
work with us. Keyboards and mice will not
continue to dominate computer
interfaces. Keyboard input will be replaced
in large measure by systems that know what
we want and require less
communication. Sensors are gaining fidelity
and and ubiquity to record presence and
actions; sensors will notice when we enter
a space, sit down, lie down, pump iron, etc.
Pervasive infrastructure is recording it. This
talk will cover projects from the Context
Aware Computing Group at MIT Media
Lab.
source.
XV. CONCLUSIONS
The nineties witnessed quantum
leaps interface designing for improved man
machine interactions. The BLUE EYES
technology ensures a convenient way of
simplifying the life by providing more
delicate and user friendly facilities in
computing devices. Instead of using
cumbersome modules to gather information
about the user, it will be better to use
smaller and less intrusive units. Ordinary
household devices -- such as televisions,
refrigerators, and ovens -- may be able to do
their jobs when we look at them and speak
to them. It is only a technological forecast.
user
explicit
 of computer enhancements

ACKNOWLEDGEMENT and recovery

techniques,”IEEE”
commun. Magvol39.
The author is thankful to the
management of National Institute of [4] J.jones, L.ong, and
Technology, Hamirpur, for their guidance m.lazer,”creating and
and support. The author gratefully intelligent technology
acknowledges the support and constant network/worldwide
encouragement of the HOD and the Faculty interoperability
of E&CED, NIT Hamirpur. Finally he demonstration.”IEEEcomm
would like to thank his parents for their love un .mag.,vol 42.
and blessings, which have been instrumental
[5] BlueEyes
in the compilation of this report.
Technology,Computer
Edge,Oct.2002,pages 23-27.
REFERENCES

[1] Joseph j.carr & john

m.brown,”introduction to
blue
eyestechnology”,published
in ieee spectrum magazine.

[2] A.jajszczyk,”automatically
switched blue eyes
networks:Benefits and
Requirement,”IEEE blue
toooth.feb 2005,vol
3,no1,pp.

[3] A .Banerjee, ”Generalized

multi protocol label
switching: an over view