BLUE EYES TECHNOLOGY

1. ABSTRACT
Imagine yourself in-front of your computer that can listen feel your emotions and interact with you through special techniques merely by the touch of the mouse. For a long time emotions have been kept out of the deliberate tools of science, but no tools could sense and respond to their affective information. Blue Eyes technology aims at creating computational Machines with perceptual and sensory abilities like those of human beings . Blue eyes system consists of a mobile measuring device called Data Acquisition Unit (DAU) and a central analytical system called Central System Unit (CSU). CSU interconnected by bluetooth. DAU collects information from the sensor and sends it over the bluetooth and delivers the messages sent from CSU to the operator. CSU buffers incoming sensor data and provides visualization interface.

2. INTRODUCTION
Basics of Bluetooth :Bluetooth wireless technology is a short-range communications technology intended to replace the cables connecting portable and/or fixed devices while maintaining high levels of security. The key features of Bluetooth technology are robustness, low power, and low cost. The Bluetooth specification defines a uniform structure for a wide range of devices to connect and communicate with each other. Bluetooth technology has achieved global acceptance such that any Bluetooth enabled device, almost everywhere in the world, can connect to other Bluetooth enabled devices in proximity. Bluetooth enabled electronic devices connect and communicate wirelessly through short-range, ad hoc networks known as piconets. Each device can simultaneously communicate with up to seven other devices within a single piconet. Each device can also belong to several piconets simultaneously. Piconets are established dynamically and automatically as Bluetooth enabled devices enter and leave radio proximity. A fundamental Bluetooth wireless technology strength is the ability to simultaneously handle both data and voice transmissions. This enables users to enjoy variety of innovative solutions such as a hands-free headset for voice calls, printing and fax capabilities, and synchronizing PDA, laptop, and mobile phone applications to name a few.
2.1

Motivation for Development of System:-

Human error is still one of the most frequent causes of catastrophes and ecological disasters. The main reason is that the monitoring systems concern only the state of the processes whereas human contribution to the overall performance of the system is left unsupervised. The control instruments are automated to a large extent, so a humanoperator becomes a passive observer of the supervised system, which results in weariness and vigilance drop. Thus, he may not notice important changes of indications causing financial or ecological consequences and a threat to human life. It therefore is crucial to assure that the operators conscious brain is involved in an active system supervising over the whole work time period. It is possible to measure indirectly the level of the operators conscious brain involvement using Eye Motility Analysis. Although there are capable sensors available on the market, a complex solution enabling transformation, analysis and reasoning based on measured signals still does not exist. In large control rooms, wiring the operator to the central system is a serious limitation of his mobility and disables his operation. Utilization of wireless technology becomes essential.

2.2 What is Blue Eyes System The system developed is intended to be the complex solution for monitoring and recording the operators
3

conscious brain involvement as well as his physiological condition. For this designing a Personal Area Network is required linking all the operators and the supervising system. As the operator using his sight and hearing senses the state of the controlled system, the supervising system will look after his physiological condition. The term Blue Eye: Blue in term stands for Bluetooth, which enables reliable communication. Eye, the eye movement enables us to obtain a lot of interestingness and important information.

Blue Eyes is the technology to make computers sense and understand human behaviour and feelings.

Blue Eyes aims to design smarter devices with emotional intelligence. It also creates computational perceptual abilities. devices with

2. SYSTEM DESIGN
3.1.Design Methodologies:In creating the Blue Eyes system a waterfall software development model is used since it is suitable for
4

unrepeatable and explorative projects. During the course of the development UML standard notations are used. They facilitate communication between team members; all the ideas are clearly expressed by means of various diagrams, which is a sound base for further development. The results of the functional design phase were documented on use case diagrams. During the low-level design stage the whole system is divided into modules. Each of them has an independent, well-defined functional interface providing precise description of the services offered to the other modules. All the interfaces are documented on UML class interaction- and state diagrams. At this point each of the modules can be assigned to a team member, implemented and tested in parallel. The last stage of the project is the integrated system testing.

3.2. Design Objectives If such a system is to be practically or commercially implemented into practice then the following criterions should

be taken care of when the system is designed to be served in a particular context. A mobile Data Acquisition device should be configured upon a legitimate operator of the system. The range and connections of the two communicating Bluetooth modules should also be taken into consideration. A reliable time-buffering, processing and recording mechanism should be configured at the right location and it should also fall within the range of the Bluetooth transmitting device so that it can receive the signals and data at the optimal level. There should also be a mechanism for the customary analysation of the data that is received by the Bluetooth module. The system design implementation should also ensure a possibility of distributing the processing among a number of computers.

2.

SYSTEM OVERVIEW

Blue Eyes system provides technical means for monitoring and recording the operators basic physiological parameters. The most important parameter is saccadic activity, which enables the system to monitor the status of the operators
6

visual attention along with head acceleration, which accompanies large displacement of the visual axis. Complex industrial environment can create a danger of exposing the operator to toxic substances, which can affect his cardiac, circulatory and pulmonary systems. Thus, on the grounds of plethysmographic signal taken from the forehead skin surface, the system computes heart beat rate and blood oxygenation. The Blue Eyes system checks above parameters against abnormal (e.g. a low level of blood oxygenation or a high pulse rate) or undesirable (e.g. a longer period of lowered visual attention) values and triggers user-defined alarms when necessary. Our system consists of a mobile measuring device and a central analytical system. The mobile device is integrated with Bluetooth module providing wireless interface between sensors worn by the operator and the central unit. ID cards assigned to each of the operators and adequate user profiles on the central unit side provide necessary data personalization so different people can use a single mobile device called DAU Data Acquisition Unit. The overall system diagram is shown in Figure 1.

Data Acquisition Unit

Central System Unit

Database Blueto oth device Connectio n manager module Data logger modul e Visuali zation modul e

Voice interfa ce

Data analysis module

8051 micro control ler Physiol ogical parame ters

Blueto oth device

Fig 1: Overall System Diagram

The tasks of the mobile Data Acquisition Unit are to maintain Bluetooth connections, to get information from the sensor and sending it over the wireless connection, to deliver the alarm messages sent from the Central System Unit to the operator and handle personalized ID cards. Central System Unit maintains the other side of the Bluetooth connection, buffers incoming sensor data, performs on-line data analysis, records the conclusions for further exploration and provides visualization interface.

4.1 DATA ACQUSITION UNIT (DAU) HARDWARE:Atmel 89C52 microcontroller is said to be the core of the Data Acquisition Unit since it is a well-established industrial
8

standard and provides necessary functionality (i.e. high speed serial port) at a low price. EYE MOTILITY ANALYSIS:-

Fig: Eye Motility Analysis

Eye motility analysis means that the eye of the operator is been monitored and analyzed for the current conditions in it. The camera fixed in the Jazz Multi-sensor is used to measure and monitor the conditions in the eye (specifically
9

retina). The camera monitors the retinal structure and reports its conditions to the Jazz Multi-sensor.

Fig: Jazz Multi-Sensor

The second component of the Data Acquisition Unit is the Jazz Multi-sensor that is to be mounted on the head of the operator. This multi-sensor will sense from the forehead of the operator and then send the plethysmographic signals to the
10

Bluetooth module of the DAU for its transmission to the Central System Unit (CSU) for processing. The Jazz Multi-sensor, the name itself indicates that it is used to sense or monitor multiple parameters. It is an Eye position measuring component of the DAU. Through direct infrared oculography, it measures the movement of the macula of the eyes and accordingly interprets them and passes this to the microcontroller for processing. It also measures the Oxyhemoglobin and de-Oxyhemoglobin of operator that is used to measure the pulse rate, blood oxygenation of the operator. It has an two axial accelerometer that is used to detect the operators position (standing, lying). It also has an Ambient light sensor to sense the light coming from the external part of the macula or the eyes. All these physiological conditions that are monitored by the Jazz Multi-sensor are then passed to the Data Acquisition Unit. The DAU in turn transmits it to the CSU, where the processing or monitoring of the acquired data is done. An interesting fact to be noted about the Jazz Multi-sensor id that it can be worn by any operator (identification is done later) and spectacles can also be worn by the operator while working with the system. It makes no difference to the Multisensor.

Features Of DAU:The unit is very light in weight. Since it comprises only of the Jazz Multi-sensor, the Atmel microcontroller
11

89C52 and the Bluetooth module that transmits or transfers the data or signals to and from the Central System Unit (CSU). The power that is required by the Data Acquisition Unit for its operation is supplied by the batteries that are mounted on the headset that is to be worn by the operator. The unit has a very simple functionality and also it is easy to use by a common operator. It is simply to be worn by the operator, not to look after it. It also does not disturb the operator in his work, neither physically or morally.

This unit takes care in regard to authentication also. It only allows access to the legitimate users or operators of the system. This is accomplished by making the use of personalized ID cards that are to be inserted in the ID card interface.

4.2 CENTRAL SYSTEM UNIT (CSU) Introductory Modules:CSU software is located on the delivered Toshiba laptop, in case of larger resource demands the processing can be
12

distributed among a number of nodes. In this section we describe the four main CSU modules. The various modules that are included in this unit are:

Connection Manager Module

Data Analysis Module Data Logger Module

Visualization Module.

Connection Manager Module:Connection Managers main task is to perform low-level Bluetooth communication using Host Controller Interface commands. It is designed to cooperate with all available Bluetooth devices in order to support roaming additionally, Connection Manager authorizes operators, manages their sessions, demultiplexes and buffers raw physiological data.

Bluetooth Connection Manager is responsible for establishing and maintaining connections using all available Bluetooth devices. It periodically inquires new devices in an operating range and checks whether they are registered in the system database. Only with those devices the Connection Manager will communicate.

13

Data Analysis Module:The module performs the analysis of the raw sensor data in order to obtain information about the operators physiological condition. The separately running Data Analysis Module supervises each of the working operators. The module consists of a number of smaller analyzers extracting different types of information. Each of the analyzers registers at the appropriate Operator Manager or another analyzer as a data consumer and, acting as a producer, provides the results of the analysis. The computed features can be e.g. the operators position (standing, walking and lying) or whether his eyes are closed or opened. Data Logger Module:The module provides support for storing the monitored data in order to enable the supervisor to reconstruct and analyze the course of the operators duty. The module registers as a consumer of the data to be stored in the database.

Each working operators data is recorded by a separate instance of the Data Logger. Apart from the raw or processed physiological data, alerts and operators voice are stored. The voice data is delivered by a Voice Data Acquisition module. Visualization Module:-

14

The module provides user interface for the supervisors. It enables them to watch each of the working operators physiological condition along with a preview of selected video source and his related sound stream. Watching all the recorded physiological parameters, alarms, video and audio data the supervisor is able to reconstruct the course of the selected operators duty. Features Of CSU: The Bluetooth connection management of the two units.

Data processing that has been acquired from the mobile device or the Data Acquisition Unit. Visualization of the overall data acquired into an appropriate format. Access verification of an operator, by matching his data with that in systems database.

2. SECURITY AND APPLICATION AREAS

5.1 Security of Data in System:There are various levels of security perseverance in the system. They can be enlisted as follows:

15

 Only registered mobile devices can connect to the system by the means of unique ID cards maintained by each individual operator. Whenever a Bluetooth connection is to be established then also there is an authentication procedure followed by the system to allow access to only those operators. Bluetooth connection encryption is maintained in the system for authentication purposes. Application areas of System:-

5.1

Blue Eyes System can be applied in every working environment requiring permanent operator's attention:

The system can be used at the power plant control rooms. It can also be implemented in the flight control centres. In Medical Application.

5 MERITS AND DEMERITS

Merits:Implementation and engineering considerations THREE groups of users:

Operator
16

Supervisor System Administrators

Demerits:The prototype built has several limitations, which are not the result of the project deficiency but are rather caused by the constraints imposed by the Project Kit and small budget. In the commercial release the USB web-cam should be replaced by an industrial camera, connected to a capturing device. The use of such a camera would lessen CPU load and improve the video signal quality. Since the Bluetooth module does not support redirecting of the voice SCO connections data to the serial port (which is a part of the Bluetooth specification) PCM interface is built on the central system side similar to the one used in the DAU. This makes the Voice Data Acquisition module receive the sound using the sound card.

5 A GENERIC TOUCH TO FUTURE OF SYSTEM Future Enhancements in this System:There lies an immense scope in the improvement of this system. Rather the commercial implementation of this system holds much ground in this dynamic era of computer sciences.

17

 The use of a miniature CMOS camera integrated into the eye movement sensor will enable the system to calculate the point of gaze and observe what the operator is actually looking at. Introducing voice recognition algorithm will facilitate the communication between the operator and the central system and simplify the authorization process.

Assuming the operator is a driver and the supervised process is car driving it is possible to build a simpler embedded on-line system, which will only monitor conscious brain involvement and warn when necessary.

As in this case the logging module is redundant, and the Bluetooth technology is becoming more and more popular, the commercial implementation of such a system would be relatively inexpensive.

5 CONCLUSION
Starting with the name of the system, Blue Eyes emphasizes Bluetooth technology and the movements of the eyes. Bluetooth provides reliable wireless communication whereas the eye movements enable us to obtain a lot of interesting and important information using the eye motility analysis. The Blue Eyes system is developed because of the need for a real-time monitoring system for a human operator. The
18

approach is innovative since it helps supervise the operator not the process, as it is in presently available solutions. The system in its commercial release will help avoid potential threats resulting from human errors, such as weariness, oversight, tiredness or temporal indisposition. However, the prototype developed is a good estimation of the possibilities of the final product. No doubt, there are some lacunas in the system, but these are present only because the system is developed on the project basis. This is done only to limit the cost of the system. The commercial implementation of the system can resolve the lacunas. In recent years, Bluetooth technology is becoming more and more popular; the commercial implementation of such a system would be relatively inexpensive. And if a large scale implementation is done, then such a system will definitely prove to be a boon for the human race in course of time.

6 REFERENCES
BOOKS:

8051 Microcontroller and Embedded Systems (For Basics of 8051 family microcontrollers and Serial Communication Principles)

Wireless Technology Manual 4.5 (For Basics of Bluetooth Technology)

WEBSITES:19

www.google.com www.bluetooth.com www.poznan.net www.bluemax.com/eyessafe www.wikipedia.org

20