BRAIN CONTROLLER INTERFACE FOR HOME

AUTOMATED SYSTEM

THUM WEI SEONG

UNIVERSITI MALAYSIA PAHANG

 BRAIN CONTROLLER INTERFACE FOR HOME AUTOMATED SYSTEM

THUM WEI SEONG

This thesis is submitted as partial fulfilment of the requirements for the award of the
Bachelor of Electrical Engineering (Hons.) (Electronics)

Faculty of Electrical & Electronics Engineering

Universiti Malaysia Pahang

JUNE 2016
 UNIVERSITI MALAYSIA PAHANG

DECLARATION OF THESIS AND COPYRIGHT

Author’s Full Name : THUM WEI SEONG

Identification Card No : 910619-08-6067
Title : BRAIN CONTROLLER INTERFACE FOR
HOME AUTOMATED SYSTEM
Academic Session : SEMESTER II 2015/2016

I declare that this thesis is classified as:

(Contains confidential information under the
CONFIDENTIAL Official Secret Act 1972)

(Contains restricted information as specified

by the organization where research was
RESTRICTED
done)*

I agree that my thesis to be published as

/\ OPEN ACCESS online open access (Full text)

I acknowledge that Universiti Malaysia Pahang reserve the right as follows:

1. The Thesis is the Property of University Malaysia Pahang.

2. The Library of University Malaysia Pahang has the right to make copies for the purpose
of research only.
3. The Library has the right to make copies of the thesis for academic exchange .

Certified by:

_____________________________ _____________________________

THUM WEI SEONG DR.MOHD ZAMRI BIN IBRAHIM

Date: 27/05/2016 Date: 27/05/2016

 iii
ii

SUPERVISOR’S DECLARATION

I hereby declare that I have checked this thesis and in my opinion, this thesis is
adequate in terms of scope and quality for the award of the degree of the Bachelor
Degree of Electrical Engineering (Hons.) (Electronics).

Signature :
Name of Supervisor : DR. MOHD ZAMRI BIN IBRAHIM
Position : SENIOR LECTURER
Date : 27/05/2016
 iii
ii

STUDENT’S DECLARATION

I hereby declare that the work in this thesis is my own except for quotations and
summaries which have been duly acknowledged. The thesis has not been accepted for
any degree and is not concurrently submitted for award of other degree.

Signature :
Name : THUM WEI SEONG
ID Number : EA12056
Date : 27/05/2016
 vi
66
6

ACKNOWLEDGMENT

This research was supported by Faculty of Electrical & Electronic Engineering,

University Malaysia Pahang.

I would like to express my sincere gratitude to my supervisor Dr Mohd Zamri

Bin Ibrahim for the continuous support of my project and related research, for his
patience, motivation, and immense knowledge. His guidance helped me in all the time
of research and writing this thesis. I could not have imagined having a better adv isor
and mentor for my research.

I would like to show my gratitude to Esther Ling, Edu Engineer from

TechSource S ystems for sharing her pearls of wisdom with me during the training
session provided by TalentCorp Malaysia.

I thank my fellow friends and course mates for helping me to do the data
analysis. Thank you for supporting me spiritually and cheer me up always.

Last but not the least, I would like to thank my family: my parents and to my
brothers for supporting me spiritually throughout writing this thesis and my life for
general.
 vi
77
7

ASTRACT

In year 2016, the population number of Malaysia was expanded to 30 572 466
peoples. This is an increasing of 1.51 % (455 964 people) compared with population
of 30 116 502 in year 2015. At the beginning o f 2016, total number of older people
with age above 64 years old is 1,520,369 (5% of total population). While in December
2012, there were total 359,203 disabled people under registration with Department of
Social Welfare. They all have trouble in doing da ily stuff and they hard to switch on
some electronic device too. The aim of this project is to use Brain-Computer Interface
(BCI) systems to interpret users’ intentions into command to control home automated
systems. This BCI system enable brain signals to activate or turn on device by using
NeuroSky MindWave Mobile that utilizes Electroencephalograms (EEGs) used to
decode human intentions. This BCI system is developed by using MATLAB graphical
user interface (GUI). Then, IP Webcam is used for live video streaming and then
undergo image processing using MATLAB to set the path where those command
should direct to. As for the BCI system output, Arduino UNO will be used as output
controller to activate electronic devices. Besides that, data analysis on the user
concentration based with gender and condition is conducted to test the impact of
different conditions on people concentration performance. Condition for this data
analysis are relax, happy and stress condition. Based on the results of data analysis,
people in happy condition are able to concentrate 36.67% faster than people in stress
condition. People in relax condition able to concentrate much faster which is 73.33%
compare to people in stress condition. By having this BC I system, conventional light
and switch can now be easily replaced and activated by just mind concentration of
user. This can act as cognitive tool to offset the effects of aging and alternative
method for disable people to control electronic devise at home.
 888
88

ABSTRAK

Di Malaysia, jumlah orang tua dari usia 50 tahun ke atas adalah lebih daripada
17% pada tahun 2011 dan trend ini semakin meningkat lebih cepat daripada jumlah
penduduk. Penduduk Malaysia telah mencapai 26.640.000 dalam tahun 2006,
bagaimanapun orang yang kurang upaya adalah berjumlah 197.519. Mereka
mempunyai masalah dalam melakukan kerja harian dan sukar untuk menghidupkan
peranti elektronik juga. Tujuan projek ini adalah untuk menggunakan Brain-Computer
Interface (BCI) untuk mentafsir isyarat otak pengguna ke dalam arahan untuk
mengawal sistem automatik rumah. S istem BCI ini membolehkan isyarat otak untuk
mengaktifkan atau menghidupkan peranti dengan menggunakan NeuroSky
MindWave Mobile yang menggunakan Electroencephalogram (EEGs) yang
digunakan untuk memperoleh tahap tumpuan manusia. S istem BCI ini dibangunkan
dengan menggunakan MATLAB Graphical User Interface (GUI). Kemudian, IP
Webcam digunakan untuk live video secara langsung dan menjalani pemprosesan
imej menggunakan MATLAB untuk mene ntukan laluan isyarat. Bagi output sistem
BCI, Arduino UNO akan digunakan sebagai pengawal output untuk mengaktifkan
peranti elektronik. Selain itu, analisis data kepada tumpuan pengguna berasaskan
dengan jantina dan keadaan dijalankan untuk me nguji kesan keadaan yang berbeza
kepada prestasi tumpuan. Keadaan untuk analisis data ini adalah relaks, gembira dan
tekanan. Berdasarkan hasil analisis data, orang dalam keadaan gembira dapat
menumpukan perhatian 36.67% lebih cepat daripada orang berada dalam keadaan
tekanan. Orang dalam keadaan relaks dapat menumpukan perhatian lebih cepat iaitu
73.33% berbanding dengan orang berada dalam keadaan tekanan. Dengan adanya
sistem BCI ini, cahaya konvensional dan suis kini boleh dengan mudah digantikan
dan diaktifkan dengan hanya penumpuan fikiran pengguna.
 999
99

TABLE OF CONTENTS

Page
SUPERVISOR’S DECLARATION iii
STUDENT’S DECLARATION iv
ACKNOWLEDGEMENT vi
ABSTRACT vii
ABSTRAK viii
TABLE OF CONTENTS ix
LIST OF TABLES xi
LIST OF FIGURES xii
LIST OF ABBREVIATIONS xv

CHAPTER 1 INTRODUCTION

1.1 Background Study 1

1.2 Problem Statement 3
1.3 Project Objective 3
1.4 Project Scope 4
1.5 Project Outline 5

CHAPTER 2 LITERATURE REVIEW

2.1 Introduction 6
2.2 Research of Invasive BCI 7
2.3 Research of Non-Invasive BCI 8
2.4 EEG brain wave reading headsets 8
2.4.1 Emotiv EPOC Headset 10
2.4.2 Melon Headband 11
2.4.3 NeuroSky MindWave Mobile Headset 12
2.4.4 Comparison 13
2.5 Previous works 13
2.6 Summary 20
 vi1
0

CHAPTER 3 METHODOLOGY

3.1 Introduction 21
3.2 Architecture 21
3.3 Hardware 22
3.3.1 NeuroSky MindWave Mobile headset 23
3.3.2 IP Webcam 26
3.3.3 Arduino UNO 28
3.3.4 USB Lamp 29
3.4 Software 29
3.4.1 MATLAB 29
3.4.2 Music Player 31
3.5 Methodology 31
3.6 Summary 35

CHAPTER 4 RESULTS AND DATA ANALYSIS

4.1 Introduction 36
4.2 Experimental Setup 36
4.2.1 Relax Condition 36
4.2.2 Happy Condition 38
4.2.3 Stress Condition 40
4.3 Output Result 42
4.4 Summary 49

CHAPTER 5 CONCLUSION AND RECOMMENDATION

5.1 Conclusion 50
5.2 Recommendation 51

REFERENCES 52
 vi1
1

LIST OF TABLE

Table No. Title Page

2.1 Comparison of different EEG headset 13

2.2 Average Players performance on HK and SK 15

3.1 Indicator light of MindWave state condition 24

3.2 Frequency range of different brainwave types 25

3.3 Parameter used in CHT technique 33

 77
77
77
7

LIST OF FIGURE

Figure No. Title Page

2.1 Electrode cap with electrodes placed on users head 7

2.2 Labels for points of electrode placement system 8

2.3 Emotiv EPOC headset 9

2.4 Melon Headband 10

2.5 NeuroSky Mindwave Mobile Headset 11

2.6 Nokia E72 used for HK (a) and Nokia N8 used for SK (b) 13

2.7 Value of brainwaves acquired from MindWave Mobile 15

device

2.8 The software architecture for ReadGoGo system 16

2.9 The user is reading on the computer’s screen with 17

Mindwave Mobile headset

2.10 Electrode positions (C3 and Cz) for EEG measurements 18

2.11 Depicts the hardware and software architecture of the 18

portable BCI system

3.1 Architecture of BCI system 22

3.2 Block diagram of BCI system 22

3.3 NeuroSky Mindwave Mobile headset 23

3.4 Complete setup of Mindwave Mobile headset on human 25

head

3.5 Default port setting of NeuroSky MindWave Mobile 26

Headset

3.6 IP Webcam android apps at Google Play 26

 88
88
88
8

3.7 Monitor of video live streaming in mobile device 27

3.8 Arduino UNO microcontroller board 28

3.9 Properties of Arduino UNO connected to computer 28

3.10 Flow chart of BCI system 30

3.11 Bluetooth Setting for data transmission of MindWave 31

Mobile headset

3.12 Voting mode: Range of rmax and rmin along direction of 33

gradient

3.13 Hardware circuit connection diagram 34

4.1 Facial expression of participants in relax condition 37

4.2 Attention value of participant in relax condition 37

4.3 Time taken for attention value to reach 80 after relax 38

condition

4.4 Facial expression of participants in happy condition 39

4.5 Attention value of participant in happy condition 39

4.6 Time taken for attention value to reach 80 after happy 40

condition

4.7 Facial expression of participants in stress condition 40

4.8 Attention value of participant in stress condition 40

4.9 Attention value of participant to reach 80 after stress 41

condition

4.10 Output of Attention value (a) raw data and (b) smooth data 42

4.11 Log in page of BCI system 43

4.12 Main page of BCI system 43

 99

4.13 Real-time graph of Attention value and video live 44

streaming

4.14 Difference between RGB (a), Grayscale image (b) and 46

Circular object detected from grayscale image (c)

4.15 Original image (a), red object extracted on original image 47

(b) and red circle object is detected (c)

4.16 Output result of green circular object detected with 48

attention value higher than 80

4.17 Output result of red circular object detected with attention 48

value higher than 80
 11
00

LIST OF ABBREVIATION

BCI Brain Controller Interface

EEG Electroencephalography

EMG Electromyography

EOG Electrooptigraphy

EPOC Emotiv Insight earlier version

USB Universal Serial Bus

ADHD Attention Deficit Hyperactive Disorder

PWM Pulse Width Modulation

IDE Integrated Development Environment

COM Communication Port

CHT Circular Hough Transform

GUI Graphical User Interface

WMP Window Media Player

HK Hard Key

SK Soft Key
 CHAPTER 1

INTRODUCTION

1.1 BACKGROUND STUDY

An English physician Richard Caton was discovered presence of electrical

activity in the brain in 1875s [1]. In that moment he was presented his finding about the
electrical activity of the exposed cerebral hemispheres of rabbits and monkeys in British
in Liverpool. In 1890s, physiologist Adolf Deck started studies the electrical activity of
animal brain. He placed electrodes directly on the surface of brain for sensory
stimulation testing and this result leading to the conclusion of brain waves[2].

Brain’s electrical activity measured by a German neurologist Hans Berger used

his ordinary radio equipment to magnify it on human scalp and then
electroencephalogram was invented [3].Then it used as a method of analysis of brain
functions in health and disease. In 1929s, the first report on electrical brain activity in
human published and this enabled scientists to watch the brain action directly.
Subsequently, Berger’s research propagated the development encephalographic (EEG)
technologies, which successively led to Walter’s development of the first Brain
Computer Interface (BCI) device in 1964[4].

BCI system enabled direct communication between human brain and external
device. An early research of BCI began in the 1970s at University of California, Los
Angeles (UC LA)[5]. The papers published after this research also mark the very first
appearance of the expression BCI system in scientific literature. N icolelis, C hapin, and
theirs colleagues did a lot of laboratories have managed to record signals from monkey
cerebral cortices to operate BCI to produce moveme nt in early 2000s. Monkeys been
 22

taught to navigate computer cursors on screen and commanded robotic arms to

grab a piece of fruit to their mouth. But this system had limitation of that only work
while monkey move its arm to produce the correct brain signal[6][7].
Several signal acquisition modalities are currently used for BCI operation in
human and non-human primates. Human BCI systems differentiate into three types
which is invasive, partially invasive and non- invasive. Invasive BCI is that using
electrode directly implanted into the grey matter of human brain. This way can produces
high quality of signals, but it having risk to death if there any mistake during operation.
Partially invasive BCI is that electrodes placed on the surface of cortex, it picks up
neural firing from the surface of the brain. Then, non-invasive BCI have electrode
carefully placed on a cap that the patient wear. It has no complications of brain surgery,
so it is safer compared to invasive and partially invasive.

Electroencephalography (EEG) is the most studied non- invasive interface.

Mainly due to it is ease of use, portability and low set-up cost. But, the technology is
susceptible to noise so the brainwave signals become more difficult due to the
originality data. Lawrence Farwell and Emanuel Donchin developed an EEG-based
brain-computer interface in 1980s [8]. They record the brainwave response of one
paralyzed Locked- in syndrome patient to communicate words, letters and simple
command to computer and enable them to speak through a speech synthesizer driven by
computer. In 2000s, research by Jessica Bayliss at the University of Rocherster showed
that volunteers which wearing virtual reality headset could control elements in virtual
world including turning lights on and off. Then, an EEG based brain-computer interface
has been pursued extensively by a number of research labs, recent advancement made
by Bin He and his team at the University of Minnesota suggests the potential of an EEG
based brain-computer interface to accomplish tasks close to invasive brain-computer
interface [9]. They demonstrated the ability of a non- invasive EEG based brain-
computer interface to control the flight of a virtual helicopter in 3-dimensional space.
 33

1.2 PROBLEM STATEMENT

In year 2016, the population number of Malaysia was expanded to 30 572 466
peoples. This is an increasing of 1.51 % (455 964 people) compared with population of
30 116 502 in year 2015. At the beginning of 2016, total number of older people with
age above 64 years old is 1,520,369 (5% of total population) [10]. While in December
2012, there were total 359,203 disabled people under registration with Department of
Social Welfare. S ince this registration is under volunteers based, so the actual number
of it ought to be much higher than that. They all experiencing difficulty in doing daily
stuff and they even unable switch on the light without anyone else since they are
generally reliant on their relatives.

By using EEG-based brain-computer interface can be help people with

inabilities to control home automated devices, such as turn on the light or other devices
with brain activity. But then the brainwave data is very fuzzy and hard to understand.
Furthermore, the right codes are needed be programmed to analysis those data for the
desired action. Extensive training is required to get the exact correct data for an action.

1.3 PROJECT OBJECTIVE

There are several objectives need to be achieved of this project which is:

i. To interpret and stabilize EEG signal acquired from human brain

ii. To develop a BCI system that utilizes image processing and brain signal
processing control.
iii. To analysis brainwave signal concentration based on gender and condition.
 44

1.4 PROJECT SCOPE

This project will be involved data acquisition of brainwave signals, filtering

brainwave into desired category, training process and data analysis of brainwave signal
based on different condition. BCI system will be developed to test the process mention
previously. The whole BCI system will include NeuroSky Mindwave Mobile headset,
IP Webcam, MATLAB, and Arduino UNO board.

Data acquisition will be conducted using NeuroSky Mindwave Mobile.

Brainwave signal will be acquired from human brain using EEG sensor on Mindwave
Mobile headset. The outputs of MindWave Mobile are raw EEG data as well as the
eSense™ (i.e. Attention, Meditation). The output raw data is fuzzy and unstable for
testing analysis. Some technique will be applied to stabil ize the data for a better data
analysis.

For processing part, after the stabilizing of brain signal output, it will
synchronize with the object detect by IP camera. IP Webcam is used for video live
streaming and transmits it into MATLAB. Video will be display in MATLAB and
doing image processing on the video frame by frame. The activation of hardware output
device will be controlled by Arduino UNO board. It will be reacted according to the
object detected and activation command will transmit from MATLAB.

Furthermore, data analysis on brainwave signal based on different conditions

will be carrying out to test the impact on user’s concentration. Condition set for this
project is relax, happy and stress condition. Different gender of participants will be
invited for this data analysis to prove the difference of mind concentration between
male and female based on different condition.
 55

1.5 PROJECT OUTLINE

The next chapter deals with the literature review on the research on types of BCI
system and previous works that applied BCI system. C hapter 3 shows the hardware and
software involved in developing the BCI system. This chapter also list out the technique
on the brain signal and image processing. Chapter 4 presents the result of this project
and data analysis on human concentration according gender and condition based. Last
chapter discussed the conclusion and future improvement of this project.
 CHAPTER 2

LITERATURE REVIEW

2.1 INTRODUCTION

This chapter will discuss the previous work done by other research. All the
technique to acquire EEG signal from human brain and effect on brain concentration
based on different condition will be further discuss in this chapter. This project is done
based on the problem encounter on the previous work and to improve the current BCI
system. A BCI system is a new communication channel between the human brain and a
digital computer. The goal of BCI system technology was to provide control capacities
to people with severe motor disabilities. An electroencephalogram (EEG) based brain-
computer interface was connected with a Virtual Reality system in order to control a
smart home application[11]. A typical BCI system was composed by the following
blocks which are data acquisition system, signal processing system and commands sent
to an application. For the acquisition system, the most frequently used for recording
method is EEG. Brain activity is recorded from the surface of scalp, so this method is
non- invasive with advantage of portability of the recording system[12]. The signal
processing block processes all the recorded data and transforms the signals in
commands for the application. The system must be fast enough to process the data
record to achieved real-time operation. It also needed to ensure that the extracted
features are not contaminated by electromyography (EMG), electrooculography (EOG)
or some other non-central nervous system artefacts.
 77

2.2 RESEARCH OF INVASIVE BCI

For invasive BCI, an array of electrodes is planted in the grey matter of the
patient’s brain during a surgery. Those electrodes are directly connected to neurons.
By this way, it offer a high quality for recorded signals of BCI devices, but the
problem is that the body may react to the foreign objects in the brain, creating the
possibility of scar-tissue build- up over the implanted device. Recognition of different
patterns of signals recorded from motor cortex neurons is the key in controlling
robotic arm or neuroprostheses [13]. Direct brain implants were used to treat
blindness. Scientist William Dobelle designed a working brain interface prototype in
1978s. The prototype was implanted into the visual cortex of man blinded in
adulthood. The single-array BCI contained 68 electrodes and the prototype succeeded
in producing phonophenes, the sensation of seeing light without light actually entering
the eye.

Another area of BCI is focusing on motor neuroprothetics. The aim of these BCI
is to either restore movement of individuals with paralysis or provide some special
devices to assist them. Special devices can be different types of interfaces with
computer or even robot arms for control. Previous studies on monkeys (Macaca
mulatta) proved that invasive methods are suitable for creating a prosthetic robot-arm
for control [14]. Natural arm movements can be recorded in populations of neurons
from motor cortex [15]. Moreover, these studies proved that continued training over of
the executed task. In addition, these studies revealed the possibility to develop real-time
controlling systems such as robot arm controlling system. To conclude it, invasive BCI
are currently much more suitable for real- time applications. Recent tests on monkeys
proved that, robotic arms and prostheses can be controlled n real-time with invasive
BCI. Also invasive on humans are a success, one patient can control a cursor on a
monitor, thus the subject can operate a computer only by his thoughts. But, the biggest
problem is the high costing of the required special electrodes array. A special expensive
surgery need to be done to implant those electrodes in the patient’s brain. The durabilit y
of electrodes may decrease, although the patient’s brain is not affected. Scar-tissues
might grow near to the electrodes, then the quality of recorded signal can be altered or it
can be even stop record signals.
 88

2.3 RESEARCH OF NON-INVASIVE BCI

Most non-invasive BCI systems are use electroencephalogram (EEG) signals.

Electrical brain activity being recorded from the electrodes placed on the scalp. The
main source of EEG is the synchronous activity of thousands pf cortical neurons [16].
Measuring EEG is a most simple non- invasive way to monitor electrical brain activity.
Signals recorded in this way are used for restoring the movement of patients, controlling
different devices, writing just by thoughts. But the signal resolution is poor compared to
invasive method.

Figure 2.1: Electrode cap with electrodes placed on users head [1]
 99

Figure 2.2: Labels for points of electrode placement system [1]

Moreover, it is characterized by small signal amplitudes and noisy

measurements. A common protocol used for non-invasive BCI is related to P300
evoked potentials. Most applications based on P300 use a paradigm in which the user
faces a screen that may contain letters, numbers or different commands. Each symbol
flashes for a number of times chosen before. The user makes a selection by counting
each time the symbol flashes. In order to decide which symbol was chosen by the user,
different algorithms are used. Usually, P300 applications do not require initial training
for the user. The typical communication rate is of about one word (i.e. 5 -6 letters) per
minute, but the improvement in the ability to select letters faster triggers an increase in
the communication rate.

Non-invasive systems are much cheaper than invasive ones, but the main problems
raised are related to the computational response time fro m the system. While real-time
applications are required for locked- in patients, the non-invasive methods cannot yet
meet such a requirement. Also, these systems offer a very good communication channel
for locked-in patients and can be used on a larger scale because of the reduced price.
Although the patient cannot generate a message very fast, the BCI still allows
communication with other people, which is very important for patients who cannot even
move their eyes for answering questions.
 10
1
0

2.4 EEG BRAIN WAVE READING HEADSETS

This section will explain in detail and discuss on the training method used for
EEG- based brain-computer interface. Based on previous work and latest research, there
are several types of devices that able to measure brain wave. At here, there are some
devices being picked to make comparison to determine the most suitable for my project.

2.4.1 EMOTIV EPOC Headset

Figure 2.3: Emotiv EPOC headset [17]

Emotiv EPOC EEG headset is a sleek, 14-channel, wireless headset that records
brains wave activities and translate them into data for analysis. This device can capture
emotions and thoughts in mind and transfer into readable codes. Emotiv Insight earlier
version (EPOC) was released in 21 st December 2009. It is very comfortable due to its
lightweight. The software that comes with the Emotiv headset provides the following
detection functionality such as various facial expressions, levels of engagement,
frustration, meditation, and excitement[18].This device gives full control of the external
devices that is connected with it. Usage of this device is to help people with disabilities
in their daily life. This device easy to set-up, so handicapped people will find it more
comfortable. On the device are EEG sensors that detect brain neurological activity,
which can be transferred to smartphone via Bluetooth. An Emotiv needed to plug in
 11
1
1

USB cable connected to laptop to run a calibration process. User need to relax for 10
seconds for the system to get the baseline reading. F urther, it is wireless technology
which allow user to carry around. But it requires applications that to be able to apply on
other device. The price of Emotiv Insight is quite expensive for public which cost $399
to $499.

2.4.2 MELON Headband

Figure 2.4: Melon Headband [19]

The Melon Headband is a wireless brain monitor that uses EEG to measure brain
activity. From the activity, the algorithm can able to detect user‘s level of focus,
relaxation, or meditation. It help user to understand and improve focus across activities
ranging from study to exercise. Starting, Berman and her co- founder Arye Barnehama
met as undergraduates studying cognitive and computer science at Pomona College and
doing research on EEG technology. They are using this on the treatment of ADHD and
dementia in a clinical setting. Melon Headband uses alpha and beta to calculate a neural
focus score and save those data in smartphone apps. These data can be tracked and
correlated to different tags to see the pattern of focus over time. It designed for low
power consumption and with Bluetooth 4.0 with the total cost of $149.00.
 12
1
2

2.4.3 NeuroSky MindWave Mobile Headset

Figure 2.5: NeuroSky Mindwave Mobile Headset [20]

The NeuroSkyMindWave Mobile Headset is a low cost single-electrode EEG

headset and it has been proven effective in detecting user’s mental state. NeuroSky’s
Mindwave Mobile headset as a minimally invasive method of measuring the attention
and meditation levels of a subject. It has a microchip which pre-process the EEG signal
and transmits that data via Bluetooth. It is the cheapest option as it comes with a big
library of apps and programs to run with. It even can pair it with the Pezzlebox orbit, a
brain-controlled helicopter. The meditation scale measured by the NeuroSky headset
reflects the level of relaxation of a subject, while for level of the subject’s attention was
a point of interest[21]. The headset is calculating the Raw EEG signals to produce the
based on an algorithm patented from Neurosky. The patterns of the electrical activity
are analysed with the help of specialized algorit hms by converting the EEG signals into
control commands[22]. But then, the values of attention and meditation recorded are
easily influenced by blinking[23]. NeuroSky MindWave Headset was the method which
mostly used for BCI system nowadays.
 13
1
3

2.4.4 Comparison

As overall, all EEG sensor headset having strengthens and weakness. By

comparing the EEG sensor headset above, the details of information of headsets
including cost and performance is compared to figure out the most suitable EEG headset
for this project.

Table 2.1 : Comparison of different EEG headset

Aspect Emotiv EPOC Melon headband NeuroSky

MindWave mobile
Sensor 3 mental states, 13 conscious Left and right 2 mental states,
interpretation thoughts, facial expressions, hemisphere eyeblinks
head movement prefrontal EEG
Electrodes 14 4 1
Interface Bluetooth Bluetooth Bluetooth
Cost $399 - $499 $149 $99.45

According to the comparison Table 2.1 above, NeuroSky MindWave is more

suitable for my project. This research is only required 1 mental state which is attention
value. As of its comfortable outfit that co nsist of only 1 electrode. Wireless connection
such as Bluetooth connection is required for this project. The costing of this headset is
also reasonable and believed it can achieve the highest performance for my project.

2.5 PREVIOUS WORKS

There is some research on changes of EEG signal based on some condition. The
terms of attention and meditation level are authored by Neurosky EEG information that
comprises of interpretation through eSense meter (Attention and Meditation).There is a
research that quantify the impacts of input gadgets have on player experience through
attention and meditation levels utilizing biosensors estimations, for example, EEG and
its connection to portable hard key (HK) and soft key (SK) input.
 14
1
4

(a)

(b)

Figure 2.6: Nokia E72 used for HK (a) and Nokia N8 used for SK (b) [24]

There was 10 participants were all are gamers but they ne ver played the game
prepared before. Among of those 10 participants, there are 5 female and 5 male
undergraduate students at the age group of 19-22 years old. Before the testing began,
game play and its mechanics were explained. Members were then be given 1-2 minutes
to acclimate themselves with the given cell phones. When members were prepared, Video
of their game play, levels of attention and meditation is recorded. Every gaming sessions
conducted around 8-10 minutes while the overall testing session took around
15-20 minutes. Table 2.2 displays an overall effect of general players' attention and
meditation level for arcade game on HK and SK. For the game play on first level, the
 15
1
5

finishing time for HK was quicker in contrast with SK. So also, more error rate was
stand up to on SK input.

Table 2.2: Average Players performance on HK and SK

Game Arcade Game

Mobile Input Devices HK SK
Time (seconds) 28,9 34.8
Attention (out of 100 scores) 41.26 42.9
Meditation (out of 100 scores) 46.42 44.31
Error rate 1 2

The conclusion on this research is that for HK's attention level, player appears to
have lower attention rate contrasted with SK, while HK meditation rate is higher in
correlation with SK. This outcome proposes that players required more focus on SK, but
they are ready to play more on HK with less stress. This could be to a great extent
because of HK have kinesthetic criticism as players could touch and feel the catch
physically in compare with SK. For SK, the players will focus more on the keypad and
on the screen as they rely on upon their visual methodology more for SK in examination
with HK [24]. As overall, the attention and meditation value will be differ according to
different type of input device, but this research unable to prove that attention and
meditation value do not effected by users’ emotion.

There was another research that about to propose a helpful BCI system, which
sets a proper attention value and eyes blinking to use and activate simple cell phone. By
accomplishing focus state over predefined level, the user is able to perform essential
operations like telephone call, sending SMS, email with predefined content or play
voice message [25].
 16
1
6

Figure 2.7: Value of brainwaves acquired from MindWave Mobile device [25]

EEG Analyzer acquired information of attention and meditation level from all
time. Recorded attention level was mostly around 40% to 50% under normal condition
user monitoring a cell phone. When somebody concentrates on a particular action or
tries to be concentrated, the attention level is can be achieve to a high value of
concentration that higher than 70%. But then normally drained individ ual will have
issues with accomplishing attention level higher than 50% or 60%. Next, strength of eye
blinking quality differs somewhere around 38% and 60%, yet now and again can be
higher and its worth can be more than 80%.

The predefined value for this research is 75% of attention value, since it can be
achieved by a normal individual and it is not general focus level. Higher or lower center
level may bring about improper action with the client. Besides it, eye blinking is
counted as a controller input too. It used double blinking instead on single eyes blinking
due to double blinking is not a normal human behavior. S ingle blinking is just like
human regular blinking, so it may cause unwanted error happened. As overall, this
research is talking about how to fully control over one device, but it cannot be control
over more than one output device.

Next, there is ReadGoGo which is a framework developed to remind user when

they are not concentrating on the text given. The proposed framework can consequently
 17
1
7

remind the user when she/he is losing focus and eventually help the user to rapidly
recover his/her concentration on the current content.

Figure 2.8: The software architecture for ReadGoGo system [26]

There are 20 participants in this research, which are 10 male and 10 female in
the age group of 17-23 years old. This research used EEG sensor of NeuroSky
MindWave Mobile headset to acquire user attention value and a webcam to capture user
face to determine whether user is focusing the text on screen or not.
 18
1
8

Figure 2.9: The user is reading on the computer’s screen with Mindwave Mobile
headset [26]

This ReadGoGo! utilized two input sources which are from the webcam and
from EEG sensor, to figure out whether the user is concentrating on the text on the
computer’s screen [26]. The framework helps the user to rapidly re-concentrate on the
object after psyche meandering or physical distractions. As overall, this framework can
eventually help the user know about their concentration levels which thus help them to
re-concentrate again, but it do not provide the exact way for user how to re-concentrate.

There is a field study on BCI system conducted at Graz, Austria which total 99
healthy people in the age group of 16-60 years old participated in this study. This study
is to test the efficiency of its experimental testing. The EEG was recorded with gold
anodes from two bipolar channels over the right- hand and foot representation regions
(C3 and C z) [27]. The head is viewed from above and nose pointes to the top of the
page.
 19
1
9

Figure 2.10: Electrode positions (C3 and Cz) for EEG measurements. [27]

Figure 2.11: Depicts the hardware and software architecture of the portable BCI system
[27]

Participants will able to achieve the target if the brainwave acquired from EEG
sensor is match with the desired pattern. A large group outside the research center been
participated to test the efficiency to operate the BCI system with just 20 -30 mins of
 20
2
0

training by using only two bipolar EEG inductions. As overall, t here is around 80%
accuracy of the BCI system after 20-30mins of training, but this is only on two parts
which is right hand and foot. A specific point of human brain is required to acquire
exact EEG signal, it will cause error if getting the wrong position.

2.6 SUMMARY

Based on previous works, there are some testing on the effect on EEG signal
based on difference factors, but there do not have a testing on mental states on human
being that affect the concentration. There were BCI system developed to remind user to
concentrate, but do not provide a proper guide for user how to re-concentrate. The more
channels of electrode needed for brainwave detection may cause more noise. Wrong
position of electrode will caused failure in BCI system. By having the comparison of
difference EEG headset, NeuroSky MindWave Mobile headset is more suitable for my
project as there is only one channel of electrode needed and low costing.
 CHAPTER 3

METHODOLOGY

3.1 INTRODUCTION

This chapter will discuss the detail steps from beginning with data acquisition,
pre-processing, processing, and final with output parts of this BCI system. All the
software and hardware parts involved are NeuroSky MindWave Moble headset, IP
Webcam, Arduino UNO board, US B Lamp, and MATLAB. BCI system will be
developed by utilized all the software and hardware parts.

3.2 ARCHITECTURE

In this project, NeuroSky MindWave Mobile headset will be collecting

brainwave signal from user brainwave. This headset having s ingle dry electrode with 2
mental states (Attention & meditation), attention value will be focused on this project.
Even this headset is cheap but it is suitable for my project to develop the BCI system. IP
Webcam is used for data acquisition on image capturing. For the complete BCI system
setup is show as Figure 3.1 below.
 22

Figure 3.1: Architecture of BCI system

3.3 HARDWARE

The hardware involved in this project is NeuroSky MindWave Mobile headset,

IP Webcam, Arduino UNO board, and USB Lamp. The F igure 3.2 below has shown the
block diagram of this BCI system.

Figure 3.2: Block diagram of BCI system

 23

3.3.1 NeuroSky MindWave Mobile Headset

In this project, NeuroSky MindWave Mobile headset is the wearable headset

that published by NeuroSky utilized the EEG sensor. The device consists of eight main
parts, ear clip, flexible ear arm, battery area, power switch, adjustable headband, sensor
tip, sensor arm and inside ThinkGear chipset [3]. F igure 3.3 below presents the device
design of NeuroSky Mindwave Mobile headset.

Figure 3.3: NeuroSky Mindwave Mobile headset

 24

Table 3.1: Indicator light of MindWave state condition

Light MindWave State Meaning

Off Powered off MindWave is turned off or has no battery

Solid Blue Connected MindWave is communicating with a paired

device

Single Blinking Blue Ready MindWave is waiting for a paired device to

connect

Double Blinking Blue Pairing Mode MindWave is ready to be paired to a device

Solid Red Unpaired MindWave needs to be paired device to

connect

Double Blinking Red Low Battery Need to replace the MindWave bettery

There is an indicator light on the back of MindWave Mobile headset to notify

the user about the condition of the headset. The details of explanation about the
indicator light are shown as Table 3.1 above.

This device must be set up correctly to ensure brain wave can be fully detected.
Orient the MindWave headset with the forehead sensor arm on left hand side. Make
sure the front sensor make contact with the forehead, but if the sensor does not make
contact with the forehead, remove it and readj ust the adjustable head band and sensor
arm. The Sensor Tip must maintain steady skin contact in order to properly measure the
brainwave. Then, the flexible Ear Arm should rest behind the left ear and then clip the
Ear C lip onto the earlobe. Two metal contacts on the inside of both sides of the ear clip
make sure direct clip on the earlobe. Any hair or obstructions should be removed to
avoid any error happened[28]. The complete setting up of NeuroSky Mindwave Mobile
headset on human head is shown as Figure 3.4 below.
 25

Figure 3.4: Complete setup of Mindwave Mobile headset on human head

Table 3.2: Frequency range of different brainwave types

Brainwave Type Frequency Range Mental states and condition

Delta 0.1Hz to 3Hz Deep, dreamless sleep, non-REM sleep,
unconscious

Theta 4Hz to 7Hz Intuitive, creative, recall, fantasy, imaginary,

dream

Alpha 8Hz to 12Hz Relaxed, but not drowsy, tranquil, conscious

Low Beta 12Hz to 15Hz Formerly SMR, relaxed yet focused,
integrated

Midrange Beta 16Hz to 20Hz Thinking, aware of self and surrounding

High Beta 21Hz to 30Hz Alertness, agitation

The device able to measures the raw signal, power spectrum (alpha, beta, delta,
theta), attention level, mediation level and blink detection. All different brainwa ve types
represent different human mental states and condition. Table 3.2 above shows the
frequency range of brainwave type and its representative mental states.

The outputs of MindWave Mobile is raw EEG data as well as the eSense™ (i.e.
Attention, Meditation) in scale of 1 to 100 [23]. In order to have interaction connection
with computer, The NeuroSky MindWave Mobile headset must paired with the
 26

computer by using Bluetooth. Serial COM port name of headset was identified using
Bluetooth software in order to connect or interface with some applications. Default port
setting for Bluetooth connectio n of NeuroSky MindWave Mobile with computer should
be same as Figure 3.5 below.

Figure 3.5: Default port setting of NeuroSky MindWave Mobile Headset

3.3.2 IP Webcam

IP Webcam is an android mobile apps developed by Pavel K hlebovich [29]. This

apps enable mobile device turn into a network camera and able to stream video inside
WiF i network without internet access. IP Webcam has a user friendly functional layout
setup which is very easy to unde rstand. This android mobile apps able to download at
android Google Play with free of charge.

Figure 3.6: IP Webcam android apps at Google Play

 27

In the IP Webcam mobile apps, user can customer some setting of it regrading of
its purpose. Users are able to set the resolution and quality of the video for higher
definition. The highest resolution of video can reach to 1280x720p and the highest
transmission of video can reach to 16000kbit/s. Besides that, it has password security,
user will required to log in with correct password in order to access the webcam.
Figure 3.7 below show the video live streaming monitor in mobile device.

Figure 3.7: Monitor of video live streaming in mobile device

Once the video is streaming, the IP address is shown at the bottom of the
monitor. By typing the same IP address into any internet browser or Android device to
view the video live streaming.
 28

3.3.3 Arduino UNO

Figure 3.8: Arduino UNO microcontroller board [30]

Arduino UNO board is a microcontroller board based on the ATmega328. It

having total 14 digital input/output pins and 6 analog input pins. S ix of the digital
input/output pins are able to be used as P ulse Width Modulation (PWM) output [31].
The Arduino UNO can be direct connected to computer by using a Universal Serial Bus
(USB) cable only. This board can be programmed by using Arduino Integrated
Development Environment (IDE) software. For this project, Arduino UNO will be
programmed by MATLAB coding with the help of Arduino S upport Package of
MATLAB. That package enable Arduino UNO direct link with the MATLAB. F igure
3.9 below shows the properties of Arduino UNO which connected with the computer.

Figure 3.9: Properties of Arduino UNO connected to computer

 29

3.3.4 USB Lamp

USB lamp is powered by any 5V USB port such as laptop, power bank or any
other US B charger. The slim design and bendable body is convenience for any usage.
The Average life of it is around 10,000 hours. US B Lamp in this project is act as the
output. Its interaction with other components will be discussed further in this chapter.

3.4 SOFTWARE

The software involved in this project is MATLAB and Music P layer. MATLAB
will act as signal and image processing, and then Music P layer is act as output device.
All those software will interact with hardware to develop the BCI system. The whole
BCI system will run the program flow according to the flow chart in Figure 3.10.
 30

Start

Detect image
from IP Webcam

No
No
Green Ball ? Red Ball ?

Yes Yes

Turn ON Turn ON
Indicator Light Indicator Light
(USB Lamp) (Music Player)

Gain Attention Gain Attention

Value from Value from
NeuroSky NeuroSky
MindWave Mobile MindWave Mobile

No No

No No
Attention Value > 80 Attention Value < 5 Attention Value > 80 Attention Value < 5

Yes Yes Yes Yes

Turn OFF Turn OFF

Turn ON all Indicator Light Turn ON all Indicator Light
USB Lamp & Music Player & Music Player & Music Player &
USB Lamp USB Lamp

Figure 3.10: Flow chart of BCI system

3.4.1 MATLAB

MATLAB is originally developed by The MathWorks Inc., which stands for

Matrix Laboratory [32]. MATLAB has some main parts which are its Mathematical
function library, Language, Graphics, External interfaces, Desktop tools and
development environment. MATLAB version 2015b is used for developing this project.
Then, Arduino support package of MATLAB 2016b is required for this project to get
 31

interaction connection with Arduino UNO. Besides that, Image processing toolbox also
required to conduct some image processing during processing part. Then, MATLAB
GUI will be used to develop the whole BCI system to make it more users friendly.

3.4.2 Music Player

The Music P layer used in this project is Window Media P layer (WMP). WMP is
a media player and media library application created by Microsoft that is utilized for
playing sound, video and review pictures on computer. This music player support many
formats of audio, video, and even picture files, such as .wma, .wmv, .avi, .mpeg, .mpg,
.mp3 and .vob.

3.5 METHODOLOGY

NeuroSky MindWave Mobile headset paired with computer in order for data
transmission. The connection of headset and computer will be through Bluetooth
connection. NeuroSky MindWave Mobile headset needs turn into paring mode in order
to have connection with computer. During the pairing process, under the device column,
need to check that box to enable the data transmission and identified the COM Port
name of headset too.

Figure 3.11: Bluetooth Setting for data transmission of MindWave Mobile headset
 32

After the pairing, MATLAB can now start acquire brainwave signal from
MindWave Mobile headset by calling the ThinkGear library of NeuroSky MindWave
which established based on MATLAB and Arduino IDE with C++. The outputs of
MindWave Mobile is raw EEG data as well as the eSense™ (i.e. Attention, Meditation)
in scale of 1 to 100 [23]. This project is mainly focus on the Attention value from the
user. S ince the raw output data is a bit unstable and fuzzy and hard to prove whether
user is concentrating. Then, 5-points moving average smoothing method used by C urve
Fitting Toolbox in MATLAB is applied to stabilize the data for better analysis [33].
Smooth function in MATLAB is used to smooth the Attentio n value acquired from user
brain according to the Formula 3.1 below.

�
��(��) = (�(𝒊 + ��) + �(𝒊 + 𝑵 − �)+ ⋯+ �(𝒊 − ��)) 3.1
���+�

Where ��(i) is the smoothed value for the ith data point, N is the number of
neighboring data�points on side of ���(i), and 2N+1 is the span. In this project, the value
of span is set to 5 for better smoothing effect. 5-points moving average technique can be
prove user is really concentrating and will not effected by sudden spiking of the
concentration value.

For the image process ing part, IP Webcam will start capturing image and live
video streaming on the MATLAB GUI. Every frame of the image capture by IP
Webcam will be undergoing image processing. Every image having the original color
which consists of red, green and blue (RGB). The RGB value of image will then turn
into gray scale image according the formula 3.2 below.

�.���
�× 𝑹 + �.����× 𝑮 + �.����× � 3.2

After convert the RGB value of image into grayscale, “imfindcircles” function
that available in MATLAB Image Processing Toolbox MATLAB will be used to detect
any circular object. The “imfindcircles” function using C ircular Hough Transform
(CHT) based algorithm to determine any circles in image. This algorithm having object
polarity detection that identifies the circular objects which brighter or darker then
 33

background. The purpose that turn image into gray scale is to make the circular object
able be detected easily. The CHT technique will be detect circular object based o n the
range of the radius given. By having the radius, center of circular object can be
identified easily.

Figure 3.12: Voting mode: Range of � ����and �����along direction of gradient

Table 3.3: Parameter used in CHT technique

����� Minimum search radius

����� Maximum search radius
�������
��� Radius of the circle that the candidate pixel belongs to
�
���� Center of the circle of radius ���
���

�
���� Center of the circle of radius 𝑟�
��𝑥

�
������
��� Center of the circle of radius �����������

��𝑥 ) and minimum search radius ( ���

The maximum search radius ( 𝑟� ���) of the
circular object is set into certain range to ensure only detect the desired target circular

��𝑥 is maximum 3 times of the ���

object. It is important that only set that 𝑟� , it makes
���

the process of circular identification become faster.

 34

There is two output of this project which is US B lamp and music player. For the
output hardware USB lamp, there is some circuit connection. The components used for
that circuit connection are 9V battery, PNP type transistor, two 220Ω resistors, two
LED indicators light and a US B holder. In this project, the Arduino UNO board will be
the output controller that interfaced with MATLAB. As mention before, Arduino UNO
board can be programmed by MATLAB coding by installing the Arduino Support
Package of MATLAB. By having that support package, all the output and input pin of
Arduino UNO board is now can be controlled by MATLAB.

If there is any green circular object detected, the pin 4 of Arduino UNO will be
activated to trigger the indicator light of US B lamp. But if red circular object detected,
pin 2 of Arduino UNO will be activated to trigger the indicator light of music player.
Then, while green circular object is detected and at the same time attention value of user
is higher than 80, then pin 7 of Arduino UNO will be activated to turn on the US B lamp.
In the other hand, if red circular object detected and attention value higher than 8 0, then
music player of computer will be turn on by using “winopen” function in MATLAB.
This “winopen” able to open any files which perform the same action by double click
any files in Windows Explorer program. F igure 3.13 below shows the circuit connectio n
of USB lamp with Arduino UNO board.

Figure 3.13: Hardware circuit connection diagram

 35

3.6 SUMMARY

This chapter discussed about the architecture of the while BCI system which
combined with hardware and software parts. All the software and hardware parts
involved are NeuroSky MindWave Moble headset, IP Webcam, Arduino UNO board,
USB Lamp, and MATLAB 2015b. Attention raw data values were stabilized by using
5-points moving average technique by Curve F itting Toolbox in MATLAB. For the part
of image processing of image, c ircular object can be easily detected by using CHT
technique. Threshold value of RGB will be setting to only detect green and red circular
object. O utput of the BCI system will be controlled by Arduino UNO b oard which
having direct interface with MATLAB.
 CHAPTER 4

RESULT AND DATA ANALYSIS

4.1 INTRODUCTION

This section discuss about the data analysis about the human concentration
based on gender and condition. There will both male and female participants take part in
this data analysis. The conditions take in consideration in this analysis will be relax,
happy and stress condition. The further explanation on experimental setup and data
analysis will be discussed in this chapter.

4.2 EXPERIMENTAL SETUP

For this project, there are total 8 people in the age group of 21-23 years old
participated in this data analysis, which are 4 male and 4 female. Participants will
required to undergo every condition such as relax, happy and stress before doing the
data analysis to test the impact on the human concentration.

4.2.1 Relax Condition

The first one is relax condition. Participate required to do nothing and resting for
5 minutes before taking the data analysis. While participant is resting, their brainwave
signal is continuous display on the MATLAB GUI. Facial expression of participants in
relax condition is shown at the Figure 4.1.
 37

Figure 4.1: Facial expression of participants in relax condition

100
90
80
Attention Reading

70
60
50
40
30
20
10
0
0 5 10 15 20 25 30 35 40 45 50 55
Time (seconds)

Figure 4.2: Attention value of participant in relax condition

After 5 minutes of resting, participants is now required to focus and concentrate.

Time taken for attention value to be reach 80 is recorded. The F igure 4.3 has shown the
time taken for attention value of participants to reach 80 is very short which around 16
seconds.
 38

100
90
80
Attention Reading

70
60
50
40
30
20
10
0
0 2 4 6 8 10 12 14 16
Time (seconds)

Figure 4.3: Time taken for attention value to reach 80 after relax condition

4.2.2 Happy Condition

For the second condition which is happy condition. Participants required

watching funny video before doing the data analysis. The funny video prepared for this
testing is “Just for Laughs”. “Just for Laughs” is a Canadian silent comedy/hidden
camera reality television prank show. When participants are watching the funny video,
their corresponding average attention value will be around 60 and recorded. Facial
expression of participants in happy condition is shown at the Figure 4.4.
 39

Figure 4.4: Facial expression of participants in happy condition

100
90
80
Attention Reading

70
60
50
40
30
20
10
0
0 5 10 15 20 25 30 35 40 45 50 55
Time (seconds)

Figure 4.5: Attention value of participant in happy condition

After 5 minutes by watching the funny video, participant is now required to

focus and concentrate. Time taken for attention value to be reach 8 0 is recorded. The
Figure 4.6 has shown the time taken for attention value of participants to reach 80 is 38
seconds which a bit longer compare to time taken in relax condition.
 40

100
90
80
Attention Reading 70
60
50
40
30
20
10
0
0 5 10 15 20 25 30 35
Time (seconds0

Figure 4.6: Time taken for attention value to reach 80 after happy condition

4.2.3 Stress Condition

The last condition is stress condition. Participants required solving 25

mathematical questions (Addition) by online. All the questions must be answered
correctly within 1 minute with no error a nd without using any calculator. Facial
expression of participants in stress condition is shown at the Figure 4.7 below.

Figure 4.7: Facial expression of participants in stress condition

 41

100
90
80
Attention Reading 70
60
50
40
30
20
10
0
0 5 10 15 20 25 30 35 40 45 50 55
Time (seconds)

Figure 4.8: Attention value of participant in stress condition

After solved all the mathematical questions online, all the participants required
to focus immediately to test their concentration. The F igure 4.9 below showing that
attention value of participants unable to reach to 80 within 2 minutes.

100
90
80
Attention Reading

70
60
50
40
30
20
10
0
0 5 10 15 20 25 30 35
Time (seconds0

Figure 4.9: Attention value of participant to reach 80 after stress condition

 42

4.3 OUTPUT RESULT

This project will mainly focus on one mental state which is attention from user.
The raw data of brain signal from user brain is fuzzy and unstable. By applying the 5-
points moving average smoothing technique, the output graph will be very smooth and
stabilized for further brain signal processing. The output result of this smoothing
process is shown at the Figure 4.10 below.

100
90
80
Attention Reading

70
60
50
40
30
20
10
0
0 5 10 15 20 25 30 35 40 45 50 55
Time (seconds)

(a)

100

80
Attention Reading

60

40

20

0
0 5 10 15 20 25 30 35 40 45 50 55
Time (seconds)

(b)

Figure 4.10: Output of Attention value (a) raw data and (b) smooth data
 43

For this BCI system, MATLAB GUI is developed act as a signal processor to
generate command based on the inputs received. Starting of the MATLAB GUI will
require user to enter password to access the program. If user forgot the password, click
on the “Forgot Password” to reset a new password.

Figure 4.11: Log in page of BCI system

After entering the main page of BCI system, there will be a “S TART” and
“S TOP” button to activate the program and terminate the program respectively. There is
two column on the left that shown the real-time graph of attention reading from
MindWave Mobile headset and live video streaming from IP Webcam.

Figure 4.12: Main page of BCI system

 44

By clicking the “START” button, the system will start acquire attention value
from NeuroSky MindWave Mobile headset and IP Webcam is now started to capture
image and video live streaming in the MATLAB GUI by accessing the IP address with
internet connectivity [34]. There is a column that allowed user to enter the trigger value
for activation of electronic device. Default trigger value for this BCI system was set to
80. The real-time graph of attention value and video live streaming from IP Webcam is
shown as the Figure 4.13 below.

Figure 4.13: Real-time graph of Attention value and video live streaming

The live video streaming capture by IP Webcam will be send to MATLAB to

undergo image processing frame by frame. The true-color of image is combined with color
of red, green, blue or RGB color system[35]. Every image will have those RGB which
representing the red, green, and blue components of the pixel scalar. In this project,
the main focus will be at t urning the RGB image into grayscale image for image
processing. There are many classes of grayscale image which in uint8, uint16, int16,
single, or double. The values range of different classes having different values, for
example for uint8, values range is from 0 to 255 [36]. F igure 4.14 show the difference
between RGB image, grayscale image and the circle detected from grayscale image.
 45

(a)

(b)
 46

(c)

Figure 4.14: Difference between RGB (a), Grayscale image (b) and Circular object
detected from grayscale image (c)

Then if any circular objects are detected, it will identify the original color of that
circular object. In this project will only focus on green and red color. The threshold
value for green and red color are set to ensure the fina l result will be only green or red
circular object. Threshold value for green and red color to be detected is predefined.

At first, red color object will be filter out and turn it into binary image. A binary
image contain only 2 color pixel which is black and white. The F igure 4.15 shows the
steps and result on how to extracts the red circular objects on the image using color
thresholding.
 47

(a)

(b)

(c)

Figure 4.15: Original image (a), red object extracted on original image (b) and red
circle object is detected (c)
 48

Then whenever green circular object is detected, the indicator light of USB
Lamp and the MATLAB GUI will display “Green Ball Detected”. At this moment, if
the user attention value acquired from MindWave Mobile headset is higher enough and
more than 80, then USB Lamp will be turn on.

Then in case of red circular object is detected, indicator light of Music P layer
will be turn on and MATLAB GUI will display “Red Ball Detected”. Then user
attention value is higher than 80 then Music P layer will be turn on automatically. F igure
4.16 and F igure 4.17 shown the output result of green and red circular object is detected
with attention value higher than 80.

Figure 4.16: Output result of green circular object detected with attention value higher
than 80

Figure 4.17: Output result of red circular object detected with attention value higher
than 80
 49

While user is sleeping or meditation, attention value will drop to a very low
value. Once attention value acquired is lower than 5, then all devices such as USB
Lamp and Music Player will be turns off immediately.

4.4 SUMMARY

In conclusion, based on the data analysis, time taken for people to concentrate
will be very long if under stress condition. Then, Time taken will become shorter in
happy condition compare to stress condition. Then, people are able to concentrate easily
if allowed to relax before focus on doing something. For the output result, once green of
red circular object is identified, indicators light of it will be turn on. Then, user mind
concentration if higher than 80 at that moment, US B lamp or music player will be turn
on. Then, based on the results of data analysis, people in happy condition are able to
concentrate 36.67% faster than people in stress condition. People in relax conditio n able
to concentrate much faster which is 73.33% compare to people in stress condition.
 CHAPTER 5

CONCLUSION AND RECOMMENDATION

5.1 CONCLUSION

BCI system was successfully developed to control home automated system. The
raw EEG signal acquired from human brain are able be interpreted and stabilized using
5-points moving average smoothing method used by Curve F itting Toolbox in
MATLAB. The fuzzy and sudden spiking of concentration value will not affect the real
concentration value. The stabilized attention value is now very smooth and suitable for
data analysis and output device control.

By having the image processing on the image acquired, human can now control
over multiply electrical devices with a symbolic representative object. In this project,
this concept had proved that user able to choose turn on US B lamp and music player.
Although there is only one output signal acquired from NeuroSky MindWave Mobile,
but this signal can be directed depending on what kind of object detected.

For the d ata analysis parts, a proper experimental testing been conducted to
analysis the brainwave signal concentration based on gender and condition. Based on
the data analysis, time taken for people to concentrate will be very long if under stress
condition. Then, Time taken will become shorter in happy condition compare to stress
condition. Then, people are able to concentrate easily if allowed to relax before focus on
doing something.
 51

5.2 RECOMMENDATION

As recommendation, NeuroSky MindWave Mobile headset can have a build- in

camera which having ability capture video that representing human vision. Using
Machine Learning tool in MATLAB can be recognizing the electronic devices. Once it
is recognized, user can direct control or turn on/off the device. By having this
improvement, all human being will able to manipulate more home electronic devices.
There will be a totally new living style in the future.
 52

REFERENCE

[1] M. Teplan, “Fundamentals of EEG measurement,” Meas. Sci. Rev., vol. 2, pp. 1–
11, 2002.

[2] A. Coenen and O. Zayachkivska, “Adolf Beck: A pioneer in

electroencephalography in between Richard Caton and Hans Berger.,” Adv.
Cogn. Psychol., vol. 9, no. 4, pp. 216–21, 2013.

[3] W. Sa, “Processing and spectral analysis of the raw EEG signal from the
MindWave,” West Pomeranian Univ. Technol., no. 2, pp. 169–173, 2014.

[4] J. Alexis, “Developing a Home-Based Functional Application for an EEG-based

Brain Computer Interface A dissertation submitted by,” no. October, 2011.

[5] “Jacques J. Vidal - Unknown - Toward direct brain-computer

communication.pdf.pdf.” .

[6] I. Wickelgren, “Tapping the Mind (News Focus),” Science (80-. )., vol. 299, no.
January, pp. 496–499, 2003.

[7] A. P. Georgopoulos, J. T. Lurito, M. Petrides, A. B. Schwartz, and J. T. Massey,

“Mental rotation of the neuronal population vector.,” Science, vol. 243, no. 4888,
pp. 234–236, 1989.

[8] E. Farwell, L.A., Donchin, “Farwell Donchin 88 - first P300 BCI.pdf.” 1988.

[9] A. Vallabhaneni, T. Wang, and B. He, “Brain - Computer Interface,” Neural

Eng., pp. 85–121, 2005.

[10] Jay Hafling, “Population of Malaysia,” Sandiwara, 2015. [Online]. Available:

http://countrymeters.info/en/Malaysia. [Accessed: 22-May-2016].

[11] M. Govindu and A. Rajaiah, “Automatic Home Control System Using Brain
Wave Signal Detection,” no. October, pp. 889–893, 2014.

[12] C. Postelnicu, D. Talaba, and M. Toma, “Brain Computer Interfaces for Medical
Applications,” Bull. Transilv. …, vol. 3, no. 52, pp. 99–106, 2010.

[13] B. Jarosiewicz, S. M. Chase, G. W. Fraser, M. Velliste, R. E. Kass, and A. B.

Schwartz, “Functional network reorganization during learning in a brain-
computer interface paradigm.,” Proc. Natl. Acad. Sci. U. S. A., vol. 105, no. 49,
 53

pp. 19486–19491, 2008.

[14] B. S. Carter and I. a. Awad, “Surrogate Measures of Outcome in Clinical Trials:

Clues Worth Examining?,” Neurosurgery, vol. 63, no. 2, pp. 5–10, 2008.

[15] a B. Schwartz, R. E. Kettner, and a P. Georgopoulos, “Primate motor cortex and

free arm movements to visual targets in three-dimensional space. I. Relations
between single cell discharge and direction of movement.,” J. Neurosci., vol. 8,
no. August, pp. 2913–2927, 1988.

[16] J. Del R. Millán, P. W. Ferrez, F. Galán, E. Lew, and R. Chavarriaga, “Non-

Invasive Brain-Machine Interaction,” Int. J. Pattern Recognit. Artif. Intell., vol.
22, no. 05, pp. 959–972, 2008.

[17] H. Ekanayake, “P300 and Emotiv EPOC: Does Emotiv EPOC capture real
EEG?,” Web Publ. http//neurofeedback. visaduma. info/ …, p. 16, 2010.

[18] A. Campbell, T. Choudhury, S. Hu, H. Lu, M. K. Mukerjee, M. Rabbi, and R. D.

S. S. Raizada, “NeuroPhone,” Proc. Second ACM SIGCOMM Work. Networking,
Syst. Appl. Mob. handhelds - MobiHeld ’10, p. 3, 2010.

[19] Curiositry, “Neurotech Hardware Roundup 2016,” 2016. [Online]. Available:

http://openbci.com/community/neurotech- hardware-roundup-2016/. [Accessed:
22-May-2016].

[20] J. W. Matiko, S. Beeby, and J. Tudor, “Real time eye blink noise removal from
EEG signals using morphological component analysis.,” Conf. Proc. IEEE Eng.
Med. Biol. Soc., vol. 2013, no. 2, pp. 13–6, 2013.

[21] K. Crowley, A. Sliney, I. Pitt, and D. Murphy, “Evaluating a Brain-Computer

Interface to Categorise Human Emotional Response,” 2010 10th IEEE Int. Conf.
Adv. Learn. Technol., pp. 276–278, 2010.

[22] A. Vourvopoulos and F. Liarokapis, “Robot Navigation Using Brain-Computer

Interfaces,” 2012 IEEE 11th Int. Conf. Trust. Secur. Priv. Comput. Commun., no.
JUNE, pp. 1785–1792, 2012.

[23] E. A. Larsen and A. I. Wang, “Classification of EEG Signals in a Brain-

Computer Interface System,” Nor. Univ. Sci. Technol., no. June, pp. 1–72, 2011.

[24] K. Chu and C. Y. Wong, “Player’s attention and meditation level of input devices
 54

on mobile gaming,” Proc. - 2014 3rd Int. Conf. User Sci. Eng. Exp. Eng. Engag.
i-USEr 2014, no. September 2014, pp. 13–17, 2015.

[25] J. Of and M. Informatics, “Brain-Computer Interface for Mobile,” vol. 24, 2015.

[26] M. Eid and A. Fernandez, “ReadGoGo!: Towards real-time notification on

readers’ state of attention,” 2013 24th Int. Conf. Information, Commun. Autom.
Technol. ICAT 2013, 2013.

[27] G. Pfurtscheller, “How Many People are Able to Operate an EEG-Based Brain-
Computer Interface (BCI)?,” vol. 11, no. 2, pp. 4–6.

[28] Neurosky, “MindWave Mobile : User Guide,” 2012.

[29] S. U. Prini, A. Hidayat, and T. Erlina, “AKUISISI DATA SUHU PADA

RESCUE ROBOT DAN MONITORING OBJEK MENGGUNAKAN
APLIKASI WEBCAM BERBASIS ANDROID.”

[30] SM, “Software Serial Numbers :,” 2015/08/19, 2007. [Online]. Available:

https://www.arduino.cc/en/Tutor ial/SoftwareSerialExample. [Accessed: 22-May-
2016].

[31] A. D’Ausilio, “Arduino: A low-cost multipurpose lab equipment,” Behav. Res.

Methods, vol. 44, no. 2, pp. 305–313, 2012.

[32] R. Krishnan, “Introduction to MATLAB,” p. 55, 2000.

[33] E. Alessio, a. Carbone, G. Castelli, and V. Frappietro, “Second-order moving

average and scaling of stochastic time series,” Eur. Phys. J. B - Condens. Matter,
vol. 27, no. 2, pp. 197–200, 2002.

[34] S. Saha and S. Neogy, “A case study on smart surveillance application system
using WSN and IP webcam,” Appl. Innov. Mob. Comput. (AIMoC), 2014, no.
February 2014, pp. 36–41, 2014.

[35] R. S. Ledley, M. Buas, and T. J. Golab, “Fundamentals of true-color image

processing,” [1990] Proceedings. 10th Int. Conf. Pattern Recognit., vol. i, pp.
791–795, 1990.

[36] T. Kumar and K. Verma, “A Theory Based on Conversion of RGB image to Gray
image,” Int. J. Comput. Appl., vol. 7, no. 2, pp. 5–12, 2010.