THE DEVELOPMENT OF A PROGRAMMABLE INPUT EMULATOR FOR PHYSIOLOGICAL SIGNALS Edmond Mitchell

Degree Stream : Electronic Engineering Project Supervisor : Dr Toms Ward

A Final Year Project Report submitted in partial fulfilment of the requirements of the National University of Ireland Maynooth for the degree of Batchelor of Engineering

April 2009 Department of Electronic Engineering National University of Ireland Maynooth

Final Year Project Report

ABSTRACT GlovePie is a software application that allows users to emulate inputs in Windows. However it lacks the capacity to use physiological signals as inputs. This project remedied this by replicating GlovePie for use with physiological signals. Physiological programmable input emulator (PhysPIE) will promote the development of assistive technology systems as it facilitates mappings between physiological signals and user input. Examples of possible input signals include Alpha waves emitted from the brain, breathing patterns, kinetic signals from the Wii Balance board or the flexing of arm muscles. C# was chosen as the programming language that the vast majority of the project would be written in. Common computer input controls were emulated such as the mouse and keyboard for outputs for the users inputted physiological signals. This allows real-time acquisition and processing of physiological signals to create events which can be mapped on to standard GUI operating system user-input commands. A core component was the incorporation of a scripting feature in the application which allows a user to easily select standard signal processing operations from which to generate events.

97557544.doc

ii

Edmond Mitchell

Final Year Project Report

TABLE OF CONTENTS ABSTRACT....ii 1 Introduction.............................................................................................................................................1 1.1 Academic background.....................................................................................................................1 1.1.1 GlovePie...................................................................................................................................1 2 Windows programming..........................................................................................................................2 2.1 Events..............................................................................................................................................2 2.2 Software Development....................................................................................................................2 2.2.1 C# programming......................................................................................................................2 2.2.2 C# debugging...........................................................................................................................3 2.2.3 C# Classs. Ease of use (e.g sendinput)...................................................................................3 2.2.4 Saving/loading..........................................................................................................................3 2.3 Threading.........................................................................................................................................3 2.4 Hooks...............................................................................................................................................5 3 Scripting..................................................................................................................................................5 3.1 Scripting in this project...................................................................................................................6 3.2 LUA.................................................................................................................................................6 3.3 Python etc........................................................................................................................................6 3.4 MatLab.............................................................................................................................................6 3.5 Reflection.........................................................................................................................................6 3.6 LUA functions.................................................................................................................................6 3.6.1 Using C# to debug LUA..........................................................................................................6 4 Arduino...................................................................................................................................................7 4.1 Arduino programming.....................................................................................................................7 4.2 Arduino circuits...............................................................................................................................7 4.3 Arduino no registers, delimiters......................................................................................................7 5 Serial Transmission................................................................................................................................7 5.1 In C#................................................................................................................................................7 5.2 Digital Vs Analog............................................................................................................................7 5.3 10 bit ADC.......................................................................................................................................7 5.4 BAUD rate.......................................................................................................................................7 6 DLLs ......................................................................................................................................................8 6.1 DLL used.........................................................................................................................................8 6.2 APIs.................................................................................................................................................8
97557544.doc

iii

Edmond Mitchell

Final Year Project Report

6.3 Zedgraph..............................................................................................................8 6.4 Graphing Dynamic........................................................................................................................8 7 Buffering.................................................................................................................................................8 7.1 Double Buffer..................................................................................................................................8 7.2 Circular Buffers...............................................................................................................................8 8 Bandpass filter........................................................................................................................................9 9 Key ChallengEs......................................................................................................................................9 9.1 Multiple Wii Devices.....................................................................................................................9 9.2 Realtime Graph...............................................................................................................................9 9.3 Connecting to serial port................................................................................................................9 9.4 Storing Serial Data.........................................................................................................................9 9.5 Hooking Keys.................................................................................................................................9 9.6 Embedding the scripting language ................................................................................................9

97557544.doc

iv

Edmond Mitchell

Final Year Project Report

1 INTRODUCTION 1.1 Academic background

This project lies in the region of assistive technology. This is a broad term which includes assistive, adaptive, and rehabilitative devices for people with disabilities. Assistive technology allows users to gain more independence by letting them complete tasks that they were formally unable to or had great difficulty accomplishing. Many prominent technologies have started of in this area and gone on to more global use. Short Message Service (SMS) was initially designed for quick long range communication for people with hearing impairments but has gone on to have a user base of over 2.5 billioni. 1.1.1 GlovePie Glove Programmable Input Emulator (GlovePie) is a windows application that allows users to emulate a variety of inputs using a multitude of devices. For instance it is possible to emulate a mouse with the Nintendo Wii Controller [Figure 1]. This is done by writing scripts in GlovePie and having GlovePie process them in real-time. Basic knowledge of programming is required to write these scripts and demo scripts are widely available through out the internetii. GlovePIE allows scripts are to be saved and loaded. The current project aims to replicate the functionality of GlovePIE for physiological signals. See Appendix A for an example of a GlovePie script.

Figure 1 Use of Wiimote with a Laptop and GlovePie 1

97557544.doc

Edmond Mitchell

Final Year Project Report

2 WINDOWS PROGRAMMING 2.1 Events

In DOS and earlier operating systems code was executed line after line in a predictable fashion via function calls or until a list of tasks has been accomplished. This is not the case with modern Graphical User Interface (GUI) operating systems. Windows is an event-driven operating system. In C# an event is defined as a member that enables an object or class to provide notificationsiii. In a piece of software the change of monitored properties of an object may result in an action occurring. An example of this would be the starting of a new process when a button is clicked. In PhysPIE an event would occur when the start script button is clicked. The event that would occur is the compiling and the running of the script

2.2 Software Development

For this project graphical elements in Windows needed to be controlled quickly and easily. To do this in C it would take hundreds of lines of code. This would not be an efficient use of time since all these functions have been written and complied in the .NET framework. The .NET framework is a piece of software technology that is available with several Microsoft Windows operating systems. It includes a large library of pre-coded solutions to common programming problems. It also contains a virtual machine that manages the execution of programs written specifically for the framework. The pre-coded solutions that form the framework's Base Class Library cover a large range of programming needs in a number of areas, including user interface, data access, database connectivity, cryptography, web application development, numeric algorithms, and network communications. The class library is used by programmers, who combine it with their own code to produce applications. Some languages that use this framework are Visual basic, Visual C++, Visual C# and Visual Basic.

2.2.1 C# programming C# was chosen as this projects primary programming language due to its object orientated nature and its widespread use when using the .NET framework. Microsoft Visual C# 2008 Edition was utilized to compile the C# code. This application is extremely responsive as it shows your program compiled and updates this preview in real-time. It allows easy manipulation of graphical user interfaces (GUIs) by listing common features that can be added to a GUI in a Toolbox. Examples of

97557544.doc

Edmond Mitchell

Final Year Project Report

these features include buttons, checkboxes, labels and combo boxes. It generates code for these features and links them to your own code.

2.2.2 C# debugging Microsoft Visual C# 2008 edition debug features were used extensively through out the lifetime of this project. Tools include breakpoints, watches and the output window. Breakpoints deliberately stop execution at any statement of code. When C# encounters a break point while executing code, execution is halted at the break statement, prior to it being executed. Break points enable you to query or change the value of variables at a specific instance in time, and they let you step through code execution one line at a time. The Watch Window allows developers to watch certain expressions, in certain modules and/or procedures, and then stop code execution when it changes. It also allows developers to view variables current value when a breakpoint has been reached. The Output window is used by C# to display various status messages and build errors. This references what line the compiler encountered an error and the error type. This is exceptionally helpful in correcting syntax errors. 2.2.3 C# Classs. Ease of use (e.g sendinput) 2.2.4 Saving/loading

2.3 Threading
Threading in computer programming is where two or more instructions are executed concurrently. This allows programs to perform multiple operations at once which speed up the program. The way threads are implemented differs from one operating system to another but the concept is always the same. A process, in the simplest terms, is an executing program. One or more threads run in the context of the process. A thread is the basic unit to which the operating system allocates processor time. On a single processor multithreading is generally achieved by time division multiplexing. This means that the processor switches between threads quickly enough that the user cannot distinguish it. However on a multi-core processor different threads are actually run on different cores at the same time. While processors may have these abilities, it is up to the operating system and the program to

97557544.doc

Edmond Mitchell

Final Year Project Report

take advantage of them. Certain higher level languages 1(for example Pythoniv) may implement a software lock which restricts processes running threads on separate cores. Windows runs threads by use of a scheduler. This scheduler runs threads based on a priority queue. Windows runs a get message loop to continually check and add threads to the schedulers queue. It also gives each thread a maximum runtime in order to prevent deadlock2. found in
using System; using System.Threading;

In

C# threading is relatively simple compared to some other languages (E.g. C). It uses the libraries

Threading in C# enables you to perform concurrent processing so you can do more than one operation at a time. For example, you can use threading to monitor input from the user, perform background tasks, and handle simultaneous streams of input. The System.Threading namespace provides classes and interfaces that support multithreaded programming and enable you to easily perform tasks such as creating and starting new threads, synchronizing multiple threads, suspending threads, and aborting threads. To incorporate threading in C#, all that needs to be done is to create a function to be executed outside the main thread and point a new Thread object at it. The following code example creates a new thread in a C# application:
Thread t = new Thread(another_function); t.Start(); //t.Abort(); t.Resume(); t.Suspend();

This code will create a new thread that leads to a function called another function. To run this thread all is needed is to run the Start() property. By using this method is easy to use this types other properties such as Abort, Resume and Suspend. I need threading in my program in order to make sure it can constantly monitor the feature receiver while still carrying out events at the same time. In order for this program to succeed, the program will need to be able to create new threads and abort successfully.

A high-level programming language is a programming language with strong abstraction from the details of the Deadlock is a situation in which two processes sharing the same resource are effectively preventing each other from

computer
2

accessing the resource, resulting in both programs ceasing to function

97557544.doc

Edmond Mitchell

Final Year Project Report

2.4 Hooks

3 SCRIPTING
In this project in order to give the user the most possible functionality the use of an embedded scripting language is required. A script language is a programming language that already has a useful set of components written in other languages. They differ from system programming language such as C, Fortran and C++ in the way they were designed. System programming languages were designed for building data structures and algorithms from scratch. In contrast, scripting languages are designed for gluing: they assume the existence of a set of powerful components and are intended primarily for connecting components togetherv. The advantages of scripting languagesvi

Rapid development: Due to the need for less code to be written development and debug time is reduced. Integration with existing technologies: They are built around component technologies like .NET and Java. Easy to learn and use: Only basic computer programming knowledge is required. Dynamic code: Scripting language code can be generated and executed at runtime, an advanced feature useful and necessary in some applications.

The disadvantages of scripting languages

They are not all-inclusive. They assume the presence of a system programming language. They are not conducive to best practices in software engineering and code structuring, such as object orientation and component-based development. They are usually not general purpose but tuned toward a specific application, such as web development (PHP).

97557544.doc

Edmond Mitchell

Final Year Project Report

3.1 Scripting in this project

Scripting is a main component in this project. It allows a user to easily select standard signal processing operations from which to generate events Scripting lets the user manipulate inputs and outputs more easily and with more capability. The Code below shows how a user can quickly check if the incoming data exceeds a defined tolerance and performs a GUI action if required. For the example below if the incoming data exceeds 500 then the mouse will move 5 pixels to the right. tolerance = 500 dataA = GetChannelADataLUA() if (dataA > tolerance) then MouseFunctionsLUA("MoveX",5) end This type of scripting will allow users to code their desired events without needing to write complex functions to take control of GUI inputs.

3.2 LUA 3.3 Python etc 3.4 MatLab 3.5 Reflection 3.6 LUA functions
3.6.1 Using C# to debug LUA LUA was integrated into PhysPIE by use of a dynamic link library (DLL). This DLL only included a simple light weight compiler. This compiler lacks any debugging features. It merely throws an exception when it encounters an error in the scripting code.

97557544.doc

Edmond Mitchell

Final Year Project Report

To debug LUA when it was initially embedded a c# function was used to type the equivalent code and test that it performed identical to the LUA scripts. Eventually a limited output window was added to LUA that referenced the line that it encountered an error at.

4 ARDUINO 4.1 Arduino programming 4.2 Arduino circuits 4.3 Arduino no registers, delimiters

5 SERIAL TRANSMISSION 5.1 In C#

5.2 Digital Vs Analog 5.3 10 bit ADC 5.4 BAUD rate

97557544.doc

Edmond Mitchell

Final Year Project Report

6 DLLS 6.1 DLL used 6.2 APIs 6.3 Zedgraph 6.4 Graphing Dynamic

7 BUFFERING
In computing, a data buffer referrers to a selection of memory that is used to temporally hold data while it is being transferred from two different locations. Typically it is needed when the rate of flow of data differs between devices.

7.1 Double Buffer

There are two main types of buffers in software. The first is a double buffer. Double buffering works by using two buffers. When one is being read the other is still being filled with data. Once the first buffer has had all its data read the second buffer is full and is available to have its data read while the original buffer is being refilled. This type of buffering is able to better combat the two main problems, buffer under-run and buffer overflow when only a single buffer is used. Buffer under-run occurs when data is being read in faster than data is being fed in. Buffer overflow however is where the data is being fed in faster than is can be read.

7.2 Circular Buffers

The second main type of buffering is circular buffering [Figure 5]. Here a fixed amount of buffers are placed in sequence. Visual the diagram shows that the buffer has no real end. This type of buffer is more lends itself to data streams. In circular buffers data is added until the buffer is full. Once the buffer is full the section with the oldest data is overwritten. In reality circular buffers require three pointers, one to the actual buffer in memory, one to point to the start of valid data and one to point to the end of valid data
97557544.doc

Edmond Mitchell

Final Year Project Report

Figure 2 A ring showing, conceptually, a circular buffer vii Buffering is required in this project due to the varying rates of possible inputs for the program. Here the buffering uses the first in, first out (FIFO) method. Buffering will need to be implemented in this project in case the rate of data flow changes from the DAQ to feature receiver or from the feature receiver to the GUI.

8 BANDPASS FILTER 9 KEY CHALLENGES 9.1 Multiple Wii Devices 9.2 Realtime Graph 9.3 Connecting to serial port 9.4 Storing Serial Data 9.5 Hooking Keys 9.6 Embedding the scripting language

97557544.doc

Edmond Mitchell

Final Year Project Report

97557544.doc

10

Edmond Mitchell

i ii

The UKs definitive text related information source - http://www.text.it LAST ACCESSED 18/12/08 http://www.wiili.org/index.php/GlovePIE:GlovePIE_Scripts The C# Programming Language By Anders Hejlsberg, Scott Wiltamuth, Peter Golde 2003 Threading limitations - http://docs.python.org/library/thread.html LAST ACCESSED 18/12/08 Scripting - http://home.pacbell.net/ouster/scripting.html#925015 LAST ACCESSED 18/12/08 Advantages of scripting -- http://www.builderau.com.au/program/java/soa/Java-trends-ScriptingCircular buffer image - http://www.citengineering.com/LuaVIEW/dl_pics/circular_buffer.png LAST ACCESSED

iii iv v vi

languages/0,339024620,320274773,00.htm LAST ACCESSED 18/12/08

vii

18/12/08