TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

INTRODUCTION 1.1 OVERVIEW In this chapter you will come to know about What is a Virtual Keyboard? overview of the project, background of the project, planning of the project and how exactly the project work has been organised? How we got the idea of making this project? Virtual keyboard (VKB) is a device which can be used in any environment, with any computing device, without carrying a big keyboard. This device can be placed at any flat surface like table, desk etc. In today's world there are ample portable and miniature devices produced with texting capability. Though the size of the devices are getting smaller but not the size of our figures which type the data, so it becomes very hectic to type such buttons or touch screens. VKB helps to overcome this problem by having conventional keyboard without actually carrying it to anywhere.VKB can be connected to all those devices which helps for texting data like desktop, laptops, PDAs, mobile phones, tablet PCs etc. A VKB is a form of computer input device whereby the image of a virtual keyboard is projected onto surface: when a users fingers are placed on the projected keys, the device translates them into keystrokes.

1.2 BACKGROUND An optical virtual keyboard was invented and patented by IBM engineers in 1992. It optically detects and analyses human hand and figure motions and interprets them as operations on a physically non-existence input device like surface having painted on projected keys. In that way it can emulate unlimited types of manually operated input devices (such as a mouse, keyboard and other devices). All mechanical input units can be replaced by such virtual devices, optimised for the current application and for the users physiology maintaining speed, simplicity and un ambiguity of manually data input. In 2002, the start-up company CANESTA developed a projection keyboard using their proprietary electronic perception technology. The company subsequently licensed the technology to celluon of korea.

Page 1 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

A proposed system called the P-ISM will combine the technology with a small video projector to create a portable computer of size of a fountain pen.

1.3 PROJECT PLANNING As this is a research project lot of things and information was necessary to acquire from different books and web so that the ultimate goal can be achieved. After deciding the project is was important to find out components required in this project. Once the components were gathered then testing those components for whether they meet the system requirements was important. The detailed outline of the planning the project is explained in the following flowchart.

Fig. 1.3 Flowchart showing planning of project First decided the project and according searched for the components required for the project in the market. Then testing of the components was important to see whether they satisfy the need of the system. Finally mounting them on PCB and testing the overall project.Then will see all the detailed study of the project in the later chapter.
Page 2 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

1.3 ORGANIZATION OF THE PROJECT REPORT

The detailed study and the references are incorporated in the project report.There was the need of datasheet of all the different components and the websites from where collected the data for the project organization.The completion of the project work has done in small modules and then finally assembled all the modules in to one report. All the necessary reference books, IEEE papers and the datasheets are referred from the web. Form the information all the circuit components has obtained and started with the designing of the circuit. After finalising the design actually the circuit built it on PCB, and then the code has written in C language according to the required working conditions.After that checked the hardware working of the project and then checked the output at different conditions and finally finalised the results.

At first step after deciding the project name it was necessary to get as much information as possible for the bulding up the project. Later searching for the components and detail survey of the components was done. This is explained in the chapter1. Once the components were decided block diagram and design of the circuit diagram was done which is briefly explained in the chapter 3. Later the manufacturing of the PCB for the interfacing of the camera and controller was done whose detailed study is explained in the chapter 4. All the experimentation works are explained in the chapter 5. Finally the results of the project and debugging of the errors are explained in the results and discussion chapter 6.

Page 3 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

LITERATURE SURVEY 2.1 INTRODUCTION Virtual Keyboard is just another example of todays computer trend of "smaller and faster". Computing is now not limited to desktops and laptops, it has found its way into mobile devices like palm tops and even cell phones. But what has not changed for the last 50 or so odd years is the input device, the good old QWERTY keyboard. Alternatives came in the form of handwriting recognition, speech recognition, abcd input (for SMS in cell phones) etc. But they all lack the accuracy and convenience of a full blown keyboard. Speech input has an added issue of privacy. Even folded keyboards for PDAs are yet to catch on. Thus a new generation of virtual input devices is now being paraded, which could drastically change the way we type. Virtual Keyboard uses sensor technology and artificial intelligence to let users work on any surface as if it were a keyboard. Virtual Devices have developed a flashlight size gadget that projects an image of a keyboard on any surface and lets people input data by typing on the image. The device detects movement when fingers are pressed down. Those movements are measured and the device accurately determines the intended keystrokes and translates them into text. The Virtual Keyboard uses light to project a full-sized computer keyboard onto almost any surface, and disappears when not in use. The translation process also uses artificial intelligence. Once the keystroke has been decoded, it is sent to the portable device either by cable or via wireless. The Virtual Keyboard uses light to project a full-sized computer keyboard onto almost any surface, and disappears when not in use. Used with Smart Phones and PDAs, it provides a practical way to do email, word processing and spreadsheet tasks, allowing the user to leave the laptop computer at home. The technology has many applications in various high-tech and industrial Sectors. These include data entry and control panel applications in hazardous and harsh environments and medical markets.

2.2 SUMMARY Virtual keyboard (VKB) is a device that tried to make which can be used in any environment, with any computing device, without carrying a big keyboard. This device
Page 4 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

can be placed at any flat surface like table, desk etc. As everyone knows today there are ample portable and miniature devices produced with texting capability. Though the size of the devices are getting smaller but not the size of our fingers which type the data, so it becomes very hectic to type such small buttons or touch screens. The detailed study and the references are incorporated in the project report. There was the need of datasheet of all the different components and the websites form where collected the all data for the project organization. The completionof the project work is done in small modules and then finally assembled all the modules in to one report. All the necessary reference books, IEEE papers and the datasheets are referred from the web1.

: are the references to the appendix B

Page 5 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

DESIGN AND DRAWING 3.1 BLOCK DIAGRAM The designing of the project mainly consists of the block diagram and the circuit diagram of the entire system. In this chapter you will come across the block diagram with its explanation and circuit diagram with its description. To understand any process it is very important to know the block diagram of the system, because it gives us an overall idea behind the project and how exactly the process is executed. The block diagram is shown below and explanation is followed with it.

Fig. 3.1 Block diagram of the system 3.1.1 Explanation of block diagram: The block diagram shows a VKB module it consists of three important components as: Camera Pattern projector Laser

Page 6 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

Each of these devices play very important role in the working of the project, while the heart of the project is microcontroller.

a) CAMERA: The CMOS camera continuously captures images of the region where the printed keyboard is supposed to be placed and checks these images for red color data above a specified threshold. The threshold concept works in this case because the laser shining on a typical human finger generates saturating values of red color data, which is very easily distinguishable from its surroundings.

Fig. 3.1.1 (a): Image sensor module mounted on a custom PCB

b) LASER: The laser beam is simply a conventional off-the-shelf red laser with a line-generating diffractive optical element attached to it. This assembly generates an invisible plane of red light hovering a few millimeters above the typing surface. When a finger passes through this plane, it shines bright red in that region.

Page 7 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

Fig. 3.1.1 (b): Red laser module with a line-generating DOE attached c) PATTERN PROJECTOR: The Pattern Projector or optional printed image presents the image of the keyboard or mouse zone of the system. This image can be projected on any flat surface. The projected images are that of a standard QWERTY keyboard, with all the keys and control functions as in the keyboard. The Projector features a wide-angle lens so that a large pattern can be projected from relatively low elevations. A printed image, with replaceable templates allows system flexibility, permitting most any kind of keyboard configuration for greater functionality.

Fig. 3.1.1 (c): Pattern projector

3.2 SYSTEM FLOW GRAPH The system flow graph gives the flow of working of the module of the project. It is explained in detail below with the figure of system flow graph fig. 3.2 Basically at first the pattern projector projects the actual image of the keyboard which may be of desired color for e.g. red color. Now the red laser beam is passed over the image of keyboard. After calculations it has been decided that the beam should be passed just 3mm above the image for proper working.
Page 8 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

Firstly the pattern of keyboard is generated by a pattern projector

Laser beam is passed over the projected image.

While typing our finger cuts the laser beam

Due to the cut the finger shines bright red and red color data crosses the threshold value at that position

The associated ASCII value stored in the controller is passed to the computing device. Fig. 3.2: System flow graph Now when started typing like if letter s is typed then the laser beam is cut by our finger and the finger shines bright red. This transition to bright color is captured by the image sensor module. The red color data crosses the threshold value and then the coordinate of the key is calculated. Once the camera detects the key then the associated ASCII value is generated in the controller. As per the ASCII value the controller sends this information to the device about which button has been pressed. Controller saves the set of all the ASCII scanned codes of the keys on keyboard.

3.3 INTERFACING DIAGRAM The interfacing circuit diagram of the camera and controller is shown below in fig. 3.3 and the explanation of the same is given below.

Page 9 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

Fig. 3.3: Interfacing diagram between camera and microcontroller In the above diagram the CMOS camera and the microcontroller is interfaced. The controller is a 40pin IC which is interfaced with a CMOS color sensor module.

Page 10 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

MANUFACTURING

4.1 PRINTED CIRCUIT BOARD A printed circuit board consists of one or more sheets of insulating material with thin copper lines on both surfaces that interconnect the components attached to the board. Printed circuit boards are used to route the electrical currents and signals through copper tracks which are firmly bonded to the insulating base. Thus large number of electronic components can be accommodated in small area with minimum wiring. Wiring error is also reduced. PCB is a neat, small and truly a work of art. Thus by using PCB electronic equipment become more reliable, compact in size and less costly. Single sided PCBs are used in electronic equipment like TV, radio, mobiles etc. double sided PCBS are used in sophisticated instruments like CRO, microprocessor, etc. 4.1.1 STRUCTURES, TYPES AND MATERIALS USED IN PCBs A printed circuit board is a flat laminated panel of some insulating material like glass, epoxy, polyester or paper phenol materials. Interconnections between different components mounted on the board are made by thin conducting paths on the surface of board or in alternating layer sandwiched between layers of insulating material. Conducting path are made of copper and called tracks. Other copper area is called as lands and is available on board surface for attaching and electrically connecting the components. The usual thickness of the substrate layer is 0.8 to 3.2 mm. The material used for the PCBs structure must be electrically insulating. There are three main types of PCBs1. Single sided: copper foil is only on one side. 2. Double sided: copper foil is on both side of substrate. 3. Multilayer PCB: consists of alternative layer of conducting copper foil and insulation.

4.1.2 MAKING OF LAYOUT AND ART WORK The PCB layout designing is nothing but the mirror image of components layout. The components layout is drawn by looking from the components side, whereas the PCB

Page 11 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

layout is drawn by looking from the copper side. The making of layout and art work consists of many things. The following block diagram gives us a brief outlook over the making of PCB.

Fig. 4.1.2: Making of layout and art work of PCB The explanation of the above block diagram is given below: Testing the circuit: The system circuit diagram is tested and connected as of the need in required software. Preparation of layout: Then as per the circuit diagram the layout of the PCB is designed using software. Art work generation and inspection: Art work is done on the laminates. Laminates are of two types1. Phenol Boards 2. Fiber glass epoxy boards. There are two methods of transferring PCB layout on the laminate1. Ink drawing method. 2. Screen drawing method

Page 12 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

EXPERIMENTATION

5.1 TEST SET UP

WHY TESTING IS REQUIRED? Testing is vital for the success of the system. System testing makes a logic space assumption that the all parts of the system are correct and the goal will be successfully achieved. Effective testing of the system reduces number of errors and naturally reduces the cost and time.

5.11 TESTING As the software and the hardware is created and added to the developing system, testing is performed to ensure that it is working correctly and efficiently. Testing is generally focused on two areas i.e. internal efficiency and external effectiveness. The goal of the external effectiveness is to verify that the software and the hardware is functioning accordingly and performing all necessary functions or sub functions.

5.1.2 DETAILS OF EQUIPMENT USED IN TEST SETUP System specifications: Microcontroller: Atmega-32 Camera: CMOS camera Laser: Red laser 635nm class II Pattern projector USB port, cable or RS232 port and cable (Both male female connectors)

5.2 HARDWARE IMPLEMENTATION LASER Our original plan, at the time of the project proposal, was to use an infrared laser to detect button presses using the CMOS camera, but we realized that user safety would be a major issue in that case. The user would never know even if he/she is staring directly at the laser and, therefore, there would be no way to prevent eye damage. In addition, it was realized that the CMOS camera used (OV6630) is not very effective at
Page 13 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

detecting infrared light. Hence, it was decided to use a Class II 635nm red laser instead of a infrared light. The laser module bought came with a built-in driver; therefore, there was no need to worry about biasing the laser properly to make it operational. All it had to connect the laser to a 3V power source, which we obtained using a simple 3V voltage regulator.

Fig. 5.1.2 (a): Laser line generation calculation The laser module also came with a line-generating diffractive optical element attached to it. However, since the fan-angle for this DOE is not known,so by experiment with various distances in order to obtain a line length of at least 8.5, which was required to cover the entire width of our printed keypad,then finally get the idea of fan-angle for this DOE. In the end, to place the laser at a distance of approximately 12.5 to obtain good results. CAMERA For this project it was decided to use the C3038 1/4 color sensor. The two primary reasons why chose this specific camera module of low cost and the fact that it is capable of outputting image color data in progressive scan mode.

Fig.5.1.2 (b): C3038-4928IR 1/4 Color Sensor Module Progressive scanning was an important consideration since controller do not have enough computational power available on the 16 MHz Mega32 microcontroller to process entire frames at once; however, it can certainly process images line-by-line as

Page 14 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

they come in. After rigorous testing and a lot of research, then realized that this project VKB could work with only the red channel data from the camera and still be able to identify keystrokes accurately. Hence connected the 8-bit red channel output from the camera (UV [7:0]) to PORTA [7:0] on the Mega32. 5.3 SOFTWARE IMPLEMENTATION The software component was split into 5 main components: Implementing the I2C protocol to read and write registers from camera

Reading values from camera to obtain 6 frames every second Processing the images to obtain a pressed key Converting the pressed key into a scan code which is then transmitted using the PS/2 protocol

Sending serial data from a java application to update the array of scan codes in the Mega32 5.3.1 Main Operation At first initialize PORTA on the Mega32 to take UV input from the camera and PORTC to communicate with the camera over the I2C interface. The baud rate is set to 19,200bps for serial communication.Then run the calibrate function on the camera, which looks at a black keyboard to determine a distinguishable value for red color threshold. Then call a function called "init_cam" which performs a soft reset on the camera before writing the required values to corresponding camera registers. These registers change the frame size to 176x144, turn on auto white balance, set the frame rate to 6 fps, and set the output format to 16-bit on the Y/UV mode with Y=G G G G and UV = B R B R. The code then enters an infinite loop which checks for the status of the PS2 transmitting queue and tries to process the next captured frame if the queue is empty. If not, the queue is updated and the PS2 transmission is allowed to continue. 5.3.2 Image Processing The getRedIndex function captures rows of data from the camera and processes each of them.Everyone first wait until a neg edge on the VSYNC, which indicates the arrival of

Page 15 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

new frame data on the UV and Y lines.Then people wait for one HREF to go by since the first row of data is invalid. At this point,it can clock in 176 pixels of data for a given vertical line in the Bayer format.

Fig. 5.3: Bayer color pattern In the mode where the UV line receives BR data, the output is given by: B11 R22 B13 R24 and so on. Since there ws only need of red data,then it stored an array of 88 values in which captured the data on the UV line every 2 PCLKS. The OV6630 also repeats the same set of pixels for consecutive rows and thus 2 vertical lines processed would have data about the same pixels. So to optimize this by completely dropping data about the even rows, but this was not going to save us anything since all our processing could be done between one neg edge and a pos edge (when data becomes valid again) of HREF. Since there is no enough memory to store entire frames of data to process, so do the processing after each vertical line is required. After each vertical line of valid data, HREF stays negative for about 0.8ms and the camera data becomes invalid; this gives ample time to process one line worth of data. After each vertical line is captured, loops through each pixel to check if it exceeds the red threshold found during calibration. For every pixel that meets this threshold, then check it if the pixel is part of a contiguous line of red pixels, which would indicate the presence of a key press. If such a pixel is found, then map this pixel to a scan code by binary searching through an array of x, y values. If this scan code would found to be valid, we debounce the key by checking for 4 continuous presses, and then add the detected key to the queue of keys to send to the PC.

Page 16 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

5.3.3 I2C Communication A very big part of our challenge is to figure out the correct configuration to use to capture and process the images from the camera. The communication protocol is not easy to work with and there are total of about 92 registers it could use to set up the camera. At first for this project VKB the TWI interface provided by CodeVision to communicate with the camera was using, but it was unable to do so. Thus,it needed to modify and use a version developed by Ruibing Wang, which uses a lot of the TWI settings provided on the Mega32. The protocol uses a 2-wire communication scheme, which is activated by a 10kOhm pull-up resistor. The clock signal to the camera is provided by the SCL line, and the frequency is given by: 16MHz / (16 + 2 x (TWBR) (4TWPS)). So for this the optimal solution would be to satisfy the minimum requirement which is to set the bit rate register (TWBR) to 72 and the status register (TWSR) to 0. The rest of the code just follows the standard protocol defined by Philips. The camera registers could be written by writing a start bit, followed by a target register address and then the target data.There is no need to read from the camera registers except in the initial phase when it should be make sure the protocol working properly.

5.3.4 Camera Settings This camera uses a resolution of 176x144 since that is the minimum requirement to detect an entire A3 size paper on which the keyboard would be printed. At this resolution, it could capture at most 6 frames of color images per second. The camera output format is set to capture 16 bit UV/Y data, where UV has BRBR data and Y has GGGG data. The Y data is completely ignored.

Fig. 5.3.4: Screenshot of Java GUI to create custom keyboard layouts on-the-fly

Page 17 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

5.3.5 Programming the EEPROM Since it required to be able to change the key assignments on the fly, that stored the array of scan codes corresponding to each key in EEPROM and turned on the RS-232 receive interrupt.Java applet is used for writting the code which is a simple GUI where the user can enter scan codes of the keys they desire and transmit it to the microcontroller through a standard COM port on the PC.

5.3.6 Keyboard Output (PS/2) The code is structured using two timer compare interrupts where timer1 compare is used to start transmissions of each data byte and timer2 compare is used to reset the waiting. Since the protocol allows a range of frequencies that a computer would understand, for getting the keyboard output,clock time of 250 and wait time of 700 is used. When the timer1 interrupt is fired, it transmits the bits in the following order when the clock is set to high: start bit(0), data bits, parity bit(xor of all bits), and a stop bit(1). If not, the clock state is updated. The rest of the code simply maintains a queue which would hold the elements to transmit as characters. The queue has a get and put method that updates the 2 pointers in an array.

Page 18 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

RESULTS AND DISCUSSIONS Standards 6.1 I2C-BUS SPECIFICATION VERSION 2.1 In this chapter all the results obtained and discussion over the result is done after experimenting the project is explained. Two wires in an I2C bus, serial data (SDA) and serial clock (SCL), carry information between the devices connected to the bus. Each device is recognized by a unique address and can operate as either a transmitter or receiver, depending on the function of the device. In addition to transmitters and receivers, devices can also be considered as masters or slaves when performing data transfers. A master is the device which initiates a data transfer on the bus and generates the clock signals to permit that transfer. At that time, any device addressed is considered a slave. 6.2 PS/2 KEYBOARD PROTOCOL The PS/2 keyboard interface typically uses a bidirectional synchronous serial protocol, but for our implementation we do not need the computer (host) to communicate with the microcontroller (device). Therefore, for our purposes, the device always generates the clock signal and all data is transmitted one byte at a time. Each byte is sent in a frame consisting of 11 bits, in the following order:

1 start bit. This is always 0. 8 data bits, least significant bit first. 1 parity bit (odd parity). 1 stop bit. This is always 1. The parity bit is set to 1 if there is an even number of 1's in the data bits and set to 0 if there is an odd number of 1's in the data bits. The number of 1's in the data bits plus the parity bit always add up to an odd number (odd parity.) This is used for error detection. Data sent from the device to the host is read on the falling edge of the clock signal; the clock frequency must be in the range 10 - 16.7 kHz. This means clock must be high for 30-50 s and low for 30-50 s.

Page 19 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

6.3 ETHICAL AND LEGAL CONSIDERATIONS Throughout the final project, we committed ourselves to the highest ethical and professional conduct and closely adhered to the IEEE Code of Ethics. We placed an extra emphasis on the following points mentioned in the Code of Ethics: 1. To accept responsibility in making decisions consistent with the safety, health and welfare of the public, and to disclose promptly factors that might endanger the public or the environment. 2. To avoid real or perceived conflicts of interest whenever possible, and to disclose them to affected parties when they do exist. 3. To be honest and realistic in stating claims or estimates based on available data. 4. To improve the understanding of technology, its appropriate application, and potential consequences. 5. To maintain and improve our technical competence and to undertake technological tasks for others only if qualified by training or experience, or after full disclosure of pertinent limitations. 6. To seek, accept, and offer honest criticism of technical work, to acknowledge and correct errors, and to credit properly the contributions of others. 7. To avoid injuring others, their property, reputation, or employment by false or malicious action. 8. To assist colleagues and co-workers in their professional development and to support them in following this code of ethics. Since we were using a Class II laser device, we always made sure to keep the laser targeted away from other individuals in the Lab. In addition, we designed an enclosure for our device such that the laser would not be visible to the user. We did not have any legal considerations since we did not use code or algorithms from other sources, did not use parts regulated by federal agencies, and did not infringe upon any existing patents. Although a commercial product similar to our device, called the Virtual Laser Keyboard, is currently manufactured by a company known as I-Tech, we believe that we have significantly distinguished our product such that we will not encounter any copyright issues. The commercial product is not dynamically

Page 20 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

reconfigurable, uses a red laser to project a standard QWERTY keyboard pattern onto a surface, and uses an infrared laser for keystroke detection. Our product, on the other hand, uses a customizable printed keyboard, a red laser for keystroke detection, and custom keystroke detection algorithms. 6.4 TESTING Keystroke Accuracy: As a result of the limited viewing angle of the camera and positioning of the laser, we had to design and calibrate with various keypad layouts to make sure we could detect all of the buttons with reasonable accuracy. Our final design for the generic keypad and testing results (percentage accuracy) for this layout are given in Figure 3. For the testing, we tried 100 keystrokes per key and set the acceptance threshold at 70% for side areas and 80% for the central area. This means that if we can recognize a certain key accurately at least 70 or 80 times, respectively, out of the 100 times that its pressed, that key passes the test.

Fig. 6.1: Testing results for keystroke detection accuracy 6.5 SAFETY The human body is vulnerable to the output of certain lasers, and under certain circumstances, exposure can result in damage to the eye and skin. Research relating to injury thresholds of the eye and skin has been carried out in order to understand the biological hazards of laser radiation. It is now widely accepted that the human eye is almost always more vulnerable to injury than human skin.

Page 21 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

Figure 11: EM radiation absorption characteristics of the human eye The cornea (the clear, outer front surface of the eyes optics), unlike the skin, does not have an external layer of dead cells to protect it from the environment. Hence, the cornea absorbs the laser energy and may be damaged. The figure below illustrates the absorption characteristics of the eye for different laser wavelength regions. Since we only used a Class II laser in this project and provided a proper enclosure for the device such that it isnt directly visible to the user, special protection is not required for normal users. People with sensitive eyesight or other severe vision problems, however, might want to take some precautionary measures and should not use the device for extended periods of time. In addition, users are strongly advised not to look directly into the laser beam at any time.

Page 22 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

Page 23 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

CONCLUSION AND FUTURE SCOPE OF THE PROJECT 7.1 CONCLUSION In this chapter the conclusion of the project after studying it is explained. The application and scope of the project is also explained in this chapter. Although the final project was very satisfying, the result of it did not

completely meet expectations of desired result. The keyboard worked as predicted but typing speed was minimal (about 60 characters per minute) due to limited processing capabilities of the Mega32 microcontroller. 7.2 FUTURE SCOPE If people had more time for completion of this project ,then they would have liked to increase the theoretical maximum typing speed by possibly using another microcontroller in parallel or maybe even an external FPGA to do extra image processing. In addition, they would also like to include sound effects for keystrokes and a dynamic calibration algorithm which can be used to orient the custom-printed keyboard in any direction. This sort of functionality would require performing 2D image transforms on-the-fly, which is not feasible with the existing microcontroller. Last but not least, they could certainly try to improve our current keystroke detection algorithm to improve typing accuracy. 7.3 ADVANTAGES

1. It can be projected on any surface or you can type in the plain air. 2. It can be useful in places like operation theatres where low noise is essential.

3. The typing does not require a lot of force. So easing the strain on wrists and hands. 4. The Virtual Keyboard is not restricted to the QWERTY touch-typing paradigm, adjustments can be done to the software to fit other touch-typing paradigms as well. No driver software necessary. 5. It can be used as a plug and play device. 6. High battery life. 7. The standard coin-sized lithium battery lasts about eight months before needing to be replaced.

Page 24 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

7.4 DRAWBACKS

1. Virtual keyboard is hard to get used to. Since it involves typing in thin air, it requires a little practice. Only people who are good at typing can use a Virtual keyboard efficiently. 2. It is very costly. 3. The room in which the projected keyboard is used should not be very bright So that the keyboard is properly visible.

Page 25 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING

TO DESIGN A VIRTUAL KEYBOARD TO INPUT ANY COMPUTING DEVICE

APPENDIX B REFERENCES

http://www.newscom.com/cgi-bin/prnh www.canesta.com www.procams.org www.billbuxton.com/3state.htm1 www.smarttech.com www.3m.com/us/office/meeting/product_catalog/wd.jhtml

Page 26 S.C.O.E. ELECTRONICS AND TELECOMMUNICATION ENGINEERING