Juliette TAILLANDIER Promotion 2008

INTERNSHIP REPORT Design and Development of Interfaces for Mobile Narratives

Location: HCIL, University of Maryland (USA) Dates: September 4th to December 22nd, 2006

Department where the internship was accomplished

UMIACS Institute for Advanced Computer Studies A.V. Williams Building University of Maryland College Park, MD 20742 -USA-

UMIACS Director
Professor V.S. Subrahmanian 301.405.2711 Tel: Fax: 301.314.9658 E-mail: vs@cs.umd.edu Website: http://www.cs.umd.edu/~vs/

Internship supervisor
Dr. Allison Druin Director, Human-Computer Interaction Lab www.cs.umd.edu/hcil Associate Professor University of Maryland College of Information Studies and Institute for Advanced Computer Studies Tel: 301.405.7406 E-mail: allisond@umiacs.umd.edu Website: www.umiacs.umd.edu/~allisond

Special thanks

To the University of Maryland. To Dr Allison Druin who allowed me to do my internship in the Human-Computer Interaction Laboratory (HCIL). To Jerry Fails, for giving so much of his time to help me. To the Human-Computer Interaction Laboratory employees and students, for being so friendly and patient with me

Internship subject
The laboratory has been working with children for almost ten years now. Therefore, more and more applications are designed for them, with their help. The objective of the project I worked on was to create mobile and interactive narratives on PDAs. In other words, the future application could allow a child to read a story, modify the text, image and sound and communicate with other children through wireless. A prototype version had been developed a month before I arrived but had some bugs in the sound code. My role was to fix the program and make it stable. Later, I worked on the infrared authentication program and at the end of the internship I did some research on possible extensions of the project.

Sujet du stage
Cela va faire dix ans que le laboratoire travaille avec des enfants. Par consquent, de plus en plus d'applications sont conues spcialement pour eux et avec leur aide. Le but du projet sur lequel je travaillais est de crer des livres d'histoires interactifs et mobiles (sur des PDA). En d'autres termes, le programme permettra un enfant de lire une histoire, de modifier le texte, l'image et le son, et de communiquer (sans fil) avec d'autres enfants. Un prototype avait dj t dvelopp un mois avant que j'arrive mais la partie son comportait des erreurs. Mon rle fut donc de rparer ce programme et de le rendre stable. J'ai ensuite travaill sur le programme d'authentification par infrarouges et vers la fin du stage j'ai effectu des recherches pour savoir s'il tait possible d'ajouter des fonctionnalits au projet.

Table of contents
Introduction...................................................................................................... 1 University description ..................................................................................... 2
University of Maryland ....................................................................................................... 2 College of Computer Science ............................................................................................ 2 UMIACS ............................................................................................................................ 3 HCIL .................................................................................................................................. 4

Project descriptions ........................................................................................ 6

ICDL .................................................................................................................................. 6 Mobile Storytellers ............................................................................................................. 7

Coding an audio player and recorder in C# .................................................. 9

First step: familiarize with C# ............................................................................................. 9 Creating a desktop audio player (using mciSendString)..................................................... 9 Creating a desktop audio recorder (using mciSendString) ................................................10 Switching to the WaveOut and WaveIn classes................................................................12 Creating an audio player for desktop (using WaveOut).....................................................12 Creating and audio player for PDAs (using WaveOut) ......................................................13 First step to recording data ...............................................................................................13 Recording data and saving it into a file .............................................................................14 Fixing the noise ................................................................................................................14 Desktop audio player and recorder...................................................................................15 PDA audio player and recorder.........................................................................................15

What is Kidsteam?......................................................................................... 17
History ..............................................................................................................................17 Description .......................................................................................................................17 Group formation................................................................................................................17 Sponsors ..........................................................................................................................17 Children as design partners..............................................................................................18 My role during Kidsteam sessions ....................................................................................20

Testing the Mobile Storytellers with Kidsteam........................................... 21

First session .....................................................................................................................21 Second session ................................................................................................................22

PDA interaction via Infrared ports ............................................................... 24

Why do the devices need to communicate?......................................................................24 Why use IR?.....................................................................................................................24 Program developed ..........................................................................................................25 Possible solutions for data transfer ...................................................................................28

Possible extensions to the project .............................................................. 30

Adding video to Mobile StoryTellers..................................................................................30 Implementing the program on cell phones ........................................................................31

Conclusion ..................................................................................................... 33

Internship report

Introduction
According to the NCES (National Center for Education Statistics), nearly 90% of American children enrolled in primary school use computers and half of them also surf on the Internet. This shows that today, technology represents a big part of a child's life. Children are used to have computers around them (whether it is at school or at home) and tend to know more and more how they work. The project I worked on during those four months focused on how we could take advantage of those figures by creating interactive and digital narratives for children. The idea was to be able to read, modify and create stories on a mobile technology and exchange them with others. My role was to concentrate on the sound and infrared authentication parts of the program. I was also involved in another project called Kidsteam. Finally, I have done some research to see if the Mobile Storyteller project could go any further. In the first part I will present the University of Maryland and the lab I worked in. Next I will talk about a previous project and explain further in detail what the Mobile Storyteller program will do. Then I will expose the modifications I have made to the sound code of the program. In a third part we will see what is Kidsteam and how it helped the project with its feedback. After that, I will describe my role in the IR authentication part of the project. Finally, we will see what are the possible extensions to Mobile Storytellers.

Internship report

University description
University of Maryland
Located in College Park (Maryland), the University of Maryland was created in 1856, more than 150 years ago. In 1916 it admitted the first women students. It now has more than 35,000 students enrolled each year. About 12,000 employees work on campus. Using technology, the University provides selected quality programs to audiences worldwide to share its knowledge and extend and enhance educational opportunities. The University of Maryland shares its research, educational and technological strengths with businesses, government and other educational institutions. Here is a short summary of its financial resources for 2006:

Financial Summary 2006

4% 15% 2% 21%

Contracts and Grants: $247,700,000 Tuition and Fees: $332,700,000 State Appropriation: $327,500,000 Educational Activities: $26,600,000

28%

30%

Auxiliary Enterprises: $178,100,000 Other: $43,900,000

We can see that grants and donations are a big part of the University's yearly income. In 2006, the university is 18th in the ranking of national public universities and in the top 20 for public research universities.

College of Computer Science

Computer science was introduced in American universities in the 1940s. Five universities played important roles in establishing computing: MIT, Harvard University, the University of Pennsylvania, Columbia University, and Princeton University. In the 1950s, with the availability of computers, computing was introduced at other universities. It started as a formal discipline at Maryland on February 1, 1962 (when the Computer Science Center was formed).

Internship report

The first degree program in computer science at Maryland was the Master of Science Degree, which started in September 1967. The Computer Science Center developed a proposal to initiate a Bachelor of Science degree in computer science in the spring of 1973. Whereas most departments of computer science in the United States arose out of either Engineering, Mathematics, or Physics Departments, the Department of Computer Science at the University of Maryland, College Park, arose out of the Academic Computer Center. The Computer Science Center and the Department of Computer Science were formally split on July 1, 1973. Today Maryland's Computer Science Department is ranked 13th in the US. Despite the late integration of computers in the university, the Computer Science Department's research is very developed. It covers different subjects, such as:

Algorithms and Theory of Computation Artificial Intelligence Bioinformatics and Computational Biology Computer Vision Databases Graphics High Performance Computing Human Computer Interaction Numerical Analysis Programming Languages Software Engineering Security Systems

UMIACS
The mission of UMIACS (University of Maryland Institute for Advanced Computer Studies) is to foster and enhance interdisciplinary research and education in computing across the College Park campus. Since its inception, UMIACS has played a major role at the University of Maryland in building strong interdisciplinary research programs, cutting-edge computing infrastructure, and long-term partnerships with national and international research centers. The Institute's programs are led by distinguished researchers, many of whom hold joint appointments in strong academic units such as Computer Science, Electrical and Computer Engineering, Linguistics, Geography, Philosophy, Business, Education, and College of Information Studies. Since computing is at the core of all the Institute's activities, UMIACS has a uniquely close relationship with the Department of Computer Science whose graduate program has been consistently ranked very high by all national rankings. The
3

Internship report

synergistic environment provided by UMIACS enables innovative collaborations between the Computer Science faculty and other faculty on campus. The infrastructure provided by UMIACS is primarily geared toward supporting interdisciplinary research while the core computer science projects are primarily conducted through the Department of Computer Science. UMIACS contains several research labs, which are:

CfAR: Center for Automation Research CBCB: Center for Bioinformatics and Computational Biology CDIG: Center for Digital International Government CHESS: Center for Human Enhanced Secure Systems CLIP: Computational Linguistics and Information Processing CVL: Computer Vision Laboratory DSSL: Distributed Systems Software Laboratory FCMD: Fraunhofer Center at Maryland GLCF: Global Land Cover Facility GVIL: Graphics and Visual Informatics Laboratory HCIL: Human Computer Interaction Laboratory KECK: Keck Laboratory for the Analysis of Visual Motion LAMP: Language and Media Processing Laboratory LCCD: Laboratory for Computational Cultural Dynamics LPCD: Laboratory for Parallel and Distributed Computing MIND: Maryland Information and Network Dynamics Laboratory PIRL: Perceptual Interfaces and Reality Laboratory

HCIL
Founded in 1983, the Human-Computer Interaction Lab (HCIL) at the University of Maryland conducts research on advanced user interfaces and their development processes. It designs, implements, and evaluates new interface technologies that are universally usable, useful, efficient and appealing to a broad cross-section of people. Interdisciplinary research teams study the entire technology development life-cycle, which includes initial technology design, implementation issues, and evaluation of user performance. This work has developed new theories, methodologies, and technologies. Current work includes new approaches to information visualization, interfaces for digital libraries, multimedia resources for learning communities, zooming interfaces (ZUIs), technology design methods with and for children, and instruments for evaluating user interface technologies.

Internship report

The HCIL is an interdisciplinary lab comprised of faculty and students from Computer Science, Education, Psychology and Information Studies. The HCIL has conducted a broad range of research over the years, and continues to pursue several areas in depth. All projects are listed below, and are grouped by area:

Communities Design Process Digital Libraries Education Physical Devices Public Access Visualization

Internship report

Project descriptions

ICDL
The project I worked on used another project called ICDL (International Children's Digital Library), which started in November 2002. It is a virtual library of international children's book available around the world. It was developed with two kinds of users in mind: children ages 7-11 willing to read books and parents who want to read books to their children. Only physical books are collected. To make sure that all the books are available online, they are all scanned. The website is designed in collaboration with children to facilitate navigation and searching. One of the primary goals of the ICDL is to "create a collection of more than 10,000 books in at least 100 languages that is freely available to children, teachers, librarians, parents and scholars throughout the world via the Internet". In addition, the people in charge want to give every child a possibility to read a book in his or her mother tongue. The National Science Foundation (NSF), the Institute for Museum and Library Services (IMLS) and Microsoft are the main sponsors of this project. There are also a couple of volunteers from around the world who donate their time to help test software, conduct use studies, ensure that language usage makes sense to native speakers etc. Country or culture experts select books from the library. They seek for works with historical value as well as contemporary literature. Books are presented whole in their original languages. These books come from around the world and are totally free. Every book on the website can be read in its integrity. In other words, everyone can now connect to the Internet to read a selection of children books from around the world and in different languages. Before the beginning of this project, the people in charge contacted all the authors and editors of the books they wanted to put so that the website users could read all the books legally. There were two types of books: the ones already in public domain and the ones protected by copyrights. For the latter the directors had to sign some contracts to have the right to publish the book on the Internet in spite of the copyrights. A third option was that an author, a publisher or a national library that had the copyrights for a book donated the book to the project.

Internship report

Mobile Storytellers

Description

The project I worked on consisted of developing a new software. With this application children could load a book from the ICDL to a PDA and change the text, the image and the sound. They could also reorder the pages and create new pages if they wanted to. One of the singularities of this project is that the PDAs were able to communicate with each other. Thus, kids could read the same story together but on two separate devices (the display of the story being shared between the PDAs). The role of each PDA (or clients) would be assigned by a desktop (or server). This program could also be used to exchange stories. For instance, let's say one of the kids modified one story; he could then send it to a friend. The communication will use infrareds for identification and wireless for data transfer.

Piccolo.Net and PocketPiccolo.Net

To display the pages of the books loaded, the program used a software called Piccolo. It has been developed by the lab in 2004 and is open source and totally free. It is a toolkit that allows you to build graphical applications. It is a way to create visual effects such as animation (through event-handling) and ZUI (Zoomable User Interface). The former is helpful to show little information in the overview. Then when the user zooms in, it gives more details (the data is either bigger or is more specified). For example, here are some print screens from the Mobile Storyteller program.

overview of a story

a page of the same story zoomed in

Internship report

There are three versions of Piccolo called Piccolo.Net (developed in C#), Piccolo.Java and PocketPiccolo.Net (for the Compact Framework). The program we made used both Piccolo.Net and PocketPiccolo.Net.

Why use C#?

When I arrived to start this internship, a prototype of the project had already been completed. It was written in C#. We could ask ourselves: why choose this language? First, to be able to use Piccolo the program had to be written in either Java or C#. Of course, the project could have been done without any ZUI friendly software but it was silly not to take advantage of it. Also, the project works with PDAs as well as desktop so Piccolo was the perfect match since it had a specific version for each one. But the main reason why C# was used is that it is better than Java for this project. It offers more stability. Moreover we did not need the portability of Java since desktops and PDAs have Windows and respectively the .Net Framework and .Net Compact Framework.

Internship report

Coding an audio player and recorder in C#

First step: familiarize with C#

As this project was already developed in C# (a programming language I didnt know), I spent the beginning of my internship learning it. I started by reading a book for beginners, which helped me understand the basics. C# is an object oriented programming language and as such it is structured in classes. Programs are also organized differently as the data is encapsulated, therefore allowing more security and flexibility than usual programming. Having done some Object Oriented Programming before, I already knew the essential notions of abstraction, encapsulation, polymorphism and inheritance, which was very helpful. As the last programming language I worked with was C, I was already familiar with the words and syntax. But I had to force myself to think in an object oriented way. This was quite difficult and needed some getting used to. After finishing the book, I had to get some new support to learn more about C#. I turned to the Internet to get some tutorials. Not only did they teach differently but they also gave a lot of examples. I spent the next couple of days testing each and every one of them in Visual C#. Even though it was simple console applications, it was a way for me to get accustomed to C#s way of thinking. Moreover, I had designed in the past some simple interfaces in Pascal manipulating most of the common objects (button, label) to navigate between windows for instance. I also knew how to handle events the user will generate. Therefore, I managed to adapt quite easily to graphical design. Understanding C# and being able to write some simple programs using this language was not as difficult as I thought. I was able to apply the knowledge I had from my previous years to learn faster. That way, I was operational at the very beginning of the internship.

Creating a desktop audio player (using mciSendString)

The first mission I had was to design a program in C# that could play and pause an audio file on the desktop. I started by taking a look at the desktop version of the existing project to see how it worked. It manipulated multiple buffers and several threads using the WaveOut method. As this program was bugging (it froze the application) I decided to change the playing method it called. I then went on the Internet to search for information on Microsofts default methods to play audio files. I found three solutions: WaveOut, PlaySound and mciSendstring. At first, I decided to work with the PlaySound method because it seemed to be the simplest way to play a file. Using this method taught me about dynamic libraries. As a matter of fact, the .Net Framework does not support sound. To play an audio file I needed to import a Dynamic Link Library (or DLL), which is a library of executable functions or data that can be
9

Internship report

used by a Windows application. It allowed me to call some of Windows' C++ predefined methods. Searching a little more on each function made me realize that PlaySound can play and stop a file (but not pause) and that mciSendString can do the three. As one of my objectives was to be able to pause the file, I had to find a new solution. I decided to switch and use mciSendstring instead of PlaySound. Windows' Media Control Interface (or MCI) provides a high-level, device independent interface for controlling multimedia devices. With it, your code can pretty much support any multimedia device and allows you to operate a file on a waveform audio device. MCI basically provides two ways of accessing the low-level multimedia devices: through a structure of messages, or by passing strings into the interface (using mciSendstring). This method passes a string as an argument, accomplishing different things. For instance, for playing a file the argument would be "play mysound", mysound being the device to play the sound to. The good thing about this method is that every action you do is very easy. You just have to call mciSendstring and change the command string (which is the argument of the method). Therefore, I was able to play, pause and stop an audio file without any problem.

Creating a desktop audio recorder (using mciSendString)

Since I did not have any record functionality in my program, my project was not over yet. Looking at all the MCI command strings I found out that one of them is used for recording an audio file. To call it you have to define a buffer that will store the recorded data. Then, another command string can stop the recording and a third one saves it into a file. By default, the file is saved as a WAV file. This audio format uses a lossless compression. Therefore no data is lost and the quality is higher. Moreover, a WAV file can be read by most of the media players (even by PDAs). The only con with this format is that the level of compression is very low, and sometimes null. Thus, we would expect the recorded file to be quite large. After a few tests, it turns out that a 20s recording only takes 200kB. Below we can see an image of the final version of the program. All the functionalities needed have been brought together in a single interface to help the user.

10

Internship report

This program can: open an audio file (using the openfile dialogbox) The user can browse the folders on his or her computer and choose a WAV file he or she wants to open. Once the OK button is clicked the path of the file appears in the label next to the browse button. play and pause an audio file In order to do so, the user needs to click on the play ( ) or pause ( ) buttons. Each time an action begins, it is written in the label on the right (in the device screen). That way, the user knows what is going on and does not click twice on the same button because it does not seem to work. record a WAV file When the user pushes the record button, the program starts recording data right away. To end this, you have to click on the stop button. To transfer the buffers data into a file, one should hit the save button. It will open a save dialogbox and create a new WAV file in the folder chosen.

11

Internship report

modify the volume This was not mandatory but I thought it would be handy. The user can increase or decrease the volume, using a trackbar. If the sound is set as stereo, the volume will be changed on both channels.

Switching to the WaveOut and WaveIn classes

Finishing this program helped me understand a little more about C# and WAV files. This code compiled and had all the functionalities I wanted. The only problem was that it only worked on desktops using the .Net Framework. What I also needed was a program that did the exact same things using the .Net Compact Framework from Pocket PCs. Looking at the mciSendstring function once again I saw that it was a very practical method for desktops only. As a matter of fact, the Compact Framework did not support it. The only option left was to use the WaveOut and WaveIn classes (to play and record). These are actually C++ classes that contain several methods employed to handle buffers. They compile on both the .Net Framework and Compact Framework.

Creating an audio player for desktop (using WaveOut)

At this point, I got paired with an undergraduate student to complete this program. He was working part-time but still managed to achieve a lot. After looking at our options we decided to start coding for desktops first manipulating only Compact Framework methods. We knew that the code for PDAs would not be very different. Moreover, the initial program for desktop did not play any sound so we would not work for nothing. We found some new material on the Internet. The WaveOut class provides a way to play a file working with one or two buffers. The first choice implies that the buffer will be large enough to contain the entire file. On the other hand, the other solution handles two smaller buffers. While one buffer plays, the other is loaded and queued. Thats how we chose to play a WAV file because it lets you use less RAM (Random Access Memory). It may be insignificant on desktops but its a criterion to consider on Pocket PCs (for more details see annex 1). We managed to play an audio file using WaveOut methods but we still had three problems: the playing looped, we didnt have any pause button and the stopping randomly froze the application.

I took us a bit of time to figure out how the program worked. After that we were able to locate the line in the code that made the playing looped. In fact, that line initialized the position of the cursor to 0 if the file ended. We commented that line, which made the playing stop once the end of the file was found.
We then had to introduce a pause function. This was not very complicated as the WaveOut class has a pause method. The tricky part was to add some Booleans to be sure to pause the file at the right time. Also if the file was paused we had to call the resume function to play it, otherwise we just called the play method.
12

Internship report

The last problem we had to deal with was to see why the stop button froze the application. We did not find where the error was. On the other hand, we did not spend a lot of time on this as we did not need this stop button (the file stopped by itself once it finished playing).
Creating and audio player for PDAs (using WaveOut)
We created a new project specially made to be deployed on PDAs and copied the desktop project into it. At first it didnt build because of two errors. The first one was that the GetString function had only one argument and was supposed to have three to be able to run on Pocket PCs. We switched back to three arguments, which corrected the mistake. The second error was located in the DLL import. In the desktop version we imported a library called "winmm.dll". It is a module for the Windows Multimedia API, which consists of low-level audio and joystick functions. It contains all the WaveOut functions that we called for the playing. The problem is that PDAs run on another operating system called Windows CE. This operating system does not recognize the DLL winmm. Instead, it uses another one called coredll. So we modified the DLL imported. The code then compiled and executed correctly. One of the advantages to employ the WaveOut methods was to have the exact same code for both versions (desktop and PDA). As a matter of fact, it required less maintenance and saved some time. At this point, the two programs still had two differences: First, the GetString method did not need the same number of arguments in both cases. But as the .Net Framework could also accept three arguments instead of one, we did not need to change anything. Finally, we could not use the coredll library on desktops because they were running on Windows XP. So we inserted a test that set the library to import as winmm for desktops and coredll for PDAs. We tested both versions of the program and were able to open, play and pause different WAV files. Thus, we finished the program for both desktop and PDAs.

First step to recording data

The second part of this project was to build an audio recorder for PDA and desktop. We already had a desktop version that worked (using mciSendString) but as for the playing we figured we could write the same code for both version. In order to do that, we had to call methods from the WaveIn class. We found a program on the Internet that recorded audio files on a desktop. The only problem was that it did not store the data it recorded but played it right away. That means that when you recorded your own voice, you could hear it with a 1 to 2 seconds delay. We used that program to start with.
13

Internship report

Recording data and saving it into a file

The program recorded data and stored it in a buffer (which was actually a byte array). Once it was full, the recording continued in another buffer while the first one was being played. We modified this code a little bit so that the full buffer gets copied in a larger byte array (of type MemoryStream) that would contain the whole recording. Once the user clicked the stop button the recording stopped. Then, the program created a file and wrote the header and all the recorded data into it. We could have used the buffer type to define both byte arrays (the one that stores part of the data and the one that saves the whole recorded data). We chose not to because a buffer can store several types of data and we knew we would only work with bytes. Also, it was not guaranteed to be thread safe, meaning multiple threads could not concurrently read from an instance of this type (which was a problem since we were using multithreading). Instead, we opted for a byte array for the small buffer and a MemoryStream for the larger buffer (because we did not know the exact size we would need and this type provided a resizable byte array). We tested the original program, which played back exactly what we tried to record. When we tested the modified program, it just played noise. We knew that the byte array had the correct data since the playback worked. Our first thought was that the data written into the file was incorrect. There were several possibilities. The first one was that the wrong buffer was copied into the MemoryStream byte array. We looked at the code and found no error. The second option was that the data written into the audio file was noise. That means that the MemoryStream variable contained noise too. To see if it was the case, we checked that the copy between the two byte arrays was made. We did not look at every data in each buffer because there were thousands of them. But we did compare one hundred or so, which were identical. We assumed that the copy was being made correctly. Then we had to see if the recorded data was correctly copied into the file. To check that, we looked at the saved file in the binary reader and chose a couple of lines that we compared to the buffer. Every value seemed to be copied correctly. From all those tests we assumed that the data was correct. We then checked the header of the file (for more details, please refer to annex 1). At first we thought there was a problem with it because some of the bytes were not in the correct order. After reading more about WAV files we found out that some of the fields in the header are written in little endian. This means that the least significant byte is put first. As a result the bytes look like they are reversed but there is no error.

Fixing the noise

From the previous test we did, we concluded that the error was not in the header or the data of the file. Thus, we took a closer look to the code to locate the flaw. We tried changing the quality of the recording (modifying the bit rate, number of channels etc) and it fixed the problem.
14

Internship report

Desktop audio player and recorder

We mixed together both desktop programs to have only one program that could open, play, pause, record and save a wav file. We also fixed the stop button. The problem was just that we disposed the variable used to play a file without calling the stop method first.

PDA audio player and recorder

We converted the desktop program to make it work on PDAs. The conversion problems were the same as before and we solved them the exact same way. One of the problems of PDAs is that they have very low RAM and memory in general. So we tried recording a two-minute song to see how much disk space it would take. The desktop version recorded a 43MB file, which is very heavy. The PDA program crashed around 20 seconds after the beginning. We were using the highest quality possible (16 bits and 2 channels) so we lowered it to 8bits and 1 channel. The results are shown in the chart below:
20s Desktop PDA 0.83 MB 1MB 2min 5MB 6.43MB

After changing the quality we made sure the WAV file was still audible. It almost did not change anything on desktop but there was a noticeable difference on PDAs. The quality was much lower but we could still hear the music and the lyrics quite easily. We figured it was not only due to the quality of the recording but also to the efficiency of the microphones on PDAs (which are bad quality too). Here is what the final program looks like on PDAs (the flowchart and code can be found at annex 3, 4 and 5). We had to simplify the interface because Pocket PCs have a smaller screen than desktops.

15

Internship report

the main window for the player/recorder program

the default open dialog box on PDA

the default save dialog box on PDA

16

Internship report

What is Kidsteam?
History
Kidsteam is an intergenerational and interdisciplinary design team. It was created in 1998 at the University of Maryland by the director of the HCIL and has become increasingly popular over the years.

Description
The Kidsteam project consists of gathering a group of six to eight children who come to the lab twice a week and spend two hours working on different projects. The children (aged six to eleven) are managed by at least six adults. The objective is to keep the adult per child ratio very high to work faster and in a more efficient way. The kids main role is to test and design new technologies made for children.

Group formation
The children are chosen in spring and spend two weeks in the summer learning how to be designers. During this time they will be introduced to the different techniques researchers use to test and create prototypes. They might even start working on some of the projects they will have during the school year. Participation in Kidsteam is voluntary so anyone can drop out whenever they want. After two semesters, if the participation was sufficient and if the child wants to, he or she can come back the following year. This process can go on until he or she turns eleven, which is the limit. So how are the children chosen? Some of the children in this year's Kidsteam were there last year or are related to a Kidsteam alumnus. Word of mouth is also very efficient. As for the very first group of kids, researchers created it by going to local schools (near College Park and the D.C. area) and interviewing children. They were looking for young people interested in technology and verbal about what they are thinking. Children are not paid but receive a compensatory gift at the end of the year in exchange for their hard work.

Sponsors
Sponsors as well as PhD students suggest Kidsteam projects. These companies (who are not necessarily specialized in computing) contact the HCIL to work with the children. This semester sponsors included Microsoft, Discovery and the National Park Service.
17

Internship report

Each company has a particular problem and thus is asking Kidsteam to design or test different things. For example, it can be testing a website (see if it is easy enough to navigate), a program (to see if it has all the functionalities wanted) or give ideas for a new software. Some computing companies donate hardware to the lab instead of the normal payment. Usually this hardware is tested or used to develop a program. Each project has a particular age range. Sometimes, all of the kids do not fit in this range so we divided them in two groups: one was the exact age range and the other was a bit older or younger. We asked the second group to work on finding how the project could please their generation. From time to time, we even called some alumni from Kidsteam to form a third group and work on the same project.

Children as design partners

For some people, working with children as design partners can sound strange. Aren't they too young to be considered as real designers? Some people are afraid that youngsters are not able to understand technology, meaning they do not know how it works or what it can do. Therefore, they do not ask their opinion and miss on something. Even if they do not know the proper terminology, children between six and eleven are more than capable of criticizing and designing new features for a program. Depending on the age they will need more or less assistance but in the end they will all manage to come up with new ideas. When you listen to them, you notice that not only do they have really good ideas but also they are ideas adults could not come up with. That is due to the fact that children are used to developing their imagination every day through roleplaying or creative writing for example. Thus they do not think about what is possible or not, they just give their ideas. Adults, on the contrary, need to be rational in their lives. Also, they have more knowledge of what is feasible or not. If you asked an adult to do the same thing he or she would just say the rational ideas and save the "crazy" ones for themselves. That is why children's ideas are valued and respected at Kidsteam. In general, kids are consulted at the end of the design process, just to give feedback. With Kidsteam, the children were often part of a project from the very beginning (defining big ideas for the coming product) until the end (providing feedback). That way, they were able to see the project grow. It was also a benefit for the people in charge of the project. As a matter of fact, they could add changes following the kids' ideas and come back the next session to receive feedback on those modifications. They could then see if they understood the ideas given and if the kids were pleased with the changes made. So what does a typical session look like? Before the actual day, adults prepare the meeting. They first contact the sponsor or PhD student who is in charge of the project to ask what they want to do. Once the guidelines are set they define what could come out of the session and what techniques they want to use. When the kids arrive, they explain to them briefly who is the person in charge of the project and what they will do in the session. There are three different techniques used at Kidsteam to design or test technologies: journals This is used the least. Children have their own journal that they keep all year long. Although all Kidsteam members know how to write they use their journals mainly to draw in it. As a matter of fact, it is usually faster for them and more efficient (they can put the whole
18

Internship report

idea in a single drawing). This method is regularly implemented at the beginning of the design process to specify the project's big ideas. It is also helpful when researchers want kids to change only one feature of the project and redesign it. bags of stuff This process was usually required at the beginning of the project, when no sample of the program or website was available yet. What adults often asked children to do was to design a low-tech prototype of some broad ideas. In order to do that kids were divided into groups of one to three and were given a bag containing things such as cardboard, tape, paper, foam balls, balloons, glue, scissors From all that they had half an hour to build their prototype. Once they were done, they presented it to the whole group. If we were working with sponsors, it allowed the people in charge to bring back the whole idea to their company. Not only did they have the idea but also how the child would design the concept. sticky notes This was the most popular, used mainly to test a prototype of a product and give feedback. Each kid was in front of a computer (or the device they were testing), and was paired with an adult (if there was enough). The child's job was to play with the product and find the major bugs. In addition, he or she had to give feedback on what they were testing. To keep a record of what was said during the session, adults wrote the comments on sticky notes, keeping one idea per post-it. When a child did not like a feature of the product tested we often asked him to give a design idea to answer the question "how would I make it better?". That way, the criticism was productive and helpful to whoever was developing the project. At the end of the session, all the sticky notes were collected and grouped on a white board. Adults then sorted them in three categories: likes, dislikes and design ideas. Most of the time, several kids gave the same idea or had comments on a same element. Therefore we circled those ideas and gave a generic name to it.

the frequency chart after a session of Kidsteam

After each session adults and children gathered for a debriefing, which consisted of saying the big ideas that most of the kids thought of. These debriefings were always very interesting. As a matter of fact, as each adult was working with a fraction of Kidsteam, they
19

Internship report

only heard part of the ideas. So the last meeting was a chance for everyone to catch up and see what the other kids (or groups) came up with. Also, it was fascinating to see the general ideas that emerged from the session. Even though the adults planned the session and set some guidelines before, children have this characteristic to be very unpredictable. As a consequence, no one could expect what the big ideas would be.

My role during Kidsteam sessions

Very rarely, kids were just asked to build low-tech prototypes and come up with general ideas. Usually, they worked on a device or a computer during the session. If they worked on desktops I prepared the session by making sure there was enough computers for everyone. I also checked that it had everything we needed, such as a soundcard and speakers that work, or Adobe's Flashplayer (to watch videos on a website). If the project was developed on a mobile device (such as a tablet or a PDA), I verified that they all functioned properly, that they could last at least one hour on battery and that they had everything we needed installed on the hard drive. If we planed to use the "bags of stuff" technique during the session, I prepared one plastic bag per group with plenty of material in it. During the session, I was part of the adults who were paired with a child. I looked at everything the kid did in case he or she found a bug in the program. I also asked questions to know if he or she liked the product. Even though the children could have written the feedback, we preferred to do it ourselves so that they did not do two things at a time. At the end of the session, I put the room in order and took notes of everything that was on the white board. Finally, I emailed the report to each person that participated in the session.

20

Internship report

Testing the Mobile Storytellers with Kidsteam

First session
During the first session, we tested the program that had been developed during the summer. It was just a prototype of what the final software would do, and as such had limited features. At this point it only included the visualization of a story, and the modification of the text. The communication between PDAs was yet to be developed. Stories can come in three versions: all text, all images and the complete version. The latter consists of the original page of the book (in smaller) followed by the text that it contained (if there is any). It can also include an audio file. In that case, a little symbol ( ) is present in the bottom left hand corner and the length of the sound is indicated in the bottom right hand corner.

all text

all images

text, images and sound

There are two possible ways to play a story. the "automatic mode" After clicking on play the pages are displayed one by one in full screen. If a sound file is attached to a page, it is read. the "manual mode" Instead of having the pages scrolling all by themselves, you have to push a button on the device to get to the next page. Since we only had four PDAs, we grouped the children in pairs, giving one device per group. We asked them to read a story using the program on the device. In order to be able to play a story, we had to load them on the Pocket PCs. Each story takes about 3 to 15MB (depending on the number of pages it contains and if they have audio files included) and the devices have an average of 100MB of memory. To prevent them from crashing we only loaded three stories per device. The kids could choose from any of those. They all picked a different story, the younger kids preferring the automatic mode and the older ones feeling
21

Internship report

confident with the manual playing. By the end of the session, every group had had at least one crash. No one managed to get to the end of a story. At this time I had a program that played sound on desktop but nothing yet on PDAs. For that reason we were still running the sound code made during the summer. The problem was that it had an error that made the whole program crash.

Second session
For this session I had time to finish the sound code for PDAs, which was integrated into the main program. As we did not add any other new feature to it, we decided not to make the kids test the code again. Instead, we thought we could ask them to come up with new particularities the program could have. We wanted to focus more on the communication part, meaning how could PDAs interact with each other. In order to help the children, we built some low tech PDAs using cardboard and gave them post-its (which had the exact same size as the screen) and cutouts from stories we used last time. Each kid had a cardboard PDA (which was full-sized) and each group had a story cutout that came in three versions: text and image, images only and text only. That way, they had a concrete prop to hold on to.

a cardboard PDA

the prototype has the same size as the real device

We then paired the children in groups different from the last session. The question they had to answer was "what would happen if two children had two PDAs with the MobileStoryteller program?". In other words, how can having multiple devices be helpful to read stories? Each group came out with really different ideas. The first one thought about using both PDAs to have more space to display the story. The first device would be used to show the images only and the second one could display the text only. That way, both pages were zoomed in which helped the user see better. The idea was to show a page and a half on each device so that the reader has a little idea of what was coming next. The scrolling was vertical, to take advantage of the shape of the screen on a PDA. Both kids were reading the same story at the same time, which implies that there is a way for the devices to synchronize.

22

Internship report

reading a same story on two devices using vertical scrolling

The only idea that came out from more than one group was that the kids had to be able to send messages to each other, meaning they wanted to communicate while reading. These messages could be to tell a friend what you think about a book or to send an extract of the book (for example a page or some text or an image). It could also be a voice message. The original idea that was given by one of the groups was to be capable of mixing stories. Therefore, you could imagine having the text of one story and the image of another or even a swap between characters and plots. In addition, there could be a surprise button that would also do a swap but the child does not know when in the book it will surface. That would be a new way to create stories.

23

Internship report

PDA interaction via Infrared ports

Why do the devices need to communicate?
The very fascinating part of this project was the collaborative component. Reading a book on a small device such as a PDA can be interesting but reading it with someone else or telling someone about it can be even better. Communication between devices can generate multiple applications. For example, as the children suggested during Kidsteam session, we could imagine sharing the display of a story between two or more devices. As a matter of fact, if the story includes text, the images will be smaller and less visible. Using multiple screens can solve this problem since it gives much more space to show the same thing. In addition, it allows each child to hold a device, so that they get more involved in the reading. Moreover, studies have shown that kids like to share what they think about a book. This was discovered during the development of the ICDL project. Therefore, the website now includes a book review option where Internet users can see what kids thought of the book. We could imagine having a similar feature for the Mobile Storytellers. The children could tag the books, leaving a comment on some of them. Then when someone opens a book to read it there could be a way to read the comments made first. Eventually, the main goal is for at least two children to be able to collaboratively read a story using multiple devices.

Why use IR?

Once we knew we wanted PDAs to communicate with each other we had to find a way to do it. There should be a way for the two devices to recognize each other before they actually send any data. That is the part we concentrated on. The first idea that would come into one's mind would be to connect the two devices with a cable. In order to do that, you have to go through a serial connection using either RS232 or USB, which the PDAs did not have. Also, the application we were building had the ability to run on mobile devices, so it was absurd to get limited by a wire. As a result, we chose to take advantage of one of the wireless technology the devices included. We had to pick between wireless, bluetooth and infrared. Wi-Fi uses the wireless local area network technology. It is mainly employed to connect to the Internet. A client needs an Access Point to use this kind of wireless connection. At the University of Maryland (where we were conducting the research) the whole campus is a hotspot, meaning has access to WiFi. The signal received is very strong and therefore connecting to it consumes a lot of power. Wi-Fi also allows connectivity in peer-to-peer mode, which enables devices to connect directly with each other. This connectivity mode can be used between two PDAs.

24

Internship report

Bluetooth is a short-range wireless technology, which allows connection between two devices (such as mobile phones, computers, cameras) via a secure radio frequency. It replaces cables and does not need a host PC. Power consumption is very low but in return the signal issued has very limited range (1 to 100 meters). One device (called the master) can communicate with up to seven devices (its slaves) and can change its part anytime. This creates small networks of maximum eight appliances. Data transfer reaches 723.1 kbits/s with Bluetooth 1 to 2.1Mbits/s with Bluetooth 2. Unfortunately, only two PDAs were equipped with Bluetooth and of top of that it was only the first version. infrared are well known for being used in remote controls. But these light waves (not to be mistaken with radio waves) have also an important role in communication and networking. They can be used to connect two devices that are really close to each other. The speed at which data is transferred differs from one device to another but it is around 115Kbps/s, which is rather slow. Also, the infrared ports are very small so the two devices to connect have to be very still in order for the connection to work. After looking at all our options, we chose to use infrared anyway for authentication. As a matter of fact, it offers more security than the other too, mostly because it does not use radio waves and so gets stopped by walls. That way, you do not get connected to someone you do not know. The main reason why we chose IR over bluetooth and WiFi was that we needed the kids to know when and who they were connected to. By using infrareds they had to physically get near the person they wanted to exchange information with. This action made them realize that they were trying to get connected to that person. Also, aligning two PDAs can be seen as a little fun and exciting game for a child.

Program developed
The first program that was developed before I got into the project could transfer a text file between two PDAs. By clicking the receive button on the client device then the send button on the other device the file was copied from one hard drive to the other. The communication between the two occurred by infrareds. This program was synchronous, meaning that once the file transfer ended the application closed and no other transfer could be made. The program we wanted to develop was only used for authentication. It had to display a message that there was no device connected at first. Then when a device was detected through the infrared port it had to display the name of the device and the status (the general flowchart of the program can be found at annex 6). Verification had to occur every second to see if the status changed or not. We chose to use a listbox to display the name of the devices and their status. It was the only interface we had. The first code made used only one thread (the one that displays the listbox). The program successfully managed to do IR recognition of a device. What no one knew is that using only one thread makes the program synchronous. So as soon as it detected the device, it closed. To make the program asynchronous, another thread was created. Its role was to detect a new device via IR. That way, the listbox was declared in one thread and modified in
25

Internship report

another, which kept giving an error. What happened is that the Invoke method had to be called every time the listbox in the IR recognition thread was used. It created a delegate on the interface thread that made possible the modification of the listbox. Another problem was that only one Hashtable was used to store the name of the devices. It was not possible to use the same one to read and write in. So a new one was created. Each Hashtable performs a different action. When I got into the project, the program could already detect a new device and display its name and status. The problem was to show at first a sentence saying no remote devices. This phrase had to be displayed only once and stay until a device was recognized. Then it was erased and replaced with the name and status of the device. The program uses a thread that checks every second if there is a new device. We managed to display the sentence for one second but then it disappeared. We finally found a solution by creating a flag that indicates if it is the first time that we enter the main method. If so, the message is displayed and if not all the devices statuses are set to disconnected.

the main screen

This program can be used for multiple connections. For example, if person A want to connect to person B, each of their devices will show the name of the others device with the status connected. Once they disconnect, both status will show disconnected but the name of the devices do not disappear. Then if person B wants to communicate with person C, his or her device will display: Device A: disconnected, Device C: connected. So each new device that the program recognizes will be added to the list of other devices which had connection at one point or another.

26

Internship report

two PDAs connected

two PDAs not aligned to connect

27

Internship report

Possible solutions for data transfer

For sending data between two devices, many options are possible, among which SyncTap, stitching and pick & drop. SyncTap SyncTap uses synchronization via buttons. Each PDA has two buttons: send and receive. To transfer a document between the two devices, one person has to press the receive button and the other the send button. The two persons have to press the buttons synchronously. As it is almost impossible to do this action at the exact same time, there is always a small margin; that is a span of time in which the second person can still hit the button to send or receive the file. It is a good technique to implement with kids because it can be seen as a game. Even if they do not achieve it the first time, they will have another chance and keep trying until they manage to do it. Also, by using SyncTap, they have to be right next to each other. As a matter of fact, in order for the two persons to be synchronized, they have to communicate. That way, one cannot send a file to a person at the other end of the world, which guaranties minimum security. Stitching Stitching is a new innovative way to copy or move data from one mobile device to the other. It uses only pen-based devices such as Tablet PCs or PDAs. The concept is simple: you use your pen or stylus to drag the documents on the first device then jump the gap and continue the gesture on the second device. To be identified as a stitch you have less than 1.5sec to cross the space between both devices. The devices do not necessarily have to be right next to each other for the program to work. There is no need for a special pen for this. The file transfer is made via wireless. The user does not actually need to touch the screen but has to stay within 2cm of it. It is then called "tracking stitching". This is not yet implemented on PDAs. The only software that has been developed so far is called StitchMaster and deals with digital pictures. It allows the user to share its pictures as well as his screen. As a matter of fact, it is possible to share the display of a picture on two screens or use one device to show thumbnails while the other is used to see full size images. Once again, stitching is a simple solution to device identification. Instead of searching for the devices to connect in a list or clicking on buttons to synchronize, you simply start dragging the files from the original device to the receiving device. Stitching could be implemented on Mobile Storytellers to exchange stories. For instance, a child who has just saved his or her story could grab the file and copy it on someone else's PDA using stitching. It could be a good idea because the gesture is quite simple and could be easier to perform for children. Pick and drop This technique uses a more natural gesture than stitching. It consists of picking up a file on a device by tapping on it with a pen and dropping it on another device by tapping a second time. In order for the program to work, the user needs a special pen that has an ID number. It does not need a hard drive as during the transfer the pen virtually holds the object.
28

Internship report

The file is actually stored on a server. Thanks to the pen ID the receiving device can recognize which file is being transferred. As for the stitching, the device identification is very simple: the first tap gives the sending device and the second tap the receiving device. This technique is very easy to use and could be implemented on Mobile Storytellers. It is simple enough to be used by children.

29

Internship report

Possible extensions to the project

This project had a couple of limitations that would be interesting to push further. We have been focusing mainly on two aspects: adding video capture to the program and making it accessible to more people by using cell phones instead of PDAs.

Adding video to Mobile Storytellers

Right now, it is only possible to view and modify a story made of text, images and sound. We could imagine having a modern story that uses video clips on every page. To view them, the program should have a new feature in order to play them. My role was to search if it was possible to create videos in a simple way on PDAs. The lab had a camera for PDAs. It used the SD slot to connect to the device and to store pictures and movies. It came with a software that I installed. I then started taking pictures and movies to see how the PDA responded and what quality I was working with. Here is a summary:
maximum resolution space on the disk picture movie 640x480 320x240 44kB 5,25MB for 20s

We can see that the quality of the pictures and movies is in the average for mobile devices. But we could not really see what was on the pictures. As for the movies, they were not very fluid and kept freezing every couple of seconds. This could be due to the fact that a short movie takes a lot of space on the disk (5,25MB for 20s), which represents one tenth of the total memory of the PDA we tested the program on. After a couple of tests I realized that taking films also needed a lot of RAM, making the software crash very often. This combined with the fact that the images froze during the video capture made us stop the research. Anyway, the Mobile Storytellers does not need any video playing or recording. As a consequence, it would not be a handicap if we did not implement it.

PDA equipped with an SD camera

30

Internship report

Implementing the program on cell phones

The Mobile Storyteller program had been developed to compile on every PDA on the market. The problem is that not everyone owns such a device. However, looking at existing technologies, we can see that cell phones are very popular; so popular in fact that it is rare to find someone who does not own one. The expansion of mobile technology has led to a fast development of cell phones. Some of them now include GPS or PDA functions. They are called smart phones. Today, there is no particular definition of what smart phones are. But people tend to agree to a general statement, which defines smart phones as a handheld device that integrates functionalities of mobile phones and PDAs. The only thing not clarified is which feature to include. Smart phones started in 1992 with "Simon" designed by IBM. It included a calendar, an address book, a world clock, a calculator, a notepad, e-mail and games. There was also a possibility to use a stylus. Most of those phones now have a QWERTY board or a touch screen to facilitate text writing and navigation. All smart phones use operating systems. The most famous are Symbian, Linux, Windows Mobile and Palm OS. The problem is that most phones do not come with Windows CE but with another operating system. Therefore, it would not recognize C# and we would have to find another language to program in. Java is famous for its portability, which means that it can be implemented on every platform. There is a Java language similar to the Compact Framework. It is called Java 2 Micro Edition or J2ME. My job was then to see if I could implement the sound code in this language. As I did not know any Java I first looked at some code on the Internet to see what it looked like. I realized that it was very similar to C# because it employs the same keywords. The only thing that is very different between the two languages is the notion of MIDlet and JAR files. It is essential to J2ME and does not exist in C#. A MIDlet is a Java program for the Java ME virtual machine. It is packed inside a .jar file. This Java ARchive contains compressed Java classes. That way, it allows classes to download faster as a group rather than one at a time. To work, a MIDlet also needs a manifest file, which is a member file inside the jar describing the content of the jar. Without it, the application will not compile. A MIDlet is created after four steps: coding, compiling, preverifying and packaging. This last step creates the JAR and manifest files. As we did not have any smart phones to test application on, I used an emulator. It is a software that simulates the behavior of a category of phones. I downloaded and worked on Sun's Wireless Toolkit (it is the one that most people use). At first I tried running some source codes I found on the Internet to learn how to do it. It did not work, and kept giving me a "ClassNotFound" exception. I then tried doing every step necessary to the MIDlet creation in the command window. To fix the problem I had to reinstall the program. The application successfully deployed. It was just a simple program that displayed the date and time but at least it helped me get started with the Wireless Toolkit. The next step was to make the sound code. Just like previously, it needed to play, pause and record sound files. It was already the end of my internship so I did not really have time to learn J2ME and code something by myself. Instead I found out that the software I was using included a couple of demo programs. One of them was focusing on sounds. It had much more functionalities than I wanted so I made a copy of the project to make modifications. I kept only the features I wanted. The recording part of the program did not save it to a file but made a playback, which I did not have time to change. Also, the files played had to be stored
31

Internship report

on the Internet. One had to type the address of a wav file on the Internet to hear it. I did not know how to play a file stored in the phone's memory. Below are screen captures of the program.

the first screen gives the choice of playing or recording a file

when a file is being played, hitting the menu key will make this box appear with a list of different actions to perform

32

Internship report

Conclusion
During this internship, I managed to fix the sound player/recorder by using another algorithm. It is now working and stable. In addition, I assisted my colleagues with the Kidsteam project by being there at every session and preparing what we would need. I also helped finish the program that achieves the infrared authentication, which will be used in the future. Finally, I have done some research to see if the program could be implemented on cell phones. This internship was a good opportunity for me to discover programming for mobile technologies. I learnt a new language (C#) and managed to program in the Compact Framework.

33

Annexes Table of contents

Annex 1: PDA characteristics .....................................................1 Annex 2: The WAV file format ....................................................2 Annex 3: Flowchart of the playing part of the program...............4 Annex 4: Flowchart of the recording part of the program ...........5 Annex 5: Program that plays and records a WAV file.................6 Annex 6: Flowchart of the IR authentication program ..............32 Annex 7: Bibliography ..............................................................33

Annexes

Annex 1: PDA characteristics

Brand RAM ROM

DELL 64 MB 256 MB

HP 64 MB 32 MB 57 MB

Toshiba 127 MB 32 MB 127 MB

Toshiba 62 MB 32 MB 62 MB

Total main memory 245 MB Processor Bluetooth?

624 MHz 400 MHz 400 MHz 400 MHz Yes (1.4) Yes (1.4) No No

Annexes

Annex 2: The WAV file format

The WAV file format (also known as the Waveform audio format) was developed by IBM and Microsoft. It is used for audio files and is a variant of the RIFF bitstream format. As such, it stores data in chunks (as shown below).

We can see that each chunk has its own header (its ID and size). The far left column gives the encoding type. If it is "big endian" the bytes are written normally. On the other hand, if it is "little endian" the least significant byte will become the most significant. For instance, "46 46 49 52" (which is the code for "FFIR") will need to be read backwards as "52 49 46 46" (or "RIFF"). Let's start the explanation of each fieldname by the RIFF chunk: ChunkSize equals to the size of the entire file minus 8 bytes (which is the size of ChunkID plus the size of ChunkSize)

ChunkID is always "RIFF"

Annexes

Format is usually "WAVE"

If the format is set to "WAVE", two sub-chunks will follow: the format (or fmt) subchunk and the "data" sub-chunk.

Audioformat indicates if the file is compressed or not. We did not use any audio compression so we set this field to 1. NumChannels shows the number of channels (1 equals to mono, 2 to stereo etc.). We chose one channel ByteRate is the average bytes streamed per second. It is calculated through the formula: ByteRate= SampleRate * NumChannels * BitsPerSample /8

Subchunk1ID is then "fmt" Subchunk1Size equals to the size of the rest of Subchunk1

SampleRate is the number of sample slices per second. We used 44100.

BlockAlign is the number of bytes for one sample including all channels. BlockAlign = NumChannels * BitsPerSample /8
BitsPerSample is a multiple of eight. We chose to work with 8 bits. Subchunk2ID equals to "data" Subchunk2size is the size of the data data is the audio data that the file contains

Annexes

Annex 3: Flowchart of the playing part of the program

"Open" Click

Recording?

N
Open audio file

"Play" Click

Recording?

N N
Play file

Paused?

Resume playing

"Pause" Click

"Stop" Click

Recording?

Recording?

Playing?

Stop

Stopped?

N
Pause

Annexes

Annex 4: Flowchart of the recording part of the program

"Record" Click

N Recording?

N Playing?

Save filename + path

Start recording

"Stop" Click Y Recording?

Stop recording

Create file

Write header

Write data

Close file

Annexes

Annex 5: Program that plays and records a WAV file

MainForm class
using using using using using using using using using System; System.Collections.Generic; System.ComponentModel; System.Data; System.Drawing; System.Text; System.Windows.Forms; System.Collections; System.IO;

namespace Sound_Player_Recorder { public class MainForm : System.Windows.Forms.Form { private System.ComponentModel.Container components = null; private System.Windows.Forms.Button OpenButton; private System.Windows.Forms.Button PlayButton; private System.Windows.Forms.Button PauseButton; private System.Windows.Forms.Button StopButton; private System.Windows.Forms.Button RecordButton; private System.Windows.Forms.OpenFileDialog OpenDlg; private System.Windows.Forms.SaveFileDialog SaveDlg;

public MainForm() { InitializeComponent(); } // Clean up any resources being used. protected override void Dispose(bool disposing) { if (disposing) { if (components != null) { components.Dispose(); } } base.Dispose(disposing); }

Annexes #region Windows Form Designer generated code private void InitializeComponent() { this.PlayButton = new System.Windows.Forms.Button(); this.StopButton = new System.Windows.Forms.Button(); this.OpenButton = new System.Windows.Forms.Button(); this.OpenDlg =new System.Windows.Forms.OpenFileDialog(); this.RecordButton = new System.Windows.Forms.Button(); this.PauseButton = new System.Windows.Forms.Button(); this.SaveDlg =new System.Windows.Forms.SaveFileDialog(); this.SuspendLayout(); // // PlayButton // this.PlayButton.Location = new System.Drawing.Point(130, 99); this.PlayButton.Name = "PlayButton"; this.PlayButton.Size = new System.Drawing.Size(125, 54); this.PlayButton.TabIndex = 0; this.PlayButton.Text = "Play"; this.PlayButton.Click += new System.EventHandler(this.PlayButton_Click); // // StopButton // this.StopButton.Location = new System.Drawing.Point(130, 244); this.StopButton.Name = "StopButton"; this.StopButton.Size = new System.Drawing.Size(125, 54); this.StopButton.TabIndex = 1; this.StopButton.Text = "Stop"; this.StopButton.Click += new System.EventHandler(this.StopButton_Click); // // OpenButton // this.OpenButton.Location = new System.Drawing.Point(130, 24); this.OpenButton.Name = "OpenButton"; this.OpenButton.Size = new System.Drawing.Size(125, 54); this.OpenButton.TabIndex = 2; this.OpenButton.Text = "Open"; this.OpenButton.Click += new System.EventHandler(this.OpenButton_Click); // // OpenDlg // this.OpenDlg.DefaultExt = "wav"; this.OpenDlg.Filter = "WAV files|*.wav"; // // RecordButton // this.RecordButton.Location = new System.Drawing.Point(130, 316); this.RecordButton.Name = "RecordButton";

Annexes this.RecordButton.Size = new System.Drawing.Size(125, 54); this.RecordButton.TabIndex = 3; this.RecordButton.Text = "Record"; this.RecordButton.UseVisualStyleBackColor = true; this.RecordButton.Click += new System.EventHandler(this.RecordButton_Click); // // PauseButton // this.PauseButton.Location = new System.Drawing.Point(130, 170); this.PauseButton.Name = "PauseButton"; this.PauseButton.Size = new System.Drawing.Size(125, 54); this.PauseButton.TabIndex = 4; this.PauseButton.Text = "Pause"; this.PauseButton.UseVisualStyleBackColor = true; this.PauseButton.Click += new System.EventHandler(this.PauseButton_Click); // // SaveDlg // this.SaveDlg.DefaultExt = "wav"; this.SaveDlg.Filter = "WAV files|*.wav"; // // MainForm // this.AutoScaleBaseSize = new System.Drawing.Size(5, 13); this.ClientSize = new System.Drawing.Size(365, 416); this.Controls.Add(this.PauseButton); this.Controls.Add(this.RecordButton); this.Controls.Add(this.OpenButton); this.Controls.Add(this.StopButton); this.Controls.Add(this.PlayButton); this.FormBorderStyle = System.Windows.Forms.FormBorderStyle.FixedDialog; this.MaximizeBox = false; this.MinimizeBox = false; this.Name = "MainForm"; this.Text = "Low-level audio player"; this.ResumeLayout(false); } #endregion /// <summary> /// The main entry point for the application. /// </summary> [STAThread] static void Main() { Application.Run(new MainForm()); }

Annexes private WaveLib.WaveOutPlayer m_Player; private WaveLib.WaveInRecorder m_Recorder; private WaveLib.WaveFormat m_Format = new WaveLib.WaveFormat(44100, 8, 1); private MemoryStream RecorderOutputStream = new MemoryStream(); private Stream m_AudioStream; private byte[] m_RecBuffer;//used to store the data recorded private string filename = ""; //used to store the name of //the file to record //booleans private bool is_playing = false; private bool is_paused = false; private bool is_recording = false;

/*----------------------------------------------------------------*/ /*----------------------Opening file methods----------------------*/ /*----------------------------------------------------------------*/ private void OpenFile() { if (OpenDlg.ShowDialog() == DialogResult.OK) { CloseFile(); try { WaveLib.WaveStream S = new WaveLib.WaveStream(OpenDlg.FileName); if (S.Length <= 0) throw new Exception("Invalid WAV file"); m_Format = S.Format; if (m_Format.wFormatTag != (short)WaveLib.WaveFormats.Pcm && m_Format.wFormatTag != (short)WaveLib.WaveFormats.Float) throw new Exception("Only PCM files are supported"); m_AudioStream = S; } catch (Exception e) { CloseFile(); MessageBox.Show(e.Message); } } }

Annexes private void CloseFile() { Stop(); if (m_AudioStream != null) try { m_AudioStream.Close(); } finally { m_AudioStream = null; } } /*----------------------------------------------------------------*/ /*------------------------Playing methods-------------------------*/ /*----------------------------------------------------------------*/ private void Play() { Stop(); if (m_AudioStream != null) { m_AudioStream.Position = 0; m_Player = new WaveLib.WaveOutPlayer(-1, m_Format, 16384, 3,new WaveLib.BufferFillEventHandler(Filler)); } } private void Filler(IntPtr data, int size) { byte[] b = new byte[size]; if (m_AudioStream != null) { int pos = 0; while (pos < size) { int toget = size - pos; int got = m_AudioStream.Read(b, pos, toget); if (got < toget) if (m_Player != null) { m_Player.file_done(); is_playing = false; break; } pos += got; } } else { for (int i = 0; i < b.Length; i++) b[i] = 0; }
System.Runtime.InteropServices.Marshal.Copy(b, 0, data, size);

} 10

Annexes /*----------------------------------------------------------------*/ /*------------------------Stopping method-------------------------*/ /*----------------------------------------------------------------*/ private void Stop() { is_paused = false; if (m_Player != null) try { m_Player.Reset(); m_Player.Dispose(); } finally { m_Player = null; } } /*----------------------------------------------------------------*/ /*------------------------Recording methods-----------------------*/ /*----------------------------------------------------------------*/ private void RecordStart() { RecordStop(); try { m_Recorder = new WaveLib.WaveInRecorder(-1, m_Format, 16384, 3, new WaveLib.BufferDoneEventHandler(DataArrived)); is_recording = true; } catch { RecordStop(); throw; } } private void DataArrived(IntPtr data, int size) { if (m_RecBuffer == null || m_RecBuffer.Length < size) m_RecBuffer = new byte[size]; //copies all the data into the m_RecBuffer System.Runtime.InteropServices.Marshal.Copy( data, m_RecBuffer, 0, size); int count = 0; while (count < m_RecBuffer.Length) { //writes m_RecBuffer into recorderOutputStream byte //per byte RecorderOutputStream.WriteByte(m_RecBuffer[count++]); } }

11

Annexes private void RecordStop() { if (m_Recorder != null) try { //where the file will be stored FileStream fs = new FileStream(filename, System.IO.FileMode.Create); //chunksize is length of wave data and the //header. long chunksize = RecorderOutputStream.Length+36; BinaryWriter bw = new BinaryWriter(fs); // Write out the header information WriteChars(bw, "RIFF"); bw.Write((int)chunksize); WriteChars(bw, "WAVEfmt "); bw.Write((int)16); bw.Write(m_Format.wFormatTag); bw.Write(m_Format.nChannels); bw.Write(m_Format.nSamplesPerSec); bw.Write(m_Format.nAvgBytesPerSec); bw.Write(m_Format.nBlockAlign); bw.Write(m_Format.wBitsPerSample); WriteChars(bw, "data"); bw.Write(RecorderOutputStream.Length); bw.Flush(); //writes the recorded data into the binary file bw.Write(RecorderOutputStream.ToArray()); fs.Close(); m_Recorder.ResetRecord(); m_Recorder.Dispose(); is_recording = false; } finally { m_Recorder = null; } } private void WriteChars(BinaryWriter wrtr, string text) { for (int i = 0; i < text.Length; i++) { char c = (char)text[i]; wrtr.Write(c); } }

12

Annexes /*----------------------------------------------------------------*/ /*-----------------------Button Click Events----------------------*/ /*----------------------------------------------------------------*/ private void OpenButton_Click(object sender, EventArgs e) { if (!is_recording) OpenFile(); } private void PlayButton_Click(object sender, EventArgs e) { if (!is_recording) { if (is_paused == true) { m_Player.Resume(); is_paused = false; } else { is_playing = true; Play(); } } }

private void PauseButton_Click(object sender, EventArgs e) { if (!is_recording && is_playing) { if (m_Player != null) { m_Player.Pause(); is_paused = true; } } }

private void StopButton_Click(object sender, EventArgs e) { if (is_recording) { RecordStop(); } else { Stop(); } }

13

Annexes private void RecordButton_Click(object sender, EventArgs e) { if (!is_recording && !is_playing) { if (SaveDlg.ShowDialog() == DialogResult.OK) { //Saves the name of the file to save filename = SaveDlg.FileName; RecordStart(); } } } } }

WaveOut class
using System; using System.Threading; using System.Runtime.InteropServices; namespace WaveLib { internal class WaveOutHelper { public static void Try(int err) { if (err != WaveNative.MMSYSERR_NOERROR) throw new Exception(err.ToString()); } } public delegate void BufferFillEventHandler(IntPtr data, int size); internal class WaveOutBuffer : IDisposable { public WaveOutBuffer NextBuffer; private AutoResetEvent m_PlayEvent = new AutoResetEvent(false); private IntPtr m_WaveOut; private private private private WaveNative.WaveHdr m_Header; byte[] m_HeaderData; GCHandle m_HeaderHandle; GCHandle m_HeaderDataHandle;

private bool m_Playing;

14

Annexes internal static void WaveOutProc(IntPtr hdrvr, int uMsg, int dwUser, ref WaveNative.WaveHdr wavhdr, int dwParam2) { if (uMsg == WaveNative.MM_WOM_DONE) { try { GCHandle h = (GCHandle)wavhdr.dwUser; WaveOutBuffer buf = (WaveOutBuffer)h.Target; buf.OnCompleted(); } catch { } } } public WaveOutBuffer(IntPtr waveOutHandle, int size) { m_WaveOut = waveOutHandle; m_HeaderHandle = GCHandle.Alloc(m_Header, GCHandleType.Pinned); m_Header.dwUser = (IntPtr)GCHandle.Alloc(this); m_HeaderData = new byte[size]; m_HeaderDataHandle = GCHandle.Alloc(m_HeaderData, GCHandleType.Pinned); m_Header.lpData = m_HeaderDataHandle.AddrOfPinnedObject(); m_Header.dwBufferLength = size; WaveOutHelper.Try(WaveNative.waveOutPrepareHeader (m_WaveOut, ref m_Header, Marshal.SizeOf(m_Header))); } ~WaveOutBuffer() { Dispose(); } public void Dispose() { if (m_Header.lpData != IntPtr.Zero) { WaveNative.waveOutUnprepareHeader(m_WaveOut, ref m_Header, Marshal.SizeOf(m_Header)); m_HeaderHandle.Free(); m_Header.lpData = IntPtr.Zero; } m_PlayEvent.Close(); if (m_HeaderDataHandle.IsAllocated) m_HeaderDataHandle.Free(); GC.SuppressFinalize(this); }

15

Annexes public int Size { get { return m_Header.dwBufferLength; } } public IntPtr Data { get { return m_Header.lpData; } } public bool Play() { lock (this) { m_PlayEvent.Reset(); m_Playing = WaveNative.waveOutWrite(m_WaveOut, ref m_Header, Marshal.SizeOf(m_Header)) == WaveNative.MMSYSERR_NOERROR; return m_Playing; } } public void WaitFor() { if (m_Playing) { m_Playing = m_PlayEvent.WaitOne(); } else { Thread.Sleep(0); } } public void OnCompleted() { m_PlayEvent.Set(); m_Playing = false; } } public class WaveOutPlayer : IDisposable { private IntPtr m_WaveOut; private WaveOutBuffer m_Buffers; // linked list private WaveOutBuffer m_CurrentBuffer; private Thread m_Thread; private BufferFillEventHandler m_FillProc; private bool m_Finished; private byte m_zero; private WaveNative.WaveDelegate m_BufferProc = new WaveNative.WaveDelegate(WaveOutBuffer.WaveOutProc); public static int DeviceCount { get { return WaveNative.waveOutGetNumDevs(); } }

16

Annexes

public WaveOutPlayer( int device, WaveFormat format, int bufferSize, int bufferCount, BufferFillEventHandler fillProc) { m_zero = format.wBitsPerSample == 8 ? (byte)128 : (byte)0; m_FillProc = fillProc; WaveOutHelper.Try(WaveNative.waveOutOpen(out m_WaveOut, device, format, m_BufferProc, 0, WaveNative.CALLBACK_FUNCTION)); AllocateBuffers(bufferSize, bufferCount); m_Thread = new Thread(new ThreadStart(ThreadProc)); m_Thread.Start(); } ~WaveOutPlayer() { Dispose(); } public void Pause() { WaveNative.waveOutPause(m_WaveOut); } public void Resume() { WaveNative.waveOutRestart(m_WaveOut); } public void Reset() { WaveNative.waveOutReset(m_WaveOut); } public void file_done() { m_Finished = true; }

17

Annexes public void Dispose() { if (m_Thread != null) try { m_Finished = true; if (m_WaveOut != IntPtr.Zero) WaveNative.waveOutReset(m_WaveOut); m_Thread.Join(); m_FillProc = null; FreeBuffers(); if (m_WaveOut != IntPtr.Zero) WaveNative.waveOutClose(m_WaveOut); } finally { m_Thread = null; m_WaveOut = IntPtr.Zero; } GC.SuppressFinalize(this); } private void ThreadProc() { while (!m_Finished) { Advance(); if (m_FillProc != null && !m_Finished) m_FillProc(m_CurrentBuffer.Data, m_CurrentBuffer.Size); else { // zero out buffer byte v = m_zero; byte[] b = new byte[m_CurrentBuffer.Size]; for (int i = 0; i < b.Length; i++) b[i] = v; Marshal.Copy(b, 0, m_CurrentBuffer.Data, b.Length); } m_CurrentBuffer.Play(); } WaitForAllBuffers(); }

18

Annexes private void AllocateBuffers(int bufferSize, int bufferCount) { FreeBuffers(); if (bufferCount > 0) { m_Buffers = new WaveOutBuffer(m_WaveOut,bufferSize); WaveOutBuffer Prev = m_Buffers; try { for (int i = 1; i < bufferCount; i++) { WaveOutBuffer Buf = new WaveOutBuffer(m_WaveOut, bufferSize); Prev.NextBuffer = Buf; Prev = Buf; } } finally { Prev.NextBuffer = m_Buffers; } } } private void FreeBuffers() { m_CurrentBuffer = null; if (m_Buffers != null) { WaveOutBuffer First = m_Buffers; m_Buffers = null; WaveOutBuffer Current = First; do { WaveOutBuffer Next = Current.NextBuffer; Current.Dispose(); Current = Next; } while (Current != First); } }

private void Advance() { m_CurrentBuffer = m_CurrentBuffer == null ? m_Buffers : m_CurrentBuffer.NextBuffer; m_CurrentBuffer.WaitFor(); }

19

Annexes private void WaitForAllBuffers() { WaveOutBuffer Buf = m_Buffers; while (Buf.NextBuffer != m_Buffers) { Buf.WaitFor(); Buf = Buf.NextBuffer; } } } }

WaveIn class
using System; using System.Threading; using System.Runtime.InteropServices; namespace WaveLib { internal class WaveInHelper { public static void Try(int err) { if (err != WaveNative.MMSYSERR_NOERROR) throw new Exception(err.ToString()); } } public delegate void BufferDoneEventHandler(IntPtr data, int size); internal class WaveInBuffer : IDisposable { public WaveInBuffer NextBuffer; private AutoResetEvent m_RecordEvent = new AutoResetEvent(false); private IntPtr m_WaveIn; private private private private WaveNative.WaveHdr m_Header; byte[] m_HeaderData; GCHandle m_HeaderHandle; GCHandle m_HeaderDataHandle;

private bool m_Recording;

20

Annexes internal static void WaveInProc( IntPtr hdrvr, int uMsg, int dwUser, ref WaveNative.WaveHdr wavhdr, int dwParam2) { if (uMsg == WaveNative.MM_WIM_DATA) { try { GCHandle h = (GCHandle)wavhdr.dwUser; WaveInBuffer buf = (WaveInBuffer)h.Target; buf.OnCompleted(); } catch { } } } public WaveInBuffer(IntPtr waveInHandle, int size) { m_WaveIn = waveInHandle; m_HeaderHandle = GCHandle.Alloc(m_Header, GCHandleType.Pinned); m_Header.dwUser = (IntPtr)GCHandle.Alloc(this); m_HeaderData = new byte[size]; m_HeaderDataHandle = GCHandle.Alloc(m_HeaderData, GCHandleType.Pinned); m_Header.lpData = m_HeaderDataHandle.AddrOfPinnedObject(); m_Header.dwBufferLength = size; WaveInHelper.Try(WaveNative.waveInPrepareHeader( m_WaveIn, ref m_Header, Marshal.SizeOf(m_Header))); } ~WaveInBuffer() { Dispose(); } public void Dispose() { if (m_Header.lpData != IntPtr.Zero) { WaveNative.waveInUnprepareHeader(m_WaveIn, ref m_Header, Marshal.SizeOf(m_Header)); m_HeaderHandle.Free(); m_Header.lpData = IntPtr.Zero; } m_RecordEvent.Close(); if (m_HeaderDataHandle.IsAllocated) m_HeaderDataHandle.Free(); GC.SuppressFinalize(this); }

21

Annexes public int Size { get { return m_Header.dwBufferLength; } } public IntPtr Data { get { return m_Header.lpData; } } public bool Record() { lock (this) { m_RecordEvent.Reset(); m_Recording = WaveNative.waveInAddBuffer(m_WaveIn, ref m_Header, Marshal.SizeOf(m_Header)) == WaveNative.MMSYSERR_NOERROR; return m_Recording; } } public void WaitFor() { if (m_Recording) m_Recording = m_RecordEvent.WaitOne(); else Thread.Sleep(0); } private void OnCompleted() { m_RecordEvent.Set(); m_Recording = false; } } public class WaveInRecorder : IDisposable { private IntPtr m_WaveIn; private WaveInBuffer m_Buffers; // linked list private WaveInBuffer m_CurrentBuffer; private Thread m_Thread; private BufferDoneEventHandler m_DoneProc; private bool m_Finished; private WaveNative.WaveDelegate m_BufferProc = new WaveNative.WaveDelegate(WaveInBuffer.WaveInProc); public static int DeviceCount { get { return WaveNative.waveInGetNumDevs(); } }

22

Annexes public void ResetRecord() { WaveNative.waveInReset(m_WaveIn); } public WaveInRecorder( int device, WaveFormat format, int bufferSize, int bufferCount, BufferDoneEventHandler doneProc) { m_DoneProc = doneProc; WaveInHelper.Try(WaveNative.waveInOpen(out m_WaveIn, device, format, m_BufferProc, 0, WaveNative.CALLBACK_FUNCTION)); AllocateBuffers(bufferSize, bufferCount); for (int i = 0; i < bufferCount; i++) { SelectNextBuffer(); m_CurrentBuffer.Record(); } WaveInHelper.Try(WaveNative.waveInStart(m_WaveIn)); m_Thread = new Thread(new ThreadStart(ThreadProc)); m_Thread.Start(); } ~WaveInRecorder() { Dispose(); } public void Dispose() { if (m_Thread != null) try { m_Finished = true; if (m_WaveIn != IntPtr.Zero) WaveNative.waveInReset(m_WaveIn); WaitForAllBuffers(); m_Thread.Join(); m_DoneProc = null; FreeBuffers(); if (m_WaveIn != IntPtr.Zero) WaveNative.waveInClose(m_WaveIn); } finally { m_Thread = null; m_WaveIn = IntPtr.Zero; } GC.SuppressFinalize(this); }

private void ThreadProc()

23

Annexes { while (!m_Finished) { Advance(); if (m_DoneProc != null && !m_Finished) m_DoneProc(m_CurrentBuffer.Data, m_CurrentBuffer.Size); m_CurrentBuffer.Record(); } } private void AllocateBuffers( int bufferSize, int bufferCount) { FreeBuffers(); if (bufferCount > 0) { m_Buffers = new WaveInBuffer(m_WaveIn, bufferSize); WaveInBuffer Prev = m_Buffers; try { for (int i = 1; i < bufferCount; i++) { WaveInBuffer Buf = new WaveInBuffer(m_WaveIn, bufferSize); Prev.NextBuffer = Buf; Prev = Buf; } } finally { Prev.NextBuffer = m_Buffers; } } } private void FreeBuffers() { m_CurrentBuffer = null; if (m_Buffers != null) { WaveInBuffer First = m_Buffers; m_Buffers = null; WaveInBuffer Current = First; do { WaveInBuffer Next = Current.NextBuffer; Current.Dispose(); Current = Next; } while (Current != First); } }

private void Advance()

24

Annexes { SelectNextBuffer(); m_CurrentBuffer.WaitFor(); } private void SelectNextBuffer() { m_CurrentBuffer = m_CurrentBuffer == null ? m_Buffers : m_CurrentBuffer.NextBuffer; } private void WaitForAllBuffers() { WaveInBuffer Buf = m_Buffers; while (Buf.NextBuffer != m_Buffers) { Buf.WaitFor(); Buf = Buf.NextBuffer; } } } }

WaveStream class
using System; using System.IO; namespace WaveLib { public class WaveStream : Stream, IDisposable { private Stream m_Stream; private long m_DataPos; private long m_Length; private WaveFormat m_Format; public WaveFormat Format { get { return m_Format; } } private string ReadChunk(BinaryReader reader) { byte[] ch = new byte[4]; reader.Read(ch, 0, ch.Length); return System.Text.Encoding.ASCII.GetString( ch, 0, ch.Length); }

private void ReadHeader()

25

Annexes { BinaryReader Reader = new BinaryReader(m_Stream); if (ReadChunk(Reader) != "RIFF") throw new Exception("Invalid file format"); Reader.ReadInt32(); // File length minus first 8 bytes // of RIFF description if (ReadChunk(Reader) != "WAVE") throw new Exception("Invalid file format"); if (ReadChunk(Reader) != "fmt ") throw new Exception("Invalid file format"); int len = Reader.ReadInt32(); if (len < 16) // bad format chunk length throw new Exception("Invalid file format"); m_Format = new WaveFormat(22050, 16, 2); // initialize // to any // format

m_Format.wFormatTag = Reader.ReadInt16(); m_Format.nChannels = Reader.ReadInt16(); m_Format.nSamplesPerSec = Reader.ReadInt32(); m_Format.nAvgBytesPerSec = Reader.ReadInt32(); m_Format.nBlockAlign = Reader.ReadInt16(); m_Format.wBitsPerSample = Reader.ReadInt16(); // advance in the stream to skip the wave format block len -= 16; // minimum format size while (len > 0) { Reader.ReadByte(); len--; } // assume the data chunk is aligned while (m_Stream.Position < m_Stream.Length && ReadChunk(Reader) != "data"); if (m_Stream.Position >= m_Stream.Length) throw new Exception("Invalid file format"); m_Length = Reader.ReadInt32(); m_DataPos = m_Stream.Position; Position = 0; } public WaveStream(string fileName) : this(new FileStream(fileName, FileMode.Open)) { }

public WaveStream(Stream S)

26

Annexes { m_Stream = S; ReadHeader(); } ~WaveStream() { Dispose(); } public void Dispose() { if (m_Stream != null) m_Stream.Close(); GC.SuppressFinalize(this); } public override bool CanRead { get { return true; } } public override bool CanSeek { get { return true; } } public override bool CanWrite { get { return false; } } public override long Length { get { return m_Length; } } public override long Position { get { return m_Stream.Position - m_DataPos; } set { Seek(value, SeekOrigin.Begin); } } public override void Close() { Dispose(); } public override void Flush() { } public override void SetLength(long len) { throw new InvalidOperationException(); }

public override long Seek(long pos, SeekOrigin o)

27

Annexes { switch (o) { case SeekOrigin.Begin: m_Stream.Position = pos + m_DataPos; break; case SeekOrigin.Current: m_Stream.Seek(pos, SeekOrigin.Current); break; case SeekOrigin.End: m_Stream.Position = m_DataPos + m_Length - pos; break; } return this.Position; } public override int Read(byte[] buf, int ofs, int count) { int toread = (int)Math.Min(count, m_Length - Position); return m_Stream.Read(buf, ofs, toread); } public override void Write(byte[] buf, int ofs, int count) { throw new InvalidOperationException(); } } }

WaveNative class
using System; using System.Runtime.InteropServices; namespace WaveLib { public enum WaveFormats { Pcm = 1, Float = 3 } [StructLayout(LayoutKind.Sequential)] public class WaveFormat { public short wFormatTag; public short nChannels; public int nSamplesPerSec; public int nAvgBytesPerSec; public short nBlockAlign; public short wBitsPerSample; public short cbSize;

public WaveFormat(int rate, int bits, int channels)

28

Annexes { wFormatTag = (short)WaveFormats.Pcm; nChannels = (short)channels; nSamplesPerSec = rate; wBitsPerSample = (short)bits; cbSize = 0; nBlockAlign = (short)(channels * (bits / 8)); nAvgBytesPerSec = nSamplesPerSec * nBlockAlign; } } internal class WaveNative { // consts public const int MMSYSERR_NOERROR = 0; // no error public const int MM_WOM_OPEN = 0x3BB; public const int MM_WOM_CLOSE = 0x3BC; public const int MM_WOM_DONE = 0x3BD; public const int MM_WIM_OPEN = 0x3BE; public const int MM_WIM_CLOSE = 0x3BF; public const int MM_WIM_DATA = 0x3C0; public const int CALLBACK_FUNCTION = 0x00030000 public const int TIME_MS = 0x0001; // time in milliseconds public const int TIME_SAMPLES = 0x0002; // number of wave // samples public const int TIME_BYTES = 0x0004; // current byte // offset // callbacks public delegate void WaveDelegate( IntPtr hdrvr, int uMsg, int dwUser, ref WaveHdr wavhdr, int dwParam2); // structs [StructLayout(LayoutKind.Sequential)] public struct WaveHdr { public IntPtr lpData; // pointer to locked data buffer public int dwBufferLength; // length of data buffer public int dwBytesRecorded; // used for input only public IntPtr dwUser; // for client's use public int dwFlags; // assorted flags (see defines) public int dwLoops; // loop control counter public IntPtr lpNext; // PWaveHdr, reserved for driver public int reserved; // reserved for driver } private const string mmdll = "winmm.dll";

// waveOut calls

29

Annexes [DllImport(mmdll)] public static extern int waveOutGetNumDevs(); [DllImport(mmdll)] public static extern int waveOutPrepareHeader( IntPtr hWaveOut, ref WaveHdr lpWaveOutHdr, int uSize); [DllImport(mmdll)] public static extern int waveOutUnprepareHeader( IntPtr hWaveOut, ref WaveHdr lpWaveOutHdr, int uSize); [DllImport(mmdll)] public static extern int waveOutWrite( IntPtr hWaveOut, ref WaveHdr lpWaveOutHdr, int uSize); [DllImport(mmdll)] public static extern int waveOutOpen(out IntPtr hWaveOut, int uDeviceID, WaveFormat lpFormat, WaveDelegate dwCallback, int dwInstance, int dwFlags); [DllImport(mmdll)] public static extern int waveOutReset(IntPtr hWaveOut); [DllImport(mmdll)] public static extern int waveOutClose(IntPtr hWaveOut); [DllImport(mmdll)] public static extern int waveOutPause(IntPtr hWaveOut); [DllImport(mmdll)] public static extern int waveOutRestart(IntPtr hWaveOut);

// WaveIn calls [DllImport(mmdll)] public static extern int waveInGetNumDevs(); [DllImport(mmdll)] public static extern int waveInAddBuffer( IntPtr hwi, ref WaveHdr pwh, int cbwh); [DllImport(mmdll)] public static extern int waveInClose(IntPtr hwi); [DllImport(mmdll)] public static extern int waveInOpen(out IntPtr phwi, int uDeviceID, WaveFormat lpFormat, WaveDelegate dwCallback, int dwInstance, int dwFlags); [DllImport(mmdll)] public static extern int waveInPrepareHeader(IntPtr hWaveIn, ref WaveHdr lpWaveInHdr, int uSize);

[DllImport(mmdll)]

30

Annexes public static extern int waveInUnprepareHeader( IntPtr hWaveIn, ref WaveHdr lpWaveInHdr, int uSize); [DllImport(mmdll)] public static extern int waveInReset(IntPtr hwi); [DllImport(mmdll)] public static extern int waveInStart(IntPtr hwi); } }

31

Annexes

Annex 6: Flowchart of the IR authentication program

Discover devices

Device discovered?

First time?

Clear listbox

Set all statuses to disconnected

Show: "no remote devices" Device in the listbox? N

Add device name to the listbox

Set device status to "connected"

32

Annexes

Annex 7: Bibliography

Introduction

http://nces.ed.gov/pubs2005/2005111rev.pdf

University Description

http://www.newsdesk.umd.edu/facts/2006rank.cfm http://prism.cs.umd.edu/papers/M02:facultytimeline/history_new.pdf http://www.cs.umd.edu/groups/areas.shtml http://www.cs.umd.edu/newsletters/InsideCS_2006s.pdf http://www.umd.edu/university/factcard2004.pdf http://www.newsdesk.umd.edu/facts/quickfacts.cfm http://www.umiacs.umd.edu/aboutus.htm http://www.umiacs.umd.edu/docs/umiacsbro.pdf http://www.umiacs.umd.edu/research.htm http://www.cs.umd.edu/hcil/about/ http://www.cs.umd.edu/hcil/research/ http://www.provost.umd.edu/Strategic_Planning/Mission2000.html

Project descriptions

http://www.icdlbooks.org/about/mission.shtml http://www.icdlbooks.org/about/goals.shtml http://www.cs.umd.edu/hcil/jazz/

Coding an audio player and recorder in C#

http://www.techdis.ac.uk/index.php?p=9_5_6_14 http://www.codeproject.com/audio/moemeka7.asp http://www.webopedia.com/TERM/D/DLL.html http://msdn.microsoft.com/library/default.asp?url=/library/enus/dnnetcomp/html/WaveInOut.asp http://msdn2.microsoft.com/en-us/library/system.buffer.aspx http://msdn2.microsoft.com/en-us/library/system.io.memorystream.aspx http://msdn.microsoft.com/library/default.asp?url=/library/enus/cpref/html/frlrfSystemByteClassTopic.asp

33

Annexes

What is Kidsteam?

http://www.cs.umd.edu/hcil/kiddesign/ Mona Leigh Guha, Allison Druin, Gene Chipman, Jerry Alan Fails, Sante Simms, Allison Farber. Mixing Ideas: A New Technique for Working with Young Children as Design Partners. http://hcil.cs.umd.edu/trs/2004-01/2004-01.pdf, 2004

PDA interactions via infrared ports

http://en.wikipedia.org/wiki/Bluetooth http://www.bluetomorrow.com/content/section/10/37/ http://en.wikipedia.org/wiki/Wi-Fi http://en.wikipedia.org/wiki/Infrared#Communications http://msdn.microsoft.com/library/default.asp?url=/library/enus/cpref/html/frlrfsystemwindowsformscontrolclassinvoketopic.asp Ken Hinckley, Gonzalo Ramos, Francois Guimbretiere, Patrick Baudisch, Marc Smith. Stitching: Pen Gestures that Span Multiple Displays. http://www.cs.umd.edu/~francois/Papers/2004-Hinckley-AVI04-Stitching.pdf, 2003 Jun Rekimoto. Pick-and-Drop: A Direct Manipulation Technique for Multiple Computer Environments. http://www.csl.sony.co.jp/person/rekimoto/papers/uist97.pdf, 1997

Possible extensions to the project

http://en.wikipedia.org/wiki/Smartphone http://en.wikipedia.org/wiki/Midlet http://en.wikipedia.org/wiki/Java_package http://en.wikipedia.org/wiki/JAR_%28file_format%29

Annex 1: The WAV file format

http://en.wikipedia.org/wiki/WAV http://ccrma.stanford.edu/CCRMA/Courses/422/projects/WaveFormat/

34