ACKNOWLEDGEMENT

First of all, I express my sincere thanks to Principal Dr.K.K.Falgunan for givin

g permission to conduct this seminar and giving lot of encouragement.I am thankf
ul to all faculty members in MCA department for their cooperation towards thisse
minar presentations.At last, but not the least I would like to express my thanks
to omnipotent god give thestrength to conduct the seminar, and also sincere tha
nks to my friends, who have participated inthe seminar and encouraged me much.
ABSTRACT
We present Skinput, a technology that appropriates the human body for acoustictr
ansmission, allowing the skin to be used as an input surface. In particular, we
resolve the locationof finger taps on the arm and hand by analyzing mechanical v
ibrations that propagate through thebody. We collect these signals using a novel
array of sensors worn as an armband. This approachprovides an always available,
naturally portable, and onbody finger input system. We assess thecapabilities, a
ccuracy and limitations of our technique through a twopart, twentyparticipant user
study. To further illustrate the utility of our approach, we conclude with sever
al proofofconceptapplications we developed.Author KeywordsBioacoustics, finger inpu
t, buttons, gestures, onbody interaction, projected displays, audiointerfaces
CONTENTS
1.
INTRODUCTION 12.RELATED WORK 23.SKINPUT 44.BIO-ACOUSTIC 45.SENSING 56.ARMBAND PR
OTOTYPE 77.PROCESSING 78.EXPERIMENT 99.DESIGN AND SETUP1110.RESULTS13
11. SUPPLEMENTAL EXPERIMENTS 1512. EXAMPLE INTERFACES AND INTERACTIONS 1813. CON
LUSION 2014. REFERENCE 21

ARMBAND PROTOTYPE
Our final prototype, shown in Figures 1 and 5, features two arrays of five sensi
ng elements,incorporated into an armband form factor. The decision to have two s
ensor packages was motivatedby our focus on the arm for input. In particular, wh
en placed on the upper arm (above the elbow),we hoped to collect acoustic inform
ation from the fleshy bicep area in addition to the firmer area onthe underside
of the arm, with better acoustic coupling to the Humerus, the main bone that run
sfrom shoulder to elbow. When the sensor was placed below the elbow, on the fore
arm, one packagewas located near the Radius, the bone that runs from the lateral
side of the elbow to the thumb sideof the wrist, and the other near the Ulna, w
hich runs parallel to this on the medial side of the armclosest to the body. Eac
h location thus provided slightly different acoustic coverage and information,he
lpful in disambiguating input location.Based on pilot data collection, we select
ed a different set of resonant frequencies for eachsensor package (Table 1). We
tuned the upper sensor package to be more sensitive to lowerfrequency signals, a
s these were more prevalent in fleshier areas. Conversely, we tuned the lowersen
sor array to be sensitive to higher frequencies, in order to better capture sign
als transmittedthough (denser) bones.
PROCESSING
In our prototype system, we employ a Mackie Onyx 1200F audio interface to digit
allycapture data from the ten sensors. This was connected via Firewire to a conv
entional desktopcomputer, where a thin client written in C interfaced with the d
evice using the Audio StreamInput/Output (ASIO) protocol.
7

Each channel was sampled at 5.5kHz, a sampling rate that would be considered too
low forspeech or environmental audio, but was able to represent the relevant sp
ectrum of frequenciestransmitted through the arm. This reduced sample rate makes
our technique readily portable toembedded processors. For example, the ATmega16
8 processor employed by the Arduino platformcan sample analog readings at 77kHz
with no loss of precision, and could therefore provide the fullsampling power re
quired for Skinput.Data was then sent from our thin client over a local socket t
o our primary application,written in Java. This program performed three key func
tions. First, it provided a live visualization of the data from our ten sensors,
which was useful in identifying acoustic features. Second, itsegmented inputs f
rom the data stream into independent instances (taps). Third, it classified thes
einput instances.The audio stream was segmented into individual taps using an ab
solute exponential averageof all ten channels. When an intensity threshold was e
xceeded, the program recorded the timestampas a potential start of a tap. If the
intensity did not fall below a second, independent closing threshold between 100ms
and 700ms after the onset crossing, the event was discarded. If start andend cr
ossings were detected that satisfied these criteria, the acoustic data in that p
eriod (plus a 60msbuffer on either end) was considered an input event (Figure 6,
vertical green regions). Althoughsimple, this heuristic proved to be highly rob
ust, mainly due to the extreme noise suppressionprovided by our sensing approach
. After an input has been segmented, the waveforms are analyzed.The highly discr
ete nature of taps (i.e. point impacts) meant acoustic signals were not particul
arlyexpressive over time. Signals simply diminished in intensity overtime. Thus,
features are computedover the entire input window and do not capture any tempor
al dynamics.
8
We employ a brute force machine learning approach, computing 186 features in tot
al, manyof which are derived combinatorially. For gross information, we include
the average amplitude,standard deviation and total (absolute) energy of the wave
forms in each channel (30 features). Fromthese, we calculate all average amplitu
de ratios between channel pairs (45 features). We also includean average of thes
e ratios. We calculate a 256point FFT for all ten channels, although only thelowe
r ten values are used, yielding 100 features. These are normalized by the highes
tamplitude FFTvalue found on any channel. We also include the center of mass of t
he power spectrum with in thesame 0Hz to 193Hz range for each channel, a rough e
stimation of the fundamental frequency of thesignal displacing each sensor. Subs
equent feature selection established the allpairs amplitude ratiosand certain ban
ds of the FFT to be the most predictive features.These 186 features are passed t
o a Support Vector Machine (SVM) classifier. A fulldescription of SVMs is beyond
the scope of this paper. Our software uses the implementationprovided in the We
ka machine learning toolkit. It should be noted, however, that other, moresophis
ticated classification techniques and features could be employed. Thus, the resu
lts presentedin this paper should be considered a baseline.Before the SVM can cl
assify input instances, it must first be trained to the user and thesensor posit
ion. This stage requires the collection of several examples for each input locat
ion of interest. When using Skinput to recognize live input, the same 186 acoust
ic features are computedonthe fly for each segmented input. These are fed into the
trained SVM for classification. We usean event model in our software
once an in
put is classified, an event associated with that location isinstantiated. Any in
teractive features bound to that event are fired. As can be seen in our video, w
ereadily achieve interactive speeds.