10/24/13

EQEM Print

Print Page

THE OUTER LIMITS

8/1/2000

Over the years, innovative approaches to using physical gestures inelectronic music have been highlighted in t pages of EM.Although some of the technologies weve covered have hadmainstream commercial success, ma technologies haveremainedeither by design or by accidenton thefringes.

However, like artistic and musical works, the commercial success ofa new controller shouldnt be the sole crite for judging itsworth. Like hammers and chisels, controllers are merelytoolsalbeit, in some cases, rather sophisticated tools that mayrequire a major paradigm shift in order to understand their fullpotential. Ultimately, these new tools are only a means to a musicalend; a controllers effectiveness at getting across musical ideasw be the greatest factor in its success.

David Wessel, esteemed researcher of gestural controllers anddirector of the Center for New Music and Audio Technologies (CNMAT) atthe University of California at Berkeley, put it best when he recentlynoted, Were on t verge of a controllerrenaissance. This is primarily due to the growing number ofmusicians and engineers fightin to keep electronic music a uniquemedium of expression rather than a means of mimicking establishedforms. M composers and musicians are using input devices that retainelements of traditional instruments, while others ar tracking gesturesin new ways by measuring motion, light, gravity, temperature, airpressure, proximity, and anything else you can imagine. Rarely in thehistory of music has there been so much workand such variedresultsin instrument development.

With that in mind, we decided it was high time to survey the currentapproaches to alternative input devices for electronic music. Some ofthe approaches are of the most personal and intimate kind, whereasothers are geared the mass market. This article will cover bothextremes, as well as the universe of approaches in between.Commercially available MIDI controllers based strictly on conventionalinstruments, such as percussio pads, guitar, bass, and windcontrollers, and variations on the standard piano keyboard (such asaccordion controllers) will not be covered. ITS ALL ABOUT INPUT

Over the centuries, every culture has developed ways to expressitself musically. By contrast, the field of electronic music, just overa century old, is perhaps moving into its adolescent stage,metaphorically speaking. F of the earliest performance-basedelectronic instruments have survived this short test of time, thenotable exceptions being the theremin and the Ondes Martenot. But untilrecently, technology imposed strict limitations what a performercould do in real time.

These days, the tools needed to analyze and use multiple streams ofinput data in musical contexts are readily available and shrinking insize and price. In fact, everyday computer input devices, such asjoysticks and graphi tablets, are sophisticated and inexpensiveenough to work in musical contexts. And as musical- instrumenttransducer systems get smaller and more powerful, acoustic and gesturalsensors are becoming easi to use with traditional instruments.

Already, external cable connections are beginning to disappear oninstruments; external cables will disappear completely asmicrotechnologies become more available. Imagine the possibilitiespresented by miniature computers (including power supply and wirelesstransmitter) that are less than a cubic millimeter in size. Thistechnology is what Kris Pister at U.C. Berkeleys Department ofElectrical Engineering and Computer Scien terms smartdust (http://robotics.eecs.berkeley.edu/~pister/SmartDust):microtechnology that, perhaps within th decade, will have a majorimpact on real-time performance applications. MIDI OR BUST Even though technologies change rapidly, well-designed instrumentsnever become obsolete.

Recent technological progress has allowed designers to use powerfulprocessors in smaller spaces and to overcome difficulties in precisepitch extraction; however, they still come up against the bandwidthlimitations of MIDI. No matter how small and unobtrusive the sensorsare, MIDI still imposes a speed limit of 31.25 Kbps. An althoughfaster data-transmission schemes abound, none have gotten enoughpopular support to dethrone the M

www.emusician.com/prntarticle.aspx?articleid=145127

1/12

10/24/13

EQEM Print

1.0 specification. Yet engineersand artists have worked around the problems presented by MIDI to createinput devices that allow new and exciting ways of making music. CONTROL SURFACES

MTC Express. One of the newest and most promising touchsurfaces being developed is a 3-D controller that uses smartfabrica soft material develop by the Canadian Space Agencythat contains an array of sensors interconnected by a network offiber-optic cables. The MTC Express ($495) from Tactex Controls (www.tactex.com), is asquare of smart fabric, measu 5.75 by 3.75 inches, that is coveredwith a padded surface and housed in a anodized aluminum slab (seeFig. 1). The entire unit weighs just 17 ounces.

Fig.1 : The MTC Express, from Tactex Controls, can track the Velocity andposition of several stimuli simultaneously.

The MTC Express can track multiple contact points within a 2-D (x-y)field, with a sensitivity of around 100 dots per inch. It also measures256 levels o pressure. The surface of the smart fabric is covered withnumerous taxels, each connected to a pair of fiber-optic cables.Because of the sensor densi the MTC Express can track very subtlephysical gestures.

Light from an LCD is sent down one of the cables and is returnedover the other. Sensors are used to track the deformation of the fibersby measuring the amount of light sent back down the cables when thefabric is touched As pressure is exerted on the surface of the MTCExpress, variations in light are returned to the sensors.

Jam Bass. The Jam Bass ($253.50), by Kellar Bass Systems, isa MIDI controller (with internal synth) that attaches to the neck of anelectric guitar bass. The control surface is made up of two rows of14 pads that mimic the fret layout below the E and A guitar strings.The pads are played with the thumb while the other fingers are on thefretboard. Using a ribbon cable, the control surface is attached to theCircuit Pack, which contains the processor, synth, and audio and MIDIoutputs You can change Synth Voice and Performance modes using thepads.

FIG.2: More than just a ribbon controller, the Kurzweil ExpressionMate cansend Note On and a variety of controllermessages.

ExpressionMate. At first, the Kurzweil ExpressionMate ($549)looks like the ribbon controllers that were used with analog synths inthe 60s and 70s (se Fig. 2). But unlike olderribbons, this one can send an impressive array of M messages. Theribbon surface is divided into three sections, each individuallyassignable. The unit also includes three 16-step arpeggiators th cansend and receive data on separate MIDI channels. The control box can beconveniently mounted on the keyboard or on a mic stand.

Thunder. Although it debuted more than a decade ago, theThunder ($1,990 by Buchla and Associates, gives the performer asophisticated touch-contro surface. The thunderbird-style surfacedesign is the result of ergonomic considerationsthe layout comesfrom tracings done around the designers hands, and the long,featherlike control strips sit nicely under the fingers. E of thesecontrol strips can track two control dimensions: pressure and position(see Fig. 3). All ten fingers can send pressure and positiondata simultaneously. Although the Thunder is no longer manufactured, alimited number of the controllers are still available directly fromBuchla and Associates.
FIG.3: David Wessels custom Thunder interface. Notice that some ofthe controllers have two sliders: one for vertical finger position, theother for pressure.

A more recent development by the company is the MarimbaLumina ($2,995 an instrument that combines the familiar design ofa 31/2-octave mallet controller with advanced electronic technology andBuchla-style ingenuity. Designed by Donald Buchla with input frompercussionists Mark Goldstein a Joel Davel, the Marimba Lumina ismanufactured by Nearfield Multimedia, specialists in precision antennatechnology.

The Marimba Lumina comes with four color-coded, foam-covered malletsthat contain tuned circuitry. Embedded each bar, strip, and pad onthe surface of the instrument is a radio antenna that can track andidentify the mallet allowing each mallet to have independent controlfunctions. Although cosmetically it bears some resemblance to moreconventional mallet controller, the Marimba Lumina is a highlysophisticated instrument capable of mapping variety of responses toperformance gestures over its various control surfaces.

A special Gold Edition Marimba Lumina ($8,000) is a41/2-octave version of the instrument that features gold- plated barsand a curved frame that allows players to reach the furthest noteseasily. (See Whats New in the October 1999 issueof EM.) Both instruments include a Yamaha DB51 XG synthesizer,so they can be used with an external sound source.

Finally, there is the Marimba Lumina 2.5 ($1,995), a new21/2-octave version of the instrument (see WhatsNew this issue).

www.emusician.com/prntarticle.aspx?articleid=145127

2/12

10/24/13

EQEM Print

Starr Labs controllers. Harvey Starr, who heads Starr Labs(http://catalog.com/starrlab), has designed a number variations on the MIDIguitar controller theme. Avoiding the common pitch-to-MIDI schemes usedin guitar controllers, Starrs devices are more user interfacesthan MIDI guitar controllers. Starrs controllers often combineguitarlike fingerboards with keys on the neck instead of strings.Sometimes a breath controller and joys are thrown in for goodmeasure.

The following examples by Starr Labs represent only a fragment ofthe companys output. Any of the configurati can be customizedto fit the needs of the musician.

The Mini-Z ($1,395) is a Velocity- and Pressure-sensitive 24-fretfingerboard designed for tapping. Options include a 4-way joystick, abreath controller, a a programmable strip along the side of the neckthat can add Modulation, P Bend, or crossfades between sounds. TheMini-ZS ($1,795) includes the optional joystick and a set of sixVelocity-sensitive string triggers. The Mini ($1,695) adds a set ofdrum machine style trigger pads. The Mini-ZXS ($1,995) containsall of the above: joystick, trigger pads, and string triggers (seeFig. 4). Note that the prices quoted above are for basic modelsonly; an added options or customization costs extra.

Starr Labs MT-48DD ($2,195) was originally designed forbassist Billy Sheehan, who wanted pedals that were better suited forbass playing than t FIG.4: Instruments from Starr Labs can traditional pedal configuration. The unitconsists of a 4 5 12 array of 2-inch be customized with variouscontrollers to fit the performers needs. Shown here rubber mounds that are playablewith mallets as well as with feet. Each mou are theMini-ZXS and Mini-ZS, which is individuallyprogrammable and can send a MIDI event or group of events include Velocity-sensitive stringtriggers. up toeight per moundincluding notes, chords, or even SysEx messages. floor unit connects to a stand-mountable programmer that holds upto 32 user programs.

A collaboration between Harvey Starr, Stephen Taylor, andmicrotonalist Ervin Wilson has produced the Wilson GeneralizedKeyboard Controller ($7,500). Inspired by the Generalized Keyboard thatR.H.M. Bosanquet develop in 1875, it is designed to work both as amicrotonal performance controller (with the ability to map multipletonal systems) and as a more traditional controller offering multiplemini-keyboards on a single surface.

The Wilson 990 includes nine ranks of 90 keys. Each rank is assignedits own MIDI channelin fact, each key transmit multipleuser-defined MIDI messages. The instrument comes with tuning softwarethat is compatible wit E-mu and Ensoniq sound modules, as well asSymbolic Sounds Kyma system. Groups of keys can be defined for different musical purposes, anddifferent setups can be edited, stored, and recalled using a providededitor/librarian program. In addition to the banks of keys, the Wilson990 has four assignable sliders and a 4-way programmable joystick. Asmaller 288-key version ($3,200) is also available.

Besides these commercially available products, there are a couple ofproprietary touch-sensitive controllers that offer interestingsolutions to specific performance requirements.

JamODrum. Developed by Tina Bean Blaine and TimPerkis at Interval Research in Palo Alto, California, the JamODrum ismeant to inspire people to engage in spontaneous, collaborative musicmaking. In fact, the community drum circle became a metaphor thatguided the form and content of the teams work. The designersa intended the JamODrum as a way for participants to explore therelationships between rhythm and graphics. The resultant objecta7-foot circular table that includes six MIDI drum pads and doubles as avideo-projection surfa was something that people could gatheraround for jamming.

Blaine and audio engineer Kris Force spent several months slicing,dicing, and processing thousands of sample create a custom libraryfor the project. During the JamODrums development phase, severalinteraction methods were available. For example, in thecall-and-response method, the sequencer plays short rhythmic patternsthat trigger synchronized flashing of the call area inthe center of the screen. The call patterns are followed by spac forplayers to copy the pattern, directed by response cues. YourTurn indicators allow everyone at the table to p together, tobe split into subgroups, or to support solo sections. Once the playerscatch on to the system of whe play and when to listen,opportunities emerge for more-experienced players to improvise withinthe form. Althoug some players have found this rhythmic learningapproach too structured to be entertaining, others have enjoyed itsSimon says aspect. Of the many interaction methods explored, the JamODrum designersfound that call-and-response was the mos successful in bringing noviceand expert players together for musical collaboration. A JamODrum installation that scales from 6 to 12 participants iscurrently on exhibit at the Experience Music Project in Seattle. Athree-person JamODrum was recently donated by Paul Allen/VulcanVentures to the Entertainment Technology Center at Carnegie MellonUniversity in Pittsburgh (www.etc.cmu.edu).

Talking Stick. Under the direction of Bob Adams, a smallgroup of musicians and technologists at IntervalResea including Geoff Smith, John Eichenseer, JesseDorogusker, Mark Goldstein, and guitarist/producer MichaelBr joined forces to create a touch-sensitive, tubularinstrument called the Stick (no relation to the Chapman Stick

www.emusician.com/prntarticle.aspx?articleid=145127

3/12

10/24/13

EQEM Print

Theproject combines customized circuitry with an array of force-sensitiveresistors (FSRs) that take multiple da measurements with one touch.Because Brook intended the Stick to be played in a way similar to anacoustic ba a vertical strip of FSR linear potentiometers was usedin lieu of a fretboard. Location and velocity information is determinedby the amount of surface pressure applied by the playersfingers. To independently measure force a position, the teamdeveloped a custom library of Max patches for the Stick.

With plans under way for her work Songs and Stories from MobyDick, Laurie Anderson, with the help of Bob Bielecki and incollaboration with Interval Research, extended the capabilities of theStick controller. Repurposed specifically for Andersonshybridized approach to music, movement, and spoken word, the newTalking Stick features a linear potentiometer and a pressure-sensitiveactuator for the manipulation of sampled audio, as well a wirelesstransmitter for sending control information.

Anderson uses the Talking Stick to evoke the clicking patterns inthe language of sperm whales and the creaks and groans of a ship. JohnEichenseer, Lukas Girling, and Dominic Robson used Cycling74s Max/MSP to crea variety of granular synthesispatches for the short sound fragments. During performance, thesefragments are modified and resequenced in real time. Besides the four Talking Sticks featured in Moby Dick, threeother Sticks are in existenceone at Stanford University, one atthe Berklee College of Music in Boston, and the one in the possessionof Michael Brook.

MIDIBall. Fans rushing a stage at a concert in Tokyo inspiredCandice Pacheco, cofounder of DCckoo, to des a giganticbeach ball that creates music as the audience bats it around. TheMIDIBall, a wireless 5-foot sphere, converts radio signals into MIDIcommands that trigger audio samples and real-time 3-D graphics withevery blow The biggest challenge was finding a plastic material thatwould be durable enough to protect embedded sensors and withstand heavyhitting, yet would appear to float. The MIDIBall also required wireless technology with a thresholdsensitive enough to reliably interpret a range of raps, slaps, andpunches without double-triggering. After experimenting with severaldifferent sensors, Pacheco ended up inserting an RF transmitter in asleeve sewn into the middle of the MIDIBall. The MIDIBall debuted att Grateful Deads Mardi Gras show at the Oakland Coliseum in1992. STRINGS N THINGS

Research and development in the field of bowed-string controllershave been going on for some time; examples include IRCAMs(Institute of Research and Coordination in Acoustics and Music)SuperPolm for violinist Suguru Goto, Tod Machovers work with thehypercello, and Peter Beylss use of multiple infrared sensors onviolins at Brussels University in Belgium.

However, the international ambassador of the extended violin is JonRose. Over the years, the English-born inventor/performer hasimplemented a number of technological innovations for the instrumentand has also developed some unique deconstructedbowed-string instruments. Roses interactive setups, developedwith hel from STEIM (STudio for Electro-Instrumental Music), combinean accelerometer on the bowing arm, an ultrasou sensor mounted on thebow, a bow-mounted sensor that measures bow pressure on the strings,and three MIDI footpedals (www.euronet.nl/users/jrviolin). BOLD AS GLOVE

Sophisticated glovelike controllers continue to be popular withperformers of electronic music. At the end of the 80s, TomZimmerman and Jaron Laniers DataGlove had scored some notoriety.When Mattel used the technolo in its own junior version of thecontrollercalled the PowerGlovemusicians such as MarkTrayle were able to hack into the greater potential of the $79 toy.

The Hands. In 1984 in the Netherlands, Michel Waisvisz beganperforming with an instrument he helped develop called The Hands. Theoriginal controller was made up of a group of keys and sensors mountedunder his fingers and thumbs. The data collected by the sensors wastranslated into MIDI using a microcomputer and custom software thateventually became the SensorLab, marketed by STEIM. Over theyearswith engineering help from Frank Balde, Johan denBiggelaar, Bert Bongers, Peter Cost, Tom Demeijer, Wim Rijnsburger, andHans Venma Waisvisz has made incremental improvements to hisgestural hand controllers. But Waisvisz keeps technological upgrades toa minimum. This allows him to master the performance techniquesnecessary for exploring the limits presented by the controller. The latest version, Hands II, is a more refined version of theoriginal controllerwith upgraded wiring andcomponentsthat measures finger, hand, and arm movements. Thedistance between the hands is also measured, using ultrasound sensors.Hands III is currently being developed at STEIM by Jorgen Brinkman.

Ladys Glove. In 1991, Laetitia Sonami (www.sonami.net) beganher work on the Ladys Glove by attaching magnetic sensors to thefingers of latex gloves. Since then, the Ladys Glove has gonethrough a series of radic design changes, most recently with the helpof engineer/designer Bert Bongers through a sponsorship from STE

In its present implementation, the Ladys Glove (see Fig.5) is a full-length Lycra glove with an accelerometer that measureshand speed; numerous mo

www.emusician.com/prntarticle.aspx?articleid=145127

4/12

10/24/13

EQEM Print

and pressure sensors; and ultrasoundtransmitters and receivers that detect distance between the gloveand the floor.

Sonami uses the Ladys Glove in live performances to controlsound, mechanical devices, and lights via MIDImostly in solosituations but also improvisations with other instrumentalists. Hercurrent setup includes a STE SensorLab processor and a Mac laptoprunning Max/MSP. WE SING THE BODY ELECTRIC

BioMuse. Researchers Hugh S. Lusted and R. Benjamin Knapphave created BioMuse (www.biocontrol.com), an interface that analyzes andinterprets the signals from up to eight simultaneous bioelectricsensors and translates them into MIDI data.

FIG.5: Laetitia Sonamis Ladys Glove includes sensors thatdetect the distance between the glove and thefloor.

For decades, composers have been using electrical signals from thebody as source for electronic music. Most of these systems were builtusing parts originally designed for medical or scientific purposes.Although artists workin with biofeedback continue in this mannertoday, a number of companies are marketing systems that are directlyapplicable to music.

BioMuse has been used primarily with electromyographic (EMG) sensorsthat measure the flexion and extension muscles. However, othersensors can be used with the system, including those that read eyemovement (electrooculographic) and brain waves(electroencephalographic, or EEG).

BodySynth. The BodySynth ($1,499), developed by EdSeveringhaus and Chris Van Raalte, uses EMG sensors attached to thebody of the performer (see Fig. 6). The bas setup comes withfour EMG sensors, a two-position switch, the Body Unit, a wirelesstransmitter, and a remote processor that handles the necessary A/Dconversio and signal processing and includes a collection ofalgorithms for musical handling of th MIDI data.

Input from each of the four EMG sensors can be mapped to any of theremote process eight MIDI output channels (or systemchannels, as the designers call them). The BodySynth can handle up to12 EMG electrodes simultaneously.

FIG.6: The BodySynth can track up to 12 EMG sensors simultaneously fortranslating muscle movement into MIDI data.

Once they are attached to the body, the EMG sensors are plugged intothe Body Unit, device roughly the size of a cigarette pack that isworn by the performer. The Body Un comes with four EMG amplifiers anda gain control for each of the four channels. The performer also wearsa wireless transmitter that sends the signals from the Body Unit t theremote processor. Each BodySynth is configurable to fit the needs ofthe user.

FIG.7: All of the I-CubeXsensors come prewired and ready to useno soldering is necessary.As many as 32 sensors can be used with the system at onetime.

I-CubeX. The I-CubeX ($625), manufactured by Infusion Systems(www.infusionsystems.com comprehensive system that can translateup to 32 analog voltages into MIDI data from a varied host gesturaland environmental sensors (see Fig. 7 I-CubeX Digitizerhandles the I/O; besides MIDI information, the unit can also output1-bit voltages (that is, 0 or 5 volts). In performance, the I-CubeX canbe used with or without a computer. It has edit for both the Mac OSand Windows 95 that allow yo to store sensor setups, enabling it tofunction as a stand-alone unit.

Each I-CubeX system comes with a Turn sensor a See actuator.Additional sensors are available that measure temperature, light,pressure, acceleration, distance and proximity; all are specificallycreated for use with the I-CubeX. For example, the TapTile is a12-inch-square that measures pressure when it is stepped on ordanced on. Another device, the TouchGlove, contains sixsenso one used in the palm and five on the fingertips.

GoFly/IRFly. Infernal Devices (www.infernaldevices.com) offers two environmentalsensors in its Sensopede se of products. The GoFly ($95) is a tinyheat sensor (1.25 by 1.5 inches) that can be used to detect the motionof people in a room. The IRFly Ranging Detector ($59) is a1.25-by-1.5-inch infrared detector that ignores other ligh and heatsources. Both devices work with processors made by Infusion Systems,with Beehive Technologies A I/O, and with STEIMsSensorLab (see the sidebar Preaching to theConverters). IBVA. IBVA Technologies (www.ibva .com) sells a system that couples EEGbrain-wave sensors with Mac software specifically tailored to musicalapplications. The single-channel IBVA (which stands for InteractiveBrainwave Visual Analyzer) Core system ($1,300) includes a headbandwith electrode sensors, a wireless transmitter and receiver, and a PinInput extension pack for connecting other types of

www.emusician.com/prntarticle.aspx?articleid=145127

5/12

10/24/13

EQEM Print

biofeedbacksensors.

Software accompanying the Core system includes the Step 1 ExpansionPak (containing various control applications), Alps+ for brainwavetoMIDI control, and a software synthesizer. In fact,the IBVA system comp with the General MIDI specification.

MIDIVox. An interesting technique for tracking the voicenonacoustically involves electroglottography (EGG). An EGG sensormeasures laryngeal behavior through changes in electrical impedanceacross the throat. The MIDIV ($1,295) uses EGG sensors for convertingpitch and intensity data from the larynx as MIDI, binary, analog, andg output. Invented by SynchroVoice, the MIDIVox is now availablefrom HealingMusic.net.

The MIDIVox is made up of two components: a neck band (available inblue or black) with four biosensors, and 1U rack-mount interfacemodule. The hypoallergenic neck band is wrapped around thesingers neck and affixed Velcro so that two of the sensorsare on either side of the Adams apple. The neck band attaches tothe process with a ribbon cable. Since its review in the July 1992issue of EM, the MIDIVox has been upgraded with new motherboardsand EPROMs, larger biosensors, and a new Velcro neck band. TOUCHE PAS

Until the 20th century, the sense of touch has been one of the mostimportant elements in playing musical instruments. With few exceptions(primarily the voice and the aeolian harp), earlier instrumentsrequired physical contact to make them work. Recently, however, thisfundamental principle has changed.

Some gestural controllers operate by measuring the capacitance of anobject. The capacitance (or ability to hold electrical charge)varies based on the objects distance from an adjacent object.Once the measurements of thes changes are recorded, they can beconverted to MIDI information with the assistance of A/Dconverters.

Other controllers use beams of light or ultrasound. For example,when an object is moved within a field of infrare light, opticalsensors measure the increasing and decreasing amount of reflected lightand generate MIDI data based on the values produced by thesemeasurements. (For a more technical description, see ScottWilkinsons article Interactive Light in theDecember 1995 issue of EM.) This particular technology is not arecent developm several MIDI guitar controllers have beendeveloped that measure reflected light as an alternative topitch-to-M conversion, where fingers placed on the fretboardinterrupt an infrared beam, and the altered length of the beam thenconverted to a MIDI Note message.

Radio Drum/Radio Baton. Developed by Bob Boie and MaxMathews, the Radio Baton and the Radio Drum track the 3-D movement oftwo or more batons over a base unit. The ends of the batons are coveredwith copper tape topped with large foam balls that resemblemallets. Each baton transmits a discrete frequency that is localized bymeasuring the electrical capacitance between the tip of the baton andthe array of receiving antennas embedd in the base unit.

The system allows performers to predefine behaviors of thebatons/mallets. For example, Mathews wrote a conductor program toprovide new ways of interpreting and performing traditional musicscores. This software, coupled with the Radio Baton, enables singersand soloists to create their own orchestral accompaniment, processtheir voice, or work with algorithmic compositions. A program writtenby Andrew Schloss lets a performe conduct Standard MIDI Files. In thismode, the baton sends motion information to a computer, which theninterpr the batons signals and sends MIDI commands to asynthesizer for playback.

An interesting example of a work created using the Radio Drum iscomposer David Jaffes pairing with Schloss the duo Wildlife.With Jaffe playing a Zeta violin and Schloss using the Radio Drum, twocomputers interpret and respond to each players actions,superimposing the output of one instrument onto the other. For example,a glissando on the Zeta violin can change the pitch of notes played onthe Radio Drum, or the drum can modify th output of the Zeta.Performing in such situations requires the musicians to develop newinteractive skills, especi when their musical intentions arewrested away from them by other musicians.

Ethervox MIDI theremin. The theremin is arguably one of thefinest and best-known gestur controllers. Big Briars Ethervox(www.bigbriar.com) offers several enhancements to the traditional theremin,including the addition of more complex waveforms and a filter that can modulated by pitch (see Fig. 8). The Ethervox theremin cancontrol external sound source and can be played from a MIDIsequencer.

In many ways, the Ethervox is fairly straightforward in its MIDIimplementation. Pitch Bend transmitted, as is Control Change 7(volume). The device can also send and receive Note Note Off,Velocity, Program Change, and System Exclusive messages. As you wouldprobably guess, the Ethervox sends a constant stream of Pitch Bendmessages. Fortunately, the user can scale the updating of Pitch Bendand Volume information to thin the MIDI data stream. Big Briars most popular theremin, the Etherwave ($369assembled; $299 kit), doesnt hav MIDI capabilities but doesboast a 5-octave range as well as controls for waveshape andbrightness, and it comes with an instructional videotape.

www.emusician.com/prntarticle.aspx?articleid=145127

6/12

10/24/13

EQEM Print

Dimension Beam. Developed by Interactive Light, the DimensionBeam uses an infrared beam to track the position of an object movingwithin its field. This invisible beam is shape like an elongated eggbalanced on one end, and it has 128 layers that can be assignedspecific MIDI values.

Lightning II. Buchla and Associates Lightning II($1,990) is a spatial controller that feature pair of wirelesswands, a half-rackspace processor, and a stand-mountable remotereceiver The Lightning II uses infrared trigonometry to track thevertical and horizontal position as w as the velocity of each wand.It divides the performance space into eight Zones, configure a 4 52 array. A Stimulus from either wand can be assigned to each of theZones. Stimuli include movement with a Zone; entering or exiting aZone; and clicking, double-clicking, or releasing the buttons. You canalso use footswitches with the Lightning II, for yet another level ofcontrol.

FIG.8: The BigBriar Ethervox theremin is a world-class performance instrument withextensive MIDI capabilities.

One of the interesting aspects of the Dimension Beam is that,although the device may ha seemed a bit esoteric when it was firstreleased in the mid-90s, it has since found commercialacceptance: Roland licenses the technology (now referred to as DBeam,and n longer sold to the public by Interactive Light) for use in manyof its products, including the SP-808 Groove Sampler, the HPD-15HandSonic drum controller, and the MC series of Groove Boxes.

The processor contains a 32-voice, 18-bit Kurzweil MASSsample-playback chip that provides a General MIDI sound bank. Eachpostage stampsize memory card can store 30 presets. A preset canhold up to 40 patches th contain user-definable mapping of gesturesto responses.

Soundbeam. Developed more than a decade ago by EMS, theEnglish company famous for its popular Synthi and VCS3 analogsynthesizers, the Soundbeam (www.soundbeam.co.uk) is an ultrasonic MIDI-controlsystem favored in special-education situations where noninvasivebiofeedback is needed (see Fig. 9). With the recent release ofthe Soundbeam 2 ($2,777), EMS has enhanced the feature set of theoriginal model to include the abilit track the speed of a movingobject within the ultrasonic field, as well as proximity and on/offstatus. The Soundbeam 2 also allows you to divide eac ultrasonic beaminto 64 sections, each of which can be assigned its own notes, chords,or MIDI control parameter.
FIG.9: The ultrasonic sensing capabilities of the Soundbeam 2 make thesystem ideal for situations that require noninvasivebiofeedback.

Up to four ultrasound beams can be used simultaneously, each withits own MIDI channel. The Switchbox ($250) sends data from eightadditional controllers (such as switches and joysticks) to theSoundbeam 2 controller, and it works simultaneously with the beams. Thebeams have a variable range from 2 to 20 feet, whereas senso can beplaced more than 150 feet away from the controller. The Soundbeam 2comes with presets containing pit sequences, as well as a large bankfor user-defined sequences.

OptiMusic. The OptiMusic system (www.optimusic.com)uses reflected, visible light beams as controllers and is also usefulin physical-therapy situations. The system comes in two versions: thesingle-beam OM-L1 and the 7- beam OM-L7. Both versions are controlled bythe OM-PCI lighttoMIDI control software.

The lights sense the reflection from performers moving within thebeams. The color, shape, angle, sensitivity, a distance between thelights is customizable. The OptiMusic system allows up to 99 notes perbeam and can wor with up to 32 light beams. It will be available inSeptember. HOT OFF THE SHELF

One thing changing the alternative-controller landscape is thatstandard control devices from the graphics and gaming worlds arebecoming increasingly powerful, plentiful, and lower in price. And,more and more of them feat Universal Serial Bus inputs. Infact, notes CNMATs David Wessel, USB is just nowbeginning to mature as we speak.

Among the devices popular with electronic musicians are the Wacomgraphics tablets. For example, Wacoms U 1212-R offers, amongother things, 32,000 points along an x-y axis and pressure and anglesensitivity. In additio each UltraPenthe little stylus that isused as an input devicecan have its own ID.

Matt Wright, musical-systems designer at CNMAT, uses a Wacom tabletwhen performing. In recent performanc with Wessel and Pakistanisinger Shafqat Ali Khan, Wright used the tablet to playadditive-synthesis sound descriptions (using MSP) of Khansvoice. Wright uses templates to indicate how the sound descriptions aremapped horizontally across the tablet. This allows him both to scrubthrough the sound in either direction (wit the pitch kept independentof the scrubbing speed) and to visually locate particular musical partsso he can set t UltraPen directly onto them. Joysticks have also matured over the years, commentsWessel, and some include enough control options buttons,triggers, movement directionfor sophisticated musicmaking. After some research into the subject,

www.emusician.com/prntarticle.aspx?articleid=145127

7/12

10/24/13

EQEM Print

composer BobOstertag suggested to Wessel and CNMAT that the Cyborg 3D USB by Saitek(www.saitek.com one such controller worthy of investigation in this area. BRAIN OPERA

Perhaps the most prolific source of research into alternativecontrollers is MITs MediaLab, under the direction o composerTod Machover and Joseph Paradiso. The MediaLabs manydevelopments have moved beyond the ivo tower and are now used inprestigious venues around the globe. Paradiso is the technology director of the Things That Thinkconsortium, a group captivated with the infusion of intelligence intoeveryday objects. He also leads MITs Responsive Environmentsgroup, which has developed controllers used in musical/graphical boardgames, as well as smart sneakers that, when worn duringperformances, endow dancers with the ability to produce continuousmusical output.

Since the mid-1980s, Machover has been passionate about augmentingthe expressivity of traditional instrumen keyboards, strings,and percussionwith computer systems that measure and interprethuman expression and feeling. These hyperinstruments,which are tailored specifically for professional musicians, expand thecapabilit of existing instruments and redefine the ways in whichpeople interact with objects that may or may not typically beassociated with making music. As sensors, software, and signalprocessing have become more sophisticated the development ofhyperinstruments has evolved to include interactive musical instrumentsfor amateur musicia as well.

In 1996, Machover created a work that blends livehyperinstrumentalists with audience participationboth live andvia the Internetin the polymorphous production known as theBrain Opera(http://brainop.media.mit .edu). After twoyears of touring, the Brain Opera and its associated gadgetry are stillundergoing constant revisions an upgrades. Many of the controllersdescribed here will become permanent installations in ViennasHouse of Musi when it opens this summer.

For some attendees of the Brain Opera, the most engaging aspect isnot the actual performance but the opportunities for hands-on musicalinteraction. This takes place within an eclectic assortment ofsculptures know collectively as the Mind Forest.

Rhythm Trees. From the gigantic pods of the Rhythm Trees hang300 translucent rubber drum pads that trigger vocal samples when struck(see 10). Implanted in each pad is a pressure-sensitivepiezo-electric strip that c accommodate a wide range of strikevelocities. Networks of 30 pads are rou to a PC running customsoftware under Windows NT. As performers strike t pads, the softwarelooks for patterns between players and generates new rhythms inresponse, creating a kaleidoscopic array of sounds, lights, and imagesin the process.

Singing and Speaking Trees. The Singing Trees respond withaudiovisual feedback to the tonal quality of notes sung next to them. Awell-sung note FIG.10: The translucent, pressure- sensitive rubber pads of the Rhythm results in a calm, atmospheric response from the system,while badly sung Treeare an inviting interface for sonic notes receive a more complex response. Speaking Trees,on the other hand exploration in the MindForest. allow people to record stories, memoirs, lyricalphrasesin other words, to speak on any topic at all. SpeakingTrees automatically edit and process the recordings and incorporate thesoun bytes into each Brain Opera performance.

Harmonic Driving. One sculpture that closely approximates anarcade-style video game is the Harmonic Driving controller. Respondingto video generated by Rolf Randos 3-D rendering system on an IBMRS/6000 computer, participants use a steering wheel to navigate througha multicolored course complete with bends, turns, and potholes.Melodies and harmonies are created that correspond to the path taken;icons placed in the course sign hot orcool musical tracks. The driver determines whether theride is a rhythmic journey or a wandering detour ambient sound. Inthe true spirit of a video game, the player is given a skill rating atthe end.

Sensor Chair. Normally, one would think of a chair assomething you sit on while making music with an instrument. But theSensor Chair captures movement of the arms and upper body and convertsthose motions into mu while the performer is sitting in a chair (seeFig. 11).

Using the Sensor Chair, the performer becomes an extension of atransmitt antenna embedded in the chairs cushion. Fourreceiving antennas mounted poles around the Sensor Chair allow bodygestures to control sound. Joe Paradiso, Neil Gershenfeld, and EdHammond developed the chairs hardwa and Pete Rice and EranEgozy devised the interpretive software. The mysti nature of thisnew controller even attracted the magicians Penn & Teller, wh usedit in a routine to highlight the innovations of music, magic, andmachine
FIG.11: Tod Machover of MITs MediaLab performs in the Sensor Chair,which senses the position and movement of

Gesture Wall. Movement from a participants entire bodymodifies the on-go

www.emusician.com/prntarticle.aspx?articleid=145127

8/12

10/24/13
the performers armsand upper body.

EQEM Print

sounds and projected images as they stand nearthe Gesture Walls. Benea the floor are plates that transmit alow-voltage electrical signal to the participants body throughhis or her shoes. Sensors mounted around the projection screen abovethe walls recei these electrical signals as they leave the body. Thisallows the instrument to follow precisely even the subtlest ofgestures.

Future Music Blender. After wandering through the MindForest, visitors to the House of Music will have an opportunity toexplore a new addition to the Brain Operathe Future MusicBlender. Chris Dodge, Alex Westner, Peter Colao, and Ed Hammond workedwith Tod Machover to create a sonic sculpture designed to collect musicsamples that could easily be incorporated into a performance medley.While some of the sounds are activated from a prepared database, othersamples will be generated by visitors to the Mind Forest. Using aSen Chair retrofitted with a multimodal mixer forcontrol, people will be able to access, select, and play samples.Simply waving a hand in the air will enable a visitor toblend sounds that are then complemented by musicalaccompaniment (generated through custom algorithmic software) to createlarger compositions.

Music toys. The latest round of controllers is disguised asmusic toys. With names like Simple Things, the Big Thing,and Music Shapers, they are made from as many squeezable, stretchable,and reconfigurable parts and materials as possible. (Even Play-Doh isused as a conductive material to manipulate sounds.) The ultimate goalbehind the creation of these controllers is to distribute them tomusic-education programs in five host cities (New York, Boston, London,Berlin, and Tokyo). The culmination of the project will be a series ofToy Symphony compositions and performances with local symphonyorchestras.

As the name implies, Simple Things make simple sounds, such asindividual notes or sample playback, and are intended to be handheldstand-alone devices. Josh Strickon, Abie Flaxman, Tristan Jehan, andDiana Young use infrared links to exchange sounds between Simple Thingsand to synchronize groups of Simple Things.

Music Shapers are fabrics, furniture, balls, and spatial instrumentsthat offer fun, tactile ways to explore differen aspects of music.Designed by Maggie Orth and Gili Weinberg, these malleable instrumentsprovide new ways to physically shape and manipulate musical sounds andtextures by measuring the exertion of force and pressure stretchableobjects. This tangible approach to playing music is inspired by thedesire to create flexible interfaces complement the multidimensionalaspects of synthesizers and computer-generated music.

The Big Thinga multitiered structure designed by Orth,Weinberg, Dan Overholt, and Mary Farboodis intende to be thebrain that controls the network of interconnected music toys. The BigThing enables young people to compose, arrange, and perform music in a3-D construction kit, letting them experience musical expression bychanging the physical relationships between a variety of sensingobjects, connectors, and chunks. Each chu containssounds or commands that can be reorganized by moving, twisting, andinterconnecting with other chunk to create islands ofsound. The sensor objects can be used to add touch sensitivity orgesture-controller functio Obviously, children will need a bit ofcoaching before they can fully understand the capabilities of thesegiant Leg like controllers and music toys, but the prospects forcollaboration are encouraging.

There are many more controllerstoo many to list in thisarticlethat have also been an outgrowth of developmentthroughout MITs MediaLab. Among these are musicalbottles that control different sounds and patterns of coloredlight when the stoppers are removed; musical threads that are sewn intoa denim jacket and e sounds; and gesture-tracking digital batonsthat provide position, orientation, and surface-pressureinformation.

Conductors Jacket. Wearable computers are not anessential part of most peoples wardrobeunless you happento be hanging around Teresa Marrin Nakra. Interested in finding newavenues for musical and emotional expression while studying at theMediaLab, Marrin Nakra mounted a collection of EMG sensors to her upperbod with the wires strapped inside a conductorsjacket to collect a variety of physiological measurements.

After taking readings on heart rate, respiration, skin conductivity,temperature, and mus tension, Marrin Nakra hoped to find acorrelation by mapping these measurements to th gestures and rhythmictiming of conducting. She was able to convince a number of wel knownconductors, including Keith Lockhart of the Boston Pops, to try out theConducto Jacket in performance so that data pertaining tothese unknown relationships could be collected (see Fig. 12). Toher surprise, Lockhart was more than willing to get wired for hisaudiences; he even requested a flashier jacket than the one that wasoriginally provided. STEIM

FIG.12: Boston Pops conductor Keith Lockhart wears the ConductorsJacket during 1998s Tech Night at thePops.

As a research center dedicated to the performing arts, STEIM (STudiofor Electro- Instrumental Music) has been a hotbed ofalternative-controller research (www.steim.nl Notable developments include TheHands, developed with Michel Waisvisz, as well as Sweatstick andthe MIDI Conductor. Important computer applications for use in liveperformance have also come from STEIM, such as LiSa, for live sampling;BigEye, which converts video into MIDI data; and Image/ine, forreal-time video manipulation. A number of prominent performers using live electronics havedeveloped highly

www.emusician.com/prntarticle.aspx?articleid=145127

9/12

10/24/13

EQEM Print

personalized instruments at STEIM, including Jon Rose,Bob Ostertag, Laetitia Sonami, Miya Masaoka, and Ka Matthews.

STEIM also features an exhibition called the Electro Squeek Club,where visitors can experience firsthand a collection of audio and videopieces. Some are completed installations, while others are indevelopment at the center. Visitors are encouraged to explore theindividual properties of each of the exhibits, which include CrackleBoxes, Babblephones, the Electronic Baby Mirror, and the BeBop Table.The artists represented include Michel Waisvisz, Bert Bongers, JorgenBrinkman, and Tom Demeyer, among many others. IRCAM

The research center in Paris known as IRCAM (Institute of Researchand Coordination in Acoustics and Music) been at the forefront ofmusic and technology for decades. Research into musical inputstructures has led to the development of several controller prototypes,including the SuperPolm and an extension of the computer mouse dubbedJerry.

This and other information has been assembled for a CD-ROM titledTrends in Gestural Control of Music, Battier and MarceloWanderly, and is available from the Electronic Music Foundation (www.cdemusic .org).Sections of the disc are dedicated to performance issues, definitionsof gesture, and gestural analysis. It a includes a roundtablediscussing the Present and Future of Gestural Control inMusic, with contributions by suc luminaries in the field asDonald Buchla, Mark Goldstein, Joel Chadabe, Tod Machover, TeresaMarrin Nakra, Robert Moog, Jean-Claude Risset, Laetitia Sonami, andMichel Waisvisz. FINAL PERSPECTIVE

It is interesting to note that many of the controllers surveyed hereare continually being refinedsome could eve be considered worksin progress. Some readers may feel that there is a gratuitous use ofnew technologies behin some of these new musical-input devices, butdeeper investigation reveals that musical results are the chiefmotivating force behind most of the controllers.

Although some of the input devices in this survey are theatrical,the ultimate goal is for the results to transcend novelty of thevisual aspects that a device presents. In most cases, the artisticsuccess of a particular controlle will depend on its transparency inthe creation of the music: the main purpose behind all of these devicesis the natural and immediate translation of physical gestures intomusic.

Some of the controllers weve examined here are based on thehighly personal goals of their developers, wherea others are createdwith mainstream applications in mind. Performers who want to developtheir own controllers ne to define the gestures they wish to use andthen find the best way to measure these gestures and translate theresults into a useful data format, such as MIDI. Sometimes a series ofgestures may require more than one t of sensor.

A number of companies have already developed the various componentsneeded in an interactive control system (such as sensors, signalconverters, and software). All that inquiring musicians, dancers, orartists need to do is spend some time configuring their own systems andlearning how to use them.

Ever since Marty Cutler started as assistant editor forEM, he has referred to his banjo as a real-time5-string arpeggiator. Armchair thereminist GinoRobair is an associate editor for EM.Beans music-making methods incl sneakinginto schools around the Bay Area with her group, RhythMix. Specialthanks to David Wessel, David Ja Donald Buchla, Alex Artaud, PeterElsie, Joel Chadabe, Mary Gallardo, Miya Masaoka, Pamela Z, Bela Fleck,B Applebaum, and Jimi Tunnell. ALTERNATIVE CONTROLLERS IN EM Reviews Big BriarEthervox Buchla andAssociates Lightning II Buchla andAssociates Thunder Cycling 74MSP Interactive LightDimension Beam Issue 05/98 08/96 08/90 10/98 07/96

www.emusician.com/prntarticle.aspx?articleid=145127

10/12

10/24/13

EQEM Print

KurzweilExpressionMate Nearfield MultimediaMarimba Lumina SynchroVoiceMIDIVox Articles BodySynth/BioMuse CNMAT/Wacomtablet/Buchla Thunder DCckoo Hydra/DimensionBeam LadysGlove MIDIBall/Interactive stageshow Soundmapping/GPS/Interactiveluggage Tech Page:Interactive Light GET WITH THE PROGRAM

05/00 06/00 07/92 Issue 09/97 10/98 02/98 11/97 06/98 05/97 06/99 12/95

One of the most popular commercially available software applicationsfor sound artists who need flexibility in mapping physical gestures toMIDI is an object-oriented programming language called Max (Mac; $395)available from Cycling 74 (www.cycling74.com). Created in 1987 by MillerPuckette at IRCAM and later developed into a commercial product byDavid Zicarelli, Max provides a graphical user interface for combiningthe basic building blocks used in an object-oriented environment. AWindows version is on the horizon.

Cycling 74s MSP lets you create, analyze, and processaudio and is designed for use with Max. The full versio of MSP ($295)requires Max 3.5.8 or higher. A free runtime version of MSP, whichcomes with the free MaxPlay application, allows you to play, but notcreate, MSP patches. PREACHING TO THE CONVERTERS A number of converters are available for translating analog signalsto digital ones.

The SensorLab (www.steim .nl/sensor.html) is a voltage-to-MIDIconverter developed at STEIM for use with any type of interactivecontroller. Although popular with many artists, the SensorLab iscurrently out of production. Ke an eye on the developers Website for further details.

The ADB I/O ($199) by Beehive Technologies (www.bzzzzzz.com)allows you to use up to eight input sensors w a Mac. You can connectup to four units, for a total of 32 I/O channels. The device acceptssensors from Inferna Devices and Infusion Systems and can work withHypercard, Supercard, AppleScript, Macromedia Director, Symantec C/C++,and Cycling 74s Max. A beta version of a Max objectcreated by David Zicarelli for the ADB can be downloaded for freefrom the Beehive downloads page.

To accommodate different types of sensors (both gestural andenvironmental), IRCAM has begun marketing its AtoMIC Pro ($665). TheAtoMIC Pro translates sensor information (electrical voltages) intoMIDI data. The devic was designed to be open-ended, so it can workwith any type of controller or signal. It has 32 analog inputs (usingtwo multipin connectors), eight digital inputs and outputs on multipinconnectors, one MIDI input, and four

www.emusician.com/prntarticle.aspx?articleid=145127

11/12

10/24/13

EQEM Print

MIDI outputs.

CNMAT has developed a low-latency, high-quality multichannelinterface for use with laptop computers in live performance situations.Code-named the Rimas Box (after its primary developer, researcher RimasAvizienis), th interface uses the 100BaseT Ethernet port tocommunicate with the computer, allowing the device to simultaneouslyhandle 64 channels of sample-synchronous control-rate gesture data, 10time-stamped MIDI I/O streams, and up to 10 channels of 24-bit audio.Latency measurements show that we can get signals into and backout of Max/ MSP in less than 7 milliseconds, says CNMAT directorDavid Wessel. One of the most important features of ourinterface is that data from sensors is treated as signals that aresynchronized at the sample level with the audio. This provides the userwith a very high degree of control intimacy. The Rimas Box, which is designed to sit neatly under a laptopcomputer, is currently being manufactured in a limited quantity forfinal beta testing. Print Page

www.emusician.com/prntarticle.aspx?articleid=145127

12/12