Playing With Lights: Music Generation Using The LED Cube

Dariusz Jackowski, Jakub Stepniewicz University of Wrocaw Institute of the Computer Science, Wrocaw, Poland

Overview
New instruments/installation based on the musical interpretation of the light intensity. The system consists of the light source in form of the LED cube and instruments which generates sound based on the measurement of the intensity of light. All devices (cube and instruments) are fully programmable. The system can be easily replicated using cheap parts.

The Instruments - Hardware Each of our instruments consists of the same building blocks - the Arduino board for main processing, one or more light sensors and speakers or buzzers. Arduino is an open-source single-board microcontroller based on the ATmega chipset. It allows us to build fully programmable instruments. Additionally we are given option to program on-board USB controller, thus changing the way (i. e. hardware type) in which the instrument is perceived by the computer. Due to the lack of the internal light source the instruments are dependent on the external one. As sensors we used simple LDRs (light dependent resistors). The instruments can be assembled by someone without previous experience with electronics. The Instruments - Software The most basic way to use our instruments is as optical theremins. The frequency of the generated tone is directly connected with the reading of the light intensity . Movement of hand in front of the sensor affects the sound in the similar way as with the theremin (only the aural effect is similar, the mechanism is completely different). Additional controllers are used for calibration, volume control and loop playing and recording. The nite state machines (FSMs) with light intensity as input and frequency of sound as output. In our experiments it proves to be most useful for generating long sounds for drones or harmony. Instead of producing one sound as output the FSM can be used to choose one of the prepared loops. Melody generated using Markov chains. As a controller for the software running on the computer. Example of Use - Installation On the cube we display some predene animations separated by segments of 3D game of life. As some of animations involves words we are able to present additional information for the audience. The instruments use as an input differences between measurements of the light intensity taken from different points in the neighbourhood of the cube, thus making the resulting sound independent form the variations of the room ambient lighting. One instrument provides the bassline with means of the nite states machine, with longer patterns associated with each state. Another one generates lead melody using Markov chain (in our case based on the analysis of European folk tunes) additional speakers can be used for lling in internal voices of harmony.

The Cube Our light source is 8x8x8 cube of LEDs with integrated programmable controller. It has the following features: capable of displaying 3D images and animations more than half thousand blue diodes used fully programmable integrated controller easy PC connection unlimited ways of customization could work either controlled by the computer or standalone The Cube - Software Non-interactive animations. They are useful both for installations and for live performance. In installation their usage allows predicting the effects or replaying installation the same way every time (it depends on the congurations of the instruments). In life performance it introduced structure in improvisation of participants. Another possible variant is to introduce human control for choosing one of previously prepared animations. 3D version of Conways Game of Life life. It has similar advantages to the animations save the fact that it tends to be chaotic at the beginning and periodic afterwards. Human-controlled (we tend to call the controlling person the conductor) movable cursor for toggling the state of diodes (we have named it the XOR Snake). The conductor is also able to set whole cube into one of predened congurations. If used with live players there can be set some additional rules for players actions in accordance to the state of cube.

Example of Use - Performance The conductor is able to play animations from predened set and at any time freeze the animation and use the XOR Snake instead. Other players use optical theremins and also allowed to change their location, use loop recording and playing abilities and hand gestures for the manipulation of sound. Additionally we introduced the following rule - the number and speed of actions of player should be related to the intensity of light on his side of the cube. For the best effect the game should take place in the dark room, which, in addition to make visual aspect of performance more striking for the audience, allows for additional special achieved by the usage f another light source - we sometimes end the performance by switching on the room lights, which causes raising of the pitch of the generated sounds by all the instruments.

Possible extensions
The simplest extensions is usage of alternative light sources in addition or instead of the cube. One of the experiments was to use few stationary optical theremins and give participants some ash lights with the regulation of light intensity. Another way to extend our system lies in programmability of all the devices - both the cube and the instruments can be used in various ways. Currently we are experimenting with some sort of feedback - the cube displays images based on the sounds which in turn are generated by the algorithm dependable on the state of the cube. Last but not least the human factor proves to be very important - most players, after the initial fascination with the novelty of the instruments and their sound, tend to explore the possibilities of the system in interesting ways .

http://www.ii.uni.wroc.pl/~yagyu/light/

dariusz.jackowski@ii.uni.wroc.pl