Ego-Jections: A “Theatrical” Reactive System of
Kinetic Structures for Augmented Multimedia
Performance
Medea Electronique, Athens, Greece, medea_electronique@yahoo.com
Keywords
Installation Interactive Robotics
System Kinetic Artificial Life
Abstract — This is a paper describing EgoJections, an
ongoing project, dealing with the research and development
of a series of kinetic structures for augmented multimedia
performance practices, in immersive, reactive environments.
The structures are interconnected in a network forming a
system of globally controlled, sensorial input processing and
actuating event processes, proposing a complex animated
artwork. The use of video, sound and light reinforce the
notion of an immersive environment, in which the presence
of the human agents merge with the machine agent network
in a cybernetic loop, presenting a unified system, able to
express artistic objectives. The implementation of the first
machine entity of the system, called “EgoJections: Try once
for better experience”, is the presented showcase here,
supporting the description of a network as such. “Try once
for better experience” took place in Electromedia Works ‘08
Athens festival. The design strategies, protocol, actuating,
and sensing technology and works, involved in presenting
the above system are explained. The autonomous behavior
of a single machine entity justifies it as a separate ‘wearable’
installation, capable of forming a smaller cybernetic loop
between the augmented man/machine entity and the human
observers, resulting in emergent behavioral tactics and
formations.
I.INTRODUCTION
Robotic art seems an emerging domain where research
and practice are connected, created from a mixture of
several fields and disciplines such as electronics,
programming, mechanics. All these disciplines combined
with some by definition, immersive media, like sound
and light, facilitate the creation of environments capable
of expressing artistic purpose. The sensorial stimulation
of evolutionary convergence of human perception,
facilitated from the use of diverse means and evolving
animated forms and functions, support the merging of the
human agents with the machine environment into an
evolved ego-space creature, forming in larger scale, a
“social”/systemic network. Conditioning the network
renders it able to express artistic purposes and at the same
time raising and process/elaborate fundamental
sociological issues/questions dealing with the
man/machine relationship and the evolutionary race of
human beings in the two last centuries.
Multimedia Immersion
Man Machine Sensing
Raising questions about the evolutionary course of
humans, their social structure and formations, in the
contexts of art whilst using the means and media of the
time, is being done by research groups and individuals for
several decades. Specifically robotics are being seen as a
new medium helping artists pushing the limits in these
pursuits. Especially the last 20 years artists like Simon
Penny, Bill Vorn, Norman White, Stelios Arcadiou (the
famous Greek/Australian Stelarc), Robotlab and many
more have been exploring these fields by using dynamic
virtual/artificial architectures to simulate organic
functions, bringing closer the living and the artificial the
organic and the machine.
Keeping as a basic framework the production of
artwork, the pursuit is being affected by advances in
research concerning Artificial Life, intelligence, genetic
algorithms, neural networks, system theory, emergence
and adaptive behaviors. Investigating at the same time
concepts such as connectionism, reactivity, behavioral
tactics (local and emergent) and immersive environments,
in order to produce aesthetics using the machine medium.
During this quest there are some basic questions that
arise, while, providing answers to these issues, formulates
very specific design strategies and architectures. One of
these questions has to do with the elements that make a
machine diversify from its own function and form,
attaining at the same time characteristics similar to those
of a living being.
Is it a mimetic approach to form or functionality of the
structure (anthropomorphic form, familiar characteristics,
speech, humanoid movement etc.)? Is it a more abstract
approach like intelligence and behavior? And if is
possible to approach a simulation of a living organism by
formulating abstract characteristics and impose it on a
structure, how is it possible for these characteristics to
become complex enough to really refer to an organic
living entity? Finally even if this happens, are the above
arguments enough to ensure that the resulting outcome is
possible to facilitate meaningful artistic purpose?
Supporting in this way the term ‘theatrical’ (this term
is borrowed from artist/researcher Bill Vorn), referring to
machine/robotic agents integrating multimedia objects to
their functionalities (metabolic processes).
II. DESIGN PROCESS
The design process was based on two parallel
procedures. One being an aesthetic approach to the actual
appearance and form of a particular machine agent,
proposed by artist and Medea Electronique founding
member Christos Laskaris, the other being a continuous
dynamic collective process taking place in Medea
Electronique’s regular brainstorming meetings, with the
purpose of answering the questions mentioned above.
Christos came with the idea of a short of “wearable”
installation, initially to be suspended from the ceiling,
functioning as a ‘head projectile’ for human agents. In
this way an intimate, peculiar, connection between the
environment/space and the human user was established.
Figure 1. Initial drawing
Figure 2. Initial drawing evolved
The above photos show the initial ‘blueprints’. The
final form of the installation is very similar to these
proposals.
Having these initial ideas to work with it was obvious
that the form of the installation itself proposed a
modified, augmented relationship between the user and
the physical space in one level and the augmented
machine/man creature and the environment, including the
rest of the human agents being present, on another.
Given the fact that this augmentation was implemented
by the technological extension/intervention, it was
sensible to consider the relationship between human
beings in a sociological manner and the relationship each
human being develops (self image) towards its self,
through technological advancements and means. The
projection of the latter in the social boundaries and
interactions, affects the processes taking place in the
former. Thus the name EgoJection was formed,
suggesting a network of technological enhanced
organisms developing interactions inside specific artistic
context.
This relationship made obvious the presence of
possible emergent phenomena concerning the
functionality of a sole machine itself. At this stage it was
also realized that however complex the movement and
functionality of one man/machine entity would be,
additional machine agents would have to be added, in
order to create a complex system enabling the research
team to analyze and control emergent behaviors, thus
creating a unpredictable but still at some level
controllable, immersive environment.
Taking into account the above discussion two different
scenarios were discussed, these scenarios are described
along with the current status of the first machine agent
created for presentation at Electromedia Works
(emw’08) festival, at the User Interaction chapter. The
next chapter describes the actual structure and the
technology used for its implementation.
III. HARDWARE SETUP
This first machine agent is a simple, interactive,
robotic structure, comprised from a sensing system, an
actuating system and a control system, functioning in
essence as an autonomous nervous system. The structure
uses video, sound and movement (current version one
degree of freedom only), to express its own status and its
understanding of the surrounding environment status.
It displays its function through two different states,
depending on whether the cybernetic loop is closed,
through the presence of a human agent at its main input
position.
The main components of the structure are:
A.Sensing Subsystem
State control of the installation is achieved through
processing the information of the sensing device in front
of the ‘cockpit’ of the machine. Presence or not of a
participant is detected and the desired control state is
initiated.
Sensing devices are also placed on the body of the
installation giving it the ability of determine whether
there are potential users in the perimeter around it.

Figure 3. Sensors arrangment
In total 8 infrared sensors are used in the current
version.
B.Control/communication Subsystem
Figure 4. Microcontroller board
Communication between the computer and all the
different sensors and actuators was implemented using a
Make Controller board. The Make Controller
(http://www.makingthings.com/products/KIT-MAKE-
CTRL/) is built around an Atmel AVR family architecture
RISC Flash memory microcontroller, the
AT91SAM7X256. The controller’s board provides
control, computation, along with CAN and Ethernet
communication.
Figure 5. Analog inputs digital outputs
The application board supports connectivity through
USB, CAN interface and Ethernet. It provides 8 analog
inputs, 8 digital outputs and 4 servo outputs. These input
and output ports are used to read values from the sensors
and control all the actuating devices.
The boards are connected on a local Ethernet network
with a 2 GHz Intel double core MacBook, running Mac
OS X version 10.4.8, controlling the actuating devices
and processing sensorial inputs, a desktop pc running
Windows Xp controlling and synchronizing the state of
the machine with the video output, and a macmini
controlling the sound parameters. This network of
computers is used for parallel processing of information
and control of the different subsystems simultaneously
and in real time. It is the global control of the installation
and able to process information coming from a system
with much bigger complexity.
C.Actuating Subsystem
The basic actuator of the machine is a 1Hp AC motor
made by Bonfiglioli mounted on the main structure with
handmade steel/copper mechanical joints.
An AC motor was used to provide sufficient direction
and speed control and still being able to handle heavy
loads and provide smooth silent movement. Other
technologies like servo or stepper motors were not used
since accuracy of positioning was not of an issue for this
setup.
D.Sound sources
As sound sources 4 loudspeakers (passive) are used
two inside the main body of the machine and 2 outside.
The pairs of speakers do not perform simultaneously, but
in relation to the current state of the system. This
arrangement is implemented using the Kyma sound
processing platform.
E.Electrical Panel
An electrical panel is used to convert PWM signals
and digital signals, from the board’s digital outputs, to a
drive signal controlling an AC, 1Hp, Bonfiglioli motor.
The main component is a Telemecaique Altivar 31
inverter, processing the signals coming from the
MakingThings board.
Figure 6. Adding components to the electrical panel the last minute is
always a drag
 In general the installation’s control panel (electrical Electronique members, using milling machines and metal
panel along with the microcontroller board) is a ‘multi- process equipment.
converter’ of OSC and MIDI messages to voltages
capable of operating different electrical, pneumatic,
mechanical components (motors, solenoids, pneumatic
pistons etc.) and assuring electrical safety during the
process.
F.Main Wooden Frame and Mechanical Joints
The body of the installation is made of wood and
paper. Stainless steel joints have been used in order to
transmit the motion of the actuator to the rest of the
structure. The front part of the installation is a projection
screen, made from a PVC panel.
A wooden boxlike frame, 2700x1400x1200 mm, has
been constructed to shape the structure and to support the
mounted projection and camera. Paper cartons have been
used to seal the frame to provide an immersive, solitary Figure 9. Assembling the steel structure
space for the main participant.

Figure 7. Wooden frame

Figure 10. Frame, steel joints, actuator

Figure 8. Wooden frame sideways

Figure 11. Dressed with paper carton

The whole frame is supported by stainless steel leg-

like structures and joined using steel and copper
components allowing it to move in the vertical direction.
All the mechanics and frames were tailor made by Medea

Figure 12. Stainless steel kinematic system
Figure 13. Ready for test
G.Video/Projection subsystem
A video screen in the front of the construction, was
used to project live video input from the installation’s
‘cockpit’, seen both from the user ‘inside’ the installation
and those from the outside.
Figure 14. ‘Cockpit’ with mounted projector and camera
A web cam mounted on the frame and a video
projector also mounted inside the structure, provided the
footage and allowed a short of communication to the
environment of the ‘internal’ structure’s processes.
II.SOFTWARE ARCHITECTURE
A.Firmware
Interfaces between the sensors and actuators and the
microcontroller interface device (Making Controller)
have been developed for the following systems:
1.Processing: Java based programming environment
for real time animated graphics synthesis and
information processing. http://www.processing.org
2.Kyma: Graphic programming environment for real-
time sound synthesis and control.
http://www.symbolicsound.com
3.VVVV: Real time modular/nodal programming
environment for video and animated graphics
processing. http://www.vvvv.org
These interfaces are implemented on the basis of a
RTOS (Real Time Operating System). This software
activates all the useful hardware components for
computation and communication. The firmware is
programmed in C programming language and includes
files to:
­Provide network functionality
­Provide basic operating system functions
­Provide OSC and USB functionality
Protocols and functions supported on the firmware are:
­TCP/IP, used to provide network operations over the
internet. Both UDP and TCP are supported
­OSC, is used as an option to connect all the
subsystems of the board
­Controller library functions, giving access to the
controllers subsystems, for example the CAN interface or
PWM devices
­Application Board library functions, providing
access to additional subsystems for hardware control. For
example the AnalogIn() functions help read any of the 8
inputs or the Motor() functions provide DC motor
control.
The RTOS uploaded on the microcontroller
communicates with Processing. Processing performs all
the parameter computations and controls the flow of
information between the software and hardware
components.
The Processing engine is also used to create a simple
virtual model of the structure and its sensing devices,
providing visual feedback in the virtual plane on the
status of these devices and in effect the status of the
interaction between human and machine agents. This is
achieved by a simple 2d representation of the structure
and sensing devices range, along with their state (active human-agents around it. The essential information are
or not), using a color code simple system to represent it provided from the seven sensors attached to its perimeter.
(red color equals active area equals presence in that
space). This allows control of the machine’s behavior
according to environmental input information processing.

Figure 16. Transition from active to idle state

In the second state the machine ”recognizes” an active

participant in front of its predefined drive position, and
starts performing its routine, always affected from the
Figure 15. Simple 2d virtual status representation diagram information of human presence around it. This
information, is given by the sensor attached at the
‘cockpit’, which is the predefined drive position.
A uniform protocol was used on the basis of UDP the
OSC (Open Sound Control) protocol. This architecture Multimedia themes are presented in accordance to the
gives a uniform “transparency” to the different software machine-man interaction status supporting the ongoing
and hardware platforms. performance.
OSC messages are exchanged throughout the networks
components and platforms allowing the parallel
processing of information and essentially unified control.
Messages are exchanged from the microcontroller board
to Processing, from Processing to VVVV, from VVVV to
Kyma and the microcontroller board, controlling the state
of the installation, in a continuous mode.
III.INTERACTION MODEL PROTOTYPE
The development goals were directed towards creating
an autonomous robotic agent, still able to perform and
function as a subcomponent inside a superimposed more
complex system if necessary. Since a system as such is
yet to be implemented, the characteristics of a single Figure 17. Active state
agent are going to be presented in this chapter.

A. Behavior:
The behavioral tactics of the machine-entity depends
mainly on the environmental information provided by the
sensing devices all around it. There are two main states,
including several sub-states in each one, reflecting the
basic behavior of the system towards its human
counterparts.
The first is an idle state, where there is no human
agent, completing the cybernetic loop with the machine,
in front of its “drive” position. In this state the machine
performs ‘by itself’, inviting the participants to join. It
shows elements of “life” through periodical animation of Figure 18. Active state multimedia
its structure, always depending on the presence or not of
 V.FUTURE WORK
Additional software development and investigation of
communication protocols, allowing the remote exchange
of data (music, video) is required. The project presented
is the first in a series of installations employing the same
structural design elements able to form a larger scale,
more complex systems presenting emergent properties
and characteristics. Larger networks are going to enable
us to explore swarm networking between remote places,
connected through Internet, exchanging cultural and
environmental elements from different
locations/cities/countries.
For the completion of different installations the
Figure 19. Transitions between active substates utilization of various OSC to voltage converters is
necessary, providing flexibility during the design process.
It is also considered essential the development of
different sensor devices, for the implementation of
different interaction scenarios. Diverse designs and
technologies allow the implementation of sensors,
varying from noticeable, personal, wearable, (single
agent monitoring), to invisible global ones, for crowd
behavior and environmental monitoring.
ACKNOWLEDGMENT
Many thanks to “Constrinox SA”, D.Giannoukakis and
A. Giannoukakis for providing means and materials used
in the installation, D.Tomaras and G.Tsotsos, helping in
the realization of the installation with their experience in
mechanical engineering and electronics.
Figure 20. P.Tsangarakis programming vvvv
Assigning character to the machine agent is of course
an illusive attribute implemented through the
connection/relation between the agents and the
performance of each subsystem. It is the artificial
organism’s performance, utilized with machine motion
and producing sound and light while functioning, that
generates the notion of behavioral structures.
The resultant behaviors in a system of machine/man
organisms as such, can vary, from peaceful to aggressive,
linear to chaotic, attractive to repulsive, individual to
flocking, according to the systems emergent
characteristics.
IV.DISCUSSION
Appropriate design and architecture, even in small
scale interaction systems can allow the creation of
complex, meta-systems able to perform structured,
identifiable behavioral strategies, through simple
interactions between the system components.
Permitting and conditioning agents to operate in
unison, allows the system to generate an integrated
output. There are key points in architecture morphology
of such systems, to facilitate multi-agent interaction. The
focus while designing these systems is not their
appearance, but the flow and processing of information.
Conditioning the unified sub-systems enables us to
express specific design and functional scenarios,
including the pursuit of valid artistic purpose.