Beruflich Dokumente
Kultur Dokumente
BRUCE W. PENNYCOOK
Department of Music and Department of Computing and Information Science, Queen’s University, Kingston,
Ontario. Canada
This paper is a study of the unique problems posed by the use of computers by composers
and performers of music. The paper begins with a presentation of the basic concepts
involved in the musical interaction with computer devices, followed by a detailed
discussion of three musical tasks: music manuscript preparation, music language
interfaces for composition, and real-time performance interaction. Fundamental design
principles are exposed through an examination of several early computer music systems,
especially the Structured Sound Synthesis Project. A survey of numerous systems, based
on the following categories, is presented: compositions and synthesis languages, graphics
score editing, performance instruments, digital audio processing tools, and computer-
aided instruction in music systems. An extensive reference list is provided for further
study in the field.
Permission to copy without fee all or part of this material is granted provided that the copies are not made or
distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its
date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To
copy otherwise, or to republish, requires a fee and/or specific permission.
0 1985 ACM 0360-0300/85/0600-0267 $00.75
FORMAL
SEVERAL SECONDS TO SEVERAL
MUSIC
MINUTES
STRUCTURES
v
PERFORMANCE
-.05 SECONDS TO SEVERAL
EXECUTION
SECONDS
RATES
.I MILLISECONDS TO 01 SECOND
SOUND SAMPLE
-44K SAMPLES/SECOND/CHANNEL
CONVERSION
The section on technical considerations ing and printing music notations. This mul-
presents certain factors that must be sat- tiplicity of purpose is most pronounced in
isfied in real-time computing environ- recent commercial instruments such as the
ments, although many of the interfaces pre- Fairlight Computer Music Instrument
sented in this survery are software packages (manufactured by Fairlight Instruments
operating in nonreal time. Limited, Sydney, Australia).
In Section I, first the unique nature of
musical tasks as opposed to general com-
Technical Considerations
puting tasks is illustrated, and then four
innovative systems are discussed to estab- The system requirements for music inter-
lish some basic principles. faces vary according to the task and to the
Section II has been subdivided into five nature of the musical environment. In a
broad categories: composition and synthe- real-time setting the following hierarchy of
sis languages, graphics score editing tools, timing requirements exists, as shown in
real-time performance systems, digital au- Figure 1: (1) control of formal music struc-
dio processing tools, and computer-aided tures such as instrumentation, number of
instruction systems. There is a great deal output channels; (2) performance execution
of overlap within these five divisions; for from the input devices; (3) merging the
example, graphics score editing tools are performance and synthesis data into a con-
used for composing as well as for transcrib- tinuous stream and transmitting it to. the
1. MUSIC USER INTERFACE CONCEPTS l Select suitable manuscript paper for the
current instrument (considering the
Interface specifications vary dramatically number of staves, staff size, spacing be-
with respect to the five levels of communi- tween staves, etc.).
cation shown in Figure 1. This makes it l Determine if the instrument requires a
impossible to devise evaluation schemes transposed part (i.e., a part written in a
that can be generally applied to all music different key as required by transposing
interface specifications. Unlike text-editing instruments such as clarinets and French
environments, in which measures of pro- horns).
ductivity can be gathered empirically, in l Choose the correct number of measures
most musical settings productivity and aes- per page, so that page turns are preceded
thetic value become hopelessly confused. by a rest of sufficient duration.
How can we measure the effectiveness of a l Choose the corrrect number of measures
piano keyboard? Measures used in evalu- per line of music to facilitate reading and
ating text editing environments (e.g., cor- visual interpretation.
rect keystrokes per unit time [Embley and l Select the correct pen nib sizes for the
Nagey 19811) do not apply. Instead, the manuscript.
interface designer must rely on individual l Prepare the manuscript paper by adding
assessments by users performing a variety the appropriate clef, key, and time sig-
of tasks. The choice of tasks is necessarily natures.
limited by the biases inherent in the system l Copy the part from score.
and by the musical preferences of the user. l Check the transcription against the orig-
A user interface that satisfies the needs of inal part in the score and make correc-
one musician in an efficient, well-ordered tions as necessary.
damental criteria for effective music inter- important feature of the synthesis interface
face design. These first appeared in the is the generous use of defaults in the data
report Design Issues in the Foundation of specifications. Users may focus on certain
a Computer Based Tool for Music Compo- aspects of the composition process, such as
sition [Buxton 19781. This report was fol- entering, testing, and correcting the notes
lowed by Buxton [1981], Buxton et al. and rhythms, without having to be con-
[1978a, 1978b, 1979, 1980, 19821, and Fe- cerned with other parameters. This ap-
dorkow et al. [1978], all of which address proach is very helpful for the novice as well
user-interface issues to some extent. The as being convenient for the experienced
major features of the user interface are user.
summarized in the following sections.
2.2.2.1 The SSSP Conduct System.
2.2.1 SSSP Graphics Display Several unique devices and control struc-
tures have been designed and implemented
The graphics interface provides the basis by the SSSP team that can be used to
for nearly all interaction with the music control the output of the synthesizer during
system. Commands and graphics actions playback of prepared musical events. Live
such as score editing and sound synthesis performance using the SSSP Conduct sys-
specification have been organized into a tem generally involves the manipulation of
combined iconic selection and typed re- these devices in a similar fashion to Ma-
sponse menu-driven format. Since one of thew’s Groove system. As it would be highly
the primary objectives of the SSSP has impractical to move the graphics display
been to minimize the learning curve for system to each performance site, programs
musically sophisticated but computer-naive were devised that enabled the user to rap-
users, most of the dialog is system initiated idly execute commands at a standard video
and is couched in familiar musical terms display terminal from the following motion-
and symbols. sensitive input sources:
Most communications are executed
through manipulation of a cursor con- (1) two continuously variable sliders;
trolled by a tablet input device, thus reduc- (2) r-y mouse (actually a digitizing pad
ing the need for typed input. Actions by and tracker);
the user always invoke a response from (3) x-y touch-sensitive pad;
the system (unlike UNIX2 [Ritchie and (4) clavier (piano) keyboard;
Thompson 19741, e.g., where an absence of (5) four variable-slope, straight-line seg-
response from the operating system usually ments for control of output rates or
means that the input was syntactically cor- amplitude contours implemented in the
rect and that the request is being serviced). Conduct software.
The most significant feature of the graphic
interface is that all of the actions needed Other real-time user variables include
to compose and listen to a musical event simple switches, set by placing the cursor
appear as symbolically informative images. over a label and depressing the appropriate
An example of the display during a session button on the mouse, and continuously
of editing waveforms for the synthesis proc- variable parameters set by direct typing,
ess is shown in Figure 3. depressing a button to invoke the “last-
typed” value or the “default” value, or by
2.2.2 SSSP Sound Synthesis Interface dragging, whereby the cursor is placed over
Audio feedback from the SSSP is instan- a parameter field and by moving the mouse
taneous. User input and internal data are up or down vertically the current value is
organized and stored in efficient data struc- shifted appropriately.
tures so that performance specifications, The most interesting feature of Conduct
called m-events, can be interpreted directly is that the parameters that describe musical
by the synthesizer control processor and attributes such as pitch, duration, and
subsequently by the synthesizer units. An timbre selection may be arbitrarily grouped
together. Each motion detected from one
’ UNIX is a trademark of AT&T Bell Laboratories. of the many input channels can modify a
WAVEFORMS FUNCTIONS
E
u sine default -freq
Pw
2
steady -an
+ default -ind
1 PLAY
DIST.
amp5J
df -ws-obj
* WAVE SHAPING*
COMPARE
* NOTE MODE*
* SINGLE*
PITCH VOL DURATION EXIT
*WORKING OBJECTS*
Figure 3. The SSSP interactive display. (Reproduced from Buxton [1981, p. 541.)
co 16-20 A
t; (ENV.4)
W
II-15
g (ENV.3)
g (EL%
3
l-5
Figure 4. POD mask. (Reproduced 2
(ENV. 5) TIME+
from Truax [1977, p. 301.) II 80” 9d’
FREQUENCY “B
(Hz)
DENSITY
sounds /set
I
(Hz) I--
DENSITY
C 0.5 2.6
L 0.4
soundslsec. TIME-+
0” 20” 50” 60” 70” 80” 90”
19851. POD is based on an aesthetic and and values derived from musical symbols
pedagogical preference for compositional and produces a time-ordered list of para-
strategies that focus on the specification of metric data for a variety of synthesis lan-
time-varying functions, which control sto- guages. Although the input formats are
chastic distributions of several musical pa- rigid and somewhat restrictive, Score has
rameters (pitch, timbre, vertical density, been successfully used for many important
amplitude, spatial distribution). This strat- computer music compositions. There are
egy provides the composer with strong, ac- several similar music-encoding and -ma-
tive parameters that produce immediate nipulation languages modeled on Score,
auditory results, as opposed to weak, such as Scot [Good 19781 and Score11
general systems that often frustrate the [Brinkman 19811. Score and its descend-
composer with system complexities and ants use terminology that is familiar to
poor response turnaround times. Figures 4 musicians, and automate repetitive tasks.
and 5 illustrate two different sets of prob- This simplifies the work of translating
ability masks controlling sound density, musical information into data that can use
frequency, and object (sound synthesis rou- a graphics device or control a synthesis
tines) selection. device. Sample input and output files are
Score. Score [Smith 19721 is a FOR- shown in Figure 6.
TRAN programme that reads a file of codes Further discussion of music languages
<input to Score>
P3 NOTES/C4/D/E
/F/G/A/B/C5; 0.. . . . . C-major scale starting on middle C
END;
PLAY;
TOOT 0.00 .125 C F3 Fl;. parameter fields designate the instrument
name, start time in seconds, note duration
in seconds, pitch, functions.
Figure 6. Score input and output files. (Reproduced from Pennycook [1983a, p. 91.)
+ +
--- --,
- - -
z -
-- -_ -
/a---- --
-
‘;
. i
x A
-
Figure 7. Example of a Scriva screen format. (Reproduced from Buxton et al. [1979, p. 231.)
enough for performance applications. This laser-jet type device. A reproduction from
system offers several traditional music no- one of the Musprint manual pages appears
tation formats and facilities for transcrib- in Figure 9.
ing them to modern music notation. Personal Composer. Personal Composer
Musprint. Musprint [Hamel 19841 is a [Miller 19841 is a software package for the
new software package written by Keith Ha- IBM PC. It requires a Hercules’” mono-
me1 for the Apple Macintosh. The author chrome graphics card, which increases the
constructs musical symbols using the screen resolution of an IBM PC to 820 x
MacPaint program, and then (like the 640 pixels, to obtain sufficient display qual-
SSSP tools) uses a mouse to place them on ity for music notation. Of greater interest
the bit-mapped screen. Although the reso- here, though, is that this product includes
lution of the Macintosh is much less than a direct interface to MIDI (see Section 3.3)
a terminal such as the Dorado, the vertical data. Music performed on a piano-type key-
resolution of the output is a function of the board that can generate MIDI data can be
printer. Hence dot-matrix output that directly displayed as musical notation-a
shows discontinuities in oblique and curved highly desirable feature. Conversely, any of
lines appears smooth when printed on a the 32 channels of music notation can be
-5- p---SW
-3- s
--L- -ti-
.O .I .2 .3
TIME
Figure 10. Perspective plot of amplitude X harmonic number X time for a
violin tone. The fundamental harmonic is plotted in the background with higher
frequency harmonics in the foreground. (Reproduced from Grey 11975, p. 1221.)
like Bruel and Kjaar and Texas Instru- detailed study, The Role of Graphics in
ments. Computer Aided Instruction in Music
A trend toward small, powerful UNIX- [Pennycook 1983b], offers a detailed dis-
based computers has provided the means cussion of the most widely distributed
to produce specialized audio workstations packages.
with capabilities previously limited to large A new product built in Canada, the Ex-
computers. One such system, the ORFIAS cercette [Sallis 19831, offers instantaneous
32/320 [Pennycook et al. 19851, utilizes pitch detection for training in solfkge and
a Texas Instruments TMS32020 digital- voice intonation. A pitch is displayed in
signal-processing microprocessor within Common Musical Notation on a medium-
a commercially available 32-bit com- resolution screen and played by the synthe-
puter. The signal-processing power of the sis mechanism. The student responds to a
TMS32020 is enhanced by other circuitry request for a new pitch by singing into a
that provides interfaces for disks and audio microphone. The response is promptly dis-
conversion systems as well as numerous played with an indication of accuracy (cor-
oscillators. In addition to an extended rect, high, low). The user can select four
UNIX command structure, the user inter- degrees of error resolution ranging from
face offers graphics aids for signal manip- +/- one-quarter tone to “just detectable.”
ulation. The capacity of the device is currently
There is a host of commercially available being expanded to include pitch sequences
digital sound-processing products being (melodies).
used in both the recording studio and live Musicland [Lamb 19821 is a music in-
performances. These include digital delay struction system available for the Alpha
units, phasors, flangers, reverberation sys- Syntauri system. (A description of the Uni-
tems, harmonizers (which generate one or versity of Canterbury computer music sys-
more pitches in parallel at a specifiable tem in which Lamb first formulated certain
interval to the input signal), and units that aspects of his methodologies appears in
combine one or more of these effects. Man- Frykberg and Bates [1978].) The Music-
ufacturers’ attempt to provide user-friendly land system is unique in that musical con-
control panels is of particular interest. For structs are assembled by the manipulation
example, the Ursa Major Space Station, of colored boxes with a mouse input device.
manufactured by Ursa Major, Belmont, The boxes contain musical fragments that
Massachusetts, offers both a variety of re- can be presented to the student from a
verberation and delay effects, and custom- library or constructed with free-hand
built circuit boards that simulate the rever- graphics drawing routines. The interface
beration characteristics of specific rooms. allows a young person to experiment with
The front panel reduces the complex de- compositional constructs before encounter-
scription of reverberation specifications to ing the complex vocabulary and syntax of
a set of rotary knobs which control the music.
delay taps and switches for selecting pro-
grammed settings.
4. CONCLUSIONS
In this survey a number of fundamental
3.5 Computer-Aided Instruction
criteria in the design of music interfaces
in Music Systems
have been established. The success and ef-
Several entries in the computer-aided in- fectiveness of each of the implemented sys-
struction field have used interface tech- tems described here are the result of exper-
niques developed for text editing and tise and insight in both music and computer
graphics display of musical notation. engineering. Many of the interface designs
Menu-driven systems substantially reduce developed in other fields, especially docu-
the learning curve for the user, an ex- ment preparation, have contributed to mu-
tremely important consideration in the de- sic interface design by defining the man-
sign of interactive instructional software. A machine communications issues.