The Relation of Timbre to Dissonance and Spectral Fusion

Robin Shields
Roger Kendall
University of California, Los Angeles
Department of Ethnomusicology

ABSTRACT
This paper describes two experiments relating dissonance and
spectral fusion to the timbre of time-variant steady-state dyads.
Listeners rated dissonance and blend levels for a set of dyads
involving fourteen interval sizes and twenty-five orchestral
combinations.
Interval size and orchestration significantly
influenced both dissonance and blend ratings. Spectral centroid
was computed and used to explain the differences across timbres.

1.

INTRODUCTION

The role of timbre in music is not clearly understood. While it is

widely recognized that timbre plays a vital role in musical
experience, theory traditionally concerns itself mainly with pitch
and duration. This paper examines the influence of timbre on the
perception of musical harmony.
Specifically, it tests the
hypothesis that timbre is significantly related to the phenomena of
dissonance and spectral fusion, or blend.

2.

BACKGROUND

Research on the acoustic properties of timbre dates to Hermann

von Hemholtzs On the Sensations of Tone as a Physiological
Basis for the Theory of Music (1885), in which he argues that tone
quality arises from the presence and strengths of partial tones in
relation to the fundamental. The importance of harmonic spectral
data in timbre classification has been confirmed by Plomp (1976)
and provides the functional paradigm for most contemporary
studies in timbre. Saldanha and Corsos (1964) study of timbre
categorization shows that both changes in harmonic content and
amplitude envelope caused statistically significant levels of
confusability. Thus, research suggests a number of factors
contribute to the sensation of timbre and hence to sound source
recognition, but it does not provide conclusive evidence of what
cues human listeners in this process.
More recent studies in timbre have centered on the effort to
discover perceptually relevant features in acoustic spectra and to
determine where the principle dimensions of the timbral
perceptual space lie. Plomp (1970) first conducted a relatively
simple scaling of instrument tones and organ stops based on multidimensional scaling techniques developed by Shepard (1962).
This was followed by Greys 1977 work, which scaled 16
instruments in a three dimensional space based on similarity and
confusability; this served as the standard mapping of timbre for
years to follow. Kendall and Carterette (1991) followed this line

ISBN 1-876346-50-7 2004 ICMPC

596

of work by constructing a mapping of dyads played on

simultaneous wind instruments. Among the most important
finding to come from this study is the significance of spectral
centroid as the primary dimension in timbre perception. This
measurement is used extensively in the analysis presented below,
and is of great utility as it provides the researcher with a
continuous, ratio scale with which to measure timbre
As with timbre, research in dissonance perception dates back to
Hemholtz (1885), who attributes dissonance to interference among
the individual components of a complex tone. He concludes that
this is a result of rapid beating of intervals formed between
different harmonics, reaching a maximum as the beating
approaches 35 Hz (Hemholtz, 1885). While so convincing as to
remain the standard theory of dissonance for over half a century,
this conclusion was ultimately overturned by Plomp and Levelt
(1965), who made the crucial link between dissonance and the
critical bandwidth. According to Plomp and Levelt, dissonance
reaches a maximum as the separation between two components of
a complex tone approaches the critical bandwidth, the necessary
separation between tones at which their activation patterns on the
basilar membrane no longer overlap and their loudness begins to
increase (Zwicker, Flottorp, and Stevens, 1957). This was
confirmed by the measurements of perceived consonance across
the octave (Kameoka and Kuriyagawa, 1969a, 1969b) and in the
numerous models of dissonance that compute roughness by
summing concurrent activity within critical bands (Terhardt, 1974
; Hutchinson and Knopoff, 1978). An interesting implication of
Plomp and Levelts findings is that dissonance levels are not
dependent on the harmonicity of the complex tones involved.
This was, in fact, confirmed by Geary (1980) who measured
dissonance levels on inharmonic tones and found that no relation
exists between harmonicity and consonance.
Research on blend is relatively scant, especially
compared to that on timbre and dissonance. Kendall and
Carterette (1991) include an experiment on blend in their study on
simultaneous timbres and find that blend closely correlates to
closeness on their multidimensional scaling: as distance between
instruments increases, they are less likely to blend and the ability
of listeners to identify each instrument increases. Sandell (1995)
performs a comprehensive study of blend in dyads of woodwinds
and finds that composite centroid, average centroid, and onset
synchrony all have significant relations to blend. Synthetic
manipulation of the centroids of the stimuli confirmed this, as the
relation held even when the distribution of spectral energy had
been systematically altered.

ICMPC8, Evanston, IL, USA

Experiment 1

This experiment investigates the effect of timbre and interval size

on subjects dissonance ratings. Stimuli were 350 dyads,
synthesized from natural recorded sounds. Five instruments
(Flute, Bb Clarinet, Trumpet, Oboe, and French Horn) were used
to create twenty-five possible orchestral combinations. Fourteen
different interval sizes were used, spanning from a minor second
to a major ninth in semitone increments. For all dyads, C4 (262
Hz) served as the lower note, while the upper voice was varied

Methods and Materials

Sounds were taken from the McGill University Music Samples
(Opolko and Wapnick, 1998); a 1.25 second steady state portion
of the tone was selected beginning at the 1.5 second mark of each
tone. A 0.1 second artificial attack and decay was added to each
tone, and the 350 dyads described above were synthesized using
summation of amplitudes from each tone. Finally, a standard
tone was synthesized using two triangle waves one octave apart.
All stimuli were equalized for loudness. The signals were first
normalized using equal loudness contours then manually adjusted
by two trained lab technicians. Subjects (N = 11) were taken from
the UCLA student body. All subjects were enrolled in classes
within the music department, although not all subjects were music
majors. One set of subjects (N = 11) performed the experiment on
the first group of stimuli with synthesized attacks.
The experiment involved rating the dissonance of each
sound from the stimulus set on a scale of 0 to 100, the endpoints
of the scale were labeled not dissonant and dissonant,
respectively. For each rating, the standard tone was first played
and then followed by one dyad from the stimulus set. Subjects
were instructed to use the standard tone as reference
corresponding to 0, or not dissonant, on the scale and to rate the
second tone accordingly. A silence of 0.25 seconds separated the
two tones, and the presentation order was randomized for each
subject. Before performing the task, subjects underwent a training
session in which the term dissonant was explained in terms
similar to Plomps (1965); they were also presented with
examples of stimuli and given opportunities to practice ratings.
Sounds were sampled in mono at 44.1 KHz with a 16-bit
sample size and were played back binaurally through a set of
Beyerdynamic stereo headphones. The experiment was designed
and implemented on a personal computer using the Music
Experiment Development System (MEDS) by Roger Kendall
(2002).

Results
Perceptual. Interval size, lower voice timbre, and upper voice
timbre significantly influenced subjects dissonance rating; all
two-way interactions also showed significance (Table 1). The
mean dissonance values by interval size are shown below (Figure
1) and closely follow the results given by Plomp (1965), and
Kameoka and Kuriyagawa (1969b). Analysis of the effects
revealed several interesting patterns within the data. First,
Newman-Keuls testing shows that significance among intervals

597

was limited to only the most consonant and dissonant intervals.

For instance, minor seconds were significant compared to perfect
fifths (p < 0.035) and octaves (p < 0.003), but not compared to
minor thirds or major sixths. Thus significant effects caused by
interval are confined to the extremes of the range.
Effect
p Value
Interval Size
0.0005
Lower Voice Timbre
0.0005
Upper Voice Timbre
0.0005
Interval Size * Lower Voice Timbre
0.013
Interval Size * Upper Voice Timbre
0.0005
Upper Voice Timbre * Lower Voice Timbre
0.010
Table 1: Significance values for main and interaction effects for
experiment one.

100
90
80

Dissonance Ratings

3.

August 3-7, 2004

70
60
50
40
30
20
10
0

10

11

12

13

14

Interval Size
(Semitones)

Figure 1: Mean dissonance ratings by interval.

Similarly, the significance of timbre, both in the upper and lower
voices, was found to be closely related to the interval size. For
instance, Newman-Keuls testing reveals that on certain intervals
(major third, perfect fifth, perfect fifth, octave) all or nearly all
levels of interaction were significant. For less consonant intervals
(minor seventh, minor sixth), only one or two levels of timbre
were significant, usually the flute and French horn. Moreover,
interactions with the upper voice had more significant levels than
those with the lower voice, suggesting that timbre of the upper
voice has the greatest perceptual salience. Further studies with
more than two voices might better define this relationship and
indicate the influence of outer versus inner voices in dissonance
ratings
Acoustical. In order to more fully explain the results given
above, we performed spectral analysis on the stimulus set and
correlated the outputs with mean dissonance ratings. Spectral
centroid, a measurement of the distribution of spectral energy
within the signal, was the first measurement studied as it has been
demonstrated to have close links to timbre perception (Kendall,

ICMPC8, Evanston, IL, USA

August 3-7, 2004

1991). Centroid values (in Hertz) were calculated for all 350
stimuli using the formula1:

Centroids were computed using a 512 point Fast Fourier

Transform (FFT). To accurately represent the variance in each
signal, centroid values were computed for ten FFT windows, five
on either side of the signals midpoint. A value of 263 Hz was
used for F1 across all dyads, as they share the common root of C4.
The linear average of these ten values was then used as the longterm average centroid, and used in the analysis described below.
Correlation of centroid values to mean dissonance ratings were
near zero (r = 0.06); however, when correlation was calculated for
each interval independently (Figure 2), it was found that levels of
correlation closely match the pattern found for mean dissonance
ratings by interval (Figure 1).
This indicates that spectral
centroid does influence dissonance judgments if the level of
dissonance is relatively high across all centroid values. Therefore,
one may assume that for consonant intervals (octave and perfect
fifth) timbre will have little or no influence on dissonance
perception, but for dissonant intervals (minor second, minor ninth)
timbre significantly influences reported dissonance levels, with
higher centroids correlating to higher dissonance levels.

0.4

(Pearson r)

The same sets of dyads used in the previous experiment

were used here; in one set attacks were synthesized while the
second set included the instruments natural attacks. Subjects
(N=11) were recruited from UCLA students taking courses in the
music department; some subjects participated in both experiments
1 and 2, but not consecutively. Subjects were instructed to rate
the blend of each dyad on a scale of 0 to 100 with endpoints
labeled twoness and oneness, respectively.
Before
performing the task, subjects were presented with an introduction
to the term blend, heard a sample spanning the range of the
stimuli, and practiced rating several dyads. The experiment was
administered on the same equipment as experiment one, also using
MEDS (Kendall, 2002).

Results
Perceptual. Interval size, timbre of the lower voice, and timbre
of the upper voice all had a significant effect on subjects blend
ratings. Additionally, all two way and three way interactions were
significant (Table 2). Mean dissonance values by interval are
given in figure 3, and mean blend ratings correlate to mean
dissonance ratings from experiment one at -0.77. One can
therefore conclude that blend and dissonance are inversely related:
Dissonant intervals are less likely to blend regardless of timbre
and intervals with a high degree of blend are less likely to be
considered dissonant.
Effect
p Value
0.0005
Interval Size
0.001
Lower Voice Timbre
0.0005
Upper Voice Timbre
0.011
Interval Size * Lower Voice Timbre
0.0005
Interval Size * Upper Voice Timbre
0.001
Upper Voice Timbre * Lower Voice Timbre
Table 2: Significance values for main and interaction effects for
experiment two.

0.6

Correlation

Methods and Materials

0.2

0.0

-0.2

-0.4

10

11

12

13

14

Interval Size
(Semitones)

Figure 2: Correlation of dissonance rating to spectral centroid.

4.

Experiment 2

This experiment investigates the effect of timbre and interval size

on perceptual blend of dyads. The stimulus set was identical to
Experiment 1, only the subject instructions and response methods
were changed.

Where C is the spectral centroid in Hertz, n is the bin number of the Fast
Fourier Transform, j is the total number of analyis bins, An is the linear
amplitude at bin n, Fn is the center frequency of bin I, and F1 is the
fundamental frequency.

598

Acoustical. Spectral centroid values calculated in experiment one

were correlated with mean blend ratings. When calculated across
all intervals, correlation values were near zero. However, when
correlations were calculated for each interval separately,
correlation values varied greatly (Figure 3). While discerning a
strong patternization is difficult, it appears that for intervals with
low blend values, correlation to spectral centroid is relatively
high, while for intervals with high blend the correlation to
centroid is comparatively low. However, notable exceptions do
exist (e.g. the octave), and it should be noted that all correlation
values are less than 0.3.

ICMPC8, Evanston, IL, USA

August 3-7, 2004

Centroid Correlation

Blend Rating

6.

Kameoka, A. and Kuriyagawa, M. (1969b). Consonance

theory part II: Consonance of complex tones and its
calculation method. Journal of the Acoustical Society of
America, 45, 1460 1471.

7.

Kendall, R. & Carterette, E. (1991). Perceptual scaling of

simultaneous wind instrument timbres. Music Perception,
8(4), 369-404.

8.

Kendall, R. (2002). Music Experiment Development

System [Computer program]. Los Angeles, CA: UCLA
Department of Ethnomusicology. (Version 2002-B-1).

9.

Plomp, R. & Levelt, W.J.M. (1965). Tonal consonance

and critical bandwidth. Journal of the Acoustical Society
of America, 38, 548-560.

10.

Plomp, R. (1970). Timbre as a multidimensional attribute

of complex tones. In R. Plomp, & G.F. Smoorenburg
(eds.), Frequency analysis and periodicity detection in
hearing (pp. 397-414). Leiden: Sijthoff.

11.

Saldanha, E. & Corso, J. (1964). Timbre cues and the

identification of musical instruments. Journal of the
Acoustical Society of America, 36(11), 2021-2026.

12.

Sandell, G. (1995) Roles for spectral centroid and other

factors in determining blended instrument pairings in
orchestration. Music Perception, 13(2), 209 246.

13.

Shepard, R. (1962). The analysis of proximities:

Multidimensional scaling with an unknown distance
function. II. Psychometrika 27, 229 237.

14.

Terhardt, E. (1974). Pitch, consonance and harmony.

Journal of the Acoustical Society of America, 55, 10611069.

15.

Zwicker, E., Flottorp, G., & Stevens, S.S. (1957). Critical

bandwidth in loudness summation.
Journal of the
Acoustical Society of America, 29, 548 557.

80

0.3

75
70
65

0.1

60
0.0

55
50

-0.1

Blend Rating

Centroid Correlation
(Pearson r)

0.2

45
40

-0.2

35
-0.3

10

11

12

13

14

30

Interval Size
(Semitones)

Figure 3: Correlation of dissonance rating to spectral centroid

5.

Conclusion

The results indicate that listeners perception of dissonance and

blend is significantly influenced by timbre and that spectral
centroid can be used to explain these differences, especially when
each interval is considered independently.
Furthermore,
dissonance ratings confirm that the results reported by Plomp
(1965) and Kameoka and Kuriyagawa (1969ab) when ratings are
made across different orchestral combinations with natural
signals.
When considered in the traditional context of timbre as
orchestration, these findings suggest that choices concerning
orchestration will influence listeners perceptions of harmonic
structures. In turn, the perceived level of blend will be influenced
by the harmonic context. More importantly, the study reveals that
interactions between timbre and harmony are substantial and
complex. This should lead to a reconsideration of timbres role in
the musical experience, with special attention to how timbre
fulfills roles generally assigned to pitch and harmony.

6.

References

1.

Hemholtz, H. (1885/1954). On the Sensations of Tone as

a Physiological Basis for the Theory of Music. (Ellis, A.J.
Trans.). New York: Dover.

2.

Geary, J.M. (1980). Consonance and dissonance of pairs

of inharmonic sounds. Journal of the Acoustical Society
of America, 67, 1785 1789.

3.

Grey, J. Multidimensional perceptual scaling of musical

timbres. (1977). Journal of the Acoustical Society of
America, 61, 1270-1277.

4.

Hutchninson, W. & Knopoff, L. (1978). The acoustic

component of Western consonance. Interface, 7, 1 29.

5.

Kameoka, A. and Kuriyagawa, M. (1969a). Consonance

theory part I: Consonance of dyads. Journal of the
Acoustical Society of America, 45, 1451 1459.

599