An Examination of Intrasubject Variability

on the Dysphonia Severity Index

*Shaheen N. Awan, †Sara A. Miesemer, and ‡Teresa A. Nicolia, *Bloomsburg, Pennsylvania, yOwings Mills, Maryland, and
zChicago, Illinois

Summary: Objectives/Hypothesis. The purpose of this study was to expand on previous literature regarding the
intrasubject variability of the Dysphonia Severity Index (DSI) by incorporating a larger sample of participants and by
examining the test-retest mean differences and intrasubject variability not only in the DSI but also within its component
measures.
Study Design. Longitudinal and within-subject design.
Methods. The DSI and component measures of jitter, maximum phonation time (MPT), lowest vocal intensity, and
highest phonational frequency (F0) were obtained from 49 normal voiced participants (21 males and 28 females) be-
tween the ages of 18–25 years. Each participant was tested in three testing sessions with an approximate 1-week interval
between each testing session.
Results. These results indicated that the mean DSI and its component measures of high F0 and MPT are quite stable
across time (ie, no significant differences across time and strong intraclass correlation coefficients [ICCs]), whereas
Bland-Altman analyses indicated that the within-subject 1-week test-retest variability on the DSI for normal voiced sub-
jects may be expected to be within ±2.27, and the 2-week test-retest variability may be expected to be within ±2.66. In
addition, weak ICC results indicated that increased variability may also be expected in the DSI component measures of
low decibel and jitter. No evidence of a test-practice effect on repeated DSI measurements was observed.
Conclusions. The findings of this study are in agreement with previous data regarding the intrasubject variability of
the DSI. Changes in DSI that exceed ±2.27–2.66 may be necessary to reflect significant changes in voice function. The
observed test-retest variability in the DSI is relatively large in relation to the initially reported DSI range of +5 (normal
voice) to 5 (severely disordered voice). Rather than the use of ±2 standard deviations (SDs) as a guideline for signif-
icant change (±2.27–2.66), DSI change of ±1.5 SDs may be clinically useful, resulting in a more liberal level of ±1.74
(within 1 week) to 2.04 (within 2 weeks) as a guideline for significant changes in voice function.
Key Words: Dysphonia Severity Index–Jitter–Maximum phonation time–High phonational frequency–Low vocal
intensity.

INTRODUCTION phonia), although the DSI is not necessarily restricted to the +5

The Dysphonia Severity Index (DSI) is a multivariate method that
provides clinicians with an objective and quantitative measure of
vocal function. Developed by Wuyts et al,1 the DSI makes use of
a combination of several voice measures that may be obtained
from the voice range profile, as well as basic aerodynamic and
acoustic analyses: the highest phonational frequency (F0 high
in Hertz), lowest intensity (in decibels [dB]), maximum phona-
tion time (MPT in seconds), and jitter (in percentage). Using
data obtained from 387 subjects (68 normal controls vs 319 voice
disordered subjects), the DSI was obtained via multiple regres-
sion analyses and consists of four weighted variables in the equa-
tion: DSI ¼ 0.13 3 MPT (s) + 0.0053 3 F0 high (Hz)  0.26 3 I
low (dB)  1.18 3 jitter (%) + 12.4. Wuyts et al1 reported an in-
verse relationship between the DSI and both grade (G: overall se-
verity) of dysphonia and the Voice Handicap Index (VHI). The
DSI was transformed such that a DSI ¼ +5 corresponded to G0
(normal voice), and a DSI ¼ 5 corresponded to G3 (severe dys-
Accepted for publication April 18, 2012.
From the *Department of Audiology & Speech Pathology, Bloomsburg University of
Pennsylvania, Bloomsburg, Pennsylvania; yGenesis Rehabilitation Services at Atrium
Village, Owings Mills, Maryland; and zEBS Healthcare, Chicago Public Schools,
Chicago, Illinois.
Address correspondence and reprint requests to Shaheen N. Awan, Department of Audi-
ology & Speech Pathology, Bloomsburg University of Pennsylvania, Centennial Hall, 400
East Second St, Bloomsburg, PA 17815-1301. E-mail: sawan@bloomu.edu
Journal of Voice, Vol. 26, No. 6, pp. 814.e21-814.e25
0892-1997/$36.00
Ó 2012 The Voice Foundation
doi:10.1016/j.jvoice.2012.04.004
to 5 range. A DSI of +1.6 was determined to be the cutoff for
perceptually normal voices. In recent years, several studies
have used the DSI to investigate the possible effects of vocal train-
ing on voice2–4; the possible influence of age and gender on the
DSI5; the possible relationship between the DSI and other voice
rating scales, such as the Grade, Roughness, Breathiness, Asthe-
nia, Strain GRBAS scale and the VHI6–8; and the possible effects
of smoking on vocal function.9
For voice professionals using the DSI, information regarding
the intrasubject variability of the measurement is needed, partic-
ularly because previous literature regarding the degree of variabil-
ity for the component measures of the DSI has been conflicting.
Several studies have examined the degree of variability for
MPT, with conflicting results reported. Studies by Shanks and
Mast10 and Speyer et al11 concluded that MPT was a reasonably
stable measure across trials and over time. In contrast, Stone12 re-
ported increases in an individual’s MPT with as many as 15 trials
and concluded that earlier trials did not elicit the patient’s true
MPT, and Lee et al13 indicated that significant variability in
MPT may be observed when vocal intensity was not being con-
trolled. For measures of jitter, Lee et al13 reported that jitter scores
were not significantly variable over time, whereas studies by
Ferrand,14 Dwire and McCauley,15 and Bough et al16 indicated
that significant variability in measures of jitter may be expected
over multiple trials and over time. For measures of vocal intensity,
Sihvo et al17 indicated that softest intensity measurements
Shaheen N. Awan, et al An Examination of Intrasubject Variability on the DSI
produced more variation than highest intensity measurements,
whereas studies by Stone and Ferch,18 Gelfer and Young,19 and
Brown et al20 indicated that vocal intensity variability was within
reasonable levels and reproducible within one standard deviation
(SD) of the mean intensity level. The final measurement, highest
frequency, has also shown to have possible excessive variation
over time, with both highest and lowest frequency measures ob-
served to vary by several semitones over multiple trials.21
Specific to the DSI, Hakkesteegt et al22 have reported on both
interobserver variability and test-retest variability. This study
initially evaluated 30 nonsmoking subjects (19 females and 11
males; mean age, 26 years) who did not complain of voice dis-
orders. The subjects were tested over three different days with
a mean of 9 days between each session for an approximate total
of 3 weeks for the 22 subjects who completed all three measure-
ments. Results indicated that interobserver variability was not
a significant factor in test-retest DSI measurements. For intra-
subject variability, the DSI was observed to increase with each
time period, with a reported change in mean DSI from measure-
ment 1–2 of +0.60 and from measurements 2–3 of +0.06. The re-
searchers concluded that slight increases in DSI over time may
have been because of a practice effect. Hakkesteegt et al22 con-
cluded that intrasubject variability only becomes significant
when the difference of the DSI within a single subject and
between two measurements is 2.49 or greater.
The DSI has shown potential to be useful as an overall measure
of vocal function. However, it is clear that the component mea-
sures of the DSI may be prone to variability that may affect the
stability of the DSI itself. The purpose of this study was to expand
on the intrasubject variability study by Hakkesteegt et al,22 by in-
corporating a larger sample of participants and by examining the
test-retest mean differences and intrasubject variability not only
in the DSI but also within its component measures.
METHODOLOGY
Participants
The methodology used in this study adhered to the basic ethical
considerations for the protection of human participants in
research and was approved by the Bloomsburg University Institu-
tional Review Board. DSI data were obtained from 49 participants
(21 males and 28 females) between the ages of 18–25 years (mean
age, 21.5 years; SD, 2.14). Subjects had perceptually normal
speaking voices and reported no significant history of voice or
laryngeal disorders. All subjects passed a hearing screening of
25 dB at 0.5, 1, 2, and 4 kHz.
Recording and analysis procedures
Each participant was tested in each of the four areas of the DSI:
jitter, highest frequency (high F0), lowest intensity (low dB),
and MPT (MPT in seconds) in three testing sessions with an ap-
proximate 1-week interval between each testing session. The
mean number of days between week 1 and 2 was 7.1 (SD,
0.60), and the mean number of days between week 1 and 3
was 14.3 (SD, 1.14). All recordings were carried out as close
as possible to the time of day at which the first testing session
was completed.
814.e22
Voice samples were recorded using a Yamaha Audiogram 6
preamplifier (Yamaha, Buena Park, CA) with a Shure SM10A
headset microphone (Shure Inc., Niles, IL) at a 2-inch mouth-
to-microphone distance. Voice samples were digitized using
the Audacity v.1.2.623 sound recording and analysis program
(Audacity Team, http://www.audacity.sourceforge.net) in 44-
KHz/16-bit format onto a Windows-compatible computer.
These recordings were subsequently analyzed for the compo-
nent DSI measures of low intensity, jitter, and highest F0 using
TF3224 (P. Milenkovic, Madison, WI) acoustic analysis soft-
ware. For measurements of jitter (%), participants were asked
to chant/count from one to four at a comfortable pitch and loud-
ness level, followed by a sustained vowel /ɑ/ for 2–3 seconds at
the same comfortable pitch and loudness level as they used in
the initial chanting/counting. This method tends to produce
a fundamental frequency similar to the habitual frequency
used in the speaking voice.25 The clinician provided the partic-
ipants with a model, and three trials were elicited. The lowest
jitter value (in percent) from the three trials was used in the
DSI analysis. For measurements of high F0, the participants
were asked to start at a comfortable pitch level and go up a semi-
tone scale (using the vowel /ɑ/) until they reached their highest
pitch level in falsetto register without losing control of their voi-
ces (no pitch or phonation breaks) and had to sustain the phona-
tion for at least 1–2 seconds. The clinician provided the
participants with a model. Three trials were elicited, and the
highest frequency (in Hertz) from the three trials (measured
using TF32) was used in the DSI analysis.
For the measurement of vocal intensity, the headset micro-
phone was calibrated using procedures described by Winholtz
and Titze26 and Asplund.27 Using a Wavetek function generator
(TekNet Electronics, Atlanta, GA) and a Realistic Minimus-1
speaker (Radio Shack, Fort Worth, TX), a 150-Hz square wave
calibration tone was recorded with the Shure SM10A microphone
placed 2 inches away from the speaker, while a sound level meter
held 12 inches away was used to measure the intensity of the cal-
ibration tone (66 dB at 12 inches). Therefore, the root mean
square (RMS) amplitude of the calibration tone recorded into
the headset microphone at 2 inches was considered to be equiv-
alent to the 66-dB tone at 12 inches. All subsequent voice
samples were recorded using the same 2 inches mouth-to-
microphone distance, and preamplifier settings. During low-
intensity elicitation, the participants were asked to hold out the
vowel /ɑ/ at their lowest pitch level without losing control of their
voice or whispering. The clinician provided the participants with
a model. Three trials were elicited and analyzed using TF32. For
each vowel, the RMS amplitude of the central 1-second portion
of the vowel was calculated and then input into a standard decibel
formula using the RMS amplitude of the previously recorded
calibration tone as the reference. The lowest intensity production
from the three trials was used in the DSI analysis.
For the elicitation of MPT, the participants were asked to
maximally inhale and then hold out the vowel /ɑ/ for as long
as possible. A model was provided for the participants. Three
trials were elicited, and the time was recorded in seconds using
a high-quality stopwatch. The longest duration MPT from the
three trials was used in the DSI analysis.
814.e23
RESULTS
A series of repeated-measures mixed-model ANOVAs (1 be-
tween: gender, two levels; 1 within: three levels of time) were
computed using SPSS v.16.028 (SPSS Inc., Chicago, IL) statis-
tical software to assess possible changes in the DSI and the
component measures of high F0, low dB, MPT, and jitter. To
protect against inflation of type I error rate, the alpha level
was set to P < 0.01 (Bonferroni adjustment ¼ 0.05/5 ANOVAs).
In the event of violations of sphericity, ANOVA results were
evaluated using Greenhouse-Geisser corrections. In addition,
single-measures intraclass correlation coefficients (ICCs)
were also calculated as indices of the reliability of the measure-
ments for a typical single subject. The ICC accounts for both the
relationship and the agreement between scores.29
Dysphonia Severity Index
ANOVA results indicated no significant main effects of time (F(2,
94) ¼ 2.70; P ¼ 0.072) or gender (F(1, 47) ¼ 3.76; P ¼ 0.059). In
addition, there was no significant interaction of time 3 gender
(F(2, 94) ¼ 3.08; P ¼ 0.051). Although the ANOVA results
were nonsignificant at the P < 0.01 level, a review of the mean
DSI data across time indicated an overall tendency for DSI to
slightly decrease over the three testing sessions. Table 1 provides
the mean DSIs and SDs across time. In addition, Table 1 also pro-
vides the single-measures ICC and associated ICC 95% confi-
dence intervals for the DSI and other component variables.
Across the measurement time points, the single-measures ICC
for the DSI (0.72) was considered to indicate moderate-to-
strong test-retest reliability.29
Jitter
ANOVA results indicated no significant main effects of time
(F(Greenhouse-Geisser: 1.65, 77.30) ¼ 2.91; P ¼ 0.07) or gen-
der (F(1, 47) ¼ 0.05; P ¼ 0.83). In addition, there was no signif-
icant interaction of time 3 gender (F(Greenhouse-Geisser:
1.65, 77.30) ¼ 0.56; P ¼ 0.54). Table 1 provides mean jitter
values and SDs across time. Across the measurement time
points, the single-measures ICC for jitter (0.61) was considered
to indicate moderate test-retest reliability.29
Highest frequency
ANOVA results indicated no significant main effect of time (F(2,
94) ¼ 0.606; P ¼ 0.55) and, as expected, a significant effect of
TABLE 1.
Mean Values for the DSI and Component Variables for All Subjects (Males and Females Combined) Across Three Testing
Weeks
Dependent Variable Week 1 Week 2
DSI 3.67 (1.56) 3.54 (1.76)
Jitter (%) 0.33 (0.17) 0.36 (0.13)
Highest frequency (Hz) 584.69 (203.71) 574.51 (174.47)
Lowest intensity (dB) 54.42 (3.23) 54.68 (3.55)
MPT (s) 20.83 (6.18) 21.04 (6.80)
Standard deviations are provided in parentheses.
* Single-measures ICC: An index for the reliability of the ratings for a typical single subject.
Journal of Voice, Vol. 26, No. 6, 2012
gender (F(1, 47) ¼ 32.5; P < 0.001) with females observed to pro-
duce a significantly higher mean maximum F0 productions than
males (673.4 [SD, 20.4] vs 443.2 Hz [SD, 20.4], respectively).
There was no significant interaction of time 3 gender (F(2,
94) ¼ 2.2; P ¼ 0.12) with the mean high frequency values
observed to be quite stable for both males (week 1: mean,
436.58 Hz; week 2: mean, 466.13 Hz; and week 3: mean,
426.95 Hz) and females (week 1: mean, 695.78 Hz; week 2:
mean, 655.79 Hz; and week 3: mean, 668.61 Hz). Table 1 pro-
vides the mean high frequency values and SDs across time col-
lapsed across both genders. Across the measurement time
points, the single-measures ICC for high frequency (0.80) was
considered to indicate strong test-retest reliability.29
Lowest intensity
ANOVA results indicated no significant main effects of time (F(2,
94) ¼ 1.86; P ¼ 0.162) or gender (F(1, 47) ¼ 3.13; P ¼ 0.083). In
addition, there was no significant interaction of time 3 gender
(F(2, 94) ¼ 3.04; P ¼ 0.053). Although the interaction was non-
significant, a review of the mean intensity data across time indi-
cated an overall tendency for intensity levels to slightly
increase over the three testing sessions. Table 1 provides the
mean low intensities and SDs across time. Across the measure-
ment time points, the single-measures ICC for low intensity
(0.45) was considered to indicate poor test-retest reliability.29
Maximum phonation time
ANOVA results indicated no significant main effects of time
(F(Greenhouse-Geisser: 1.72, 80.67) ¼ 0.77; P ¼ 0.45) or gen-
der (F(1, 47) ¼ 0.01; P ¼ 0.99). In addition, there was no signif-
icant interaction of time 3 gender (F(Greenhouse-Geisser:
1.72, 80.67) ¼ 1.36; P ¼ 0.26). Table 1 provides the mean
MPTs and SDs across time. Across the measurement time
points, the single-measures ICC for MPT (0.79) was considered
to indicate strong test-retest reliability.29
Bland-Altman plots
Figures 1 and 2 show Bland-Altman plots of the first and second
DSI measurement and the first and third DSI measurement, re-
spectively. Bland and Altman30–32 have described a method by
which the concordance/agreement between two quantitative
measures may be qualitatively assessed. In the Bland-Altman
plot, the difference between two measurements is plotted
Week 3 ICC* ICC: 95% Confidence Intervals
3.31 (1.87) 0.72 0.60–0.83
0.32 (0.14) 0.61 0.45–0.74
565.04 (199.68) 0.80 0.71–0.88
55.25 (4.09) 0.45 0.28–0.61
20.39 (6.32) 0.79 0.68–0.86
Shaheen N. Awan, et al An Examination of Intrasubject Variability on the DSI
FIGURE 1. Bland-Altman plot showing the difference between the
first and the second measures (DSI 2- DSI 1) plotted against the mean
of the first and the second measurement (mean of DSI 1 and 2).
against the mean of the two measures. Ninety-five percent con-
fidence intervals (ie, ±1.96 SDs from the mean difference) are
also provided. Bland-Altman plots allow the test user to identify
and evaluate any possible relationship between the measure-
ment differences and averages.
In Figure 1, the y-axis depicts the difference between the
first and second measurements (DSI 2  DSI 1), and the x-axis
shows the mean of DSI 1 and DSI 2. Although the median
of the DSI 2  DSI 1 differences (median DSI 2  DSI 1 differ-
ence ¼ 0.29) was somewhat lower than the mean DSI
2  DSI 1 difference (mean DSI 2  DSI 1 difference ¼ 0.13;
SD ¼ 1.16), a Shapiro-Wilk test of normality indicated that
the distribution of DSI 2  DSI 1 difference scores did not
deviate significantly from normality (Shapiro-Wilk (49) ¼ 0.95;
P ¼ 0.06). In addition, the vast majority of difference scores
(47/49 ¼ 95.92%) were observed to be within ±1.96 SDs of the
mean difference (±1.96 SD ¼ ±2.27). In Figure 2, the y-axis de-
picts the difference in DSI over a 2-week interval (DSI 3  DSI
1), and the x-axis shows the mean of DSI 1 and DSI 3. The DSI
3  DSI 1 difference scores were again normally distributed
(Shapiro-Wilk (49) ¼ 0.98; P ¼ 0.59) with a mean difference of
0.36 (SD ¼ 1.36) and the median difference of 0.35 in very
close proximity. Again, the vast majority of difference scores
(47/49 ¼ 95.92%) were observed to be within ±1.96 SDs of the
mean difference (±1.96 SD ¼ ±2.66).
FIGURE 2. Bland-Altman plot showing the difference between the
first and the third measures (DSI 3- DSI 1) plotted against the mean of
the first and the third measurement (mean of DSI 1 and 3).
814.e24
DISCUSSION
This study examined the test-retest reliability of the DSI in
a group of perceptually normal subjects participating in three
testing sessions, approximately 1 week apart. These results in-
dicated that the mean DSI and its component measures of high
F0 and MPT are quite stable across time (ie, no significant dif-
ferences across time and strong ICCs), whereas Bland-Altman
analyses indicated that the 1-week test-retest variability on the
DSI for normal voiced subjects may be expected to be within
±2.27, and the 2-week test-retest variability may be expected
to be within ±2.66. In addition, weak ICC results indicated
that increased variability may also be expected in the DSI com-
ponent measures of low dB and jitter.
Overall, the findings of this study regarding test-retest mean
DSI differences and expected variability in the DSI over time
are consistent with data reported by Hakkesteegt et al22 of ex-
pected DSI variation of ±2.49 in DSI test-retest measurements.
However, in contrast to Hakkesteegt et al,22 the results of this
study did not indicate a tendency for increased DSI scores over
time. In the present study, the mean DSI scores in this study
were observed to slightly decrease over time, although both 1-
and 2-week DSI differences were observed to be nonsignificant.
In addition, the Bland-Altman analyses showed that test-retest
difference scores (both week 2–week 1 and week 3–week
1 scores) were normally distributed, with a similar number of sub-
jects observed to increase or decrease in their DSI scores over
time. This finding differs somewhat from Hakkesteegt et al,22
where Bland-Altman analyses indicated that a substantial number
of subjects had higher DSI scores at week 3 versus week 1. Al-
though Hakkesteegt et al22 had speculated that DSI may increase
over time due to familiarity with the DSI procedures and an over-
all test-practice effect, the larger number of subjects in the present
study as compared with Hakkesteegt et al22 (N ¼ 49 vs 22, respec-
tively) may have resulted in improved sampling in DSI measure-
ments, with the overall results of the present study indicating that
DSI differences over time approximate a normal distribution.
It should be noted that the overall DSI scores found in this
study were lower on average than those found in the study by
Hakkesteegt et al.22 Although this study reports similar values
in many of the DSI components to the study by Hakkesteegt
et al,22 the main difference between the studies is in terms of
the high-frequency results, with the subjects in this study pro-
ducing lower high-frequency values than the mean high fre-
quency reported by Hakkesteegt et al.22 This study used very
specific standardized instructions that elicited a controlled
high-frequency production (ie, conditions of no phonation or
pitch breaks and 1–2 seconds in duration). Using very specific
instructions for every participant may have resulted in the lower
high-frequency values reported in this study and, therefore, the
overall decrease in DSI values as compared with the study by
Hakkesteegt et al.22 Regardless, the mean DSI values reported
for the normal voiced subjects included in this study are sub-
stantially greater than the +1.6 normal/disordered DSI cutoff
value reported by Wuyts et al.1
As indicated in several previous studies,10,11,13,20 changes in
the mean DSI component measures were observed to be
814.e25
nonsignificant across time, resulting in relatively stable mean test-
retest measures of the DSI. Although the ICC for the overall DSI
was slightly lower than that reported by Hakkesteegt et al22 (0.72
vs 0.79, respectively), the ICC results are highly comparable and
quite strong. However, as in the study by Hakkesteegt et al,22 ICC
analysis indicated that the lowest vocal intensity (low dB) and jit-
ter may be prone to weaker intrasubject reliability than the mea-
sures of MPT and highest F0. In particular, changes in low dB can
have a substantial effect on the overall DSI and has been reported
to be one of the strongest contributors to the DSI in normal voiced
subjects.9 The importance of the use of highly standardized in-
structions regarding the DSI component measures, accompanied
by clinician modeling and multiple trials, is stressed if clinically
meaningful DSI scores are to be obtained over time.
CONCLUSION
Various previous studies have shown that the DSI can be bene-
ficial in describing differences in vocal capability and discrim-
inating disordered from normal subjects. Although previous
research has shown the DSI to be beneficial, previous research
on the stability of the DSI and its component measures has been
limited. In agreement with Hakkesteegt et al,22 the results of
this study clearly indicate that mean DSI scores across subject
groups can be highly stable; however, for the individual subject,
changes in DSI that exceed ±2.27–2.66 may be necessary to re-
flect significant change in voice function. The result of ±2.27–
2.66 for significant patient change is substantial in light of the
reported range of +5 to 51 or even +8 to 8 reported by Hak-
kesteegt et al.22 This degree of expected DSI change may show
poor sensitivity to the relatively subtle changes in voice that
may occur as the severity of the patient’s voice disorder changes
over time. Hakkesteegt et al22(p89,90) had also stated that,
‘‘In more subtle voice changes it is very well possible that
a measured difference in DSI will not be significant.’’ Rather
than use of ±2 SDs as a guideline for significant change
(±2.27–2.66), DSI change of ±1.5 SDs may be clinically useful,
resulting in a more liberal level of ±1.74 (within 1 week) to 2.04
(within 2 weeks) as a guideline for significant change in voice
function. Although the mean overall DSI score and mean com-
ponent measures (ie, MPT, jitter, high frequency, and low inten-
sity) did not show significant differences over the three testing
sessions, it is suggested that detailed instructions and models
may influence stability of the intrasubject test-retest DSI and
its component measures. Future studies that investigate intra-
subject variability for the DSI in other age groups (eg, senescent
voice subjects) would also be beneficial to determine the degree
of expected DSI variability throughout the life span.
REFERENCES
1. Wuyts FL, De Bodt MS, Molenberghs G, et al. The Dysphonia Severity
Index: an objective measure of vocal quality based on a multiparameter
approach. J Speech Lang Hear Res. 2000;43:796–809.
2. Timmermans B, De Bodt M, Wuyts F, Van de Heyning P. Voice quality
change in future professional voice users after 9 months of voice training.
Eur Arch Otorhinolaryngol. 2004;261:1–5.
3. Timmermans B, De Bodt MS, Wuyts FL, Van de Heyning P. Analysis and
evaluation of a voice-training program in future professional voice users.
J Voice. 2005;19:202–210.
Journal of Voice, Vol. 26, No. 6, 2012
4. Awan SN, Ensslen AJ. A comparison of trained vs. untrained vocalists on
the Dysphonia Severity Index (DSI). J Voice. 2010;24:661–666.
5. Hakkesteegt MM, Brocaar MP, Wieringa MH, Feenstra L. Influence of age
and gender on the dysphonia severity index. A study of normative values.
Folia Phoniatr Logop. 2006;58:264–273.
6. Woisard V, Bodin S, Yardeni E, Peuch M. The Voice Handicap Index: cor-
relation between subjective patient response and quantitative assessment of
voice. J Voice. 2007;21:623–631.
7. Hakkesteegt MM, Brocaar MP, Wieringa MH, Feenstra L. The relationship
between perceptual evaluation and objective multiparametric evaluation in
dysphonia severity. J Voice. 2008;22:138–145.
8. Hakkesteegt MM, Brocaar MP, Wieringa MH. The applicability of the Dys-
phonia Severity Index and the Voice Handicap Index in evaluating effects of
voice therapy and phonosurgery. J Voice. 2010;24:199–205.
9. Awan SN. The effect of smoking on the Dysphonia Severity Index in
females. Folia Phoniatr Logop. 2011;63:65–71.
10. Shanks SJ, Mast D. Maximum duration of phonation objective tool for
assessment of voice. Percept Mot Skills. 1977;45:1315–1322.
11. Speyer R, Bogaardt HCA, Passos VL, et al. Maximum phonation time: var-
iability and reliability. J Voice. 2010;24:281–284.
12. Stone RE. Issues in clinical assessment of laryngeal function: contraindica-
tions for subscribing to maximum phonation time and optimum fundamental
frequency. In: Bless DM, Abbs JH, eds. Vocal Fold Physiology: Contempo-
rary Research and Clinical Issues. San Diego, CA: College Hill Press; 1983:
410–424.
13. Lee L, Stemple JC, Kizer M. Consistency of acoustic and aerodynamic
measures of voice production over 28 days under various testing conditions.
J Voice. 1999;13:477–483.
14. Ferrand C. Effects of practice with and without knowledge of results of
jitter and shimmer levels in normally speaking women. J Voice. 1995;9:
419–423.
15. Dwire A, McCauley R. Repeated measures of vocal fundamental frequency
perturbation obtained using the Visi-Pitch. J Voice. 1995;9:156–162.
16. Bough D, Heuer RJ, Sataloff RT, Hills JR, Cater JR. Intrasubject variability
of objective voice measures. J Voice. 1996;10:166–174.
17. Sihvo M, Laippala P, Sala E. A study of repeated measures of softest and
loudest phonations. J Voice. 2000;14:161–169.
18. Stone RE, Ferch PA. Intra-subject variability in F-SPLmin voice profiles.
J Speech Hear Disord. 1982;47:134–137.
19. Gelfer MP, Young SR. Comparisons of intensity measures and their stabil-
ity in male and female speakers. J Voice. 1997;11:178–186.
20. Brown WS, Morris RJ, DeGroot T, Murry T. Reliability of single sample
experimental designs: comfortable effort level. J Voice. 1998;12:453–459.
21. Gelfer MP. Stability in phonational frequency range. J Commun Disord.
1989;22:181–192.
22. Hakkesteegt MM, Wieringa MH, Brocaar MP, Mulder PGH, Feenstra L.
The interobserver and test-retest variability of the Dysphonia Severity
Index. Folia Phoniatr Logop. 2008;60:86–90.
23. Audacity v.1.2.6 [Computer software]. Audacity Development Team.
Available at: http://audacity.sourceforge.net/. Accessed January 2010.
24. TF32 [Computer Software]. Madison, WI: P. Milenkovic; 2002.
25. Awan SN. The Voice Diagnostic Protocol: A Practical Guide to the Diag-
nosis of Voice Disorders. Austin, TX: Pro-Ed Inc; 2001.
26. Winholtz WS, Titze IR. Conversion of a head-mounted microphone signal
into calibrated SPL units. J Voice. 1997;11:417–421.
27. Asplund A. How loud was it? A calibration system for voice recording in
clinical and research applications. Available at: http://www.clinsci.umu.
se/enheter/logopedi/personal/anders-asplund. Accessed January 2010.
28. SPSS 16.0 for Windows [Computer Program]. Chicago, IL: SPSS, Inc; 2006.
29. Portney LG, Watkins MP. Foundations of Clinical Research: Applications
to Practice. 2nd ed. Upper Saddle River, NJ: Prentice Hall Health; 2000.
30. Bland JM, Altman DG. Statistical methods for assessing agreement between
two methods of clinical measurement. Lancet. 1986;1:307–310.
31. Bland JM, Altman DG. Comparing methods of measurement: why plotting
difference against standard method is misleading. Lancet. 1995;346:
1085–1087.
32. Bland JM, Altman DG. Measuring agreement in method comparison stud-
ies. Stat Methods Med Res. 1999;8:135–160.