The European Organization for Research and Treatment
of Cancer Breast Cancer-Specific Quality-of-Life
Questionnaire Module: First Results From
a Three-Country Field Study
By Miriam A.G. Sprangers, Mogens Groenvold, Juan I. Arraras, Jack Franklin, Adrienne te Velde, Martin Muller,
Luisa Franzini, Anna Williams, Hanneke C.J.M. de Haes, Penny Hopwood, Ann Cull, and Neil K. Aaronson
Purpose: To construct a breast cancer-specific qual-
ity-o-ITfe questionnaire (QLQ) module to be used in con-
junction with the European Organization for Research
and Treatment of Cancer (EORTC) QLQ-C30 and to test
its reliability and validity cross-culturally.
Patients and Methods: Module construction took
place after the EORTC guidelines for module develop-
ment. The module-the QLQ-BR23-consists of 23 items
covering symptoms and side effects related to different
treatment modalities, body image, sexuality, and future
perspective. This module was tested in 170 Dutch, 168
Spanish, and 158 American cancer patients at two points
in time. The timing for the Dutch and Spanish patients
was before and during treatment with radiotherapy or
chemotherapy. For the American patients, the question-
naire was administered at admission at the breast clinic
and 3 months after the first assessment.
THE LONG-TERM goal of the European Organiza-
tion for Research and Treatment of Cancer
(EORTC) Study Group on Quality of Life is to develop
an integrated measurement system for evaluating the
quality of life (QL) of cancer patients participating in
international clinical trials. The Study Group recognizes
the need to reconcile two levels of application of such
QL assessments-generalizability of results across stud-
ies and sensitivity to specific research questions. This has
From the Netherlands Cancer Institute, Amsterdam; Academic
Medical Center, Amsterdam, the Netherlands; University of Copen-
hagen, Denmark; Hospital de Navarra, Pamplona,Spain; University
of Houston, Houston; University of Texas School of Public Health,
Houston, TX; Christie Hospital, Manchester; and the Western Gen-
eral Hospital, Edinburgh, United Kingdom.
Submitted October 18, 1995; accepted April 11, 1996.
Supported by grantsfrom the Dutch Cancer Society, Amsterdam; the
Netherlands (NKI 91-30 and NKI 90-A); the Department of Health of
the Government of Navarra; the University Cancer Foundation and the
Physicians Referral Service of the M.D. Anderson Cancer Center.
This research was conducted while Dr Sprangerswas on the staff
of the Netherlands CancerInstitute.
Address reprint requests to MirjamA.G. Sprangers, PhD, Depart-
ment of Medical Psychology, Academic Medical Center, University
of Amsterdam, J404, Meibergdreef 15, 1105 AZ Amsterdam, the
Netherlands; Email M.A.Sprangers@AMC. UVA.NL.
1996 by American Society of Clinical Oncology.
0732-183X/96/1410-0019$3.00/0
2756 Journal of Clinical Oncology, Vol 14, No 10 (October), 1996: pp 2756-2768
Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134
Copyright 2017 American Society of Clinical Oncology. All rights reserved.
Results: Multitrait scaling analysis confirmed the hy-
pothesized structure of four of the five scales. Cronbach's
alpha coefficients were, in general, lowest in Spain
(range; .46 to .94) and highest in the United States
(range; .70 to .91). On the basis of known-groups com-
parisons, selective scales distinguished clearly between
patients differing in disease stage, previous surgery,
performance status, and treatment modality, according
to expectation. Additionally, selective scales detected
change over time as a function of changes in perfor-
mance status and treatment-induced change.
Conclusion: These results lend support to the clinical
and cross-cultural validity of the QLQ-BR23 as a supple-
mentary questionnaire for assessing specific quality-of-
life issues relevant to patients with breast cancer.
J Clin Oncol 14:2756-2768. 1996 by American So-
ciety of Clinical Oncology.
led to the adoption of a modular measurement approach
to QL assessment whereby a core instrument-the QL
questionnaire (QLQ)-C301 (see Appendix 1)-has been
designed to cover a range of QL issues relevant to a broad
spectrum of cancer patients. The QLQ-C30 is intended
to be supplemented by additional subscales (modules) to
assess aspects of QL of particular importance to specific
patient subgroups. The combination of the QLQ-C30 and
supplemental modules will enhance the ability of the in-
strument to detect clinically meaningful differences be-
tween treatment arms and clinically important changes in
QL over time. Detailed guidelines have been established
for international collaboration in generating diagnosis-
and/or treatment-specific, supplemental questionnaires to
be used in conjunction with the core instrument. 2 Modules
related to head-and-neck cancer, colorectal cancer, esoph-
ageal cancer, and prostate cancer, have been or are being
constructed following these guidelines. An internationally
validated lung cancer-specific module is available.3
In the current study, the focus is on women with breast
cancer. In this article, the construction of a breast cancer-
specific questionnaire module is described. This module
was designed to be used in conjunction with the EORTC
QLQ-C30 for assessing the QL of breast cancer patients
participating in international clinical trials. The primary
study objectives were to test the reliability and validity
of this module among Dutch, Spanish, and American
breast cancer patients.
EORTC QLQ-BR23
THE CONSTRUCTION OF THE EORTC BREAST
CANCER MODULE
Construction
The development of a breast cancer module was a
collective effort of primarily Dutch, Danish, and Ameri-
can members of the Study Group. On the basis of discus-
sions within the Study Group, an initial decision was
reached to include a relatively broad spectrum of do-
mains: symptoms and side effects related to different
treatment modalities (ie, surgery, chemotherapy, radio-
therapy, and hormonal treatment), body image, sexuality,
and future perspective. To compile an exhaustive list of
relevant QL issues that cover the identified domains, liter-
ature searches were conducted, 6 existing questionnaires
were reviewed, 6 and interviews were held with breast
cancer patients7 and medical specialists. 6 These combined
sources resulted in a list of 26 issues.
These issues were then made into questionnaire items
according to a range of criteria. 8 For example, questions
were designed to be compatible with the response catego-
ries adopted for the QLQ-C30 (ie, from "not at all" to
"very much"). The items on body image were selected
from a 10-item scale devised by Hopwood 9 on the basis
of their content validity. The resulting, provisional ques-
tionnaire module contained 35 items.
Pretesting
This first version of the module was pretested among
Dutch (n = 23) and Danish (n = 110) breast cancer
patients, representative of the population of interest, to
identify and solve potential problems in its administra-
tion. Structured interviews were conducted with the Dutch
patients after completion of the QLQ-C30 and the provi-
sional module, while the Danish breast cancer patients
were mailed the QLQ-C30, the module, and a number of
debriefing questions.
In general, the module was well received, with the
majority of patients stating that the questionnaire items
were clear and relevant. However, 22% of the Dutch and
12% of the Danish patients found some of the sexuality
and/or body image items either too personal or too diffi-
cult to answer, although the majority of these patients did
complete these items. On the basis of such qualitative
findings and quantitative results of the Dutch and Danish
samples combined, the questionnaire was revised, ac-
cording to preset criteria. 8 The resulting module-the
QLQ-BR23-consists of 23 items.
Translation and Pilot-Testing
The module was subsequently translated from Dutch-
the language of origin, with the exception of the body
Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134
Copyright 2017 American Society of Clinical Oncology. All rights reserved.
2757
image items, which originated in English-into English
after rigorous forward-backward translation procedures,
involving three translators.2" Five items of the original
breast cancer module were further adapted on the basis
of the comments provided by the reviewers of the module
and other members of the Study Group. These changes
were made after the module was translated. The Dutch
item: "To what extent have you been sexually active"
was altered in the English version by adding "with or
without intercourse" to indicate the broad spectrum of
possible sexual activities. In the Dutch version, the four
items on breast symptoms were designed to be completed
only by patients who have had breast-conserving therapy.
To render these four items equally applicable to patients
having undergone mastectomy, the items of the English
version referred to "the area of your affected breast,"
whereas the Dutch version refers to "your affected
breast."''
Pilot-testing, aimed at identifying and solving potential
problems in the translation, took place among 13 British
patients (from Edinburgh and Manchester) and 15 Ameri-
can patients (from Houston, TX)." In the American ver-
sion, the British words "hot flushes" were changed into
the American equivalent "hot flashes." In general, pa-
tients were positive about the questionnaire, stating that
it covered relevant issues and was clear, although five
British patients found the body image and/or sexuality
items very personal. However, this did not preclude them
from completing these items. The English translation was
reviewed and approved by the Study Group, and was
subsequently translated into Spanish according to the
same iterative forward-backward translation procedures,
now involving four translators. Pilot-testing involved 17
patients from Madrid. The questionnaire was generally
well received, although three women refused to complete
the sexuality items. This Spanish version of the module
was approved by the Study Group and was used in the
current study.
Hypothesized Scale Structure of the QLQ-BR23
The English version of the QLQ-BR23 is provided in
Appendix 2. It incorporates two functional scales (body
image and sexuality) and three symptom scales (arm
symptoms, breast symptoms, and systemic therapy side
effects). Because the item on sexual enjoyment is condi-
tional on having been sexually active, and the item on
being upset by hair loss is conditional on having experi-
enced hair loss, these two items are completed by only a
subset of patients and are therefore handled singly. The
remaining single item assesses future perspective.
2758
The following sections report on the results of larger
scale testing of the reliability and validity of the QLQ-
BR23 when used among Dutch, Spanish, and American
patients with breast cancer.
METHODS
Dutch Sample
Patients. Breast cancer patients receiving either radiotherapy
after breast-conserving surgery for local/locoregional disease or che-
motherapy for distant metastases or locoregional disease were re-
cruited from the Antoni van Leeuwenhoek Hospital in Amsterdam.
The following patients were excluded: (1) those who had a life
expectancy of less than 3 months; (2) those who lacked basic profi-
ciency in Dutch; or (3) those who were participating in concurrent
QL investigations. No restrictions were made with regard to age or
performance status.
Additional study measures. Sociodemographic data included
age, marital status, and education. Clinical data included disease
stage and nature and schedule of treatment. This information was
extracted from the patients' medical records. To establish the level
of patients' performance status, a short interview was conducted
with the patients according to guidelines recommended by Schag et
al.' 2 The response patterns to the 11 interview items were rescored
to obtain a Karnofsky performance status (KPS) score." Finally, a
debriefing form was used to ascertain the time needed for completion
of the questionnaire, the degree and kind of help provided, the pres-
ence of items that were confusing, difficult to understand, or intru-
sive, and whether patients had skipped questions and, if so, the
reason(s) why.
Data collection procedure. The QLQ-C30, the breast cancer
module, and the debriefing questions were administered on two occa-
sions. Patients receiving radiotherapy completed these measures be-
fore their treatment and on the last day of the radiotherapy course.
Patients receiving chemotherapy completed the questionnaires on
the first day of the first treatment cycle and on either the first day
of the second cycle or during the second cycle, depending on the
chemotherapy schedule. The questionnaires were self-administered
either at the outpatient clinic or in the hospital. A research assistant
was present during the assessments to provide help whenever needed
and to record the problems patients may have had with completing
the questionnaire module.
Spanish Sample
Patients. Breast cancer patients receiving radiotherapy or che-
motherapy for local, locoregional, or metastatic disease were re-
cruited from the cancer ward of the Hospital de Navarra in Pamplona.
The following patients were excluded: (1) those who had a life
expectancy of less than 3 months; or (2) those who lacked basic
proficiency in Spanish. There were no further restrictions.
Additional study measures. Sociodemographic data included
age, marital status, and education. Clinical data included disease
stage and the nature and schedule of treatment. This information was
extracted from patients' medical records. Additionally, the physician
provided a KPS score on the same day as the interview.
Datacollection procedure. The QLQ-C30 and the breast cancer
module were administered at two points in time. Patients receiving
radiotherapy completed these measures on the day of the simulation,
Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134
Copyright 2017 American Society of Clinical Oncology. All rights reserved.
SPRANGERS ET AL
before radiotherapy, and on the last day of radiotherapy. Patients
receiving chemotherapy completed the questionnaires on the first
day of the first treatment cycle and on the first day of the second
cycle. The questionnaires were administered in the form of an inter-
view, while patients had the questionnaire in front of them because
this is the common mode of administering questionnaires in the
Hospital de Navarra.
American Sample
Patients. Breast cancer patients attending the M.D. Anderson
Cancer Center in Houston, TX, for regular care were recruited for
the study as part of a larger research program concerned with the
cost-effectiveness of cancer treatments. Some patients were about
to start active treatment (ie, surgery, chemotherapy, radiotherapy,
hormonal therapy, or a combination of these therapies), while others
were in follow-up. No restrictions were used with regard to patient
accrual.
Additional study measures. Information regarding age, disease
stage, and nature of treatment was extracted from the patients' medi-
cal records.
Data collection procedure. The QLQ-C30 and the breast cancer
module were administered at two points in time. The first question-
naire was given at the first visit to the center. If patients were to
receive treatment, the timing of this first administration was before
the start of the therapy. Patients may have received treatment pre-
viously elsewhere. Patients completed the second questionnaire after
3 months, irrespective of their treatment trajectory. The second ques-
tionnaire was mailed to the patients. Patients could contact the re-
searchers or a social worker in case they needed help with completion
of the questionnaire.
Statistical Analyses
A range of analyses was conducted to test empirically the hypothe-
sized scale structure of the QLQ-BR23, to establish scale reliability,
and to evaluate the validity of the questionnaire scales and single
items. Scaling analyses and reliability estimation were conducted on
the data of the first and second assessments for all three data sets.
Multitraitscaling. Multitrait scaling analysis was used to exam-
ine the extent to which the items of the questionnaire could be
combined into the hypothesized multiitem scales. The technique is
based on an examination of item-scale correlations." Evidence of
item convergent validity was defined as a correlation of 2 .40 (cor-
rected for overlap) between an item and its own scale. Item discrimi-
nant validity is indicated when an item correlates significantly higher
with its own scale (corrected for overlap) than with another scale
(referred to as scaling success).
Reliability. The internal consistency of the multiitem question-
naire scales-for the entire sample and for subgroups of patients-
was assessed by Cronbach's alpha coefficient.s Internal consistency
estimates of a magnitude of .70 were sought.' 6 Score distributions
(ie, skew, floor and ceiling effects) were examined for the multiitem
scales and single items.
Clinical validity. The clinical validity of the QLQ-BR23 was
assessed in two ways. First, the method of known-groups compari-
son'7 was used to evaluate the extent to which the QLQ-BR23 was
able to discriminate between subgroups of patients differing in clini-
cal status. The clinical parameters used to form mutually exclusive
patient subgroups for the baseline analyses included disease stage
(local/locoregional v metastatic disease) for the three samples, and
EORTC QLQ-BR23 2759

previous surgery (mastectomy v lumpectomy) and initial perfor- Table 1. Sociodemographic and Clinical Characteristics
mance status (KPS scores) for the Dutch and Spanish samples. For of the Study Samples at Baseline
the analysis of the data from the second administration, the clinical Dutch Spanish American
parameters used were on-treatment performance status (KPS score) In = 170) (n = 168) In = 1581
for the Dutch and Spanish samples, and treatment modality (radio- Characteristic No. % No. % No. %
therapy v chemotherapy) for the three samples. One-way analysis of
variance (ANOVA) was used to test for the statistical significance Age, years
(P < .01) of group differences. To examine the magnitude of such Median 51 55 52
differences, effect sizes based on standardized differences between Range 27-82 28-82 25-83
mean scores were calculated. Following Cohen,'8 effect sizes of .20, Marital status NA
.50, and .80 were considered small, medium, and large, respectively. Unmarried 16 9 16 10
Second, changes in the QLQ-BR23 scores over time were exam- Married 123 73 130 77
ined in relationship to changes in performance status (KPS) scores Divorced 13 8 3 2
(Dutch and Spanish samples) and treatment-induced change (all sam- Widowed 17 10 19 11
ples). Improvement or deterioration in performance status was de- Education NA
fined as a shift of at least one level upward or downward on the Compulsory only 110 65 136 81
KPS. Repeated-measures ANOVA was used to test for statistically Advanced vocational 45 26 13 8
significant changes in QLQ-BR23 scores over time. Only statistically University 15 9 18 11
significant (P < .01) data are reported. Disease stage
Local 63 37 89 53 76 51
RESULTS Locoregional 42 25 50 30 38 26
Metastatic 65 38 29 17 34 23
Patient Recruitment and Follow-Up Treatment
Dutch sample. Two hundred eighteen patients were None - - 50 32
Chemotherapy 94 55 112 67 68 43
asked to participate in the study. Forty-eight patients
Radiotherapy 76 45 56 33 15 9
(22%) declined for the following reasons: (1) the study Other - - 25 16
was perceived as too confronting or too burdensome (n Surgery NA
= 20); (2) perceived lack of time (n = 7); and (3) being Mastectomy 53 31 71 42
too ill (n = 5). The remaining 16 patients reported diverse Lumpectomy 117 69 97 58
KPS score NA
reasons, or their reasons for nonparticipation were not
Mean (SD) 78 (14) 93 (10)
recorded. At the second assessment point, 10 patients Range 30-100 70-100
were lost to follow-up for various reasons, including feel-
Abbreviation: NA, not available.
ing too ill and administrative failure. The average time In the American sample, 10 patients were not staged.
between the two assessments was 32 days (SD, 7.2).
Spanish sample. One hundred seventy-six patients were
asked to participate in the study. Eight patients (5%) de- median age ranging from 51 to 55 years. In general,
clined, primarily because they perceived the study as too Spanish patients had a lower level of education than their
confronting or burdensome or because they felt too nervous. Dutch counterparts. Additionally, Spanish patients gener-
At the second assessment, 28 patients were lost to follow- ally appeared to have a better health status as indicated
up. The reasons for patient drop-out and the time interval by the lower prevalence of advanced disease and higher
between the two assessments were not recorded. KPS scores.
American sample. The number of patients who were
Feasibility of the Questionnaire
asked to participate in the study and the number of patients
who declined were not registered. Data are presented for In the Dutch sample, the average time required to com-
the first 158 patients who completed the first and second plete the QLQ-BR23 in combination with the QLQ-C30
assessment. While the time interval was intended to be 3 at baseline was 9.2 minutes (SD, 4.7 minutes). One hun-
months for all patients, the time elapsed between administra- dred fifty-four patients (91%) completed the questionnaire
tions varied widely (mean, 121 days; SD, 48.1 days). without assistance. The questionnaire was administered
in interview form in four cases, while help was provided
Sociodemographic and Clinical Data in the remaining 11 cases (eg, the interviewer circled the
The sociodemographic and clinical characteristics of answers for the patient). These latter patients were older,
the 170 Dutch, 168 Spanish, and 158 American patients had a worse KPS score, and had a lower level of educa-
at baseline are listed in Table 1. The age distribution tion.
was highly comparable across the three samples, with the In general, items were well accepted and clear to the

Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134

Copyright 2017 American Society of Clinical Oncology. All rights reserved.
2760 SPRANGERS ET AL

majority of patients. The only items that elicited a number
of comments were those related to sexuality. Between
11% and 14% of the patients found one or more of these
items to be too personal. While 9% of the Dutch patients
refused to complete these items at baseline, this percent-
age was considerably lower among Spanish (1%) and
American patients (4%). The number of illogical response
patterns caused by the two conditional items (ie, enjoy-
ment of sex and being upset by hair loss) was negligible
in the Dutch and Spanish samples (where a research assis-
tant or interviewer was present). However, a considerable
number of American patients (ie, 16% to 21%) either
failed to complete these items when they should have (ie,
when the preceding item applied) or erroneously com-
pleted these items when they should not have (ie, when
the preceding item did not apply).
Scaling and Reliability Estimation
Multitrait scaling. Results of the multitrait scaling
analysis are listed in Table 2. In the Dutch sample, item-
scale correlations (corrected for overlap) exceeded the
.40 criterion for item-convergent validity for the body
image and sexual functioning scales at baseline, and for
the sexual functioning and breast symptoms scales at the
second assessment. In the Spanish sample, support for
item convergent validity was found for the body image
and sexual functioning scales at both assessment points.
The American data provided evidence for item-conver-
gent validity for the body image, sexual functioning, arm
symptoms, and breast symptoms scales at the two assess-
ments. While all of the subscales (except the one ad-
dressing sexuality) displayed some problems across the
three samples, the scale assessing systemic therapy side
effects did so consistently.
For both assessments, there were 80 (20 items X [5-
1] scales) tests of item-discriminant validity. Again, the
only scale that evidenced consistent problems was that
assessing systemic therapy side effects. For the two as-
sessments combined, scaling successes were noted in
69% of the cases (91% without the systemic therapy side
effects scale) in the Dutch sample, in 70% of the cases
(79% without the systemic therapy side effects scale) in
the Spanish sample, and in 81% of the cases (89% without
the systemic therapy side effects scale) in the United
States sample. The relatively low number of scaling errors
provided support for the hypothesized structure of four
of the five scales (with the exception of the systemic
therapy side effects scale).
Reliability. Cronbach's alpha coefficients for the
multiitem scales were, in general, lowest in the Spanish
sample (ranging from .46 to .94), and highest in the Amer-
ican sample (range, .70 to .91), with the coefficients of
the Dutch sample holding an intermediate position (range,
.57 to .89) (Table 3). While all multiitem scales in the
American sample met the .70 criterion for internal consis-
tency reliability, this was not the case for the arm symp-
toms and the systemic therapy side effects scales in the
Dutch sample and the three symptom scales in the Spanish
sample before and during treatment. No systematic differ-
ences in scale reliabilities were found for patients dif-

Table 2. Item Convergent Validity, Item Discriminant Validity, and Item Discriminant Validity Test Per Sample, First and Second Assessment
Dutch Sample Spanish Sample American Sample
Scales CON' DISt TESTt CON' DISt TESTt CON' DISt TESTt
Time 1
BI .57-.75 .00-.32 100 .44-.58 .01-.25 100 .61-.73 .00-.46 94
SX .76 .03-.33 100 .81 .01-.15 100 .77 .02-.21 100
ARM .38-.61 .03-.44 92 .26-.34 .00-.29 50 .52-.75 .03-.51 83
BR .38-.64 .02-.40 81 .23-.39 .00-.31 63 .55-.79 .01-.42 100
SYS .18-.49 .02-.38 25 .00-.42 .00-.42 54 .13-.60 .00-.48 61
Time 2
BI .30-.71 .01-.34 88 .63-.78 .07-.21 100 .72-.83 .13-.55 100
SX .77 .05-.25 100 .90 .02-.22 100 .80 .13-.24 100
ARM .28-.65 .06-.40 75 .27-.49 .01-.31 67 .60-.73 .06-.58 75
BR .49-.77 .02-.25 100 .23-.36 .01-.49 63 .45-.72 .08-.66 69
SYS .09-.47 .01-.48 32 .25-.37 .02-.41 54 .34-.63 .06-.49 71
Abbreviations: BI, body image scale (4 items); SX, sexual functioning scale (2 items); ARM, arm symptoms scale (3 items); BR, breast symptoms scale
(4 items); SYS, systemic therapy side effects scale (7 items); CON, item convergent validity; DIS, item discriminant validity; TEST, item discriminant validity
test.
'CON is the range of item-scale correlations (corrected for overlap).
tDIS is the range of correlations between an item and other scales.
tTEST is the percentage of cases in which an item correlates significantly higher with its own scale (corrected for overlap) than with other scales.

Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134

Copyright 2017 American Society of Clinical Oncology. All rights reserved.
EORTC QLQ-BR23 2761

fering in age, educational level, or performance status
(data not shown).
Score Distributions
After the scoring procedures for the QLQ-C30, all scale
and single-item scores were linearly transformed to a 0
to 100 scale. For the functional scales and single items
(ie, body image, sexuality, and future perspective), higher
scores represent a better level of functioning. For the
symptom scales and single item, a higher score represents
a higher level of symptoms.
Table 3 lists the means and SDs for the two assess-
ments, per sample. For the functioning scales and single
items, the full range of possible scores was observed
for both assessments across the three samples (data not
shown), with the sexual functioning scale in the Dutch
sample being the only exception (ie, none of the Dutch
patients endorsed the maximum score for both items).
This latter finding may be attributed, in part, to the fact
that the item "to what extent have you been sexually
active" was not altered by adding the phrase "with or
without intercourse" as was the case in the English and
Spanish versions. Score distributions were roughly sym-
metrical, with the exception of the body image and sexual
functioning scales in the Spanish sample. These latter
scales exhibited a positive and negative skew, respec-
tively (ie, more patients scoring toward maximum and
minimum levels of body image and sexual functioning,
respectively).
The score distributions of the symptom scales were
generally restricted to the lower and upper middle part
of the response scale at both assessments (ie, higher and
maximum scores were not observed). The only exceptions
were the arm and breast symptoms scales in the American
sample, which exhibited the full range of scores. The
score distribution of the single item on being upset by
hair loss covered the entire range and was symmetrical
across assessments and samples.
Clinical Validity
Known-groups comparisons. Dutch patients with
metastatic disease reported significantly (P < .001)
lower levels of sexual functioning and future perspec-
tive and higher levels of systemic therapy side effects
than patients with local or locoregional disease. While
the same pattern of mean scores was found in the Span-

Table 3. Descriptive Statistics and Scale Reliability of the QLQ-BR23

Dutch (n = 170) Spanish n = 168) American (n = 158)

Scales/Items Item No. Mean SD a Mean SD a Mean SD

Time 1
Functioning*
BI 4 84.5 20.7 .86 94.4 12.0 .69 77.0 27.5 .85
SX 2 22.0 21.0 .87 9.9 19.5 .89 23.1 24.9 .89
SE 1 56.8 25.5 - 42.2 26.3 - 55.7 34.2 -
FU 1 51.9 23.8 - 62.3 32.1 - 36.9 33.5 -
Symptomst
ARM 3 26.0 21.0 .62 15.8 15.8 .48 23.7 25.0 .75
BR 4 21.6 19.5 .74 18.3 14.8 .56 26.6 26.4 .85
SYS 7 15.4 14.3 .62 10.3 11.1 .56 16.6 15.6 .70
HU 1 28.1 38.9 - 37.0 35.1 - 21.7 36.0 -
Time 2
Functioning*
BI 4 85.4 20.5 .83 92.3 15.3 .82 63.0 33.1 .91
SX 2 22.9 22.0 .89 9.4 19.3 .94 23.9 25.4 .88
SE 1 59.1 23.6 - 50.7 21.8 - 47.1 33.8 -
FU 1 61.8 29.1 - 65.0 26.4 - 45.1 34.9 -
Symptomst
ARM 3 20.3 20.1 .61 9.6 13.9 .57 22.2 24.6 .79
BR 4 28.4 21.1 .81 18.4 14.5 .46 21.7 23.2 .81
SYS 7 23.1 16.3 .57 14.7 13.4 .59 30.4 22.9 .78
HU 1 42.5 37.4 - 34.4 36.5 - 46.2 39.5 -

Abbreviations: BI, body image; SX, sexual functioning; SE, sexual enjoyment; FU, future perspective; ARM, arm symptoms; BR, breast symptoms; SYS,
systemic therapy side effects; HU, upset by hair loss.
*Scores range from 0 to 100, with higher scores representing a higher level of functioning.
tScores range from 0 to 100, with higher scores representing a higher level of symptomatology.

Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134

Copyright 2017 American Society of Clinical Oncology. All rights reserved.
2762 SPRANGERS ET AL

ish sample, statistically significant differences were ob-
served only for the systemic therapy side effects scale
(P < .01). The effect sizes of the significant group
differences were of a medium magnitude (range, .60
to .68). None of these differences achieved statistical
significance in the American sample (data not presented
in tabular form).
Previous surgery was also used as a grouping variable
for the Dutch and Spanish samples. According to expecta-
tion, patients who had received a mastectomy reported
significantly (P < .001) poorer body image than patients
who had undergone breast-conserving surgery in both
the Dutch (effect size .85) and Spanish (effect size .60)
samples. While Dutch patients who had undergone a mas-
tectomy also reported poorer future perspective than pa-
tients who had received lumpectomy (effect size .42),
none of the other comparisons achieved statistical levels
of significance in either the Dutch or Spanish samples
(data not shown in tabular form).
Mutually exclusive subgroups of patients were formed
on the basis of their initial performance status scores
(Table 4). Different cutoff points were used for the Dutch
sample (KPS - 70 v KPS - 80) and the Spanish sample
(70 - KPS - 90 v KPS = 100) because of differences
in sample composition (ie, the KPS scores of the Spanish
patients were skewed positively). As expected, Dutch pa-
tients with a poorer performance status had significantly
(P < .001) lower levels of body image, sexual functioning
and future perspective, and a higher level of systemic
therapy side effects than those with a better performance
status. The same pattern of mean scores emerged for
these scales among Spanish patients, although significant
differences (P < .01) were found only for the systemic
therapy side effects scale. The magnitude of the effect
sizes were medium to large (range, .43 to 1.1).
When on-treatment performance status was used as the
grouping variable, statistically significant group differ-
ences (P < .001) were in the expected direction for body

Table 4. Summary of ANOVA of the QLQ-BR23 by KPS at Time I

KPS 70 KPS 80
No. of Patients Mean SD No. of Potients Mean SD F

Dutch sample
Functioning'
BI 54 77.2 23.1 112 88.1 18.5 3.28
SX 49 8.2 13.2 105 28.4 20.9 6.21
SE 14 47.6 33.9 67 58.7 23.3 1.49
FU 54 40.7 32.2 113 57.2 32.0 3.11
Symptomst
ARM 55 28.7 25.6 114 24.7 18.4 1.17
BR 27 27.2 21.0 87 19.8 18.9 1.72
SYS 54 24.6 16.4 112 10.9 10.8 6.42
HU 8 29.2 37.5 11 27.3 41.7 0.10

KPS= 70-90 KPS 100

Spanish sample
Functioning'
BI 75 93.0 11.3 93 95.5 12.5 1.84
SX 75 9.3 18.6 92 10.3 20.2 .11
SE 13 43.6 25.0 21 41.3 27.7 .06
FU 75 57.3 31.7 93 66.3 32.0 3.28
Symptomst
ARM 74 18.5 17.3 93 13.7 14.2 3.76
BR 75 21.2 16.4 93 15.9 13.0 5.59
SYS 74 12.9 12.9 93 8.2 8.9 7.64
HU 2 .0 .0 7 47.6 32.5 3.89
Abbreviations: BI, body image; SX, sexual functioning; SE, sexual enjoyment; FU, future perspective; ARM, arm symptoms; BR, breast symptoms; SYS,
systemic therapy side effects; HU, upset by hair loss.
'Scores range from 0 to 100, with higher scores representing a higher level of functioning.
'Scores range from 0 to 100, with higher scores representing a higher level of symptomatology.
tP < .01.
P < .001.
iRefers to one-way analysis of variance (ANOVA).

Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134

Copyright 2017 American Society of Clinical Oncology. All rights reserved.
EORTC QLQ-BR23
image (Dutch/Spanish), systemic therapy side effects
(Dutch/Spanish), future perspective (Dutch), and arm
symptoms (Spanish). Effect sizes ranged from .51 to .95
(data not presented in tabular form).
When treatment was used as the grouping variable,
patients receiving radiotherapy reported a significantly
higher level (P < .001) of breast symptoms (Dutch/Span-
ish), while patients receiving chemotherapy reported a
significantly higher level of systemic therapy side effects
(Dutch). The latter difference also achieved statistical sig-
nificance in the Spanish and the American samples at P
< .01. Effect sizes ranged from .44 to as high as 1.5.
Additionally, Dutch patients receiving radiotherapy re-
ported significantly better body image and future perspec-
tive than patients undergoing chemotherapy (effect sizes
.81 and .44, respectively). None of these or other compari-
sons achieved statistical levels of significance in the Span-
ish or American samples (data not shown in tabular form).
Change in QLQ-BR23 scores as a function of KPS and
treatment. Since the KPS scores of the Spanish patients
had a positive skew, with many patients receiving the
maximum score, we combined patients whose perfor-
mance status had improved (n = 14) or remained un-
changed (n = 100) and compared these with patients
whose performance status had deteriorated (n = 22). To
render the Dutch data comparable, the same classification
was used, including 104 and 46 patients, respectively.
Repeated-measures ANOVA was used to test for be-
tween-group (the two performance status subgroups) dif-
ferences over time (first and second assessment) in scores
on the QLQ-BR23. Statistically significant between-
group differences over time (in ANOVA terms, group x
time interactions) were observed only for the systemic
therapy side effects scale (P < .01) in the Dutch sample
and the body image scale (P < .001) in the Spanish
sample. In these cases, the change in mean scores was in
the same, expected direction, albeit of a different magni-
tude* (data not presented in tabular form).
Repeated-measures ANOVA was used to test for be-
tween-group (radiotherapy v chemotherapy) differences
over time (first and second assessment) in scores on the
QLQ-BR23 (Table 5). We expected patients who had
received radiotherapy to report a higher level of breast
*In the Dutch and Spanish samples, statistically significant (P <
.01) between-group differences over time were also observed for
the systemic therapy side effects scale and the body image scale,
respectively, when a different classification was used (ie, with the
patients whose KPS scores had remained unchanged as a separate,
third group).
Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134
Copyright 2017 American Society of Clinical Oncology. All rights reserved.
2763
symptoms at the second assessment than at baseline, and
patients undergoing chemotherapy to report higher levels
of systemic therapy side effects over time. As expected,
statistically significant between-group differences over
time were observed for the breast symptom scale and
the systemic therapy side effects scale in the Dutch and
Spanish samples. While the same pattern of results was
found in the American sample, the differences did not
achieve statistical significance.
DISCUSSION
We have described the construction of a breast cancer-
specific questionnaire module to be used in conjunction
with the EORTC QLQ-C30. Module construction took
place following the Study Group's guidelines, which re-
quire a high level of scientific rigor to ensure that the
resulting modules have high content validity. These
guidelines included the identification of relevant QL is-
sues, operationalization of issues into questionnaire items,
pretesting the resulting module, translating the module
into other language versions, and pilot-testing these dif-
ferent language versions. The resulting module consists
of 23 items. Because this module was devised to supple-
ment the core instrument, a number of items are relevant
for, but not entirely specific to breast cancer patients.
These items pertain to chemotherapy side effects, body
image, sexuality, and future perspective, areas that are
not or not sufficiently covered by the core instrument.
Conversely, a number of items of the core instrument
may be highly relevant for assessing the QL of breast
cancer patients participating in clinical trials (eg, items
assessing chemotherapy or radiotherapy side effects). The
QLQ-BR23 can therefore not be recommended as a free-
standing instrument for assessing the QL of breast cancer
patients, but rather, should be administered in conjunction
with the core questionnaire.
To date, the cancer-specific QL questionnaires used
most frequently in clinical trials among breast cancer pa-
tients include the Rotterdam Symptom Checklist
(RSCL),' 9 the Cancer Rehabilitation Evaluation System
Short Form (CARES-SF), 20 and the Functional Living
Index-Cancer (FLIC). 2' These questionnaires were de-
signed to be applicable to a broad spectrum of cancer
patients. The only QL questionnaires that have been de-
vised specifically for breast cancer patients include the
Breast Cancer Chemotherapy Questionnaire (BCQ) 22 and
the Functional Assessment of Cancer Therapy-Breast
(FACT-B). 23 While the BCQ is intended to be applicable
to stage II breast cancer patients undergoing chemother-
apy, the FACT-B, like the QLQ-BR23, was designed for
2764 SPRANGERS ET AL

Table 5. Repeated-Measures ANOVA to Test for Group* x Timet Interactions

Time 1 Time 2
Scale Treatment No. of Patients M SD M SD P
Dutch sample:
Breast Radiotherapy 73 22.6 18.4 37.1 18.4 .00
Chemotherapy 34 16.9 16.6 12.5 15.7
Systemic Radiotherapy 71 12.7 13.3 13.7 12.5 .00
Chemotherapy 82 16.5 14.7 30.8 14.9
Spanish sample
Breast Radiotherapy 41 18.1 13.3 26.4 13.7 .00
Chemotherapy 95 19.6 16.0 14.9 13.5
Systemic Radiotherapy 41 09.5 10.1 10.1 10.4 .01
Chemotherapy 95 10.1 10.9 16.6 14.1
American sample
Breast Radiotherapy 14 21.4 28.9 29.2 23.1 .04
Chemotherapy 60 34.4 26.4 24.8 25.7
Systemic Radiotherapy 12 17.5 17.6 25.0 18.3 .04
Chemotherapy 47 18.0 15.1 41.3 21.8
*Radiotherapy versus chemotherapy.
tFirst and second assessment.
tThe relatively large number of missing cases in the Dutch sample is due to missing values of individual items. At this stage of the module development
process, missing values were not replaced.
American patients who received either radiotherapy or chemotherapy were included. Since the majority of the American patients received combination
therapy, the number of patients included in the current analysis is relatively low.

use among breast cancer patients with a range of disease
stages and undergoing different treatments. In comparison
to the QLQ-BR23, the FACT-B is shorter (42 v 53 items,
including the core instruments) and covers fewer symp-
toms and treatment-related side effects. None of these
cancer-specific or breast cancer-specific QL question-
naires were originally constructed for use in international
research settings. While the cancer-specific question-
naires have been translated and validated in a number of
languages, the breast cancer-specific questionnaires have
not yet been validated cross-culturally.
In large-scale testing among Dutch, Spanish, and
American patients with breast cancer, the QLQ-BR23 was
found to be feasible and was generally well accepted by
patients. On average, it required 9 minutes to complete
both the QLQ-C30 and the QLQ-BR23 and, in most cases,
it could be filled out by the patients themselves with little
or no assistance (Dutch data only). However, as was noted
during the pretesting and pilot-testing phases, a number
of women (range, 1% to 9%) found the questions relating
to sexuality too intrusive and chose not to complete them.
The majority of these women stated that sexuality was
irrelevant for them as they were generally older and/or
had been widowed for a long period of time. However,
none of the patients refused to complete the entire ques-
tionnaire. This confirms the general notion that missing
data can be expected when sensitive areas are addressed
in a questionnaire. However, at the same time, the results
indicate that probing about sexuality need not be avoided
as long as patients are left free to leave such questions
unanswered. The two conditional items (ie, sexual enjoy-
ment and being upset by hair loss) caused some confusion
among the American patients who completed the ques-
tionnaire entirely on their own either at the outpatient
department or via mail. This finding illustrates the advan-
tage of having a research assistant available to explain the
purpose of the questionnaire, to provide help whenever
needed, and to check the questionnaire for missing data.2 4
Regarding the module's psychometric properties, the
multitrait scaling analysis confirmed the hypothesized
scale structure of the QLQ-BR23, with the exception of
the systemic therapy side effects scale, which evidenced
consistent problems across the three samples. While all
multiitem scales in the American sample met the minimal
standards set for internal consistency reliability at both
assessments, this was not the case for the arm symptoms
and the systemic therapy side effects scales in the Dutch
sample, and the three symptom scales in the Spanish
sample before and during treatment. Scale reliability was
generally lowest in the Spanish sample. This result may,
in part, be attributed to the homogeneity of the Spanish
sample. The majority of these patients appeared to be in
relatively good health, as indicated by the high prevalence
of local or locoregional disease stages and the relatively

Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134

Copyright 2017 American Society of Clinical Oncology. All rights reserved.
EORTC QLQ-BR23
high mean level of KPS scores compared to their Dutch
counterparts. However, because the KPS score of the
Spanish patients was obtained from their physicians while
the KPS score in the Dutch sample was interviewer based,
these scores are not entirely comparable.
The validity of the QLQ-BR23 was evidenced by its
ability to discriminate between subgroups of patients
known to differ in clinical status. According to expecta-
tion, selective scales distinguished clearly between pa-
tients differing in disease stage, previous surgery, perfor-
mance status, and treatment modality. Despite its
relatively weak scale structure and relatively low reliabil-
ity, the systemic therapy side effects scale, in particular,
was able to discriminate between patient subgroups and,
in general, yielded large effect sizes. The ability of the
QLQ-BR23 to reflect changes in patients' KPS scores
over time was limited to the systemic therapy side effects
scale in the Dutch sample and the body image scale in
the Spanish sample. As expected, the breast symptom
scale and the systemic therapy side effects scale were
responsive to treatment-induced changes. Cross-cultural
validity was evidenced by the similarity of the results
across the three samples.
It is not surprising that the systemic therapy side effects
scale is the weakest scale from a classic psychometric
perspective. In fact, the use of such a scale can also
be disputed on clinical grounds because the side effects
involved are not necessarily expected to occur together.
In the absence of a clear-cut rationale for grouping these
items, we explored a number of alternative combinations,
also including appropriate symptoms from the QLQ-C30.
Regardless of how the items were grouped, the reliability
was relatively low, while the validity-in terms of
known-groups comparisons and responsiveness--was
adequate. In other words, the systemic therapy side effects
scale was able to discriminate between patient subgroups
and to detect change over time, despite suboptimal inter-
nal consistency. Guyatt et al25 26' have argued that respon-
siveness, and not reliability, is the essential psychometric
property of an instrument intended to assess treatment-
induced change. While their technical stance has not re-
mained undisputed,2 7 we stand by our decision to combine
these items into one scale. Given the heterogeneity inher-
ent in such a symptom scale, it is unclear if efforts to
improve its internal consistency (eg, by means of item
bias analyses 28 ) would yield large gains in validity. How-
ever, an alternative way to handle these items is to analyze
them at an individual level.
The relatively large number of scales and/or individual
items generated by the core instrument and the breast can-
Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134
Copyright 2017 American Society of Clinical Oncology. All rights reserved.
2765
cer module combined has the advantage of increasing the
level of informational detail, but unfortunately, also the
number of outcome parameters. In hypothesis testing, this
may result in multiple-testing problems where differences
in mean scores between trial arms may achieve statistical
levels of significance on the basis of chance alone. To limit
the number of such outcome parameters, the EORTC Study
Group on Quality of Life is currently developing higher-
order summary scores for both the core instrument and
its supplemental modules. Until such summary scores are
available, we recommend that investigators identify a priori
a limited set of the questionnaire subscales and single items
considered to be of primary interest. The statistical analysis
should then be focused primarily on this subset, while the
remaining scales and single items can be analyzed in a
more exploratory manner.
Taken together, these results lend support to the clinical
and cross-cultural validity of the QLQ-BR23. The combi-
nation of the QLQ-C30 and QLQ-BR23 will enhance the
ability to detect clinically meaningful differences between
treatment arms and clinically important changes in QL
of breast cancer patients over time. However, at the same
time, one should be aware of the limitations of the current
investigation. The three data sets differed in sample com-
position, the assessment schedule, the data collection pro-
cedure, and the version of the QLQ-BR23 used (with a
slightly different version being used in the Dutch sample
compared with the version used in the Spanish and Ameri-
can samples). The responsiveness of the QLQ-BR23 over
time could not be demonstrated among the American
patients. This finding may be attributed to the use of a
convenience sample where the number of patients declin-
ing participation remained unknown, and to the high de-
gree of sample heterogeneity with respect to the treat-
ments received. Additionally, the range of follow-up at
the second assessment was large, and information on pa-
tients' health or treatment status was not available. To
address these shortcomings, the EORTC QL Study Group
has launched an international study involving more than
16 countries to test the psychometric and cross-cultural
validity and reliability of the QLQ-BR23 by using the
same version of the questionnaire (ie, the version used
among the Spanish and American patients; see Appendix
2) and a common study design (ie, in terms of sample
criteria, timing of assessments, and mode of administra-
tion) across countries. In addition to this psychometric
study organized within the EORTC Study Group, the
module is currently being used in selective phase III clini-
cal trials. The data generated by these studies will provide
further evidence of the module's suitability for use in
international cancer clinical trials.
2766 SPRANGERS ET AL

ACKNOWLEDGMENT study, Juan Pedro Arbizu and colleagues of the Oncology Department
of the Hospital de Navarra for their help in conducting the Spanish
We thank Aafke Buitelaar, Liesbeth Abbink, and Irma Kruyver of study, and Linda Greer and Rebecca Sanchez from M.D. Anderson
the Netherlands Cancer Institute for their contribution to the Dutch Cancer Center for their contribution to the American study.

Appendix 1: The EORTC QLQ-C30 (Version 2.0)*

We are interested in some things about you and your health. Please answer all of the questions yourself by circling the number that best
applies to you. There are no "right' or "wrong" answers. The information that you provide will remain strictly confidential.
Please fill in your initials:
Your birthdate (Day, Month, Year):
Today's date (Day, Month, Year):
No Yes
1. Do you have any trouble doing strenuous activities, like carrying a heavy shopping bag or a suitcase? 1 2
2. Do you have any trouble taking a long walk? 1 2
3. Do you have any trouble taking a short walk outside of the house? 1 2
4. Do you have to stay in a bed or a chair for most of the day? 1 2
5. Do you need help with eating, dressing, washing yourself or using the toilet? 1 2
Not at A Quite Very
During the past week: All Little a Bit Much
6. Were you limited in doing either your work or other daily activities? 1 2 3 4
7. Were you limited in pursuing your hobbies or other leisure time activities? 1 2 3 4
8. Were you short of breath? 1 2 3 4
9. Have you had pain? 1 2 3 4
10. Did you need to rest? 1 2 3 4
11. Have you had trouble sleeping? 1 2 3 4
12. Have you felt weak? 1 2 3 4
13. Have you lacked appetite? 1 2 3 4
14. Have you felt nauseated? 1 2 3 4
15. Have you vomited? 1 2 3 4
16. Have you been constipated? 1 2 3 4
17. Have you had diarrhea? 1 2 3 4
18. Were you tired? 1 2 3 4
19. Did pain interfere with your daily activities? 1 2 3 4
20. Have you had difficulty in concentrating on things, like reading a newspaper
or watching television? 1 2 3 4
21. Did you feel tense? 1 2 3 4
22. Did you worry? 1 2 3 4
23. Did you feel irritable? 1 2 3 4
24. Did you feel depressed? 1 2 3 4
25. Have you had difficulty remembering things? 1 2 3 4
26. Has your physical condition or medical treatment interfered with your family
life? 1 2 3 4
27. Has your physical condition or medical treatment interfered with your social
activities? 1 2 3 4
28. Has your physical condition or medical treatment caused you financial
difficulties? 1 2 3 4
For the following questions please circle the number between 1 and 7 that best applies to you
29. How would you rate your overall health during the past week?
1 2 3 4 5 6 7
Very poor Excellent
30. How would you rate your overall quality of life during the past week?
1 2 3 4 5 6 7
Very poor Excellent
*The EORTC QLQ-C30 is a copyrighted instrument. It is currently available in Bulgarian, Chinese, Czech, Danish, Dutch, English, Finnish, French,
German, Greek, Hungarian, Italian, Japanese, Norwegian, Polish, Portuguese, Russian, Slovak, Slovanian, Spanish, Swedish, and Turkish. Requests for
permission to use the instrument and for scoring instructions should be sent to Gwendoline Kiebert, PhD, Head of the Quality of Life Unit, The EORTC Data
Center, Avenue EMounier 83 Bte 11, 1200 Brussels, Belgium.

Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134

Copyright 2017 American Society of Clinical Oncology. All rights reserved.
EORTC QLQ-BR23 2767

Appendix 2: The EORTC QLQ-BR23*

Patients sometimes report that they have the following symptoms or problems. Please indicate the extent to which you have experienced
these symptoms or problems during the past week.
Not at A Quite Very
During the past week: All Little a Bit Much
31. Did you have a dry mouth? 1 2 3 4
32. Did food and drink taste different than usual? 1 2 3 4
33. Were your eyes painful, irritated or watery? 1 2 3 4
34. Have you lost any hair? 1 2 3 4
35. Answer this question only if you had any hair loss: Were you upset by the
loss of your hair? 1 2 3 4
36. Did you feel ill or unwell? 1 2 3 4
37. Did you have hot flushes? 1 2 3 4
38. Did you have headaches? 1 2 3 4
39. Have you felt physically less attractive as a result of your disease or
treatment? 1 2 3 4
40. Have you been feeling less feminine as a result of your disease or treatment? 1 2 3 4
41. Did you find it difficult to look at yourself naked? 1 2 3 4
42. Have you been dissatisfied with your body? 1 2 3 4
43. Were you worried about your health in the future? 1 2 3 4
Not at A Quite Very
During the past four weeks: All Little a Bit Much
44. To what extent were you interested in sex? 1 2 3 4
45. To what extent were you sexually active? (with or without intercourse) 1 2 3 4
46. Answer this question only if you have been sexually active: To what extent
was sex enjoyable for you? 1 2 3 4
Not at A Quite Very
During the past week: All Little a Bit Much
47. Did you have any pain in your arm or shoulder? 1 2 3 4
48. Did you have a swollen arm or hand? 1 2 3 4
49. Was it difficult to raise your arm or to move it sideways? 1 2 3 4
50. Have you had any pain in the area of your affected breast? 1 2 3 4
51. Was the area of your affected breast swollen? 1 2 3 4
52. Was the area of your affected breast oversensitive? 1 2 3 4
53. Have you had skin problems on or in the area of your affected breast (e.g.,
itchy, dry, flaky)? 1 2 3 4
The EORTC QLQ-BR23 is a copyrighted instrument. It is currently available in Bulgarian, Danish, Dutch, English, French, German, Greek, Hungarian,
Italian, Norwegian, Polish, Portuguese, Spanish, Swedish, and Turkish. Requests for permission to use the instrument and for scoring instructions should
be sent to Gwendoline Kiebert, PhD, Head of the Quality of Life Unit, The EORTC Data Center, Avenue E Mounier 83 Bte 11, 1200 Brussels, Belgium.

REFERENCES

1. Aaronson NK, Ahmedzai S, Bergman B. et al: The EORTC
QLQ-C30: A quality of life instrument for use in international clini-
cal trials in oncology. J Natl Cancer Inst 85:365-376, 1993
2. Sprangers MAG, Cull A, Bjordal K, et al: The European Orga-
nization for Research and Treatment of Cancer approach to quality
of life assessment: Guidelines for developing questionnaire modules.
Qual Life Res 2:287-295, 1993
3. Bergman B, Aaronson NK, Ahmedzai S, et al: The EORTC
QLQ-LC13: A modular supplement to the EORTC core quality of
life questionnaire (QLQ-C30) for use in lung cancer clinical trials.
Eur J Cancer 30A:635-642, 1994
4. Haes de JCJM: Onderzoek naar de gevolgen van het bevol-
kingsonderzoek naar borstkanker voor de kwaliteit van leven in
de Koning et al (eds): De Kosten en Effecten van Bevolking-
sonderzoek naar Borstkanker. Erasmus Universiteit Rotterdam,
1990
5. Groenvold M: The quality of life with adjuvant therapy for
cancer of the breast. Ugeskr Laeger 153:1185-1189, 1991
6. Sprangers MAG, Groenvold M, Velde te A, et al: The Con-
struction of a Breast Cancer Module. Amsterdam/Copenhagen/Man-
chester/Edinburgh, Internal report of the EORTC Study Group on
Quality of Life, 1992
7. Groenvold M: Quality of life in breast cancer adjuvant therapy:
What should be measured? Content validation of a questionnaire
using qualitative interviews. Presented at the 6th European Confer-
ence on Clinical Oncology and Cancer Nursing, Florence, Italy,
October 27-31, 1991
8. Sprangers MAG, Cull A: Guidelines For Module Development.
Amsterdam/Edinburgh: Internal report of the EORTC Study Group
on Quality of Life, 1992, revised version, 1993
9. Hopwood P: Body image. Internal report of the EORTC Study
Group on Quality of Life, 1992

Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134

Copyright 2017 American Society of Clinical Oncology. All rights reserved.
2768
10. Cull A, Sprangers MAG, Aaronson NK: EORTC Quality of
Life Study Group Translation Procedure. Edinburgh/Amsterdam, In-
ternal report of the EORTC Study Group on Quality of Life, 1993
11. Sprangers M, Cull A, Hopwood P, et al: The translation of
the Dutch breast cancer module into English. Amsterdam/Edinburgh/
Manchester, Internal report of the EORTC Study Group on Quality
of Life, 1993
12. Schag CC, Heinrich RL, Ganz PA: Kamofsky performance
status revisited: Reliability, validity, and guidelines. J Clin Oncol
2:187-193, 1984
13. Karnofsky DA, Burchenal JH: The clinical evaluation of che-
motherapeutic agents in cancer, in MacLeod CM (ed): Evaluation
of Chemotherapeutic Agents. New York, NY, Columbia University,
1949, pp 199-205
14. Hays RD, Hayashi T, Carson S, et al: User's Guide for the
Multitrait Analysis Program (MAP). Santa Monica, CA, Rand Cor-
poration, 1988
15. Cronbach LJ: Coefficient alpha and the internal structure of
tests. Psychometrika 16:297, 1951
16. Nunnally JC, Bernstein IH: Psychometric Theory. New York,
NY, McGraw-Hill, 1994
17. Kerlinger FN: Foundations of Behavioral Research. New
York, NY, Holt, Rinehart & Winston, 1973
18. Cohen J: Statistical Power Analysis for the Behavioral Sci-
ences. Hillsdale, NJ, Lawrence Erlbaum Associates, 1988
19. de Haes JCJM, van Knippenberg FCE, Neijt JP: Measuring
psychological an physical distress in cancer patients: Structure and
Downloaded from ascopubs.org by 5.12.50.134 on February 25, 2017 from 005.012.050.134
Copyright 2017 American Society of Clinical Oncology. All rights reserved.
SPRANGERS ET AL
application of the Rotterdam Symptom Checklist. Br J Canc
62:1034-1038, 1990
20. Schag CAC, Ganz PA, Heinrich RL: Cancer Rehabilitation
Evaluation System-Short Form (CARES-SF): A cancer specific re-
habilitation and quality of life instrument. Cancer 68:1406-1413,
1991
21. Schipper H, Clinch J, McMurray A, et al: Measuring the
quality of life of cancer patients: The Functional Living-Index-Can-
cer: development and validation. J Clin Oncol 2:472-483, 1984
22. Levine M, Guyatt GH, Gent M, et al: Quality of life in stage
II breast cancer, an instrument for clinical trials. J Clin Oncol 6:1798-
1810, 1988
23. Cella DF, Tulsky DS, Gray G, et al: The Functional Assess-
ment of Cancer Therapy Scale: Development and validation of the
general measure. J Clin Oncol 11:570-579, 1993
24. Osoba D: The quality of life committee of the clinical trials
group of the National Cancer Institute of Canada: Organization and
functions. Qual Life Res 1:211-218, 1992
25. Guyatt G, Walter S, Norman G: Measuring change over time:
Assessing the usefulness of evaluative instruments. J Chron Dis
40:171-178, 1987
26. Guyatt GH, Deyo RA, Charlson M, et al: Responsiveness
and validity in health status measurement: A clarification. J Clin
Epidemiol 42:403-408, 1989
27. Hays RD, Hadorn D: Responsiveness to change: An aspect
of validity, not a separate dimension. Qual Life Res 1:73-75, 1992
28. Groenvold M, Bjorner JB, Klee MC, et al: Test for item bias
in a quality of life questionnaire. J Clin Epidemiol 48:805-816, 1995