Patient Care
FUNDAMENTALS OF

PATIENT SATISFACTION
MEASUREMENT
n Eric Shirley, MD; Gary Josephson, MD; and James Sanders, MD
In this article
Know the ins and outs of measuring patient satisfaction to be sure youre not
misreading the scores.
PATIENT SATISFACTION IS A COGNITIVE EVAL-
uation and an emotional reaction to medical care that is
strongly influenced by underlying expectations.1,2 Since the
Institute of Medicine emphasized the delivery of patient-cen-
tered care in the early 1990s,3 attention to patient satisfaction
has increased.
Patient satisfaction is now an endpoint for the evalua-
tion of care, physician payment structures 4 and a measure
for insurance companies to determine hospital selection and
reimbursement.
Given the importance of these data, nearly all health care
employees are responsible for the scores. Proper analysis and
response to these scores requires understanding the principles
of satisfaction measurement such as methods, psychometric
properties and theories of interpretation.
MEASUREMENT METHODS T he goal of patient satisfac-
tion surveys is to understand from the patient perspective a
hospitals or institutions specific strengths and weakness in
order to improve the delivery of care.3 Satisfaction can be
measured with qualitative or quantitative methods.
Qualitative methods use professional observations, focus
groups or interviews and have the potential to generate higher
response rates.5 However, due to disadvantages such as influ-
encing provider behavior during observation, lack of statistical
validity and reliability, and greater cost,6 these methods are
typically used only during survey development.
12 JANUARY/FEBRUARY n 2016
Quantitative methods are more common for acquiring
ongoing satisfaction data as they provide a more accurate
method to measure and compare the patient experience.
Satisfaction may be scored on a global measure or within
multiple distinct domains. With a single global dimension or
question, it is difficult to determine why a patient rated sat-
isfaction a certain way.
More often, multidimensional measures are used to provide
further detail within distinct domains, such as interpersonal
manner of the provider, technical quality of care, availability,
outcomes of care and the physical environment.7 As the mul-
tidimensional surveys tend to be longer, it is important that
the time required for completion is acceptable to patients.8
Survey instruments may be generic and applicable to mul-
tiple conditions or disease-specific. Generic measures evaluate
the process of care including the service provided by the hospi-
tal, medical facilities and patients relationship with health care
professionals. Disease-specific measures evaluate satisfaction
with treatment outcome for a specific disease or disorder.
Surveys consist of carefully worded questions and use mul-
tiple response formats, ranging from simple yesno formats
to multipoint satisfaction scales. Interval-type scales such as
the five-point Likert format (very poor, poor, fair, good, very
good),9 are helpful as they can be easily converted into numeri-
cal values.10 A comments section helps to obtain information
not solicited by the questions.

PSYCHOMETRIC PROPERTIES M  easurement tools must be
valid and reliable in order to make appropriate changes to care
processes.11 Therefore it is important to understand these and
other psychometric terms as they relate to the measurement
of patient satisfaction.
VALIDITY Validity for a patient satisfaction survey means
that the questionnaire results actually reflect patient satisfac-
tion. This is the most difficult psychometric property to assess
in satisfaction surveys, as there is no gold standard for com-
parison testing. Subsequently, an extensive item-generation
process is required to ensure the different types of validity.12
n Face validity implies that an instrument measures what
it is intended to measure.
n Content validity is the concept that a questionnaire aim-
ing at a certain topic should cover the content of the
topic. For a patient satisfaction questionnaire to have
content validity, it should measure all of the dimen-
sions considered important for patients to recommend
a provider.
n Criterion validity means that another criterion, such as
the current instrument or outcome, corresponds with
the instrument under development. As customer loy-
alty is often a sign of satisfaction in the business world,
likelihood to recommend a provider is often used for
comparison in health care.
The goal of patient satisfaction surveys is to under-
stand from the patient perspective a hospitals or
institutions specific strengths and weaknesses in
order to improve the delivery of care.3
RELIABILITY Reliability reflects the ability of the instru-
ment to produce consistent and reproducible results. The three
common methods of reliability testing are the interclass cor-
relation coefficient (ICC), Pearson correlation coefficient and
the kappa statistic. The kappa statistic is superior for ordinal
data typically found in surveys.
When testretest results are compared for the number of
exact matches, exact matches can occur by chance alone, and
the kappa statistic accounts for this chance factor. Accepted
criteria for kappa coefficients are: k > 0.80, almost perfect;
k= 0.61 to 0.80, substantial; k = 0.41 to 0.60, and moderate;
k = 0.21 to 0.40, fair; k = 0.00 to 0.20.13.
BIASES Questionnaires may be limited by biases in several
ways. Types of biases are:
n Optimizing: Difficult-to-interpret questions require a
lot of judgment, making answers unreliable. As an ex-
ample, a patient could interpret were you comfort-
able? as relating to their surgery but also as relating
to the softness of the bed.
n Recall bias: Occurs when a patient is asked about how
they were in the past, such as how comfortable was
your current admission compared to other hospitals for
prior admissions?
n Satisficing: In longer burdensome questionnaires, pa-
tients may do what it takes to finish the questionnaire,
Physician Leadership Journal 13
 rushing through the questionnaire just to get it done.
n Social desirability: Patients may also subconsciously
want a result to be better than it is, or even lie about
being good (i.e., faking good). The opposite of this is
deviation (i.e., faking bad).
n End-aversion: Patients may not want to answer at the
extremes. This is also called the central tendency bias.
n Positive skew: Patients may have a tendency to skew
results toward the positive.
n Halo effect: An overall positive experience tends to
cause all answers to reflect this even if the individual
items would not have matched this level.
RESPONSIVENESS The responsiveness of an instrument
also is called its sensitivity to change. If satisfaction improves
with process improvement initiatives, then scores also should
improve. If scores do not improve, then the instrument may
be insufficiently sensitive, may measure the wrong items (lack
of validity) or the improved issues did not relate to patient
satisfaction.
SELECTING OR SWITCHING TO
A DIFFERENT PATIENT SATISFACTION
SURVEY IS A DIFFICULT TASK.
Response rates of 30 percent are usually acceptable, with
50 percent considered high and 80 percent considered very
high.14 However, these figures are not standardized and are
in stark contrast to the response rate of 80 percent proposed
as an absolute minimum for epidemiological studies.15
In addition, the minimal clinically important difference
(MCID) for satisfaction scores is not known. The MCID is a
change in score that actually means something to a patient.
For example, if a domain score change from 20 to 25 means
a patient would seek another physician, then the score is
important. But if the change is completely unimportant to
the patient, then the two scores are functionally equivalent.
INTERPRETATION OF RESULTS Appropriate understanding
of satisfaction data has the potential to improve the way care
is delivered. However, interpretation of these data may result
in conclusions at opposite ends of a spectrum.
At one end is over-interpretation, with all findings and
comparisons among institutions being accepted as factual
despite obvious differences in survey administration and prac-
tice setting.
At the other end, under-interpretation (particularly by
physicians) may dismiss all of the data as inaccurate or not
applicable. Appropriate analysis requires understanding of
the tool that was used and leads to conclusions between the
ends of this spectrum.
14 JANUARY/FEBRUARY n 2016
Interpretation begins with an examination of the measure-
ment tool that was used. It is important to verify the psycho-
metric properties, as many institutional or author-developed
measurement tools were developed with little or no attention
to verifying their validity and reliability.3,11,16,17
For example, one study17 found that 60 percent of the pa-
tient satisfaction instruments in the literature reported no psy-
chometric data. Further, it was reported that only 71 of 3,000
articles using patient satisfaction as an outcome measure used
instruments that had undergone psychometric development.12
Although consensus on an appropriate response rate is
lacking, the response rate for the survey results should be
noted with attention drawn toward rates falling below 30
percent. Conclusions can then be drawn from the data itself
using different theoretic models of interpretation of satis-
faction data. These include the discrepancy model and the
top-box model.
Under the discrepancy model, satisfaction is defined by
the perceived discrepancy between the expectations and the
experience.18 In this manner, positive responses do not indi-
cate that care was good, but instead indicate that nothing
extremely bad occurred.18
In comparison, the top-box model uses the highest rating
as the basis of comparisons. This method draws from the
consumer model, where only the most extremely satisfied
customers can be counted on to remain loyal and recommend
the practice to others.19 Despite the popularity of the top box
model, the rationale for use in the health care setting is not
well-documented.
Individual item scores can be used to highlight opportu-
nities for improvement. Although open comments are not
amenable to scoring and statistical analysis, these comments
can be particularly helpful to physicians making an effort to
improve their patient interactions and evaluation of different
care processes.
Comparisons between institutions or to benchmarks
should be made cautiously. The results of different types
of surveys should not be compared as the results of single-
question satisfaction surveys tend to be more optimistic than
multidimensional surveys.20
Other survey variables also can affect results. For example,
telephone surveys are able to achieve higher response rates
than mail-based methods.9,11 Interviews may not solicit accu-
rate or thorough information due to loss of anonymity, tim-
ing or fear of the impact on care bring delivered. As a result,
interviews tend to produce socially desirable responses with
higher positive ratings than mail surveys.5,21
Timing of administration also affects results, with satisfac-
tion ratings declining over time22 if patients experienced billing
or other problems related to the visit. Thus higher satisfaction
scores are often seen in handout vs. mail surveys.23
Finally, variations in practices (such as patient sociodemo-
graphics, patient health status, practice environment) may
lead to inappropriate comparisons.
SURVEY OPTIONS The increasing importance of patient
satisfaction data has been associated with the development
of thousands of surveys (Table 1).24 These surveys are admin-
istered from two main sources, external surveys directed from
national bodies and internal surveys directed from hospitals
or clinics.
TABLE 1. PATIENT SATISFACTION SURVEYS
Client Satisfaction Questionnaire
Consumer Assessment of Healthcare Providers and Systems
Consultation Satisfaction Questionnaire
DrScore.com
Patient Satisfaction Index
Patient Satisfaction Questionnaire
Short Assessment of Patient Satisfaction
The Consumer Assessment of Healthcare Providers and
Systems (CAHPS) surveys are generic tools created by the
government Agency for Healthcare Research and Quality
(AHRQ).25,26
Different CAHPS surveys are available that assess experienc-
es across multiple settings: ambulatory, surgical, health plans,
and medical groups. The surveys have standard core items but
can be customized with optional supplemental items.
CAHPS surveys ask whether patients never, sometimes,
usually, or always experienced certain practices (such as com-
munication, pain control, cleanliness) during the care episode
in an effort to generate an objective measure of the patient
experience.
The Patient Satisfaction Questionnaire and the Client Sat-
isfaction Questionnaire are generic tools created by private
organizations. The Patient Satisfaction Questionnaire, devel-
oped by the RAND Corp. in Santa Monica, California, divides
satisfaction into multiple domains of care.7 This survey consists
of 43 items reflecting six dimensions of service: access to care,
availability of services, technical quality of care, interpersonal
care, communication and financial.
The Client Satisfaction Questionnaire (CSQ-18), developed
at the University of California, San Francisco, and distributed
by Tamalpais Matrix Systems in Mill Valley, California, is an-
other generic multidimensional measure27 that consists of
18 items assessing satisfaction with health care services. 28
Dimensions measured by the CSQ include physical surround-
ings, general satisfaction, interpersonal communication and
technical aspects of care.
The Short Assessment of Patient Satisfaction (SAPS) scale
is a newer validated generic measure of patient satisfaction.29
Advantages include its length of seven items requiring just
over a minute to complete and its wide applicability.29
Other generic measures include the Consultation Satisfac-
tion Questionnaire (ConsultSQ) that has 18 items assessing
clinician consultations30 and the Patient Satisfaction Index
(PSI) with 23 items covering satisfaction with medical care.31
Web-based satisfaction surveys exist as well. DrScore.com
is an online patient satisfaction survey that uses two patient
satisfaction scales: satisfaction with physician care and sat-
isfaction with office policy and procedures.32 The physician
scale consists of nine items, and the office scale consists of
five items. Responses are measured on a scale from zero (not
at all satisfied) to 10 (extremely satisfied).32
More commonly, hospitals select a vendor that gathers and
reports patient satisfaction data. These vendors use databases
to determine benchmarks for comparable organizations ac-
cording to hospital size and region (Table 2).
TABLE 2. PATIENT SATISFACTION SURVEY VENDORS
Center for Medicare and Medicaid Services (CAHPS)
NRC (National Research Corporation) Picker
Pinnacle Quality Insight
Press Ganey Associates
Professional Research Consultants
SullivanLuallin Group
The patient-satisfaction measurement firms with the most
clients (listed in order) include National Research Corp. Picker,
Press Ganey Associates, Professional Research Consultants
(PRC), Pinnacle Quality Insight and SullivanLuallin Group.33
CHOOSING A SURVEY S electing or switching to a different
patient satisfaction survey is a difficult task. As an agreed-upon
single standardized measurement tool does not exist,3,34 a spe-
cific recommendation for the best survey cannot be made.
However, it is clear that although it may be tempting to
create a new survey to individualize questions and decrease
costs, the steps required to generate a validated questionnaire
are exceedingly time consuming.12 Therefore existing tools
with established psychometric properties should be used to
ensure that satisfaction is measured and reported accurately.12
Selecting a satisfaction survey requires identifying what the
data will be used for; such as participation in benchmarking,
meeting insurance requirements or improving processes of
care. In addition, the domains of interest should be identi-
fied.29,34
Domains may include access to care and the environment in
which care is provided;35,36 health information is provided;35,36
doctor-patient relationship;36 participation in decision mak-
ing;36 technical quality of the care;35,36 effectiveness of treat-
ment relative to patient expectations of care;36 and general
satisfaction.28,30
It also is important to consider whether a survey that mea-
sures what patients actually experienced or one that measures
perceptions or opinions of that experience is desired, as the
former tend to be more accurate.37-39
Other factors to consider are cost and method of admin-
istration. The utility index, which consists of five components
(validity, reliability, educational impact, cost efficiency, accept-
ability) can assist in choosing the right instrument.40
CONCLUSION It is important for health care employees
to understand the measurement techniques used to obtain
Physician Leadership Journal 15
patient satisfaction data as they are as important as the data
itself. Employees should also be aware of measurement vari-
ations that may result in inappropriate data interpretation.
Knowledge of these concepts can help identifying strengths
and weakness from the patient perspective in order to improve
the delivery of care.
Eric Shirley, MD, is director of sports medicine
at Nemours Childrens Specialty Care and an
assistant professor at Mayo Clinic College of
Medicine in Jacksonville, Florida.
Eric.Shirley@nemours.org
Gary Josephson, MD, is chairman of the
Department of Surgery at Nemours Childrens
Specialty Care and an associate profes-
sor at Mayo Clinic College of Medicine in
Jacksonville, Florida.
GJosephs@nemours.org
James Sanders, MD, is chief of the Division
of Pediatric Orthopaedics and professor of
orthopaedics and pediatrics at the University
of Rochester and the Golisano Childrens
Hospital in Rochester, New York.
REFERENCES
1. Urden LD. Patient satisfaction measurement: current issues and implications.
Outcomes Manag. 6(3):125-31, July-Sept 2002.
2. Crow R, Gage H, Hampson S, et al. The measurement of satisfaction
with healthcare: implications for practice from a systematic review of the
literature. Health Technol Assess6(32):1244, 2002.
3. Espinel AG, Shah RK, McCormick ME, et al. Patient satisfaction in pediatric
surgical care: A systematic review. Otolaryngol Head Neck Surg. 150(5):739-
49, May 2014.
4. Morris BJ, Richards JE, Archer KR, et al. Improving patient satisfaction in
the orthopaedic trauma population. J Orthop Trauma. 28(4): e804, Apr
2014.
5. Burroughs TE, Waterman BM, Cira JC, et al. Patient satisfaction
measurement strategies: a comparison of phone and mail methods. Jt
Comm J Qual Improv. 27(7):349-61, July 2001.
6. Fottler MD, Ford RC, Bach SA. Measuring patient satisfaction in healthcare
organizations: qualitative and quantitative approaches. Best Pract
Benchmarking Healthc. 2(6):227-39, Nov-Dec 1997.
7. Ware JE Jr, Snyder MK, Wright WR, Davies AR. Defining and measuring
patient satisfaction with medical care. Eval Program Plann. 6(3-4):247-63,
1983.
8. Beattie M, Lauder W, Atherton I, et al. Instruments to measure patient
experience of health care quality in hospitals: a systematic review protocol.
Syst Rev 3(4):1-8, Jan 2014.
16 JANUARY/FEBRUARY n 2016
9. Clark PA. Medical practices sensitivity to patients needs: Opportunities and
practices for improvement. J Ambul Care Manage. 26(2):110-23, Apr-June
2003.
10. Lund CM. Benchmarking patient satisfaction. Best Pract Benchmarking
Healthc. 1(4):203-6, July-Aug 1996.
11. Urden LD. Patient satisfaction measurement: Current issues and implications.
Outcomes Manag. 6(3): 125-31, July-Sept 2002.
12. Barnett SF, Alagar RK, Grocott MPW, et al. Patient-Satisfaction Measures
in Anesthesia: Qualitative Systematic Review. Anesthesiology. 119(2):452-78,
Aug 2013.
13. Garbuz DS, Masri BA, Esdaile J, et al. Classification systems in orthopaedics.
J Am Acad Orthop Surg. 10(4):290-7, July-Aug 2002.
14. Sitzia J, Wood N. Response rate in patient satisfaction research: an analysis
of 210 published studies. Int J Qual Health Care. 10(4):311-7, Aug 1998.
15. Rose G, Barker DJ. Epidemiology for the uninitiated: Conduct of surveys. Br
Med J. 2(6146):1687-8, Dec 16, 1978.
16. Hamann J, Leucht S, Kissling W. Shared decision making in psychiatry. Acta
Psychiatr Scand. 107(6):403-9, June 2003.
17. Sitzia J. How valid and reliable are patient satisfaction data? An analysis of
195 studies. Int J Qual Health Care. 11(4): 319-28, Aug 1999.
18. Williams B. Patient satisfaction: a valid concept? Soc Sci Med. 38(4):509-16,
Feb 1994.
19. Jones TO, Sasser WE. Why satisfied customers defect. Harvard Bus
Rev.73(6):80-110, Nov-Dec 1995.
20. Vuori H. Patient satisfaction--does it matter? Qual Assur Health Care.
3(3):183-9, 1991.
21. Burroughs TE, Waterman BM, Cira JC, et al. Patient measurement
strategies: a comparison of phone and mail methods. Jt Comm J Qual
Improv.27:349-61, July 2001.
22. Bendall-Lyon D, Powers TL, Swan JE. Time does not heal all wounds:
Patients report lower satisfaction levels as time goes by. Mark Health Serv.
21(3):10-4, Fall 2001.
23. Gribble RK, Haupt C. Quantitative and qualitative differences between
handout and mailed patient satisfaction surveys. Med Care.43(3):276-81,
Mar 2005.
24. Morris BJ, Richards JE, Archer KR, et al. Improving patient satisfaction in
the orthopaedic trauma population. J Orthop Trauma. 28(4): e804, Apr
2014.
25. Agency for Healthcare Research and Quality/Consumer Assessment of
Healthcare Providers and Systems website. Accessed January 21, 2015.
https://www.cahps.ahrq.gov
26. Centers for Medicare & Medicaid Services. Hospital care quality information
from the consumer perspective (website). Accessed January 21, 2015.
Available at: www.hcahpsonline.org.
27. Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/
patient satisfaction: development of a general scale. Eval Program Plann.
2(3):197-207, 1979.
28. Larsen DL, Attkisson CC, HargreavesWA, Nguyen TD. Assessment of client/
patient satisfaction: development of a general scale. Eval Program Plann.
2(3):197-207, 1979.
29. Hawthorne G, Sansoni J, Hayes L, et al. Measuring patient satisfaction with
health care treatment using the short assessment of patient satisfaction
measure delivered superior and robust satisfaction estimates. J Clin Epi.
67(5): 527-37, May 2014.
30. Baker R. Development of a questionnaire to assess patients satisfaction
with consultations in general practice. Br J Gen Pract. 40(341):487-90, Dec
1990.
31. Guyatt GH, Mitchell A, Molloy DW, et al. Measuring patient and relative
satisfaction with level or aggressiveness of care and involvement in

Art
care decisions in the context of life threatening illness. J Clin Epidemiol
48(10):1215-24, Oct 1995.
SHOWCASE YOUR
32. Camacho FT, Feldman SR, Balkrishnan R, et al. Validation and reliability of

of MEDICINE
2 specialty care satisfaction scales. Am J Med Qual. 24(1):12-8, 2009. Epub
Dec 5, 2008.

33. Sandler M. By the numbers. Largest patient-satisfaction measurement firms.

Mod Healthc.42(18):32, Apr 30, 2013. The American Association for Physician Leadership is
34. Hawthorne G. Review of patient satisfaction measures. ISBN: 0 642 82800 seeking submissions for a new humanities-focused fea-
8. Canberra, Australia: Australian Government Department of Health and ture in the Physician Leadership Journal that showcases
Ageing; 2006. your creative impulses related to physician leadership,
35. Fox JG, Storms DM. A different approach to sociodemographic predictors of including:
satisfaction with health care. Soc Sci Med. 15(5):557-64, Sept 1981.
36. Hardy G, West M, Hill F. Components and predictors of patient satisfaction. n POETRY
Br J Health Psychol. 1:65-85, 1996.
n SHORT-FORM PROSE
37. Sofaer S, Firminger K. Patient perceptions of the quality of health services.
Annu Rev Public Health. 26:51359, 2005. n VISUAL ART
38. Sixma H, Kerssens J, Campen C, et al. Quality of care from the patients traditional or digital
perspective: from theoretical concept to new measuring instrument. Health n HANDCRAFTS AND SCULPTURE
Expect 1(2):8295, Nov 1998.
images only, please
39. Coulter A, Fitzpatrick R, Cornwell J. The point of care measures of patients
experience in hospital: purpose, methods and uses. The Kings Fund, July So indulge your creative side and show us your interpre-
2009, pp. 132. tation of the Art of Medicine. Send your submissions
40. Van der Vleuten C. The assessment of professional competence: to Gina Bowden-Pierce.
developments, research and practical implications. Adv Health Sci Educ gbowden-Pierce@physicianleaders.org
Theory Pract. 1(1):4167, Jan 1996.

Physician Leadership Journal 17

Copyright of Physician Leadership Journal is the property of American Association of
Physician Leadership and its content may not be copied or emailed to multiple sites or posted
to a listserv without the copyright holder's express written permission. However, users may
print, download, or email articles for individual use.