Beruflich Dokumente
Kultur Dokumente
ONCE YOU GET STARTED: Im going to read through some things so bear
with me
Introduction
Hi, and thank you for agreeing to participate in this session today. My name
is <FIRST and LAST NAME>, and I work for Westat, a research company
based in Rockville, MD. Today Id like to get your reactions to a website that
allows users to search for different doctors and other clinicians who provide
medical services to people enrolled in Medicare.
Westat is conducting an evaluation of the Physician Compare website for the
Centers for Medicare and Medicaid Services (CMS), the Federal agency that
oversees the Medicare program. Physician Compare is currently available
through Medicare.gov, but today well be looking at a redesigned version of
the Physician Compare website. Were interested in evaluating potential new
pages for the website and identifying areas for improvement.
Your responses will be kept confidential. For our report, the comments that
you share will be grouped with comments from other people like you who are
providing feedback. Your name will not be associated with anything you say.
Todays session is completely voluntary. If youd like to stop at any time,
please tell me and well stop immediately. If there are any questions you
dont want to answer, just tell me and we can skip them.
Some team members working on this project and Medicare representatives
are logged into this session as observers. Wed like to record this session so
we can make sure we dont miss anything. Because well be recording,
please try to speak in a voice at least as loud as the one Im using now so
that we can make sure the tape is picking up our voices.
Page 1
Consent
Interviewer Instructions: Show consent form on screen (Consent form will
be sent to respondent in advance of session). READ THE FOLLOWING ALOUD
TO PARTICIPANT:
Westat is a private company conducting this project on behalf of the
Centers for Medicare and Medicaid Services (or CMS).
We will be asking you to review a set of sample web pages to obtain
your feedback.
The session will last approximately 60 minutes.
Your participation is voluntary, and you have the right to stop at any
time or to refuse to answer any question.
The things you say may be put in a written summary of this discussion,
but your name will not be used anywhere.
We would like to record the interview in case we need to review a
portion of the session as we make recommendations for the website.
The recording will only be reviewed by those working on the website.
You will be sent a check for $60 for your participation.
Interviewer instructions: Answer any questions. Obtain consent to
proceed and to record.
Do we have your consent to record the session?
EXPLAIN FEATURE TO RECORD, TURN IT ON, AND THEN
OBTAIN CONSENT AGAIN SO THAT CONSENT IS RECORDED.
Okay, now that we are recording: Today is [DAY AND TIME]. Do we have your
permission to record, and do you agree to participate in this study?
Im going to ask you to do some tasks using the website. There are no right
or wrong answers. I am simply interested in how you go about completing
each task and how easy or difficult you find the site to use. Its important
to know that we are not testing you were testing possible new pages for
this website and identifying potential ways to improve them. If something
that we go through today is unclear or confusing, its important to share this
with me since its likely to be confusing to other people, too. I didnt design
the site, so you wont hurt my feelings if you criticize it. All of your positive
and negative feedback is welcome. Well save some time at the end of the
session to cover your general impressions.
Page 2
One last thingthis website is NOT fully developed at this point. If you try to
get toor land onsomething that is not yet developed, Ill tell you.
Do you have any questions before we begin?
-
Before we begin with the tasks, Im going to let you use, and familiarize
yourself with the website for about 5-10 minutes. Feel free to use the site as
you normally would if I wasnt here, click around, and explore the different
features. Ill be available if you get lost at any point. If you could, just think
aloud as you work.
Tasks
Now Im going to ask you to do some specific tasks. As we get started, here
are some things to keep in mind:
Ill read each task out loud to you then work on the task as if you were
doing it alone at home without me.
Feel free to talk aloud as you work.
When you find what youre looking for, let me know that youre done
with the task. Then, Ill tell you when you can move on to the next
task.
Page 3
Page 4
a. Starting on the home page, find two doctors who you consider to
be in a good location for [you/the person you care for].
i. Typed in the location bar Washington DC and searched
primary care. She then selected the blue box to go to a
page that showed internal medicine doctors
ii. Added William Harrison MD and Annick P Hebou to
compare, and was going to click compare
iii. R expects to see biographical information, where they went
to school, and qualifying information.
b. How would you compare these doctors?
i. R adds two physicians to the compare tab and clicked the
green compare button
2) Think of your own medical condition(s) or a health issue of interest to
you [or the person you care for].
a. Starting on the homepage, find a doctorother than your [or
his/her] own current doctorwho could give you a second
opinion about alternative treatments you might consider.
i. R searches by just typing in Washington DC and selects
another doctor to see his information; sees where the
doctor is located, educational background
ii. R wants to know how old the physician is; it is important
for her to know
iii. Expects to see information on how this doctor does
things when asked the clinical quality of care tab. R
October 2016 Moderators Guide
Page 5
Page 6
g. Take a minute to look over this page. What can you tell me about
this page and the information available?
3) Pretend that youre experiencing pain in your foot, but you arent sure
what type of doctor to see.
a. If they already noticed Search by body part: Using the
search by body part button on the home page, find a doctor to
help with your foot pain.
Page 7
i. She would type her city and then would click search by
body part. R chooses the woman option and clicks on feet
and clicks next.
ii. R says the images are weird and x-ray and opaque
looking says the highlight color should be more vibrant.
b. If they have NOT noticed Search by body part yet: Starting
from the home page, how would you go about finding a doctor
for your foot pain?
Page 8
Debriefing
(Cover points noted during task performance first, then summary reactions
to site pages and features.)
Task Performance
Were there any tasks that you werent sure how to do? Why was that?
o No, R says it was very straightforward and not difficult to do
[FOR ANY TASK THAT YOU NOTED A USABILITY ISSUE FOR THAT
PARTICIPANT DID NOT ALREADY COVER]:
o I noticed that on Task # __, you [FILL WITH WHAT YOU
OBSERVED]. Tell me about what you were thinking there
At any time, did you feel you needed some type of help in order to
figure out what to do next?
o [IF NEEDED]: Where was that? What were you doing at that time?
o What would you do to get help with the website if you were on a
website like this at home on your own?
Homepage
When you first looked at this page, what did you notice? Tell me about
that What made you notice [FILL BASED ON RESPONDENTS
ACTIVITY DURING TASKS]?
o R: It was trying to solicit more specific information from her
which is smart, a lot of these things just has generic boxes; it
drew R out, especially for someone who is not knowledgeable
about some of these things and it also gave you a way to look in
a couple different ways.
What are your thoughts about other areas of the homepage, the
[COVER ANY ELEMENTS NOT PREVIOUSLY MENTIONED]
o Header, More Medicare Compare Websites, Resources, Carousel,
etc.
They are different ways to find those solutions; its different
kinds of searches and drills down on more specificity; its
not just a doctor thing it offers other health care
professionals part of the equation. A lot of patients do not
know where to go when they are looking, so this is helpful.
Page 9
How easy or difficult was it to use the search box to find what you were
looking for?
o It was not difficult to find or use the search box.
o Did you know that you could search for names, specialties, or
medical conditions?
Out of these options, how would you normally search for a
doctor on a website like this?
R: She would normally type in the specialty, she
knows what ones she is looking for; likes that it gives
her multiple ways to search.
o When you searched for [back] from the home page, you were
brought to this screen:
Page 10
Did you see the Search By Body Part button on the home page?
o What do you think youll find when you click on this button?
R: You pretty much know what you are looking for but I
guess if you didnt this helps you. If you are having foot
pain, you click foot pain and it will break it down by
specialty and it will eventually get you to a list of doctors,
by virtue of using it as a research tool
o Would this type of search be helpful to you when looking for a
new doctor?
Results page
When you got to a Results page, did you see what you expected, or
not? Tell me more about that
o Were the results listed in the way you expected?
o PROBE: From the information provided on this page, can you tell
which doctors are part of a group practice?
[DID THEY NOTICE THAT GROUPS AND INDIVIDUALS WERE
LISTED TOGETHER?]
o What do you like about the way results are presented?
This site aggregates a lot of things in one place for me,
rather than.. Im thinking of such and so place, you have to
really drill down on that Dr. you may have to google some
information for yourself. This is all in one place and thats a
good thing.
o What, if anything, do you suggest changing about the display of
results?
R: As I said it does aggregate more information than I have
seen elsewhere and I think its a dramatic improvement
over some of the others. Thinks it is good for the most part.
October 2016 Moderators Guide
Page 11
Map
[IF THEY MENTIONED THE MAP ALREADY]
o Did you think the map was helpful or not helpful for finding a
doctor?
R: it is what it is technology wise. It does bring up that
specific practice or doctor with phone number, locations;
its pretty standard.
[IF THEY DIDNT MENTION THE MAP]
o Did you notice the map on this page?
o Did you notice the pins on the map changed color when you
hovered over the doctors name?
R: As I was going along, that is what it would have popped
up. Maybe it was a more specific answer to the query I was
making. It became green for a reason.
October 2016 Moderators Guide
Page 12
Did you notice the Search Box at the top of this page?
Do you think this would be helpful or not helpful when searching for a
doctor or a group practice?
Compare Feature
Profile pages
Clinician Profile Pages
When you got a doctors profile page, did you see what you expected,
or not? Tell me more about that
o Was the information available what you expected to see, or not?
R: Yes
o What do you like about the way profile pages are presented?
o What, if anything, do you suggest changing about the display of
information on doctors profile pages?
R: Again, I would think a little more about the font or the
size being somewhat different. Maybe make the topics
more block or uppercase. Its just all the same, too much.
Needs to be distinguished more
R: The Accepting New Patients thing. Almost impossible b/c
it changes all of the time. Maybe you have something that
says when they are not accepting new patients.
October 2016 Moderators Guide
Page 13
When you got a group practices profile page, did you see what you
expected, or not? Tell me more about that
o Was the information available what you expected to see, or not?
o What do you like about the way profile pages are presented?
o What, if anything, do you suggest changing about the display of
information on group practice profile pages?
You explain what anesthesiology is. It came from that other
page by specialties. If you didnt know what it is, you
would know what an explanation is.
Sticky Header
Did you notice that the contact information for the group practice
remained at the top of the screen when you scrolled down the page?
o Yes the top stays static. It was clear.
o Is this something that would be helpful, or not helpful?
R: Well its helpful to know you are not lost. If you get
bogged down with your results, you can retrace your steps
without completely closing yourself out which is a good
thing. If you are not good with technology, this helps. The
older you get, the more you need this technology thing so
dont resist.
R: Attention to patient medication cost!! That is a really
good item especially if you are older with Medicare. That is
a really good question to ask if you ask to create the page.
If someone considers asking a Dr this question, that is a
very important thing. A lot of people dont get care b/c
they cant afford it. That is a really good element to look
for. A really good thing.
Overall Reactions
Page 14
Page 15
Closing
Thank you for your time and sharing your opinions about the Physician
Compare website.
Page 16