Scenario-based multiple-choice

that assess knowledge at higher

Bloom’s levels

Janet Lindsley, PhD

Associate Professor of Biochemistry,
Assistant Dean for Curriculum
Member of the National Board of Medical Examiners
USLME Step 1 Integrated Review Committee
Past Step 1 Item Writing Committee
 Workshop objectives
By the end of this workshop, participants will be
able to:
1. identify the Bloom's taxonomy levels at which
well written multiple choice questions can
assess knowledge,
2. distinguish between well written and poorly
written multiple choice questions,
3. create well written multiple choice questions
which assess knowledge higher Bloom's levels.
 Bloom’s taxonomy for the
cognitive domain

Handbook I: Cognitive
(Bloom et al. 1956)

Image accessed on 9/21/15

http://www.maggiehosmcgrane.com/2
014/09/flipping-grade-4-and-flipping-
blooms.html
Analyze the following NBME
questions to identify key
elements/similarities
 A 64-year-old man comes to the physician because of
swelling in his feet for the past 2 years. He says that his
skin is dry and itchy and his feet "feel heavy." One of his
legs is shown. Which of the following is the most likely
cause of his condition?
(A) Arteriolar constriction and arteriolar hypertension
(B) Arteriolar dilation and venous hypertension
(C) Venous constriction and arteriolar constriction
(D) Venous hypertension and incompetent valves
(E) Venous hypertension and venous constriction

2015samples_step1.pdf
 A 12-year-old girl with a 1-year history of systemic lupus erythematosus is
brought to the physician for a routine follow-up examination. During the past
year she has done well with the exception of occasional mild frontal
headaches, fatigue, and arthralgias; the results of regular laboratory
evaluations have been stable. She has missed 20 days of school in the past 4
months. Her parents express concern that sending her to school might cause
her to be exposed to children with contagious diseases that might exacerbate
their daughter's condition. The girl tells the physician that she misses seeing
her many friends at school. Passive motion of the elbows, wrists, and knees
produces mild discomfort. Physical examination shows no other abnormalities.
Which of the following is the most likely cause of this patient's excessive school
absences?
(A) Lupus arthritis
(B) Lupus cerebritis
(C) Malingering
(D) School phobia
(E) Vulnerable child syndrome

2015samples_step1.pdf
 A placebo-controlled clinical trial is conducted to assess whether a new
antihypertensive drug is more effective than standard therapy. A total of 5000
patients with essential hypertension are enrolled and randomly assigned to one of
two groups: 2500 patients receive the new drug and 2500 patients receive
placebo. If the alpha is set at 0.01 instead of 0.05, which of the following is the
most likely result?
(A) Significant findings can be reported with greater confidence
(B) The study will have more power
(C) There is a decreased likelihood of a Type II error
(D) There is an increased likelihood of statistically significant findings
(E) There is an increased likelihood of a Type I error

2015samples_step1.pdf
A 32-year-old woman with type 1 diabetes mellitus has had progressive
renal failure over the past 2 years. She has not yet started dialysis.
Examination shows no abnormalities. Her hemoglobin concentration is
9 g/dL, hematocrit is 28%, and mean corpuscular volume is 94 μm3. A
blood smear shows normochromic, normocytic cells. Which of the
following is the most likely cause of her anemia?
(A) Acute blood loss
(B) Chronic lymphocytic leukemia
(C) Erythrocyte enzyme deficiency
(D) Erythropoietin deficiency
(E) Immunohemolysis
(F) Microangiopathic hemolysis
(G) Polycythemia vera
(H) Sickle cell disease
(I) Sideroblastic anemia
(J) β-Thalassemia trait

2015samples_step1.pdf
Key elements/similarities?
Appropriately constructed MCQs can
be effective at assessing higher level
cognitive skills

• MCQs with clinical or research vignettes can

assess ability to:
– Analyze information (data interpretation)
– Synthesize information
– Apply knowledge to solving novel problems

Of course, the MCQs should be aligned with

curricular learning objectives.
UUSOM current practice of tagging
questions by “Bloom’s level”
• Recall of factual knowledge
• Applied knowledge
• Data Interpretation
– Including interpretation of images
• Scientific (Research) scenario
– Complex interpretation of research data
• Clinical Scenario
– Complex interpretation of clinical data
 Anatomy of a USMLE-style MCQ
A 7-month-old infant is brought to the physician's office because of poor
weight gain despite large food intake. He has had two episodes of
pneumonia and has frequent bulky stools. He coughs frequently. X-rays
of the lungs show increased markings and hyperinflation. Trypsin is
absent in a fresh stool sample, and the fat content is increased. Which of
the following is the most likely cause of this infant's disorder?
(A) Autoimmune disorder
(B) Defective ion transport at epithelial surfaces
(C) Disaccharidase deficiency
(D) Inability to synthesize apolipoprotein B
(E) Villous atrophy of the jejunum

2015samples_step1.pdf
The stem poses a clear question; it is usually possible to arrive at the
answer with the options covered.
Answer choices are homogeneous.
 Anatomy of a USMLE-style MCQ
A 7-month-old infant is brought to the physician's office because of poor
weight gain despite large food intake. He has had two episodes of
pneumonia and has frequent bulky stools. He coughs frequently. X-rays
of the lungs show increased markings and hyperinflation. Trypsin is
absent in a fresh stool sample, and the fat content is increased. Which of
the following is the most likely cause of this infant's disorder?
(A) Autoimmune disorder
(B) Defective ion transport at epithelial surfaces
(C) Disaccharidase deficiency
(D) Inability to synthesize apolipoprotein B
(E) Villous atrophy of the jejunum

2015samples_step1.pdf
The stem poses a clear question; it is usually possible to arrive at the
answer with the options covered.
Answer choices are homogeneous.
 Tables of answer choices can be
helpful
A 4-year-old boy is brought to the physician because of slow growth during the
past year. He has had recurrent urinary tract infections since the age of 1 year.
He is at the 10th percentile for height and 25th percentile for weight. Physical
examination shows pallor. Laboratory studies show a normochromic, normocytic
anemia and increased serum concentrations of urea nitrogen and creatinine.
Urinalysis shows a low specific gravity. Which of the following sets of additional
serum findings is most likely in this patient?
Calcium Inorganic 1,25-Dihydroxycholecalciferol Erythropoietin
Phosphorous
A    
B    
C    
D    
E    
F    
2015samples_step1.pdf
 Balancing answer choices prevents
cuing to the correct answer
A 4-year-old boy is brought to the physician because of slow growth during the
past year. He has had recurrent urinary tract infections since the age of 1 year.
He is at the 10th percentile for height and 25th percentile for weight. Physical
examination shows pallor. Laboratory studies show a normochromic, normocytic
anemia and increased serum concentrations of urea nitrogen and creatinine.
Urinalysis shows a low specific gravity. Which of the following sets of additional
serum findings is most likely in this patient?
Calcium Inorganic 1,25-Dihydroxycholecalciferol Erythropoietin
Phosphorous
A    
B    
C    
D    
E    
F    
2015samples_step1.pdf
Avoid giving away the answer by
cuing
 Good questions are not
purposefully tricky or deceptive
 Summary: A baker’s dozen of “rules” for writing
effective, NBME-style MCQs
1. MCQs with vignettes can test higher levels of knowledge than simple MCQs.
2. Avoid pseudo-vignettes, in which the vignette is not needed.
3. Question stems should be long and answer choices short and homogeneous
(e.g. all mechanisms described in similar depth).
4. Don’t be negative. (No “which of the following is NOT...”)
5. Tables of answers can be useful.
6. Use as many distracters as necessary to avoid giving away the answer.
7. Questions should never be purposefully tricky or deceptive.
8. Order answers alphabetically or numerically.
9. Avoid giving away the answer. (No cuing by having the right answer longest or
more detailed, spelling or grammatical errors in distractors, having 2 choices
similar and one being correct.)
10.Students should be able to answer the question before reading the options
(No “which of the following is true?”)
11.The lead-in question should include “most likely” (or similar).
12.No questions that would result in malpractice.
13. Avoid “buzz” words.
Gather ideas throughout the year
Application of these concepts
 How could this question be
improved?
A 68-year-old man with no known cardiovascular disease has
normal adult hemoglobin that displays cooperative
interactions between its polypeptide subunits. Which of the
following functional characteristics of hemoglobin does this
cooperativity most strongly affect?

a) Fraction of hemoglobin present as methemoglobin in the

peripheral circulation
b) Fraction of hemoglobin present in tetrameric form in
erythrocytes
c) Fraction of hemoglobin subunits bound to a heme group
d) Fraction of hemoglobin's bound oxygen transferred from
erythrocytes to tissues
 Better?
A female newborn was referred for hemoglobin analysis after a newborn
screen detected a HbF or HbA variant. DNA sequence analysis revealed a
heterozygous alpha-globin Asp126Tyr mutation. Functional analysis of this
mutant protein gave the following data. What is the most likely consequence
of this mutation?
Patient

Reference

A. Decreased fraction of globin chains with heme attached

B. Decreased methemoglobin fraction
C. Decreased oxygen transfer from erythrocytes to tissues
D. Increased fraction of globin chains with heme attached
E. Increased methemoglobin fraction
F. Increased oxygen transfer from erythrocytes to tissues
 Better?
A 68-year-old man with chronic obstructive pulmonary disease (COPD) is
brought to the ED because of confusion and lethargy. His pulse is 96/min,
respirations 22/min, blood pressure 148/82 mm Hg. Physical exam shows
erythema of the skin and mucosal membranes, as well as cyanosis of the nail
beds. There is no splenomegaly. Lab studies show hemoglobin of 17 g/dL and
hematocrit of 49%. Arterial blood gas on room air reveals the following:
pH = 7.33
pCO2 = 58 mm Hg
pO2 = 61 mm Hg
HCO3- = 36 mEq/L
This patient most likely has a decrease in which of the following relative to a
healthy patient?
A. affinity of hemoglobin for oxygen
B. ferritin concentration
C. heme synthesis
D. percent HbA1c
E. red blood cell volume
 Good question?
A 26-year-old woman presents with pustular skin lesion on the right
arm for 2 days and is found to have acute arthritis of the left knee and
right wrist. Gram stain of the pustule is shown in the attached image.
Which of the following is the most likely initial portal of entry for this
pathogen in this patient?

a) Cutaneous
b) Gastrointestinal
c) Genital
d) Pulmonary
e) Urinary
 How could this question be
improved?
A 25-year-old woman with cystic fibrosis is receiving care for
her condition in a patient-centered medical home. Which of
the following practices best exemplifies the patient-centered
aspect of care in a medical home model?

a) Delivering outpatient and emergency room care

b) Facilitating access to the emergency room when symptoms
are exacerbated
c) Following national guidelines for care
d) Reducing cost of prescription medications
*e) Teaching self-management of symptoms based on
condition severity
 Better?
A 25-year-old woman with cystic fibrosis is receiving care for
her condition in a patient-centered medical home. Her care is
most likely to differ from traditional services by including:

a) Coverage for both outpatient and emergency room care.

*b) Instruction on how to manage her GI and respiratory
symptoms.
c) Only protocols that follow US Preventative Services
guidelines for care of patients with cystic fibrosis.
d) Rapid access to emergency room care when she
experiences exacerbated shortness of breath.
e) Reduced cost of Pancreatic enzymes, Fluticasone, Albuterol,
nebulized DNase and Montelukast.
 Good question?
A 5-year-old boy was brought to the emergency room,
unconscious, and in shock after he accidentally overdosed on
his mother's iron pills, mistaking them for sweet-sour candy.
He is hyperglycemic with leukocytosis and the empty
container recovered from the scene suggests that he had
ingested about 33 ferrous sulfate tablets. How would you
treat this patient?

a) Chelation therapy with dimercaprol

b) Detoxification with activated charcoal
*c) Gastric lavage with deferoxamine
d) IV infusion with British Anti-Lewisite
e) Oral succimer, then pennicillamine
 How could this question be
improved?
A 20-year-old man presents to the ER after an automobile
accident. Radiography shows extensive fractures of the skull,
including a fracture and partial closure of the foramen ovale.
This fracture would damage the nerve that supplies all of the
following muscles, EXCEPT:

*a) Buccinator
b) Lateral pterygoid
c) Masseter
d) Medial pterygoid
e) Temporalis
 Better?
A 20-year-old man presents to the ER after an automobile
accident. Vital signs are T 97.3 F, RR 22/min, HR 105/min, BP
110/75 mm Hg. Radiography shows extensive damage to his
skull, including a fracture and partial closure of the foramen
ovale. This fracture most likely results in nerve damage to which
of the following muscles?

Buccinator Lateral Masseter Medial Temporalis

pterygoid pterygoid
A Yes Yes Yes Yes No

B Yes Yes Yes No Yes

C Yes Yes No Yes Yes

D Yes No Yes Yes Yes

E No Yes Yes Yes Yes

What is one thing that you are
going to start, stop or continue
doing as a result of this
workshop?
 References
• NBME item writing guide (see:
http://www.nbme.org/publications/item-
writing-manual-download.html)
Questions?