Opinion
VIEWPOINT
Accreditation Council for Graduate Medical
Education (ACGME) Milestones—
Time for a Revolt?
Ronald M. Witteles, In theory, there is no difference between theory and
MD practice. But, in practice, there is.
Department of Jan L. A. van de Snepscheut
Medicine, Stanford
University School of
Medicine, Stanford, The pleasure of leading a residency program and shap-
California. ing the development of postgraduate trainees comes
with increasingly formidable administrative chores. In the
Abraham Verghese, 2013 to 2014 academic year, the Accreditation Council
MD, MACP
Department of
for Graduate Medical Education (ACGME) ushered in the
Medicine, Stanford “Next Accreditation System” for all residency and fel-
University School of lowship training programs.1 One of its stated goals was
Medicine, Stanford, to address a problem that was widely recognized: “As ad-
California.
ministrative burdens have grown, program directors
have been forced to manage programs rather than men-
tor residents.”1
Viewpoint The most notable mandatory change was the insti-
tution of Milestones1-3 to track each trainee’s progress
from amateur to skilled practitioner in his or her field of
specialization. For example, surgical trainees could prog-
ress along a Patient Care Milestone as they first mas-
tered knot tying, then simple surgical procedures, and
eventually complex operations. The Milestones essen-
tially subsumed the 6 core competencies (patient care,
medical knowledge, practice-based learning, systems-
based practice, professionalism, and interpersonal skills
/communication) that had formed the basis of the last
major paradigm shift by the ACGME. Although the theory
underlying Milestones seemed sensible, the concept
broke down in implementation. For large internal medi-
cine programs, it specifically failed on one of the most
basic metrics: it became nearly impossible to actually fol-
low the rules.
In each residency program, a group of faculty (the
Clinical Competency Committee4) is charged with twice
annually assessing each trainee across 22 unique Mile-
stones. Every Milestone is graded on a 1-to-9 scale, with
extensive descriptions of characteristics mapped to each
level on the scale and with a check box under each of the
9, so that the evaluators can pick the box that best fits
the trainee’s degree of progress. But some simple arith-
metic for a moderate to large program like ours (120
trainees) illustrates the fundamental problem: if the com-
mittee spends just 3 minutes assessing and discussing
each Milestone (and a thoughtful committee would need
Corresponding to spend more to be fair to the resident given that each
Author: Ronald M.
Witteles, MD, Stanford
Milestone contains up to 18 performance descriptors),
University School of it means an investment of 15 840 minutes per year for
Medicine, 300 Pasteur each committee member. This equates to close to 7 stan-
Dr, Lane Building 158,
dard work weeks per year of doing nothing other than
Stanford, CA
94305-5133 (witteles Milestones assessments for every member of the com-
@stanford.edu). mittee, which is obviously impractical.
jamainternalmedicine.com (Reprinted) JAMA Internal Medicine Published online September 26, 2016
Copyright 2016 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a Cornell University User on 09/26/2016
Further problems quickly become apparent. By
ACGME’s definition, “level 3” on the 1-to-9 scale is sup-
posed to reflect the expected level of an “early learner,”3
that is, the level of a new intern. Yet, descriptions of level
3 behaviors for 2 of the Interpersonal and Communica-
tion Skills Milestones include “health records are disor-
ganized and inaccurate” and “resists offers of collabora-
tive input.”3 Is this truly indicative of the skills of medical
school graduates? Furthermore, can a clinical compe-
tency committee be expected to meaningfully assess an
intern 6 months into his or her intern year across 4 sepa-
rate categories of practice-based learning, in addition to
18 other Milestones? There is a disconnect between the
educational theorists who thought this to be a good plan
and those on the ground level trying to implement the
theory. The reality is that each training program figures
out its own workaround, such that the necessary boxes
can be checked without creating an unfeasible system.
Some creative workarounds have already been
developed,5 designed to fit the square peg of more
sensible evaluation systems into the round hole of the
labyrinthian Milestones requirements.
Recent evidence suggests that clinical competency
committees are taking the expedient (and practical) path
for evaluations, largely checking the boxes based on
where trainees are supposed to be rather than based on
actual meaningful performance data.6 In some cases,
even the ability to make assessments stretches credu-
lity; did the committees have adequate data to evaluate
90% of interns for the Learns and Improves via Perfor-
mance Audit Milestone, as happened in the first year of
Milestone implementation?6 Even more concerning—if
the data are believable—is that residents transitioning
from the first to the second year of training averaged only
a 5 on the 1-to-9 scale for Interpersonal and Communi-
cation Skills Milestone. This level includes the descrip-
tors, “inconsistently engages in collaborative com-
munication,” “health records…miss key data or fail to
communicate clinical reasoning,” and “requires guid-
ance or assistance to engage in communication with per-
sons of different socioeconomic and cultural
backgrounds.”6 Far more likely is that committees are
choosing level 5 for end-of-the-year interns simply be-
cause level 5 is where entering second-year residents are
expected to be.
Training programs are being judged by the ACGME
on the basis of how trainees score on the Milestones.
According to the ACGME, “the Milestone data will be
used as formative assessment of the quality of residency/
fellowship programs.”2 A perverse incentive is there-
fore in place to tick the box in the direction of every
E1
 Opinion Viewpoint

resident meeting every Milestone, particularly because the Mile-
stones themselves are likely seen as being impractical in the first
place.
The validity of applying Milestones to every trainee in every field
is also of concern. Does a fellow pursuing a 1-year interventional car-
diology fellowship really need to focus on 4 subcompetencies of sys-
tems-based practice, or should he or she be focused on attaining the
procedural expertise required to perform complex coronary inter-
ventions and transcatheter valve replacements in the single year al-
lotted to this training? By definition, an interventional cardiology fel-
low has already met expectations across all 22 Milestones during both
residency and fellowship training.
In our experience, many residents either have not heard of or
have a poor understanding of the Milestones. Indeed, even the 6 core
competencies—more than a decade after their implementation—
remain mysterious to many trainees and to the faculty who are do-
ing the evaluating. Ask a resident to define the difference between
practice-based learning and systems-based practice, and be pre-
pared for eyes to glaze over. Going from a manageable (if still some-
what confusing) 6 core competencies to 22 Milestones assess-
ments has only increased the confusion.
The recent experience with the American Board of Internal
Medicine’s Maintenance of Certification program is instructive.7,8 Al-
though the theory seemed sound to those who formulated the re-
quirements, the implementation of the program frustrated clini-
cians. Many viewed the system as meaningless and expensive
without any clear evidence that it made them better physicians who
delivered better care.7,8 The groundswell of discontent led to a re-
volt and to fundamental changes to the program.8 There are clear
parallels with the Milestones program: the burden on program di-
rectors and the near-impossibility of execution are apparent and fre-
quently discussed. For now, however, programs may be unwilling
to push back at the very council that accredits them and allows them
to exist.
It is time to ask the fundamental question about Milestones:
Why are we doing this? It is the same question to be asked about
work-hour rules and other such changes introduced into the train-
ing of physicians. Theories of learning and assessment, like all
theories, are a good place to start an experiment—but hands-on
experience and reports from the field are necessary feedback for
theories to be validated. A simplified assessment of proficiency in
a small number of core skill sets (often called “entrustable profes-
sional activities”) would likely prove to be more honest, efficient,
and accurate—with a fraction of the bureaucracy. Three years into
the present experiment, the conclusion is clear: the Milestones are
an administrative millstone. The ACGME should restore its prom-
ise to help program directors mentor residents rather than man-
age programs.

ARTICLE INFORMATION
Published Online: September 26, 2016.
doi:10.1001/jamainternmed.2016.5552
Conflict of Interest Disclosures: None reported.
REFERENCES
1. Nasca TJ, Philibert I, Brigham T, Flynn TC. The
next GME accreditation system—rationale and
benefits. N Engl J Med. 2012;366(11):1051-1056.
2. Accreditation Council for Graduate Medical
Education (ACGME). Frequently asked questions:
milestones. http://www.acgme.org/Portals/0
/MilestonesFAQ.pdf. Accessed August 15, 2016.
3. Accreditation Council for Graduate Medical
Education; American Board of Internal Medicine.
The Internal Medicine Milestone Project. http:
//www.acgme.org/portals/0/pdfs/milestones
/internalmedicinemilestones.pdf. Accessed August
15, 2016.
4. Andolsek K, Padmore J, Hauer KE, Holboe E.
Clinical competency committees: a guidebook for
programs. http://www.acgme.org/Portals/0
/ACGMEClinicalCompetencyCommitteeGuidebook
.pdf. Accessed August 15, 2016.
5. Choe JH, Knight CL, Stiling R, Corning K, Lock K,
Steinberg KP. Shortening the miles to the
Milestones: Connecting EPA-based evaluations to
ACGME Milestone reports for internal medicine
residency programs. Acad Med. 2016;91(7):943-950.
6. Hauer KE, Clauser J, Lipner RS, et al. The internal
medicine reporting Milestones: cross-sectional
description of initial implementation in US
residency programs [published online May 10,
2016]. Ann Intern Med. doi:10.7326/M15-2411.
7. Loscalzo J. Maintenance of certification: good
intentions gone awry. Trends Cardiovasc Med. 2015;
25(4):312-314.
8. Eichenwald K. To the barricades! the doctors’
revolt against ABIM is succeeding! Newsweek.
September 15, 2015. http://www.newsweek.com
/abim-american-board-internal-medicine-doctors
-revolt-372723. Accessed October 15, 2015.

E2 JAMA Internal Medicine Published online September 26, 2016 (Reprinted) jamainternalmedicine.com

Copyright 2016 American Medical Association. All rights reserved.

Downloaded From: http://archinte.jamanetwork.com/ by a Cornell University User on 09/26/2016