Archives of Physical Medicine and Rehabilitation

journal homepage: www.archives-pmr.org

Archives of Physical Medicine and Rehabilitation 2015;96(8 Suppl 3):S173-7

INTRODUCTION

Traumatic Brain Injury Rehabilitation Comparative

Effectiveness Research: Introduction to the Traumatic
Brain InjuryePractice Based Evidence Archives
Supplement
Susan D. Horn, PhD,a John D. Corrigan, PhD,b Marcel P. Dijkers, PhDc
From the aInstitute for Clinical Outcomes Research, International Severity Information Systems, Salt Lake City, UT; bDepartment of Physical
Medicine and Rehabilitation, Ohio State University, Columbus, OH; and cDepartment of Rehabilitation Medicine, Icahn School of Medicine at
Mount Sinai, New York, NY.
Current affiliation for Horn, Department of Population Health Sciences, University of Utah School of Medicine, Salt Lake City, UT.

Abstract
This supplement of the Archives of Physical Medicine and Rehabilitation is devoted to the Traumatic Brain InjuryePractice Based Evidence
study, the first practice-based evidence study, to our knowledge, of traumatic brain injury rehabilitation. The purpose of this preface is to
place this study in the broader context of comparative effectiveness research and introduce the articles in the supplement.
Archives of Physical Medicine and Rehabilitation 2015;96(8 Suppl 3):S173-7
ª 2015 by the American Congress of Rehabilitation Medicine

This supplement of the Archives of Physical Medicine and Reha-
bilitation offers an initial set of results and analyses of the TBI-
PBE study, funded by the National Institutes of Health, National
Institute on Disability and Rehabilitation Research, and Ontario
Neurotrauma Foundation. It is the first practice-based evidence
study, to our knowledge, of traumatic brain injury (TBI) rehabili-
tation. Practicing therapists provided detailed documentation of
therapy sessions, and this information was combined with medical
record abstracted data, downloaded medication data, surveys of
patients’ postdischarge outcomes, and clinician profiles of their
An audio podcast accompanies this article.
Listen at www.archives-pmr.org.
Supported by the National Institutes of Health, National Center for Medical Rehabilitation
Research (grant no. 1R01HD050439-01); National Institute on Disability and Rehabilitation
Research (grant no. H133A080023); and Ontario Neurotrauma Foundation (grant no. 2007-ABI-
ISIS-525).
The opinions contained in this article are those of the authors and should not be construed as an
official statement from the National Institutes of Health, National Center for Medical Rehabilitation
Research; National Institute on Disability and Rehabilitation Research; or Ontario Neurotrauma
Foundation.
Publication of this article was supported by the American Congress of Rehabilitation
Medicine.
Disclosures: none.
training and experience in rehabilitation, in general, and with TBI
specifically. The purpose of this preface is to place this study in the
broader context of comparative effectiveness research (CER) and
introduce the articles in the supplement, which offer conclusions
based on what we believe is the richest dataset on TBI rehabili-
tation ever assembled.
The Problem
The Centers for Disease Control and Prevention (CDC) estimated
that in 2010 approximately 2.5 million people in the United States
sustained a TBI.1 Of these, 2.2 million had emergency department
visits, 280,000 were hospitalized and survived, and 52,000 died.1,2
The Brain Injury Association of America and researchers at the
CDC estimated that 3.1 to 5.3 million people in the United States
live with long-term disability arising from physical, emotional, or
cognitive consequences of TBI.3-5 Although the personal and fa-
milial consequences of TBI are devastating, so are the financial
implications. In 2010, the CDC estimated that the annual cost to
society of TBI for direct medical care, rehabilitation, and indirect
costs (eg, lost productivity) totaled >$76.5 billion, with 90% of
these costs attributed to severe injury.6

0003-9993/15/$36 - see front matter ª 2015 by the American Congress of Rehabilitation Medicine
http://dx.doi.org/10.1016/j.apmr.2015.03.027
S174 S.D. Horn et al

Inpatient rehabilitation after TBI was started experimentally in .designed to inform health-care decisions by providing
the 1970s, but it now is received by growing numbers, not just evidence on the effectiveness, benefits, and harms of different
adolescents and young adults who were the recipients originally, treatment options. The evidence is generated from research
but increasingly by adult and older adult patients. Cuthbert et al7 studies that compare drugs, medical devices, tests, surgeries, or
recently reported that from 2008 to 2010, on average, 20,000 ways to deliver health care. Comparative effectiveness research
people in the United States aged >15 years received rehabilitation requires the development, expansion, and use of a variety of data
for a primary diagnosis of TBI. sources and methods to conduct timely and relevant research and
Although there have been hundreds of publications on aspects of disseminate the results in a form that is quickly usable by clini-
TBI inpatient rehabilitation, the process is still largely a black box: cians, patients, policymakers, and health plans and other payers.16
we do not know what services are being offered, with what results,
The Institute of Medicine has a similar definition.17
if they are individually or in combination, and how this might differ
Most existing medical research focuses on benefits or harms of
by severity of injury, nature and seriousness of patient deficits, or
new medications or other single-component interventions, using
general social, psychological, and health characteristics of patients.
randomized controlled trial (RCT) designs involving an academic
Substantial attention has been given to describing etiologies of TBI
researcher (rather than community-active clinicians), a small ho-
and characterizing its consequences. Far less effort has been paid to
mogeneous sample of patients, and (often) outcomes that are a
studying interventions, particularly in acute rehabilitation. The 1999
proxy for the real-world outcomes in which patients are interested.
National Institutes of Health consensus statement8 recommended
The results often are not applicable to many patients with the
that rehabilitation programs for persons with moderate and severe
specified condition of interest. These explanatory studies (also
TBI be interdisciplinary and comprehensive. However, the authors
known as efficacy studies) differ in a number of very important
of this statement admitted that scientific evidence for effectiveness
aspects from CER (known as effectiveness studies).18 Briefly, RCT
of comprehensive rehabilitation was limited to uncontrolled studies
study patients usually have no or just minor comorbidities in
and 1 nonrandomized controlled trial.8 A more recent review of TBI
addition to the index disorder being studied, in contrast with many
studies concluded that more research is needed to identify which
patients being treated by primary care physicians or most specialists.
interdisciplinary rehabilitation programs are promising practices
In RCTs, interventions must adhere exactly to the treatment regimen
and provide the greatest improvement in outcomes and which pa-
(protocol) or the participant will be considered nonevaluable, in
tient subgroups benefit the most from various forms of interdisci-
contrast with routine care in which patients are prescribed a treat-
plinary rehabilitation.9
ment regimen but may not follow instructions meticulously. RCTs
Over a decade ago, Chesnut et al conducted a systematic re-
are powered to test a single (usually short-term) outcome selected as
view of evidence available to answer several questions related to
the primary outcome, in contrast with real-world care in which
interventions for TBI, including whether intensity of acute inpa-
longer-term outcomes and >1 outcome are of interest to physicians
tient rehabilitation is related to outcome.10 They lamented the lack
and their patients. The result of this highly controlled but narrowly
of high-quality research, finding only 1 quasi-experimental com-
focused research is that patients and clinicians still do not have
parison of inpatient rehabilitation effectiveness and 4 observa-
answers to questions (eg, Does it work for someone with my health
tional studies of intensity effects.10-14 Chesnut concluded that the
issues? Is it any better than the treatment I am currently receiving?
single quasi-experimental study of effectiveness suggested that
What are the long-term effects? What are the side effects and
acute rehabilitation may make a difference; however, there was
likelihood and seriousness of each side effect?).19
little evidence that therapeutic intensity, as measured by hours of
The CER approach addresses these limitations in a number of
treatment, is related to this beneficial effect. Spontaneous recovery
ways, including conducting systematic reviews and comparing 2
after TBI is known to be significant and confounds conclusions in
treatments for their effectiveness. Systematic reviews analyze
this area. As a possible explanation for this lack of relation be-
existing research reports and determine whether there is information
tween intensity of treatment and outcomes, Chesnut observed that
relevant for treatment decision-making. These reviews, however,
specific impairments and comorbidities were not taken into ac-
may be limited by existing research, mostly RCT-type studies,
count. Results from a practice-based evidence study of stroke
where only 1 treatment is compared at a time with routine care or a
rehabilitation that took into account impairments and comorbid-
placebo often for a short duration of time using a proxy for the
ities found that hours of treatment per week did not predict out-
desired outcome. In contrast, CER primary research evaluates
comes; however, accounting for time spent doing specific
multiple treatments for outcomes that are relevant to a more typical,
activities per week (eg, gait, upper-extremity control, problem-
real-world patient population. This approach involves new, pro-
solving) significantly improved prediction of outcomes.15
spective data collection or mining of administrative and clinical
databases using new statistical techniques (eg, propensity scores,
instrumental variables, severity of illness measures) to address se-
CER on TBI rehabilitation lection bias and confounding by lack of comparability of patients
who receive different treatments. Practice-based evidence studies
CER has received much attention recently. The Agency for
are a type of CER that allows comparison of treatment differences
Healthcare Research and Quality describes CER as:
that arise naturally in the course of treatment provided in different
facilities by diverse professionals.
List of abbreviations:
CDC
CER
Centers for Disease Control and Prevention
comparative effectiveness research
Practice-based evidence study
RCT randomized controlled trial methodology for TBI rehabilitation
TBI traumatic brain injury
Rehabilitation is an example of a complex intervention with many
TBI-PBE Traumatic Brain InjuryePractice Based Evidence
different components delivered in an individualized manner;

www.archives-pmr.org
Preface to Traumatic Brain Injury Supplement
therefore, it is impossible to use a single RCT, or even a series of
successive RCTs, to determine the optimal timing, duration,
content, and combination of all the therapies that may be used.
Most previous rehabilitation research on interventions or man-
agement strategies has been based on retrospective data abstracted
from clinical and administrative records and contains only those
data elements that are readily available and exportable from
rehabilitation center medical record systems. Data are aggregated
to achieve larger samples, as is done in the TBI Model Systems’
National Database.20 However, the medical record often includes
only procedure codes used for billing and does not contain mea-
sures designed to capture the active ingredients of treatments
provided by various disciplines. In addition, the observational
research in rehabilitation that has been published commonly lacks
information on comparators and often uses weak, incomplete, and/
or untested outcome measures that happen to be available.19
Practice-based evidence research has addressed the shortcom-
ings of previous research by developing a methodology that can
handle multiple differences in rehabilitation programs resulting
from a large number of therapists, at diverse sites, customizing
many aspects of their treatment program, and treating patients
with various levels of severity of illness and injury.18 Practice-
based evidence studies in rehabilitation (eg, stroke,15 orthopedic
joint replacement,21 spinal cord injury22) have the following
characteristics19: (1) they are prospective, multisite observational
studies that capture practice variations existing both between and
within sites; (2) sites enroll large numbers of consecutively
admitted patients, which permit analysis of subgroups of patients
and cross-validation of findings; (3) treatment data are either
abstracted from the medical record or provided by point of care
documentation of therapies that are given during each treatment
session (additional data may be collected through downloads of,
for example, medication information; (4) outcome data are
abstracted from the medical record, supplemented with informa-
tion obtained from patient interviews after various spans of time
postdischarge; (5) clinicians (organized into a clinical project
team) and former patients are involved in design of the study,
including creating the study design, developing the content of
point of care documentation forms to describe the care that they
provide, and generating content of the follow-up interview; (6)
analysis of this comprehensive database is led by the clinicians
and former patients who helped design the study and contributed
data on rehabilitation treatments administered, with a focus on
basic effectiveness questions (eg, which type of treatment,
administered for how long, during what phase of the patient’s
admission, produces the best outcomes?); and (7) because clini-
cians and former patients are involved in analysis and reporting,
this partially closes the circle of knowledge translation: clinicians
and former patients are involved in initial design through final
dissemination.
Various experts have argued whether practice-based evidence
studies can prove efficacy and effectiveness of treatment alterna-
tives, or whether their findings need to be confirmed by an RCT.
However one views this question, one conclusion is clear:
practice-based evidence studies are superior to RCTs when it
comes to external validity (generalizability)23: (1) placebos are
replaced by real-life treatments (eg, alternative approaches being
used by therapists); (2) narrowly defined patient samples using
multiple inclusion/exclusion criteria are replaced by a sample
including every patient who consents to be observed (and because
there are no required treatment regimens that may pose varying
risks, most patients consent); (3) surrogate outcome measures are
www.archives-pmr.org
S175
replaced by real-life outcomes (eg, function, participation, long-
term use of health-care resources); and (4) to mirror clinical re-
ality, limitations on treatment setting or minimal level of training
of therapists are not used in recruiting sites or treatment providers.
Therefore, practice-based evidence studies address all the re-
quirements of well-designed CER. Findings have the potential to
influence decisions of patients, clinicians, and administrators.
Practice-based evidence research is an outcomes-directed,
model-building methodology that has established effective and
efficient treatments in other diagnostic groups and clinical set-
tings.24,25 Strengths of the practice-based evidence approach are
the extensive array of data collected, their level of detail, and as a
result, the breadth of issues that can be examined in testing of a
priori or post hoc hypotheses. Practice-based evidence research
uses detailed descriptions of actual rehabilitation practices to
examine the relations among patient characteristics, content of
therapy, and their associations with rehabilitation outcomes. The
practice-based evidence approach does not disrupt the routines of
the treatment setting in the way an RCT does. It offers a natu-
ralistic view of rehabilitation treatment by examining what actu-
ally happens in the care process, not altering the treatment
regimen to evaluate the efficacy of a particular intervention.18
The practice-based evidence approach also offers the advan-
tage of large numbers of patients that often cannot be attained in
an RCT constrained by stringent selection criteria and/or costs
involved in identifying, treating, and measuring patients. It con-
trols for patient differences by taking into account important
clinical covariates (eg, severity of illness, injury severity, comor-
bidities, functional status). This inclusiveness allows greater
generalizability of findings.
In addition, detailed data on interventions allow researchers to
focus on the most meaningful level of resolution regarding the
types of care rendereddconsistent with current knowledge and
insights offered by clinical participants. Therefore, the practice-
based evidence approach can answer study questions and hy-
potheses initially at a basic level, but it also allows drilling down
into the data with the help of additional insights offered by the
interdisciplinary clinical project team.
Practice-based evidence methodology allows important statisti-
cal associations to be identified. Although causality cannot firmly
and definitely be established, alternate hypotheses regarding
possible cause and effect can be tested using the large number of
available variables to identify mediating and moderating influences
on outcomes. Results of these analyses can be used to eliminate
most alternative explanations about causality and to generate spe-
cific additional analytic questions. Because of the multivariable
methods used in practice-based evidence studies to determine fac-
tors significantly associated with outcomes, findings are based on
strong statistical association. In the past, when changes in practice
were made based on findings of a practice-based evidence study,
evaluation of the effects of the change demonstrated the correctness
of the practice-based evidence study’s conclusions.24,25
This supplement offers an initial set of results from the TBI-PBE
study. Funding by the National Institutes of Health, National Insti-
tute on Disability and Rehabilitation Research, and Ontario Neu-
rotrauma Foundation made possible what we believe is the richest
dataset on TBI rehabilitation ever assembled. The analyses reported
in this supplement only scratch the surface and hopefully will only
be the beginning of an extensive analysis and reporting process.
The TBI-PBE study enrolled 2205 individuals with TBI
receiving initial inpatient rehabilitation at 9 rehabilitation centers
across the United States and 1 in Canada over a 2.5-year time
S176
period. Three of the 10 study sites were part of the TBI Model
Systems, funded by the National Institute on Disability and
Rehabilitation Research. This 6-year study developed treatment
taxonomies for each rehabilitation discipline and used paper and
web-based electronic data capture systems to document the in-
terventions occurring each time a clinician had a treatment session
with a consented patient, or a nurse interacted with or on behalf of
the patient. This resulted in >1000 clinicians providing detailed
information about hundreds of specific treatment activities in
>350,000 encounters/treatment sessions. Extensive medical re-
cord abstracting documented patient and injury characteristics,
severity of principal and comorbid conditions, and ancillary
treatments (eg, medications, nutritional support) to augment in-
formation collected on therapy sessions. In addition, in-depth
follow-up interviews (with patients or, where appropriate, their
proxies) were conducted at 3 and 9 months postdischarge to
capture outcomes in the first year postinjury for most participants.
This supplement includes 12 articles, starting with an intro-
ductory article describing in detail how the practice-based evi-
dence methodology was applied in this investigation of TBI
rehabilitation and the measures and methods used in the project.
The subsequent articles aim to answer 1 of the following
questions: What patient and injury characteristics contributed to
patient outcomes? What clinical events occurred in the course of
treatment that may have mediated outcomes? What treatments
were used in response to what clinical problems? Where various
treatment options were available, which one(s) is best, as
demonstrated by superior outcomes achieved (taking into account
TBI severity and other patient deficits and strengths that may
affect outcomes)? We believe the findings will provide guidance in
developing guidelines for clinical decision-making and other
evidence-based practices.
Keywords
Brain injuries; Comparative effectiveness research; Rehabilitation
Corresponding author
Susan D. Horn, PhD, University of Utah School of Medicine,
Department of Population Health Sciences, Health System Inno-
vation and Research Program, Williams Bldg, Rm 1N461, 295
Chipeta Way, Salt Lake City, Utah 84108. E-mail address: susan.
horn@hsc.utah.edu.
Acknowledgments
We thank the clinical and research staff at each of the 10 inpatient
rehabilitation facilities represented in the study for their contri-
butions. The study center directors included the following: John
D. Corrigan, PhD, and Jennifer Bogner, PhD (Department of
Physical Medicine and Rehabilitation, Ohio State University,
Columbus, OH); Nora Cullen, MD (Toronto Rehabilitation Insti-
tute, Toronto, ON, Canada); Cynthia L. Beaulieu, PhD (Brooks
Rehabilitation Hospital, Jacksonville, FL); Flora M. Hammond,
MD (Carolinas Rehabilitation, Charlotte, NC [now at Indiana
University]); David K. Ryser, MD (Neuro Specialty Rehabilitation
Unit, Intermountain Medical Center, Salt Lake City, UT); Murray
E. Brandstater, MD (Loma Linda University Medical Center,
Loma Linda, CA); Marcel P. Dijkers, PhD (Mount Sinai Medical
S.D. Horn et al
Center, New York, NY); William Garmoe, PhD (Medstar National
Rehabilitation Hospital, Washington, DC); James A. Young, MD
(Physical Medicine and Rehabilitation, Rush University Medical
Center, Chicago, IL); and Ronald T. Seel, PhD (Crawford
Research Institute, Shepherd Center, Atlanta, GA). We also thank
Michael Watkiss for contribution in data collector training and
support during data collection and Randall J. Smout for contri-
bution in data cleaning and analysis.
References
1. Centers for Disease Control and Prevention. Traumatic brain injury in
the United States: fact sheet. 2014. Available at: http://www.cdc.gov/
TraumaticBrainInjury/get_the_facts.html. Accessed March 6, 2014.
2. Faul M, Xu L, Wald MM, Coronado VG. Traumatic brain injury in the
United States: emergency department visits, hospitalizations, and
deaths. Atlanta: Centers for Disease Control and Prevention, National
Center for Injury Prevention and Control Centers for Disease Control;
2010.
3. Brain Injury Association of America. Brain injury facts: brain
injury statistics. 2012. Available at: http://www.biausa.org/
LiteratureRetrieve.aspx?IDZ104992. Accessed March 6, 2014.
4. Thurman D, Alverson C, Dunn K, Guerrero J, Sniezek J. Traumatic
brain injury in the United States: a public health perspective. J Head
Trauma Rehabil 1999;14:602-15.
5. Selassie AW, Zaloshnja E, Langlois JA, Miller T, Jones P, Steiner C.
Incidence of long-term disability following traumatic brain injury
hospitalization in the United States, 2003. J Head Trauma Rehabil
2008;23:123-31.
6. Finkelstein E, Corso P, Miller T, editors. The incidence and economic
burden of injuries in the United States. New York: Oxford Univ Pr;
2006.
7. Cuthbert JP, Corrigan JD, Whiteneck GG, et al. Extension of the
representativeness of the Traumatic Brain Injury Model Systems Na-
tional Database: 2001 to 2010. J Head Trauma Rehabil 2012;27:
E15-27.
8. Consensus conference. Rehabilitation of persons with traumatic brain
injury. NIH Consensus Development Panel on Rehabilitation of Per-
sons With Traumatic Brain Injury. JAMA 1999;282:974-82.
9. Brasure M, Lamberty GH, Sayer NA, et al. Multidisciplinary rehabilitation
programs for moderate to severe traumatic brain injury in adults: future
research needs: identification of future research needs from comparative
effectiveness review no. 72 [Internet]. 2013. Available at: http://www.ncbi.
nlm.nih.gov/books/NBK143629/. Accessed March 6, 2014.
10. Chesnut R, Carney N, Maynard H, Patterson P, Mann NC, Helfand M.
Evidence report: rehabilitation for traumatic brain injury. Rockville:
Agency for Health Care Policy and Research; 1999. AHCPR Pub No
99-E005.
11. Aronow HU. Rehabilitation effectiveness with severe brain injury:
translating research into policy. J Head Trauma Rehabil 1987;2:
24-36.
12. Heinemann AW, Hamilton B, Linacre JM, Wright BD, Granger C.
Functional status and therapeutic intensity during inpatient rehabili-
tation. Am J Phys Med Rehabil 1995;74:315-26.
13. Blackerby WF. Intensity of rehabilitation and length of stay. Brain Inj
1990;4:167-73.
14. Spivack G, Spettell CM, Ellis DW, Ross SE. Effects of intensity of
treatment and length of stay on rehabilitation outcomes. Brain Inj
1992;6:419-34.
15. Horn SD, DeJong G, Smout RJ, Gassaway J, James R, Conroy B.
Stroke rehabilitation patients, practice, and outcomes: is earlier and
more aggressive therapy better? Arch Phys Med Rehabil 2005;86(12
Suppl 2):S101-14.
16. Agency for Healthcare Research and Quality. What is comparative
effectiveness research. Available at: http://www.effectivehealthcare.
www.archives-pmr.org
Preface to Traumatic Brain Injury Supplement
ahrq.gov/index.cfm/what-is-comparative-effectiveness-research1/.
Accessed August 31, 2014.
17. Sox HC, Greenfield S. Comparative effectiveness research: a report
from the Institute of Medicine. Ann Intern Med 2009;151:203-5.
18. Horn SD, DeJong G, Deutscher D. Practice-based evidence research in
rehabilitation: an alternative to randomized clinical trials and tradi-
tional observational studies. Arch Phys Med Rehabil 2012;
93(8 Suppl):S127-37.
19. Dijkers MP, Whiteneck GG, Gassaway J. CER, PBE, SCIRehab,
NIDRR, and other important abbreviations. Arch Phys Med Rehabil
2013;94(4 Suppl):S61-6.
20. Dijkers MP, Harrison-Felix C, Marwitz JH. The traumatic brain injury
model systems: history and contributions to clinical service and
research. J Head Trauma Rehabil 2010;25:81-91.
21. DeJong G, Hsieh C, Gassaway J, et al. Characterizing rehabilitation
services for patients with knee and hip replacement in skilled nursing
www.archives-pmr.org
S177
facilities and inpatient rehabilitation facilities. Arch Phys Med Rehabil
2009;90:1269-83.
22. Whiteneck GG, Gassaway J. SCIRehab uses practice-based evi-
dence methodology to associate patient and treatment character-
istics with outcomes. Arch Phys Med Rehabil 2013;94(4 Suppl):
S67-74.
23. Dijkers MP. External validity in research on rehabilitation in-
terventions: issues for knowledge translation. Austin: SEDL, National
Center for the Dissemination of Disability Research; 2011. Focus
Technical Brief 33.
24. Hudak S, Sharkey S, Engleman M, Horn SD, Spector W,
Limcangco R. Pressure ulcer plan is working. Provider 2008;34:34-9.
25. Olsho LE, Spector WD, Williams CS, Rhodes W, Fink RV,
Limcangco R, Hurd D. Evaluation of AHRQ’s on-time pressure ulcer
prevention program: a facilitator-assisted clinical decision support
intervention for nursing homes. Med Care 2014;52:258-66.