Journal of Critical Care 39 (2017) 220224

Contents lists available at ScienceDirect

Journal of Critical Care

journal homepage: www.jccjournal.org

Sepsis/Infection

Filtering authentic sepsis arising in the ICU using administrative codes

coupled to a SIRS screening protocol
Christopher L. Sudduth, MD, Elizabeth C. Overton, MSPH, Peter F. Lyu, MSPH,
Ramzy H. Rimawi, MD, Timothy G. Buchman, PhD, MD
Critical Care Center, Emory University School of Medicine, Emory Healthcare, Atlanta, GA, United States

a r t i c l e i n f o a b s t r a c t

Keywords: Purpose: Using administrative codes and minimal physiologic and laboratory data, we sought a high-specicity
Sepsis identication strategy for patients whose sepsis initially appeared during their ICU stay.
Systemic inammatory response syndrome Materials and methods: We studied all patients discharged from an academic hospital between September 1, 2013
Epidemiology
and October 31, 2014. Administrative codes and minimal physiologic and laboratory criteria were used to iden-
Administrative codes
tify patients at high risk of developing the onset of sepsis in the ICU. Two clinicians then independently reviewed
Detection
Intensive care unit the patient record to verify that the screened-in patients appeared to become septic during their ICU admission.
Results: Clinical chart review veried sepsis in 437/466 ICU stays (93.8%). Of these 437 encounters, only 151
(34.6%) were admitted to the ICU with neither SIRS nor evidence of infection and therefore appeared to become
septic during their ICU stay.
Conclusions: Selected administrative codes coupled to SIRS criteria and applied to patients admitted to ICU can
yield up to 94% authentic sepsis patients. However, only 1/3 of patients thus identied appeared to become septic
during their ICU stay. Studies that depend on high-intensity monitoring for description of the time course of sep-
sis require clinician review and verication that sepsis initially appeared during the monitoring period.
2017 Elsevier Inc. All rights reserved.

1. Background administrative data typically include physician claims, insurance claims,

Sepsis is the life-threatening inammatory response to infection [1].
In 2011, septicemia ranked as the most expensive condition for all Unit-
ed States hospitalizations accounting for 20.3 billion dollars [2]. Sepsis
frequently causes inpatient death [3]. Survivors fare poorly: 26% require
readmission within 30 days and 48% within 180 days [4,5]. Almost half
of all patients who survive severe sepsis (sepsis accompanied by organ
failure) to hospital discharge die within the following year [6]. Early de-
tection and treatment of sepsis is therefore important in preventing de-
compensation to organ failure.
Large-scale epidemiologic studies of sepsis often leverage adminis-
trative databases to retrospectively identify patients [7-11]. Those
Abbreviations: ICD-9-CM, International Classication of Diseases, Ninth Revision,
Clinical Modication; MS-DRG, Medicare Severity Diagnosis Related Group; ICU,
intensive care unit; SD, standard deviation; IQR, interquartile range.
Corresponding author at: Emory University, Robert W. Woodruff Health Sciences
Center, 1440 Clifton Road NE, 313A, Atlanta, GA 30322, United States.
E-mail addresses: sudduth@ohsu.edu (C.L. Sudduth), elizabeth.overton@emory.edu
(E.C. Overton), peter.lyu@emoryhealthcare.org (P.F. Lyu), ramzyrimawi@emory.edu
(R.H. Rimawi), tbuchma@emory.edu (T.G. Buchman).
hospital discharge and emergency visit records encoded using the Inter-
national Classication of Diseases, Ninth Revision, Clinical Modication
(ICD-9-CM) and Medicare Severity Diagnosis Related Group (MS-DRG)
codes. The specic combinations of these codes used to identify sepsis
and severe sepsis vary. Two selections of administrative codes are com-
monly used: the Angus set and the Martin set [7,8]. Administrative code
selection strategies yield inconsistent results: the incidence of severe
sepsis varies by up to 3.5 fold depending on the details of the codes se-
lected [10]. The correspondence of populations selected via administra-
tive codes with populations selected as septic from a chart review
performed by clinical experts is also poor [12]. The need for a standard
and reliable method to retrospectively identify a group of patients
with authentic sepsis is apparent [13]. Furthermore, with the increasing
interest in early detection of sepsis in the ICU, a protocol that identies
the subset of the cohort that develops sepsis in the ICU is needed [14,
15].
We sought a standardized, high-specicity strategy to accurately
identify a cohort of patients whose sepsis initially appears during their
intensive care unit (ICU) stay. In addition, we aimed to describe demo-
graphic and mortality data for said cohort. Our intent was to propose a

http://dx.doi.org/10.1016/j.jcrc.2017.01.012
0883-9441/ 2017 Elsevier Inc. All rights reserved.
 C.L. Sudduth et al. / Journal of Critical Care 39 (2017) 220224
methodology for rapidly yet effectively identifying patient records ap-
propriate for subsequent analysis towards features that would herald
sepsis before that condition became obvious.
2. Methods
2.1. Ethical approval, study design, search strategy
This study was conducted under the approval of the Emory Univer-
sity Institutional Review Board. In this work, we retrospectively studied
6 ICUs (1 27-bed neurosurgical, 1 20-bed general surgical, 2 7-bed
medical, and 2 9-bed cardiothoracic) at Emory University Hospital, a
605-bed acute care teaching facility, between September 1, 2013 and
October 31, 2014. The search strategy began with all 24,323 hospital dis-
charges that occurred during this timeframe. The screening protocol
was applied to only ICU patients and was comprised of 2 components:
(1) administrative codes and (2) physiologic and laboratory criteria.
The administrative codes used in the screening protocol are listed in
Fig. 1. This code set included a highly rened set of ICD-9-CM and MS-
DRG codes with components of both the Angus set and the Martin set
[7,8,16]. These were also enhanced by 3 codes that did not appear in
either the Angus or Martin sets. Details are given in Fig. 1.
The physiologic eligibility criteria ltered the selected encounters to
include only those that met at least 2 of the 4 pre-dened SIRS criteria
within a 4-hour moving window (temperature N 38 C or b36 C,
heart rate N 90 beats/min, respiratory rate N 20 breaths/min, and/or
white blood cell count N 12.000/L or b 4000/L). Further, the patient
had to meet SIRS criteria for the rst time while in the ICU during a
given hospitalization. These criteria were developed based on the orig-
inal 1991 American College of Chest Physicians (ACCP) and the Society
of Critical Care Medicine (SCCM) consensus denitions for sepsis [1].
Four hours was deemed to be a sufcient interval such that septic
Fig. 1. Venn diagram for administrative codes used to dene sepsis.
221
manifestations could be adequately captured based upon frequencies
of vital sign measurements and blood draws.
2.2. Chart review
Two clinicians independently reviewed each patient's chart. First,
clinicians attempted to identify clinical changes or physiologic or labo-
ratory events that heralded sepsis. Such elements included clinical
worsening (such as respiratory distress dened as respiratory
rate N 20 breaths/min or increased ventilator requirements and hemo-
dynamic instability dened as heart rate N 90 beats/min, blood
pressure b 90 mm Hg systolic or increased vasopressor requirements)
and laboratory indications (such as leukocytosis dened as white
blood cell count N 12.000/L or b 4000/L and lactic acidosis dened as
lactic acid 4.0 mmol/L) that are commonly accepted to signal sepsis
and associated decompensation of vital organ systems. The earliest indi-
cation was termed the acute event. Other aspects of the patient chart,
specically positive culture data and diagnostic imaging reports con-
taining ndings pointing to a localized infection (for example, new
and localized inltrate on a chest radiograph) were reviewed to deter-
mine the most likely cause of clinical worsening. Two parameters relat-
ing to the acute event were then subjectively scored on a 10-point scale:
(1) likelihood the acute event occurred rst in the ICU and (2) likeli-
hood the acute event occurred due to an infectious etiology. The scales
for these parameters are described in Fig. 2.
2.3. Scoring and assignment
Analysis was restricted to those who met SIRS criteria in the ICU (not
prior to the ICU). Those patients were assigned one of three labels. Pa-
tients were labeled as non-septic if the record failed to yield a clinical
impression conrming sepsis. The subjective score for likelihood the
222 C.L. Sudduth et al. / Journal of Critical Care 39 (2017) 220224

Fig. 2. Scales for different parameters evaluated during chart review.

acute event occurred due to an infectious etiology was used to validate
this categorization criterion. The second label was septic non-ICU and
identied patients with authentic sepsis with onset outside of their ICU
stay. To be labeled as septic non-ICU, patients had to record a score of
7 or less for the subjective score likelihood acute event occurred in the
ICU. The third label was septic ICU and identied patients with au-
thentic sepsis with onset during their ICU stay. This captured the re-
mainder of patients. These patients (1) had authentic sepsis and (2)
were given an 8 or higher for their likelihood acute event occurred in
the ICU score. When discrepancies in scoring or assignment were en-
countered, patient charts were reviewed in detail until an agreement
could be reached (CS, RR and TB).
3. Results
A total of 24,323 encounters were discharged from the 605 hospital
beds between September 1, 2013 and October 31, 2014 (Fig. 3). The ini-
tial physiologic exclusion criteria resulted in 466 encounters that (1) re-
ceived an administrative code and (2) met 2 of 4 SIRS criteria initially in
the ICU, and thus these 466 encounters constituted the initial study
population. Of the 466 encounters with administratively coded sepsis
and valid SIRS physiology, clinical chart review identied sepsis in 437
encounters (93.8%). In other words, the rened code set yielded a 6%
error rate. In only 151/437 (34.6%) encounters (Fig. 3) the onset of sep-
sis occurred while the patient was in the ICU. Encounter characteristics,

Fig. 3. Patient selection ow diagram.

 C.L. Sudduth et al. / Journal of Critical Care 39 (2017) 220224 223

Table 1
Encounter characteristics.

All study encounters Non-septic encounters Encounters with veried sepsis

(n = 466) (n = 29)
All Septic non-ICU Septic ICU
(n = 437) (n = 286) (n = 151)

Age, years, mean (SD) 58 (17) 56 (16) 58 (17) 58 (17) 58 (17)

Male, n (%) 266 (57) 14 (48) 252 (58) 161 (56) 91 (60)
White, n (%) 259 (56) 12 (41) 247 (57) 164 (57) 83 (55)
Admitted from non-healthcare facility 185 (40) 16 (55) 169 (39) 105 (37) 64 (42)
Culture collected or antibiotics administered within 24 h of rst SIRS criteria 462 (99) 29 (100) 433 (99) 286 (100) 147 (97)

ICU, intensive care unit; SD, standard deviation; SIRS, systemic inammatory response syndrome.

chart review data and outcome measures are summarized in Tables 1, 2
and 3.
For the 437 patients identied as septic via the rened administra-
tive codes, SIRS criteria, and chart review, overall mortality was 19%.
This was uniform among the non-septic (21%), septic non-ICU (17%),
and septic ICU (22%) groups.
4. Discussion
The purpose of this effort was to evaluate a protocol that could rap-
idly and reliably identify a small cohort of patients whose sepsis initially
appeared during their ICU stay. These patients represent a rst cohort
for precision medicine: early identication and rapid, tailored response
to this life-threatening condition. We found that we could reliably iden-
tify a patient population with sepsis by coupling a rened code set with
SIRS criteria. Manual review revealed a 6% error rate. The patients erro-
neously identied as septic were nevertheless critically ill. Among the
patients with authentic sepsis, only about 1/3 became septic in the
ICU. Regardless of the time or location of sepsis onset, in-hospital mor-
tality remained high at 22%.
Our ndings are consistent with prior reports. Stringent application
of screens based on administrative codes yield positive predictive values
of 88.9% and 70.7% for the Martin and Angus sets, respectively [8,17].
We do not claim improvement as our screen was intended to increase
specicity at the expense of sensitivity. Indeed, only 34.6% (151/437)
had an onset of sepsis while in the ICU. This corresponds to 10.8% of all ad-
ministratively coded septic patients who received ICU care and only
6.3% of administratively coded septic patients in any hospital setting.
Again, our purpose was not to capture all septic patients but rather to
identify a cohort of patients arriving in the ICU without clinical evidence
of sepsis and becoming septic during that ICU stay. Additionally, by
selecting for septic patients in the ICU, our protocol is in line with the re-
cently published third international consensus denition for sepsis [18].
This new denition recommends clinicians (and investigators) limit the
use of the word sepsis to only those patients with evidence of life-
threatening organ dysfunction.
Of note, 100% (29/29) of encounters deemed non-septic per chart re-
view had a blood culture drawn or an antibiotic administered within
24 h of meeting SIRS criteria. Clinician notes for these patients
documented no concern for sepsis. It appeared that most of these pa-
tients had acute events that triggered a change in physiologic signals
to meet SIRS criteria, but the clinicians did not attribute these perturba-
tions to sepsis. In each one of the 29 cases an alternative diagnosis ade-
quately explained the patient's clinical picture, the most common being
cardiogenic shock.
This study has several limitations. It was a single institution explor-
atory analysis covering a 14-month period. The methodology excluded
septic ICU patients that failed to meet our sampling criteria. No
attempt was made to estimate the number of excluded patients. We
have presented a tool towards identifying a research cohort with a
low false-positive rate at the expense of specicity. The tool should
therefore not be used for population-wide predictions. In addition,
our initial screening protocol and chart review methodology were
novel. No claim is made regarding external validity. While it is possi-
ble our protocol failed to capture the entire population of septic pa-
tients and while it is possible that our protocol failed to accurately
identify the time of sepsis onset, analysis of the cohort suggests that
it is close to complete and sufciently precise to ask more detailed
questions aimed at early detection and intervention, which is central
to modern sepsis care. We also note that several patients had multiple
acute events during a single encounter and as such the acute events
marked by coders as septic and the acute events reviewed by clini-
cians retrospectively may not represent the same events. These limita-
tions aside, when screening for patients with septic physiology rst
manifesting in the ICU, we can now estimate that approximately 1/3
of patients thus screened actually developed their sepsis in the ICU
and do not seem to have outcomes better than patients who devel-
oped their sepsis outside of the ICU.
5. Conclusions
Coupling a rened administrative code set with SIRS criteria screens
for authentic sepsis yielded a 93.8% positive predictive value. Errors
arose from shock states subsequently attributed to non-septic etiolo-
gies. Approximately 1/3 of patients screened manifested sepsis for the
rst time during (not prior to) the ICU admission. This subpopulation
is of signicant importance to studies using high-intensity monitoring
for early detection and targeted treatment of sepsis.

Table 2
Chart review results.

All study encounters Non-septic encounters Encounters with veried sepsis

(n = 466) (n = 29)
All (n = 437) Septic non-ICU (n = 286) Septic ICU (n = 151)

Acute event occurred due to infection

Likelihood, median (IQR) 8 (510) 1 (12) 9 (610) 9 (610) 8 (610)
Likelihood 8, n (%) 290 (62) 0 (0) 290 (66) 187 (65) 103 (68)
Acute event occurred in ICU
Likelihood, median (IQR) 5 (110) 2 (110) 5 (19) 2 (15) 10 (910)
Likelihood 8, n (%) 160 (34) 8 (28) 152 (35) 1 (0) 151 (100)

ICU, intensive care unit; IQR, interquartile range.

224 C.L. Sudduth et al. / Journal of Critical Care 39 (2017) 220224

Table 3
Encounter outcomes.

All study encounters Non-septic encounters Encounters with veried sepsis

(n = 466) (n = 29)
All (n = 437) Septic non-ICU (n = 286) Septic ICU (n = 151)

Length of stay
ICU LOS, days, median (IQR) 6 (315) 3 (25) 7 (315) 5 (211) 13 (621)
Hospital LOS, days, median (IQR) 14 (725) 8 (414) 14 (825) 11 (722) 19 (1129)
Discharge disposition
Expired, n (%) 89 (19) 6 (21) 83 (19) 50 (17) 33 (22)
Hospice, n (%) 66 (14) 2 (7) 64 (15) 38 (13) 26 (17)

ICU, intensive care unit; IQR, interquartile range.

Conicts of interest
None.
Funding
This work was supported by the Emory University Center for Critical
Care; and the Surgical and Critical Care Initiative. The funding sources
had no role in study design, data collection, data analysis, data interpre-
tation, in writing the report, nor in the decision to submit the article for
publication.
Acknowledgements
The authors acknowledge the assistance of Qiao Li, PhD, Gari Clifford,
PhD, Monica Crubezy, PhD, Terry Willey, and Sara Gregg, MHA.
Appendix A. Supplementary data
Supplementary data to this article can be found online at http://dx.
doi.org/10.1016/j.jcrc.2017.01.012.
References
[1] Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Denitions for
sepsis and organ failure and guidelines for the use of innovative therapies in sepsis.
Chest 1992;101(6):164455.
[2] Torio CM, Andrews RM. National Inpatient Hospital Costs: The Most Expensive Con-
ditions by Payer, 2011: Statistical Brief #160. Rockville, MD: Agency for Health Care
Policy and Research (US); 2013.
[3] Liu V, Escobar GJ, Greene JD, Soule J, Whippy A, Angus DC, et al. Hospital deaths in
patients with sepsis from 2 independent cohorts. JAMA 2014;312(1):902.
[4] Goodwin AJ, Rice DA, Simpson KN, Ford DW. Frequency, Cost, and Risk Factors of
Readmissions Among Severe Sepsis Survivors. Crit Care Med 2015;43(4).
[5] Elixhauser A, Steiner C. Readmissions to U.S. Hospitals by Diagnosis, 2010(Rockville,
MD) ; 2013.
[6] Prescott HC, Langa KM, Liu V, Escobar GJ, Iwashyna TJ. Increased 1-year healthcare
use in survivors of severe sepsis. Am J Respir Crit Care Med 2014;190(1):629.
[7] Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemi-
ology of severe sepsis in the United States: analysis of incidence, outcome, and asso-
ciated costs of care. Crit Care Med 2001;29(7):130310.
[8] Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United
States from 1979 through 2000. N Engl J Med 2003;348:154654.
[9] Dombrovskiy VY, Martin AA, Sunderram J, Paz HL. Rapid increase in hospitalization
and mortality rates for severe sepsis in the United States: a trend analysis from 1993
to 2003. Crit Care Med 2007;35(5):124450.
[10] Gaieski DF, Edwards JM, Kallan MJ, Carr BG. Benchmarking the incidence and mor-
tality of severe sepsis in the United States. Crit Care Med 2013;41(6):116774.
[11] Sands KE, Bates DW, Lanken PN, Graman PS, Hibberd PL, Kahn KL, et al. Epidemiol-
ogy of sepsis syndrome in 8 academic medical centers. JAMA 1997;278:23440.
[12] Jolley RJ, Sawka KJ, Yergens DW, Quan H, Jett N, Doig CJ. Validity of administrative
data in recording sepsis: a systematic review. Crit Care 2015;19(1).
[13] Cooke CR, Iwashyna TJ. Sepsis mandates: improving inpatient care while advancing
quality improvement. JAMA 2014;312(14):13978.
[14] Dellinger RP, Levy MM, Rhodes A, Annane D, Gerlach H, Opal SM, et al. Surviving
sepsis campaign: international guidelines for management of severe sepsis and sep-
tic shock, 2012. Intensive Care Med 2013;39(2):165228.
[15] Giuliano KK. Physiological monitoring for critically ill patients: testing a predictive
model for the early detection of sepsis. Am J Crit Care 2007;16(2):12230.
[16] Angus DC, van der Poll T. Severe sepsis and septic shock. N Engl J Med 2013;369:
84051.
[17] Iwashyna TJ, Odden A, Rohde J, Bonham C, Kuhn L, Malani P, et al. Identifying pa-
tients with severe sepsis using administrative claims: patient-level validation of
the angus implementation of the international consensus conference denition of
severe sepsis. Med Care 2014;52(6):e3943.
[18] Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al.
The third international consensus denitions for sepsis and septic shock (sepsis-3).
JAMA Feb. 2016;315(8):801.