BRIEF REPORTS

Orbital Fracture Clinical Decision Rule

Development: Burden of Disease and Use
of a Mandatory Electronic Survey Instrument
Kabir Yadav, MDCM, MS, Ethan Cowan, MD, MS, Stephen Wall, MD, MS, and Paul Gennis, MD

Abstract
Objectives: In preparation for development of a clinical decision rule (CDR) to promote more efficient
use of computed tomography (CT) for diagnosing orbital fractures, the authors sought to estimate the
annual incidence of orbital fractures in emergency departments (EDs) and the usage of CT to make these
diagnoses. The authors also sought to evaluate a mandatory electronic data collection instrument (EDCI)
administered to providers to facilitate CDR data collection.
Methods: National estimates were made by analyzing the 2007 National Hospital Ambulatory Medical
Care Survey (NHAMCS) database, while hospital billing system and coding data were used to make local
estimates. An EDCI was integrated into the CT ordering system such that providers had to complete the
form to perform a CT. Because the EDCI had to be filled out for every CT ordered, data collection effi-
ciency was measured by compliance (counting the number of unrealistic data collection instrument
answers) and by timing a convenience sample of providers completing the EDCI.
Results: Out of 116.8 million ED visits in the United States in 2007, 4.1 million patients were treated for
injuries of the eye and face. Of those, 820,252 patients underwent CT imaging, with 102,999 patients
(12.5%) diagnosed with an orbital fracture. In our local hospital system with 122,500 annual ED visits,
752 CTs of orbits were performed, with 172 (23%) orbital fractures. The EDCI compliance rate was
94.9% and took less than 5 minutes to complete.
Conclusions: National and local data demonstrate a low yield for CT imaging in identifying orbital frac-
tures. Data collection using a mandatory EDCI linked to computerized provider order entry can provide
prospective, consecutive patient data that are needed to develop a CDR for the selective use of CT imag-
ing in orbital trauma. Such a decision rule could increase the efficiency in diagnosing orbital fractures,
thereby improving patient care, reducing radiation exposure, and decreasing costs.
ACADEMIC EMERGENCY MEDICINE 2011; 18:313316 2011 by the Society for Academic Emergency
Medicine

tion.2,3 On the other hand, timely and accurate diagnosis

I
n 2006, U.S. emergency departments (EDs) treated
approximately 3.79 million patients for eye and face
injuries.1 Computed tomography (CT) is the estab-
lished imaging method used for assessing orbital trauma,
but has certain disadvantages including limited availabil-
ity, high cost, and exposure to substantial ionizing radia-
From the Department of Emergency Medicine, The George
Washington University (KY), Washington, DC; the Department
of Emergency Medicine (EC, PG), Jacobi Medical Center,
Bronx, New York; and the Department of Emergency Medi-
cine, New York University (SW), New York, NY.
Received August 2, 2010; revision received September 3, 2010;
accepted September 4, 2010.
The authors have no relevant financial information or potential
conflicts of interest to disclose.
Supervising Editor: Shahriar Zehtabchi, MD
Address for correspondence and reprints: Kabir Yadav,
MDCM, MS; e-mail: kyadav@mfa.gwu.edu.
is critical because missed orbital fractures can have per-
manent consequences, including diplopia and loss of bi-
nocular vision.4 Weighing the costs and long-term risks
of orbital CTs against the consequences of missed frac-
tures likely results in overuse of radiographic imaging.
However, to the best of our knowledge, estimates of the
prevalence of orbital trauma and the usage of CT
imaging for diagnosis of orbital fractures have not been
published.
To reduce unnecessary utilization of radiographic
testing, clinical decision rules (CDRs) have been derived
and validated for ankle films, knee films, cervical spine
films, and head CTs for blunt trauma.5 Studies have
demonstrated decreased resource utilization without
compromising patient safety, and CDRs have been
widely accepted into the clinical practice of emergency
and primary care physicians.6,7 Developing CDRs
requires knowledge of the signs and symptoms

2011 by the Society for Academic Emergency Medicine ISSN 1069-6563

doi: 10.1111/j.1553-2712.2011.01017.x PII ISSN 1069-6563583 313
314
associated with the outcome of interest. We did not
find any published studies investigating the ability of
the clinical exam to predict the presence of orbital frac-
ture. Clinical signs and symptoms of orbital trauma
have only been described in association with severity of
fractures or their anatomic location.8,9
Data acquisition for CDR research can be challenging
and expensive. It often requires on-site research assis-
tants gathering data as patient care is administered.
Traditionally this involves manual coding of medical
records or acquiring data using a paper data collection
instrument. This process is costly and remains suscepti-
ble to inaccuracy.10,11 To our knowledge, there is no
published CDR research linking a computerized data
collection instrument to an electronic ordering system
to accomplish paperless data collection.
Our first hypothesis was that we can successfully
estimate the prevalence of orbital fractures and CT
imaging using a national database and local hospital
system records. Our second hypothesis was that a man-
datory electronic data collection instrument (EDCI) can
efficiently collect clinical predictor data with minimal
noncompliance.
METHODS
Study Design
This study was conducted in three phases. Phase I was an
analysis of the latest publicly available National Hospital
Ambulatory Medical Care Survey (NHAMCS).12 Phase II
was a review of billing and coding data in our public
urban hospital system, North Bronx Healthcare Network
(NBHN). Phase III was implementation of a mandatory
EDCI conducted as a prospective cohort study of ED
patients with blunt orbital trauma. The study received
institutional review board approval with waivers of
patient and physician consent. Phase III data collection
began July 1, 2007, and concluded October 16, 2009.
Study Populations
The NHAMCS is a national probability sample of visits
to U.S. hospital emergency and outpatient departments;
only the ED data were used in this analysis. NHAMCS
uses a four-stage sampling design covering geographic
primary sampling units (PSUs), hospitals within PSUs,
EDs within hospitals, and patient visits within EDs.
Hospitals are selected with a probability proportional
to the number of ED visits after being stratified by
region, class, and size. Patient visits are systematically
selected over a randomly assigned 4-week period, while
hospitals are stratified by region, type, and size.
The NBHNs hospital system consists of a Level I
trauma center (75,000 annual ED visits) and a Level II
trauma center (47,500 annual ED visits). For Phase III,
ED patients (age 18 years) suffering blunt orbital
trauma undergoing orbital CT imaging were eligible for
inclusion and were enrolled in a consecutive fashion
through the completion of the mandatory EDCI.
Study Protocol
For Phase I, International Classification of Diseases 9th
Revision (ICD-9) coding identified patients diagnosed
with orbital fractures (801.x and 802.6802.8) using
Yadav et al. ORBITAL FRACTURE CDR DEVELOPMENT
appropriate survey methods in the Stata version 10.1
statistical software package (StataCorp, College Station,
TX). Patient weights in the database were used to esti-
mate national values. To identify all patients with trau-
matic eye and facial injuries, we used a broader set of
ICD-9 coding (802.x, 830.x, 848.x, 870.x, 871.x, 872.x,
873.x, 918.x, 921.x, 940.x, 941.x, and 950.x). To identify
CT usage, we used the NHAMCS diagnostic services
survey item CATSCAN.
For Phase II, ICD-9 coding identified patients diag-
nosed with orbital fractures in the NBHN network. Our
trauma protocols mandate the use of orbital CTs for the
diagnosis of orbital fracture, so we used Current Proce-
dural Terminology (CPT) codes (70480, 70481, 70482) to
quantify CT usage.
For Phase III, emergency physicians filled out the
EDCI as part of the orbital CT order. Completing the
EDCI was a mandatory part of ordering the orbital CT,
but the specific answers to the questions had no effect
on the physicians ability to order the CT. The data col-
lection instrument included 15 questions about the
presence of blunt orbital trauma and exclusion criteria
to screen out ineligible cases (see Data Supplement S1,
available as supporting information in the online ver-
sion of this paper). A paper version of this form of data
capture had been used previously at NBHN and in mul-
ticenter studies and been shown to successfully enable
consecutive patient enrollment without adversely affect-
ing patient care.13,14
Potential clinical predictors for the data collection
instrument were identified by extensive literature
review by the study investigators, followed by a con-
sensus conference with the institutional department
heads of the Craniofacial Trauma Service (a multidisci-
plinary service comprised of otolaryngologists, ophthal-
mologists, oromaxillofacial surgeons, and plastic
surgeons). Prior to implementation, we educated pro-
viders to the mandatory data collection instrument
through didactic lectures reviewing diagnosis of orbital
fractures and provision of review materials in print and
via electronic mail.
To measure compliance with the ECDI, we looked for
patterns of answers that were not likely to occur in real
life: 1) all yeswhile it is possible that a truly unfor-
tunate trauma patient would have every clinical predic-
tor, it is quite unlikely and 2) all unknownsome of
the predictors are based on simple observation of the
patients face, i.e., periorbital ecchymosis; therefore all
unknown is unlikely to be a true assessment.
Research assistants measured the time it took a conve-
nience sample of providers to complete the ECDI.
Data Analysis
National survey results were reported as estimates with
standard errors. Local billing and coding results were
reported as counts and percentages. ECDI compliance
and completion time were reported as counts and per-
centages.
RESULTS
According to NHAMCS, 4.1 million patients out of the
116.8 million ED visits in 2007 were treated for injuries of
ACAD EMERG MED March 2011, Vol. 18, No. 3 www.aemj.org
Table 1
NHAMCS and NBHN Data on Eye and Face Injuries
Source Condition
NHAMCS Eye and face injuries
NHAMCS CT usage
NHAMCS Orbital fractures
NBHN CT usage
NBHN Orbital fractures
NHAMCS = National Hospital Ambulatory Medical Care Survey; NBHN = North Bronx Healthcare Network.
the eye and face (Table 1). Of those, 820,252 patients
underwent CT imaging, leading to 102,999 patients
(12.5%) being ultimately diagnosed with orbital fracture.
Based on CPT codes, we found that 748 patients
underwent orbital CT scanning in the NBHN EDs in
2006. According to ICD-9 coding, 172 adult patients
were diagnosed with orbital fractures (23% of patients
undergoing orbital CT).
Out of 3,123 EDCIs completed, 2,552 (81.7%) were for
patients suffering traumatic injury. Only 130 (5.1%) of
those were noncompliant as defined above: four all
yes and 126 all unknown. This represents 94.9%
compliance with a mandatory ECDI. Of 142 observa-
tions of providers filling out the data collection instru-
ment, none took more than 5 minutes to complete.
DISCUSSION
Overuse of CT imaging contributes to rising health care
costs. The rate of CT usage in U.S. EDs has quadrupled
from 1996 to 2007,15 while improvements in diagnostic
yield have been called into question.16 From our
national and local estimates, almost 80% of CT scans
ordered were negative for fractures, indicating an inef-
ficient use of CT imaging. Furthermore, orbital CT
scanning exposes the patient to substantial radiationa
dose of 10130 mGray (10 mGray = 1 rad) is adminis-
tered to the lens of the eye, while the thyroid gland is
exposed to 13 mGray.17,18 While the thyroid gland is
widely recognized as radiosensitive, so too is the lens
of the eye; as little as 5002,000 mGray causes detect-
able opacities, and exposures of >4,000 mGray cause
visually impairing cataracts.19
To identify the need for a CDR, a clearly definable
and clinically important condition must be identified
where the diagnostic test of choice is used inefficiently
or where there is significant variation in practice. As
demonstrated by the national and local data, CT usage
for orbital fractures is inefficient. To derive a CDR, cli-
nicians must assess potential predictor variables in
patients in a standardized, prospective fashion. We
have demonstrated that a mandatory EDCI linked to
computerized provider order entry can efficiently
enable paperless data collection on consecutive
patients. Such a system can streamline the derivation
and validation of CDRs at reduced cost.
LIMITATIONS
The NHAMCS sampling method may create unreliable
estimates. However, with relative standard errors of less
315
Estimate Standard Error 95% CI
4,106,902 299,243.5 3,516,5124,697,292
820,252 83,947.4 654,629985,875
102,999 21481.2 60,618145,380
748
172
than 30%, our estimates are reliable according to gui-
dance from the National Center for Health Statistics.20
Without having a discrete category for orbital CT in
the NHAMCS database, it is possible that CT usage in
orbital trauma is overestimated. The NHAMCS item
CATSCAN would have tracked the performance of any
CT imaging for the patient, regardless of anatomic loca-
tion. Furthermore, even in patients with facial trauma,
the investigation of other facial injuries may have
prompted the ordering of CT imaging. It is likely that CT
imaging efficiency for orbital fracture is better reflected
by our local hospital system estimates, where we could
specifically measure orbital CT usage. However, the gen-
eralizability of our usage may be limited because it repre-
sents the practice at only one urban hospital system.
Billing and coding data have the inherent limitations of
retrospective data, such as being coded properly for data
extraction. However, procedural codes are captured
from the electronic radiology ordering system, and diag-
nostic coding is captured from the electronic hospital
discharge system, so data fidelity should be high.
Although our approach to evaluating the accuracy of
EDCI completion is similar to prior studies,21,22 there
may have been other patterns of noncompliant EDCI
completion that we were unable to detect. Other possi-
ble patterns of noncompliance, such as alternating
yes and unknown answers (the keyboard keys are
next to each other) were not found. Because providers
were aware that the EDCI is part of the medical record,
it is unlikely that they would have fabricated clinical
signs and symptoms simply to skip the data collection
instrument. This awareness may explain why noncom-
pliant EDCI results skewed heavily to documenting all
unknown rather than all yes.
CONCLUSIONS
National and local data demonstrate a low yield for CT
imaging in identifying orbital fractures. Data collection
using a mandatory electronic data collection instrument
linked to computerized provider order entry can effi-
ciently provide prospective, consecutive patient data
needed to develop a clinical decision rule for the selec-
tive use of CT imaging in orbital trauma. Such a deci-
sion rule could increase the efficiency in diagnosing
orbital fractures, thereby improving patient care, reduc-
ing radiation exposure, and decreasing costs.
The authors acknowledge the assistance of Susan M. Hailpern,
DrPH, MS, with the NHAMCS survey analysis and Zachary Ash-
well with the data collection.
316 Yadav et al. ORBITAL FRACTURE CDR DEVELOPMENT

References X-Radiography Utilization Study (NEXUS). Ann

Emerg Med. 1998; 32:4619.
1. Pitts SR, Niska RW, Xu J, Burt CW. National Hospi-
14. Mower WR, Hoffman JR, Herbert M, et al. Develop-
tal Ambulatory Medical Care Survey: 2006 Emer-
ing a decision instrument to guide computed tomo-
gency Department Summary. National Health
graphic imaging of blunt head injury patients. J
Statistics Reports No. 7. Hyattsville, MD: National
Trauma. 2005; 59:9549.
Center for Health Statistics, 2008.
15. National Center for Health Statistics. Health, United
2. Gilbard SM. Management of orbital blowout frac-
States, 2009: with special feature on medical tech-
tures: the prognostic significance of computed
nology. Hyattsville, MD: National Center for Health
tomography. Adv Ophthalmic Plast Reconstr Surg.
Statistics, 2010.
1987; 6:26980.
16. Pines JM. Trends in the rates of radiography use
3. Rowe-Jones JM, Adam EJ, Moore-Gillon V. Subtle
and important diagnoses in emergency department
diagnostic markers of orbital floor blow-out frac-
patients with abdominal pain. Med Care. 2009;
ture on coronal CT scan. J Laryngol Otol. 1993;
47:7826.
107:1612.
17. Rehani MM. CT: caution on radiation dose. Ind J
4. Burnstine MA. Clinical recommendations for repair
Radiol Imag. 2000; 10:1920.
of orbital facial fractures. Curr Opin Ophthalmol.
18. Zammit-Maempel I, Chadwick CL, Willis SP. Radia-
2003; 14:23640.
tion dose to the lens of eye and thyroid gland in
5. Agrawal P, Kosowsky JM. Clinical practice guide-
paranasal sinus multislice CT. Br J Radiol. 2003;
lines in the emergency department. Emerg Med
76:41820.
Clin North Am. 2009; 27:55567.
19. Hopper KD, Neuman JD, King SH, Kunselman AR.
6. Graham ID, Stiell IG, Laupacis A, OConnor AM,
Radioprotection to the eye during CT scanning.
Wells GA. Emergency physicians attitudes toward
AJNR Am J Neuroradiol. 2001; 22:11948.
and use of clinical decision rules for radiography.
20. National Center for Health Statistics. Public Use
Acad Emerg Med. 1998; 5:13440.
Data File Documentation: 2007 National Hospital
7. Brehaut JC, Stiell IG, Visentin L, Graham ID. Clini-
Ambulatory Medical Care Data collection instru-
cal decision rules in the real world: how a widely
ment. Hyattsville, MD: National Center for Health
disseminated rule is used in everyday practice. Acad
Statistics, 2009.
Emerg Med. 2005; 12:94856.
21. Kline JA, Johnson CL, Webb WB, Runyon MS. Pro-
8. Osguthorpe JD. Orbital wall fractures: evaluation
spective study of clinician-entered research data in
and management. Otolaryngol Head Neck Surg.
the emergency department using an internet-based
1991; 105:7027.
system after the HIPAA privacy rule. BMC Med
9. Jank S, Schuchter B, Emshoff R, et al. Clinical signs
Inform Decis Mak. 2004; 4:e17.
of orbital wall fractures as a function of anatomic
22. Rivera ML, Donnelly J, Parry BA, et al. Prospective,
location. Oral Surg Oral Med Oral Pathol Oral
randomized evaluation of a personal digital assis-
Radiol Endod. 2003; 96:14953.
tant-based research tool in the emergency depart-
10. Hobbs FD, Parle JV, Kenkre JE. Accuracy of rou-
ment. BMC Med Inform Decis Mak. 2008; 8:e3.
tinely collected clinical data on acute medical
admissions to one hospital. Br J Gen Pract. 1997;
Supporting Information
47:43940.
11. Wilton R, Pennisi AJ. Evaluating the accuracy of The following supporting information is available in the
transcribed clinical data. Proc Annu Symp Comput online version of this paper:
Appl Med Care. 1993:27983. Data Supplement S1. Electronic data collection
12. National Center for Health Statistics. Public Use instrument.
Data File: 2007 National Hospital Ambulatory Medi- Please note: Wiley Periodicals Inc. is not responsible
cal Care Survey. Hyattsville, MD: National Center for the content or functionality of any supporting infor-
for Health Statistics, 2009. mation supplied by the authors. Any queries (other than
13. Hoffman JR, Wolfson AB, Todd K, Mower WR. missing material) should be directed to the correspond-
Selective cervical spine radiography in blunt ing author for the article.
trauma: methodology of the National Emergency