Gastrointestinal Imaging • Original Research

Poullos et al.
Multiphasic CT Reporting for HCC

Gastrointestinal Imaging
Original Research

Structured Reporting of
Multiphasic CT for Hepatocellular
Carcinoma: Effect on Staging and
Suitability for Transplant
Peter D. Poullos1 OBJECTIVE. The purpose of this study is to evaluate whether use of a standardized radi-
Joseph J. Tseng1 ology report template would improve the ability of liver transplant surgeons to diagnose stage
Marc L. Melcher 2 T2 hepatocellular carcinoma (HCC) and determine patient suitability to undergo orthotopic
Waldo Concepcion2 liver transplant (OLT).
Andreas M. Loening 3 MATERIALS AND METHODS. In this retrospective study, a standardized template
was devised, and its use was mandated for reporting of liver CT findings for patients with
Jarrett Rosenberg 4
cirrhosis and HCC. Two surgeons analyzed 200 reports (100 before and 100 after template
Jüergen K. Willmann1
American Journal of Roentgenology 2018.210:766-774.

implementation) for descriptions of cirrhosis, portal hypertension, lesion enhancement char-

Poullos PD, Tseng JJ, Melcher ML, et al.
Keywords: hepatocellular carcinoma, liver transplant,
Organ Procurement and Transplantation Network,
standard reporting, template
doi.org/10.2214/AJR.17.18725
Received July 5, 2017; accepted after revision
August 22, 2017.
Based on a presentation at the Radiological Society of
North American 2015 annual meeting, Chicago, IL.
acteristics, tumor thrombus, portal and superior mesenteric vein patency, and Organ Procure-
ment Transplantation Network (OPTN) class. Ability to determine Milan criteria and surgeon
satisfaction were also assessed. Data obtained before and after template implementation were
statistically analyzed using the Cochran-Mantel-Haenszel test.
RESULTS. Template implementation increased the percentage of reports documenting
the presence or absence of portal hypertension (74% to 88% for surgeon 1 and 86% to 87%
for surgeon 2; p = 0.042); lesion number (76% to 88% for surgeon 2 [no change for surgeon
1]; p = 0.038), size (95% to 96% for surgeon 1 and 82% to 93% for surgeon 2; p = 0.03), and
enhancement (93% to 94% for surgeon 1 and 80% to 91% for surgeon 2; p = 0.049); presence
of tumor thrombus (10% to 57% for surgeon 1 and 31% to 63% for surgeon 2; p < 0.001); and
OPTN class (8% to 82% for surgeon 1 and 2% to 81% for surgeon 2; p < 0.001). The surgeons
were significantly more able to determine the presence of T2 disease and qualification for ex-
ception points after implementation of the template (increasing from 80% to 94%; p = 0.025).
Satisfaction with reports also improved (p < 0.0001).
CONCLUSION. The reporting template improved determination of patient suitability to
undergo transplant according to the Milan criteria.
epatocellular carcinoma (HCC) Patients with cirrhosis who do not have

1
Department of Radiology, Division of Body Imaging,
H is a major cause of death among
patients with end-stage liver dis-
ease (ESLD). For patients with
Stanford University, 300 Pasteur Dr, Stanford, CA HCC and ESLD, potentially curative options
94305-5621. Address correspondence to J. J. Tseng include surgical resection, ablation, and or-
(jtseng3@stanford.edu).
thotopic liver transplant (OLT). For trans-
2
Department of Surgery, Division of Multi Organ plant candidates who are ineligible for cura-
Transplantation, Stanford University, Stanford, CA. tive resection, OLT is a desirable method for
simultaneously removing the HCC and re-
placing the failing cirrhotic liver. OLT, how-
3
Department of Radiology, Division of Body MRI,
Stanford University, CA.
ever, requires that the patient be placed on a
4
Department of Radiology, Stanford University, waiting list. In 2016, a total of 14,750 pa-
Stanford, CA. tients were listed as waiting for transplant,
but only 7841 transplants were performed;
AJR 2018; 210:766–774
1206 patients died while waiting, and an ad-
0361–803X/18/2104–766 ditional 1404 patients were removed from
the list because they were too sick to under-
© American Roentgen Ray Society go transplant [1].
HCC are prioritized by calculating the model
for end-stage liver disease (MELD) score on
the basis of bilirubin, creatinine, and interna-
tional normalized ratio (INR) values. MELD
alone, however, does not reflect the progno-
sis for HCC in patients [2] in whom tumor
progression, vascular invasion, and metastat-
ic disease can preclude transplant. To rec-
tify this disadvantage, the United Network
for Organ Sharing, which organizes the Or-
gan Procurement and Transplantation Net-
work (OPTN) in the United States, has cre-
ated an exception categorization that allows
patients with HCC to obtain a higher MELD
score that more appropriately reflects their
medical urgency for liver transplant. Candi-
dates meeting these criteria receive an ini-
tial MELD score equivalent of 22, and fur-

766 AJR:210, April 2018

 Multiphasic CT Reporting for HCC

ther points are awarded based on laboratory
values [3].
To obtain points, HCC lesions must meet
specific OPTN imaging-based criteria [4]. Le-
sions are designated OPTN class 1 (definite-
ly benign) through class 5 (definitely HCC),
depending on their size and arterial enhance-
ment as well as the presence of washout, cap-
sule, threshold growth, or a combination of
these features. Candidates must have at least
one OPTN class 5 lesion and stage T2 disease,
as defined by the Milan criteria (Appendix 1),
to qualify for exception points. To apply, the
transplant team must complete standard Unit-
ed Network for Organ Sharing documentation
that includes comprehensive imaging details
(Appendix 2). Surgeons rely on radiology re-
ports that, unfortunately, often omit the de-
tails required for these forms. Significant time
is spent reevaluating imaging studies because
of information missing from the reports, es-
sentially requiring a second radiologist to re-
American Journal of Roentgenology 2018.210:766-774.
The purpose of the present study is to
evaluate whether the use of a standardized
radiology report template would improve the
ability of liver transplant surgeons to diag-
nose stage T2 HCC and suitability for OLT.
Materials and Methods
Template Design
An institutional review board waiver was
granted for this study, and informed consent was
waived because of the retrospective methods used.
The study was HIPAA compliant.
Two abdominal radiologists created the tem-
plate for patients with stage T2 HCC (Appendix
3) to include all imaging details needed to apply
for exception points. This template was derived in
part from templates freely available through the
Radiological Society of North America Informat-
ics Reporting website [11], and additional modifi-
cations were made to mimic the required OPTN
documentation completed by the transplant team.
After creation of this template, the chief of clinical
template listed as a “body CT abdomen liver tri-
phasic” (Appendix 4) was also created for reports
assessing patients without stage T2 HCC (with
this less detailed template hereafter referred to as
the “non-OPTN template”). Before template im-
plementation, radiologists used their own personal
style of dictation to create reports.
Report Collection
Eighteen months after the start date, reports of
patients with HCC were collected for analysis, with
an arbitrary goal of collecting 100 reports dictated
before template implementation and 100 reports dic-
tated after template implementation. The lead author
requested that the radiology information technol-
ogy team search the radiology information system
for multiphasic (late arterial, portal venous, and de-
layed) liver CT reports containing the terms “cir-
rhosis” and either “hepatocellular carcinoma” or
“HCC.” The search results were returned in a chron-
ologically sorted order with the use of spreadsheet
software (Excel, version 15.0, Microsoft) with an ac-

interpret the case (for example, at liver trans-
plant tumor boards).
Structured reporting has been endorsed
by the American College of Radiology and
other professional societies as a method of
communicating the most pertinent and use-
ful information with the use of standardized
language and formatting [5, 6]. Template cre-
ation should be a multidisciplinary effort,
and studies have shown that both referring
physicians and radiologists favor structured
over conventional nonstructured reports [7–
9]. Although widely adopted, the clinical
impact of such templates has not been well
studied in the literature. However, findings
from early studies are promising [10].
transplantation reviewed and approved its use. Of
importance, the surgeons surveyed in the present
study were unaware of our research intentions and
their future role as report analysts.
Template Implementation
On January 13, 2014 (the start date), an e-mail
was sent to all radiology faculty and trainees,
designating our template mandatory for use in
reports regarding patients with stage T2 HCC.
OPTN educational references were also included.
The template was made available in a real-time
radiology reporting platform (Power Scribe 360,
Nuance) under the listing “body CT abdomen liver
OPTN HCC.” This template will hereafter be re-
ferred to as the “OPTN template.” A less detailed
companying word-processed document (Microsoft
Word 2013, Microsoft) containing the reports.
Inclusion criteria (Table 1) were designed to fa-
cilitate collection of a variety of reports for pa-
tients who were close to meeting the Milan criteria,
for whom accurate staging would be particular-
ly important. Exclusion criteria were designed to
minimize reports including likely benign lesions
or early HCC. Although hyperenhancing lesions
smaller than 1 cm and hypoenhancing lesions may
probably represent HCC (according to the Liv-
er Imaging Reporting and Data System or OPTN
class 4), these patients do not qualify for exception
points and therefore were excluded. Likewise, we
excluded patients with advanced HCC, who also
do not qualify for exception points.

TABLE 1: Inclusion and Exclusion Criteria for Chart Review

Inclusion Criteria Exclusion Criteria
The report states that a lesion is “consistent with HCC” or equivalent language Report specifies that the patient does not have cirrhosis (making hepatectomy
an option)
Multiphasic CT of adequate quality (judged to be adequate unless stated Lesions that are “likely benign,” “unlikely to represent HCC,” or have an
otherwise) equivalent status
The report either states that the patient has cirrhosis or makes no mention of Lesions that are less than 1 cm or larger than 10 cm in maximum diameter
cirrhosis
Lesions that are reported as “too numerous to count, ”innumerable” or as
having an equivalent status
Infiltrative (non–mass-forming) HCC
Hypoenhancing HCC
Prior hepatectomy or liver transplant for HCC
Organ Procurement Transplantation Network class 3
Technically limited examination (unenhanced or single phase)
Nonabdominal study (CT chest or interventional procedure)
Note—HCC = hepatocellular carcinoma.

AJR:210, April 2018 767

 Poullos et al.

tree, as well as information about portal hyperten-

Report review of triphasic Report review of triphasic sion (splenomegaly, ascites, varices, and related el-
liver CT done for HCC liver CT done for HCC
evaluation prior to evaluation after ements). Report syntax was not altered in any way.
January 13, 2014 January 13, 2014
Survey Creation
397 Reports excluded while 282 Reports excluded while A survey was created to analyze each report (Ta-
reaching goal of 100 reports reaching goal of 100 reports ble 3). The first eight questions (regarding funda-
meeting inclusion criteria meeting inclusion criteria
mental information) were questions with yes or no
responses that assessed the adequacy of informa-
100 Reports meeting 100 Reports meeting
tion crucial and fundamental to all reports of pa-
inclusion criteria inclusion criteria
tients with ESLD and HCC. Questions nine and 10
(regarding OPTN designation) assessed the inclu-
200 Reports truncated, sion and correctness of OPTN grading. Questions
sanitized, and randomized 11 and 12 (regarding making the diagnosis) were
for review by surgeons more subjective and included an option for uncer-
tainty. Question 13 assessed overall satisfaction.
Fig. 1—Report selection flowchart. HCC = hepatocellular carcinoma.
Report Analysis
Reports were reviewed and collected chrono- abdominal CT reports was lesions that were un- The 200 reports were randomized using a free
logically, working both backward and forward likely to represent HCC (n = 123), followed by le- online randomizer (Research Randomizer, ver-
from the start date. The lead author provided to sions with indeterminate HCC status (n = 17) and sion 4.0, Social Psychology Network) and were
another author both OPTN literature and verbal presence of too many lesions (n = 17). collated into a single 200-page PDF document,
American Journal of Roentgenology 2018.210:766-774.

instruction regarding inclusion and exclusion cri- with each page containing only one unique re-
teria. The two authors then collected 20 reports to- Report Sanitization and Truncation port. Each surgeon, blinded to study goal and de-
gether, practicing selection using the inclusion and Reports were sanitized to remove all protect- sign, received a copy of this document, along with
exclusion criteria. The author who had received ed health information, clinical information, and a link to the electronic survey. Study survey data
instruction then collected 20 additional reports on dates. They were further truncated by deleting in- were collected and managed using research elec-
his own, which the lead author then reviewed for formation related to IV contrast medium and ra- tronic data capture tools (REDCap, version 6.9.3,
accuracy. Once trained satisfactorily, the author diation dose, as well as statements related to the Stanford University).
trained by the lead author collected the remaining thorax, pancreas, adrenals, kidneys and ureters,
160 reports, which were then reviewed once more bowel, lymph nodes, abdominal wall, and bones. Response Tabulation
by the lead author to determine adherence to the All that remained (when included) was text de- Questions 1–9 elicited yes or no responses,
inclusion and exclusion criteria. scribing liver morphologic findings, hepatic vas- whereas questions 10–12 offered a third response
To find 100 pretemplate reports meeting inclu- culature, liver lesions, gallbladder, and biliary option (unsure) because they inquired about the
sion criteria, we reviewed 497 reports dating up to
TABLE 2: Reasons for Report Exclusion Among Abdominal CT Studies
27 months before the start date. To find 100 post-
template reports, we reviewed 382 reports dating up Reason for Report Exclusion Before Template After Template
to 20 months after the start date. A total of 397 pre-
Not likely HCC 169 123
template and 182 posttemplate reports were exclud-
ed (Fig. 1). Detailed reasons for exclusion are listed Nonabdominal CT 95 88
in Table 2. If a single patient had multiple unique Prior hepatic surgery 26 2
reports, those were not excluded. All posttemplate Prior OLT 23 7
reports meeting inclusion criteria were included, re-
Too many lesions 19 17
gardless of whether the OPTN template was used.
It was decided a priori to do an intention-to-treat Metastatic disease 14 7
analysis, avoiding overoptimistic estimates of tem- Indeterminate lesion 13 17
plate effectiveness resulting from inclusion bias. Maximum lesion diameter < 1 cm 11 4
Ninety-five pretemplate and 88 posttemplate
Technically limited 9 4
reports were eliminated because they were not
for abdominal CT. These included interventional Maximum lesion diameter > 10 cm 7 1
radiology procedures and CT scans of the chest Infiltrative HCC 5 0
that were inadvertently captured in our radiology Tumor thrombus 4 5
information system search results. In the pretem-
OPTN class 3 2 7
plate group, the two most common reasons for ex-
clusion of abdominal CT reports were lesions that Hypoenhancing HCC 0 0
were unlikely to represent HCC (n = 169) and pre- Total 397 282
vious hepatic surgery (n = 26). In the posttemplate Note—HCC = hepatocellular carcinoma, OLT = orthotopic liver transplant, OPTN = Organ Procurement
group, the most common reason for exclusion of Transplantation Network.

768 AJR:210, April 2018

 Multiphasic CT Reporting for HCC

TABLE 3: Questions Appearing on the Survey Submitted to Surgeon Readers

1. Does the report adequately convey presence or absence of cirrhosis? [yes or no]
2. Does the report adequately convey the presence or absence of portal hypertension? [yes or no]
3. Does the report adequately state the number of lesion(s)? [yes or no]
4. Does the report adequately state the size of the lesion(s)? [yes or no]
5. Does the report clearly state the enhancement characteristics (both arterial and portal venous and/or delayed phases) of the lesion(s)? [yes or no]
6. Does the report adequately communicate the presence or absence of tumor thrombus? [yes or no]
7. Does the report adequately communicate portal vein patency? [yes or no]
8. Does the report adequately communicate SMV patency? [yes or no]
9. Does the report mention OPTN class? [yes or no]
10. Is the OPTN class correct? [yes, no, or unsure]
11. Assuming the imaging description is correct, does the patient have hepatocellular carcinoma? [Yes, no, or not enough information to judge]
12. Based purely on this report, in the absence of contraindications to transplant, can you determine whether or not this patient falls within the Milan criteria
and qualifies for MELD priority score? [yes, he qualifies; no, he does not qualify; or unsure and need more imaging details]
13. How satisfied are you with the information contained in this report? [1, not at all satisfied; 2, dissatisfied; 3, neutral; 4, satisfied; 5, very satisfied]
Note—Information appearing in brackets in each question indicates possible responses. SMV = superior mesenteric vein, OPTN = Organ Procurement Transplantation
Network, MELD = Model for End Stage Liver Disease.

ability to draw conclusions on the basis of report obtained after that date. Of the 100 re-
American Journal of Roentgenology 2018.210:766-774.

Overall Survey Results

information. Both “yes” and “no” were considered
positive responses, whereas “unsure” was consid-
ered a failure of reporting. Thus, for questions 10–
12, yes and no responses were grouped together
and compared with unsure responses. For ques-
tion 10, the surgeon reading the report made the
determination of whether the OPTN designation
was correct.
Statistical Analysis
The data from the survey results were export-
ed from the research electronic data capture tool
into spreadsheet software. An author who was not
involved in either the study design or data collec-
tion analyzed the data using statistical software
(Stata, version 14, StataCorp). Differences in re-
sponses to questions before and after implemen-
tation of the template were tested using Cochran-
Mantel-Haenszel tests stratified by surgeon, with
a null hypothesis of no difference between preim-
plementation and postimplementation responses.
The effect of report attributes on postimplementa-
tion satisfaction was assessed by multivariable lo-
gistic regression with the surgeon considered a co-
variate. A significance level of p < 0.05 was used.
Subgroup analysis was also performed using
similar statistical methods to compare the post-
template reports that used the OPTN template (n =
47) with posttemplate reports that deviated from
the OPTN template (n = 53).
Results
Template Compliance
One hundred reports meeting our inclu-
sion criteria were obtained before the tem-
plate implementation date, and 100 were
ports obtained after template implementa-
tion, 47% complied with the OPTN template
format, and 53% did not. Intention-to-treat
analysis was performed, and all 100 re-
ports obtained after template implementa-
tion were compared with all 100 reports ob-
tained before implementation.
Patient Characteristics
The 100 pretemplate reports referred to 88
individual patients; three patients each had
three reports, six patients had two, and 79 pa-
tients had one. Sixty-four reports were for men
and 36 were for women. The age range of the
patients was 39–80 years, with a mean and
median age of 60 years. The 100 posttemplate
reports referred to 78 patients; six patients had
three unique reports, 10 patients had two, and
62 patients had one. Sixty-nine reports were
for men and 31 were for women. The patient
age range was 49–73 years, with a mean age of
62 years and a median age of 64 years. Patient
characteristics are summarized in Table 4.
TABLE 4: Patient Characteristics in 100 Reports Obtained Before Versus
100 Reports After Template Implementation
Characteristic
No. of unique patients
Age range (y)
Mean age (y)
Median age (y)
No. of men
No. of women
The total number of positive respons-
es and the interobserver agreement (kappa)
between two surgeons for pretemplate and
posttemplate reports are presented in Ta-
ble 5. Question 9, regarding the mentioning
of OPTN criteria, had the greatest level of
agreement, with a kappa value of 0.84 (95%
CI, 0.755–0.900).
Of the 13 total questions on the survey, re-
sponses for ten improved significantly after
template implementation. These ten ques-
tions were questions 2, 3, 4, 5, 6, 9, 10, 11, 12
and 13. Questions 2, 3, 4, 5 and 11 showed
marginally significant improvements (p =
0.025–0.049). Questions 6, 10, 12, and 13
showed highly significant improvements
(p < 0.0001–0.001).
Inclusion of Fundamental Information:
Questions 1–8
With regard to these eight questions, ade-
quacy improved for five, and three showed no
change after template implementation.
Before Template After Template
88 78
39–80 49–73
60 62
60 64
64 69
36 31

AJR:210, April 2018 769

 Poullos et al.

TABLE 5: Total Number of Positive Responses and Interobserver Agreement (Kappa Value) Between Two Surgeons
for Reports Obtained Before and After Template Implementation
Positive Responses (%) Kappa Value
Before Template After Template
Before After Overall
Question Number and and Topic Surgeon 1 Surgeon 2 Surgeon 1 Surgeon 2 p Template Template Overall 95% CI
Question 1: cirrhosis 97 93 95 92 0.52 0.37 0.59 0.5 0.216–0.762
Question 2: portal hypertension 74 86 88 87 0.042 0.27 0.41 0.32 0.159–0.489
Question 3: no. of lesions 98 76 98 88 0.038 0.04 −0.04 0.01 −0.046 to 0.122
Question 4: size of lesions 95 82 96 93 0.03 0.10 0.14 0.12 0.045–0.313
Question 5: enhancement 93 80 94 91 0.049 −0.12 0.07 −0.05 −0.113 to 0.074
Question 6: tumor thrombus 10 31 57 63 < 0.001 0.22 0.46 0.46 0.327–0.567
Question 7: PV patency 80 68 72 65 0.22 0.49 0.52 0.51 0.379–0.631
Question 8: SMV patency 37 37 34 37 0.76 0.70 0.72 0.71 0.597–0.805
Question 9: OPTN mentioned 8 2 82 81 < 0.001 0 0.63 0.84 0.755–0.900
Question 10: OPTN correct (% of question 9) 50 0 100 96 < 0.001 — — — —
Question 11: HCC 80 80 95 93 0.025 0.31 0.14 0.27 0.131–0.415
Question 12: Milan criteria 78 80 89 96 < 0.0001 0.25 0.40 0.34 0.249–0.444
American Journal of Roentgenology 2018.210:766-774.

Question 13: satisfaction 3 30 50 80 < 0.0001 0.01 0.28 0.36 0.047–0.182

Note—PV = portal vein, SMV = superior mesenteric vein, OPTN = Organ Procurement and Transplantation Network, HCC = hepatocellular carcinoma, — = not applicable.

The five items that improved were pres-
ence or absence of portal hypertension,
number of lesion(s), size of the lesion(s), en-
hancement characteristics, and the presence
or absence of tumor thrombus. Of these five
items, the most statistically significant im-
provement was the mention of tumor throm-
bus, which increased from 10% (surgeon 1)
or 31% (surgeon 2) before template imple-
mentation to 57% (surgeon 1) or 63% (sur-
geon 2) after template implementation (p <
0.001). Subgroup analysis after template im-
plementation showed that 91% (surgeon 1) or
89% (surgeon 2) of the 47 OPTN template
reports reported the presence or absence of
tumor thrombus, compared with 74% (sur-
geon 1) or 68% (surgeon 2) of the 53 non-
OPTN template reports (for surgeon 1, p =
0.035; for surgeon 2, p = 0.015). One patient
in the pretemplate group and one patient in
the posttemplate group had a tumor throm-
bus. The three items for which no improve-
ment was seen were presence or absence of
cirrhosis, portal vein patency, and superior
mesenteric vein patency.
implementation (p < 0.001). Subgroup analy-
sis reveals that four of 47 OPTN template re-
ports (8.5%) did not mention OPTN. None of
these four patients, however, had T2 disease,
so not using the OPTN template was a cor-
rect decision. One patient had a 6.6-cm le-
sion, another patient had six tumors, a third
patient had a lesion with questionable wash-
out, and the last patient had a lesion that was
actually OPTN class 4, although that was
not explicitly stated. In contrast, 12 of the 53
non–OPTN-template reports (22.6%) did not
mention OPTN. Of those 12 reports, seven
(58%) were for patients with T2 disease, and
therefore should have mentioned OPTN.
Surgeon 1 stated that of the 8% of pre-
template reports that mentioned OPTN,
50% were correctly classified. Of the 82% of
posttemplate reports that mentioned OPTN,
100% were correctly classified. Surgeon 2
stated that of the 2% of the pretemplate re-
ports that mentioned OPTN, 100% were cor-
rectly classified. Also, of the 81% of post-
template reports that mentioned OPTN, 96%
were correctly classified.
Transplantability
Before implementation of the template,
20% (surgeon 1) or 22% (surgeon 2) of the
reports studied were perceived as not con-
taining enough information for the surgeons
to decide whether the patient met the Milan
criteria. This value decreased to 4% (surgeon
1) or 11% (surgeon 2) after template imple-
mentation (p < 0.0001) (Fig. 3).
Satisfaction
Before template implementation, surgeon
1 was satisfied or very satisfied with 3% of
the reports. However, this value increased to
50% for reports created after template im-
plementation (p < 0.0001). Before template
implementation, surgeon 2 was satisfied or
very satisfied with 30% of the reports. This
value increased to 80% for reports created
after template implementation (p < 0.0001)
(Fig. 4). When multivariate analysis was per-
formed, the items that were correlated with
report satisfaction were portal hypertension,
tumor thrombus, portal vein patency, and
OPTN class (p < 0.001 for all).

Organ Procurement Transplantation Network Making the Diagnosis Discussion

Designation: Questions 9 and 10
The surgeons responded that the percent-
age of reports that mentioned the OPTN were
8% (surgeon 1) or 2% (surgeon 2) before tem-
plate implementation, compared with 82%
(surgeon 1) or 81% (surgeon 2) after template
Before template implementation, 80% of
reports (for both surgeons 1 and 2) were per-
ceived to adequately convey the presence or
absence of HCC. After template implementa-
tion, this finding improved to 95% (surgeon
1) or 93% (surgeon 2) (p = 0.025) (Fig. 2).
Determining whether patients with HCC
are eligible for MELD exception points is
challenging because of the various clinical
and imaging data that must be reported. By
utilizing structured reporting, we significant-
ly improved communication of relevant in-

770 AJR:210, April 2018

 Multiphasic CT Reporting for HCC

100 100

80 80

Percentage of Responses
Percentage of Responses

60 60

40 40

20 20

0 0
Before After Before After Before After Before After
Template Template Template Template Template Template Template Template
Surgeon 1 Surgeon 2 Surgeon 1 Surgeon 2

No Unsure Yes No Unsure Yes

American Journal of Roentgenology 2018.210:766-774.

Fig. 2—Bar graph of reader response to survey question 11 (“Assuming imaging Fig. 3—Bar graph of reader response to survey question 12 (“Based purely on this
description is correct, does patient have hepatocellular carcinoma?”). report, in absence of contraindications to transplant, can you determine whether
or not this patient falls within Milan criteria and qualifies for MELD priority
score?”). MELD = model for end-stage liver disease.

formation as well as surgeon satisfaction.
Specifically, we improved the ability of sur-
geons to diagnose HCC and assess suitability
for MELD exception points.
Our results are concordant with those of
other studies that have shown that structured
reports provide superior description and
evaluation of the clinical question and im-
prove the confidence of referring physicians
[7–9, 12]. Brook et al. [12] compared struc-
tured versus traditional reports of multipha-
sic CT for the ability to assess for resectabil-
ity of pancreatic carcinoma. They found that
structured reports provided more complete
reporting of 12 key features (7.3 of 12 fea-
pressions were compared with arthroscopic
findings, and the overall accuracy increased
from 53% to 76% when readers used the
structured template as opposed to traditional
reports. Dickerson et al. [13] also compared
the differences in report thoroughness and
satisfaction between reports obtained before
and after template implementation that de-
scribed multiple sclerosis findings on brain
100
80
Percentage of Responses
MRI examinations. By utilizing a 12-item
structured reporting template for brain MRI
examinations of patients with known or sus-
pected multiple sclerosis, the authors were
able to show that reports obtained after tem-
plate implementation yielded a significant in-
crease in multiple sclerosis–relevant findings
and positive ratings by neurologists. Reports
obtained after template implementation men-

tures for nonstructured reports vs 10.6 of 12

features for structured reports (p < 0.01), and
they increased the sufficiency of information 60
needed for surgical planning [12].
The results of our study compare favor-
ably with those of other studies in the medi- 40
cal literature. There was an 80% (surgeon 1)
or 50% (surgeon 2) perceived improvement
20
in the completeness of information needed
to determine whether patients met the Milan
criteria in our study.
0
Ellerman et al. [10] did a similar analysis Fig. 4—Bar graph of
Before After Before After
by comparing the ability of structured and reader response to
Template Template Template Template
Surgeon 1 Surgeon 2
traditional MRI reports to predict osteochon- survey question 13
dritis dissecans lesion stability. International (“How satisfied are
Very Unsatisfied Neutral Satisfied
you with information
Cartilage Repair Society criteria were used contained in this Unsatisfied Very Satisfied
in the structured template. Radiologic im- report?”).

AJR:210, April 2018 771

 tioned 11.1 findings, whereas those obtained
before implementation mentioned 5.8 find-
ings (p < 0.001). The posttemplate reports
also were more likely to receive positive rat-
ings, compared with the pretemplate reports
(56% vs 28%; p = 0.01) [13]. Other authors
have also shown that structured reports are
preferred, clearer, and result in greater inter-
reader agreement [7, 8, 14].
Implementing a standardized reporting
template can be labor intensive and can take
time to gain traction. Although we had held
division meetings and circulated e-mails
about the clinical importance of lesion cate-
gorization with either the Liver Imaging Re-
porting and Data System or OPTN since Jan-
uary of 2013, routine voluntary reporting by
radiologists was sporadic. In January 2014,
when the OPTN template was made man-
datory, the number of reports classifying le-
sions with the use of these criteria substan-
tially improved; nonetheless, only 47% of
reports complied with template use within
the 20 months after introduction.
Correctness of classification also im-
American Journal of Roentgenology 2018.210:766-774.
proved. The template contained a built-in
decision support tool; essentially, an OPTN
cheat sheet was included below the impres-
sion section of the report, along with instruc-
tions indicating to delete the cheat sheet be-
fore report approval. This likely improved
efficiency and accuracy because dictating
trainees no longer had to spend time search-
ing online for OPTN classification materials,
which may or may not be reliable. Undoubt-
edly, simultaneous education of trainees and
faculty also had an effect.
One obvious reason for improved confi-
dence in the diagnosis of HCC might be that
the OPTN template is much more detailed
than the non-OPTN template. A detailed re-
port not only communicates more informa-
tion, but it also likely conveys more radiologist
confidence and competence. In addition to its
detail, it is arranged more logically in a format
(United Network for Organ Sharing paper-
work) that is recognizable and makes sense.
Perhaps no other outcome of our study is
more important than the ability to determine
whether a patient meets the Milan criteria and
thus qualifies for the MELD priority score. Al-
though the template improved the ability of the
surgeon to determine this, a larger study with
longer follow-up would be needed to determine
whether there was a difference in applications
for exception points between the two groups.
Even if that were not the case, less lofty goals
are also worthwhile, such as decreasing trans-
plant coordinator documentation time to make
it easier for radiologists to prepare or present
772 AJR:210, April 2018
at liver transplant tumor board and to improve
Poullos et al.
communication and surgeon satisfaction.
The present study has several limitations.
First, there may have been an inherent bias
of the surgeons to prefer reports containing
OPTN terminology, especially because they
had been lobbying for us to report it. We were
also concerned that the readers might guess the
goal of the study, thereby further biasing the
results. However, we took measures to conceal
this goal: the readers had no input into study
design, we did not discuss the reason for the
survey (which was administered nearly 2 years
after we implemented the OPTN template),
and all the reports were randomized and made
to look similar.
A potential source of bias is the inclusion
of multiple reports from the same subjects be-
cause similar disease burden and alliterative
reporting may have influenced subsequent re-
ports. Furthermore, the retrospective structure
of the study is a weakness. A prospective ran-
domized trial of reporting may have provided
more valid results; however, with 10 body im-
aging fellows and 20 faculty members (includ-
ing some from other divisions) interpreting
a large number of studies at multiple clinical
sites, randomization would have been imprac-
tical. Also, the fact that only 47% of the post-
template reports actually followed the OPTN
template could be viewed as a weakness. How-
ever, the inclusion of non-OPTN template
studies in our analysis actually strengthens the
results because improvement was still shown.
Of course, it is impossible to know which com-
ponent of the improvement was the result of
education and which was the result of the tem-
plate, but likely both are important. It would
be reasonable to expect that future studies that
only included the OPTN template would show
even more significant improvements.
In conclusion, the results of the present
study suggest that the implementation of a
standardized reporting template was valu-
able in improving communication between
radiologists and transplant surgeons. By
prompting the radiologist to comment on
specific details, the template was able to im-
prove the quality and completeness of tripha-
sic liver CT reports, as well as the satisfac-
tion of surgeons. With implementation of a
standardized reporting template, institutions
may be able to increase the accuracy of ra-
diologic diagnosis and staging of HCC and
be well equipped to determine whether pa-
tients qualify for MELD exception points.
References
1. Health Resources and Services Administration
(HRSA). Organ Procurement and Transplantation
Network: national data. Health Resources and
Services Administration website. optn.transplant.
hrsa.gov/data/view-data-reports/national-data.
Published 2017. Accessed April 9, 2017
2. Bernardi M, Stefano G, Maurizio B. The MELD
score in patients awaiting liver transplant: strengths
and weaknesses. J Hepatol 2011; 54:1297–1306
3. Health Resources and Services Administration
(HRSA). OPTN: policies. HRSA website. Policy
3.6 organ distribution: allocation of livers. optn.
transplant.hrsa.gov/governance/policies/. hrsa.gov/
policiesAndBylaws/policies.asp. Published 2012.
Accessed April 9, 2017
4. Wald C, Russo MW, Heimbach JK, ­ Hussain
HK, Pomfret EA, Bruix, J. New OPTN/UNOS pol-
icy for liver transplant allocation: standardization of
liver imaging, diagnosis, classification, and report-
ing of hepatocellular carcinoma. Radiology 2013
266:376–382
5. Kahn CE Jr, Langlotz CP, Burnside ES, et al. To-
ward best practices in radiology reporting.
­Radiology 2009; 252:852–856
6. Dunnick NR, Langlotz CP. The radiology report
of the future: a summary of the 2007 Intersociety
Conference. J Am Coll Radiol 2008; 5:626–629
7. Schwartz LH, Panicek DM, Berk AR, Li Y,
­Hricak H. Improving communication of diagnos-
tic radiology findings through structured report-
ing. Radiology 2011; 260:174–181
8. Naik SS, Hanbidge A, Wilson SR. Radiology reports:
examining radiologist and clinician preferences re-
garding style and content. AJR 2001; 176:591–598
9. Grieve FM, Plumb AA, Khan SH. Radiology re-
porting: a general practitioner’s perspective. Br J
Radiol 2010; 83:17–22
10. Ellermann JM, Donald B, Rohr S, et al. Magnetic
resonance imaging of osteochondritis dissecans:
validation study for the ICRS classification sys-
tem. Acad Radiol 2016; 23:724–729
11. RSNA Informatics Reporting website. Radiological
society of North America (RSNA). radreport.org/
index.php. Accessed January 30, 2018
12. Brook OR, Brook A, Vollmer CM, Kent TS, S ­ anchez
N, Pedrosa I. Structured reporting of multiphasic CT
for pancreatic cancer: potential effect on staging and
surgical planning. R ­ adiology 2015; 274:464–472
13. Dickerson E, Davenport MS, Syed F, et al. Effect of
template reporting of brain MRIs for multiple sclero-
sis on report thoroughness and neurologist-rated
quality: results of a prospective quality improvement
project. J Am Coll Radiol 2017; 14:371.e1–379.e1
14. Ghoshhajra BB, Lee AM, Ferencik M, et al. Inter-
preting the interpretations: the use of structured
reporting improves referring clinicians’ compre-
hension of coronary CT angiography reports. J Am
Coll Radiol 2013; 10:432–438
15. Mazzaferro V, Regalia E, Doci R, et al. Liver
transplantation for the treatment of small hepato-
cellular carcinomas in patients with cirrhosis.
New Engl J Med 1996; 334:693–700
(Appendixes start on next page)
 Multiphasic CT Reporting for HCC

APPENDIX 1: Milan Criteria for Liver Transplant for Patients With Hepatocellular Carcinoma

The Milan criteria for patient suitability to undergo liver transplant for hepatocellular carcinoma include the following [15]:
• Single tumor less or equal to 5 cm
• Presence of two to three tumors, none of which exceed 3 cm
• No indication of vascular invasion, extrahepatic manifestations, or both

APPENDIX 2: Standardized Paperwork Required for Placement of Patients on Transplant Waiting List

Figures 5–7 are templates provided by United Network of Organ Sharing (UNOS) for centers to use when recording hepatocellular carci-
noma imaging findings. Documentation of these findings is required by UNOS when applying for model for end-stage liver disease (MELD)
exception points for patients with end-stage liver disease and hepatocellular carcinoma who require placement on a transplant waiting list.
They are not to be submitted with the report, and centers may choose to develop their own documentation methods.
American Journal of Roentgenology 2018.210:766-774.

Fig. 5—Optional reporting template for MRI. OPTN = Organ Procurement Fig. 6—Optional reporting template for CT. OPTN = Organ Procurement
Transplantation Network, SI = signal intensity. © 2018 UNOS. Used with permission. Transplantation Network, SI = signal intensity. © 2018 UNOS. Used with permission.

Fig. 7—Optional summary report template. OPTN = Organ Procurement

Transplantation Network, HCC = hepatocellular carcinoma, MELD = model for end-
stage liver disease. © 2018 UNOS. Used with permission.

AJR:210, April 2018 773

 Poullos et al.

APPENDIX 3: Standard Reporting Template for APPENDIX 4: Template for Patients Without Stage T2
­Hepatocellular Carcinoma (HCC) Hepatocellular Carcinoma (HCC)
Assessment
LIVER:
LIVER: Morphology:
Cirrhosis: Focal lesions:
Portal hypertension: Hepatic vasculature:
Liver lesions: REMAINING ABDOMEN:
Largest lesion size: Biliary tree:
Lesion location: Gallbladder:
Late arterial hyperenhancement: Spleen:
Portal venous phase washout: Pancreas:
Portal venous phase pseudocapsule enhancement: Adrenal glands:
Delayed phase washout: Kidneys and ureters:
Delayed pseudocapsule enhancement: Gastrointestinal tract:
Tumor thrombus: Peritoneal cavity:
Largest axial dimension: (This does not include the pseudocapsule in Vasculature:
measurement)
Lymph nodes:
Largest CC dimension:
American Journal of Roentgenology 2018.210:766-774.

Abdominal wall:
Contrast phase used for measurement:
Musculoskeletal:
Size on prior study:
IMPRESSION:
Prior study date:
1. Number of HCC lesions:
Infiltrative hepatocellular carcinoma:
2. Largest lesion size:
Portal vein:
3. Findings meet OPTN criteria class:
Superior mesenteric vein:
4. Cirrhotic changes:
Splenic vein:
5. Portal hypertension:
Hepatic venous system:
Note—OPTN = Organ Procurement Transplantation Network.
Hepatic arterial system:
REMAINING ABDOMEN:
Biliary tree:
Gallbladder:
Spleen:
Pancreas:
Adrenal glands:
Kidneys and ureters:
Gastrointestinal tract:
Peritoneal cavity:
Vasculature:
Lymph nodes:
Abdominal wall:
Musculoskeletal:
IMPRESSION:
1. Number of HCC lesions
2. Largest lesion size:
3. Findings meet OPTN criteria class:
4. Cirrhotic changes:
5. Portal hypertension:
Note—CC = craniocaudal, OPTN = Organ Procurement Transplantation Network.

774 AJR:210, April 2018