ARTICLE IN PRESS

Education

Influence of Artificial Intelligence on

Canadian Medical Students'
Preference for Radiology Specialty: A
National Survey Study
D1X XBo Gong, D2X XMSc, D3X XJames P. Nugent, D4X XBSc, D5X XWilliam Guest, D6X XMD, PhD, D7X XWilliam Parker, D8X XMD, BMSc,
D9X XPaul J. Chang, D10X XMD, D1X XFaisal Khosa, D12X XMD, MBA, D13X XSavvas Nicolaou, D14X XMD

Abbreviations Rationale and Objectives: Artificial intelligence (AI) has the potential to transform the clinical practice of
radiology. This study investigated Canadian medical students’ perceptions of the impact of AI on radiol-
AI ogy, and their influence on the students’ preference for radiology specialty.
Artificial Intelligence
Materials and Methods: In March 2018, an anonymous online survey was distributed to students at all
CAR 17 Canadian medical schools.
Canadian Association of
Radiologists Results: Among 322 respondents, 70 students considered radiology as the top specialty choice, and
133 as among the top three choices. Only a minority (29.3%) of respondents agreed AI would replace
RSNA
radiologists in foreseeable future, but a majority (67.7%) agreed AI would reduce the demand for radiol-
Radiological Society of North
ogists. Even among first-choice respondents, 48.6% agreed AI caused anxiety when considering the
America
radiology specialty. Furthermore, one-sixth of respondents who would otherwise rank radiology as the
CaRMS first choice would not consider radiology because of the anxiety about AI. Prior significant exposure to
Canadian Resident Matching radiology and high confidence in understanding of AI were shown to decrease the anxiety level. Inter-
Service ested students valued the opinions of local radiologists, radiology conferences, and journals. Students
CACMS were most interested in “expert opinions on AI” and “discussing AI in preclinical radiology lectures” to
Committee on Accreditation of understand the impact of AI.
Canadian Medical Schools Conclusion: Anxiety related to “displacement” (not “replacement”) of radiologists by AI discouraged
LCME many medical students from considering the radiology specialty. The radiology community should edu-
Liaison Committee on Medical cate medical students about the potential impact of AI, to ensure radiology is perceived as a viable
Education long-term career choice.
Key Words: Artificial intelligence; Radiology; medical students; Specialty; Education; Survey.
© 2018 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.

INTRODUCTION Specifically, deep learning algorithms utilizing convolutional

neural networks have shown great potential in medical image

I
n recent years, there has been extensive discussion about the
rapid advancement of artificial intelligence (AI) technology,
and its potential impact on the future of radiology (1).
Acad Radiol 2018; &:1 12
From the MD Undergraduate Program, University of British Columbia, Van-
couver, British Columbia, Canada (B.G., J.P.N.); Department of Radiology,
Vancouver General Hospital, University of British Columbia, Vancouver, 899
12th Avenue West, British Columbia V5Z 1M9, Canada (B.G., J.P.N., W.G.,
W.P., F.K., S.N.); Department of Radiology, University of Chicago Medical Cen-
ter, Chicago, Illinois (P.J.C.). Received July 23, 2018; revised September 26,
2018; accepted October 1, 2018. Funding: This research did not receive any
specific grant from funding agencies in the public, commercial, or not-for-profit
sectors. Address correspondence to: B.G. e-mail: bogong.ustc@gmail.com
© 2018 The Association of University Radiologists. Published by Elsevier Inc.
All rights reserved.
https://doi.org/10.1016/j.acra.2018.10.007
processing (2). Furthermore, the availability of open-source
deep learning tools and public medical imaging datasets enables
independent software developers, without sophisticated medical
knowledge, to create an increasing number of AI products for
radiology (3). Although AI algorithms have been shown to out-
perform human radiologists in certain specific tasks, experts in
the field appear to agree that the complex work of a radiologist
requires general intelligence and thus cannot be solved by nar-
row AI solutions alone (1,3 5). In the foreseeable future, radi-
ology may shift to a model where radiologists are augmented
with AI, but a full integration of AI solutions into routine radiol-
ogy workflow could be slowed by existing barriers in IT infra-
structure, data curation, and regulatory approval (1,4,6,7).

1
 ARTICLE IN PRESS
GONG ET AL
Despite the limitations of existing AI technology and the
challenges associated with its utilization, there has been a
widespread misconception that equated the narrow tasks AI
could perform to the full scope of radiologists’ work, leading
to an exaggeration of the impact of AI on radiology (1,8).
Geoffrey Hinton, a prominent AI researcher, once famously
said radiologists were “like Wile E. Coyote in the cartoon,”
“already over the edge of the cliff,” and as a result medical
schools “should stop training radiologists now” (9). Fueled by
such bold statements, a pervasive pessimism that predicted an
imminent demise of the field of radiology was evident not
only in the general media (10,11), but also in influential med-
ical and radiology journals (12 14). Even among radiologists
who are optimistic about a future of AI-augmented radiol-
ogy, there has been discussion about “the likely scenario” of
the displacement of radiologists due to productivity gains and
the resultant reduction in demand for manpower (4).
In the midst of this pessimism and uncertainty, experts in
the field have warned about the potential negative influence
on medical students (15,16). For students who are potentially
interested in radiology, misinformation about AI and its
impact may discourage them from exploring this field and
choosing radiology as their career. The lost appeal to the
brightest medical students could result in a vicious cycle of
declining quality of applicants (17). For medical students not
interested in radiology, misconceptions about the roles of AI
in radiology would likely perpetuate during their nonradiol-
ogy professional careers, and may hinder collaboration with
radiologists to effectively utilize AI technology in health care.
Anecdotally, medical students interested in radiology have
increasingly sought opinions regarding the potential impact
of AI, from local radiologists, online radiology forums, or
experts in the field. Often, the students demonstrated a strug-
gle of shunning away from an otherwise deeply interesting
field (18,19). There have been stories about well-intentioned
medical school career advisors discouraging medical students
from considering radiology (20). The first and second authors
of this study are medical students and can attest to the increas-
ing anxiety about the impact of AI among ourselves and
other interested students. Despite the critical importance of
this issue, however, the literature on the influence of AI on
medical students’ specialty preference has been scant. In a
recently published article in European Radiology (21), Santos
et al distributed an electronic survey among medical students
in three major German medical schools. They examined 263
respondents’ prior knowledge about AI, and their perceptions
of AI's impact on the future of radiology and medicine. This
study however, did not identify the respondents’ career inter-
est, and therefore was not able to examine the influence of
their perceptions of AI on their preference for radiology as a
choice of specialty (21).
Therefore, it is important to focus on medical students
who are interested in radiology, and examine their per-
ceptions of the impact of AI on radiology, the factors con-
tributing to such perceptions, and the perceptions’
influence on their preference for radiology as a career. We
2
Academic Radiology, Vol &, No &&, && 2018
hypothesized that Canadian medical students who were
interested in radiology held an overall pessimistic percep-
tion regarding the impact of AI on radiologists, and as a
result experienced substantial anxiety in choosing radiol-
ogy as a career. We further hypothesized that these inter-
ested students had a low-to-moderate understanding of AI
technology, received conflicting opinions regarding AI's
impact, and were interested in learning more about AI.
To test these hypotheses, we developed an electronic
survey (Supplementary Document 1) and distributed it to
students in all 17 Canadian medical schools in March 2018.
In this 17-item survey, respondents were asked to indicate
their agreement with statements on a 7-point Likert scale
regarding the impact of AI on radiology, their level of anxiety
in pursuing radiology in the residency match, and their confi-
dence in their own understanding of basic principles of AI.
To directly assess the impact of AI on their specialty prefer-
ence, respondents were asked to specify their ranking of radi-
ology as a specialty choice, with or without the consideration
of AI. They were also asked to answer five true/false ques-
tions to demonstrate their level of understanding of deep
learning technology. At the end of the survey, respondents
were asked to identify the origins of opinions that contrib-
uted to their perceptions about AI, and rank a list of impor-
tant initiatives that could help medical students make
informed career decisions regarding AI and radiology. Our
results could provide important insight into medical students’
perceptions of AI in radiology, and their impact on their spe-
cialty preference. This in turn could help the radiology com-
munity develop effective strategies to educate medical
students and attract the best candidates to the field.
MATERIALS AND METHODS
Ethics Approval
This prospective anonymous online survey study received
ethics approval at the Institutional Behavioural Research
Ethics Board (H18-00373). Informed consent was confirmed
when a respondent agreed to start the survey online.
Survey Design
The survey was developed from a literature review of topics
in medical student specialty preference, recent advancement
in AI technology, and expert opinions on the impact of AI in
radiology. The survey underwent several rounds of internal
testing and feedback that involved nine medical students,
two radiology residents, and three staff radiologists.
The final survey (Supplementary Document 1) consisted
of 17 items, including multiple-choice questions, true/false
questions, 7-point Likert scale items, a slider scale question,
and a narrative question for additional comments. Questions
were focused on seven broad topics: (1) demographics
including medical school and year level; (2) prior exposure to
radiology and AI; (3) perceptions of the impact of AI on
 ARTICLE IN PRESS
Academic Radiology, Vol &, No &&, && 2018
radiology; (4) ranking of the radiology specialty with or with-
out the consideration of AI; (5) assessment of their confidence
and understanding of AI; (6) sources of information that con-
tributed to the perceptions of AI; and (7) recommendations
to help medical students make informed career decisions.
Survey Distribution
The target survey audience was Canadian medical students
who were interested in radiology. Because the target audi-
ence could not be easily accessed as a single group, we
attempted to distribute the survey to all Canadian medical
students, and asked the respondents to self-identify their spe-
cialty preference.
The electronic survey was hosted on the institutional
Qualtrics website. An email with a description and a link to
the online survey was sent to contacts in all 17 Canadian
medical schools (a full list of contacts in Acknowledgment).
For three medical schools (McMaster, Toronto, Saskatche-
wan), the survey was distributed by the Faculty of Medicine
offices to all currently enrolled medical students via internal
school mailing lists. For the remaining 14 medical schools,
the survey was distributed by student volunteers, mostly lead-
ers of local student radiology interest groups, who then for-
warded the link to students by various methods. Distribution
methods included emailing all students via an internal mailing
list, if such channel existed, emailing student members of the
radiology interest groups, or most commonly, posting the
survey link on closed Facebook groups of medical classes. For
these 14 schools, at least one round of reminder emails was
sent to the student contacts one week after the initial emails.
Participation in this anonymous survey was voluntary, with
no monetary incentives. Once started, a respondent had to
answer all but the last narrative comment question to com-
plete and submit the survey.
Data Analysis
Only completed and submitted responses were collected for
data analysis. Results were exported from the Qualtrics web-
site to local spreadsheets and analyzed using SPSS (IBM Cor-
poration) and SigmaPlot (Systat Software).
We considered a “conventional response rate” as the pro-
portion of respondents among Canadian medical students
who received the survey invitation. In our study, because dif-
ferent distribution methods were used at different medical
schools, the total number of students who received the survey
invitation was not exactly known. A lower bound of conven-
tional response rate was calculated based on the national
enrolment of medical students: 322/11,444 = 2.8% (22).
Because the survey was intended for students who were
interested in radiology, we defined a “relevant response rate” as
the proportion of survey respondents who indicated an interest
in radiology (R) among Canadian medical students who were
interested in radiology (S). The size of S was estimated to be
3.87 £ 82 = 317. Here, 3.87 is the average length of Canadian
INFLUENCE OF ARTIFICIAL INTELLIGENCE
MD programs in years, and 82 is the average number of
Canadian medical graduates who applied to radiology as first
choice in the residency match from 2016 to 2018 (23). Due to
the often limited exposure to radiology in medical school curric-
ulum (17), students who considered radiology as second or third
specialty choices in junior years may decide to commit to radiol-
ogy in senior years. Thus, a lower bound of the relevant response
rate was determined when considering R as only respondents
indicating radiology as the first choice (n = 70): 70/317 = 22%.
A more accurate estimate of the relevant response rate was
determined when considering R as follows: a combination of
year 1 respondents indicating radiology as one of the top three
specialty choices (n = 46), year 2 and 3 respondents indicating
radiology as one of the top two choices (n = 27 and 19), and
year 4 respondents indicating radiology as the first choice
(n = 17). The relevant response rate was thus estimated as
(46 + 27 + 19 + 17)/317 = 34%.
Statistics
Statistical analyses were performed by using the x2 analysis,
Mann-Whitney rank sum test, and Kruskal-Wallis one-way
analysis of variance (ANOVA) test. For x2 analysis, the Likert
responses strongly agree, agree, and somewhat agree were com-
bined into one group. The responses neither agree nor disagree,
somewhat disagree, disagree, and strongly disagree were combined
into a second group. For Mann-Whitney rank sum test and
Kruskal-Wallis one-way ANOVA, the Likert responses were
converted to ordinal data by assigning the following values:
strongly disagree, 1; disagree, 2; somewhat disagree, 3; nei-
ther agree nor disagree, 4; somewhat agree, 5; agree, 6; and
strongly agree, 7 (or similarly, strongly pessimistic 1, pessimis-
tic 2, somewhat pessimistic 3, mixed or neutral 4, somewhat
optimistic 5, optimistic 6, and strongly optimistic, 7). For all
tests, a p < 0.05 was considered statistically significant.
RESULTS
Responses From Canadian Medical Students
We focused on investigating the impact of AI on medical
students' preference for the radiology specialty, thus the tar-
get audience of the survey were Canadian medical students
who were interested in radiology. Because this target audi-
ence could not be accessed as a single group, we attempted
to distribute the survey to all Canadian medical students,
and asked the respondents to self-identify their specialty
preference. The distribution to the larger student body was
to ensure we fully reached the target audience.
A total of 322 complete responses were received from
medical students in all 17 Canadian medical schools, with a
national enrolment of 11,444 (22). A detailed breakdown of
the survey responses by medical school and student year level
is provided in Supplementary Table S1. Forty-four partial
responses were received, which were not included in the
analysis.
3
 ARTICLE IN PRESS
GONG ET AL Academic Radiology, Vol &, No &&, && 2018

TABLE 1. Characteristics of the Respondents Respondent Characteristics

Number of Percentage Among the 252 respondents who did not consider radiology as
Category Responses (Total = 322) their first specialty choice, 136 respondents disclosed their first
Radiology as specialty choice
specialty choices, which included internal medicine specialties
First choice 70 21.7% (n = 30, 22.1%), surgical specialties (n = 27, 20.0%), family med-
Second choice 29 9.0% icine (n = 26, 19.1%), emergency medicine (n = 10, 7.4%),
Third choice 34 10.6% psychiatry (n = 10, 7.4%), pediatrics (n = 9, 6.6%), neurology
Below third choice 75 23.3% (n = 8, 5.9%), anesthesiology (n = 6, 4.4%), radiation oncology
Uninterested 114 35.4% (n = 4, 3.0%), and pathology (n = 3, 2.2%), etc.
Educational background Other characteristics of the respondents, including their
Computer science/software 37 11.5% educational background, exposure to the field of radiology,
engineering and exposure to AI are summarized in Table 1.
Other sciences 251 78.0%
Other engineering 30 9.3%
Arts/Humanity 44 13.7%
Perceptions of the Potential Impact of AI on Radiology
Other 25 7.8%
Graduate degrees To assess the respondents’ perceptions of the potential impact of
In all fields 74 23.0% AI on the field of radiology, respondents were asked to rate
In computer science/software 6 1.9% their agreement with four statements on a seven-point Likert
engineering scale (results in Fig 1). Among the 133 respondents who con-
Exposure to radiology
sidered radiology as one of their top three specialty choices, the
Radiology shadowing 141 43.8%
vast majority (94.7%) agreed with (combining “strongly agree,”
Interest group events 138 42.9%
Radiology research 72 22.4%
“agree,” and “somewhat agree”) the statement regarding AI's
Radiology rotation/elective 58 18.0% technical capability (“AI will augment radiologist's capability, and
Radiology conferences 33 10.2% make radiologists more efficient”). Regarding the potential impact
None of the above 98 30.4% on the radiology workforce, slightly more than half of these
Exposure to AI respondents (58.6%) disagreed with (combining “strongly dis-
AI courses 80 24.8% agree,” “disagree,” and “somewhat disagree”) the statement
Computer science AI projects 52 16.1% “AI will replace radiologists in my lifetime.” However, a significant
Radiology AI projects 18 5.6% proportion of these respondents (67.7%) agreed with the state-
None of the above 221 68.6% ment “The impact of AI alone will reduce the number of radiologists
AI, Artificial intelligence. that are needed.” Despite the perception that AI could potentially
displace radiologists, an overwhelming majority (90.2%) of
these respondents agreed that “Radiologists should embrace artificial
Among the 322 responses, 70 respondents (21.7%) ranked intelligence, and work with the IT industry for its application.”
radiology as their first specialty choice, 29 respondents (9.0%)
as the second choice, 34 respondents (10.6%) as the third
Anxiety About AI Was Associated with Lowered
choice, 75 respondents (23.2%) below the third choice, and
Preferences for the Specialty of Radiology
114 respondents (35.4%) were not interested in radiology
(Table 1). To assess the level of anxiety about the potential impact of
AI, the respondents were also asked to indicate their agree-
ment with the “anxiety” statement: “The uncertain impact of
Conventional and Relevant Response Rates
artificial intelligence makes me worried to choose radiology as my
A conventional response rate is defined as the proportion of career.” As illustrated in Figure 2, even among respondents
respondents among medical students who received the invita- who considered radiology as their first choice, close to half
tion. A lower bound of the conventional response rate based on (48.6%) of these respondents agreed with the statement. Fur-
national enrollment was calculated to be 2.8% (detailed meth- thermore, the level of agreement increased as the ranking of
odology in the Materials and Methods section). radiology declined from the first to third choice (statistically
Because the survey was intended for Canadian medical stu- significant difference between the first and third choice
dents who were interested in radiology, a relevant response rate groups: p = 0.013, Kruskal-Wallis one-way ANOVA; level
is defined as the proportion of survey respondents who indi- of agreement: 62.1% in second-choice group, 67.6% in
cated an interest in radiology among the target audience. third-choice group, 56.4% in all three groups combined).
While a lower bound of the relevant response rate was calcu- Such results indicate that anxiety about AI, instead of a lack
lated as 22%, a more accurate estimate of the relevant response of intrinsic interest in radiology, at least partially contributes
rate was determined to be 34% (detailed methodology in the to the lowered preference for the radiology specialty in cer-
Materials and Methods section). tain medical students.

4
 ARTICLE IN PRESS
Academic Radiology, Vol &, No &&, && 2018 INFLUENCE OF ARTIFICIAL INTELLIGENCE

Figure 1. Interested students’ perceptions of the potential impact of AI. AI, Artificial intelligence.

Medical Students’ Perceptions of the Impact of AI
Discouraged Them From Considering the Specialty of
Radiology
To directly assess how medical students’ perceptions about
the impact of AI would influence their preference for the
radiology specialty, we asked the respondents to rerank radi-
ology as a specialty choice “if the potential impact of AI was not a
consideration” (results in Fig 3). The number of respondents
who considered radiology as their first specialty choice
increased from 70 to 84, by 20.0%. Notably, one respondent
with such attitude shift previously stated “not interested in
radiology.” Other respondents previously ranked radiology as
the second choice (8), the third choice (6), and lower than
the third choice (1). These results suggest that a considerable
proportion (one-sixth) of medical students, who would oth-
erwise consider radiology as their first specialty choice, have

Figure 2. Please rate your agreement: “The uncertain impact of AI makes me worried to choose radiology as my career.” AI, Artificial intelligence.

5
 ARTICLE IN PRESS
GONG ET AL Academic Radiology, Vol &, No &&, && 2018

the 160 respondents who, if not considering the potential

Figure 3. Perceived impact of AI discouraged interested students
from considering radiology specialty. AI, Artificial intelligence.
been discouraged from considering radiology based on their
perceptions of the impact of AI.
Similarly, if no consideration of the potential impact of AI was
made, the total number of respondents who considered radiology
as one of the top three specialty choices also increased substantially
(from 133 to 160, by 20.3%). A visual illustration of a detailed
breakdown of such ranking shift is provided in Figure 3.
Factors Associated with Medical Students’ Anxiety
About AI
To understand the factors that could be associated with medi-
cal students’ anxiety about AI, we analyzed responses from
impact of AI, ranked radiology as one of their top three spe-
cialty choices. These respondents were categorized into two
subgroups: those who agreed with the “anxiety” statement
(who answered “somewhat agree,” “agree,” or “strongly
agree”), and those who did not agree (who answered “neither
agree nor disagree,” “somewhat disagree,” “disagree,” or
“strongly disagree”).
Results of a univariable analysis of variables are shown in
Table 2. Among the variables, significant exposure to radiol-
ogy (conducting radiology research, attending radiology conferences,
or completing radiology rotations/electives) led to a lower fre-
quency of agreement with the “anxiety” statement (nonex-
posure: 76.6% vs exposure: 53.0%, p = 0.003 with x2
analysis). Exposure to AI (taking AI courses, conducting computer
science projects involving AI, or conducting radiology research projects
involving AI) also led to lower frequency of agreement, albeit
not reaching statistical significance (nonexposure: 68.9% vs
exposure: 56.1%, p = 0.148 with x2 analysis). Consistently,
respondents in the subgroup that did not agree with the
“anxiety” statement demonstrated a higher level of confi-
dence in their understanding of AI (median: “agree” vs “some-
what agree” with the statement “I have a good understanding of
what AI is,” p = 0.044 with Mann-Whitney rank sum test).
These results suggest that significant exposure to either radi-
ology or AI helps ease medical students’ anxiety about AI.
Another variable that showed a negative correlation with
the frequency of agreement with the “anxiety” statement
was respondent's year level (agree-subgroup: 2.0 § 0.1 years,
vs not-agree-subgroup: 2.5 § 0.2 years, p = 0.005 with

TABLE 2. Univariable Analysis of Variables Associated with Agreeing with the “Anxiety” Statement
Frequency of Agreement
Variable with the “Anxiety” Statement p Value

Diagnostic (DR) vs interventional

radiology (IR)
More interested in DR 35/58 (60.3%) 0.528
More or equally interested in IR 68/102 (66.7%) (x2 analysis)
Exposure to radiology
Research, conference, rotation 44/83 (53.0%) 0.003
None of the above 59/77 (76.6%) (x2 analysis)
Degree in computer science
Yes 18/25 (72.0%) 0.523
No 85/135 (63.0%) (x2 analysis)
Exposure to AI
Courses, research projects 32/57 (56.1%) 0.148
None of the above 71/103 (68.9%) (x2 analysis)
Did Not Agree with
Agreed with the “Anxiety” the “Anxiety”
Variable Statement (n = 103) Statement (n = 57) p Value

Year level (mean § SEM) 2.0 § 0.1 2.5 § 0.2 0.005 (Mann-Whitney)
“I have a good understanding “Somewhat agree” “Agree” 0.044 (Mann-Whitney)
of what AI is” (median)
AI, Artificial intelligence.

6
 ARTICLE IN PRESS
Academic Radiology, Vol &, No &&, && 2018 INFLUENCE OF ARTIFICIAL INTELLIGENCE

Mann-Whitney rank sum test). It is possible that the longer
time spent in medical school is associated with more opportu-
nities for meaningful exposure to radiology/AI, and thus
contributing to a lower level of anxiety about AI.
Medical Students’ Understanding of AI
While the potential impact of AI has been an important topic
among the radiology community, it is unclear how much
medical students actually understand about AI technology.
This survey attempted to assess medical students’ level of
understanding of AI using both self- and objective assessment.
First, respondents were asked to indicate their level of agree-
ment with the statement “I have a good understanding of what
AI is.” Among all the 322 respondents, the majority (258
respondents, 78.9%) agreed with the statement, including
responses “strongly agree” (54 respondents, 16.8%), “agree”
(90 respondents, 28.0%) and “somewhat agree” (110
respondents, 34.2%), reflecting a high level of confidence in
their understanding of AI (Table 3).
Second, we developed five statements regarding AI (all
statements are true, adapted from reference 24) that were pre-
sented as true/false questions for respondents to answer. These
five statements described basic principles of deep learning (use
of neural networks, direct learning from data, requirement of
large amount of labelled data, opaque thought process, and
inability in deductive reasoning) that are important for an indi-
vidual to have a meaningful understanding of AI's technical
capacity and limitations in the clinical setting (Table 3).
Respondents were encouraged to choose the “I do not know”
option as needed to avoid guessing. In contrast to the high
confidence level reflected in respondents’ self-assessment, only
half (49.7%) of the respondents answered at least three of the
five (60%) questions correctly. Notably, 30.7% of the respond-
ents failed to answer any question correctly. Only a small pro-
portion (14.0%) of respondents answered all five questions
correctly. The correct rate for the five questions ranged from
34.2% to 57.8% (average: 45.2%; Table 3). These results dem-
onstrate a gap between the respondents’ self- and objective
assessment of their level of AI understanding.
A univariable analysis of variables associated with the correct
rate reveals that either having a degree in computer science or
having prior exposure with AI through education or projects was
strongly associated with the medical student's higher number of
correct answers to the five questions regarding the key features of
AI (for both variables: median correct rate increased from 0.4 to
0.8; p < 0.001 with Mann-Whitney rank sum test; Table 3).
Sources of Opinions That Influenced Medical Students’
Perceptions of AI
To understand what sources of opinions influenced the med-
ical students’ perceptions of the impact of AI on the field of
radiology, respondents were asked to identify sources of
opinion and strength of influence (slider scale, 0 = no influ-
ence, 100 = extremely influential).

TABLE 3. Self-Assessment and Objective Assessment of Respondents’ Understanding of AI

“I have a good understanding of what AI is.” Respondents (Percentage) (n = 322)

Strongly disagree 2 (0.6%)

Disagree 15 (4.7%)
Somewhat disagree 22 (6.8%)
Neither agree nor disagree 29 (9.0%)
Somewhat agree 110 (34.2%)
Agree 90 (28.0%)
Strongly agree 54 (16.8%)
Statements Regarding AI Presented As True/False Questions Correct Rate

Deep learning is a class of machine learning algorithms that use multiple layers of neural networks. 51.2%
Deep learning methods learn directly from data, without the need of hand-engineered feature extraction. 34.2%
Application of deep learning in radiology requires large databases of labeled medical images. 57.8%
Deep learning systems are often opaque: it can be difficult to delineate the underlying “thought process.” 37.6%
Existing deep learning technology can achieve good pattern recognition, but lacks the ability of deductive reasoning. 45.0%
Variable Correct Rate (Median, [25%, 75%]) p Value

Degree in computer science

Yes (n = 37) 0.8, (0.4, 1.0) <0.001
No (n = 285) 0.4, (0, 0.8) (Mann-Whitney)
Exposure to AI
Courses, research projects (n = 101) 0.8, (0.6, 0.9) <0.001
None of the above (n = 321) 0.4, (0, 0.6) (Mann-Whitney)
AI, artificial intelligence.

7
 ARTICLE IN PRESS
GONG ET AL
Results from the 160 respondents who, if not considering
the potential impact of AI, ranked radiology as one of their
top three specialty choices, were analyzed (Fig 4). The most
common sources of opinion were local radiologists (n = 115),
fellow medical students (n = 115), nonradiology physicians
(n = 88), and the general medial (n = 74). There was a statisti-
cally significant difference of the strengths of influence of var-
ious opinion sources (p < 0.001, Kruskal-Wallis one-way
ANOVA). The most influential sources of opinions were, in
descending order: local radiologists (median influence: 80,
n = 115), radiology conferences (median: 67, n = 20), and
radiology journals (median: 62, n = 37). The medium tier
includes radiologists from online forums, such as Aunt Min-
nie, Student Doctor Network, or PreMed 101 (median
Figure 4. Sources of opinion on the impact of AI. AI, Artificial intelligence.
8
Academic Radiology, Vol &, No &&, && 2018
influence: 50, n =57), nonradiology physicians (median; 50,
n = 88), and computer scientists (median: 50, n = 48). The
least influential sources were the general media (median
influence: 20, n = 74), fellow medical students (median: 25,
n = 115), and family members (median: 38, n = 44).
Respondents were also asked to indicate the level of opti-
mism or pessimism from these sources regarding the impact
of AI on radiologists (radiology workforce), which again
showed statistically significant difference (p < 0.001, Krus-
kal-Wallis one-way ANOVA). The most optimistic group
of opinions were from local radiologists, radiology confer-
ences, and radiology journals (for all three sources,
median = “somewhat optimistic”), followed by radiologists
from online forums (median = “mixed or neutral”). For the
 ARTICLE IN PRESS
Academic Radiology, Vol &, No &&, && 2018
remaining opinion sources (nonradiology physicians, fellow
medical students, computer sciences, family members, and
general media), their median results were all “somewhat pessi-
mistic.” Interestingly, those opinions that respondents relied
upon more, were also more optimistic about the impact of
AI. An illustrative breakdown of results for each opinion
source is provided in Figure 4.
What Can the Radiology Community Do to Help Medical
Students?
At the end of the survey, respondents were asked to select up
to three initiatives they felt were most important, that could
be done by the radiology community to help medical stu-
dents make informed specialty decisions regarding the impact
of AI on radiology. Notably, as shown in Figure 5, whether
or not respondents were primarily interested in radiology, the
rankings of their recommendations were similar. For both
subgroups, the most recommended initiatives were “inviting
experts to provide opinions on the impact of AI” (71.4% from
respondents ranking radiology as top three choices, 69.8%
from respondents ranking radiology lower than the third
choice), and “discussing AI in preclinical radiology lectures”
(59.4%, 73.5%). The next three recommended initiatives
were “publishing position statements of radiology organizations”
(31.6%, 20.6%), “offering courses on AI” (29.3%, 30.2%), and
“explaining the perspectives of radiologists in general media”
(29.3%, 29.6%).
Figure 5. What are the most important initiatives to help medical students make informed decision regarding the impact of AI? AI, Artificial
intelligence.
INFLUENCE OF ARTIFICIAL INTELLIGENCE
The respondents were also asked if they were aware of ini-
tiatives within the radiology community that were underway
at the time of survey (March 2018): the launch of a new jour-
nal Radiology: Artificial Intelligence by the Radiological Society
of North America (25), the formation of the Canadian Asso-
ciation of Radiologists (CAR) AI Working Group (26) and
its plan to publish a White Paper (1), and a list of available AI
recourses on the CAR website (27). The awareness of such
initiatives was overall low: Radiological Society of North
America new AI journal: 7.5% among respondents who
ranked radiology as one of top three choices, and 2.1% in
remaining respondents; CAR AI Working Group: 6.8% and
3.7%; CAR AI resource list: 9.0% and 2.1%.
Narrative Comments
As an optional question, the respondents were asked to pro-
vide any additional comments by free text with no length
limit. A total of 52 comments were received.
Several respondents correctly pointed out that radiology
was not the only medical specialty to be affected by AI. Some
other affected specialties could include pathology, dermatol-
ogy, family medicine, internal medicine, and surgery.
A number of respondents emphasized that although AI was
not likely to replace radiologists, the improved efficiency by
an AI-augmented model could lead to uncertain ramifica-
tions. Many commented that AI would add value to patient
care, but at a cost of a reduced demand for radiologists. They
9
 ARTICLE IN PRESS
GONG ET AL Academic Radiology, Vol &, No &&, && 2018

emphasized that, instead of the doomsday predication of DISCUSSION

being entirely replaced, this potential displacement was the
As many believe in this era of rapid advancement in AI tech-
true concern for interested medical students. However, one
nology, we are on the verge of a far-reaching shift in radiol-
respondent shared his/her view that the declined workload
ogy that will profoundly affect practising radiologists,
caused by AI could be compensated for by an overall trend of
trainees, other physician colleagues, and patients (1). As the
increasing utilization of medical imaging. One other respon-
radiology community prepares for this shift with a sustained
dent predicted radiologists would spend more time on com-
commitment to uphold high standards of patient care, it is
plex cases, and another respondent wondered if radiologists
critical to inform and engage medical students in the process,
would spend the extra time in direct patient education and
to ensure a bright future for the field.
consultation. A few respondents predicted that the field
Although a recent survey study explored German medical
would shift to a greater focus on interventional radiology.
students’ perceptions of the impact of AI on radiology and
While the survey focused on questions regarding the
medicine (21), no study has been published to specifically
workforce of radiologists, a few respondents emphasized
examine the influence of such perceptions on medical stu-
that the goal of the radiology profession was to improve
dents’ preference for the radiology specialty. Thus, the nov-
patient care. One respondent specifically tackled the issue
elty of our findings in this national survey study is three-fold:
from a moral standpoint: “I truly hope that radiologists will
(1) our results revealed considerable anxiety about AI in med-
not let greed win the moral battle and engage in advanced technol-
ical students interested in radiology, with a substantial pro-
ogy, like AI, to develop superior diagnostic instruments for the
portion (one-sixth) of interested medical students
next generation and, as a result, help improve the quality of
discouraged from considering radiology solely because of the
care.” Another respondent said: “Radiologists will not be able
uncertain impact of AI. (2) Although most respondents dis-
to stop this change in medicine so they better get ahead of it and
agreed with a “replacement” view, the respondents agreed
embrace it.” There was also a call for a “collaborative”
with a potential “displacement” of radiologists by improved
approach to obtain the “perspective of the computer sci-
efficiency with AI augmentation, which could have
ence community” in this process.
caused the anxiety among interested students. (3) Medical
Several respondents touched upon the importance of
students considered radiologists’ opinions with high regard,
educating medical students and the radiology community
and demonstrated a strong interest for education on AI tech-
at large about the “fundamentals” of AI technology. Many
nology in the medical school curriculum.
specifically emphasized that AI “has not been mentioned at all
Our results show that medical students considered the
in the entire medical school curriculum so far” despite student
opinions from local radiologists, radiology conferences, and
interest. These students stated that they would like to have
journals more influential, which were also groups whose
this “elephant in the room” topic addressed in lectures.
opinions were more optimistic. This could explain that only
Another respondent predicted that education on this sub-
a minority of our respondents agreed with the doomsday
ject “would contribute to a less fatalistic view on the impact of AI
“replacement” prediction, consistent with the German sur-
on radiology,” and advocated for broader medical school
vey study (21). However, the respondents still experienced
admission policy to encourage enrolment of “students
considerable anxiety about AI, even among about half of
with a background in computer science, engineering,
those who ranked radiology as their first specialty choice.
math, and physics.”
The anxiety was confirmed by the results that one-sixth of
One particular respondent shared a detailed recollection
respondents who would potentially consider radiology as
of his/her attitude shift regarding AI, and the influence of
their first choice would not do so if they considered the
online expert opinions on his/her decision to apply to the
impact of AI. The increasing level of anxiety in respondents
radiology specialty. This personal experience was consis-
who ranked radiology lower further demonstrated that AI
tent with the main results of the survey, and could be a
was one major factor contributing to many respondents’
representative example for many respondents. The full
lower preference for the radiology specialty.
comment is quoted below:
The above results suggest that the “replacement” view was
“Some radiologists I met seemed to say that artificial intelligence in not a major cause of medical students’ career-altering anxiety.
diagnostic radiology is not ready to replace radiologists and that it Instead, a large proportion of respondents agreed that there
is probably going to take many years before we come up with some- would be potential reduction of demand for radiologists as a
thing powerful and useful. However, at first, I was scared when I result of improved efficiency with AI. Several respondents
heard of this AI thing because I thought that radiologists will lose emphasized in the narrative comments that this “displacement”
their jobs so I started hesitating about applying to this specialty. view was the true trigger for their anxiety about AI.
Then, I watched conferences on Youtube made by other radiolog- Although the potential displacement of the radiology work-
ists and they were saying that AI will more likely be a tool for force is indeed a possibility (4), it is likely that medical stu-
radiologists and that it will allow them to work faster and that's dents have over-estimated the extent of the displacement,
why I stopped hesitating about applying to this specialty.” leading to significant stress. In fact, the univariable analysis in

10
 ARTICLE IN PRESS
Academic Radiology, Vol &, No &&, && 2018
Table 2 suggests that an improved understanding of the scope
of the radiologists' work (through prior exposure to radiol-
ogy), or an improved understanding of AI technology
(potentially through prior exposure to AI), could lead to a
deeper understanding of the potential roles of AI in radiol-
ogy, and hence ease the medical students’ anxiety about AI.
It is important to note that, despite significant anxiety
about the uncertain impact of AI, respondents overwhelm-
ingly (90.2%) supported the notion of collaborating with the
IT industry to facilitate the development and application of
AI in radiology, for AI's potential value to improve patient
care. Such commendable attitudes toward the patient-
centered practice of radiology, despite perceived detrimental
consequences to the radiologist workforce, were further con-
firmed by the narrative comments of several respondents.
Despite respondents’ overall high level of confidence in
their AI understanding, the correct rate of the five true/false
AI knowledge questions was low (45.1%). This discrepancy
indicates there is significant room for education on the basic
principles of AI technology among medical students. Our
univariable analysis further shows that taking AI courses or
getting involved in AI projects were equally effective as for-
mal computer science education in improving medical stu-
dents’ understanding of AI.
In recent months, there have been discussions about the
notion that “radiologists who use AI will replace radiologists
who don't” (28). As the practice of radiologists shifts with the
application of AI, it is possible that such changes would attract
trainees who are more open to technology and discourage
those who are more tech-averse. In our survey, a relatively
high proportion (11.5%) of respondents had formal training
in computer science/software engineering, which could be
due to a selection bias because those individuals could be
more interested in completing a survey regarding AI. But it
would be interesting to examine if such trend does occur in
the composition of radiology applicants and accepted train-
ees, through longitudinal national data collection of radiology
residency programs. Such information would be helpful for
the field to align residency curriculum with the needs of
future AI-augmented radiology practice.
One thing that we did not anticipate was the number of
responses from medical students who were not primarily
interested in radiology, constituting 58.7% of the total
responses, despite the survey's stated focus on radiology.
Responses from these students could be driven by a general
interest in AI: As many respondents correctly pointed out,
radiology will not be the only medical specialty affected by
AI. It is interesting to note that the respondents, whether or
not primarily interested in radiology, provided almost identi-
cal rankings for initiatives that would help medical students
become informed about the potential impact of AI, indicat-
ing a high acceptance of these highly ranked initiatives.
Similar to their American counterparts, Canadian medical
students undergo a typical educational path that usually starts
with broad undergraduate studies, followed by 3 4 years of
medical school education, and then residency training in
INFLUENCE OF ARTIFICIAL INTELLIGENCE
radiology with a national residency match (the Canadian Resi-
dent Matching Service, CaRMS). The Canadian MD pro-
grams are jointly accredited by the Committee on
Accreditation of Canadian Medical Schools and the Liaison
Committee on Medical Education, the latter of which is also
responsible for the accreditation of MD programs in the United
States. The shared standards of medical school curriculum sug-
gest that the results of this Canadian study could possibly be
extended and generalized to the medical student population in
the United States. Further studies should be carried out in other
jurisdictions, to assess if the same influences of AI can also be
observed in medical students in other regions.
LIMITATIONS
A potential limitation of our survey study is its rather low rele-
vant response rate, referred to as “response rate” below for sim-
plicity (lower bound: 22%, more accurate estimate: 34%). This
low response rate could reflect selection bias of our respond-
ents, and thus limit the generalizability of the interpretation of
certain results. On the other hand, this response rate is in the
range of typical response rates in surveys of medical students
(29 33). As confirmed by many student volunteers who dis-
tributed our survey, Canadian medical students are inundated
with internal and external surveys, and as a result experience
significant survey fatigue, which could contribute to the low
response rate. Furthermore, as demonstrated in marketing
research, surveys with a response rate of this range could per-
form the same or better than surveys with higher response rates
(34 36). Therefore, this concern may be over-stated.
The survey could be improved by further exploring the
issue of interventional versus diagnostic radiology. Some
respondents mentioned a potential shift toward interven-
tional radiology in future radiology practice, echoing some
trainees’ intention to be trained in interventional radiology as
a coping strategy in preparation for the potential displace-
ment of diagnostic jobs (19). Unlike the relatively recent
implementation of separate residency training pathways for
interventional and diagnostic radiology in the United States,
the Canadian radiology residency programs remain unified.
Future studies would be helpful to further elucidate this issue.
CONCLUSION
Medical students are the future of radiology. Our study
revealed considerable anxiety among medical students inter-
ested in radiology, caused by their perceptions of the impact
of AI. Although interested students generally disagreed with
a “replacement” view, many were concerned about the
potential “displacement” of radiologists by improved effi-
ciency with the future AI-augmented model. Such anxiety
discouraged a significant number of medical students from
considering the radiology specialty. In anticipation of a signif-
icant shift in radiology practice as a result of AI technology,
the radiology community should help educate medical
11
 ARTICLE IN PRESS
GONG ET AL
students about the potential impact of AI, to relieve student
anxiety and ensure the long-term prosperity of the field.
ACKNOWLEDGMENTS
The authors thank medical students Yuhao Wu, John Liu,
Sally Choi, Jeremy Mak, Sarah Yeo, Cody Lo, and David
Jung for providing feedback on the drafts of the survey. The
authors also thank the following individuals and organizations
for assistance in distributing the survey, in alphabetic order of
the medical school name: Dalhousie University: Malik Ali,
Meghan Wentzell; McGill University: Ann-Sophie Lafreni
ere;
McMaster University: Michael G. DeGroote School of Medi-
cine Student and Resident Affairs; Memorial University of New-
foundland: Andrew Fenwick; Northern Ontario School of
Medicine: Omar Daher; Queen's University: Lauren Mak; Uni-
versite de Montreal: Nohad El Sayed, Hetoum Misirliyan;
Universite de Sherbrooke: Aisin Ibrahim; Universite Laval: Marie-
Lyne Bourque; University of Alberta: Ruojin Bu, Mariam Nar-
ous; University of British Columbia: Ariel Kwan, Jasmine Lo;
University of Calgary: Charlene Blomquist, Parthiv Amin,
Danielle Smith; University of Manitoba: Maria Koleva, Prabjot
Singh; University of Ottawa: Maria Doubova, Kathryne
DesAutels, Chris Welsh, Stefan Siedlikowski, Coralea Kap-
pel, Alexander Pearson, Alexandre Rochon, Mohamed Ali,
Vanessa Russell; University of Saskatchewan: College of Medi-
cine Undergraduate Medical Education Office; University of
Toronto: Vice Dean of the MD Program Dr. Patricia Hous-
ton; and Western University: Lucy Samoilov, Brian Robertson.
The authors thank Dr. Chen Wan for critical reading of the
manuscript, and suggestions on figure preparation.
REFERENCES
1. Tang A, Tam R, Cadrin-Chenevert A, et al. Canadian association of radi-
ologists white paper on artificial intelligence in radiology. Can Assoc
Radiol J 2018; 69:120–135.
2. McBee MP, Awan OA, Colucci AT, et al. Deep learning in radiology. Acad
Radiol 2018; 25(11):1472–1480.
3. Choy G, Khalilzadeh O, Michalski M, et al. Current applications and future
impact of machine learning in radiology. Radiology 2018; 288(2):318–328.
4. Liew C. The future of radiology augmented with artificial intelligence:
a strategy for success. Eur J Radiol 2018; 102:152–156.
5. Recht M, Bryan RN. Artificial intelligence: threat or boon to radiologists?
J Am Coll Radiol 2017; 14:1476–1480.
6. McGinty GB, Allen Jr. B. The ACR Data Science Institute and AI Advisory
Group: harnessing the power of artificial intelligence to improve patient
care. J Am Coll Radiol 2018; 15:577–579.
7. Kahn Jr. CE. From images to actions: opportunities for artificial intelli-
gence in radiology. Radiology 2017; 285:719–720.
8. Munk PL. A brave new world without diagnostic radiologists? Really?
Can Assoc Radiol J 2018; 69:119.
9. Mukherjee S. A.I. versus M.D. The New Yorker. New York City, NY,
United States: Conde  Nast, 2017.
10. Why scan-reading artificial intelligence is bad news for radiologists.
The Economist. London, England. 2017.
11. Musa M. Opinion: rise of the robot radiologists. The Scientist. New York
City, United States: LabX Media Group, 2018.
12
Academic Radiology, Vol &, No &&, && 2018
12. Schier R. Artificial intelligence and the practice of radiology: an alterna-
tive view. J Am Coll Radiol 2018; 15:1004–1007.
13. Obermeyer Z, Emanuel EJ. Predicting the future—big data, machine
learning, and clinical medicine. N Engl J Med 2016; 375:1216–1219.
14. Chockley K, Emanuel E. The end of radiology? Three threats to the future
practice of radiology. J Am Coll Radiol 2016; 13:1415–1420.
15. Chang P. Video from RSNA 2016: AI and radiology—separating hope
from hype. http://www.youtube.com; 2016.
16. Parker W. Cutting through the AI hype B.S.: Drs. Eliot Siegel, Rasu
Shrestha, Greg Moore and Mr. Elad Benjamin talk AI and the future of
radiology. www.medium.com; 2017.
17. Gunderman RB. Medical students are our future. J Am Coll Radiol 2005;
2:795–797.
18. Siegel E, Schilling R. Rest assured, Joa ~o, you are safe from artificial intel-
ligence. Appl Radiol 2017; 46:4–8.
19. How will AI affect radiology over the next 20 years? http://www.auntmin-
nie.com; 2017.
20. Gagnon L. Radiology leaders must counter hype over AI in radiology.
http://www.auntminnie.com; 2018.
21. Pinto Dos Santos D, Giese D, Brodehl S, et al. Medical students' attitude
towards artificial intelligence: a multicentre survey. Eur Radiol 2018.
22. The Association of Faculties of Medicine of Canada. In: 2017 Canadian
medical education statistics; 2017.
23. Canadian Resident Matching Service. R-1 match interactive data. http://
www.carms.ca; 2018.
24. Chartrand G, Cheng PM, Vorontsov E, et al. Deep learning: a primer for
radiologists. Radiographics 2017; 37:2113–2131.
25. Radiological Society of North America. RSNA to publish three new journals.
http://www.rsna.org; 2017.
26. Canadian Association of Radiologists. Artificial intelligence. http://www.
car.ca; 2018.
27. Canadian Association of Radiologists. AI resources. http://www.car.ca;
2018.
28. Stanford University Center for Artificial Intelligence in Medicine & Imag-
ing. RSNA 2017: Rads who use AI will replace rads who don't. http://
aimi.stanford.edu; 2017.
29. Arleo EK, Bluth E, Francavilla M, et al. Surveying fourth-year medical stu-
dents regarding the choice of diagnostic radiology as a specialty. J Am
Coll Radiol 2016; 13:188–1895.
30. Oskvarek JJ, Brower JV, Mohindra P, et al. Educational impact of a
structured radiation oncology clerkship curriculum: an interinstitutional
comparison. J Am Coll Radiol 2017; 14:96–102.
31. Isik U, Wouters A, Ter Wee MM, et al. Motivation and academic perfor-
mance of medical students from ethnic minorities and majority: a com-
parative study. BMC Med Educ 2017; 17:233.
32. Kerneis S, Jacquet C, Bannay A, et al. Vaccine education of medical stu-
dents: a nationwide cross-sectional survey. Am J Prev Med 2017; 53:
e97–e104.
33. Bruce AN, Kumar A, Malekzadeh S. Procedural skills of the entrustable
professional activities: are graduating US medical students prepared to
perform procedures in residency? J Surg Educ 2017; 74:589–595.
34. Visser P, Krosnick J, Marquette J, et al. Mail surveys for election fore-
casting? An evaluation of the colombia dispatch poll. Public Opin Q
1996; 60:181–227.
35. Keeter S, Kennedy C, Dimock M, et al. Gauging the impact of growing
nonresponse on estimates from a national RDD telephone survey. Public
Opin Q 2006; 70:759–779.
36. Bosmans JM, Weyler JJ, De Schepper AM, et al. The radiology report as
seen by radiologists and referring clinicians: results of the COVER and
ROVER surveys. Radiology 2011; 259:184–195.
SUPPLEMENTARY DATA
Supplementary data related to this article can be found at
https://doi.org/10.1016/j.acra.2018.10.007.