Clinical learning
The development and validation of a knowledge, attitude
and behaviour questionnaire to assess undergraduate
evidence-based practice teaching and learning
Janice M Johnston, Gabriel M Leung, Richard Fielding, Keith Y K Tin & Lai-Ming Ho
Objectives Most evidence-based practice (EBP) edu-
cational assessment tools evaluated to date have
focused on specific knowledge components or tech-
nical skills. Other important potential barriers to the
adoption of EBP, such as attitudinal, perceptual and
behavioural factors, have yet to be studied, especially
in the undergraduate setting. Therefore, we devel-
oped and validated a knowledge, attitude and beha-
viour questionnaire designed to evaluate EBP
teaching and learning in an undergraduate medical
curriculum.
Methods We derived the questionnaire from a compre-
hensive literature review, informed by international and
local experts and a Year 5 student focus group. We
determined its factor structure and refined and valid-
ated the questionnaire according to the responses of a
cohort of Year 5 and a combined group of Years 2 and
3 students using principal components factor analysis
with varimax rotation. Factor reliability was computed
using Cronbach’s alpha coefficient. We assessed con-
struct validity by correlating the factors with other
Introduction
The evidence-based practice (EBP) approach to med-
ical care has been recognised as a key competency for
doctors.1,2 To provide optimal care, clinicians must be
able to locate, interpret and apply current best evidence
to a given clinical situation. However, they often lack
confidence in this process and in their own decision-
making skills.3,4
Previous research has shown that certain educational
interventions are effective in improving cognitive and
Department of Community Medicine, University of Hong Kong,
Hong Kong
Correspondence: Gabriel M Leung MD, MFPHM, Department of
Community Medicine, University of Hong Kong, Faculty of Medicine
Building, 21 Sassoon Road, Pokfulam, Hong Kong. Tel.: 00 852 2819
9209; Fax: 00 852 2855 9528; E-mail: gmleung@hku.hk
992  Blackwell Publishing Ltd M ED I C AL E D UC AT I O N 2003;37:992–1000
measures of EBP activity and examined responsiveness
through paired t-test of the pre ⁄ post factor mean scores.
Results A 43-item questionnaire was developed. Four
factors were identified from both student groups. The
overall questionnaire as well as each factor had high
construct validity (Cronbach’s alpha > 0Æ7 for each
scale). No significant correlations were found between
the 4 factors, confirming their orthogonality. Positive
correlations, however, resulted between factor mean
scores and other EBP activities. The responsiveness of
the questionnaire was satisfactory.
Conclusion A reliable knowledge, attitude and beha-
viour measure of EBP teaching and learning appropri-
ate for undergraduate medical education has been
developed and validated.
Keywords education, medical, undergraduate ⁄ *methods;
evidence based medicine ⁄ *education; curriculum; atti-
tude; questionnaires; reproducibility of results; Hong
Kong; literature review.
Medical Education 2003;37:992–1000
technical EBP skills.5,6 The main emphasis in this
literature tends to be on specific aspects of EBP skill
acquisition, such as asking clinical questions,6–8 search-
ing for evidence,6–8 retrieving appropriate evidence8
and critical appraisal techniques.6 These skills are
normally assessed through knowledge-based exercises
such as pre ⁄ post performance tests,3,5 EBP reports9
and questionnaires examining computer literacy3 and
self-assessed competencies.6,10
However, there is good evidence to indicate that
undergraduates may be inhibited in adopting the EBP
approach not only because they lack the aforementioned
cognitive and technical skills, but also because of other
equally important factors such as the lack of recognition,
support and use of EBP methods by clinical teach-
ers,10,11 time pressures,12 lack of peer group support,11
and an inability to adopt and use in a timely and efficient
KAB questionnaire for evidence-based practice • J M Johnston et al.
Key learning points
There is a lack of appropriate assessment tools for
evidence-based practice activities and processes,
especially in the undergraduate setting.
A reliable evaluation instrument, in the form of a
knowledge, attitude and behaviour questionnaire,
was developed and validated to measure the
effectiveness of evidence-based practice teaching
and learning activities.
Face, content, criterion and construct validity was
confirmed, as was responsiveness of the question-
naire.
manner evidence-based search and appraisal tech-
niques.12 These latter attributes of attitudes, percep-
tions and behaviour cannot be assessed through existing
modes of skill-based evaluation formats.
In addition, while much previous EBP research has
focused on postgraduate and continuing medical edu-
cation,5,10,13,14 very few reports have looked at the
undergraduate learning environment. A growing body
of literature exploring EBP teaching and learning
confirms the research deficit in undergraduate EBP
education and calls for further work in this area.3,6,9,15
We therefore developed and validated a knowledge,
attitude and behaviour (KAB) questionnaire at the
University of Hong Kong (HKU) designed to evaluate
EBP teaching and learning in the undergraduate
medical education setting.
Subjects and Methods
Subjects
The HKU undergraduate medical school curriculum
follows a 5-year, system-oriented, problem-based learn-
ing ⁄ lecture hybrid curriculum. The first 2 years focus
on preclinical activity and the final 3 on patient care
and clinical clerkships. Clinical exposure and experi-
ence with EBP increases as students progress through
the curriculum. All Year 5 students (n ¼ 159) were
recruited for the questionnaire development and pilot
testing. All Years 2 and 3 students (n ¼ 338) were
recruited for the questionnaire validation.
Identifying issues important to EBP teaching and
learning
The general content and specific items of the question-
naire were initially derived from a review of the
 Blackwell Publishing Ltd M ED I C A L E D UC A T I O N 2003;37:992–1000
993
published literature evaluating EBP teaching, educa-
tional interventions in medicine for undergraduates or
postgraduate and practising clinicians, and EBP teach-
ing evaluation questionnaires. A comprehensive search
of Medline from 1966 to 2001 using the MeSH terms
evidence-based medicine, education, medical-under-
graduate, clinical clerkship and educational measure-
ment yielded 66 papers. Thirteen of these papers
report specific educational interventions and the eval-
uation of EBP skills such as asking clinical questions,7,8
acquiring evidence,8,16 appraising evidence16 and
applying evidence in the clinical context.17 We found
only 1 paper that reported the development and
validation of a questionnaire to evaluate an educational
intervention for practising doctors in a continuing
medical education context.18
Questionnaire development
A 4-step approach was used to select items. Firstly, a
thematic review of EBP educational assessment and
specific questionnaire items10,18–20 described above
identified issues which were assembled into 33 multi-
nomial categorical response questions and 11 questions
requiring estimates, dichotomised or other responses.
These questions address:
1 knowledge (8 multinomial items, 1 item requiring an
estimate);
2 attitude (11 multinomial items);
3 practice (7 multinomial items, 8 discrete response
items, 1 open-ended item);
4 actual use of EBP (6 multinomial items, 1 item
requiring an open-ended estimate), and
5 anticipated future use of EBP (1 multinomial item).
(See Table 1).
Secondly, a focus group of Year 5 students (n ¼ 10)
was convened to inform the questionnaire development
process. Using semistructured questions, the focus
group explored students’ experiences of EBP teaching,
their perceptions of the usefulness of EBP to patient
care and learning, and the anticipated impact of EBP
on their daily clinical practice (Table 2).
Thirdly, a panel of international experts comprising
specialists in EBP, medical education, clinical medi-
cine, psychometrics, and evaluation and measurement
was asked to assess the preliminary questions and
provide structured comments with respect to face and
content validity, comprehensibility and comprehensive-
ness. The amended questionnaire was further refined
by a local panel, the members of which had expertise in
psychometrics, evaluation and measurement and were
familiar with questionnaire development in the local
994 KAB questionnaire for evidence-based practice • J M Johnston et al.

Table 1 Content of questionnaire items grouped under 5 categories

1 Knowledge of EBP13,19,20
Content: asking clinical questions, acquiring evidence, appraising evidence, applying evidence to a clinical
situation, and assessing treatment effectiveness in terms of patient outcomes
Question format: statements rated on a Likert scale (1 ¼ strongly disagree, 6 ¼ strongly agree)
2 Attitudes towards EBP6,13,20
Content: perceived need for information, willingness to practice EBP, perceived role of EBP in clinical
practice, attitude about EBP’s threat to clinical practice
Question format: statements rated on a Likert scale (1 ¼ strongly disagree, 6 ¼ strongly agree)
3 Practice of EBP6,13,19–21
Content: access and acquisition of evidence, application to patient care, influence of positive role models on
EBP adoption, barriers to adopting EBP, contribution of EBP to clinical reasoning and learning
Question format: statements rated on a Likert scale (1 ¼ never, 5 ¼ every day; 1 ¼ not at all, 6 ¼ completely)
4 Actual use of EBP
Content: current proportion of clinical activity based on EBP principles, frequency of actual use of EBP,
perceived need for EBP each day or week and for each patient encounter, overall use of EBP in the past year
Question format: mostly open-ended responses with some multiple choices
5 Future use of EBP19
Content: perceived future importance of EBP to medical practice, willingness to practise EBP in the future,
usefulness of EBP in the future, potential barriers to the adoption of EBP currently and in the future
Question format: statements rated on a Likert scale (1 ¼ very unwilling, 6 ¼ very willing; 1 ¼ completely
useless, 6 ¼ very useful; 1 ¼ not at all, 6 ¼ completely)

Cantonese-speaking context. While individual items
were not formally rated, all feedback was incorporated
through an iterative process from primary to secondary
feedback. An additional 3 multinomial categorical
items were generated as a result of this process.
Fourthly, the Year 5 (n ¼ 10) undergraduate med-
ical students were then asked to comment on the
comprehensibility and relevance of the items in the
questionnaire. Students were not specifically asked to
advise on comprehensiveness of the items as their
limited experience was considered likely to restrict their
appraisal of the full range of relevant issues. The
resultant preliminary draft questionnaire contained 36
multinomial categorical response questions and 11
questions requiring estimates, dichotomised or other
responses. This questionnaire was then returned to the

Table 2 Year 5 focus group discussion questions

1 How do you define EBP ⁄ EBM?

2 Can you tell me about the challenges or problems that you encountered in your previous EBP learning?
3 What did you find most useful about EBP learning? Why? Tell me more
4 What did you find least useful about EBP learning? Why? Tell me more
5 What aspects of EBP learning do you think will be most useful in facilitating your care for your patients?
(Break down into different areas, e.g. understanding the disease ⁄ illness, patient-related outcomes, clinical
decision making) Why? Tell me more. What else? (Probe the students to consider different areas in their
clinical practice)
6 Which aspect(s) in this EBP learning experience (focus on the tool they were using) did you find most
useful in clinical practice? Why? Tell me more. What else? (Probe the students to consider different areas
in their clinical practice)
7 Which aspect(s) in this EBP learning did you find least useful in clinical practice? Why? Tell me more.
What else?
8 Do you think you will use EBM in your daily clinical practice as a result of this EBP learning: Why? Or: Why not?
9 Has your intention to use EBM in your daily clinical practice changed (increased or decreased)? Which aspects
in this learning opportunity enhanced or discouraged your intention to integrate EBM in your clinical practice?
Why? Tell me more
10 Do you have other opinions about this EBP learning experience?
11 What about the perceptions of others, patients, clinical staff, professors?
What about the students’ reactions to these perceptions?

 Blackwell Publishing Ltd M ED I C AL E D UC AT I O N 2003;37:992–1000

KAB questionnaire for evidence-based practice • J M Johnston et al.
expert panel for final reconfirmation of its face and
content validity.
Questionnaire refinement
The preliminary 47-item questionnaire was adminis-
tered to all Year 5 undergraduate students (n ¼ 159) at
the Faculty of Medicine, University of Hong Kong. The
questionnaire contained 7 items using a 5-point scale
and 29 items using a 6-point scale, adopting both Likert
and adjectival scales (strongly agree)strongly disagree,
not at all)completely, never)all the time, very diffi-
cult)very easy, completely unprepared)completely
prepared, completely useless)very useful, very unwill-
ing)very willing). Negative statements were recoded for
the analysis. As none of the items were severely skewed,
the data can be analysed as interval data without
introducing bias.21 The 36 categorical response items
were included in a principal components factor analysis
with varimax rotation. Pairwise deletion of missing
values, eigenvalues ‡ 1 and factor loading scores
‡ 0Æ4 were used to sort items into factors.18,22 The
Scree plot was used to confirm the optimum number of
factors to include in the final set. Five items did not reach
the preset factor loading threshold and were excluded
from the questionnaire. The revised questionnaire
contained 42 items, 31 of which were multinomial.
The 4 factors derived from the factor analysis included
EBP knowledge (9 items), attitudes towards EBP
(6 items), personal application and use of EBP (8 items)
and future use of EBP (8 items). A summary score (the
mean score of all the items in each factor) and
Cronbach’s alpha coefficient were calculated for each
factor.
Evaluation of the final questionnaire
The final version of the questionnaire containing 43
items (with 31 categorical response items) was distri-
buted, completed and collected during a whole class
session for the Years 2 and 3 classes (n ¼ 293) at the
beginning of the academic year. The Year 2 students
completed the same questionnaire again at another
whole class session at the conclusion of the school year
8 months later.
The data were analysed according to the methods
described above. Following the determination of the
factor structure, Cronbach’s alpha was used to assess
the internal consistency of each factor.13,18,22,23 In
order to establish the independence of the factors and
to further examine factor construct validity, between-
factor correlations were calculated.10,18 Comparisons
between the factors and estimates of actual or perceived
 Blackwell Publishing Ltd M ED I C A L E D UC A T I O N 2003;37:992–1000
995
EBP activity were used to assess criterion validity.10,18
The students estimated:
1 the proportion of current medical practice that is
based on evidence;
2 their perceived need for evidence, and
3 their perceived change in self-behaviour in looking up
evidence compared to that of the previous year.
The respondents also reported how frequently they
engaged in EBP and whether they considered them-
selves to be EBP practitioners. As this approach
required multiple comparisons, only those correlations
with P £ 0Æ01 were considered significant.18
Responsiveness, or the extent to which the instru-
ment can detect change,10 was assessed on the basis of a
pre ⁄ post comparison of the Year 2 students, who had
undergone a full year of EBP teaching through a series
of 6 learning modules during the academic year. Paired
Student’s t-test was used to compare differences
between the pre ⁄ post mean factor scores.10,18 Effect
size was calculated to assess sensitivity to change.21
Ethics approval
This study received approval from the Undergraduate
Education Committee and the Faculty Ethics Com-
mittee, Faculty of Medicine, University of Hong
Kong.
Results
A total of 158 of 159 Year 5 students (response rate
99%) completed the preliminary, 47-item question-
naire. Individual item response rates ranged from 97%
to 100%. Students ranged in age from 21 to 29 years
but 94% were either 23 or 24 years old. The
male : female ratio was 2 : 1.
After the refinement procedure, the confirmatory
factor analysis of the 31 categorical items in the revised
questionnaire yielded 4 factors or subscales. The Scree
plot supported a 4-factor solution explaining 50Æ7% of
the variance. The 4 components derived were: factor
1 – EBP knowledge (9 items; Cronbach’s alpha ¼
0Æ88); factor 2 – attitudes towards EBP (6 items;
Cronbach’s alpha ¼ 0Æ79); factor 3 – personal applica-
tion and use of EBP (8 items; Cronbach’s
alpha ¼ 0Æ75), and factor 4 – future use of EBP (8
items; Cronbach’s alpha ¼ 0Æ76) (Table 3). Satisfac-
tory Cronbach’s alphas were noted for each factor.
(Means, standard deviations and factor loadings for
each item are detailed in the Appendix.)
The final version of the questionnaire was completed
by Years 2 and 3 students (293 ⁄ 338; response rate
996 KAB questionnaire for evidence-based practice • J M Johnston et al.

Table 3 Factor analysis for Year 5 and combined Years 2 and 3 students

Factor analysis for Year 5 students

Variance
Number Mean explained by Cronbach’s
Factor of items scale score SD each factor alpha

EBP knowledge 9 4.42 0.64 19.1 0.88

Attitudes towards EBP 6 3.73 0.56 11.8 0.79
Personal application and use of EBP 8 2.62 0.54 10.4 0.75
Future use of EBP 8 4.12 0.53 9.4 0.76
Total variance
explained ¼ 50.7

Factor analysis for Years 2 and 3 combined

Variance
Number Mean explained by Cronbach’s
Factor of items scale score SD each factor alpha

Future use of EBP 9 3.68 0.61 16.3 0.88

Attitudes toward EBP 6 3.86 0.45 11.8 0.74
EBP knowledge 5 4.63 0.58 11.8 0.80
Personal application and use of EBP 6 2.33 0.56 10.4 0.71
Total variance
explained ¼ 50.4

SD ¼ standard deviation.

86%). These students were predominately 20 (76%) or

21 (14%) years of age (range 19–24 years). The
male : female ratio was 1 : 1. Individual item response
rates ranged from 89% to 100%. Principal components
analysis indicated 4 components, as did the Scree plot,
which explained 44Æ8% of the variance. Although the 4
components derived were similar in content to the
factors in the Year 5 group, the factor order changed and
3 items were found to have moved between factors.
These were removed for the final Years 2 and 3 analysis.
Factor 1, representing EBP knowledge, loaded lower for
the Years 2 and 3 students, whereas their expected
future use of EBP loaded higher (Table 3). Satisfactory
Cronbach’s alphas were noted for each factor.
Correlation between factor scores
Correlation between factors and other
evidence-based activity
To further assess the criterion validity, a correlation
analysis was performed between the factor scores and
other evidence-based activity reported by the students.
Future use of EBP was positively correlated with the
students’ perceptions of themselves as EBP practition-
ers, frequency of practising EBP and need for evidence
per day. Personal application and use of EBP was also
positively correlated with the students’ perception of
how frequently they engaged in EBP and how fre-
quently they accessed evidence compared to during the
previous year. Evidence-based practice knowledge was
positively correlated with the need for evidence per
week (Table 5).

The factors derived from the combined Years 2 and 3

group were found to be coherent and represented
separate scales. As part of construct validation, interfac-
tor correlation coefficients comparing the mean score for
each of the 4 factors for each student were computed. No
significant correlations were found between the 4 factors,
confirming their orthogonality (Table 4).
Responsiveness
The Year 2 students completed the survey once at the
beginning of the academic year and then again
8 months later, following 6 modules of EBP teaching.
The pre ⁄ post assessment Year 2 mean factor scores
calculated on the basis of the items identified in the

 Blackwell Publishing Ltd M ED I C AL E D UC AT I O N 2003;37:992–1000

KAB questionnaire for evidence-based practice • J M Johnston et al. 997

Table 4 Correlation matrix between the combined Years 2 and 3 factor scores

Future use of Attitudes toward Evidence-based Personal application and

evidence-based evidence-based practice: use of evidence-based
practice practice knowledge practice

Future use of evidence-based practice 1.000

Attitudes toward evidence-based practice 0.059 1.000
Evidence-based practice: knowledge 0.016 )0.007 1.000
Personal application and use of 0.023 0.010 )0.038 1.000
evidence-based practice

F score ¼ 1.118; P-values all greater than ¼ 0.01.

Table 5 Correlation matrix between the 4 factors and other evidence-based activity for the combined Years 2 and 3 students

Future use Attitudes EBP Personal application

Item n of EBP towards EBP knowledge and use of EBP

Amount of current clinical activity 163 0.150 )0.031 0.015 0.061

based on evidence
Need for evidence per day 135 0.196* 0.095 0.122 0.142
Need for evidence per week 157 0.123 0.080 0.160* 0.153
Current frequency of accessing 153 0.078 0.108 0.013 0.387*
evidence compared to previous year
Frequency of practising EBP 163 0.365* 0.024 0.011 0.513*
Self-rating as an EBP practitioner 149 0.353* 0.026 0.118 0.147

*Significant at P £ 0.01.

Table 6 Year 2 pre ⁄ post-assessment comparison of mean differences in mean factor scores

Paired
Mean score Mean score t-test Effect
Factor n pre-assessment* post-assessment* (T score) P-value size

Future use of EBP 108 3.93 3.98 )0.777 0.439 0.075

Attitudes towards EBP 104 3.98 3.92 1.003 0.32 0.098
EBP knowledge 108 4.60 4.83 )3.498 0.001 0.33
Personal application 107 2.46 2.48 )0.329 0.74 0.031
and use of EBP

*Scored out of a maximum of 6.

combined Years 2 and 3 component structure were
compared. Evidence-based practice knowledge (factor 1)
was significantly different at the post-assessment and
demonstrated the largest effect size (Table 6).
Discussion
This paper has focused on the development and
validation of a self-administered questionnaire to assess
EBP knowledge, attitude, behaviour and perceptions in
undergraduate medical students. To our knowledge it is
the first to assess these non skill-based attributes for
EBP and also the first to assess EBP in the undergra-
duate setting.
Our findings indicate that the questionnaire has
satisfactory reliability and validity. The development of
the questionnaire was informed by issues identified as
important in the literature by international and local
experts and as relevant by the Year 5 focus group. The
acceptability of the questionnaire to the students was
demonstrated by the high response rates to individual
items and by the year groups as a whole. While the

 Blackwell Publishing Ltd M ED I C A L E D UC A T I O N 2003;37:992–1000

998 KAB questionnaire for evidence-based practice • J M Johnston et al.
combined Years 2 and 3 student response rate was
lower than that for Year 5, this most likely reflects the
lack of direct patient contact for these students and
therefore the lack of perceived direct relevance of some
of the questions to their learning routine.
The derived factors for both the confirmatory Year 5
and Years 2 and 3 analyses appeared to be independent,
coherent, consistent and multidimensional. They inclu-
ded knowledge of EBP, personal application and use of
EBP, attitudes towards EBP and future use of EBP. Each
factor had good internal consistency and construct
validity. The proportion of explained variance for each
factor was similar for Year 5 and combined Years 2 and 3.
While the content of each factor was similar, some
differences were noted and the 3 items loading on
different scales between Year 5 and Years 2 and 3 were
removed from the analysis to ensure coherence.
Although the factor loading order varied between the
Year 5 and combined Years 2 and 3 groups, the
consistency of the factors demonstrates suggests con-
gruence between these 2 subpopulations. Such variation
likely reflects the different educational and patient care
experiences of these students. To account for these
expected differences, we tested the questionnaire across
multiple year groups to ensure robustness.
The combined Years 2 and 3 correlation matrix of
the factors with other evidence-based activities sup-
ports this assessment. Three of the 4 scales (except for
attitudes towards EBP) were associated with increased
EBP activity as assessed by actual or anticipated
behaviour. While their current need for evidence
reflected their lack of patient contact, students did
perceive the need to acquire and use evidence in the
future. The mean difference between pre- and post-
assessment and the regression of the pre- on the post-
assessment for the combined Years 2 and 3 students
suggested the questionnaire was capable of measuring
change in EBP knowledge in undergraduate medical
students.
Evidence-based practice and the evaluation of EBP
teaching and learning is often reduced to one of its
several components, such as asking good clinical
questions, accessing and retrieving quality evidence,
appraising the evidence and applying this evidence to a
clinical situation.13 As changes in attitudes and per-
ceptions as well as knowledge are important precursors
to changes in behaviour, we set out to develop an
instrument to assess change in students’ EBP attitudes,
perceptions, behaviour and knowledge. Our analysis
demonstrates that this questionnaire supports the
assessment of EBP teaching in the undergraduate
medical curriculum and is useful in evaluating evi-
dence-based learning and teaching.
 Blackwell Publishing Ltd M ED I C AL E D UC AT I O N 2003;37:992–1000
Future research and development
We have presented preliminary results from a multistage
validation of an EBP evaluation questionnaire for use in
the undergraduate medical curriculum. This research
has contributed to the development of tools to assess the
effectiveness of undergraduate medical education. Fur-
ther research should focus on assessing the effectiveness
of the realisation of EBP in practice and on evaluating the
generalisability of these findings in other settings.
Contributors
JMJ participated in questionnaire development, data
analysis and the writing of the paper. GML conceived
and designed the study and participated in question-
naire development, data analysis and the writing of the
paper. RF participated in questionnaire development,
data analysis and the writing of the paper. KYK
undertook project co-ordination and data analysis.
LMH participated in questionnaire development and
data analysis.
Acknowledgements
We thank Dave Slawson, Mark Ebell, Howard Stras-
berg, Sunita Stewart and Wendy Lam for their invalu-
able input and advice during the drafting of the
questionnaire; and Marie Chi for expert secretarial
assistance in the preparation of the manuscript.
Funding
This study was funded by a Teaching Development
Grant from the University of Hong Kong.
References
1 World Federation for Medical Education. In: Walton H, ed.
Proceedings of the World Summit on Medical Education. Med
Educ 1994;28 (1):1–3.
2 Muller S. Physicians for the twenty-first century: report of the
project panel on the general professional education of the
physician and college preparation for medicine. J Med Educ
1984;59:127–8, 155–67.
3 Barnett SH, Kaiser S, Morgan LK et al. An integrated pro-
gramme for evidence-based medicine in medical school. Mt
Sinai J Med 2000;67:163–8.
4 Poses RM. Money and mission? Addressing the barriers to
evidence-based medicine. J Gen Intern Med 1999;14:262–4.
5 Smith CA, Ganschow PS, Reilly BM et al. Teaching residents
evidence-based medicine skills: a controlled trial of effective-
ness and assessment of durability. J Gen Intern Med
2000;15:710–5.
KAB questionnaire for evidence-based practice • J M Johnston et al.
6 Ghali WA, Saitz R, Eskew AH, Gupta M, Quan H, Hershman
WY. Successful teaching in evidence-based medicine. Med
Educ 2000;34:18–22.
7 Ellis P, Green M, Kernan W. An evidence-based medicine
curriculum for medical students: the art of asking focused
clinical questions. Acad Med 2000;75:528.
8 Rosenberg WM, Deeks J, Lusher A, Snowball R, Dooley G,
Sackett D. Improving searching skills and evidence retrieval.
J R Coll Physicians Lond 1998;32:557–63.
9 Thomas PA, Cofrancesco J Jr. Introduction of evidence-based
medicine into an ambulatory clinical clerkship. J Gen Intern
Med 2001;16:244–9.
10 Green ML, Ellis PJ. Impact of an evidence-based medicine
curriculum based on adult learning theory. J Gen Intern Med
1997;12:742–50.
11 Sackett DL, Parkes J. Teaching critical appraisal: no quick
fixes. [Editorial.] Can Med Assoc J 1998;158:203–4.
12 Sackett DL, Straus SE. Finding and applying evidence during
clinical rounds. The evidence cart. JAMA 1998;280:1336–8.
13 Bordley DR, Fagan M, Theige D. Evidence-based medicine: a
powerful educational tool for clerkship education. Am J Med
1997;102:427–32.
14 Green ML. Evidence-based medicine training in graduate
medical education: past, present and future. J Eval Clin Pract
2000;6:121–38.
15 Srinivasan M, Weiner M, Breitfeld PP, Brahmi F, Dickerson
KL, Weiner G. Early introduction of an evidence-based
medicine course to preclinical medical students. J Gen Intern
Med 2002;17:58–65.
 Blackwell Publishing Ltd M ED I C A L E D UC A T I O N 2003;37:992–1000
999
16 Wayland WC, Barry HC, Farquhar L, Holzman C, White A.
Training medical students in evidence-based medicine: a
community campus approach. Fam Med 1999;31:703–8.
17 Schoenfeld P, Cruess D, Peterson W. Effect of an evidence-
based medicine seminar on participants’ interpretations of
clinical trials: a pilot study. Acad Med 2000;75:1212–4.
18 Taylor R, Reeves B, Mears R et al. Development and valid-
ation of a questionnaire to evaluate the effectiveness of evi-
dence-based practice teaching. Med Educ 2001;35:544–7.
19 McAlister FA, Graham I, Karr GW, Laupacis A. Evidence-
based medicine and the practising clinician. J Gen Intern Med
1999;14:236–42.
20 McColl A, Smith H, White P, Field J. General practitioners’
perceptions of the route to evidence-based medicine: a ques-
tionnaire survey. BMJ 1998;316:361–5.
21 Streiner DL, Norman GR. Health Measurement Scales. A
Practical Guide to their Development and Use. 2nd edn. Oxford:
Oxford Medical Publications 1995;38,168.
22 Cork RD, Detmer WM, Friedman CP. Development and
initial validation of an instrument to measure physicians’ use
of, knowledge about and attitudes toward computers. JAMA
1998;5:164–76.
23 Bland JM, Altman DG. Validating scales and indexes. BMJ
2002;321:606–7.
Received 21 August 2002; editorial comments to authors 12 December
2002; accepted for publication 26 March 2003
1000 KAB questionnaire for evidence-based practice • J M Johnston et al.

Appendix
Combined Years 2 and 3 factor scores for individual items included in each component

Mean Factor loading

Statement (range 1–6) SD score

Future use of evidence-based practice

Compared to 1 year ago, how useful do you believe evidence-based 4.30 0.913 0.623
medicine will be in your future practice as a doctor?
Compared to 1 year ago, how willing are you to practise evidence-based 4.15 0.871 0.589
medicine as a doctor in the future?
You personally appreciate the advantages of practising evidence-based medicine 3.99 0.680 0.456
Evidence-based medicine should be an integral part of the 3.95 0.825 0.556
undergraduate medical curriculum
Compared to 1 year ago, how much do you support the principles of 3.74 1.015 0.788
evidence-based medicine?
Compared to 1 year ago, how much do you support lifelong learning using 3.71 1.103 0.698
evidence-based medicine techniques?
How much do you consider the practice of evidence-based medicine a 3.23 1.009 0.739
routine part of your learning?
How much has the practice of evidence-based medicine changed the way you learn? 3.18 1.012 0.763
How easy or difficult has it been for you to practise evidence-based medicine 2.93 0.879 0.556
as a medical student in the last month?
Attitudes toward evidence-based practice
If evidence-based medicine is valid, then anyone can see patients 4.04 0.75 0.626
and do what doctors do
There is no reason for me personally to adopt evidence-based medicine because 4.02 0.57 0.622
it is just a fad (or fashion) that will pass with time
Evidence-based medicine is cook-book medicine that disregards clinical experience 3.92 0.622 0.732
Doctors, in general, should not practise evidence-based medicine because medicine 3.89 0.657 0.678
is about people and patients, not statistics
Evidence-based medicine ignores the art of medicine 3.79 0.729 0.703
Previous work experience is more important than research findings in 3.43 0.833 0.417
choosing the best treatment available for a patient
Evidence-based practice – knowledge
Evidence-based medicine requires the use of critical appraisal skills to 4.81 0.844 0.782
ensure the quality of all the research papers retrieved
Effective searching skills ⁄ easy access to bibliographic databases and 4.76 0.811 0.773
evidence sources are essential to practising evidence-based medicine
Critically appraised evidence should be appropriately applied to the 4.64 0.772 0.747
patient using clinical judgement and experience
The evidence-based medicine process requires the appropriate identification 4.57 0.754 0.684
and formulation of clinical questions
Practising evidence-based medicine increases the certainty that the 4.38 0.734 0.547
proposed treatment is effective
Personal application and use of evidence-based practice
How frequently do you access medical evidence from a textbook? 3.91 1.19 0.553
How frequently to you access medical evidence in general? 2.87 1.01 0.710
How frequently do you access medical evidence on the Internet 2.47 0.880 0.692
(excluding Medline and Cochrane Reviews)?
How frequently do you access medical evidence from original research papers? 1.96 0.784 0.713
How frequently do you access medical evidence from the Cochrane database? 1.40 0.664 0.565
How frequently do you access medical evidence from secondary sources such as 1.30 0.601 0.554
the ACP Journal Club, the Journal of Evidence-Based Medicine, POEMs
(Patient Oriented Evidence that Matters) or CATs (Critically Appraised Topics)?

 Blackwell Publishing Ltd M ED I C AL E D UC AT I O N 2003;37:992–1000