Espellardo EBM Self-Instructional Manual

EVIDENCE BASED MEDICINE:
Self-Instructional Manual
Noel L. Espallardo, MD, MSc
Department of Family and Community Medicine

UP College of Medicine and Philippine General Hospital
Evidence-based Medicine: Introductory Module
PARTICIPANTS’ HANDOUT
EVIDENCE BASED MEDICINE
OBJECTIVES

The purpose of the reading assignment is to introduce to the participants the concept of
Evidence‐based Medicine in medical decision making. It also serves to introduce
Medline and how an efficient literature search can be done through PubMed.

The methods on how the course will be conducted will also be discussed.

At the end of the reading session, you should be able to actively participate in the group
discussions that will subsequently be done in the succeeding sessions.

READING ASSIGNMENT
EVIDENCE BASED MEDICINE
INTRODUCTION

Medicine is a dynamic science. Everyday challenging problems arise, new modalities of
treatment are promoted, disease management done in minimal or far from ideal
conditions. Suppose a patient who consulted for cough productive of yellowish phlegm
and asked for a prescription of an antibiotic. Will you prescribe it or not? These are
common problems that may escape our attention and diminish the quality of care we
give if we make inappropriate decisions.

In the old practice faced with this question, a physician will just ask a colleague or an
expert for the answer or rely on his/her prior knowledge of the disease. He may also
prescribe a drug because of the promotional lecture sponsored by the manufacturer.

In evidence‐based medicine (EBM) a new paradigm is introduced. Before he makes a
decision, the physician will first try to retrieve his latest article about the topic that he
kept from his file, appraise the article then makes a decision. Later, he evaluates the
effectiveness of his decision. This loop ensures improvement in the quality of care.

DEFINITION OF EBM

Evidence‐based medicine is defined as the process of systematically finding, appraising,
and using contemporaneous research findings as the basis for clinical decisions (NLM,
2008). Evidence‐based medicine follows four steps:

• formulate a clear clinical question from a patient's problem
• search the literature for relevant clinical articles
• evaluate (critically appraise) the evidence for its validity and usefulness
• implement useful findings in clinical practice.

THE PARADIGM SHIFT

The traditional method of answering clinical problems was based on the following
assumptions (User’s Guide, 1992):
• clinical experience is the way of building and maintaining one's knowledge
• basic mechanisms of disease is a sufficient guide for clinical practice
• traditional medical training and common sense is sufficient to allow one to
evaluate new tests and treatment.
• clinical experts are a sufficient to generate valid guidelines for clinical practice

The traditional assumptions are now being questioned with the new paradigm. The
assumptions of the new paradigm are (User’s Guide, 1992):
• clinical experience are crucial but in the absence of systematic observation one
must be cautious in the interpretation of information derived from clinical
experience for it may at times be misleading.
• understanding of basic mechanisms of disease are necessary but insufficient
guides for clinical practice
• understanding certain rules of evidence is necessary to correctly interpret
literature on causation, prognosis, diagnostic tests, and treatment strategy

The new paradigm makes learning new things more self‐directed and less reliant on
teachers. Students can gain the skills to make independent assessments of evidence,
and thus evaluate the credibility of opinions being offered by experts. The purpose of
this course is to introduce the concept of evidence based medicine and the use of these
concepts to improve the quality of his/her own practice.

WHY TEACH EBM

Medical education faces a problem in a present setting: too much information, too little
time, too many students in crowded rooms, and exams that discouraged real life‐long
learning (Rangachari, 2007). There is a need to make students asks questions about
useful information and try to seek the answer for themselves.

The term "evidence based medicine" was coined at McMaster Medical School in Canada
in the 1980's to label an “active clinical learning” strategy, which people at the school
had been developing for over a decade (NLM, 2008). Randomized controlled trials have
shown that evidence‐based medicine learning is more effective than didactic learning
among medical interns in family medicine. They provide better care in terms of
providing treatment to patients with hypertension (Espallardo, 2006).

MEDICAL STUDENTS LEARNING EBM

Currently most physicians report a moderate amount of exposure to EBM. But
physicians in clinical research careers were more favorable towards EBM than those in
the clinical practice careers (Luebbe, 2007). But among those who were already exposed
to EBM there was mismatch perceived competence and their actual performance. This
suggests that better education in EBM is needed (Caspi, 2006). Starting the training in
the undergraduate program (medical students) is the logical approach. Early experience
helps medical students learn, helps them develop appropriate attitudes towards their
studies and future practice and orientates medical curriculums towards society's needs
(Littlewood, 2005).

OBJECTIVES OF THE COURSE

After going through the readings and workshops of this manual, the participants should
be able to:

• define and describe the steps in applying evidence‐based medicine into his/her
own clinical practice
• appraise and use randomized controlled trials and other types of studies in
solving clinical problems in clinical practice
• make an efficient literature search and identify problems and solutions in the
application of evidence based medicine

TEACHING METHODS

This is a series of self‐reading materials and group discussions. Allot a fix time for you to
read the reading assignment in the manual. Conduct the group discussion with at least 5
of your colleagues. Assign a facilitator and a co‐facilitator/scribe for each discussion.
Observe the rules enumerated in the succeeding section. Monitor your progress by
reviewing the checklist provided before each section.

The pace of learning depends on your time schedule. However, I suggest that you allot
one day a week for the reading time and group discussion. You can also try to apply
critical appraisal by yourself with other topics of interest.

RULES OF DISCUSSION

The ground rules that are encouraged to be observed during the group discussion are as
follows:

• Honor the established time limits.
• Allow one person to talk at a time.
• Focus on the topic. Avoid sideline conversation.
• Listen to what others have to say. All ideas are valued.
• Encourage participation in the discussion for all participants.
• Critique on ideas and thoughts, not on the person.

ROLES AND RESPONSIBILITIES

FACILITATOR

The facilitator serves as the process facilitator. He/she is also responsible for the
content and final outcome of the discussion. His/her responsibilities are to:

• Provide the process to achieve the objectives and desired outcomes.
• Pose probing but non‐threatening questions to provoke thought, clarify
discussion and bring insight on some points.
• Provide balance. Facilitate rather than lead the discussion.
• Remain neutral on content and avoid evaluation and decisions on ideas.
• Encourage equal participation among group members.

CO‐FACILITATOR/SCRIBE

The co‐facilitator helps the facilitator to achieve his/her objectives. He/she may join the
group discussion, but must bear in mind of his/her other functions:

• Be a timekeeper to ensure progress.
• Contribute ideas to the topic being discussed.
• Meet with the facilitator during the break to discuss the process and ideas on
how to proceed.
• Record essential information (content) and observation (group process) for post‐
discussion processing and evaluation.

PARTICIPANTS

The participants are encouraged to actively participate in the discussion. He/she
working agreement set by the group. The amount of learning you will get from this
course is proportional to the degree of your participation.

REFERENCES

Caspi O, McKnight P, Kruse L, Cunningham V, Figueredo AJ, Sechrest L. Evidence‐based medicine:
discrepancy between perceived competence and actual performance among graduating medical students.
Med Teach. 2006 Jun;28(4):318‐25.

Espallardo NL. Effectiveness of Critical Appraisal Workshop as a Method for Disseminating a Clinical
Practice Guideline on Hypertension. Fil Fam Phys 2006; 44 (2): 54‐60.

Littlewood S, Ypinazar V, Margolis SA, Scherpbier A, Spencer J, Dornan T. Early practical experience and
the social responsiveness of clinical education: systematic review. BMJ. 2005 Aug 13;331(7513):387‐91.

Luebbe AM, Radcliffe AM, Callands TA, Green D, Thorn BE. Evidence‐based practice in psychology:
perceptions of graduate students in scientist‐practitioner programs. J Clin Psychol. 2007 Jul;63(7):643‐55.

National Library of Medicine. Medical Subject Headings. www.ncbi.nlm.nih.gov/sites/entrez (May 27,
2008).

Rangachari PK. Back to the future? Active learning of medical physiology in the 1900s. Adv Physiol Educ.
2007 Dec;31(4):283‐7.

Users' Guides to Evidence‐based Medicine. JAMA. 1992;268(17):2420‐5.

READING ASSIGNMENT
SEARCHING THE MEDLINE
DATABASE
CLINICAL SCENARIO

Supposing a patient with cough came in to your clinic and asks for an anti‐biotic drug
because he wants to be relieved of his cough right away. Will you prescribe it?

Patients usually come in to the clinic for problems. Unfortunately these problems are
vague and sometimes not clearly stated. To state the problem clearly, you must bear in
mind that there are only three important elements that the patient want to know:

• What their diseases are
• What treatment they should be given
• What is the expected outcome of the treatment

These three important elements in clinical research are basically:

• the patient (P)
• the intervention/exposure (I)
• the outcome (O)

Sometimes the researcher can add

• method (M)

Going back to our scenario, my clearly stated problem will be: “Among patients with
cough (P) will anti‐biotic (I) provide symptom relief faster?”

TRADITIONAL METHOD OF LITERATURE SEARCH

A recent survey of important knowledge sources that influence clinical practice was
conducted among faculty members, fellows and residents of a large teaching tertiary
care hospital. The results showed that the most important resources were English
journals, text books and experience (Yousefi‐Nooraie, 2007). This dominance of the
traditional information resources and experience‐based medicine may be one of the
barriers to the dissemination of evidence‐based medicine.

Thus educational programs to develop skills of efficiently searching the research
literature need to be developed. Brief (two‐hour) instructional intervention on EBM‐
based techniques for searching Medline for evidence related to a clinical problem
provided to the students have been shown to be effective. With this training, students
had fewer search errors and correspondingly higher quality searches. The most common
search errors were a lack of Medical Subject Headings (MeSH) explosion, missing MeSH
terms, lack of appropriate limits, failure to search for best evidence, and inappropriate
combination of all search concepts (Gruppen, 2005).

PUBMED, ENTREZ AND MEDLINE

PubMed was developed by the National Center for Biotechnology Information (NCBI) at
the National Library of Medicine (NLM), located at the U.S. National Institutes of Health
(NIH). PubMed provides access to bibliographic information that includes MEDLINE, as
well as:

• out‐of‐scope citations (e.g., articles on plate tectonics or astrophysics) from
certain MEDLINE journals, primarily general science and chemistry journals
• citations that precede the date that a journal was selected for MEDLINE
indexing.
• additional life science journals that submit full text to PubMed Central and
receive a qualitative review by NLM.

Entrez is the text‐based search and retrieval system used at NCBI for services including
PubMed.

MEDLINE is the largest component of PubMed and is the freely accessible online
database of biomedical journal citations and abstracts created by the U.S. National
Library of Medicine (NLM). Approximately 5,200 journals published in the United States
and more than 80 other countries have been selected and are currently indexed for
MEDLINE. A distinctive feature of MEDLINE is that the records are indexed with NLM's
controlled vocabulary, the Medical Subject Headings (MeSH).

In the internet, you can access MEDLINE through PUBMED or GRATEFULMED or through
other organizations.

PARTS OF PUBMED HOMEPAGE

Side bar

Page header

Entrez databases
Query box
Features tab

BASIC SEARCH STRATEGY

The first step in using PubMed is to first develop a search strategy, a plan that helps you
look for the information you need. This can be done by doing the following steps:

• Identify the key concepts (should include the PIOM discussed earlier)
• Determine alternative terms for these concepts (can be facilitated with MESH
term search)
• Refine your search (use limits like publication dates, study subjects, study
designs, patient age, etc)

Our clinical question in the previous scenario was “Among patients with cough (P) will
anti‐biotic (I) provide symptom relief faster?” The key concepts in my search terms are:
• Cough
• Antibiotics
• Relief of symptoms

When I type the key term “cough” in the search box the yield was 27,751 articles and it
will be impossible for me to browse these articles. When I typed “cough AND
antibiotics" the yield was 2,175 and when I typed “cough AND antibiotics AND relief”
the yield was 15 articles. Now I can browse through these articles. How did this happen?

THE BOOLEAN PRINCIPLE
Searching MEDLINE
THE BOOLEAN LOGIC
A B
1 10 11
2 3 9
4 5 8 12
6 7 13
14
15 16
C
A OR B (A AND B) OR C
A AND B A AND (B OR C)
A AND B AND C A AND (B AND C)

In the Boolean principle, elements labeled such as 1, 2, 3, etc., can belong to set A, B, C,
etc. Some of these elements can belong to two or more sets, and some of these sets
may contain no elements. In the above example, if you want to combine elements in
two sets you use the word OR, i.e. the elements in set A OR set B are 1, 2, 3, 4, 5, 6, 7, 8,
9, 10, 11, 12, 13 and 14. If you want the elements common to the two sets you use the
word AND, i.e. the elements that belong to both set A AND set B are 7, 8 and 9.

In the MEDLINE, articles are indexed together as set of articles based on their key
words. Just like the Boolean principle the main operators in the MEDLINE are also the
words AND and OR. As a beginner this may be enough for you.

Searching MEDLINE
THE BOOLEAN LOGIC
TB RCTs
1 10 11
2 3 9
4 5 8 12
6 7 13
14
15 16
MENINGITIS
TB OR RCTs
TB AND RCTs
TB AND RCT AND MENINGITIS

In the example above, if I want articles about tuberculosis (TB), I will type “tuberculosis”
in the search window and the result will give me 9 articles. But if my concern is only to
read randomized controlled trials (RCT) on tuberculosis I will type in the search window
“tuberculosis AND randomized controlled trial” and it will give me 3 articles. Reading 3
articles instead of 9 will save me a lot of time.

THE ADVANCED SEARCH STRATEGY

The advance search page can be accessed by clicking the link Advanced Search (beta) on
the right side of the query box. I used the advance search option and got the results
shown below. When I type cough AND antibiotic, the yield was 2, 175 articles and when
I type cough AND antibiotic relief, the yield was 15 articles. If I have no time to browse
through 15 articles, I can limit this further by checking other boxes for relevance such as
date of publication, type of studies, age of subjects etc. This can also be done in the
basic search but it will take several steps.

REFERENCES

Gruppen LD, Rana GK, Arndt TS. A controlled comparison study of the efficacy of training medical students
in evidence‐based medicine literature searching skills. Acad Med. 2005 Oct;80(10):940‐4.

National Library of Medicine. PubMed OVerview. www.ncbi.nlm.nih.gov/sites/entrez (May 27, 2008).

Yousefi‐Nooraie R, Shakiba B, Mortaz‐Hedjri S, Soroush AR. Sources of knowledge in clinical practice in
postgraduate medical students and faculty members: a conceptual map. J Eval Clin Pract. 2007
Aug;13(4):564‐8.

Evidence-based Medicine: Learning Module on Differential Diagnosis
CRITICAL APPRAISAL OF AN
ARTICLE ABOUT DIFFERENTIAL
DIAGNOSIS

OBJECTIVES

The purpose of the reading assignment is to introduce to the participants the
concept of medical decision making using an article about differential diagnosis.

At the end of the reading session, you should be able to answer the user guides
questions for the workshop.

INSTRUCTIONS

Read the assignment for an article about a differential diagnosis. Focus on the
critical appraisal questions, why they are asked and how to get the answers from
the paper. After reading the paper you can proceed to conduct the group
workshop.

READING ASSIGNMENT
CLINICAL DECISION ON
DIFFERENTIAL DIAGNOSIS

INTRODUCTION

Differential diagnosis is the method of limiting the possible causes of the
patient’s symptoms before making a final diagnosis. Identifying the right
differentials will make patient management more focused and efficient.

Differential diagnosis can be arrived at by using the anatomic approach i.e.
considering the possibilities based on organs that may be affected within the
proximity of the symptom like chest pain may have differential diagnosis like
herpes zoster (skin), costochondritis (ribs), pneumonia (lings) or angina (heart).
If the symptom is systemic like fever, the differentials can by be pathophysiology
i.e. vascular, inflammatory/infectious, neoplastic/neurologic, degenerative,
intoxication/idiopathic, congenital, allergic/autoimmune, trauma and endocrine
(VINDICATE) (Friedland, 1998). With these approaches however the frequency or
probability of each differential will not be known.

Probabilities of Differential Diagnosis and Recommended Diagnostic Strategies

Differential Diagnosis Diagnostic Tests Treatment

Working diagnosis – Choose test(s) with high Start empiric
specificity and LR+ much
most possible cause larger than one.
treatment
that should be ruled in

Alternative diagnosis – Choose test(s) with high Start supportive
sensitivity and LR- much
other possibilities that smaller than one.
treatment
should be ruled‐out

Remote diagnosis None None

If we consider all known causes equally possible (the ‘possibilistic’ approach),
then the patient will have unnecessary diagnostic tests performed on them.
Instead, we must considering first those that are more common (a ‘probabilistic’
approach), or more more serious if left undiagnosed and untreated (a
‘prognostic’ approach) or more responsive to treatment (Richardson, 1999). And
they are important because the probabilities of the individual differential will
help us focus our diagnostic strategies as shown in the table below.

The disease probabilities can be taken from population prevalence statistics or
from original research. Research studies focus more directly on the frequency of
diseases that cause symptoms (Kroenke, 1997) are preferred over population
survey because they are more associated with presenting symptoms.

CLINICAL SCENARIO

Suppose a 35‐year old female patient came in to your clinic for fever and
abdominal pain for a week. There was neither diarrhea nor dysuria. You read in
the papers that there was an increased incidence of dengue fever in children.
How will you optimize (request only for what is essential) the diagnostic
laboratory tests for this patient?

Naturally you can do that if you already have an initial diagnosis in mind.
Unfortunately there might be many of them. This session will help you trim
down the differential diagnosis and request only for the laboratory tests that are
essential.

SEARCH

You found an unpublished retrospective study in the archives of your
department written by a previous resident Santos AR, entitled “Differential
diagnosis of typhoid fever in the emergency room”.

CRITICAL APPRAISAL

RELEVANCE QUESTION

• Is the objective of the article on differential diagnosis similar to your
clinical dilemma?

To answer this question, look at the objective of the study. It is important that
your article is relevant to the question you have raised in order for you to make
maximum use of the results of the study and be able to apply it to decision
making that influences patient care. For differential diagnosis it is important that
the focus is to find the cause of symptoms, clinical and laboratory presentation
among patients similar to your patient or case scenario.

VALIDITY GUIDES

• Did the study patients represent the full spectrum of patients who
present with this clinical problem?

Study designs that answer clinical questions like differential diagnosis can be a
cross‐sectional study or cohort study. An important element with these designs
is how the subjects are recruited so they can represent other patients who may
also have the same symptoms i.e. representativeness.

The definition of the clinical problem under study describes the population to
which the study will be applied. The problem usually is a symptom or an
abnormal physical examination such as headache or abdominal mass or a
combination of symptoms and abnormal physical findings like headache and
facial asymmetry. This is usually defined in the inclusion and exclusion criteria of
the study.

With the symptom already defined, the other strategies that can assure
representativeness are any of the following:
• Random selection – not always possible in clinical setting
• Consecutive patient recruitment – most feasible
• Recruitment in defined setting – must always be done

The Article by Santos included patients consulting for fever in the emergency
room. Although the inclusion criteria were fever alone as the chief complaint,
there was a subgroup analysis of patients with fever and abdominal complaint.
The total number of patients included in the study was 235. This coincides with
your case scenario.

• Were the criteria for each final diagnosis explicit and credible?

Determination of final diagnosis must be clearly described, may not necessarily
be based on the ultimate reference standard. However the criteria must be
explicit enough to make sure that different clinician will arrive at the similar
diagnosis (inter‐rater reproducibility).

The final diagnosis in Santos’s paper was based on clinical syndromes and
criteria. Blood cultures, ultrasound and other tests were not done to establish
the final diagnosis in only 48% of the cases.

• Was the diagnostic work‐up comprehensive and consistently applied?

The set of diagnostic work‐up should be thorough to come up with an accurate
diagnosis. Then a minimum set of diagnostic work‐up that includes a thorough
history and physical examination and a few initial laboratory tests should have
been applied consistently for all patients. This can be answered when the study
described a prospective approach in identifying patients in the study.
Retrospective approach is usually limited because, records cannot guarantee a
standard diagnostic approach for everybody.

The diagnostic tests done for 85% of patients in the Santos study were CBC,
urinalysis and stool examination. Temperature was measured using a mercury
type thermometer and records with “febrile” as reported documentation of
fever were excluded.

• For initially undiagnosed patients, was follow‐up to come up with a
diagnosis sufficiently long and complete?

Sometimes the diagnosis at the early stage of the disease is really difficult and
the patient may be classified as not having the disease or undetermined. To
assure ourselves with the eventual diagnosis of undetermined cases, we may
have to observe them over time.

The Santos study observed the patients for 24 to 48 hours in the emergency
room. In most patients antibiotics were started and the patients were sent home
without fever.

OVERALL, IS THE STUDY VALID?

Although the study was a retrospective study, you decided that you can use this
article because it is the only available study in your setting.

WHAT ARE THE RESULTS?

What were the diagnoses and their probabilities? How precise are the estimate
of the probabilities?

The probabilities of the differential diagnosis are reported as either incidence or
prevalence with their 95% confidence interval.

In the Santos study, the following top three diseases were the most common
diagnosis given to adult patients with fever: a) typhoid fever, 34%; b) urinary
tract infection, 32%; and c) acute gastroenteritis, 29%. No confidence intervals
were reported.

CAN THE RESULTS HELP ME IN CARING FOR MY PATIENTS?

• Are the study patients similar to my own?

For a study on differential diagnosis be applied to your patient you have to be
assured that the characteristics of your patient is similar to the study’s inclusion
criteria.

In the Santos study, they included patients consulting in the emergency room
but were eventually sent home. The cases seen in this study seemed to be the
milder cases similar to your patient.

• Do you think the disease probabilities in the study still apply today?

Disease prevalence and incidence change across time. Old disease can be
controlled because of effective treatment. Thus a paper on differential diagnosis
may still include smallpox for patients with fever and skin lesions in the 1950’s,
the probability is almost zero today. The probability of Dengue fever may differ
in different times of the year. A little knowledge on epidemiology of disease
across time may be necessary to have an accurate answer to this question.
However if the disease in question does not vary over time then this is not a
problem.

The study of Santos was a three‐year retrospective study from January 1988 to
December 1991. Seasonal variation may have been accounted for but the study
is already 9 years old. Unfortunately you cannot find a more recent one.

RESOLUTION OF THE PROBLEM IN THE SCENARIO

After appraising the study of Santos you decided that your diagnostic tests will
focus on ruling in or ruling out typhoid fever, urinary tract infection and
gastroenteritis.

REFERENCES

Friedland ed. Evidence‐based Medicine: A framework for clinical practice. Appleton and Lange,
1998.

Kroenke K. Symptoms and science: the frontiers of primary care research [Editorial]. J Gen Intern
Med 1997; 12: 509 ‐ 510.

Richardson WS, Wilson MC, Guyatt GH, Cook DJ, Nishikawa J, and the Evidence Based Medicine
Working Group. JAMA, 1999 Apr 7; 281(13):1214‐9.

WORKSHOP
CRITICAL APPRAISAL OF
AN ARTICLE ABOUT
DIFFERENTIAL DIAGNOSIS
(SESSION BRIEFING)

OBJECTIVES

The purpose of the workshop is to introduce to the participants the concept of
medical decision making about differential diagnosis. Another objective is to
introduce concepts of critical appraisal of an article regarding differential
diagnosis focusing on the following:

• validity
• interpretation of the results
• applicability of the results

CASE SCENARIO
Your neighbor is worried about her 14 year old daughter who complained of a
lump in the breast. The family had no history of breast cancer. She is afraid to
consult a doctor because her daughter is afraid of surgery. Knowing you are a
medical student, she was asking you about the possibilities.

You recalled reading a journal on the differential diagnosis of breast mass among
adolescents. What will you tell her?

Appraisal Sheet
CLINICAL DECISION ON DIFFERENTIAL DIAGNOSIS

CLINICAL SCENARIO OR
QUESTION

SEARCH

CRITICAL APPRAISAL

RELEVANCE Is the objective of the article on differential diagnosis
similar to your clinical dilemma?

PRIMARY VALIDITY Did the study patients represent the full spectrum of
GUIDES patients who present with this clinical problem?
Definition of the clinical problem or the patient whom the
study will be applied.

Were the criteria for each final diagnosis explicit and
credible?
Determination of final diagnosis must be clearly described,
may not necessarily be the ultimate reference standard.

SECONDARY VALIDITY Was the diagnostic work‐up comprehensive and
GUIDES consistently applied?


For initially undiagnosed patients, was follow‐up to come
up with a diagnosis sufficiently long and complete?

OVERALL, IS THE STUDY
VALID?

WHAT ARE THE What were the diagnoses and their probabilities? How
RESULTS? precise are the estimates of probabilities?

CAN THE RESULTS HELP Are the study patients similar to my own?
ME IN CARING FOR MY Inclusion criteria, exclusion criteria, clinical definition
PATIENTS?

Do you think the disease probabilities in the study still
apply today?
Is the study recent? Could the probabilities change since the
study publication?

RESOLUTION OF THE
PROBLEM IN THE
SCENARIO

Evidence-based Medicine: Learning Module on Diagnostic Tests
AN ARTICLE ABOUT A DIAGNOSTIC
TEST
(SESSION BRIEFING)

OBJECTIVES

The purpose of the reading assignment is to introduce to the students the concept of medical
decision making using an article about a diagnostic test. At the end of the reading session, you
should be able to answer the user guides questions for the workshop.

INSTRUCTIONS

Read the reading assignment for an article about a diagnostic test. Focus on the critical
appraisal questions, why they are asked and how to get the answers from the paper.

After reading the paper you can proceed to conduct the group workshop.

READING ASSIGNMENT
CLINICAL DECISION ON A
DIAGNOSTIC TEST

INTRODUCTION

Diagnostic test are often requested routinely like CBC, urinalysis and stool exam. This is
expensive and detrimental to patient care and therefore should not be encouraged. Doctors
should be requesting for the diagnostic test because it will give valuable information and may
change the way patient will be treated. This can be done by decision analysis using diagnostic
testing thresholds.

This will entail the following steps:
• Establish the upper and lower testing threshold
• Establish the probability that the patient has the disease
• Using the likelihood ratio determine if the test will change your management

Upper testing threshold – the probability of disease in a patient that you will stop testing and
start treatment

Lower testing threshold – the probability of disease in a patient that you will stop testing and
rule out the disease

Testing zone – is the probability of the disease that fall between the lower and upper testing
threshold

Pre‐test probability – the probability that a patient has the disease before doing a diagnostic
test

Post‐test probability – the probability that the patient has the disease after doing the diagnostic
test

Testing zone

Lower testing threshold Upper testing threshold

We will use this analysis in approaching our clinical scenario.

CLINICAL SCENARIO

A 70 years old female patient came in to the clinic complaining of forgetfulness. She’s afraid
that she has dementia just like her sister who was sent to a nursing care institution. She does
not want to be subjected to MRI or CT scan. You referred her to the psychiatric resident and she
suggested that you perform the Mini‐mental State Examination (MMSE), but you doubt her
decision.

The next weekend you went to the library and try to learn more about the MMSE.

In this scenario I consider the probability of 30% that the patient has dementia. I also consider
institutional care for these patients if the probability is 75% and maybe home intervention if the
probability is 25%. This means I have to make some diagnostic test for the patient.


Home care 25 % 30% 75% Institutional Care

Lower testing threshold upper testing threshold

SEARCH

After searching in the MEDLINE you found the article by Mulligan et al entitled “A comparison
of alternative methods of screening for dementia in clinical settings” published in the Archive of
Neurology, June 1996. Luckily the full text was also available.

CRITICAL APPRAISAL

RELEVANCE

• Was the objective of the paper relevant to your clinical question?

Most of the time we read journal articles because the topic is interesting. Because of this
application to clinical practice is not ensured. We can only ensure that the results of the article
are applied to practice if the objectives of the article are relevant to the clinical problems we
see in clinical practice. Thus the objective of the study must determine the accuracy (outcome)
of the contemplated diagnostic test (intervention/exposure) among patients (population)
similar to your case scenario.

VALIDITY GUIDES

• Was there an independent comparison with a reference standard?

There are two elements in this guide question i.e. use of a reference standard and independent
comparison. A reference standard for a diagnostic test is the test that gives the information
nearest to the “truth”. Thus the accuracy of the test should be compared against the standard.
If the diagnostic test approximated the standard, that means the test also approximates the
“truth”. An independent comparison means that the reader of the reference standard did not
know the result of the diagnostic test being evaluated (Jaeschke, 1994). Awareness of the initial
test result may lead to increase confirmation with reference standard leading to bias on the
accuracy of the diagnostic test being evaluated. Thus the first question you should answer is
whether there was a comparison with the reference standard and whether the reference
standard used was acceptable to your setting. The second is whether the reader of the
reference standard was blinded to result of the diagnostic test being evaluated.

In the study by Mulligan et al the reference standard used was the diagnosis of dementia based
on the DSM‐III‐R.

• Did the patient sample include an appropriate spectrum of patients to whom the test
will be used?

The accuracy of a diagnostic test among patients with low risk for the disease is different from
patients with high risk of the disease. The clinical utility of a test can be seen when used among
persons who are healthy, patients who are very sick and mostly those in‐between because
these are the patients who will be requiring the test. Patients consulting in family practice
usually belong to the healthy and in‐between groups while patients consulting in the hospitals
are those in the in‐between and more severe groups. The in‐between groups may give an
underestimate of the accuracy of the test (but it is the accuracy value to whom the test will be
used) while the healthier and more severe may give an overestimate of the accuracy. If all
groups are equally represented the average accuracy will be obtained.

The study of Mulligan et al included elderly patients consulting in a geriatric hospital and
memory clinic. The elderly age group is the population with the highest risk of dementia thus
the results from this study may be an overestimate.

• Was the reference standard done regardless of the result of the diagnostic test being
evaluated?

In some studies, the accuracy of a diagnostic test is examined retrospectively (chart review of
actual practice). In actual practice however, physicians request to perform the reference
standard based on the initial result of the diagnostic test. The reference standard is used to
verify the initial finding i.e. when positive. When this happens most of the data available will be
those positive for the diagnostic test and will likely be positive in the reference standard. This
will increase the accuracy of the test. This is called verification bias (Jaeschke, 1994). To avoid
this, the study must show that the reference standard was done regardless of the result of the
diagnostic test being evaluated.

It was mentioned in the Mulligan et al study that the comparison with the DSM‐III‐R was blind
and independent.

• Were the methods for performing the test described in sufficient detail to permit
replication?

This is necessary so that the reader will be able to duplicate the test in his/her own setting and
get the same valid result. Description should include preparation for the patient such as diet,
drugs to avoid, precautions, ideal conditions for performing the diagnostic test and a step by
step description of how the diagnostic test is done and interpreted.

There are a lot of papers dealing with instructions on how to administer the MMSE and its
interpretation.


Yes. You accepted the validity since most of the questions were answered adequately.


• What were the likelihood ratios for the different possible test results?

Likelihood ratio indicates by how much a given test result increases the pre‐test probability of
the disease. A likelihood ratio of 1 means the post‐test probability is similar to the pre‐test
probability. A likelihood ratio of greater than 1 increase the chance that the disease is present,
and the greater the likelihood ratio the greater is the increase in chance.

Some papers give the sensitivity and specificity values rather than the likelihood ratio. The
formula for computing the likelihood ratio from sensitivity and specificity is shown below:

Likelihood ratio of a positive test LR (+) = Sn/1‐Sp
Likelihood ratio of a negative test LR (‐) = 1‐Sn/Sp

A rough guide in evaluating LR values:
• LRs >10 or < 0.1 generate large, and often conclusive changes from pre‐ to post‐test
probability;
• LRs of 5‐10 and 0.1‐0.2 generate moderate shifts in pre‐ to post‐test probability;
• LRs of 2‐5 and 0.5‐0.2 generate small (but sometimes important) changes in probability;
and
• LRs of 1‐2 and 0.5‐1 alter probability to a small (and rarely important) degree.

In the Mulligan study the LR (+) MMSE is 2.46 and the LR (‐) is 0.14. The alternative clinical test
is the Antisaccadic Eye Movement Test (AEMT) but did not do very well compared with the
MMSE.

Let us go back to our case scenario. To use the likelihood ratio in my decision making, I have to
do the following steps:

• Convert pre‐test probability to pre‐test odds (p/1‐p)
o Pre‐test odds = 0.30/1‐0.30 = 0.43

• Multiply pre‐test odds by the likelihood ratio to get the post‐test odds
o If the MMSE is negative = 0.43 x 0.14 = 0.06
o If the MMSE is positive = 0.43 x 2.46 = 1.06

• Convert the post‐test odds to post‐test probability (o/o+1)
o If the MMSE is negative = 0.06/0.06+1 = 0.05 or 5%
o If the MMSE is positive = 1.06/2.06 = 0.51 or 51%

Thus if the test is negative the probability that the patient has dementia is 5% and I will can
recommend home care. But if the MMSE is positive the probability becomes 51% and did not
reach my threshold for institutional care. The MMSE will only change my management if the
result is negative but if positive, I will have to do another test.

Home care 25 % 30% 75% Institutional Care

Lower testing threshold upper testing threshold


• Will the reproducibility of the test result and its interpretation be satisfactory in my
setting?

Even if the test was described very well, the reproducibility of the interpretation is necessary
for the test to be adequately applied in your setting. The paper should report measures of
agreement between interpreters or raters.

If agreement is not reported, decide for yourself. Is the interpretation simple enough? Is the
basis of the interpretation clear and specific?

The MMSE questionnaire also contains instruction on how to administer the test and interpret
the result.

• Are the results applicable to my patient?

If your setting is somewhat similar to that in the study, and the inclusion criteria include
characteristics of your patient, then you can apply the results to your patient. Sometimes your
clinical judgment is required.

• Will the results change my management?

The usefulness of a diagnostic test result is whether it will help the clinician manage his/her
patient. If the diagnostic test will lead the doctor decide to give treatment or not, then the test
is helpful and may be requested. However if after applying the LR value to determine the post‐
test probability and this did not help in the decision, then you don’t have to perform the test.


After looking at the Mulligan et al study, you agreed with the psychiatry resident and even
asked her help to administer the MMSE to the patient yourself.

REFERENCES
Jaeschke R, Guyatt G, Sackett D, and the Evidence Based Medicine Working Group. How to Use an Article About a
Diagnostic Test: Validity Guides. JAMA, 1994; 271(5):389-391.
Jaeschke R, Guyatt G, Sackett D, and the Evidence Based Medicine Working Group. How to Use an Article About a
Diagnostic Test: Results and Applicability. JAMA, 1994; 271(9):703-707.

WORKSHOP
CRITICAL APPRAISAL OF AN ARTICLE
ABOUT A DIAGNOSTIC TEST
(SESSION BRIEFING)

OBJECTIVES

The purpose of the workshop is to introduce to the participants the concept of medical decision
making about the usefulness of a diagnostic test. Another objective is to introduce concepts of
critical appraisal of an article regarding a diagnostic test focusing on the following:

• Validity
• Reference standard
• Sensitivity and specificity
• Likelihood ratios
• Applicability of the results

CASE SCENARIO

You have a 25 year old friend who is a tennis player. He sustained a fall during a practice game
two weeks ago but still suffers pain in the knee joint especially when walking. His doctor
requested for MRI. But because of the cost of MRI he wants a second opinion and asks you
about its accuracy.

How will you reply?

Appraisal Sheet
CLINICAL DECISION ON A DIAGNOSTIC TEST

QUESTION

SEARCH

CRITICAL APPRAISAL

RELEVANCE Is the objective of the study relevant to your clinical
question?

VALIDITY GUIDES Was there an independent and blind comparison with a
reference standard?
What was the reference standard. Were they assessed
independently?

Did the patient sample include an appropriate spectrum of
patients to whom the test will be used?

Was the reference standard done regardless of the result
of the diagnostic test being evaluated?

Were the methods for performing the test described in
sufficient detail to permit replication?

VALID?


WHAT ARE THE What were the likelihood ratios for the different possible
RESULTS? test results?

CAN THE RESULTS HELP Will the reproducibility of the test result and its
ME IN CARING FOR MY interpretation be satisfactory in my setting?
PATIENTS?

Are the results applicable to my patient?

Will the results change my management?

RESOLUTION OF THE
PROBLEM IN THE
SCENARIO

Evidence-based Medicine: Learning Module on Therapy or Prevention
ABOUT THERAPY OR PREVENTION
(SESSION BRIEFING)

OBJECTIVES

medical decision making using an article about therapy or prevention.

At the end of the reading session, you should be able to answer the user guides questions for
the workshop.

INSTRUCTIONS

Read the reading assignment for an article about therapy or prevention. Focus on the critical
appraisal questions, why they are asked and how to get the answers from the paper.


READING ASSIGNMENT
CLINICAL DECISION ON THERAPY OR
PREVENTION

RANDOMIZED CONTROLLED TRIALS

Randomized controlled trials are the standards design to prove effectiveness of drugs or other
forms of intervention. When done properly, it can provide the best evidence of effectiveness. In
this type of design, individuals are randomly assigned (randomization) to either of the two or
more groups, one with the intervention the other without the intervention being tested or
another intervention. Randomization tries to make the two groups similar for both known and
unknown factors that may affect the outcome other than the intervention being tested. Then
they are observed forward in time and their outcome compared. The outcome can be the cure
of a disease, relief of symptoms or improvement in quality of life (Espallardo, 2000).

A brief comparison of the advantages and disadvantages of different types of clinical studies
that might help us answer our questions is presented below:
• Case‐Control Study
• Cross‐Sectional Survey
• Cohort Study
• Randomized Controlled Trial
• Crossover Design

Case‐Control Studies
• Advantages:
o quick and cheap;
o only feasible method for very rare disorders or those with long lag between
exposure and outcome;
o fewer subjects needed than cross‐sectional studies.
• Disadvantages:
o reliance on recall or records to determine exposure status;
o confounders;
o selection of control groups is difficult;
o potential bias: recall, selection.

Cross‐Sectional Survey
• Advantages:
o cheap and simple;
o ethically safe;.
• Disadvantages:
o establishes association at most, not causality;
o recall bias susceptibility;
o confounders may be unequally distributed;
o Neyman bias;
o group sizes may be unequal.

Cohort Study
• Advantages:
o ethically safe;
o subjects can be matched;
o can establish timing and directionality of events;
o eligibility criteria and outcome assessments can be standardized;
o administratively easier and cheaper than RCT.
• Disadvantages:
o The controls may be difficult to identify;
o exposure may be linked to a hidden confounder;
o blinding is difficult;
o randomization not present;
o for rare disease, large sample sizes or long follow‐up necessary.

Randomized Controlled Trial
• Advantages:
o unbiased distribution of confounders;
o blinding more likely;
o randomization facilitates statistical analysis.
• Disadvantages:
o expensive: time and money;
o volunteer bias;
o ethically problematic at times.

Crossover Design
• Advantages:
o all subjects serve as own controls and error variance is reduced thus reducing
sample size needed;
o all subjects receive treatment (at least some of the time);
o statistical tests assuming randomization can be used;
o blinding can be maintained.
• Disadvantages:
o all subjects receive placebo or alternative treatment at some point;
o washout period lengthy or unknown;
o cannot be used for treatments with permanent effects.

CLINICAL SCENARIO

You are working at the out‐patient clinic in your institution when a 55 year‐old, male bank
executive came to your clinic for constricting chest pain located at mid‐sternum precipitated by
exertion and relieved by rest. His blood pressure was 130/80, heart rate was 85/minute and
the respiratory rate was 20/minute. An ECG was done revealing lateral wall ischemia. You sent
the patient home on oral nitrates and aspirin. However, further work‐ups revealed
hypercholesterolemia with a level of 6.5 mol/liter. You gave dietary advice but the patient
claimed that he has been on a high fiber diet with low fat intake for the past 6 months.

He is now inquiring if he should take a drug for his elevated cholesterol.

SEARCH

You decided to translate this clinical dilemma to an answerable question. You also decided that
the article should include a population of patients that has an elevated cholesterol who have
already undergone dietary therapy for at least 6 months to be consistent with your patient’s
profile. The drug intervention should be compared with placebo. The article must report
clinically important outcomes, such as reduction in cardiovascular deaths. Finally, you wanted
an article that employed randomization.

The article you finally retrieved included 4444 patients randomized to either simvastatin or
placebo. However, before applying the results to your patient, you decided to appraise the
article using the following guides.

CRITICAL APPRAISAL

RELEVANCE

• Is the objective of the article comparing therapeutic interventions similar to your
clinical dilemma?

Before going any further, first ascertain if the objective of the study addresses the clinical
problem you face. Appraising an irrelevant article would not be helpful to your clinical dilemma
and will be a waste of time. Below are a few tips that will help you decide on relevance:

• Population of the study (P) – should be similar to the characteristic of your patient.
• Intervention/comparative intervention/exposure (I) – should include the therapeutic
intervention you want to test.
• Outcome of the study (O) – one of the outcomes measured should be the goal you and
your patient wish to work for.

VALIDITY GUIDES

The issue of validity speaks to the "truthfulness" of the information. The validity criteria should
be applied before an extensive analysis of the study data. If the study is not valid, the data may
not be useful. The evidence that supports the validity or truthfulness of the information is
found primarily in the study methodology. Here is where the investigators address the issue of
bias, both conscious and unconscious. Study methodologies such as randomization, blinding,
and accounting for all patients help insure that the study results are not overly influenced by
the investigators or the patients.

• Was the assignment of patients to treatment randomized?

In order to answer this question, the reader is advised to look into the article’s abstract or
methodology section. For the 4S study, randomization was done and was written both in the
abstract and methods section.

The strength of randomization is that if the sample size is sufficiently large, it assures that both
known and unknown determinants of outcome are evenly distributed between the treatment
and control groups. In the absence of randomization, these factors might be difficult to control
and might be the one strongly influencing outcome rather than the treatment itself (Guyatt,
1994). Research has shown that random allocation comes closest to insuring the creation of
groups of patients who will be similar in their risk of the events you hope to prevent.
Randomization balances the groups for prognostic factors (i.e. disease severity) which eliminate
over‐representation of any one characteristic within the study groups. Randomization should
also be concealed from the clinicians and researchers of the study to help eliminate conscious
or unconscious bias. This is done so that the clinicians won't be aware of which treatment the
next patient would receive.

At times though because of the rarity of the disease and small patient sample size,
randomization might not be feasible. In these cases, a clinician must rely on weaker studies but
should be aware of its potentials for errors.

• Were all patients who entered the trial properly accounted for and attributed at its
conclusion?

o Was follow‐up complete?

Every patient who entered the trial should be accounted for at its conclusion. If substantial
numbers are “lost to follow‐up”, the validity of the conclusions are open to question. A drop‐
out rate of 20% or more is usually declared as substantial. If the number lost to follow‐up is
less than this the reader can decide if this affects the conclusion by assuming a “worst case
scenario”. This means that the numbers lost in the treatment group are assumed to have bad
outcomes and the numbers lost in the control group are assumed to have been cured and if the
conclusions differ, a substantial number was lost to follow‐up.

This is best checked by looking at the number of patients enrolled at the outset and comparing
this with the number of patients reported in the results table.

Another way of deciding whether follow‐up was complete is to check whether an intention to
treat analysis was done. If this is reported one can safely assume that follow‐up was complete.

o Were patients analyzed in the groups to which they were randomized?

It simply means that all those belonging to the control group or treatment group are analyzed
from beginning to end in this same grouping including those who were dropped or withdrawn
or changed treatment. No crossing over treatment modalities were done as this would likely
lead to biased results.

Excluding non‐compliant patients from the analysis leaves behind those who may be destined
to have a better outcome and destroys the unbiased comparison provided by randomization.
This principle of attributing all patients to the group to which they were randomized results in
an "intention‐to‐treat" analysis. This strategy preserves the value of randomization: prognostic
factors that we know about, and those we don't know about, will be, on average, equally
distributed in the two groups, and the effect we see will be just that due to the treatment
assigned (Guyatt, 1994).

• Were patients, their clinicians, and study personnel "blind" to treatment?

Blinding is the process by which the intervention being given is concealed from the patient, the
clinicians and the one who analyzes the data. Patients, clinicians and data analysts are likely to
have an opinion regarding the experimental treatment. These opinions, whether optimistic or
pessimistic, can systematically distort reporting of treatment outcomes. As to avoid these
“reporter and observer” bias, blinding is necessary.

Blinding is not always possible (such as in surgery trials) and in these situations, we should
check to see if outcome events were assessed by investigators or adjudication committees who
are not directly involved in the trial.

To answer this question the reader is again advised to look into the abstract or the
methodology section.

• Were the groups similar at the start of the trial?

To answer this question, one should look for a report of the comparison of the baseline
characteristics of the experimental and control group. For most studies, this is labeled as table
1. In the 4S trial, baseline characteristics were similar.

For reassurance about the study’s validity, readers would like to be informed that the
treatment and control groups were similar for all the factors that determine clinical outcomes
of interest save for the experimental therapy. The greater the similarity between known
prognostic factors for the control and experimental group, the more likely that the results can
be attributed to the intervention, rather than due to the differences in these factors.

• Aside from the experimental intervention, were the groups treated equally?

Interventions other than the treatment under study, when differentially applied to the
treatment and control groups, are called “co‐interventions”. This might distort the results since
they in themselves might cause changes in reported outcomes.


If the study fails any of the above criteria, we need to decide if the flaw is significant and
threatens the validity of the study. If this is the case, we'll need to look for another study.

The 4S study yielded yes to all appraisal questions hence you decided that over‐all the study
was valid. You now proceed to analyze the results.


Once you have determined that the study methodology is valid, you must examine the results
and their applicability to the patient. Clinicians may have additional concerns such as whether
the study represented patients similar to his/her patients, whether the study covered the
aspect of the problem that is most important to the patient, or whether the study suggested a
clear and useful plan of action.

• How large was the treatment effect?

Most randomized controlled trials report outcomes either as treatment success or treatment
failures and adverse effects. Examples of outcomes include cure rates, side effects or death.
Patients either do or do not suffer these events and the article frequently reports the
proportion of patients who develop such events. In the 4S study, 11.5% died in the placebo
group and 8.2% died in the simvastatin group. By eyeballing these figures, simvastatin seems
better in reducing deaths. But how else could these figures be compared? The following
simple computations could help:

Risk in Control (Rc) = Death in control/N patients in the control

Risk in Treatment (Rt) = Death in treatment/N patients in treatment

Absolute Risk Reduction (ARR) = Rc – Rt = 0.115 ‐ 0.082 = 0.033

Relative Risk (RR) = Rt/Rc = 0.082/0.115 = 0.71

Relative Risk Reduction (RRR) = 1 – RR = 1‐ 0.71 = 0.29 (29%)

Absolute risk reduction is the absolute difference between the proportion who died in the
placebo group compared to the simvastatin group. Relative risk is the risk of events in the
simvastatin group or new treatment relative to the placebo or control group. The most useful
measure to use in explaining the benefit of treatment to patients is the relative risk reduction.
In this case, simvastatin treatment reduces deaths 29% more than placebo.

• How precise was the estimate of treatment effect?

To decide regarding precision, one should look at the reported 95% confidence interval. The
closer these values, the more precise your estimates. If this is not reported check the p‐value,
anywhere from </= .10 is acceptable.

Ideally, the reported minimum and maximum values of this interval should all be positive or all
be negative for the absolute risk reduction, all above below one for the relative risk and relative
risk reduction to be considered precise.


• Can the results be applied to my patient care?

If your patient meets all of the inclusion criteria and none of the exclusion criteria, the
applicability of the study’s results to your patient is without question. It is rare however that
we get a patient who conforms to all the characteristics of the study subjects. In these cases,
we should decide if the reason is compelling enough not to apply the results of our study to our
particular patient.

• Were all clinically important outcomes considered?

Clinically important outcomes may range form decreasing mortality, morbidity, improving
quality of life. These are outcomes that are important to the patients and will lead directly to
reducing symptoms or decreasing death. Some studies might report improvement in
cholesterol levels, improvement in PFTs but these are what might be labeled as surrogate
endpoints. That is, the researchers have substituted these physiologic measures for important
outcomes we have just mentioned. For reduction in these laboratory parameters does not
always translate into decrease in morbidity and mortality.

A dramatic example of the danger of substitute endpoints was found in the evaluation of the
usefulness of clofibrate as anti‐cholesterol drug. It shown to decrease serum cholesterol but
was shown to increase all‐cause mortality. Similar findings were noted in an anti‐arrhythmia
trial hen the investigators had to stop the trials when they discovered that mortality was
substantially higher in patients receiving antiarrhythmic treatment than in those receiving
placebo.

• Are the likely treatment benefits worth the potential harm and costs?

Computation for cost effectiveness and checking for side effects might be done to check if the
treatment benefits are worth the potential harm and costs.


The article was valid and giving simvastatin reduces all deaths by 29% compared to placebo.
However when we look into costs, giving this drug treatment is fairly expensive. However due
to the marked cardiac risks for this patient you decide to give simvastatin since your patient
could also afford drug treatment.

At this point, we hope that you are encouraged to adapt this new paradigm in your medical
decision making. The steps are relatively simple. First, define the problem clearly. Second, use
one of several search strategies to come up with relevant articles. Third, appraise the article.
Assess the results and the applicability of your article to your patient. Guided by this, you then
decide on the action to take.

This may sound like a tedious process but as they say practice makes perfect. And if each of us
will continue asking, searching, we will continue learning and hopefully improving our patient
care.

REFERENCES

Espallardo, NL. Research Protocol Development for Resident Physicians. Family Medicine Research Group, Inc.
Manila, 2000.

Guyatt GH, Sackett D, Cook DJ, for the Evidence Based Medicine Working Group. How to Use an Article About
Therapy or Prevention: Validity. JAMA, 1993;270(21):2598‐2601.

Guyatt GH, Sackett D, Cook DJ, for the Evidence Based Medicine Working Group. How to Use an Article About
Therapy or Prevention: Results and Apllicability. JAMA, 1994;271(1):59‐63.
WORKSHOP
AN ARTICLE ABOUT THERAPY OR
PREVENTION
(SESSION BRIEFING)

OBJECTIVES

making about treatment. Another objective is to introduce concepts of critical appraisal of an
article regarding treatment focusing on the following:

• validity
• randomization
• intention‐to‐treat
• interpretation of the results
• applicability of the results
• clinically relevant endpoints
• cost‐effectiveness

CASE SCENARIO

Your 65 year old grandfather who suffered a heart attack 5 years ago is currently taking 5
medicines daily. Two were anti‐hypertensives, an anti‐cholesterol drug, aspirin and vitamin E.
He wants to stop vitamin E since he has difficulty swallowing it because of its large size. But
according to his doctor he should take it because it will prevent recurrence of the heart attack.

What will you advice him to do?

Appraisal Sheet
CLINICAL DECISION ON THERAPY OR PREVENTION

QUESTION

SEARCH

CRITICAL APPRAISAL

RELEVANCE Is the objective of the article comparing therapeutic
interventions similar to your clinical dilemma?

VALIDITY GUIDES Was the assignment of patients to treatment randomized?
Randomization vs. random selection

Were all patients who entered the trial properly accounted
for and attributed at its conclusion?

Was follow‐up complete?
Dropouts, withdrawals

Were patients analyzed in the groups to which they were
randomized?
Intention‐to‐treat analysis

Were patients, their clinicians, and study personnel "blind"
to treatment?

VALID?


WHAT ARE THE How large was the treatment effect?
RESULTS? Risk in control, risk in treatment, relative risk, relative risk
reduction, absolute risk reduction

How precise was the estimate of treatment effect?
95% confidence interval, p value

CAN THE RESULTS HELP Can the results be applied to my patient care?
ME IN CARING FOR MY Inclusion criteria, exclusion criteria
PATIENTS?

Were all clinically important outcomes considered?
Outcome, results

Are the likely treatment benefits worth the potential harm
and costs?
Side effects, NNT, costs

RESOLUTION OF THE
PROBLEM IN THE
SCENARIO

Evidence-based Medicine: Learning Module on Harm
ABOUT HARM
(SESSION BRIEFING)

OBJECTIVES

medical decision making using an article about harm. At the end of the reading session, you
should be able to answer the user guides questions for the workshop.

The students should also be able to identify the advantages and disadvantages of the different
study designs to determine the harmful effect of intervention or other exposure.

INSTRUCTIONS

Read the reading assignment for an article about harm. Focus on the critical appraisal
questions, why they are asked and how to get the answers from the paper.


READING ASSIGNMENT
CLINICAL DECISION ON HARM

INTRODUCTION

Harmful or adverse effects are sometimes associated with a particular treatment. It can be
studied in the context of RCT’s simply by measuring side effects or adverse events in addition to
benefits of the intervention. However, trials are not adequately powered to look for side effects
i.e. sample size computation is based on the beneficial effect.

However, when outcomes are rare, or studying the exposure or intervention is unethical, other
study designs are frequently required. As a result, we usually find evidence about harm in
cohort studies (groups of patients who are and aren't exposed to the treatment are followed up
for the outcome of interest) and case‐control studies (patients with the outcome of interest are
matched with patients without the outcome and investigators look retrospectively to
determine exposure). Case‐control studies are useful when the outcome of interest is rare or
when the required follow‐up is long.

COHORT AND CASE-CONTROL STUDIES

Harmful effects of drugs or other exposure can be studied with cohort or case‐control study
designs. A cohort is any group of individuals who share the same characteristics. In a cohort
study, selection of subjects starts with identifying individuals who have the same characteristics
or presence or absence of a particular cause or exposure. They are then divided into two
groups, those with the characteristics or causes and those without the characteristics. They are
then observed forward in time and determine who among them develop the outcome or effect
(Espallardo, 2000).

Cohort studies can also be prospective or retrospective depending on the manner of patient
recruitment. If recruitment is being done forward in time it is prospective, but if the cohort
already existed in the past and data gathering is being done by reviewing existing clinical
records then it is retrospective (Espallardo, 2000).

In a case‐control study, inclusion of subjects starts with defining or selecting those who have
the outcome or effect. These are considered as cases. Then this group is compared with
subjects who don’t have the outcome or effect. These are considered as the controls. Both the
cases and the control should be taken from within the same population. Then the two groups
are investigated as to the presence or absence of hypothesized causes or risk factors for the
outcome (Espallardo, 2000).

Case‐control study can be prospective or retrospective depending on the manner of patient
recruitment. If recruitment is being done as cases develop forward in time it is prospective, but
if the cases have already developed in the past and patient recruitment is being done by
reviewing existing clinical records then it is retrospective (Espallardo, 2000).

CLINICAL SCENARIO

You are the resident physician covering for the consultant in his clinic. You met one of his
patient, 45 year old male who is on anti‐cholesterol medication. You noted that his cholesterol
is now only 4.35 mmol/L but the patient was still prescribed with anti‐cholesterol drug from his
last week visit. You approach the consultant and mentioned it to him. He told you its okay; it’s
the high cholesterol that we should be concerned with anyway.

Still doubtful, you went to the library and look for the harmful effect of very low cholesterol.

SEARCH

You found the article by Zureik et al, entitled “Serum cholesterol concentration and death from
suicide in men: Paris prospective study 1” published in the British Medical Journal, September
1996.

CRITICAL APPRAISAL

RELEVANCE

• Is the objective of the article on harm similar to your clinical dilemma?

Your formulated clinical question must be addressed by the objective of the study. For a
decision to stop the harmful exposure the article must be designed to determine the harmful
effect (outcome) of the a drug, chemical or environmental substances (intervention/exposure)
to the patient (population). This must be clearly stated by the objective of the study.

VALIDITY GUIDES

• Were there clearly identified comparison groups?

Just like studies of effectiveness of treatment, studies of harm require that a control group for
comparison should be done. Unfortunately, randomized controlled trials to prove harm is not
ethical, so studies of weaker design like cohort or case‐control studies may be relied upon. Both
designs has a control group for comparison, in the former controls are chosen by absence of
exposure and in the latter by absence of the outcome or disease.

In a cohort study, a group of patients are observed. They are divided into those with the
exposure and those without the exposure. Then the outcome is observed forward in time. In a
case‐control study, patients with the outcome are gathered. Then a group of patient without
the outcome matched to the preceding group for certain characteristics other than the
exposure is also gathered. The presence of the exposure in both groups is then ascertained
(Levine, 1994).

In a cross‐sectional study, patients are grouped and analyzed with respect to their outcome and
exposure. This study design can also be a basis for establishing harmful effect, but the temporal
relationship of the exposure occurring before the outcome cannot be established.

The study by Zureik observed 6,728 men who had measurements of serum cholesterol. They
grouped and categorized cholesterol levels into low (<4.78 mmol/L), normal (4.78 to 6.21
mmol/L) and high (>6.21 mmol/L). The changes were also categorized. These were the
comparison groups.

• Were the exposures and outcomes measured in the same way in the groups
compared?

Measurement of outcomes must be similar in both groups. In cohort study, the investigators
must show that diligent observation for the outcome was done in the groups with the exposure
(high risk) as well as the groups without the exposure (low risk). When the patient with the
exposure were observed more diligently, there will be a higher detection rate of the outcome
leading to increase incidence of the disease in the exposed group. This is called surveillance bias
(Levine, 1994). This must be avoided. This bias may also occur in case‐control studies, when the
detection of exposure is more diligent in the group with the disease or outcome.

Suicide data were taken from the national databases and death certificates in all groups in the
Zureik study.

• Was follow‐up sufficiently long and complete?

The length of follow‐up must be sufficiently long enough to detect the outcome. If follow‐up is
short, the chance of underestimating the effect of the exposure is high. When the relation
between asbestos and lung cancer was being investigated the relative risk was only 1.4 in the
early years of observation compared to the subsequent relative risk of 18.2 when the years of
observation was extended to 15 years and beyond.

The Zureik study observed their patients for 4 years after enrolment. They had 95% follow‐up.

• Is the temporal relationship between the exposure and outcome correct and dose
response gradient present?

For an exposure to cause an effect, two important criteria may be considered. First the
exposure must be present before the outcome and second there must be a dose response
gradient, i.e. the higher the dose the higher is the probability of the outcome. Cross‐sectional
studies usually cannot establish temporal relationship but it can establish a dose response
gradient and a comparison between groups.

The Zureik study measured serum cholesterol before the event of suicide so the temporal
relationship was correct.


Since all the answers to the validity guides, the study can be considered valid.


• What is the magnitude of the association between exposure and outcome? Was the
estimate of the risk precise?

The relative risk is the incidence of the adverse effect in the group with the exposure divided by
the incidence of adverse effect in the group without the exposure. If the relative risk is more
than 1, then the exposure is causing harm and if less than 1 the exposure reduces harm.
Relative risk is usually computed when the design is a cohort study.

In a case control study, the odds ratio is computed. The odds ratio approximates the relative
risk, especially when the disease is rare.

The values of 95% confidence interval should be greater than 1 to say that the exposure really
causes harm. If one value of the 95% confidence interval is less than 1 and the other is more
than 1, then the effect of the exposure is uncertain.

The Zureik study showed that those with low cholesterol had increased risk of suicide with a
relative risk of 3.16 with a 95% CI of 1.38 to 7.22. The analysis was also adjusted for age,
smoking and mean corpuscular volume.



Just like in an article about a beneficial intervention, for the harmful effect to be extrapolated
to your patient you have to be assured that the characteristics of your patient is similar to the
study’s inclusion criteria.

The Zureik study recruited men between 43 to 52 years old with similar demographic
characteristics with our patient.

• Should I attempt to stop the exposure?

In answering this question you should consider the following:
• How large and precise is the risk of harm?
• What are the consequences if I withdraw the exposure?
• Do I have any alternative for the exposure?

Decision is simple when the answers to these questions are clear. For example cigarette
smoking has been associated with increase incidence of lung cancer and cardiac deaths, but
withdrawing smoking may lead to “decrease quality of life” for smokers. But recently an
alternative like nicotine patch has been shown to decrease withdrawal discomfort and
eventually improve smoking cessation. So the decision to withdraw smoking for every patient
consulting in the clinic is warranted.


Based on the appraisal you decided to go back to your consultant and inform him about the
study that you found. He thanked you for the information and promised to withdraw the anti‐
cholesterol drug when the patient comes back.

REFERENCES

Espallardo, NL. Research Protocol Development for Resident Physicians. Family Medicine Research Group, Inc.
Manila, 2000.

Levine M, Walter S, Lee H, Haines T, Holbrook A, Moyer V, for the Evidence Based Medicine Working Group. How
to Use an Article about Harm. JAMA, 1994;271(20):1615‐1619.

WORKSHOP
ABOUT HARM
(SESSION BRIEFING)

OBJECTIVES

making about harmful effect of a substance or drug. Another objective is to introduce concepts
of critical appraisal of an article regarding a harmful effect of a substance or drug focusing on
the following:

• Validity
• Cohort study, case‐control study, case series and case report
• Interpretation of the results

CASE SCENARIO

You attended a lecture in pharmacology and discussed issues about drug regulation. One
example raised was the case of rofecoxid, a drug that stayed in the market for some time
despite evidence showing harmful effect. The drug was eventually withdrawn from the market.

Your professor raised a similar issue with celecoxid and asks the class to be ready for a debate
on whether celecoxid should also be withdrawn. Which side will you be – for or against?

Appraisal Sheet
CLINICAL DECISION ON HARM

QUESTION

SEARCH

CRITICAL APPRAISAL

RELEVANCE Is the objective of the article on harm similar to your
clinical dilemma?

PRIMARY VALIDITY Were there clearly identified comparison groups?
GUIDES

Were the exposures and outcomes measured in the same
way in the groups compared?

Was follow‐up sufficiently long and complete?
Is the temporal relationship between the exposure and
outcome correct and dose response gradient present?

VALID?

WHAT ARE THE What is the magnitude of the association between
RESULTS? exposure and outcome? Was the estimate of the risk
precise?


ME IN CARING FOR MY
PATIENTS?

Should I attempt to stop the exposure?

RESOLUTION OF THE
PROBLEM IN THE
SCENARIO

Evidence-based Medicine: Learning Module on Prognosis
ABOUT PROGNOSIS
(SESSION BRIEFING)

OBJECTIVES

medical decision making using an article about prognosis.

At the end of the reading session, you should be able to answer the user guides questions for
the workshop.

INSTRUCTIONS

Read the reading assignment for an article about prognosis. Focus on the critical appraisal
questions, why they are asked and how to get the answers from the paper.


READING ASSIGNMENT
CLINICAL DECISION ON PROGNOSIS

WHAT IS PROGNOSIS

Prognosis refers to the development of possible “outcome” of disease i.e. death in patient with
cancer. Prognostic factors are characteristics of a particular patient can be used to predict that
patient's eventual outcome i.e. patients advanced TNM cancer stage may have more death
than those with less advance TNM cancer stage. Prognostic factors need not necessarily cause
the outcomes but just predict their development. Thus prognosis is a prediction of the probable
outcome of a disease based on a individual's condition and the usual course of the disease as
seen in similar situations. Risk factors on the other hand are patient characteristics associated
with the development of the disease rather than the outcome of the disease.

The study designs for prognostic and risk factors are cohort study and case‐control study. Cross‐
sectional studies do not give valid conclusions about prognostic or risk factors because
temporal relationship between factors and outcome is not established.

A cohort study follows one or more groups (cohorts) of individuals who have not yet suffered
an adverse event and monitor the number of outcome events over time. An ideal cohort study
consists of well defined sample of subjects representative of the population of interest, and
uses objective outcome criteria.

Investigators can also collect "cases" of individuals who have already suffered the outcome
event (death due to cancer) and compare them to "controls" who have not (cancer patients
who are alive). In these "case‐control" studies the investigators count the number of individuals
in each group with a particular prognostic factor (advance or less advance TNM cancer stage).

To be valid studies on prognosis, these observational studies must be conducted and reported
with information that address this appraisal guide (Von Elm, 2007)

CLINICAL SCENARIO

Your brother consulted you because of what happened to his wife lately. She just had a
miscarriage (after 6 months of pregnancy) last week and she had not taken her meals lately.
They are already in their five years of marriage and they don’t have a child yet. Your brother is
asking you if they should take a vacation despite his being very busy at work and he is trying to
save for the house mortgage. What will you advice him?

If you advice him to take a vacation, they might loss their house or even his job. If you advice to
continue working and let time heal his wife’s grief, the chance of a psychological problem
worsening is great.

Since you have attended a workshop on evidence based family practice, you formulated the
question “What is the chance that my brother’s wife will go further into grief after the
miscarriage considering that they still don’t have any child yet?” and went to the library.

SEARCH

You typed the combination of the terms, pregnancy loss and prognosis and grief and you were
able to get across the article of Janssen et al. study entitled “A prospective study of risk factors
predicting grief intensity following pregnancy loss” publish in the Archive of General Psychiatry
last January 1997.

Now you proceed to see if the information in this study can be used to answer your brother’s
question.

CRITICAL APPRAISAL

RELEVANCE

• Is the objective of the article on prognosis similar to your clinical dilemma?

Your formulated clinical question must be addressed by the objective of the study. The PIO can
still be applied in this type of article. The objective of the study must clearly state that it is
determining the prognosis (outcome) of some patients with the prognostic factor
(intervention/exposure) among patients with the disease being studied (population).

VALIDITY GUIDES

• Was there a representative sample of patients without the outcome at the start of
observation?

The authors must specify how they defined or diagnosed the patients included in the study. The
authors should also specify at what stage of the disease they started observing their patients. If
these were not done bias can distort the result of the study. If the study included patients who
are more severe, the prognosis will naturally be poor and if they include patients who are mild,
the prognosis will be good. However if you mix these patients in one study without subgroup
analysis, the results will be mixed and biased.

If this is not explicit in the study, you can look at the inclusion criteria or examine the setting
where the study was done. The inclusion criteria may give the basis for the diagnosis, and the
setting may give the stage of the disease i.e. outpatient setting may have included patients in
the earlier stage and hospital setting may be patients in the late stage.

In the Janssen et al study, 221 women were recruited through a magazine add. They had a
stable marriage and reported a recent pregnancy loss. So these women may have been
recruited at a relatively similar stage.

• Was follow‐up sufficiently long and complete?

Just like in the paper about harm, the length of follow‐up must be sufficiently long enough to
detect the outcome. If follow‐up is short, the chance of arriving at a good prognosis is high
because few will develop the outcome resulting to false hopes for the patient. If it is too long,
the prognosis will be poor because everybody will eventually die in the long term. Measuring
prognosis over a given period is usually acceptable i.e. 5 year survival for chronic diseases, 6‐24
months survival for cancer, 30 days survival after ICU admission etc.

The number of lost to follow‐up will also lead to bias results especially when the outcome is
unknown. If patients were lost to follow‐up because they felt bad about the outcome,
prognosis will be better if they are excluded in the analysis. If they were lost to follow‐up
because they felt better and the investigators assumed the worse scenario, the prognosis will
look bad.

In the Janssen et al study, follow‐up was 94%, a relatively high rate.

• Were the criteria for determining the prognostic factor and outcome explicit and
credible?

The criteria for determining the outcome in study about prognosis are usually straightforward
I.e. mortality. Mortality or survival can be taken from death certificates and other medical
records; morbidity can be taken from hospitalization records, etc. In some cases outcomes are
recurrence of disease or disease progression in which case this must be clearly defined.
Definitions can be taken from the NLM MESH definitions or ICD 10 classification of the WHO.

The Janssen et al study, measured the following prognostic factors; a) Symptom Checklist‐90, b)
Dutch Personality Inventory, and c) information about quality of partnership, education,
religion, social support etc. using existing records and surveys. The outcomes were measured
using the Perinatal Grief Scale immediately after pregnancy loss and at 6, 12 and 18 months.

• Was there adjustment for other prognostic factors?

Age and sex are factors that can affect prognosis but something we cannot do about. Thus
many prognostic studies look at the effect of other modifiable prognostic factors by adjusting
for age and sex. Doing subgroup analysis does this. In subgroup analysis, the results of the study
are presented for each subgroup i.e. age and sex. Thus the prognosis of different TNM stage for
cancer can be presented in different age group or in different sex.

Another method is through multivariate analysis or regression model approach. In this
approach the basic variables included in the model are the prognostic factor and the outcome.
To adjust for age and sex, these variables are included into the model. This is usually described
in the analysis section of the methodology.

In the Janssen et al study, multivariate analysis was done.


Since all the validity questions were fulfilled, the study can be considered to be valid.


• How large is the likelihood of outcome to occur in those with the prognostic factor in a
specified period of time? Was it statistically significant?

The relative risk is the incidence of the outcome in the group with the prognostic factor divided
by the incidence of the outcome in the group without the prognostic factor. If the outcome
being measured is death and the relative risk is more than 1, then the factor results into poor
prognosis and if less than 1 the factor causes good prognosis. Relative risk is usually computed
when the design is a cohort study. In a case control study, the odds ratio is computed. The odds
ratio approximates the relative risk, especially when the disease is rare.

If the relative risk or odds risk is 1.11, it means the chance of developing the outcome is just
slightly higher if the patient has the prognostic factor. If the relative risk or odds risk is 1.99 it
means the chance of developing the outcome is almost two times (2x) and if the relative risk or
odds risk is 9.89 the chance is almost ten times (10x). The question of “how large is the chance”
involves preferential judgment from the patient and the physician.

To be statistically significant, the upper and lower values of 95% confidence interval should be
greater than 1 to say that the factor gives a bad prognosis when the outcome is death. If one
value of the 95% confidence interval is less than 1 and the other is more than 1, then the effect
of the prognostic factor is uncertain. In some cases, studies report the p value for statistical
significance i.e p <0.05 as significant.

The Janssen et al study reported that grief intensity was higher for a) women who had been
pregnant longer, b) pre‐loss neurotic personalities, c) pre‐loss psychiatric symptoms, and d) did
not have any living children. All these factors were significant at p <0.05.



Just like in an article about harm, for the prognostic factor to be extrapolated to your patient
you have to be assured that the characteristics of your patient is similar to the study’s inclusion
criteria. The setting may also be important. Patients being observed in setting where the
facilities are advanced and complete may have better prognosis than among patients who are
being observed in resource poor setting even though they have the same prognostic factor.

The subjects in the Janssen et al study were women who reported recent pregnancy loss with
stable marital relationship. The subjects were similar to the sister‐in‐law’s case.

• Can I use the results to decide on the intervention or reassure my patient?

Prognostic data should be used in decisions about therapy. Knowing the probability of the
outcome based on the prognostic factors present in the patient should influence the decision to
give or withhold treatment. For example surgical excision for cancer with the hope of improving
survival should be withheld in favor of palliation treatment if the prognosis of the patient is
very poor.

Prognosis data may also be helpful in reassuring anxious patients about their outcome. For
example some patients with dyspepsia may become too worried about the chronic epigastric
symptom and can be reassured and counseled about the low prognosis of dyspepsia leading to
bleeding ulcer or cancer.


Based on the Janssen et al study, you would rather advice your brother to take a vacation,
because his wife’s grief may even intensify based on the results of Janssen et al study.

REFERENCES

Andreas Laupacis, George Wells, W. Scott Richardson, Peter Tugwell for the Evidence‐Based Medicine Working
Group. How to Use an Article about Prognosis. JAMA. 1994;272(3):234‐237.

National Library of Medicine. Medical Subject Headings. www.ncbi.nlm.nih.gov/sites/entrez (May 27, 2008).

Von Elm E, Altman D, Egger M, Pocock S, Gøtzsche P, Vandenbroucke J. STROBE Initiative. Strengthening the
reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational
studies. BMJ 2007; 335: 806‐808.

WORKSHOP
ABOUT PROGNOSIS
(SESSION BRIEFING)

OBJECTIVES

making using an article about prognosis. Another objective is to introduce concepts of critical
appraisal of an article regarding prognosis:

• Validity
• Representative sample

CASE SCENARIO

You attended the morbidity and mortality conference in the Department of Medicine where a
case of 55 year old male diagnosed with lung cancer was presented. He was diagnosed to have
lung cancer 3 years ago and was given combined therapy (radiation and chemotherapy). He was
told that his cancer was already “cured” and was able to go back to work after the treatment.
Three months ago he came back to his doctor for difficulty of breathing and subsequent CT scan
showed a mass encroaching on the right bronchus suggesting recurrence of lung cancer. A
month later he was admitted to the ICU but eventually died after 3 days. During the
presentation one consultant suggested that the medical resident should be accountable for
“not doing enough” to save the patient.

Do you think the resident should be held accountable for the outcome?

Appraisal Sheet
CLINICAL DECISION ON PROGNOSIS

QUESTION

SEARCH

CRITICAL APPRAISAL

clinical dilemma?

PRIMARY VALIDITY Was there a representative sample of patients without the
GUIDES outcome at the start of observation?

Was follow‐up sufficiently long and complete?

Were the criteria for determining the prognostic factor and
outcome explicit and credible?

Was there adjustment for other prognostic factors?

VALID?

WHAT ARE THE How large is the chance of the outcome to occur in a
RESULTS? specified period of time? How precise were they?


ME IN CARING FOR MY
PATIENTS?

Can I use the results to decide on the intervention or
reassure my patient?

RESOLUTION OF THE
PROBLEM IN THE
SCENARIO

Evidence-based Medicine: Learning Module on Health Economic Analysis
ARTICLE ABOUT HEALTH
ECONOMIC ANALYIS
(SESSION BRIEFING)

OBJECTIVES

concept of medical decision making using an article about a health economic
analysis.


INSTRUCTIONS

Read the reading assignment for an article about a health economic analysis.
Focus on the critical appraisal questions, why they are asked and how to get the
answers from the paper.


READING ASSIGNMENT
CLINICAL DECISION USING AN
ARTICLE ON HEALTH
ECONOMIC ANALYSIS

HEALTH ECONOMIC ANALYSIS

Health economic analysis is a formal, quantitative methods used to compare
alternative strategies with respect to their cost and their expected outcomes
(Eisenberg, 1989). Its purpose is to inform decisions on resource allocation. It can
potentially inform decisions in institutions like hospitals and in regional or
national health policy (Russell, 1996). In this design the cost of a particular
intervention is estimated. Estimation include direct and indirect costs. There are
three types of economic analysis depending on the type of outcome. If the
outcome being considered is effectiveness of treatment, it is called cost‐
effectiveness analysis. If the outcome is savings in terms of monetary units it is
called cost‐benefit analysis. If the outcomes are equal and the cost is the only
one being compared it is called cost minimization.

Health economic analysis is becoming a popular tool for decision making by
health care providers and policy makers. There are three types:
• Cost minimization – aims to identify cost and areas for minimizing cost of
a given treatment option or program with no attempt to examine its
effect on effectiveness.
• Cost‐benefit – cost benefit measures the cost of two alternatives and
compares outcomes in terms of monetary value i.e. clinical outcomes are
assigned an estimated monetary value.
• Cost‐effectiveness – a full economic a analysis where there is comparison
of alternative options in terms of outcomes expressed as ratio of cost per
unit of effectiveness

Cost‐effectiveness analysis compares the outcome of alternative options in
terms of their monetary cost per unit of effectiveness i.e. cost per life saved, cost
per disease prevented etc. It is an important tool when setting priorities for
allocation of resources. It is most useful when it is compared with other
alternatives.

CLINICAL SCENARIO

Your father is the mayor of one of the town in the Visayas. His budget for health
care is minimal but he wants to start a program for pregnant women. Although
most of the pregnancies in the area were low risk, most patients still go to the
nearest obstetrician who is in the city 50 kilometers away. He is planning to set‐
up a maternity hospital with a trained obstetrician in your hometown. He is now
asking you for a recommendation.

What is your recommendation?

SEARCH

Having attended an EBM workshop you thought that the best way to convince
your father is to show cost‐effectiveness of handling low risk deliveries with
different types of trained physicians. You were able to get an article by Ratcliffe
and Tucker entitled “The costs of alternative types of routine antenatal care for
low‐risk women: obstetrician‐led shared care vs care by general practitioners
and midwives” published in the Journal of Health Services and Policy, 1996.

Appraising the article and conveying the information to your father seemed to
be a good strategy, so you proceeded in appraising the article.

CRITICAL APPRAISAL

RELEVANCE

• Is the objective of the article on economic analysis similar to your
clinical dilemma?

Your scenario must be addressed by the objective of the study. The perspective
or “point of view” in economic analysis usually refers to the one who will pay for
the intervention. Often, the point of view of the economic analysis is stated in
the objective and this is important to determine its relevance to your case
scenario. If you are using an economic analysis for policy decision like the
government pay for this kind of drug or facility, then the point of view must be
the societal point of view. If your scenario is to assist a patient to make a
decision on an intervention in a “pay‐for‐service” setting, then the perspective
must be from the patient or “payer” perspective. Health insurance perspective is
also a payer perspective.

VALIDITY GUIDES

• Did the analysis provide a full economic comparison of health care
strategies?

Physicians usually choose between two alternatives. The range of alternative
strategies examined must include at least the currently accepted standard and
the new alternative (O’Brien, 1997). Another alternative is the “do nothing”
alternative but may not be realistic in some cases because of ethical issues.

When we compare the cost of giving each of the two alternatives, this is cost
analysis. When we use this to make a decision, we only give the alternative with
the lowest cost that may not be necessarily effective. When we use comparison
of effectiveness such as a randomized controlled trial in making a decision, we
give an effective alternative that the patient may not be able to afford. Thus it
makes sense to consider cost and effectiveness when making clinical decisions.

A full economic analysis compares not only the cost of the two alternatives but
also integrate information about efficacy of the alternatives. Thus cost and
outcomes should both be analyzed for each alternative strategies being
compared. This can be achieved if the study design is a cost‐benefit or a cost
effectiveness approach.

The Ratcliffe and Tucker study was a full economic valuation of the cost of tests,
investigations, personnel, cost incurred by the patients of pregnancies delivered
by obstetricians vs family physicians. The study included patients enrolled in a
randomized controlled trial to answer the effectiveness outcome.

• Were the costs and outcomes properly measured and valued?

Cost pertains to resources used and this must be differentiated from charges or
prices of commodities. The point of view of costing refers to the one who will
pay for the cost and this may differ. For example cost to government hospital or
funder may be different from the point of view of the patient. What may be cost
saving for the funder may actually be an increased cost for the patient. The ideal
point of view is from the society’s view, but this is difficult to measure. Thus
proper measurement of cost may differ from the health care system. In a system
where the payment is a fee‐for‐service set‐up, an economic analysis on the point
of view of the paying patient may be a good basis for making decision. In a
system where health care is being paid for by the government, an economic
analysis from the point of view of society may be a good basis. Thus in measuring
cost, the point of view of the analysis must be established (O’Brien, 1997).

Outcomes in health care must be an outcome that is of value to the patient and
society. It should be something that can be appreciated by the patient. For
example in making a decision about the treatment for hypertension, outcomes
like decrease in incidence of mortality or stroke, decrease in hospitalization or
myocardial infarction instead of just the lowering of blood pressure should be
the outcome to be considered. In addition these outcomes must be measured in
the best possible designs i.e. randomized controlled trials for treatment,
controlled comparison for complex intervention etc. Systematic reviews or meta‐
analysis of these interventions are better methods for establishing outcomes.

Lastly, the cost and outcome must be expressed as a ratio i.e. cost per outcome
(cost per life‐year gained, or cost per death avoided etc.). This expression of
result will give the most relevant information for decision making.

In the Ratcliffe and Tucker study costs were extracted from clinical data that
came from a randomized controlled trial. They included cost per patient, staffing
cost, non‐health services cost and mean societal cost. Cost data was available in
about 94% of subjects included in the trial.

• Was appropriate allowance made for uncertainties in the analysis?

Economic analysis usually depends on analysis of secondary data and since not
all data are available for each alternative, some assumptions need to be made.
There are uncertainties to these assumptions. A good economic analysis is one
that recognizes these uncertainties and look at how it affects their findings. This
method is called sensitivity analysis (O’Brien, 1997).

In sensitivity analysis the estimates for key variables in costs are changed in
order to assess their impact they on the results. The changes can be based on
variation in price index, opportunity costs, geographical price differences etc. In
terms of outcomes, the variation can be from confidence intervals or from
lowest and highest effect noted from the studies that were reviewed.


Since the study was a full economic comparison and wide perspective of cost
was considered you decided that the study was valid.


• What were the costs and outcomes of each strategy?

Economic analysis papers should have tables that report the costs of resources
used in the alternative intervention such as drugs, personnel services, facilities,
supplies etc. It should also contain tables about the outcome of each alternative.
Lastly, this is expressed as a cost‐effectiveness ratio or cost‐benefit.

The table below is a good guide to help decide the alternative to choose. “C” will
be the best choice since it is more effective and less cost. “B” is not a good
choice because it is less effective but more expensive. “A” is more effective but
more expensive as well.

Effectiveness
High Low
Cost High A B
Low C D

Sometimes, an alternative may be more effective but also more expensive. To
make a decision in this scenario, an incremental analysis should be done.
Incremental cost is the amount we pay for the added effectiveness of the
alternative. Usually the availability of resources and personal judgment may be
needed to decide whether the incremental cost is worth it.

Ratcliffe and Tucker showed that the total societal mean cost for GP or midwife
care was lower by P 2,178 and was statistically significant.

• How much does allowance for uncertainty change the result?

Looking at how uncertainties affect the results is called sensitivity analysis. A
direct approach for doing this is by computing for the cost effectiveness ratio
using the lower and upper limit of the 95% confidence interval of the
effectiveness outcome. If the cost‐effectiveness values are reversed with
sensitivity analysis then the results are considered to be soft and its reliability is
less.


• Could my patients expect similar outcomes?

The inclusion criteria of the cited clinical trial or other studies reviewed to
determine the outcome and the setting from which the trial was done can be
duplicated in your setting will play an important factor. If the patients in the
study are similar to your patient and setting for which the intervention was given
can be duplicated, then you can expect the same outcome (O’Brien, 1997).

• Could my patients expect similar costs?

The health care system may be different from the setting where the economic
analysis was done. This difference may lead to difference in costing once applied
for decision making in your setting. Cost data may be different for two reasons:
1) clinical practice vary in resource consumption associated with the treatment
and 2) prices for resources differ from those used in the study (O’Brien, 1997).
This can only be answered if the analysis presented the detailed cost so the
reader can decide whether the costing in the study can also be applied in his/her
own setting.

Countries may differ with respect to the value they place on health benefits. If
$50,000 per life‐year is an acceptable cost‐effectiveness threshold for the US it
may not be affordable in the Philippines. Countries vary in their willingness to
pay for health care (O’Brien, 1997).


Since most pregnancies in your hometown were low risk, you showed to your
father that the town will save more money if they hire family physicians or
midwives rather than an obstetrician. You called up your father one month later
and the town council opted to hire more midwives.

REFERENCES

Espallardo, NL. Research Protocol Development for Resident Physicians. Family Medicine
Research Group, Inc. Manila, 2000.

Eisenberg JM. Clinical economics. A guide to the economic analysis of clinical practices. JAMA,
1989; 262:2879‐86.

O'Brien B, Heyland D, Richardson WS, Levine M, Drummond M, for the Evidence‐Based Medicine
Working Group. How to use an Article on Economic Analysis of Clinical Practice: Validity Guides.
JAMA, 1997; 277(19):1552‐1557.

O'Brien B, Heyland D, Richardson WS, Levine M, Drummond M, for the Evidence‐Based Medicine
Working Group. How to use an Article on Economic Analysis of Clinical Practice: Results and
Applicability. JAMA, 1997; 277(22):1802‐1806.

Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC. The role of the cost‐effectiveness
analysis in health and medicine. JAMA, 1996; 276(1)‐1172‐7.

WORKSHOP
ARTICLE ON HEALTH
ECONOMIC ANALYSIS
(SESSION BRIEFING)

OBJECTIVES

medical decision making about an economic analysis. Another objective is to
introduce concepts of critical appraisal of an article regarding an economic
analysis focusing on the following:

• Validity
• Costs in health care
• Incremental costs

CASE SCENARIO

You’re in a summer internship program with the Philippine Health Insurance
together with some medical students from Ateneo de Manila University College
of Medicine and Public Health. You were asked to give a report on a benefit
package for patients with diabetes mellitus. One of the issues raised is whether
to include a screening program in the package or just wait for a definite
diagnosis of diabetes mellitus to avail of the package. Another issue is should the
screening program include pre‐diabetes screen with impaired glucose tolerance?

What will you recommend?

To answer these questions, you need to read about diabetes and how it is
diagnosed.

Appraisal Sheet
CLINICAL DECISION ON ECONOMIC ANALYSIS

QUESTION

SEARCH

CRITICAL APPRAISAL

RELEVANCE Is the objective of the article on economic analysis similar
to your clinical dilemma?

VALIDITY GUIDES Did the analysis provide a full economic comparison of
health care strategies?

Were the costs and outcomes properly measured and
valued?

Was appropriate allowance made for uncertainties in the
analysis?

VALID?

WHAT ARE THE What were the incremental costs and outcomes of each
RESULTS? strategy?

How much does allowance for uncertainty change the
result?

CAN THE RESULTS HELP Could my patients expect similar outcomes?
ME IN CARING FOR MY
PATIENTS?

Could my patients expect similar costs?

RESOLUTION OF THE
PROBLEM IN THE
SCENARIO

Evidence-based Medicine: Learning Module on Meta-analysis
ARTICLE ABOUT SYSTEMATIC
REVIEWS OR META-ANALYSIS
(SESSION BRIEFING)

OBJECTIVES

concept of medical decision making using an article about systematic reviews or
meta‐analysis.


INSTRUCTIONS

Read the assignment for an article about systematic reviews or meta‐analysis.
Focus on the critical appraisal questions, why they are asked and how to get the
answers from the paper.


READING ASSIGNMENT
CLINICAL DECISION ON
SYSTEMATIC REVIEW OR
META-ANALYSIS

REVIEWS, SYSTEMATIC REVIEWS AND META-
ANALYSIS

A review is secondary study design that integrates findings of two or more
studies that discuss similar topic usually defined by PIO. There may be some bias
when the reviewer subjectively decides which studies to include or exclude in
the review. A systematic review is similar to review but has a way to
systematically search the literature searching and has systematic rules in
combining the studies to be reviewed. The results of systematic reviews are
more often objective than a review. Meta‐analysis is like a systematic review but
applies some statistical analysis to the results.

Sample of meta‐analysis result

A meta‐analysis is a procedure that integrates and combine the results of two or
more primary studies that are similar in the population enrolled the intervention
used and the outcome measured. The pooled result is then subjected to a
statistical analysis. A well conducted meta‐analysis allows a more objective
appraisal of the existing evidence about a problem than a traditional review or
systematic review. Meta‐analysis may also be biased owing to the inclusion or
exclusion of some irrelevant or relevant studies respectively (Espallardo, 2000).

CLINICAL SCENARIO

Your grandmother had a history of fall a week ago after taking a bath. She was
brought to the hospital for treatment of minor bruises in her knees. The x‐rays
were normal. Your mother asked you if she needs a walker or cane to prevent
falls and subsequent injury.

SEARCH

You ask a colleague from Rehabilitation Medicine and she gave you an article
from the NHS Center for Reviews and Dissemination she got from the internet
entitled “Preventing falls and subsequent injury in older people”.

CRITICAL APPRAISAL

RELEVANCE

• Is the objective of the article on meta‐analysis similar to your clinical
dilemma?

Your formulated clinical question must be addressed by the objective of the
study. The objective of an appropriately done meta‐analysis is often a focused
clinical objective with PIO and the method being clearly defined. This is often
used for the systematic literature search and basis for inclusion or exclusion.

VALIDITY GUIDES

• Did the review address a focused clinical problem?

Systematic reviews of the medical literature try to summarize publications
related to a similar topic. Because several articles are combined together,
sometimes the purpose of the review is not clear or very broad. It therefore
becomes difficult to determine what the review is trying to achieve.

In order to know what the objective of the review is, the clinical problem must
be focused. There must be a clear description of the patient and its relation with
an exposure or an outcome.

The NHS study specifically stated that they tried to identify strategies that
prevent falls and subsequent injury in older people. The focused objective
seemed to apply to your problem.

• Were the criteria for searching and selecting articles for inclusion and
exclusion explicit and credible?

In conducting systematic reviews, a systematic search and appraisal of the
literature should be done in order to:
• ensure that no relevant articles were missed
• studies were included because they are good studies and not because
they agree with the authors opinion
• studies were excluded because they are bad studies and not because
they disagree with the author’s opinion.

Thus paper should describe the method of searching for the medical literature.
Statements like “an electronic search of published articles in the MEDLINE using
the terms . . . from 1966 to 2000 was done” must be found somewhere in the
methodology section. This assures the readers that the findings of the study
were based on a wide range of literature source and represent the most current
and complete information about the clinical problem.

The paper should also describe how they include or exclude retrieved articles.
The paper must contain statements like “all retrieved abstracts were reviewed
by three independent reviewers and articles that were randomized controlled
trial on . . . using the intervention . . .” somewhere in the method section. This
assures the readers that the articles used in the study were objectively chosen
and not because they agree with the authors opinion.

The NHS study identified trials published in computerized databases like Social
Science Citation Index, PSYCHLIT, EMBASE, RCN database, AMED and UNCOVER.
The citations also identified reviews and peer contribution from reviewers and
other experts in the field.

• Was the validity of included studies appraised and the appraisal
reproducible?

Even if all included studies are randomized controlled trials, there may be some
small differences among different trials that might affect the results of the study.
Thus a standard appraisal of each article must be done. Peer review may not be
a reliable method of appraisal. Differences in peer perception and interest may
lead to differences in the result of the appraisal.

While there are no agreed standards to evaluate validity, the review must have
at least developed a checklist of criteria that focused on the methodology of the
study being appraised.

The NHS study included only randomized controlled trials that evaluated
strategies to prevent falls.


Overall the study is valid.


• What are the overall results of the systematic review?

When looking at the results of a systematic review or meta‐analysis, you should
look for clinically relevant presentation like lower mortality rates in one group
compared to the other, or the difference in quality of life scores between the
two groups. Sometimes subgroup analysis i.e. patients with high risk and
patients with low risk, to see the different effect of an exposure may also be
helpful.

In a meta‐analysis, the confidence interval of the overall results can also be
computed and this can provide information about the precision of the results.

Thirty‐six randomized controlled trials were included in the meta‐analysis. The
results showed that 10‐24 weeks of exercise including balance training showed
an effective risk reduction by as much as 37% with an adjusted fall incidence
ratio of 0.90 and with a 95% CI of 0.81 to 0.99.



In a systematic review, the patient’s characteristics are varied because they
came from different studies. Application to patients therefore becomes wider.
When variation in patient inclusion may influence the effect, subgroup analysis
between different patient characteristics may also help decide what kind of
patient will benefit from the intervention or will be affected by the exposure.

This information can be seen in the methodology section where the researchers
describe the type of patients in the literature search and inclusion criteria of the
studies.

The NHS meta‐analysis included only studies done on elderly.

• Are the results of the review relevant to my patient?

When the clinical question of the review is focused, the answer to this question
becomes evident. You only need to focus on the outcome measured and decide
whether this is relevant to your patient.

When the outcome differs between studies, they are combined and this is
reported in systematic reviews or meta‐analysis as effect size. This is a difficult
situation because the outcomes are combined and you cannot easily decide
whether the outcome is relevant or not. In this case you can look at the results of
individual studies and choose studies that give relevant outcomes and use them
to make a decision.


Based on the review, balance training rather than walker devices will help
prevent further fall and subsequent injury.

REFERENCES

Espallardo, NL. Research Protocol Development for Resident Physicians. Family Medicine
Research Group, Inc. Manila, 2000.

Oxman A, Cook D, Guyatt G, for the Evidence Based Medicine Working Group. How to Use an
Overview. JAMA, 1994;272(17):1367‐71.

WORKSHOP
ARTICLE ABOUT A SYSTEMATIC
REVIEW OR META-ANALYSIS
(SESSION BRIEFING)

OBJECTIVES

medical decision making using an article about a systematic review or meta‐
analysis. Another objective is to introduce concepts of critical appraisal of an
article regarding differential diagnosis focusing on the following:

• Validity
• Review, systematic review or overview, meta‐analysis

CASE SCENARIO

Your 55 year old mother is asking if she should be taking hormone replacement
therapy. She had menopause three years ago and is having hot flushes every
now and then but says it’s tolerable. Her friend is taking oral estrogen but
confided that she heard it has side effects and is consulting her doctor if she can
stop it. Your mother is also contemplating to ask for prescription on hormone
replacement. What advice will you give her?

Appraisal Sheet
CLINICAL DECISION ON SYSTEMATIC REVIEW OR META‐ANALYSIS

QUESTION

SEARCH

CRITICAL APPRAISAL

clinical dilemma?

PRIMARY VALIDITY Did the review address a focused clinical problem?
GUIDES

Were the criteria for searching and selecting articles for
inclusion and exclusion explicit and credible?

Was the validity of included studies appraised and the
appraisal reproducible?

VALID?

WHAT ARE THE What are the overall results of the systematic review?
RESULTS?


ME IN CARING FOR MY
PATIENTS?

Are the results of the review relevant to my patient?

RESOLUTION OF THE
PROBLEM IN THE
SCENARIO

Evidence-based Medicine: Learning Module on Using Clinical Practice Guideline
CRITICAL APPRAISAL OF A
CLINICAL PRACTICE GUIDELINE
(SESSION BRIEFING)

OBJECTIVES

concept of medical decision making using a clinical practice guideline.


INSTRUCTIONS

Read the assignment for an article about a clinical practice guideline. Focus on
the critical appraisal questions, why they are asked and how to get the answers
from the paper.


READING ASSIGNMENT
HOW TO USE A CLINICAL
PRACTICE GUIDELINE

CLINICAL PRACTICE GUIDELINES

"Clinical Practice Guidelines are systematically developed statements to assist
practitioner decisions about appropriate health care for specific clinical
circumstances."(Field, 1990). Guidelines were developed to:

• Make evidence‐based management explicit.
• Make clinical decision making more objective and scientific.
• Assess professional performance.
• Educate the patients and practitioners about current "best practice."

If guidelines are to improve practice, they need to be developed by the people
who are actually going to have to apply them. Guidelines, like so much else in
healthcare today are no longer as immutable as the Laws of the Medes, you will
have to periodically check that they are working, that they are getting to the
people who are going to use them and that they are up to date.

In general, good topics for guidelines are those which:

• contribute a high workload
• poor treatment risks disastrous outcomes
• change is practical with the resources you have available
• there is variation in current management (i.e. there is uncertainty about
the best management strategy), but some hope of reaching a consensus
• the changes you wish to implement will be acceptable to patients
• there is good evidence to back up the protocols

It might be useful to consider the criteria used to select illnesses for screening
programs (how common, how serious, how preventable, how acceptable?

Clinical practice guidelines, which have been defined as "systematically
developed statements to assist practitioner and patient decisions about
appropriate health care for specific clinical circumstances," represent an attempt
to distill a large body of medical knowledge into a convenient, readily useable
format. Like overviews, they gather, appraise and combine evidence. Guidelines,
however, go beyond most overviews in attempting to address all the issues
relevant to a clinical decision and all the values that might sway a clinical
recommendation. Like decision analyses, guidelines refine clinical questions and
balance trade‐offs. Guidelines differ from decision analyses in relying more on
qualitative reasoning and in emphasizing a particular clinical context.

Clinical guidelines are likely to be an important part of medical practice for the
foreseeable future. They are tools for consistency and effectiveness in patient
care. Its development methodology has evolved such that there are
internationally agreed standards with which to develop and assess guidelines.
Key to the process is the rigor of the systematic review in order to assess the
best evidence on which to base recommendations. In order to safeguard trust,
guideline development groups must have editorial independence of government,
industry and special interest groups, while at the same time having regard to the
implications of their recommendations (Keeley, 2003).

There is an imperfect evidence base to support decisions about which guideline
dissemination and implementation strategies are likely to be efficient under
different circumstances. Decision makers need to use considerable judgment
about how best to use the limited resources they have for quality improvement
activities. In order to effectively disseminate and implement clinical practice
guidelines among health care providers, it is essential to determine the level of
knowledge, attitudes and practices of physicians on the use of CPGs (Espallardo,
2007).

A total of 169 physicians participated in the survey coming from 17 hospitals.
Majority of the respondents were from hospitals in the National Capital Region
and three hospitals were from the provincial area, one from Ilocos Norte and
two from Oriental Mindoro. Most of the respondents were internists (18.9%)
followed by obstetricians and gynecologists (16.6%) and pediatricians (16.0%).

Overall there was a high level of awareness on CPG among the doctors surveyed.
The average awareness score was 89.0%. An excellent level of awareness (more
than 92.0%) was noted in terms of the general description of CPG and its
evidence‐based process of development. In terms of attitude, there were more
respondents (76.9%) who seemed to be open and have a favorable attitude to
CPG. In terms of overall attitude score, the mean score was 54.2% out of the
possible 100%. There is a mild negative correlation between awareness to CPG
and overall attitude to CPG (Pearson r = ‐0.23; p value = 0.004). This means the
higher the level of awareness the less favorable are the doctors’ attitude. The
actual practice of physicians is only between 33‐37% when compared to CPG
recommendations as standard criteria. The physicians’ actual practice score is
lower than their current knowledge of CPG recommendations (Espallardo, 2007).

Figure 1 Awareness, Attitudes, Knowledge and Actual Practice Scores of
Physicians Surveyed
Awareness Score % Unfavorable Favorable

(SD)
Awareness to CPG 89 (SD 12)
Attitudes Score % Unfavorable Favorable

(SD)
Overall attitude to CPG 54 (SD 6)
Chart Review Scores Score % Unfavorable Favorable

(SD)
Acute gastroenteritis 33 (SD 11)
Normal delivery 37 (SD 9)

CASE SCENARIO

At the end of a busy day in your clinic, you are glad to find that your last patient
is a 45/male previously diagnosed to have hypertension. He claims that his
highest blood pressure for the past month was 160/90 and his usual blood
pressure was 130/90 and he has been taking a beta‐blocker for the past year. He
was relatively symptom free save for occasional headache and nape pains during
BP spikes. Thinking that this was just another run of the mill hypertensive
patient you were ready to refill his prescription and give your usual advice
regarding diet and exercise. As you were about to do just that, your patient
began asking you the relative benefits of the alternative drugs available in the
market. He was also asking if he needed to have an ECG done together with
blood chemistries, urinalysis and a 2D‐Echo since his friend who consulted
another physician was advised to do that. Since he was not overweight he was
asking if a regular exercise program would add any additional benefit in
controlling his symptoms. You gave him the usual advice you knew based on
your knowledge of pathophysiology and pharmacokinetics. However, as you
were finally closing your clinic you decided that you were not satisfied with the
answers you gave him and decided to do a search for the best available
evidence.

SEARCH FOR CPG

You decide that in order to find relevant answers to a variety of clinical
questions, a clinical practice guideline would be the best article to retrieve. You
initially searched ww.guidelines.gov and found numerous guidelines for
hypertension. However, you did not have sufficient time to download and print
the full text version of these guidelines. You then remembered a copy of the
Philippine compendium sent to you by mail 2 years ago and taught of the
Philippine Clinical Practice Guidelines on the Detection and Management of
Hypertension.

You then go home, sit at your desk and decide to review this article in order to
be more prepared for your next patient encounter.

CRITICAL APPRAISAL

RELEVANCE

• Is the objective of the article on clinical practice guideline similar to
your clinical dilemma?

Your formulated clinical question must be addressed by the objective of the
clinical practice guideline. Usually guideline objectives are broad i.e. answers
questions about the best diagnostic test, the recommended treatment, the
expected outcome or prognosis etc.

VALIDITY GUIDES

• Were all important options and outcomes considered?

Guidelines aid us in our decision making skills and we make better judgment calls
if we know all the alternative options open to us and the relative harm and
benefits of each choice. Guideline developers then should present most of the
reasonable options seen in practice and their corresponding outcomes.

In the case of the Philippine Clinical practice guideline on the detection and
management of hypertension, several treatment options were presented
ranging from beta‐blockers, diuretics, ACE inhibitors, calcium channel blockers
with recommendations of the best alternative for hypertensive patients with co‐
morbid conditions. The guideline however did not include the newer generation
anti‐hypertensives.

As important as presenting all the options, the corresponding outcomes such as
morbidity and mortality data, prevention of complications and other measures
that improve health related quality of life should be reported. Inasmuch as all
these will be helpful and clinically relevant to individual patients.

In this hypertension guideline, mortality and morbidity data together with
prevention of hypertensive complications were the outcomes given emphasis.
However costs and side effects were not well mentioned.

• Was an explicit and sensible process used to identify, select, and
combine evidence?

Guideline developers must allow the reader to know how the evidence has been
tracked, reviewed, appraised and combined in order to allow them to ascertain
the validity of the gathered evidence. Developers should specify a focused
question, search the literature for available evidence, critically appraise this
evidence and summarize the results in an easy to understand material.

The Philippine Clinical Practice Guidelines for Hypertension was not very clear
regarding how they searched the literature and what database they used.
Mention of tracking, retrieving and appraising was done in Phase 1 of the
introduction section, but the complete way on how this was done was not
mentioned. However, summary on how the articles were reviewed and graded
was provided.

• Is the guideline likely to account for important recent developments?

You should look for two important dates: the publication date of the most recent
evidence considered and the date on which the final recommendations were
made. Some authorities also identify important studies in progress and new
information that could change the guideline. Ideally, these considerations may
be used to qualify guidelines as "temporary" or "provisional," to specify dates for
expiration or review, or to identify key research priorities. For most guidelines,
however, you must scan the bibliography to get an impression of how current a
particular guideline may be.

Once you are confident that the clinical practice guideline addresses your clinical
question and is based on a rigorous up‐to‐date assessment of the relevant
evidence, you can review the recommendations to determine how useful they
will be in your practice.

The reader is advised to check the bibliography section of the guideline and
check the dates of the most recent articles included. Ideally, the evidence
should be within the last 2 years before the guideline was published. Since
medical knowledge rapidly transforms, this will ensure that our
recommendations will not be outdated. Hence, there is a need to revise
guidelines periodically.

This guideline on hypertension was released in 1995 and the latest evidence
upon looking at the bibliography was in that same year. Being at present in the
year 2000 and with the rapid developments in antihypertensive medications, the
guideline should be due for review and revision.

• Has the guideline been subjected to peer review and testing?

People may interpret evidence differently and their values as to what important
options and outcomes are may differ. As such, a guideline that has been
subjected to scrutiny by external reviewers and tested in an actual clinical
practice setting and found acceptable might be easier to use.

OVERALL, IS THE GUIDELINE VALID?

Once you are confident that the guideline meets at least 2 out of the 3
requirements above, you can review the recommendations and its applicability
to our individual patients. At present, the Philippine Clinical Practice Guidelines
on Detection and Management of hypertension although a little bit outdated at
the present time will do.

WHAT ARE THE RECOMMENDATIONS

• Are practical, clinically important, recommendations made?

To be useful guidelines should give practical, unambiguous advice addressing a
particular clinical situation. Recommendations should be simple and specific at
the same time comprehensive enough to allow the reader a chance to assess the
benefits and costs of following the particular recommendation.

In this hypertension guideline, recommendations are divided into every aspect of
any encounter with a hypertensive patient. The following 6 questions are
addressed by the guideline:

• How should blood pressure be measured?
• How should hypertension be diagnosed?
• How should hypertension be worked up?
• What advice should hypertensive patients receive regarding lifestyle
modification?
• How should hypertension be treated?
• How can hypertension be prevented among normotensives?

This then allows the clinician to answer aspects regarding diagnosis, laboratory
work‐ups, treatment options, non‐pharmacologic advice and preventive
measures.

• How strong are the recommendations?

The "strength," "grade," "confidence," or "force" of a recommendation should
be informed by multiple considerations:

• the quality of the investigations which provide the evidence for the
recommendations
• the magnitude and consistency of positive outcomes relative to negative
outcomes (adverse effects, burdens to the patient and the health care
system, costs)
• the relative value placed upon different outcomes.

Thus, grading of the recommendations are based on the methodological
soundness of the available evidence, the number of positive outcomes in
relation to negative ones and the consistency of findings across different
evidences available. It is not enough to look into the fact that randomized
controlled trials were used as evidence but also if findings across different trials
were consistent. Inconsistent findings are at times the reason why different
guideline developers have different recommendations regarding certain clinical
issues.

It is also important to note that different guideline developers use different
standards for grading their recommendations and that this should explicitly be
placed in the guideline for ease of understanding.

The Philippine Clinical Practice Guidelines for Hypertension used a system
adopted by the Canadian Hypertension Society. Therapy wise, a lot of Grade A
recommendation meaning that evidence is based from well‐conducted trials was
made.

WILL THE RECOMMENDATIONS HELP YOU IN CARING FOR YOUR PATIENTS?

• Is the primary objective of the guideline consistent with your
objectives?

The purpose of the guideline developers for coming up with recommendations
may vary from your own. Guidelines may be disseminated to assist physicians in
decision making (clinical algorithms), to evaluate their practice and the standard
of care they give to their patient (quality assurance) or to set limits for physician
choices (reimbursements, recertification). In any case, in order to find
recommendations most suited to your needs, the purpose of the guideline
should be in line with your intended objective.

This hypertension guideline was made to ensure the availability of a local
guideline for the detection and management of hypertension. Since your
questions dealt with management issues, this guideline is appropriate for your
purpose.

• Are the recommendations applicable to your patients?

You must determine if the kind of patients you have are similar to those patients
targeted by the guideline. If your patients have a different prevalence or risk of
disease, if the diagnostic and therapeutic options recommended are not
available in your area, the guideline might not apply.

The advantage of reviewing and applying a Philippine practice guideline is that it
takes into account the characteristics of our setting and hopefully allows it to be
more responsive.


Having deemed that the aforementioned guideline is valid, you would still opt to
give this patient with uncomplicated hypertension a beta‐blocker. In terms of
diagnostics, you would request for an FBS, Serum Creatinine, Serum potassium
and urinalysis. You would request for these tests since they would have an
effect on the antihypertensive you would choose. Furthermore they would
provide the following additional benefits: a)detection and early treatment of
diabetes, b)detection of asymptomatic renal disease, c)detection of possible
secondary hypertension. You would explain to your patient that since he has no
symptoms of any cardiac disease, performing an ECG and Echo is not routinely
recommended.

In terms of non‐pharmacologic advice, you would encourage him to go with
regular aerobic exercise such as walking, jogging or cycling 30 minutes per day 3‐
4x/week since regular physical activity reduces blood pressure and results in a
decrease in all cause mortality.

A lot of decision making, considerations of options and outcomes came into play
for a relatively simple case of hypertension. And as medical knowledge
improves, more options will be made available. As such the need for relevant,
well‐constructed and tested guidelines to improve our clinical decision making
will always be there. Again, we as clinicians should be able to adapt guidelines
that are valid and whose recommendations will be most appropriate and
feasible in our respective settings.

REFERENCES
Hayward R, Wilson M, Tunis S, Bass E, Guyatt G for the Evidence Based Medicine Working
Group. How to Use a Clinical Practice Guideline: Validity. JAMA, 1995;274(7):570-4
Hayward R, Wilson M, Tunis S, Bass E, Guyatt G for the Evidence Based Medicine Working
Group. How to Use a Clinical Practice Guideline: Recommendations and Applicability. JAMA,
1995;274(20):1630-2.

WORKSHOP
CRITICAL APPRAISAL OF A
CLINICAL PRACTICE GUIDELINE
(SESSION BRIEFING)

OBJECTIVES

medical decision making using an article about a clinical practice guideline.
Another objective is to introduce concepts of critical appraisal of an article
regarding clinical practice guidelines focusing on the following:

• Validity
• Interpretation of the recommendations
• Applicability of the recommendations
• Decisions for the patient (case scenario)

CASE SCENARIO

Your 65 year old grandmother is apparently healthy. Except for occasional joint
pains, she does not complain of anything. She has not been diagnosed to have
any chronic disease. But she wants to go to your clinic for a general check‐up.
What clinical tests and procedures will you do?

Appraisal Sheet
HOW TO USE A CLINICAL PRACTICE GUIDELINE

CASE SCENARIO OR
QUESTION

SEARCH

CRITICAL APPRAISAL

RELEVANCE Is the objective of the article on clinical practice guideline
similar to your clinical dilemma?

VALIDITY GUIDES Were all important options and outcomes considered?
Alternatives, expected results

Was an explicit and sensible process used to identify,
select, and combine evidence?

Is the guideline likely to account for important recent
developments?
Last update

Has the guideline been subjected to peer review and
testing?

OVERALL, IS THE
GUIDELINE VALID?

WHAT ARE THE Are practical, clinically important, recommendations
RECOMMENDATIONS made?


How strong are the recommendations?
Grading

WILL THE Is the primary objective of the guideline consistent with
RECOMMENDATIONS your objectives?
HELP YOU IN CARING
FOR YOUR PATIENTS?

Are the recommendations applicable to your patients and
setting?

RESOLUTION OF THE
PROBLEM IN THE
SCENARIO

Espellardo EBM Self-Instructional Manual

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Espellardo EBM Self-Instructional Manual

Hochgeladen von

Copyright:

Verfügbare Formate

EVIDENCE BASED MEDICINE:

Noel L. Espallardo, MD, MSc

Department of Family and Community Medicine

THE ADVANCED SEARCH STRATEGY

Awareness Score % Unfavorable Favorable

Awareness to CPG 89 (SD 12)

Attitudes Score % Unfavorable Favorable

Overall attitude to CPG 54 (SD 6)

Chart Review Scores Score % Unfavorable Favorable

Acute gastroenteritis 33 (SD 11)

Normal delivery 37 (SD 9)

Das könnte Ihnen auch gefallen