CRITICAL APPRAISAL

OF
LITERATURE
Rahmatina B. Herman
Fakultas Kedokteran
Universitas Andalas

Introduction
Research involves gathering, collecting and
analyzing of data to produce meaningful
information
Many of research did not perform in good
quality
Even many of studies were biased and the
results were untrue
It can lead to false conclusions

What is Critical Appraisal?

Critical appraisal is an important element of
evidence-based medicine
Critical appraisal is the process of systematically
examining research evidence to assess its validity,
results, and relevance before using it to inform a
decision (Hill and Spittlehouse, 2001)
Critical appraisal is the process of carefully and
systematically examining the research paper to
judge its trustworthiness, and its value and
relevance in a particular context (Burls 2009)

..What is Critical Appraisal?

Critical appraisal is a systematic approach to the
results of scientific papers:
- reading,
- understanding,
- interpreting,
- identifying the limitations,
- deciding upon the usefulness

..What is Critical Appraisal?

Critical appraisal is an essential step in the
process of putting research into practice:
- Asking questions about an articles research
methodology,
- Scrutinizing its data collection and analysis
methods,
- Evaluating how the presented findings can
help us to determine whether the conclusions
can influence us in decision-making

..What is Critical Appraisal?

The aim of Critical Appraisal is helping
people to develop the necessary skills
to make sense of scientific evidence
based on validity, results and relevance.

Why Do We Need Critical Appraisal Skill

Problem to obtain scientific information:
Vast and expanding literature.
Limited time to read.
Reason to read
Different reasons to read mean different
strategies.
- Keeping up to date.
- Answering specific clinical questions.
- Pursuing a research interest.

Stages of Obtaining Information

I. Clarify your purpose for reading

- Keeping up to date
Skimming the main journals and summary bulletins

- Answering specific questions

Finding good quality literature on the subject

- Pursuing a research interest

Extensive literature searching.

..... Stages of Obtaining Information

II. Specify your information need
- What kind of reports do I want?
- How much detail do I need?
- How comprehensive do I need to be?
- How far back should I search?
The answers to these questions should flow from
the reasons for reading

.....Stages of Obtaining Information

III. Identify relevant literature
- Selectivity is the key to successful critical
appraisal
IV. Critically appraise what you read
- Separating gold from ore
- Time is limited, stop reading the
unimportant information
- Simple checklists enable the useful
information to be identified.

Kind of Information in A Research Paper

During Critical Appraisal, try to find the following information:
1. Research question
2. Study design
3. Selection issues
4. Outcome factors and study factors; method to measure
5. Confounders
6. Statistical method
7. Statistical Results
8. Conclusions
6. Study limitations
7. Application the findings into daily clinical practice or
research
An ideal abstract contains at least the red color information

Abstract
After you read the abstract:

Do you want to know more after reading the

abstract?
Are there reasons to doubt the findings without
reading the whole article?

Introduction Section
The introduction sections must help
us to identify:
- key concepts,
- goals,
- subjects, and
- themes of the research.

Methodology Section
The Methodology will give us a step-by-step
description of exactly how the study was
carried out.
Study design:
Different research questions require different
study designs.
For example the best design for studies
evaluating the effectiveness of an intervention
or treatment is a Randomized controlled trial

..Methodology Section
Where the study done

Data collection:
- From whom data was collected
- Primary or secondary data
- How data collected
- How good the data
How the study control for differences between
the groups being compared

..Methodology Section
Sample: Is there any selection bias?
In an intervention study, how were subjects
recruited and assigned to groups?
Studies are subject to bias and also
confounders.
It is important that researchers take steps to
minimize this bias and confounders.
For example use of a control group,
randomization, blinding
Sample size to produce significant results

..Methodology Section
Methods to measure the outcome factors and
study factors
- Whether there was any error in measurement
- Consider the sensitivity and specificity of each
detection method
How good the measures:
- Validity
- Whether the measures well established
in either prior research or through pilot
testing by the researcher

..Methodology Section
What is statistical method used?
- Statistical method used should be appropriate and
be able to identify and characterize the effect of
potential factors
- The effects of confounder(s) should be controlled.
- Probability : Find the P values, and confidence
interval (CI)
- Significances
- significance or not significance means
- Bias play devils advocate

..Methodology Section
Whether important potential confounders were
considered
For example:
- You found a significant association between smoking
and arteriosclerosis risk
- You want to compare the proportion of cigarette
smokers between arteriosclerosis cases and controls
- You found more smokers in arteriosclerosis cases,
and more males in arteriosclerosis cases than controls
- Since a proportion of smokers in males usually higher
than females, your finding might be distorted because of
the different sex distribution
After adjusting the effect of sex distribution, there was no difference in
the association between smoking and GC risk.

Discussion and Conclusion Section

Must be detailed and precise in discussing the
findings
How well are the results related to other
research on the same topic?
- Is there a review of how these results
compare or contrast with prior research?
- If this report found something different from
previous research, then its important to
question on appraising the reliability of the
findings.

..Discussion and Conclusion Section

What are the authors conclusions?
- Whether the results reported in the conclusions
consistent with what is reported in the tables
- Whether the interpretation consistent with
what the actual findings

This section will also explain:

- The limitations of the research
- Possible implications or application of the
findings into daily clinical practice or research

Example of Guides in CA
Are the results of the study valid?
> Primary:
- Whether the exposure status clearly defined and
measured
- Whether the outcome of interest measured
- What was the follow up time and was it
adequate to measure the outcome of interest?
- Whether the outcome measured in the same
way for both exposed & not exposed

> Secondary:
- How much was the attrition? (Loss to follow up)
- Are there confounders that the investigator did
not address?

..Example of Guides in CA
What were the results?
- How strong is the association between exposure
and outcome?
- How precise is the estimates of the risk ?
(Confidence Interval and p value for statistical
test of significance)

Will the results help me?

- Whether the results applicable to my
population?
- What is the magnitude of the risk ?
- Should I attempt to stop the exposure?