MSc Aesthetic Medicine

Dissertation Guidelines
Guidelines for dissertation
The MSc dissertation aims to demonstrate the candidates ability to critically
appraise a specific area of Aesthetic Medicine, acquisition of in-depth knowledge
and understanding of the field, ability to provide a rational and systematic
discussion of the subject, as well as attainment of academic writing skills.
Candidates should aim to achieve a standard that is suitable for publication in an
international peer-reviewed aesthetic journal such as Journal of Cosmetic
Dermatology, Aesthetic Surgery Journal or the Journal of Plastic Reconstructive
& Aesthetic Surgery (JPRAS).
First of all, identify an aesthetic related topic of interest (a list of topics is
provided below). You may include clinical findings, images or cases to illustrate
your points of discussion. If you are unsure of a topic, you may want to discuss
potential areas of interest with the Course Lead.
Once you have your topic and title, you should then perform a thorough up-to-
date extensive literature search* to provide the background to your dissertation.
It also ensures that your idea is novel and has the potential to provide new
insight for readers of a peer-reviewed journal. Once you have read the literature,
you can then formulate your own research questions that would inform your
practice and perhaps contribute to the field of interest.
The literature search would serve as your introduction and most of your
discussion chapter and the methodology and result chapters would be based on
three research questions identified through your initial review and answered
according to the Best Evidence Topics (BETs) methodology introduced to you
through the course (http://www.bestbets.org).
The dissertation is then written accordingly in your own words. You are
advised to arrange your dissertation according to the following structure and
format. Alternatively, you may follow the structural layout similar to a peer
reviewed paper. A list of topics and examples of how to formulate BETs
research questions is provided below.

Appendix 1 provide guidelines for how to perform a systematic literature

review.

Ali M Ghanem MD PhD

Dissertation Topics

Scar management Optimizing the aesthetic outcome

Photoaging An evidence based treatment approach
Toxins in aesthetic medicine Innovations and new trends
Toxins in aesthetic medicine Treatment of a particular indication
Soft tissue augmentation in aesthetic medicine The upper face
Soft tissue augmentation in aesthetic medicine The middle face
Soft tissue augmentation in aesthetic medicine The lower face
Soft tissue augmentation in aesthetic medicine The periocular area
Soft tissue augmentation in aesthetic medicine Risk reduction strategies
Soft tissue augmentation in aesthetic medicine Any other aspect
Peels in aesthetic medicine Evaluation of efficacy and safety
Peels in aesthetic medicine Office based choices
Hair loss Evidence based treatment modalities
Hair transplantation Evaluation of technical details
Excess hair treatment A critical evaluation of treatment modalities
Laser treatment for vascularized lesions An evidence based protocol
Laser skin rejuvenation An evidence based treatment strategy
Evidence for aesthetic medicine intervention for better quality of life
Psychological consideration in aesthetic medicine When the practitioner says no
Medicolegal considertations in aesthetic medicine UK, USA and Australian law
Aesthetic medicine and the law Optimum regulation strategies (UK law)
Outcome measures in aesthetic medicine
Cosmeceuticals Evidence based treatment approach
Cosmeceuticals Critical analysis of current popularized strategies
Sclerotherapy Evaluation of current modalities and alternatives
Optimum skin resurfacing Microneedling , laser or chemical peels
Mesotherapy Evaluation of current treatment modalities
Thread lifts A critical analysis of current treatment modalities
Collagen stimulation agents A critical analysis of current treatment modalities
Platelet rich plasma in aesthetic medicine Evidence based treatment approach
Microlipofilling in aesthetic medicine Evaluation of office based procedures
Non-invasive lipolysis technology in aesthetic medicine A critical analysis

Ali M Ghanem MD PhD

Example on how to formulate BETs research questions

Dissertation topic:

Collagen stimulation agents A critical analysis of current treatment modalities

Introduction and basis for discussion

Conduct an (abstract only) systematic review of the literature on collagen,
collagen synthesis and collagen stimulation products
Identify 7 potential modalities
-
-
-
-
-
-
-

Poly L Lactic Acid (PLLA)

Calcium Hydroxylapatite (CaHA)
Polycarpolactone (PCL) and Carboxymethylcellulose (CMC)
Threads (eg polydioxanone (PDO))
Radiofrequency combined with/without magnetic energy devices
Fractioned laser
Micro-needling

Note the first three are injectable whereby the fourth is thread, the fifth/sixth are
energy (non-invasive) and laser device and the seventh is mechanical
stimulation.
We try to evaluate like with like through a systematic BETs approach
through a well-thought and practically orientated 3part questions
(target population / indication intervention / outcome).

Given the wider range of applications and interventions as well as outcome

measures the 3-Part Best Evidence Topic (BET) Questions would present a
huge number of possible evaluations and comparisons.
For the purpose of the dissertation, it is necessary to choose an appropriate
target population, indications (commonly presenting conditions such as facial
aging or hands etc), interventions (popular options such as injectable or
technology available devices) and outcome (standardized tools such as
PROMs, doctor evaluation, 3-D scanning, biopsy, complications, costs etc).
For example, one might want to evaluate the efficacy and safety of injectable
agents vs threads as soft tissue augmentation and collagen stimulation tools
in facial rejuvenation.
The thesis would focus on facial rejuvenation (population), and compare all
three injectables identified in the literature search above to available threads
and discuss this from efficacy, safety and cost effectiveness point of view.
Appendix 1

Ali M Ghanem MD PhD

1] Efficacy: look up all papers published and evaluate them according to BETs
critical appraisal (CA) sheets and compare their important messages
(indications, number of subjects, protocol, outcome measure etc) in tables.

2] Safety: look up all papers published on complications and evaluate them

according to severity, frequency, predictive factors (the CA sheets are helpful
here too as they will include analysis of follow up duration and if all patients are
accounted for etc).
3] Cost analyses (US & UK): Try to obtain pricing of products from websites,
aesthetic pharmacies and practitioners then use the economic study CA sheet as
it would give you the principles of how to conduct a good cost analysis
evaluation.
Report these in tables and discuss them according to BETs methodology or
other method to report the outcome of systematic reviews in the literature**.

Other students may choose other angles to discuss in the same topic and
accordingly other 3-part questions or dissertations on the same topic but
different focus might be;
Question
1

Target population/
Indication
Facial rejuvenation

Hand rejuvenation

Dcolletage

Intervention

Outcome

PLLA/CaHA/PCL Basic science

vs
Treatment burden
PDO threads
Efficacy (objective* vs
subjective#)
*3D scan/photo/biopsy
#Doctor evaluation
#Patient evaluation
Durability
Complications
Cost analysis
PLLA/CaHA/PCL Any of above outcomes
vs
Radiofrequency/
Laser
Laser vs micro-
Any of above outcomes
needling

Use BETs CA sheets (Appendix 2 A & B): include the ones relevant to

cohort and case control studies as most of the aesthetic literature would
fall under these categories. More sheets are available on
http://bestbets.org/links/BET-CA-worksheets.php)
Use BETs CA sheet (Appendix 2 C)
** Appendix 3: 5 steps to systematic review.
Ali M Ghanem MD PhD

Dissertation Structure
1. Title page This should include the title of your dissertation, your name
and institution, the year of completion
2. Abstract page
3. Acknowledgements
4. Table of contents
5. List of figures
6. List of tables
7. List of abbreviations
8. Introduction & objectives the aims and objectives of your dissertation
should be at the end of your introduction chapter.
9. Methods & materials
10. Results & discussions if you wish, you may have your discussion in
a separate chapter.
11. Conclusion
12. References
13. Appendices (if applicable)
Each of the above headings should begin on a new page.

Dissertation Format
1. The Dissertations for the MSc courses must be at least 15000 words (13000
17000) excluding references.
2. Use a standard, easy-to-read word processor font such as Times or Arial.
3. Font size 11 or 12.
4. All pages should be numbered in the bottom right corner or bottom centre.
5. Paragraphs should be made clearly visible by leaving an additional blank line
between paragraphs.
6. Line spacing: 1.5 or double-spaced.
7. Page alignment: justified
8. Images and figures: please ensure that the images do not contain patient
identifiable information and that you have patients consent to the images.
9. References: We recommend that you follow the style for references according
to the format of PRS or JPRAS. Please be accurate and consistent with your
format.

Dissertation Marking Scheme

The dissertation will be double marked according to the marking scheme
provided below in appendix 4.
Appendix 4

Ali M Ghanem MD PhD

Appendix 1
QMUL - MSc Aesthetic Medicine
Guidelines for Dissertation - Systematic Reviews
Adapted from Mary Simons & Karen Marks of the Library Services, Macquarie University Sydney, Australia,
A systematic review is an overview of primary studies that used explicit and reproducible methods.
Systematic reviews apply scientific strategies that limit bias by the systematic assembly, critical
appraisal and synthesis of all relevant studies on a specific topic
A meta-analysis is a mathematical synthesis of the results of two or more primary studies that
addressed the same hypothesis in the same way
Systematic reviews as well as meta-analyses of appropriate studies can be the best form of evidence
available for health care practitioners

SYSTEMATIC REVIEW STEPS:

1. Research Question
2. Research Protocol
3. Literature Search
4. Data Extraction
5. Quality Appraisal
6. Data Analysis and Results
7. Interpretation of Results

1. Research Question
The first step in performing a systematic review is to formulate a primary research question as part of
the research protocol. Appropriate questions to be addressed include: (1) phenomena associated
with disease or interventions, (2) disease or condition frequency, (3) diagnostic accuracy, (4) disease
etiology and/or risk factors, (5) prognosis, and (6) intervention effects.
The aims of a systematic review can be varied and include: (1) clarifying the relative strengths and
weaknesses of the literature on the question, (2) summarizing a large amount of literature, (3)
resolving literature conflicts, (4) evaluating the need for a large clinical trial, (5) avoiding a redundant
unnecessary trial, (6) increasing the statistical power of smaller studies, (7) improving the precision or
identify a smaller treatment effect, and (8) improving the generalizability of treatment outcomes.
2. Research Protocol
Once the research question is formulated, the research protocol is developed. The goal of developing
a research protocol is to develop formulation of the questions and methods of the review before
retrieving the literature. The methods for literature searching, screening, data extraction, and analysis
should be contained in a written document to minimize bias before starting the literature search.
3. Literature Search
Sources to search for studies for systematic reviews:
Cochrane Central Register of Controlled Trials (via Cochrane Library)
Cochrane Database of Systematic Reviews (via Cochrane Library)
Database of Abstracts of Reviews of Effects (DARE) database ie: other reviews (via
Cochrane Library)
Medline and Embase (Library does not subscribe to Embase)
Other databases, eg CINAHL, Scopus, PsycINFO
Journals
Conference proceedings

Bibliographies and references listed in primary sources

Unpublished and ongoing studies, including known experts in the field (seek by personal
communication)
Raw data from published trials (seek by personal communication)
Foreign language literature (do not limit searches to English)
Grey literature (theses, internal reports, non peer-reviewed journals, pharmaceutical industry
files)
PubMed Clinical Queries (www.ncbi.nlm.nih.gov/entrez/query ) includes search filters offering
4 categories of aetiology, prognosis, treatment, diagnosis, and choice of emphasizing
sensitivity or specificity.
Cochrane Handbook appendix contains a list of approximately 30 clinical trials registries for
finding unpublished primary studies (See Reference List for Library call number)
A minimum of two reviewers performs a first-stage screening of titles and abstracts based on the
research question and its study design, population, intervention, and outcome to be studied. Based on
the initial screening, selected full-text articles are obtained for the second-stage screening. Using the
full text a second-stage screening is performed by at least two reviewers. The studies selected are
then submitted for data extraction.
4. Data Extraction
A standardized form (paper or electronic) assists in the task of data extraction. For example:
Sample Data Extraction Form Items:
Referenceincluding journal, title, author, volume in page numbers
Objectivethe study objective as stated by the authors
Study designtype of trial
Populationdemographics of the participants in the study
Interventiondescription of the intervention
Controldescription of the control group or alternative intervention
Outcomeresults of the intervention and how measured including statistics used
Commentsdetails regarding the study quality
5. Quality Appraisal
A checklist to assess for biases is important: several quality scales and checklists have been
developed for this.
Each trial should be evaluated in terms of its:
Methodological qualitythe extent to which the design and conduct are likely to have
prevented systematic errors (bias)
Precisiona measure of the likelihood of random errors (usually depicted as the width of the
confidence interval around the result)
External validitythe extent to which the results are generalisable or applicable to a particular
target population
6. Data Analysis and Results
After including and excluding studies based on the quality appraisal, data analysis and results of the
studies should be undertaken. The initial step for this process involves a simple descriptive evaluation
of each study, commonly presented in tabular format. Tables should include the population under
study, the interventions, and outcomes.
7. Interpretation of Results
Most of this information can be presented in the data analysis and results table in the manuscript. The
strengths and weaknesses of the included studies must be discussed. Conclusions should be based
on the best available scientific evidence. Recommendations regarding future studies can also be
made.
PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses
(www.prisma-statement.org). It is an evidence-based minimum set of items for reporting in systematic

reviews and meta-analyses. PRISMA should be a helpful resource to improve reporting of systematic
reviews and meta-analyses.

References used for this summary (all are available from the Library):
Wright, R. W., R. A. Brand, et al. (2007). "How to Write a Systematic Review." Clinical Orthopaedics
and Related Research 455: 23-29
Systematic reviews in health care a practical guide / Paul Glasziou ... [et al.]. Cambridge ; New York :
Cambridge University Press, 2001. eBook: Available via Library catalogue
Greenhalgh, T. (1997). "How to read a paper: Papers that summarise other papers (systematic
reviews and meta-analyses)." BMJ 315(7109): 672-675.
Oxman, A. D. (1994). "Systematic Reviews: Checklists for review articles." BMJ 309(6955): 648-651.
Systematic reviews and meta-analysis /Julia H. Littell, Jacqueline Corcoran, Vijayan Pillai.
Series Title: Pocket guides to social work research methods Oxford ;New York : Oxford University
Press. Available in Library: H62 .L497 2008
Harvey, R. (2007). "Systematic review as a primer rather than endpoint for clinical research: The
training perspective." Otolaryngology-Head and Neck Surgery 137(4): S66-S68.
Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for
Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(6): e1000097.
doi:10.1371/journal.pmed1000097
Cochrane handbook for systematic reviews of interventions / edited by Julian P.T. Higgins Chichester,
England ; Hoboken, NJ : Wiley-Blackwell and Sally Green., c2008. Available in Library: R723.7 .C63
2008

Appendix 2 A

CASE-CONTROL (INCLUDING HARM) CHECKLIST

How do you rate this paper?
1.0

OBJECTIVES AND HYPOTHESES

1.1

2.0

1 2 3 4 5 6 7 8 9 10

Are the objectives of the study clearly

stated?

DESIGN
2.1

Is the study design suitable for the

objectives?

2.2

Who/what was studied?

2.3

Was this the right sample to answer the

objectives?

2.4

Did this include a clearly identified

comparison group, identical in all
aspects other than the exposure?

2.5

Did the exposure precede outcome?

2.6

Is the study large enough to achieve its

objectives? Have sample size estimates
been performed?

2.7

Were all subjects accounted for?

2.8

Were all appropriate outcomes

considered?

2.9

Has ethical approval been obtained if

appropriate?

3.0

4.0

MEASUREMENT AND OBSERVATION

3.1

Is it clear what was measured, how it was

measured and what the outcomes were?

3.2

Were the exposures to the agent and

outcomes measured in the same way
in all of the groups compared?

3.3

Were the assessments of exposure

blinded to outcome?

3.4

Was follow up sufficiently long and

complete?

3.5

Are the measurements valid?

3.6

Are the measurements reliable?

3.7

Are the measurements reproducible?

PRESENTATION OF RESULTS
4.1

Are the basic data adequately described?

4.2

Are the results presented clearly,

objectively and in sufficient detail to
enable readers to make their own
judgement?
Can you construct a 2x2 table of
exposure and outcome?

4.3

4.4

Was there a dose response effect?

4.5

Are the results internally consistent, i.e.

do the numbers add up properly?

5.0

6.0

7.0

ANALYSIS
5.1

Are the data suitable for analysis?

5.2

Are the methods appropriate to the data?

5.3

Are any statistics correctly performed and

interpreted?

5.4

Are relative risks or odds presented

with confidence intervals?

DISCUSSION
6.1

Are the results discussed in relation to

existing knowledge on the subject and
study objectives?

6.2

Is a causal relationship between

exposure and outcome suggested?

6.3

If so, is this causal relationship

justified?

6.4

Is the discussion biased?

INTERPRETATION
7.1

Are the authors conclusions justified by

the data?

7.2

What level of evidence has this paper

presented? (using CEBM levels)

7.3

Does this paper help me answer my

problem?

How do you rate this paper now?

1 2 3 4 5 6 7 8 9 10

In addition, answer the following questions with regards to local practice.

8.0

IMPLEMENTATION
8.1

Can any necessary change be

implemented in practice?

8.2

What aids to implementation exist?

8.3

What barriers to implementation exist?

Appendix 2 B

COHORT CHECKLIST
How do you rate this paper?
1.0

OBJECTIVES AND HYPOTHESES

1.1

2.0

1 2 3 4 5 6 7 8 9 10

Are the objectives of the study clearly

stated?

DESIGN
2.1

Is the study design suitable for the

objectives?

2.2

Who/what was studied?

2.3

Was a control group used if

appropriate?

2.4

Were outcomes defined at the start of

the study?

2.5

Was this the right sample to answer the

objectives?

2.6

Is the study large enough to achieve its

objectives? Have sample size estimates
been performed?

2.7

Were all subjects accounted for?

2.8

Were all appropriate outcomes

considered?

2.9

Has ethical approval been obtained if

appropriate?

3.0

4.0

MEASUREMENT AND OBSERVATION

3.1

Is it clear what was measured, how it was

measured and what the outcomes were?

3.2

Was the assessment of outcomes

blinded?

3.3

Was follow up sufficiently long and

complete?

3.4

Are the measurements valid?

3.5

Are the measurements reliable?

3.6

Are the measurements reproducible?

PRESENTATION OF RESULTS
4.1

Are the basic data adequately described?

4.2

Are the results presented clearly,

objectively and in sufficient detail to
enable readers to make their own
judgement?
How large are the effects within a
specified time?

4.3

4.4

Are the results internally consistent, i.e.

do the numbers add up properly?

5.0

6.0

7.0

ANALYSIS
5.1

Are the data suitable for analysis?

5.2

Are the methods appropriate to the data?

5.3

Are any statistics correctly performed and

interpreted?

DISCUSSION
6.1

Are the results discussed in relation to

existing knowledge on the subject and
study objectives?

6.2

Is the discussion biased?

INTERPRETATION
7.1

Are the authors conclusions justified by

the data?

7.2

What level of evidence has this paper

presented? (using CEBM levels)

7.3

Does this paper help me answer my

problem?

How do you rate this paper now?

1 2 3 4 5 6 7 8 9 10

In addition, answer the following questions with regards to local practice.

8.0

Implementation
8.1

Can any necessary change be

implemented in practice?

8.2

What aids to implementation exist?

8.3

What barriers to implementation exist

8.4

Are the study patients similar to your

own?

8.5

Does the paper give any conclusions

that will affect what you will offer or
tell your patient?

Appendix 2 C

ECONOMIC CHECKLIST
How do you rate this paper?
1.0

OBJECTIVES AND HYPOTHESES

1.1

2.0

1 2 3 4 5 6 7 8 9 10

Are the objectives of the study clearly

stated?

DESIGN
2.1

Is the study design suitable for the

objectives?

2.2

Who/what was studied?

2.3

Was this the right sample to answer the

objectives?

2.4

Is the study large enough to achieve its

objectives? Have sample size estimates
been performed?

2.5

Were all subjects accounted for?

2.6

Were all appropriate outcomes

considered?

2.7

Has ethical approval been obtained if

appropriate?

2.8

Does this economic analysis cite valid

evidence on the clinical efficacy of the
alternative?

2.9

From whos perspective were costs

measured?

2.10 Are all costs and effects identified?

3.0

4.0

MEASUREMENT AND OBSERVATION

3.1

Is it clear what was measured, how it was

measured and what the outcomes were?

3.2

Were consequences and costs

measured accurately in appropriate
units?

3.3

Were opportunity costs measured?

3.4

Are the measurements valid?

3.5

Are the measurements reliable?

3.6

Are the measurements reproducible?

PRESENTATION OF RESULTS
4.1 Are the basic data adequately described?

4.2

4.3

5.0

Are the results presented clearly,

objectively and in sufficient detail to
enable readers to make their own
judgement?
Are the results internally consistent, i.e.
do the numbers add up properly?

ANALYSIS
5.1

Are the data suitable for analysis?

5.2

Are the methods appropriate to the data?

5.3

Are any statistics correctly performed and

interpreted?

6.0

7.0

DISCUSSION
6.1

Are the results discussed in relation to

existing knowledge on the subject and
study objectives?

6.2

Is the discussion biased?

6.3

Has a sensitivity analysis been

performed (was appropriate allowance
made for uncertainties)?

INTERPRETATION
7.1

Are the authors conclusions justified by

the data?

7.2

What level of evidence has this paper

presented? (using CEBM levels)

7.3

Does this paper help me answer my

problem?

How do you rate this paper now?

1 2 3 4 5 6 7 8 9 10

In addition, answer the following questions with regards to local practice.

8.0

IMPLEMENTATION
8.1

Can any necessary change be

implemented in practice?

8.2

What aids to implementation exist?

8.3

What barriers to implementation exist?

8.4

Do the costs apply in my practice?

8.5

Are the treatments likely to be

effective in my setting?

Appendix 3
JOURNAL

OF

THE ROYAL

SOCIETY OF

MEDICINE

Volume

96

March

2003

Five steps to conducting a systematic review

Khalid S Khan MB MSc

Regina Kunz MD MSc 1

Jos Kleijnen MD PhD 2

Gerd Antes PhD 3

J R Soc Med 2003;96:118121

Systematic reviews and meta-analyses are a key element of

evidence-based healthcare, yet they remain in some ways
mysterious. Why did the authors select certain studies and
reject others? What did they do to pool results? How did a
bunch of insignificant findings suddenly become significant?
This paper, along with a book1 that goes into more detail,
demystifies these and other related intrigues.
A review earns the adjective systematic if it is based on a
clearly formulated question, identifies relevant studies,
appraises their quality and summarizes the evidence by use
of explicit methodology. It is the explicit and systematic
approach that distinguishes systematic reviews from
traditional reviews and commentaries. Whenever we use
the term review in this paper it will mean a systematic review.
Reviews should never be done in any other way.
In this paper we provide a step-by-step explanation
there are just five stepsof the methods behind reviewing,
and the quality elements inherent in each step (Box 1). For
purposes of illustration we use a published review
concerning the safety of public water fluoridation, but we
must emphasize that our subject is review methodology, not
fluoridation.
EXAMPLE: SAFETY OF PUBLIC WATER
FLUORIDATION

You are a public health professional in a locality that has

public water fluoridation. For many years, your colleagues
and you have believed that it improves dental health.
Recently there has been pressure from various interest
groups to consider the safety of this public health intervention
because they fear that it is causing cancer. Public health
decisions have been based on professional judgment and
practical feasibility without explicit consideration of the
scientific evidence. (This was yesterday; today the evidence is
available in a York review2,3, identifiable on MEDLINE
through the freely accessible PubMed clinical queries
interface [http://www.ncbi.nlm.nib.gov/entrez/query/
static/clinical.html], under systematic reviews.)
Education Resource Centre, Birmingham Womens Hospital, Birmingham B15
2TG, UK; 1German Cochrane Centre, Freiburg and Department of Nephrology,
Charite, Berlin, Germany; 2Centre for Reviews and Dissemination, York, UK;
3

118

German Cochrane Centre, Freiburg, Germany

Correspondence to: Khalid S Khan

E-mail: khalid.khan@bham-womens.thenhs.com

STEP 1: FRAMING THE QUESTION

The research question may initially be stated as a query in

free form but reviewers prefer to pose it in a structured and
explicit way. The relations between various components of
Box 1 The steps in a systematic review
Step 1: Framing questions for a review
The problems to be addressed by the review should be
specified in the form of clear, unambiguous and structured
questions before beginning the review work. Once the review
questions have been set, modifications to the protocol should
be allowed only if alternative ways of defining the populations,
interventions, outcomes or study designs become apparent
Step 2: Identifying relevant work
The search for studies should be extensive. Multiple resources
(both computerized and printed) should be searched without
language restrictions. The study selection criteria should flow
directly from the review questions and be specified a priori.
Reasons for inclusion and exclusion should be recorded
Step 3: Assessing the quality of studies
Study quality assessment is relevant to every step of a review.
Question formulation (Step 1) and study selection criteria (Step
2) should describe the minimum acceptable level of design.
Selected studies should be subjected to a more refined quality
assessment by use of general critical appraisal guides and
design-based quality checklists (Step 3). These detailed
quality assessments will be used for exploring heterogeneity
and informing decisions regarding suitability of meta-analysis
(Step 4). In addition they help in assessing the strength of
inferences and making recommendations for future research
(Step 5)
Step 4: Summarizing the evidence
Data synthesis consists of tabulation of study characteristics,
quality and effects as well as use of statistical methods for
exploring differences between studies and combining their
effects (meta-analysis). Exploration of heterogeneity and its
sources should be planned in advance (Step 3). If an overall
meta-analysis cannot be done, subgroup meta-analysis may
be feasible
Step 5: Interpreting the findings
The issues highlighted in each of the four steps above should
be met. The risk of publication bias and related biases should
be explored. Exploration for heterogeneity should help
determine whether the overall summary can be trusted, and, if
not, the effects observed in high-quality studies should be
used for generating inferences. Any recommendations should
be graded by reference to the strengths and weaknesses of
the evidence

JOURNAL

OF

THE

ROYAL SOCIETY

OF

MEDICINE

Volume 96

March

2003

Figure 1 Structured questions for systematic reviews and relations between question components in a comparative study

the question and the structure of the research design are

shown in Figure 1. This paper focuses only on the question
of safety related to the outcomes described below.
Free-form question

Is it safe to provide population-wide drinking water

fluoridation to prevent caries?
Structured question

.
.
.

The populationsPopulations receiving drinking water

sourced through a public water supply
The interventions or exposuresFluoridation of drinking
water (natural or artificial) compared with nonfluoridated water
The outcomesCancer is the main outcome of interest
for the debate in your health authority

The study designsComparative studies of any design

examining the harmful outcomes in at least two
population groups, one with fluoridated drinking water
and the other without. Harmful outcomes can be rare
and they may develop over a long time. There are
considerable difficulties in designing and conducting
safety studies to capture these outcomes, since a large
number of people need to be observed over a long
period. These circumstances demand observational, not
randomized studies. With this background, systematic
reviews on safety have to include evidence from studies
with a range of designs.

STEP 2: IDENTIFYING RELEVANT PUBLICATIONS

To capture as many relevant citations as possible, a wide

range of medical, environmental and scientific databases
were searched to identify primary studies of the effects of

119

JOURNAL

OF

THE ROYAL

SOCIETY OF

MEDICINE

Volume

96

March

2003

Table 1 Description of quality assessment of studies on safety of public water fluoridation

Quality categories

High

Moderate

Low

Prospective design
Ascertainment of exposure

Prospective
Study began within 1 year of
fluoridation
Follow-up for at least 5 years
and blind assessment
Adjustment for at least three
confounding factors (or use
of randomization)

Prospective
Study began within 3 years
of fluoridation
Long follow-up and blind
assessment
Adjustment for at least one
confounding factor

Prospective or retrospective
Study began 43 years after
fluoridation
Short follow-up and unblinded
assessment
No adjustment for confounding
factors

Ascertainment of outcome
Control for confounding

water fluoridation. The electronic searches were supplemented by hand searching of Index Medicus and Excerpta
Medica back to 1945. Furthermore, various internet engines
were searched for web pages that might provide references.
This effort resulted in 3246 citations from which relevant
studies were selected for the review. Their potential
relevance was examined, and 2511 citations were excluded
as irrelevant. The full papers of the remaining 735 citations
were assessed to select those primary studies in man that
directly related to fluoride in drinking water supplies,
comparing at least two groups. These criteria excluded 481
studies and left 254 in the review. They came from thirty
countries, published in fourteen languages between 1939
and 2000. Of these studies 175 were relevant to the
question of safety, of which 26 used cancer as an outcome.
STEP 3: ASSESSING STUDY QUALITY

Design threshold for study selection

Adequate study design as a marker of quality, is listed as an

inclusion criterion in Box 1. This approach is most
applicable when the main source of evidence is randomized
studies. However, randomized studies are almost impossible to conduct at community level for a public health
intervention such as water fluoridation. Thus, systematic
reviews assessing the safety of such interventions have to
include evidence from a broader range of study designs.
Consideration of the type and amount of research likely to
be available led to inclusion of comparative studies of any
design. In this way, selected studies provided information
about the harmful effects of exposure to fluoridated water
compared with non-exposure.
Quality assessment of safety studies

120

After studies of an acceptable design have been selected,

their in-depth assessment for the risk of various biases
allows us to gauge the quality of the evidence in a more
refined way. Biases either exaggerate or underestimate the
true effect of an exposure. The objective of the included
studies was to compare groups exposed to fluoridated

drinking water and those without such exposure for rates of

undesirable outcomes, without bias. Safety studies should
ascertain exposures and outcomes in such a way that the risk
of misclassification is minimized. The exposure is likely to
be more accurately ascertained if the study was prospective
rather than retrospective and if it was started soon after
water fluoridation rather than later. The outcomes of those
developing cancer (and remaining free of cancer) are likely
to be more accurately ascertained if the follow-up was long
and if the assessment was blind to exposure status.
When examining how the effect of exposure on
outcome was established, reviewers assessed whether the
comparison groups were similar in all respects other than
their exposure to fluoridated water. This is because the
other differences may be related to the outcomes of interest
independent of the drinking-water fluoridation, and this
would bias the comparison. For example, if the people
exposed to fluoridated water had other risk factors that
made them more prone to have cancer, the apparent
association between exposure and outcome might be
explained by the more frequent occurrence of these factors
among the exposed group. The technical word for such
defects is confounding. In a randomized study, confounding
factors are expected to be roughly equally distributed
between groups. In observational studies their distribution
may be unequal. Primary researchers can statistically adjust
for these differences, when estimating the effect of
exposure on outcomes, by use of multivariable modelling.
Put simply, use of a prospective design, robust
ascertainment of exposure and outcomes, and control for
confounding are the generic issues one would look for in
quality assessment of studies on safety. Consequently,
studies may range from satisfactorily meeting quality
criteria, to having some deficiencies, to not meeting the
criteria at all, and they can be assigned to one of three
prespecified quality categories as shown in Table 1. A
quality hierarchy can then be developed, based on the
degree to which studies comply with the criteria. None of
the studies on cancer were in the high-quality category, but
this was because randomized studies were non-existent and

JOURNAL

OF

THE

ROYAL SOCIETY

control for confounding was not always ideal in the

observational studies. There were 8 studies of moderate
quality and 18 of low quality.
STEP 4: SUMMARIZING THE EVIDENCE

To summarize the evidence from studies of variable design

and quality is not easy. The original review3 provides details
of how the differences between study results were
investigated and how they were summarized (with or
without meta-analysis). This paper restricts itself to
summarizing the findings narratively. The association
between exposure to fluoridated water and cancer in
general was examined in 26 studies. Of these, 10 examined
all-cause cancer incidence or mortality, in 22 analyses. Of
these, 11 analyses found a negative association (fewer
cancers due to exposure), 9 found a positive one and 2
found no association. Only 2 studies reported statistically
significant differences. Thus no clear association between
water fluoridation and increased cancer incidence or
mortality was apparent. Bone/joint and thyroid cancers
were of particular concern because of fluoride uptake by
these organs. Neither the 6 studies of osteosarcoma nor the
2 studies of thyroid cancer and water fluoridation revealed
significant differences. Overall no association was detected
between water fluoridation and mortality from any cancer.
These findings were also borne out in the moderate-quality
subgroup of studies.
STEP 5: INTERPRETING THE FINDINGS

In the fluoridation example, the focus was on the safety of a

community-based public health intervention. The generally
low quality of available studies means that the results must
be interpreted with caution. However, the elaborate efforts
in searching an unusually large number of databases provide
some safeguard against missing relevant studies. Thus the
evidence summarized in this review is likely to be as good as
it will get in the foreseeable future. Cancer was the harmful
outcome of most interest in this instance. No association
was found between exposure to fluoridated water and
specific cancers or all cancers. The interpretation of the
results may be generally limited because of the low quality
of studies, but the findings for the cancer outcomes are
supported by the moderate-quality studies.
RESOLUTION

After having spent some time reading and understanding the

review, you are impressed by the sheer amount of published

OF

MEDICINE

Volume 96

March

2003

work relevant to the question of safety. However, you are

somewhat disappointed by the poor quality of the primary
studies. Of course, examination of safety only makes sense
in a context where the intervention has some beneficial
effect. Benefit and harm have to be compared to provide the
basis for decision making. On the issue of the beneficial
effect of public water fluoridation, the review3 reassures
you that the health authority was correct in judging that
fluoridation of drinking water prevents caries. From the
review you also discovered that dental fluorosis (mottled
teeth) was related to concentration of fluoride. When the
interest groups raise the issue of safety again, you will be
able to declare that there is no evidence to link cancer with
drinking-water fluoridation; however, you will have to
come clean about the risk of dental fluorosis, which appears
to be dose dependent, and you may want to measure the
fluoride concentration in the water supply and share this
information with the interest groups.
The ability to quantify the safety concerns of your
population through a review, albeit from studies of
moderate to low quality, allows your health authority, the
politicians and the public to consider the balance between
beneficial and harmful effects of water fluoridation. Those
who see the prevention of caries as of primary importance
will favour fluoridation. Others, worried about the
disfigurement of mottled teeth, may prefer other means
of fluoride administration or even occasional treatment for
dental caries. Whatever the opinions on this matter, you are
able to reassure all parties that there is no evidence that
fluoridation of drinking water increases the risk of cancer.
CONCLUSION

With increasing focus on generating guidance and

recommendations for practice through systematic reviews,
healthcare professionals need to understand the principles of
preparing such reviews. Here we have provided a brief
step-by-step explanation of the principles. Our book1
describes them in detail.
REFERENCES

1 Khan KS, Kunz R, Kleijnen J, Antes G. Systematic Reviews to Support

Evidence-Based Medicine. How to Review and Apply findings of Health Care
Research. London: RSM Press, 2003
2 McDonagh M, Whiting P, Bradley M, et al. Systematic review of water
fluoridation. BMJ 2000;321:8559
3 NHS Centre for Reviews and Dissemination (CRD). A systematic
review of water fluoridation. CRD Report 18. York: University of York,
2000 [http://www.york.ac.uk/inst/crd/fluorid.htm]

121

Appendix 4

Marking Scheme
Grade A (70%+)
Original approach
Shows critical insight
Applies
models/tests/techniques
successfully

Grade B (60 - 70%)

Appropriate application of
models/tests/techniques

Grade C (50 60%)

Partially successful at
application of
models/tests/techniques

Contents 50%

Taught course material as a

starting point
Advanced comprehension
Awareness of complex issues
Extensive literature review

Taught course material applied

Adequate comprehension
Adequate awareness of issues
Comprehensive literature
review

Taught course material used

Evidence of comprehension
Some gaps
Covers the literature

Analysis 10%

Original analysis
Compares data
Analytical insight

Appropriate analysis

Attempts analysis
Some misconception

Conclusion 10%

Relevant, valid, appropriate

Critically evaluated
Offers a realistic vision or
recommendation for the future

Relevant, valid, appropriate

Some relevant and valid

conclusions

References 10%

All work referenced

Accurate references

Present but incomplete

Accurate references

Some references

Presentation and style

10%

Innovative
Well organised and structured
Consistent format
Captures and maintains readers
attention
Minimal or no error

Appropriate
Organised and structured
A few inconsistencies with
formatting
Making sense for readers

Appropriate
Some structure
Numerous inconsistencies with
formatting

Approach 10%

Finally, YOU ARE REMINDED THAT PLASGIARISM IS STRICTLY FORBIDDEN! Please refer to the section
Plagiarism and TURNITIN prior to your submission. You are advised to begin your work on the dissertation
well in advance to ensure your dissertation is completed and submitted before the deadline.

Good Luck!

The Blizard Institute, 4 Newark Street, London, E1 2AT