STUDY DESIGNS AND

ERRORS
Dr ASHA KUMARI
SR, DEPT OF
BIOCHEMISTRY

Increasing and
Measuring Impact
Sandy De Groote
Associate Professor
Scholarly Communication Librarian
University Library

Aims and Hypothesis

Aim: The purpose of the study
Hypothesis: A clear unambiguous statement that
predicts the outcome of the research
H
H
0
0

The null hypothesis

attempts to show that
no variation exists
between variables

H
H
1
1

Scholarship & Publishing

Scholars Publish :
To share findings

Build a reputation in field

Earn Tenure
Promotion
Dissemination of quality research in order to
have an impact

Matched random sampling:

here participants are first matched on
some characteristic and then
individually assigned randomly to
groups.

PANEL SAMPLING
Probability Proportional to Size
Selecting a group of participants through a
"Proportional to size" refers the number
random sampling method and then asking that
of units awarded to variables in the
group for the same information again several
assessment as determined by the
times over a period of time.
population.
This longitudinal sampling-method allows for A village that has only 2,500 residents
estimates of changes in the population.
requires fewer healthcare facilities than a
city of 250,000.

JCR Oral Dentistry (80)

Multidisciplinary (55)

Oral Dentistry - SJR

How to Measure Your Impact

using Citations
Impact based on citations & h-index
Web of Science

12,000 peer reviewed journals (plus conference

proceedings)

Scopus
20,000 peer reviewed journals (plus conference
proceedings, books)
Citation information only goes back to 1996

Google Scholar
# of journals unknown lots but not all - permission
from publishers to index
also other sources conference proceedings books,
repositories

Web of Science

Scopus

Google Scholar

De Groote SL,
Raszewski R.
Coverage of Google
Scholar, Scopus, and
Web of Science: a
case study of the hindex in nursing. Nurs
Outlook. 2012 NovDec;60(6):391-400.

H Index
An index to quantify an individuals scientific research
output. J.E. Hirsch
The h-index is based on the set of a researchers most
cited papers and the number of citations that the
researcher has received in other people's publications
A scientist has index h if h of [his/her] Np papers have at
least h citations each, and the other (Nph) papers have at
most h citations each.
variants of h-index
g-index
a-index
and more

Calculating H-Index
Article # ---- Times Cited
1-----87
2-----70
3-----46
A scientist has index h if h of
[his/her] Np papers have at least h
5-----19
citations each, and the other
6-----15
(Np h) papers have at most h
7-----10
citations each
8 ---- 9
9 ---- 8
10--- 6
11---- 4
12---- 1

Calculating H-Index
Article # ---- Times Cited
1-----87
2-----70
8 articles have been cited at least 8
or more times and the remaining
3-----46
articles have been cited 8 or less
5-----19
6-----15
7-----10
A scientist has index h if h of
8 ---- 9
[his/her] Np papers have at least h
citations each, and the other
9 ---- 8
(Np h) papers have at most h
10--- 6
citations each
11---- 4
12---- 1

Web of
Science

Scopu
s

Altmetrics / Article Level Metrics

http://altmetrics.org/manifesto/
http://www.altmetric.com/

http://impactstory.org/

PLoS

Article Level Metrics - PLoS

http://article-level-metrics.plos.org/

impactstory.org

http://www.altmetric.com/

ORCID
http://orcid.org/

More Information
Selecting journal publication venues
http://researchguides.uic.edu/journalselection

Measuring Impact
http://researchguides.uic.edu/if
Contact Sandy De Groote
Associate Professor & Scholarly Communication Librarian
sgroote@uic.edu
312-413-9494

Selected Elements of Study

Design

Measurement accuracy (variables)

Effects can only be gauged relative to baseline (provided by a control group)

Experimental studies differ from non-experimental studies (of course)
The unit of recorded measure - individual or aggregate (ecological)
Upstream and downstream causes should be considered
Measurements may be longitudinal in individuals over time
Cohort or case-control samples
Hypothesis testing (analytic) or hypothesis generating (descriptive) studies
Is the exposure randomized?
Are groups comparable at baseline (confounding)
Will you use prospective or retrospective measurements?
Incident or prevalent cases?
Matched or independent samples?
Will you use blinded subjects and/or observers?
Is the study based in an open- or closed-population?
There are too many design elements to discuss in a single week. We cant cover them all!

Epidemiologic study designs

What type of study to chose depends on:
what is the research question/ objective
Time available for study
Resources available for the study
Common/rare disease or production problem
Type of outcome of interest
Quality of data from various sources
Often there are multiple approaches which will all
work
Choosing an established design gives you a huge
head start in design, analysis and eliminating biases

OBSERVATION:
There are two types of observation:
Naturalistic:The normal environment; everything is left as it
normally is.One weakness is that it may allows extraneous
variables to effect the IV therefore the researcher cannot claim
cause and effect.
Controlled:The environment is changed by the researcher, the
situation is controlled.This reduces the number of extraneous
variable.

Finding a h-index value in

Web of Science

Citation Report (h-index) from

Web of Science

Finding a h-index value in

Scopus

Citation Overview (h-index) from

Scopus

Epidemiologic study designs

The basis for the lecture is the distinction
between descriptive epidemiology and analytic
epidemiology
Descriptive epidemiology:
seeks to measure the frequency in which
diseases occur or collect descriptive data on
possible causal factors.
Analytic epidemiology:
attempts to specify in more detail the causes of
a particular disease

Study design is the procedure under

which a study is carried out
Two main categories
Observation:
Identify subjects, then
Observe and record characteristics
Experiment
Identify subjects,
Place in common context,
Intervene, then
Observe effects of intervention

 Observation vs Experiment
Observations are readily obtained, but subject to bias,
that is systematic errors
Some observational designs are less subject to bias
than others
Experiments are hard to do well
Experiments can answer narrow questions definitively
Generalizability of results from experiments may be at
issue, eg new drug testing that excludes women subjects

Epidemiologic study designs

Factors Important in Study Design
A. Specific, testable hypotheses - NOT a fishing
expedition But where do hypothesis come from?
observation + biological understanding + social
understanding + intuition causal hypothesis

B. Biases
1. Internal validity
2. External validity

Epidemiologic study designs

Cross-sectional studies
Cross-Sectional Studies measure
existing disease and current
exposure levels.
provide a 'snapshot' of the
outcome and the characteristics
associated with it, at a specific
point in time.
sample without knowledge of
Exposure or Disease
sample at one point in time
Mostly prevalence
studies/surveys

Cross Sectional Studies

(contd)

Advantages

Good design for hypothesis generation

Can estimate prevalence and sometimes rates
Can estimate exposure proportions in the population
Can study multiple exposures or multiple outcomes or diseases
Relatively easy, quick and inexpensive
No issue of subjecting any animals or producers to particular treatments
Best suited to studying permanent factors (breed, sex, blood-type)
Often good first step for new study issue

Cross Sectional Studies

Disadvantages
Impractical for rare diseases
give no indication of the sequence of events whether
exposure occurred before, after or during the onset of the
disease outcome. This being so, it is impossible to infer
causality.
Confounding is difficult to control
No control over sample size for each exposure by disease
subclass.
Prevalence-incidence bias (also called Neyman bias).
Especially in the case of longer-lasting diseases, any risk
factor that results in death will be under-represented among
those with the disease.

Epidemiologic study designs

Cross Sectional Studies
Disadvantages

Problems with temporal sequence of data

hard to decide when disease was actually acquired
disease may cure the exposure
miss diseases still in latent period
Nonresponse is a common problem in wide-scale surveys

 Repeated cross-sectional studies may be carried out

to give a pseudolongitudinal study, where the
individuals included in the study are either chosen
from the same sampling frame or from a different one.

Epidemiologic study designs

Case-control studies
Case-Control Studies identify existing disease/s and look back
in previous years to identify previous exposures to causal
factors.
Cases are those who have a disease.
Controls are those without a disease.
Analyses examine if exposure levels are different between
the groups.

Epidemiologic study designs

Case-control studies
Advantages
Best design for rare diseases
Can be accomplished quickly since events of interest have
already occurred
Can study several potential exposures at the same time
Lends itself well to hospital-based studies and outbreaks

Epidemiologic study designs

Case-control studies
Disadvantages
Problems with temporal sequence of data
Hard to decide when disease was actually
acquired
Disease may cure the exposure
Miss diseases still in latent period
Cant calculate incidence, population
relative risk or attributable risk
HIGH potential for bias

Epidemiologic study designs

Cohort Studies
In a cohort study, subjects with an
exposure to a causal factor are identified
and the incidence of a disease over time
is compared with that of controls (persons
who do not have the exposure).
In a longitudinal study, subjects are
followed over time with continuous or
repeated monitoring of risk factors or
health outcomes, or both.

Epidemiologic study designs

Cohort Studies
Advantages

Can get best assessment of exposure and can deal

with changes in exposure
May be only design if exposure needs to be
measured directly
Allows study of rare exposures
Only way to get prospective information for rapidly
fatal diseases
Good for establishing temporal sequence and
natural history of disease
Can examine multiple outcomes linked to exposure often find other effects than

Epidemiologic study designs

Cohort Studies
Advantages

Can estimate overall and specific disease rates, usually incidence

Researcher selects, measures & records data
Lower potential for bias than a case-control study - no recall and selection bias
Results are considered more conclusive than results from case-control studies
The longer a cohort study continues, the stronger it can become

Epidemiologic study designs

Cohort studies
Disadvantages
Impractical for rare diseases - even if risk is
high, no cases may occur or results
may not be statistically significant
Larger sample size than case-control
Data collection is usually very expensive
Long time commitment for follow-up

Epidemiologic study designs

Other designs:
Time series - test if incidence of disease changes in a population over
time

Hybrid - often what is seen in practice

Can be efficient and match necessity
Can lead to bias and disaster

Meta-analysis
Combining results from a range of published studies
Established methodology, not just literature review

 Observational studies
Case Series
Case-control studies
Cohort Studies
(Meta-Analyses)

 Case series Case series

Example: Normal plasma cholesterol in an 88-year-old man who
eats 25 eggs a day [Kern J, NEJM 1991; 324:896899] Example:
Normal plasma cholesterol in an 88-year-old man who eats 25
eggs a day [Kern J, NEJM 1991; 324:896899]
Advantages
Excellent at identifying unusual situation
Good for generating hypotheses amenable to rigorous test
Disadvantages
Generally short-term
Investigators self-select (bias!)
Generally no controls

 Controls
A control is a standard of comparison for
Effects
Variability
Case-control studies
Controlled studies Retrospective

 How are Case-Control Studies Done?

Identify cases (with condition of interest, here,
coccidioidomycosis) Match to disease-free controls who
are similar with respect to known risk factors for
condition
Compare degree of exposure to possible risk factor
(here, being in dust cloud).

 The logic of Case-Control Studies The logic of CaseControl Studies

Cases differ from controls only in having the disease
If exposure does not predispose to having the disease,
then exposure should be equally distributed between
the cases and controls.
The extent of greater previous exposure among the
cases reflects the increased risk that exposure confers

 Measures of effect Measures of effect

Relative risk (ratio of probabilities of contracting
disease given exposure), or Odds ratio (ratio of the
odds of contracting disease given exposure) RR or OR
of 1 indicate no effect of exposure (equal odds)
Physically being in a dust cloud (OR 3.0; CI, 1.6-5.4;
P<.001) significantly increased the risk for being
diagnosed with coccidioidomycosis

 Cohort studies Cohort studies

Prospective Controlled Can determine causes and
incidence of diseases as well as identify risk factors
Generally expensive and difficult to carry out
Procedure for cohort study Logic of the cohort study

 Procedure for cohort study

Identify groups of exposed subjects and control
subjects Match for other risk factors Follow over time
Record the fraction in each group who develop the
condition of interest Compare these fractions using RR
or OR

 Logic of the cohort study

Differences in the rate at which exposed and control
subjects contract a disease is due to the differences in
exposure, since other known risk factors are equally
present in the two groups.

 Clinical trials
A clinical trial is a comparative, prospective experiment
conducted in human subjects.
Historical controls

 Controlled vs Uncontrolled studies

Controlled comparison made relative to a simultaneous
reference condition
Uncontrolled comparison implicit Previous experience
Historical evidence Anecdote Controlled trials
make it possible to ascribe differences in outcome to differences
in treatment Differences from expectation in uncontrolled studies
could also be due to many other factors Different kinds of patients
Change in standard of care Improving technique, etc make it
possible to ascribe differences in outcome to differences in
treatment

 Historical controls
Are better than no controls Example: ddI vs AZT for
AIDS Are better than no controls Example: ddI vs AZT
for AIDS
But not by much Example: gastric freezing for peptic
ulcer

 Randomization & Concurrent Controls Randomization &

Concurrent Controls
Randomized vs nonrandomized assignment to Rx
The randomized clinical trial is the epitome of all
research designs because it provides the strongest
evidence for concluding causation. Conclusions from
nonrandomized studies subject to many sources of bias
Randomized vs nonrandomized assignment to Rx.

 Patients can serve as own controls

This is usually beneficial The comparison removes
patient differences.
Example: Vasocon-A
Subjects received placebo eye drops in one eye,
Vasocon-A drops in the other eye Randomization
determined which eye gets placebo for each pt Catdander extract applied to both eyes Response between
eyes compared.

 Paired designs Paired designs

Work best when outcome can be observed shortly after
treatment, and Disease and treatment are both shortlived

 Blinding
Good practice: factors that can affect the evaluation of
outcome should not be permitted to influence the
evaluation process Double-blind design Neither
patient nor outcome evaluator knows Rx to which
patient was assigned Single-blind Patient or evaluator
is blinded as to Rx, but not both Triple-blind Patient,
Physician, and Data analyst are blinded as to Rx identity