CASP

Checklist: 12 questions to help you make sense of a Cohort Study

How to use this appraisal tool: Three broad issues need to be considered when appraising a
cohort study:

Are the results of the study valid? (Section A)

What are the results? (Section B)
Will the results help locally? (Section C)

The 12 questions on the following pages are designed to help you think about these issues
systematically. The first two questions are screening questions and can be answered quickly.
If the answer to both is “yes”, it is worth proceeding with the remaining questions. There is
some degree of overlap between the questions, you are asked to record a “yes”, “no” or
“can’t tell” to most of the questions. A number of italicised prompts are given after each
question. These are designed to remind you why the question is important. Record your
reasons for your answers in the spaces provided.

About: These checklists were designed to be used as educational pedagogic tools, as part of a
workshop setting, therefore we do not suggest a scoring system. The core CASP checklists
(randomised controlled trial & systematic review) were based on JAMA 'Users’ guides to the
medical literature 1994 (adapted from Guyatt GH, Sackett DL, and Cook DJ), and piloted with
health care practitioners.
For each new checklist, a group of experts were assembled to develop and pilot the checklist
and the workshop format with which it would be used. Over the years overall adjustments
have been made to the format, but a recent survey of checklist users reiterated that the basic
format continues to be useful and appropriate.
Referencing: we recommend using the Harvard style citation, i.e.: Critical Appraisal Skills
Programme (2018). CASP (insert name of checklist i.e. Cohort Study) Checklist. [online]
Available at: URL. Accessed: Date Accessed.

©CASP this work is licensed under the Creative Commons Attribution – Non-Commercial-
Share A like. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-
sa/3.0/ www.casp-uk.net

Critical Appraisal Skills Programme (CASP) part of Oxford Centre for Triple Value Healthcare Ltd www.casp-uk.net
 Journal reading
Paper for appraisal and reference:.........................................................................................................
Section A: Are the results of the study valid?

1. Did the study address a clearly Yes HINT: A question can be ‘focused’
focused issue?
✔ in terms of
Can’t Tell • the population studied
• the risk factors studied
No
• is it clear whether the study tried to
detect a beneficial or harmful effect
• the outcomes considered

Comments: The aim of the present study was to investigate the prevalence of adjuvant
chemotherapy use among patients with MOC and the impact on overall
survival using a large hospital-based database. Our primary hypothesis
was that adjuvant chemotherapy is not associated with a survival benefit.

2. Was the cohort recruited in Yes HINT: Look for selection bias which might
an acceptable way? ✔ compromise the generalisability of the
Can’t Tell findings:
• was the cohort representative of a
No
defined population
• was there something special about the
cohort
• was everybody included who should
have been

Comments: The NCDB, has been established jointly by the American Cancer Society and Commission
on Cancer of the American College of Surgeons, as a hospital-based database capturing
data of patientswith newly diagnosed cancer in the United States. Patient data are
prospectively collected from participating commission-accredited cancer programs and
are regularly audited. All data are de-identified and available for research purposes

Is it worth continuing?

2
 3. Was the exposure accurately Yes HINT: Look for measurement or
measured to minimise bias? ✔ classification bias:
Can’t Tell • did they use subjective or objective
measurements
No • do the measurements truly reflect what
you want them to (have they been
validated)
• were all the subjects classified
into exposure groups using the
same procedure

Comments: Patients with MOC were identified based on ICD-O-3 histology codes as grouped by the
International Agency for Research on Cancer. Those with stage I disease who underwent
primary surgery (as assessed fromsite-specific surgery codes) and did not have a personal
history of another tumorwere selected for further analysis. Interval between surgery and
chemotherapy administration was calculated, and similar to previous studies

4. Was the outcome accurately Yes HINT: Look for measurement or

measured to minimise bias? ✔ classification bias:
Can’t Tell • did they use subjective or objective
measurements
No • do the measurements truly reflect what
you want them to (have they been
validated)
• has a reliable system been
established for detecting all the cases (for
measuring disease occurrence)
• were the measurement
methods similar in the different groups
• were the subjects and/or
the outcome assessor blinded to
exposure (does this matter)

Comments: Categorical and continuous variables were compared between patients who did and did not receive
adjuvant chemotherapy with the chi-square and Mann-Whitney U tests respectively. A binary logistic
regression was performed to identify clinical variables independently associated with the
administration of adjuvant chemotherapy. Kaplan-Meier curves were generated to determine 5-year
OS rates while univariate analysis was performed with the log-rank test. Stratified analyses by tumor
grade, and substage were performed. A Cox multivariate model was constructed to evaluate the
impact of adjuvant chemotherapy on overall survival after controlling for variables associated with
OS in the univariate analysis.

3
5. (a) Have the authors identified Yes HINT:
all important confounding ✔ • list the ones you think might be
factors? Can’t Tell important, and ones the author missed

No

Comments: The presence of medical co-morbidities was evaluated using the Charlson-Deyo
Comorbidity index score and was categorized into absent (score 0) and present (score 1).
Who received neoadjuvant treatment, and those with unknown surgery-chemotherapy
interval were excluded from the present study.

5. (b) Have they taken account of Yes HINT:

the confounding factors in the
design and/or analysis?
✔ • look for restriction in design, and
Can’t Tell techniques e.g. modelling, stratified-,
regression-, or sensitivity analysis to
No correct, control or adjust for confounding
factors

Comments: The presence of medical co-morbidities was evaluated using the

Charlson-Deyo Comorbidity index score and was categorized into absent
(score 0) and present (score 1). Stratification by disease substage or tumor
grade

HINT: Consider
6. (a) Was the follow up of
subjects complete enough?
Yes
✔ • the good or bad effects should have
had long enough to reveal
Can’t Tell themselves
• the persons that are lost to follow-up
No may have different outcomes than
those available for assessment
• in an open or dynamic cohort, was
there anything special about the
outcome of the people leaving, or the
exposure of the people entering the
cohort
6. (b) Was the follow up of Yes
subjects long enough? ✔
Can’t Tell

No

4




Comments: Follow-up data were available for cases diagnosed between 2004 and
2014 and suppressed for patients diagnosed in 2015. For survival analysis,
a minimum of 1 month of follow-up was required.



Section B: What are the results?



7. What are the results of this study? HINT: Consider
• what are the bottom line
results
• have they reported the rate or
the proportion between the
exposed/unexposed, the
ratio/rate difference
• how strong is the association
between exposure and
outcome (RR)
• what is the absolute risk
reduction (ARR)

Comments: There was no difference in OS between patients who did and did not receive
chemotherapy, p = 0.17; 5-yr OS rates were 86.8% and 89.7% respectively. For patients
with stage IA/IB disease, there was no difference in OS between the observation (n =
2465) and the chemotherapy groups (n = 618), p = 0.46; 5-year OS rates were 90.5% and
88.1% respectively. Similarly, for patients with stage IC disease, there was no difference in
OS between the observation (n=455) and chemotherapy (n=704) groups, p = 0.11; 5-year
OS rates were 85.1% and 85.6% respectively.

8. How precise are the results? HINT:
• look for the range of the confidence
intervals, if given

Comments: After controlling for patient age, the presence of co-morbidities, the type of
insurance, disease sub-stage, tumor grade and size, and the performance
of LND, the administration of adjuvant chemotherapywas not associated
with a better overall survival (HR: 1.18, 95% CI: 0.85, 1.64)

5


9. Do you believe the results? Yes HINT: Consider
✔
• big effect is hard to ignore
Can’t Tell • can it be due to bias, chance or
confounding
No • are the design and methods of this
study sufficiently flawed to make the
results unreliable
• Bradford Hills criteria (e.g. time
sequence, dose-response gradient,
biological plausibility, consistency)

Comments: NCDB and a cohort of patients with a pathologically confirmed primary ovarian carcinoma.
Co-morbidities was evaluated using the Charlson-Deyo Comorbidity index score. The chi-square and
Mann-Whitney U tests. A binary logistic regression to identify clinical variables independently
associated with the administration of adjuvant chemotherapy. Kaplan-Meier curves were generated
to determine 5-year OS rates while univariate analysis was performed with the log-rank test

Section C: Will the results help locally?

10. Can the results be applied to Yes HINT: Consider whether
the local population? • a cohort study was the appropriate
Can’t Tell method to answer this question
• the subjects covered in this study could
No be sufficiently different from your
✔
population to cause concern
• your local setting is likely to differ
much from that of the study
• you can quantify the local benefits and
harms

Comments: Because the study result was not significant,so we cannot apllied this study
result in population



11. Do the results of this study fit Yes
with other available
evidence? Can’t Tell

No
✔

Comments: To date there is no evidence supporting the administration of adjuvant
chemotherapy for patients with early stage MOC

6


12. What are the implications of Yes HINT: Consider
this study for practice? • one observational study rarely
provides sufficiently robust
Can’t Tell
✔ evidence to recommend changes
to clinical practice or within health
No policy decision making
• for certain questions,
observational studies provide the
only evidence
• recommendations from
observational studies are always
stronger when supported by other
evidence

Comments: In this large contemporary cohort of patients with stage I MOC, approximately one in three patients
received adjuvant chemotherapy with no clear benefit on overall survival. Given the lack of clear
evidence, the decision to administer adjuvant chemotherapy should be individualized following
appropriate counseling. Randomized trials are extremely difficult to perform even with international
participation. Creation of an international databasemay further aid in the clarification of the optimal
management of these patients and identify any subgroups that can benefit from adjuvant treatment.