Sie sind auf Seite 1von 3

1838 International Journal of Epidemiology, 2016, Vol. 45, No.

International Journal of Epidemiology, 2016, 1838–1840

Commentary: Causal doi: 10.1093/ije/dyw229
Advance Access Publication Date: 27 January 2017
inference in epidemiology:
potential outcomes, pluralism and peer review
Douglas L Weed
DLW Consulting Services, LLC, University of Utah School of Medicine, Salt Lake City, UT 84132, USA.

Accepted 20 July 2016

Introduction: The Debate on the Exclusivity For those of us who have participated in these sorts of
of Potential Outcomes philosophical discussions—typically, philosophy in epi-
Now that two groups have challenged the idea that the poten- demiology—over the decades, it is not sufficient to simply
tial outcomes (counterfactual) approach (POA) is the only le- critique a method (or an approach such as POA), identify
gitimate approach to causal inference in epidemiology, I fully its problems (even errors) and call it a day. To make pro-
expect the supporters of that view to either defend their pos- gress (i.e. to make improvements), an alternative must be
ition or offer an olive branch to appease their critics. The de- proposed that does at least several things: it corrects the
bate is engaged and best left to those in academic towers errors in the original method, it keeps the features of the
where these arguments mean something. In the practice of original method that ‘work’ and it does not add new prob-
epidemiology, despite what Vandenbroucke et al.1 and lems (errors) that make it less desirable than the original
Krieger and Davey Smith2 contend, the presumed negative method. With this working model of progress in mind, I
impact of POA has yet to be realized. No journal requires it. turn to the alternative approaches offered by Krieger and
Sometimes counterfactuals and DAGs appear and often they Davey Smith2 and by Vandenbroucke et al.1
do not. In the complex and ever-expanding worlds of regula- According to Krieger and Davey Smith, ‘there is not one
tion and litigation, where epidemiology plays a key role, POA prescription for how epidemiologists can best infer caus-
is at best a whisper. Simply put, the debate about whether ation’. Rather, we epidemiologists should be ‘open to plural-
POA is the only legitimate approach to causal inference in ism about causation’ and use the approach ‘best developed
epidemiology is as much about the power of individuals at by Peter Lipton’ which will help us to ‘forge that aspire to
certain academic institutions to gain attention as it is about scope, precision, mechanism, unification, and simplicity’,
the intellectual competitions that excite so-called ‘theoret- the ‘loveliest’ and not necessarily the ‘likeliest’ explanation.
icians’ of epidemiology. It remains to be seen if the exclusivity Lovely explanations are a feature of Lipton’s philosophical
claim is really a critical issue for the future of epidemiology. treatise,3 which really cannot be said to provide epidemiolo-
gists with anything resembling a practical approach to
causal inference. Krieger and Davey Smith’s contention,2
Pluralism and Its Constraints that Lipton frequently employs epidemiological examples, is
That said, there are some disturbing features of the argu- misleading. Lipton is no epidemiologist, and his intent is not
ments by Vandenbroucke et al.1 and Krieger and Davey to provide a practical approach to causal inference in epi-
Smith2 with important implications for the practice of demiology or elsewhere. In his own words, Lipton is at-
causal inference and for peer review. Of particular concern tempting to ‘describe inductive processes’ in philosophical
are the alternative solutions (to POA) offered by these au- terms. So we are left with what Krieger and Davey Smith2
thors—what they provide as proposed improvements for articulate from their translation of Lipton’s effort, which is
causal inference. To be blunt, they advocate a type of ‘plur- that there are many kinds of study designs relevant and use-
alism’, something akin to ‘anything goes’ when attempting ful for causal inference in epidemiology, each of which in-
to assess disease causation as long as there is evidence and volves different assumptions and biases. In their words,
judgment and their approaches fall under the rubric of ‘in- there is a ‘need for causal triangulation, whereby epidemi-
ference to the best explanation’, at best a philosophical slo- ologists should employ diverse study designs, each involving
gan (according to its author, Lipton3). different and unrelated potential biases, and test our
C The Author 2017; all rights reserved. Published by Oxford University Press on behalf of the International Epidemiological Association

Downloaded from

by Stockholm University Library user
on 17 May 2018
International Journal of Epidemiology, 2016, Vol. 45, No. 6 1839

hypotheses in different populations and in different histor- preciation for how this well-known and still relevant ap-
ical periods, to see if results are robust to the confounding proach fits into the family of methods used in everyday
structures encountered and the analytic methods used’. Yes. epidemiological practice.
I agree. It is good to use more than one study design and to Beyond the idea that there is more than one type of
examine hypotheses in more than one population, which study design, the pluralistic approach to causal inference in
typically occurs in more than one historical ‘period’. Bias epidemiology advocated in these two papers appears to
and confounding are issues we need to consider, using ana- consist primarily of metaphors—did I mention ‘tale wag-
lytical and statistical methods. Judgment is essential. ging the dag?’—and judgment. This is not progress. These
According to Vandenbroucke et al.,1 the solution to the authors have latched onto pluralism rather than a unilat-
problem described above is also pluralism or what they eral POA approach, a simple solution to be sure. However,
call ‘pragmatic pluralism’ which involves judgment, ‘rag- in effect they have solved the problem of relying too much
ged evidence,’ triangulation and other ‘diverse strategies’. upon a single approach by introducing an even more vex-
They propose that ‘future epidemiologists should learn: (i) ing problem: relying upon any and all approaches as long
that causal inference remains a judgment based on integra- as they involve many different study designs or ‘triangula-
tion of diverse types of evidence; (ii) diverse strategies to tion’ or ‘interlocking (ragged) evidence’, judgment and
assess causality by ruling out alternatives, such as triangu- lovely explanations. In the pluralistic world of
lation, negative controls, and interlocking evidence from Vandenbroucke et al.1 and Krieger and Davey Smith,2 the
other types of science; (iii) the elements of all types of epi- metaphors plus judgment approach—whether actually
demiological study designs, including counterfactual de- based on Lipton’s philosophical treatise or not—is as insuf-
signs; and (iv) to reflect critically on whether potential ficient as POA and potentially less helpful. In the end, the
biases matter’. Yes. I agree. Use different study designs. alternatives introduce as many—perhaps more—problems
Rule out alternatives, including bias and confounding (and than the single problem they attempt to solve.
chance?); employ judgment. This is also not an innovative
solution to the problem.
In sum, both groups focus on judgment, and both groups Judgment
emphasize pluralism by which they mean something other As noted above, both Vandenbroucke et al.1 and Krieger
than a single approach. Remember that their arguments are and Davey Smith2 emphasize judgment as a key compo-
focused upon why one approach—POA—cannot be suffi- nent of their pluralistic approaches. This is not a new phe-
cient. Fair enough. The POA approach isn’t enough. nomenon, given that Susser4—one of the early giants of
But has the methodological pendulum swung too far in causal inference in epidemiology—emphasized the import-
the opposite direction? What these experienced epidemiolo- ance of judgment, and it continues to appear in recent dis-
gists advocate is a collection of potentially competing ideas cussions of so-called ‘weight of evidence’ approaches to
and vague approaches, not a prescription (to be sure), but causation (Adami et al.,5 Cogliano et al.6 and Krimsky7).
also not directly tied to how epidemiologists actually practise But what exactly is judgment? We can agree that judg-
causal inference in today’s world, as informed by a half cen- ment is an important part of the process of causal infer-
tury of theory and practice. If a student were to ask me how ence, and we can agree that we prefer those who exhibit
to practise causal inference based on these two papers, I ‘good’ or ‘sound’ judgment over those whose judgment is
would tell them that there is no one way to do it, that you questionable.8 When collecting, summarizing and inter-
should use different study designs, ‘triangulate’ and ‘interlock’ preting scientific evidence, we can also agree that judgment
evidence from other sciences. Above all, use your judgment is applied at many steps. The problem with judgment is
about all that ‘ragged evidence’ and assess whether confound- that it is a complex and not wel- understood mental cap-
ing and bias can explain the results. Finally, consider how acity. Krimsky7 calls it a ‘black box’, a familiar term for
‘lovely’ your explanation is. I would not be surprised if the epidemiologists.9 At the minimum, judgment includes: sci-
student would reply, ‘Yes, OK, but how do I determine if an entific reasoning, ethical reasoning, practical wisdom and
exposure causes a disease?’ values. Judgment is not, therefore, a purely subjective pro-
I feel this student’s pain. There is no mention in these cess, although it has a clear and significant subjective com-
two papers about the need to employ systematic review ponent. Relying too much upon judgment, however, is
methods, no mention of statistical significance, no mention problematic. After all, how can we objectively determine
of meta-analysis and no mention of the tools for determin- whose judgment is better?
ing study quality or study hierarchies. Oddly enough, there Judgment may be a necessary component of the process,
are brief mentions of Hill’s causal considerations deep in but is not, by itself, sufficient for making causal claims.
the guts of their arguments, but no discussion and no ap- Methods of causal inference and a systematically collected

Downloaded from

by Stockholm University Library user
on 17 May 2018
1840 International Journal of Epidemiology, 2016, Vol. 45, No. 6

body of evidence are necessary. These methods take prece- different study designs and statistical methods), summariza-
dence over judgment in any causal assessment. tion of the results across studies (when appropriate) whether
by pooled analysis or meta-analysis, the application of the
Peer review Bradford-Hill viewpoints (or criteria or whatever else you
If Vandenbroucke et al.1 and Krieger and Davey Smith2 want to call them) which creates the need to repeat the process
are correct in arguing that the POA approach cannot be described above in the biological-mechanistic world of toxi-
the only way to assess causation, then their pluralism (plus cology, and a description and critique of past reviews examin-
judgment) solution creates havoc for the peer reviewer. ing the same exposure-disease relationship. The overarching
How can those who have been tasked with reviewing any method that subsumes all these is the general scientific
submitted manuscript or published paper, that claims to method. Viewing causal inference from the perspective of how
infer (or establish) or assess causation, proceed? it is actually practised permits—indeed, invites—the inclusion
Given that the literature on peer review is remarkably quiet of the DAG (counterfactual) approach as one picturesque way
on specifics when it comes to causal inference, I offer the fol- to describe the potential effects of exposures and confounders.
lowing considerations to students and peer reviewers alike: I respect the efforts of all those who work to improve
the theory and practice of causal inference in epidemi-
i. whether the authors rely primarily upon method or
ology. After all, causality is the most important scientific
primarily upon their subjective judgment in making
problem we epidemiologists aim to solve.
claims about causation;
Conflict of interest: Dr. Weed is an independent scien-
ii. whether a method for determining causation is
tific consultant. No entity, whether non-profit or for-
profit, provided funding for this paper. Dr. Weed con-
iii. whether the method described is one generally recog-
ceived, designed, and wrote this paper without assistance
nized in the scientific community and referenced there;
from any organization or individual.
iv. whether the authors’ description of that method is ac-
curate [i.e. whether it reasonably conforms to the de-
scriptions of that method in the published literature or
misrepresents (deviates prominently from) those same 1. Vandenbroucke JP, Broadbent A, Pearce N. Causality and causal
description]; inference in epidemiology: the need for a pluralistic approach. Int
J Epidemiol 2016;45:1776–86.
v. whether that method is appropriate for the scientific
2. Krieger N, Davey Smith G. The tale wagged by the DAG:
question at hand;
broadening the scope of causal inference and explanation for epi-
vi. whether the method selected by the author was used demiology. Int J Epidemiol 2016;45:1787–808.
appropriately to interpret results. 3. Lipton P. Inference to the Best Explanation. 2nd edn. London and
New York: Routledge, 2004.
In my experience, these six considerations cover the
4. Susser M. Judgment and causal inference: criteria in epidemio-
bases reasonably well. Put another way, infractions of any
logic studies. Am J Epidemiol 1977;105:1–15.
one of these—much less several—cast serious doubt on the 5. Adami HO, Berry CL, Breckenridge CB et al. Toxicology and epi-
validity and reliability of any authors’ causal assessment. demiology: improving the science with a framework for combin-
ing toxicological and epidemiological evidence to establish causal
Conclusion inference. Toxicol Sci 2011;122:223–34.
This is not the place to fully describe the approach to causal 6. Cogliano VJ, Baan RA, Straif K et al. The science and practice of
inference that exists in the practising community of epidemio- carcinogen identification and evaluation. Environ Health Perspect
logical scientists and in the community of biomedical scientists
7. Krimsky S. The weight of evidence in policy and law. Am J Pub
in general. The approach—really a family of methods—has
Health 2005;95:S129–36.
been ignored in these papers, perhaps because it is both too 8. Weed DL. The nature and necessity of scientific judgment. J Law
complex and too well-known. It begins with a systematic col- Policy 2007;15:135–64.
lection and description of the available evidence, an assess- 9. Weed DL. Beyond black box epidemiology. Am J Public Health
ment of the quality of the individual studies (which utilize 1998;88:12–14.

Downloaded from

by Stockholm University Library user
on 17 May 2018