Sie sind auf Seite 1von 10

Abma - Social psychology & Stapel 28 Aug.

2013

Social psychology before and after the Stapel fraud case


Ruud Abma
Utrecht University
The Netherlands
On the 28th of November 2012, the three committees that had investigated the work of the
Dutch social psychologist Stapel, published their joint report Flawed science: The fraudulent
research practices of social psychologist Diederik Stapel (Levelt Committee, 2012). They had
established without any doubt that Stapel had committed fraud in 55 papers, all published in
international journals in the period 2004-2011. Moreover, they found serious evidence of
fraud in 10 earlier papers. Finally, in 10 PhD theses that Stapel supervised one or more
chapters were based on fraudulent data.
On the same day that the report was presented, Stapel appeared on Dutch television.
He stated that he was sorry for the damage he had caused to his field, his colleagues and his
PhD students. His mental condition did not allow him to be interviewed by the press, but he
referred to an autobiographical book that was about to published: Ontsporing, which literally
means derailment.
In this paper, I will discuss the meaning and significance of the Stapel fraud case, against the
background of the post-war history of social psychology. The 1950s and 1960s saw a rise of
experimental social psychology, followed by a critical intermezzo, in which basic
epistemological, methodological and ethical issues were discussed. This did not, however,
stop the development of experimental social psychology. The Stapel fraud case has renewed
the interest in current methodological problems of social psychology. What are the differences
with the earlier wave of criticism?
Stapels fraud

Fraude ratio Stapel

ESHHS Wrzburg 2013

Abma - Social psychology & Stapel 28 Aug. 2013


The graph shows Stapels production of international (English) papers (black line) and the
amount of fraudulent papers (dotted line). His high production in 1998 gave him his image of
golden boy with a knack for successful experimental work. It also helped him to get a
professorship in Groningen. His extremely low production in 2003 drove him to falsify data,
according to himself for the first time. When no one found out and instead everyone admired
his beautiful results (too good to be true), falsification became a habit, and later on his
fabrication of complete data sets guaranteed a continuing high publishing record.
Stapel even succeeded in getting his paper Coping with chaos published in Science,
on messy environments and stereotyping, which was completely made up (Stapel &
Lindenberg, 2010). Neither his co-author Siegwart Lindenberg, a reputed sociologist, nor the
reviewing editor of Science, the Dutch social psychologist Ap Dijksterhuis had any doubts
about the truthfulness of the reported experiments.
Stapel published mainly in social psychology journals, his favourite being the Journal
of Personality and Social Psychology, the leading journal in the field. More than half of his
publications in this journal (1996-2011) were based on falsified or fabricated data. The picture
for other journals is about the same (see Table). Neither his 70 co-authors, nor the journal
reviewers were alerted by the sometimes quite obvious mistakes and unlikely patterns in
Stapels papers.

J. of Personality and Social Psychology


European Journal of Social Psychology
Personality and Social Psychology Bulletin
Journal of Experimental Social Psychology
Social Cognition
British Journal of Social Psychology
Psychological Science
Basic and Applied Social Psychology

13/24
11/19
8/15
5/14
5/14
3/4
2/4
2/3

Sloppy science?
Stapel had published fraudulent papers, co-authored by leading figures in the field, in the
major social psychology journals without being noticed at all. The Levelt committee therefore
raised the question why this fraud and the widespread violations of sound scientific
methodology were never discovered in the normal monitoring process in science []
Virtually nothing of all the impossibilities, peculiarities and sloppiness mentioned in this
report was observed by all these local, national and international members of the field, and no
suspicion whatever arose. (Levelt committee, 53) Therefore they concluded that there are
certain aspects of the discipline that should be deemed undesirable or even incorrect from the
perspectives of academic standards and scientific integrity (Ibidem, 54).
Confirmation bias
The main target of the committees was the so-called verification (or confirmation) bias: One
of the most fundamental rules of scientific research is that an investigation must be designed
in such a way that facts that might refute the research hypotheses are given at least an equal
chance of emerging as do facts that confirm the research hypotheses. Violations of this
fundamental rule, such as continuing to repeat an experiment until it works as desired, or
excluding unwelcome experimental subjects or results, inevitably tend to confirm the
researchers research hypotheses, and essentially render the hypotheses immune to the facts.
(Ibidem,.48)

Abma - Social psychology & Stapel 28 Aug. 2013


The report subsequently presents a multitude of examples of verification bias in
Stapels publications. When confronted with these examples, several co-authors defended the
serious and less serious violations of proper scientific method with the words: that is what I
have learned in practice; everyone in my research environment does the same, and so does
everyone we talk to at international conferences. (Ibidem, 48) Therefore the committees
concluded that from the bottom to the top there was a general neglect of fundamental
scientific standards and methodological requirements. (Ibidem, 53)
According to the committee, this situation was aggravated by certain practices by
international journals: For instance, a co-author stated that editors and reviewers would
sometimes request certain variables to be omitted, because doing so would be more consistent
with the reasoning and flow of the narrative, thereby also omitting unwelcome results.
Reviewers have also requested that not all executed analyses be reported, for example by
simply leaving unmentioned any conditions for which no effects had been found, although
effects were originally expected. [] Not infrequently reviews were strongly in favour of
telling an interesting, elegant, concise and compelling story, possibly at the expense of the
necessary scientific diligence. (p.53)
As early as 1998, Stapel himself openly had advocated the practice of verification bias, in a
speech at the occasion of receiving the Jos Jaspars Award: We design an experiment and go
to our lab to test our conjectures. And then what happens? Our experiment fails. We dont
find what we expected to find. Moreover, we find something we cannot explain. We tweak
and fine-tune the experimental set-up until we find something we do comprehend, something
that works, something with a P-value smaller than .05. Champaign! Celebration! []
I am sure that there are other ways of doing experimental social psychology.
Sometimes, for example, our research is theory- rather than data- or observation-driven. My
point is that whatever way we arrive at our theories and hypotheses, the experiments and tests
we design are made to verify, not to falsify our conjectures. The leeway, the freedom we have
in the design of our experiments is so enormous that when an experiment does not give us
what we are looking for, we blame the experiment, not our theory. (At least, that is the way I
work). Is this problematic? No.
Our results are often paradigm-contingent. That is, we find what we are looking for
because we design our experiments in such a way that we are likely to find what we are
looking for. Of course! Should we design our experiments such that we are unlikely to find
support for our hypotheses? Should we try to prove ourselves wrong? No, for the best results,
we should use the methods that are likely to work best. Use a spoon to eat your soup and a
cup to drink your tea. Not vice versa. (Stapel, 1998, xx)
Here, Stapel appears to be a good pupil of Festinger (1953, 156) who states that there is
almost limitless room for the experimenters ingenuity to create a situation which will be best
for his experimental purpose. Fifteen years later, Aronson and Carlsmith (1968) suggested
that there are almost as many ways of building and conducting an experiment as there are
experimenters. (p.3 / 372).

Angry social psychologists


The evocation by the Levelt committees of the fundamental rules of scientific research and
the suggestion that social psychologists in their research tend to fall short of these standards,
created an uproar in social psychological circles.

Abma - Social psychology & Stapel 28 Aug. 2013


For instance, the emeritus professor of social psychology from Utrecht University, Wolfgang
Stroebe, offered as his opinion that the report was an insidious attack on the whole of social
psychology. Stroebe agreed that the reviewer blindness for Stapels misconduct is shocking,
but he added sadly it is pervasive to all fields of science and not specific to social
psychology. The use of this observation as an argument to attack the entire field of social
psychology shows therefore a breathtaking ignorance of the vast literature on scientific
misconduct. (DUB, 5 dec. 2012)
Interestingly, Stroebe added the following observation: The major difference between
the Levelt/Noorth/Drenth Committees and the reviewers of Stapels manuscripts, who
overlooked discrepancies that were obvious to the clever committee members, is that these
reviewers were reviewing articles by a star scientist of unblemished reputation, whereas the
Committee already knew that most of this research was fraudulent. Apparently, Stroebe did
not presume that there was Stapels papers were submitted to blind reviewing.
Stroebe ended his contribution by requesting an apology: I would therefore suggest that the
members of the committee read the vast literature on scientific fraud, enriched by some social
psychological reading on hindsight bias (why people are always cleverer after the fact).
They could then produce an amended version of their report, omitting all defamatory
statements about social psychology. And finally, they might offer their apology to the many
scientists they have insulted with their slanderous conclusions about social psychology.
Stroebes article was followed by a long string of comments by social psychologists,
most of whom agreed with him. For instance, Fritz Strack of Wrzburg University stated:
The report is an excellent example of "slodderwetenschap". It spreads sweeping
generalizations without clear documentation while neglecting the diligence to which it
subjects and holds up social psychology. I doubt that its claims would pass peer review and
editorial scrutiny for any scientific journal.
More officially, the European Association for Social Psychology also protested against
the conclusions of the report about the research culture in social psychology: The Levelt
report was published on November 28, but instead of providing the expected closure and a
welcome insight into one of the darkest chapters of the history of social psychology, it brings
the discipline as a whole into disrepute. For, in addition to describing Stapels fraudulent
activities, the report characterizes social psychology as a discipline with low academic
standards and limited scientific integrity. (EASP, 2012)
Not one of the angry social psychologists, however, responded to the critique that it
was the policy of the journals in this field that encouraged researchers to cut corners and
photoshop their data and analyses.
A historical view
Confirmation bias is found in experimental social psychology, but is it limited to social
psychology? Probably not (Stroebe, Postmes and Spears, 2012). But it certainly has a long
tradition here.
In Making social psychology experimental, Kurt Danziger (2000) demonstrates how
the meaning of experimentation has varied widely in psychology, but ended up in specific
types of manipulative social psychological experimentation, in which social was reduced to
interpersonal, whereby humans functioned as social stimuli only insofar as they were
directly present to each other (Danziger, 2000, 333). These experiments were limited to
exploring effects that were local, proximal, short term and decomposable. This constituted a
radical break [by F. Allport] with the more traditional conceptions of the social that were
operative in the social sciences around 1900 (334). Kurt Lewins experimental study of
group phenomena in the 1940s can be seen as an intermezzo: his conception of theory as a

Abma - Social psychology & Stapel 28 Aug. 2013


structural model became after his death in 1947 became replaced by Festingers conceptual
variables.
This tradition was especially stimulated by Leon Festinger: The laboratory
experiment should be an attempt to create a situation in which the operation of variables will
be clearly seen under special identified and defined conditions. It matters not whether such a
situation would ever be encountered in real life. (Festinger, 1953, 139) Festingers definition
of a true experiment: the investigator creates a situation with the exact conditions he wants
to have and in which he controls some, and manipulates other variables. He is then able to
observe and measure the effect of the manipulation of the independent variables on the
dependent variables. (1953, 137). Festingers preference is for strong manipulations of
variables, obtaining effects that are comparable with those in real life. To obtain these strong
effects it is a matter of course that the experimenter must use deception, prevarication,
misdirection of subjects, and the like. (1953, 170).
This reorientation brought social psychology into line with well-respected core areas
of experimental psychology and thereby boosted its prestige.
Crisis in social psychology
During the 1970s laboratory experimentation, which had become the method of choice in
social psychology research publications, was criticized from various perspectives: ethical,
methodological and epistemological. It became a contested field, even among experimental
social psychologists themselves.
Ethical
Long after he abandoned social psychology, Festinger (1980) reflected: ethical issues never
seemed extraordinarily difficult to me [] I do not see the harm in temporarily deceiving
persons in order to study some important question.
Methodological
Milgrams obedience studies constituted an extreme version of this new form of
experimentation insofar as his experiments were frequently illustrations rather than simple
inductive accounts of hypotheses tested even if, epistemologically, they were only
illustrations of Milgrams own cleverness.(Stam et al, 374)
Ecological validity
Robert Zajonc reflected in 1997 that experiments had taken on a life of their own, and
research had lost contact with everyday life. Events researched in one lab were designed to
explore effects not found in everyday life, but in other labs (Brannigan, 2004, 11)
Individualism: [the social collapses into interpersonal which then becomes individualized]
These critiques for a large part reflect the self image of experimental social psychologists. For
instance, Festinger (1953) saw subjects as barely sentient data-producing organisms, that
nevertheless had a seemingly endless ability to destroy the aims of the experiment. This was
echoed by Aronson and Carlsmith (1968), who depicted subjects as the Other the
unknowing and uncaring raw material of the scientific enterprise in social psychology. Above
all they could not be trusted all manner of deceptions, manipulations, and checks must be
built into an experiment to guard against this Other. (Stam et al., 375)

Abma - Social psychology & Stapel 28 Aug. 2013


Deception: social priming
One form of deceiving subjects is used in so-called priming experiments. Some of these
became rather famous, such as Barghs old age experiment or Dijksterhuis professor
prime. It appeared problematic to replicate the results in exact replication experiments.
Maybe this is why social psychologists favoured conceptual replications: investigating the
same hypothetical mechanism with a different task. During the last two years, in at least two
instances doubt was raised by exact replications.
This prompted Daniel Kahneman to send a letter to students of social priming (26 sept.
2012). He wrote:
As all of you know, of course, questions have been raised about the robustness of priming
results. The storm of doubts is fed by several sources, including the recent exposure of
fraudulent researchers, general concerns with replicability that affect many disciplines,
multiple reported failures to replicate salient results in the priming literature, and the growing
belief in the existence of a pervasive file drawer problem that undermines two methodological
pillars of your field: the preference for conceptual over literal replication and the use of metaanalysis. Objective observers will point out that the problem could well be more severe in
your field than in other branches of experimental psychology, because every priming study
involves the invention of a new experimental situation.
For all these reasons, right or wrong, your field is now the poster child for doubts
about the integrity of psychological research. Your problem is not with the few people who
have actively challenged the validity of some priming results. It is with the much larger
population of colleagues who in the past accepted your surprising results as facts when they
were published. These people have now attached a question mark to the field, and it is your
responsibility to remove it. []
I believe that you should collectively do something about this mess. To deal
effectively with the doubts you should acknowledge their existence and confront them straight
on, because a posture of defiant denial is self-defeating.
Although no one directly accused priming researchers of committing fraud, doubts of
confirmation bias and sloppy science were present. Besides Kahneman, it were especially
methodologists (Simmons, Nelson & Simonsohn 2011) who urged social psychologists to set
up strings of replication experiments, which was not quite realistic, because already for a long
time leading journals in the field had shown a lack of interest in publishing replication studies.
[Stroebe recently argued that replication studies would not help to detect fraud EBSP, May
2013]
Connecting the dots
The traditional response to the criticisms of sloppy science and confirmation bias has been to
advocate a stricter compliance to orthodox methodology. After Stapels fraude and the
Levelt report, all sorts of practical rules and regulations are suggested, concerning the role of
ethics officers in departments, central facilities for the storage of data, recognizing the specific
contribution of authors to journal articles, not communicating unpublished research to the
media, more strict procedures in the review process and last but not least more opportunities
for publishing replications, including failures of replication.
Fritz Strack from Wrzburg has expressed his doubts about this and, in part he seems
to follow the lead of the 1970s critique here. In EBSP he wrote: there is no methodological
procedure that links data and theory such that obeying certain rules would guarantee the
truth. In his view the scientific endeavour is more like a persuasive communication that

Abma - Social psychology & Stapel 28 Aug. 2013


requires humans both as communicators and recipients, while the methodology plays the role
of the rhetoric (Strack, 2012). Instead of supporting moralistic pseudo-methodological
standards, Strack favours a change in the content and not the rhetoric of the persuasive
communication. Researchers should not try to please journalists and mass media by creating
a wow! but instead focus on the underlying processes, the how of psychological
phenomena.
Also they should not focus on the exact replication of specific phenomena, but rather
on the demonstration of those universal underlying mechanisms that are responsible for
producing them. Nobody would expect that the relationship between cognitive contents and
behaviour, for instance the elderly stereotype as in Barghs experiment, would be the same
over varying contexts. Stracks main plea is to focus on creating a solid conceptual basis for
social psychology research (including replications).
More fundamental are the suggestions done by Naomi Ellemers in her paper
Connecting the dots: Mobilizing theory to reveal the big picture in social psychology (and
why we should do this) (Ellemers, 2013). Like Strack she deplores the frequent publication
of sexy attention grabbers and the increase of short papers being published. She is also
critical about the emphasis on methodological purity (or fetishism) to counter sloppy science
and questionable research practices: Adding rules, checks and sanctions is a standard
response displayed by organizations that try to cope with errors that have been made.
However, empirical research has revealed that this is not necessarily the best way to learn
from previous problems or to improve collective performance. (Ellemers, 2013, 3)
Instead, Ellemers advocates to examine whether and how specific findings relate to a
broader theoretical perspective. Regaining a focus on the big picture is the only way forward,
according to Ellemers. Along with this, she places great value on methodological pluralism
and triangulation: examining the same phenomenon from different angles by combining
different research approaches and types of measures: This requires that we more directly
connect highly rigorous (and for some reductionist) experimentation [] with more messy
and ambiguous methods or field observations (to establish external validity). (Ibidem, 4)
Ellemers favours meaningful research and suggests that short papers with single
study observations do not contribute much to the understanding of real-life concerns, such as
the pervasiveness of discrimination, the development and resolution of intergroup conflict, or
the tendency toward suboptimal decision making. Situations in which basic mechanisms
occur should be examined more closely, with an emphasis on contextual variables that are at
stake. Why not make use of earlier grand theories of social psychology: Heider, Festinger,
Tajfel or Allport? Or present-day authors such as Daniel Kahneman or Susan Fiske.
Ellemers concludes her paper with the advice to (a) combine different methods and
measures, (b) writing more overview articles, drawing theoretical lines between research
traditions, and (c) transcend disciplinary boundaries not just towards the harder sciences,
but also towards the humanities and of course the colleagues within the faculties of social
sciences: sociologists, cultural anthropologists, political scientists and economists.
This reflects an earlier statement by one of the grand old men from the field, Robert
Zajonc about the lack of cumulativeness in social psychology: If one were to take any
textbook and randomly reshuffle the chapters, it would matter since there is no compelling
order. So in a century of psychology nothing accumulates. And we have no consensus about
the core of our fields subject matter. (Brannigan, 2004, 11)
This might have to do with the nature of social psychological experiments. According to
sociologist Augustine Brannigan, social psychological experiments are not true experiments
in the sense of the natural sciences. They are not tests in the strict sense designed to compare
outcomes on human subjects in experimental designs based on random assignment to

Abma - Social psychology & Stapel 28 Aug. 2013


different treatment groups. Rather they are demonstrations or dramatizations with a
pedagogical or moral lesson about everyday life. Social psychological experiments also
borrow heavily from commonsense knowledge. According to Brannigan, they simply
reiterate the obvious in abstract methodological form. The experiment gives the sense of
terrific scientific precision in the form of knowledge without actually discovering anything
substantively new. (Brannigan, 2004, 20)
Conclusion
It seems as if social psychology is facing another crisis. The critique in the 1970s for a large
part had a philosophical and ethical nature, and was voiced by people from outside
mainstream psychology. This time, it is experimental psychologists from other subdisciplines
and methodologists who conclude that experimental social psychology lacks epistemological
rigour. At the same time, in the eyes of the public and the media, at least in The Netherlands,
social psychology has lost its credibility as problem-solving branch of social science.
Whether the interdisciplinary, multi-method approach that Ellemers advocates will
succeed depends for a large part on the reward structures within the academic world, which in
the end boils down to finance structures. As long as the emphasis is on so-called fundamental
research and publication in international high ranking journals, and governments and other
parties are willing to keep on subsidizing this type of research, changes will not be very
probable. If on the other hand more emphasis is put on societal relevance of research, social
psychologists will be challenged more to make an effort to increase the ecological validity of
their research which also means: a broader theoretical and methodological scope.

Abma - Social psychology & Stapel 28 Aug. 2013

References
Arkin, R.M. (2011) Most underappreciated. 50 prominent social psychologists describe their
most unloved work. Oxford: Oxford University Press.
Aronson, E. & J.M. Carlsmith (1968) Experimentation in social psychology, in G. Lindzey &
E. Aronson (Eds) The handbook of social psychology, Vol. 2, Reading MA: Addison-Wesley,
1-79.
Brannigan, A. (2004) The rise and fall of social psychology. The use and misuse of the
experimental method. New York: Aldine de Gruyter.
Danziger, K. (2000) Making social psychology experimental. A conceptual history, 19201970, Journal of the History of the Behavioral Sciences, 36, 329-347.
European Association of Social Psychology (2012) [Comment on Levelt report] European
Journal of Social Psychology, 42,
Ellemers, N. (2013) Connecting the dots: Mobilizing theory to reveal the big picture in social
psychology (and why we should do this), European Journal of Social Psychology, 43 (1), 1-8.
Festinger, L. (1953) Laboratory experiments. In: L. Festinger & D. Katz (Eds.) Research
methods in the behavioral sciences, New York: Holt, 136-172.
Greenwood, J.D. (2004) The disappearance of the social in American social psychology.
Cambridge: Cambridge University Press.
Kahneman, D. (2012) Open letter, 26 sept. 2012.
Lange, P.A.M. van, A.W. Kruglanski & E.T. Higgins (Eds.) (2011) Handbook of theories of
social psychology, vol 1 & 2. Sage: Thousand Oaks.
Lange, P.A.M. van, A.P. Buunk, N. Ellemers & D.H.J. Wigboldus (2012) Sharpening
scientific policy after Stapel. ASPO: Internal report.
Levelt Committee (2012) Flawed science: The fraudulent research practices of social
psychologist Diederik Stapel. Tilburg: Tilburg University.
https://www.commissielevelt.nl/wpcontent/uploads_per_blog/commissielevelt/2013/01/finalreportLevelt1.pdf
Morawksi, J. (2012) The importance of history to social psychology. In: A.W. Kruglanski &
W. Stroebe (Eds.) (2012) Handbook of the history of social psychology. New York and
London: Psychology Press, 19-41.
Moscovici, S. & I. Markov (2006) The making of modern social psychology: The hidden
story of how an international social science was created. Cambridge: Polity Press.
Mummendey, A. (2012) Scientific misconduct in social psychology: Towards a currency
reform in science, European Bulletin of Social Psychology, 24 (1), 4-7.

Abma - Social psychology & Stapel 28 Aug. 2013

Parker, I. (1989) The crisis in modern social psychology and how to end it. Londen;
Routledge.
Patnoe, S. (1988) A narrative history of experimental social psychology. The Lewin tradition.
New York: Springer.
Rodrigues, A. & R. Levine (Eds.) (1999) Reflections on 100 years of experimental social
psychology. New York: Basic Books.
Stam, H., Radtke, H. and Lubek, I. (2000) Strains in experimental social psychology: A
textual analysis of the development of experimentation in social psychology, Journal of the
History of the Behavioral Sciences, 36, 36582.
Stapel, D.A. (2000). Moving from fads and fashions to integration: Illustrations from
knowledge accessibility research. European Bulletin of Social Psychology, 12, 4-27.
Stapel, D.A. & S. Lindenberg (2011) Coping with chaos. How disordered contexts promote
stereotyping and discrimination, Science 332 (6026), 251-253.
Strack, F. (2012) The Wow and How of research in social psychology: Causes and
consequences, European Bulletin of Social Psychology, 24 (2), 48.
Stroebe, W. (2012) Open letter in DUB, 5 dec. 2012.
Stroebe, W. (2013) Scientific fraud: Lessons to be learnt, European Bulletin of Social
Psychology, 25 (1), 5-12.
Stroebe, W., T. Postmes & R. Spears (2012) Scientific misconduct and the myth of selfcorrection in science, Perspectives on psychological science, 7, 670-688.
Zajonc, R.B. (1997) One hundred years of rationality assumptions in social psychology. In: A.
Rodrigues & R. Levine (Eds.) (1999) Reflections on 100 years of experimental social
psychology. New York: Basic Books, 200-214.

10

Das könnte Ihnen auch gefallen