Sie sind auf Seite 1von 12

AGGRESSIVE BEHAVIOR

Volume 37, pages 133–144 (2011)

Moral Disengagement in Self-Reported


and Peer-Nominated School Bullying
Marie-Louise Obermann

Department of Psychology, Aarhus University, Aarhus C, Denmark

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
This study examined the relation between moral disengagement and different self-reported and peer-nominated positions in school
bullying. The aims of this study were to (1) investigate moral disengagement among children for whom self-reported and peer-
nominated bully status diverged and (2) compare levels of disengagement among self-reported and peer-nominated pure bullies,
pure victims, bully–victims, and children not involved in bullying. A sample of 739 Danish sixth grade and seventh grade children
(mean age 12.6) was included in the study. Moral disengagement was measured using a Danish version of the Moral
Disengagement Scale and bullying was measured using both self-reports and peer nominations. Results revealed that both self-
reported and peer-nominated bullying were related to moral disengagement, and that both pure bullies and bully–victims displayed
higher moral disengagement than outsiders. Discrepancies between self-reported and peer-nominated bullying involvement
indicates that a person’s social reputation has a stronger association with moral disengagement than so far expected. Implications
are discussed, highlighting the importance of further research and theory development. Aggr. Behav. 37:133–144, 2011. r 2010
Wiley-Liss, Inc.

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

Keywords: moral disengagement; bullying; self-reports; peer-nominations

INTRODUCTION Pershing, 2003; Piquero et al., 2005; Sykes and


Matza, 1957] or the moral hypocrisy paradigm
People often behave contrary to their overall
[Batson et al., 1997, 1999, 2002; Valdesolo and
beliefs without perceiving such behaviors as immoral.
DeSteno, 2007], also demonstrate that legitimization
This paradox plays a pivotal role in understanding
of immoral behavior is commonly used when people
how people are able to commit detrimental conduct
are committing aggressive, antisocial, and criminal
and aggression [Tsang, 2002]. The social cognitive
acts.
theory of moral disengagement offers an explana-
However, so far the specific relation between
tion of how social cognitive mechanisms allow
moral disengagement and different positions in
individuals to engage in self-serving harmful beha-
school bullying has only been investigated to some
viors that contradict their moral principles, whereas
extent [Gini, 2006; Hymel et al., 2005; Menesini
at the same time continuing to advocate these
et al., 2003]. The aim of this study is to further
principles without incurring self-evaluative emotional
investigate and clarify the relation between moral
reactions, such as guilt [Bandura, 1986, 1990, 1999,
disengagement and bullying.
2002].
Many studies have shown that moral disengage-
ment correlates positively with general aggressive- Moral Disengagement
ness toward others [Aquino et al., 2007; Bandura The theory of moral disengagement is understood
et al., 1996, 2001; Carmak and Blatney, 1995; within the frame of social cognitive theory [Bandura,
Grussendorf et al., 2002; McAlister, 2000, 2001;
McAlister et al., 2006; Osofsky et al., 2005; Paciello Correspondence to: Marie-Louise Obermann, Department of
et al., 2008; Pelton et al., 2004; Pornari and Wood, Psychology, Nobelparken, Jens Chr. Skous Vej 4, 8000 Aarhus C,
2010]. Research within theoretically related para- DK. E-mail: mariloui@psy.au.dk
digms, such as the neutralization paradigm [Copes, Received 1 September 2009; Accepted 1 October 2010
2003; Cromwell and Thurman, 2003; Evans and Published online 23 November 2010 in Wiley Online Library (wiley
Porche, 2005; Khoo and Oakes, 2000; Minor, 1984; onlinelibrary.com). DOI: 10.1002/ab.20378

r 2010 Wiley-Liss, Inc.


134 Obermann

1986]. Moral disengagement refers to the use of consequently, something one is not personally
different legitimization mechanisms conducive to a responsible for. Previous research has shown that
selective disengagement of moral censure. According people become willing to commit hurtful acts if a
to Bandura [1999, 2002], the gap between moral legitimate authority takes responsibility for such acts
beliefs and immoral behavior is covered by eight [Bandura et al., 1975; Milgram, 1974; Zimbardo,
different disengagement mechanisms belonging to 2004].
one of four major domains, namely (1) reconstruc- The third domain of disengagement involves
tion of immoral behavior, (2) obscuration of minimizing the negative consequences of one’s
personal responsibility, (3) misrepresentation of transgressions. Distortion of consequences is a
injurious consequences, and (4) blaming the victims disengagement mechanism that operates in this
(illustrated in Fig. 1). domain. Through selective inattention toward the
The first set of disengagement mechanisms focuses negative outcomes of one’s transgressions or by
on the cognitive construal of immoral behavior. focusing on the possible positive outcomes one’s
Detrimental conduct is made personally as well as transgression may produce for recipients, it becomes
socially acceptable through processes of moral easier to convince oneself that an act does not
justification, i.e. by viewing transgressions as a involve immoral aspects.
means to serve higher ends. Moral justification The fourth and final set of disengagement
seems to be widely used in different kinds of mechanisms focuses on the recipients of detrimental
legitimization of harmful behavior [Baumeister and behavior. Mistreating others involves a risk of
Vohs, 2004; Kelman and Hamilton, 1989; Rapoport personal distress. One way of relieving or avoiding
and Alexander, 1982]. By masking transgressions in such distress is through the process of dehumaniza-
euphemisms, immoral activities are downplayed and tion. This mechanism involves viewing victims as
seemingly in agreement with moral standards. subhuman creatures unable to experience feelings
Advantageous comparison means comparing one’s common to ‘‘normal’’ or ‘‘real’’ people. In this way,
own transgressions to worse inhumanities com- the conventional moral codex is suspended, as it
mitted by others; by doing so, one’s own transgres- only applies to ‘‘real people.’’ Previous studies have
sions will seem less harmful or even insignificant. found that dehumanization facilitates harmful
The second set of disengagement mechanisms behavior toward others [e.g. Bandura et al., 1975;
involves personal responsibility through minimiza- Zebel et al., 2008]. Disengagement, focusing on the
tion and obscuration processes. Diffusion of respon- recipients of detrimental behavior, furthermore
sibility refers to how, when part of a group, involves believing or stating that victims themselves
individuals are able to disclaim personal responsi- are to blame for being mistreated [Bar-Tal, 2000;
bility. Displacement of responsibility involves seeing Haslam, 2006; Kelman, 1976]. Attribution of blame
the causes for one’s transgressions as the result of refers to how individuals come to view their
situational pressures or other people’s demands and, transgressions as reactions to provocations or as

Overall Moral
Disengagement Moral Disengagement

Four Disengagement
Domains Reconstructing Obscuring Misrepresenting Blaming the
Immoral Personal Injurious Victims
Behavior Responsibility Consequences

Eight
Dehumanization
Displacement of

Consequences
Advantageous

Responsibility
Responsibility

Attribution of
Distortion of

Disengagement
Euphemistic

Comparison
Justification

Diffusion of
Labeling

Mechanisms
Moral

Blame

Fig. 1. The three levels in the moral disengagement theory.

Aggr. Behav.
Moral Disengagement in Bullying 135

defense against the victim’s initial norm violation. criteria [Roland and Idsøe, 2001]. Bullying is defined
Blame attribution is widely used as a means of as repeated aggression by one or more individuals
legitimization among perpetrators, both in case of who intend to harm or disturb another person
atrocities, such as genocide [see Monroe, 2008] and physically, verbally, or psychologically [Boulton and
in everyday-life situations, i.e. when children who Underwood, 1992; Nansel et al., 2001; Olweus, 1991,
bully others understand their bullying as a reaction 1993; Veenstra et al., 2005; Wolke et al., 2001]. One
to the victim’s initial provocation [Boulton and central way of distinguishing bullying from aggres-
Underwood, 1992]. sion in general, then is by looking at repetition over
By using these eight disengagement mechanisms, time. Unequal power balances, i.e. when victims are
individuals become able to act contrary to their unable to defend themselves from the bully/bullies
moral beliefs while still upholding the experience of [Roland and Idsøe, 2001], constitute another criter-
behaving morally. ion that qualifies aggressive behavior as bullying.
Many studies have shown associations between There are two reasons why the study of moral
high levels of moral disengagement and children’s disengagement in relation to school bullying is
and adolescents’ use of different kinds of aggression particularly interesting. First, because of the
[Bandura et al., 1996, 2001; Carmak and Blatney, unequal power balance, bullying may be considered
1995; Paciello et al., 2008; Pelton et al., 2004]. highly immoral and much more unfair than other
Adolescents with a high moral disengagement kinds of peer aggressions, and it may therefore be
acceptance tend to be more irritable, more prone argued that bullying is a particularly relevant subject
to vengeful rumination, more inclined toward in the study of moral disengagement. This assump-
physical and verbal aggression, and more frequently tion is supported by the fact that some studies
involved in violent episodes. They are also less indicate that the majority of children report negative
troubled by anticipatory feelings of guilt owing to attitudes toward bullying [Boulton and Underwood,
injurious conduct; the higher the moral disengage- 1992; Menesini et al., 1997].
ment acceptance, the weaker the felt guilt caused by Second, studies of moral disengagement in school
aggressive conducts [Bandura et al., 1996]. bullying usually measure bullying based on either
Several studies have found males (adults as well as self-reports [Hymel et al., 2005] or other reports,
children) to be more prone to moral disengagement such as reports from parents, teachers, or peers [e.g.
than females [Bandura et al., 1996; Carmak and Gini, 2006; Malti et al., 2009; Menesini et al., 2003].
Blatny, 1995; Grussendorf et al., 2002; McAlister However, although moral disengagement is shown
et al., 2006]. In their longitudinal study, Paciello et al. to be associated with both other and self-reported
[2008] found that moral disengagement decreases bullying, so far the consistency between other and
during adolescence, with the biggest decrease taking self-reports has not been investigated with regard to
place between 14 and 16 years of age. Thus, age as well moral disengagement. If we are to reach a better
as gender may influence disengagement acceptance. understanding of the social as well as personal
The Moral Disengagement Scale [MDS; developed aspects of moral disengagement, such an investiga-
by Bandura et al., 1996] is a commonly used tool in tion will provide the much-needed information.
empirical studies conducted on the relation between An overview of different studies investigating the
moral disengagement and aggression among school relationships between self-reported and peer-nomi-
children. A number of these studies found that the nated aggression shows moderate correlations ran-
eight mechanisms of moral disengagement can be ging between r 5 .2 and .4 [Caprara and Pastorelli,
traced back to one latent variable, making indivi- 1993; Juvonen et al., 2001; Österman et al., 1994;
duals more or less inclined to employ mechanisms of Pellegrini, 2001; Strohmeier et al., 2008]. The litera-
moral disengagement, with Cronbach’s coefficient ture reflects an ongoing discussion for and against
ranging between .82 and .93 [Bandura et al., 1996, the use of self-reports and peer nominations when
2001; Paciello et al., 2008; Pelton et al., 2004]. measuring aggressive behavior and bullying among
school children. Self-reported bullying might be
biased owing to social desirability which might
Different Contributions From Self-Reports
lead to over- or under-reporting [Menesini and
and Peer Nominations
Camodeca, 2008; Österman et al., 1994; Pellegrini,
All the aforementioned studies investigated moral 2001; Perry et al., 1988]. On the other hand, self-
disengagement in relation to aggression in general. reports are the only way to obtain information
Bullying, however, is understood as a particular about individual experiences of being bullied
kind of aggressive behavior that involves additional [Graham et al., 2007]. Peer nominations might be

Aggr. Behav.
136 Obermann

biased by friendship, misattribution, and prejudice, only differ in disengagement levels from (a) children
and hence social reputations might be resistant to who are neither seen as bullies by themselves nor by
change and children may tend to interpret their their peers, (b) children who are only regarded as
peers’ behavior in the light of experiences lying way bullies by themselves, and (c) children who are
back in time [Graham et al., 2007; Menesini and regarded as bullies by both peers and themselves?
Camodeca, 2008; Peets and Kikas, 2006]. On the Previous research based on either self-reports
other hand, peer nominations are the best way of [Hymel et al., 2005] or peer nominations [Gini,
measuring the reputation of or the attitudes toward 2006; Menesini et al., 2003] suggest that both self-
the subject from the group. Graham et al. [2007] reported and peer-nominated bullying relates to
suggest that both measures be combined when moral disengagement. This study thus hypothesized
studying bullying and victimization, in order to that children, who were either regarded as bullies by
decrease the risk of overlooking important differ- themselves, their peers, or both, would show higher
ences between groups for whom self- and peer views levels of disengagement than children who were not
diverge. Juvonen et al. [2001] argue that self-reports viewed as bullies at all.
measure personal experiences of being an aggressor The second aim of the study was to investigate
or a victim from the child’s personal frame of potential differences in the display of moral disen-
reference, whereas peer nominations reflect the gagement between groups that differ with regard to
social reputation (that is, to which degree peers bully status measured by self-reports and peer
agree on an individual’s social standing as aggressor nominations, respectively. Previous studies on dif-
or victim) and suggest that two different, yet related, ferences between bullies, victims, and outsiders
constructs are measured when using self-reports and based on either self-reports [Hymel et al., 2005] or
peer nominations, respectively. peer-nominations [Gini, 2006; Menesini et al., 2003]
From a moral disengagement perspective, the have consistently found bullies to display higher
combination of peer-nominated and self-reported levels of disengagement than outsiders and victims.
bullying is very interesting. The disengagement This study thus predicted that self-reports and peer
theory states that moral disengagement is based on nominations would show similar differences between
the individual’s own judgments about his or her acts. outsiders, pure bullies, and pure victims. The
It, furthermore, states that these personal moral disengagement theory [Bandura, 1999] is not explicit
standards have been adopted from and are influ- as to how the status of being both aggressor and
enced by the social norms of the surroundings victim affects moral disengagement. Rather, the
[Bandura, 1986, 1999]. However, the theory is not theory broadly assumes that all types of aggressors
explicit as to what happens when a discrepancy will need to disengage morally. Based on this
exists between the individual’s and the surrounding’s assumption, both bully–victims (children who were
evaluation of whether an act is potentially immoral both bullies and victims) and (pure) bullies were
or not. Previous studies support the notion that expected to display higher disengagement levels than
personal standards of right and wrong are influ- pure victims and outsiders.
enced by the norms and acceptance of the surround-
ings [see Zimbardo, 2004]. However, the question
remains to what extent children that are being METHOD
judged as bullies by peers, but who do not see
Participants
themselves as such, would show high moral disen-
gagement. Do they resemble or differ from the Seven hundred and thirty-nine sixth and seventh
children who view themselves as bullies or from the graders from 38 different classes in eight Danish
children who are neither viewed as bullies by their state schools participated in the study. The inclusion
peers nor by themselves? This investigation would criteria were state schools because the majority of
provide us with a better understanding of the social Danish children (about 90%) attend these schools.
aspects of moral disengagement. Private and special schools were excluded from the
project. The participating schools were located in
(higher/lower) middle class urban communities. In
Aims and Hypotheses
all, 677 children filled in a questionnaire. Sixty-two
The first aim of the study was to investigate moral (9%) were absent on the day when the researchers
disengagement with regard to the relation between visited their class, and therefore no self-report
self-reported and peer-nominated bullying. Do measures were collected for these participants,
children who are regarded as bullies by their peers only peer reports. Of the children filling in the

Aggr. Behav.
Moral Disengagement in Bullying 137

questionnaire, 355 were boys and 322 girls. The age disengagement by analyzing children’s reported moral
of the children ranged between 11 and 14 years, the emotions (guilt and shame vs. pride and indifference)
mean age being 12.6 years. while in the hypothetical role of a bully. Hymel
et al. [2005] developed a scale specifically designed
Procedure to measure levels of moral disengagement in bullying
situations, and Gini [2006] applied a MDS developed
The research followed a stringent consent proce-
by Caprara et al. [1995] designed for primary
dure including approvals from headmasters, school
school children. In order to compare findings on
councils, and teachers. Furthermore, parents and
moral disengagement in bullying to studies measuring
children were given the choice of participation or
moral disengagement in overall aggression, this study
nonparticipation. No parents disallowed their chil-
thus aimed at applying the MDS to the field of
dren’s participation in the study. One child chose
bullying.
not to participate owing to severe dyslexia.
The MDS was translated/retranslated from Eng-
Questionnaires were distributed approximately 2
lish to Danish by the author and a colleague. Before
months into the school year. The author and an
finishing the final Danish version of the question-
assistant visited each class for one lesson (45 min).
naire, the translation was discussed with native
Before the questionnaires were handed out, the
English speakers who also spoke Danish. The scale
participants were reseated to prevent them from
consists of 32 items, 4 items measuring each of the
viewing each other’s answers. Tables were rearranged
eight disengagement mechanisms. The scale items
and screens were set up between the tables. The
encompass different kinds of harmful behavior, such
participants were then introduced to the study
as verbal abuse, physically injurious and destructive
and told how to complete the questionnaire. The
conduct, deception and theft in different contexts:
researchers made a point of underlining the fact that
educational, familial, community, and peer relations
no peers, parents, or teachers would have access to the
[Bandura et al., 2001]. Examples of different items
filled-in questionnaires and that the data would be
are: ‘‘It is all right to fight to protect your friends’’
treated anonymously. The reseachers further stressed
(moral justification), ‘‘Damaging property is no big
there being no right or wrong answers, and that
deal when you consider that others are beating
each child’s personal evaluation of his or her current
people up’’ (advantageous comparison), ‘‘If kids
experiences and opinions would be of interest to
fight and misbehave in school, it is their teacher’s
the study.
fault’’ (displacement of responsibility), ‘‘Kids who
During the data collection, teachers were present
get mistreated usually deserve it’’ (attribution of
in the classroom. In case the children had questions
blame), and ‘‘Some people deserve to be treated like
regarding items or regarding how to fill in the
animals’’ (dehumanization) [see Bandura et al.,
questionnaire, the researcher and her assistant dealt
1996].
with those questions. The teachers remained in the
For each of the 32 items, the children rated their
back of the classroom and did not intervene. This
degree of endorsement or repudiation of moral
procedure was followed to ensure that all partici-
exonerations of detrimental conduct. Their
pants were given the same explanation on questions
responses were scored on a 5-point Likert scale,
concerning the same item, and to prevent the teacher
ranging from 1 (strongly disagree) to 5 (strongly
from viewing the children’s answers as promised in
agree). Low scores signify a low level of moral
the introduction.
disengagement acceptance and high scores signify a
high level of moral disengagement acceptance.
Instruments
According to previous studies, the MDS has a high
Moral disengagement. Moral disengagement internal consistency, with a Cronbach’s a reported of
acceptance was measured using a Danish version of .82–.86 [Bandura et al., 1996, 2001; Pelton et al.,
the MDS, originally developed by Bandura et al. 2004]. In this study, Cronbach’s a was .85.
[1996]. The MDS has been used in several studies Self-assessed bullying. This study used a
investigating moral disengagement with regard to Danish abbreviated version of the Norwegian and
overall aggression [e.g. Bandura et al., 1996, 2001; English versions of the Olweus Bully Victim Ques-
Pelton et al., 2004]. However, existing studies on tionnaire (OBVQ) [Olweus, 1996]. The OBVQ is
moral disengagement in bullying employ different a psychometrically sound instrument measuring
methodologies when measuring moral disengagement. two separate aspects of bullying (bullying and
Consequently, the MDS has not been used in bullying victimization), which has been used in many studies
research. Menesini et al. [2003] assessed moral of bullying in different countries [see Kyriakides

Aggr. Behav.
138 Obermann

et al., 2006; Solberg and Olweus, 2003]. The been subjected to bullying themselves. The bully–
questionnaire was translated/retranslated by the victim group (N 5 78) consisted of children who
author and a colleague, and discussed with native both recognized themselves as bullies and felt
Norwegian and English speakers before the final subjected to bullying themselves. The group of
version of the questionnaire was made. outsiders (N 5 387) consisted of children who neither
Before completing the questionnaire, the partici- perceived themselves as bullies nor as victims of
pants were given a definition of bullying stating bullying. When creating the continuous variables,
the intent to harm, the repetition over time, and scores on the ten bully questions and scores on the
the power imbalance between the involved parts. It ten victimization questions, respectively, were
was also emphasized that bullying might include summed. On each scale, scores were ranging from
physical, verbal, as well as relational kinds of 10 (no bullying) to 50 (different kinds of bullying
aggression. happening several times a week).
The initial question of the victim subscale asked Peer-nominated bullying. The children were
for an overall estimation of if and how often the asked to nominate those of their classmates who
children had experienced being victimized since the they currently regarded as bullies and victims,
beginning of the school year (2 months). Subse- respectively: ‘‘In your opinion, which of your
quently, participants were asked whether they had classmates are bullying others/being bullied by
been exposed to different kinds of physical, verbal, others?’’ The children were further asked to describe
or relational bullying. Answers ranged from 1 (low the kind of bullying behavior displayed by the
characteristic/never happened) to 5 (high character- bullies. When the questionnaire was introduced, the
istic/happens several times a week). children had been instructed to put ‘‘nobody’’ if
The initial question of the bully subscale asked for none of the classmates were bullies or victims, in
an overall estimation of how often the children, order to minimize the number of false positive
during the last 2 months, had taken part in bullying answers. In Danish state schools, children spend
other pupils. Again, children were subsequently most of their school hours with the same classmates,
asked whether they had exposed others to different ensuring high familiarity among peers. Thus, peer
kinds of physical, verbal, or relational bullying. reports from classmates were considered a reliable
Answers ranged from 1 (low characteristic/never instrument in this study.
happened) to 5 (high characteristic/happens several The children nominated an average of .96 bullies
times a week). and .83 victims, the minimum number of nomina-
Cronbach’s a for the ten items in the bully tions given for both bullies and victims was 0 and
subscale was .82 and for the ten items in the victim the maximum was 13. For each child, a bully and a
subscale was .83. victim peer nomination score was calculated: the
Both continuous as well as dichotomized variables number of nominations for each child was summed,
for bullying and victimization were created based on divided by the number of classmates completing the
the bullying and the victimization subscales. When questionnaire and multiplied by a hundred. Thus,
creating the dichotomized variables, participants the nomination scores show the percentage of
were assigned to one of four self-reported bully classmates who nominate an individual as a bully
statuses, depending on whether or not the partici- or a victim—the higher the scores, the higher/
pants reported having bullied others or having been stronger the peer reputation(s) as a bully or victim.
victimized themselves two or more times over the The bully nomination scores ranged from 0 to 59%
course of the last 2 months (the initial overall of the classmates, and the mean score was 4.3%. The
question from the bully and victims subscales, victim nomination score ranged from 0 to 65% of
respectively). Being bullied two or more times the classmates, and the mean score was 4.6%.
during the last couple of months has previously Participants were assigned to one of four peer-
been shown to be a reasonable and useful lower nominated bully statuses, depending on whether or
bound cut-off point, when dichotomizing children not two or more of their classmates had nominated
into groups of bullies/victims and nonbullies/non- them as bullies and victims. The pure victims
victims [see Solberg and Olweus, 2003]. The group (N 5 93) group consisted of children who had less
called pure victims (N 5 99) consisted of participants than two bully nominations and two or more victims
who experienced being bullied by others but who did nominations. The pure bullies (N 5 102) group
not identify themselves as bullies. The group of consisted of children who had two or more bully
pure bullies (N 5 101) consisted of children who nominations and less than 2 victim nominations.
recognized having bullied others, but who had not The bully– victim group (N 5 18) consisted of

Aggr. Behav.
Moral Disengagement in Bullying 139

children who had two or more bully and victim Confounding variables. Previous research
nominations. The group of outsiders (N 5 456) shows that age and gender may influence the level
consisted of children who had less than two bully of moral disengagement acceptance [e.g. Bandura
and victim nominations. et al., 1996; Paciello et al., 2008], hence the relation
Categorization based on self/peer consistency. between moral disengagement gender and age was
Participants were assigned to one of four groups, investigated. A one-way analysis of variance revealed
depending on the consistency between self-reported no significant age difference (F(3,659) 5 0.883,
and peer-nominated bullying status. The first group, P 5 .449). An independent sample t-test revealed a
the double identified bullies (N 5 53) constituted significant difference in level of disengagement
individuals viewed as bullies by themselves as well as acceptance among boys (M 5 85.7, SD 5 14.3,
their classmates. The second group, the self-identified N 5 349) and girls (M 5 78.0, SD 5 14.7, N 5 321);
bullies (N 5 127), were those who viewed themselves as t(668) 5 6.82, P 5 .001. According to Cohen (1988),
bullies, but were not viewed as bullies by their the effect size was moderate (d 5 .065). Gender, but
classmates. The third group, the peer-identified bullies not age, was therefore included as a covariate in the
(N 5 67), did not view themselves as bullies; however, subsequent analyses.
they were viewed as bullies by their classmates. The Factor structure of the moral disengagement
fourth group (N 5 425) were neither viewed as bullies scale. In line with the results of earlier research,
by themselves nor by their classmates. Confirmatory Factor Analysis showed that a one
factor structure provided a reasonable fit for the MDS
Missing Data (S-B w2(464) 5 1,671.72, P 5 .000, RMSEA 5 0.067,
90% CI [.064–.070], CFI 5 .85, SRMR 5 .88).
Participants answering less than 90% of items on Dependent and independent variables. Accord-
a scale were excluded from the subsequent analyses. ing to the theory of moral disengagement, the
Based on this criterion, five participants were relationship between moral disengagement and detri-
excluded from the bully subscale, one was excluded mental behavior develops in a constant self-regulatory
from the victim subscale, and seven from the MDS. interplay between (1) a person’s behavior, (2) the
In the rest of the sample, missing data were imputed conditions under which it occurs, (3) the judgment of
using the Expectation Maximization algorithm, a the behavior in relation to moral principles, and (4)
widely used and well-accepted method for dealing perceived circumstances as well as the consequences the
with missing data [Shafer and Graham, 2002]. behavior may have for the individual [Bandura, 1986,
1999]. Thus, it is not possible to claim that disengage-
ment per se anticipates detrimental behavior or vice
RESULTS versa. This indicates that disengagement and bullying
can both be used as dependent and independent
Preliminary Analysis
variables. Depending on the purpose of the studies used,
Validation of sample. In order to investigate previous research has both employed moral disengage-
whether there was a difference in the number of bully ment [e.g. Hymel et al., 2005] and aggression [e.g.
and victim nominations between the group of children Bandura et al., 1996, 2001] as dependent variables. In the
being present in class (N 5 677) and the children being subsequent analyses, however, moral disengagement was
absent from school on the day of completing the chosen as the dependent variable, because the purpose of
questionnaires (N 5 62), a chi-square for independence the study was to investigate whether moral disengage-
was conducted. No significant relation was found ment levels differ among different groups of bullies
between number of bully and victim nominations and and victims.
absence from school (Bully: Pearson chi 5 0.003,
Po.998, df 5 2, V 5 .034, P 5 .998, N 5 739. Victim:
Moral Disengagement in Self-Reported and
Pearson chi 5 2.071, df 5 2, P 5 .335, V 5 .053,
Peer-Nominated Bullying
P 5 .335, N 5 739) indicating that absentees are not
particularly involved in bullying (at least from their The correlations between moral disengagement
peers’ point of view). This finding suggests that the and the bullying variables are shown in Table I.
absentees might be missing at random. The finding also In order to further investigate the relation between
indicates that the children do not forget to nominate peer-nominated and self-reported bullying on the one
those of their classmates who are absent in the test hand and victimization and moral disengagement on
situation. Thus, bias owing to the absence does not the other, a standard multiple regression analysis was
seem to pose a problem in this study. carried out. Both peer-nominated and self-reported

Aggr. Behav.
140 Obermann

TABLE I. Correlations Between Moral Disengagement, Peer-Nominated and Self-reported Bullying and Victimization

Peer-nominated Peer-nominated Self-reported Overall moral


victimization bullying victimization disengagement

Peer-nominated victimization .028 .341 .033


Peer-nominated bullying .028 .082 .200
Self-reported victimization .341 .082 .158
Self-reported bullying .131 .305 .474 .280
Pearson correlation is significant at the .05 level (two-tailed).
Pearson correlation is significant at the .01 level (two-tailed).

TABLE II. Means and Standard Deviations for Boys and Girls on Overall Moral Disengagement Across Four Categories of
Self/Peer Consistency

Girls Boys Total

Self/peer consistency Mean (SD) N Mean (SD) N Mean (SD) N

Double assigned bully 87.25 (20.45) 12 91.28 (10.29) 40 90.35 (13.19) 52


Self bully only 86.57 (13.17) 49 87.15 (13.98) 78 86.93 (13.63) 127
Peer bully only 80.46 (17.9) 26 87.98 (14.88) 41 85.06 (16.41) 67
Not assigned bully 75.44 (13.48) 234 82.98 (14.61) 185 78.77 (14.46) 419
Total 77.99 (14.75) 321 85.48 (14.32) 344 81.87 (14.99) 665

TABLE III. Means and Standard Deviations in Overall Moral Disengagement for Boys and Girls in Four Self-Reported and
Peer-Nominated Bullying Positions

Girls Boys Total

Status Mean (SD) N Mean (SD) N Mean (SD) N

Self-status Pure bully 86.27 (15.478) 26 88.88 (13.661) 75 88.21 (14.118) 101
Pure victim 79.45 (14.141) 55 88.55 (15.799) 44 83.49 (15.502) 99
Bully-victim 87.03 (14.282) 35 87.98 (11.750) 43 87.55 (12.869) 78
Outsider 75.00 (13.874) 205 82.75 (14.306) 182 78.65 (14.584) 387
Total 77.99 (14.745) 321 85.48 (14.315) 344 81.87 (14.989) 665
Peer status Pure bully 81.90 (18.680) 29 88.47 (12.053) 73 86.60 (14.463) 102
Pure victim 80.19 (15.124) 43 83.88 (14.182) 50 82.17 (14.662) 93
Bully-victim 98.78 (15.377) 9 98.78 (15.377) 9 91.83 (18.684) 18
Outsider 84.57 (14.684) 240 84.57 (14.648) 217 80.52 (17.733) 457
Total 77.99 (14.745) 321 85.65 (14.316) 349 81.98 (15.009) 670

bullying displayed significant relations to overall moral (F(3) 5 2.0, P 5 .112, Z2 5 .009). Bonferroni post
disengagement (Peer: b 5 0.240, SE 5 .07, b 5 .133, hoc comparisons revealed that the group of children,
P 5 .001. Self: b 5 1.153, SE 5 .234, b 5 .217, P 5 .000), who were neither seen as bullies by their peers nor
whereas self-reported and peer-nominated victimi- by themselves, had a significantly lower disengage-
zation did not (Self: b 5 0.178, SE 5 .161, b 5 .049, ment level than the three groups who were viewed as
P 5 .267. Peer: b 5 0.013, SE 5 .06, b .008, P 5 .83). bullies by themselves, their classmates, or both.
The next step was to investigate possible differences in
moral disengagement between self-reported and peer-
Moral Disengagement in Different Bully
nominated bullying. Means and standard deviations for
Statuses
boys and girls in the four groups of self-peer consistency
are shown in Table II. A four (self/peer consistency) by Table III shows mean and standard deviations in
two (gender) analysis of variance was conducted. overall moral disengagement between boys and girls
A significant main effect for both self/peer consistency in the four self-reported and the four peer-nominated
(F(3) 5 13.9, P 5 .000, Z2 5 .060) and gender (F(1) 5 9.1, bully statuses.
P 5 .014, Z2 5 .014) occurred. There was no inter- A two (gender) by four (self-reported bully status)
action effect between gender and self/peer evaluation between groups analysis of variance was conducted

Aggr. Behav.
Moral Disengagement in Bullying 141

to investigate the relation between self-reported The findings of the study showed moral disen-
bully status and level of moral disengagement. gagement to be related to both the personal
A statistically significant main effect occurred for experience of being directly involved in bullying
self-reported bullying status (F(3,567) 5 14.86, and peer nomination of someone as a bully. Most of
P 5 .000, Z2 5 .06). The analysis also revealed a the findings confirmed the initial hypotheses. How-
statistically significant main effect for gender ever, the study yielded a few unexpected findings.
(F(1.567) 5 13.535, P 5 .000, Z2 5 .02), indicating Peer-identified bullies displaying the same levels of
that boys had a higher moral disengagement disengagement as self-identified bullies proves parti-
acceptance. The interaction effect between gender cularly interesting. Not only does this finding
and self-reported bully status was not significant suggests that attitudes present in the ambient
(F(3.657) 5 2.01, P 5 .110). Multiple comparisons, environment relate indirectly to moral disengage-
using the Bonferroni post hoc test, indicated a ment through the influence of an individual’s self-
significantly lower disengagement score for the evaluation of whether she/he is a bully, as suggested
outsiders than the scores of all the actively involved by the disengagement theory [Bandura, 1999]; the
groups. No significant difference was detected finding further indicates that moral disengagement
between the three actively involved groups, indicat- directly relates to the peer reputation of being a
ing that bullies, victims, and bully–victims did not bully, and that individuals may need to legitimize
differ in level of overall disengagement acceptance. aggressive behavior even in situations where they
To investigate the impact of peer-nominated themselves do not experience that they are acting
bullying status and gender on moral disengagement immorally. Future research and theory development
acceptance, a two (gender) by four (peer-nominated might seek a better understanding of the social
bully status) between groups analysis of variance influence on moral disengagement, in circumstances
was conducted. Results showed a statistically when the individual does not him- or herself
significant main effect for peer-nominated bullying recognize his or her own behavior as immoral or
status (F(3) 5 5.318, P 5 .001, Z2 5 .024) and gender as being likely to be interpreted by others as
(F(1) 5 15.207, P 5 .000, Z2 5 .022). The interaction immoral.
effect between gender and peer-nominated bullying In accordance with previous findings [e.g. Caprara
status was not significant (F(3) 5 0.841, P 5 .472, and Pastorelli, 1993; Strohmeier et al., 2008;
Z2 5 .004). Multiple comparisons, using the Bonfer- Österman et al., 1994], this study not only found
roni post hoc test, indicated that the peer-nominated that some convergence between self- and peer-
outsider group had a significantly lower disengage- experienced bullying exists, but also demonstrated
ment score than the peer-nominated pure bullies and that different points of view come into play.
bully–victims. No significant difference between the As suggested by Juvonen et al. [2001], a possible
three actively involved groups occurred, indicating explanation of this self/peer discrepancy might be
that pure bullies, pure victims, and bully–victims did found in the observation that self-reports and peer
not differ in level of overall disengagement accep- nominations measure different, yet related, con-
tance. structs (private experience/social reputation) which
will make them correlate, however, only to a limited
extent.
In the analysis conducted on the four bully
statuses findings on both peer-nominated and self-
DISCUSSION
reported data supported the initial hypothesis that
This study investigated the relation between pure bullies and bully–victims would be more prone
different kinds of bullying involvement on moral to moral disengagement than outsiders. Based on
disengagement acceptance. The main findings of the theoretical assumption that individuals behaving
the study were (1) that peer-identified and aggressively will experience the need to disengage
self-identified bullies displayed the same levels of whether or not they themselves are simultaneously
moral disengagement, (2) that children identified victimized [Bandura, 1999], bullies and bully–victims
as bullies by either themselves, their peers, or were not expected to differ significantly from each
both displayed higher disengagement levels other with regard to moral disengagement. Thus, the
than the children who were neither viewed as bullies findings of this study were in line with the
by themselves nor by their peers, and (3) that pure disengagement theory.
bullies and bully–victims displayed higher levels of The regression analysis conducted for continuous
moral disengagement than the outsiders. data confirmed the initial hypothesis that bullying

Aggr. Behav.
142 Obermann

was associated with moral disengagement, whereas ACKNOWLEDGMENTS


(self-reported as well as peer-nominated) victimiza-
I gratefully acknowledge the participation of the
tion was not associated with moral disengagement
students, teachers, and headmasters who made this
when controlling for bullying. However, some of the
project possible. I also thank Bo Sommerlund
findings regarding moral disengagement among pure
(Aarhus University) and Mark Shevlin (University
victims were neither consistent with previous re-
of Ulster) for statistical advice, and Ole Steen
search [Gini, 2006; Menesini et al., 2003] nor the
Kristensen (Aarhus University) for helpful com-
initial hypothesis that pure victims would display
ments.
significantly lower levels of disengagement than pure
bullies and bully–victims: Neither the self-reported
nor the peer-nominated pure victims displayed
significantly lower levels of moral disengagement REFERENCES
than the pure bullies as expected, and the self- Aquino K, Reed A, Thau S, Freeman D. 2007. A grotesque and dark
reported pure victims even showed significantly beauty: How moral identity and mechanisms of moral disen-
higher disengagement levels than the outsiders. To gagement influence cognitive and emotional reactions to war.
J Exp Soc Psychol 43:385–392.
further test the possibility that recipients of aggres- Bandura A. 1986. Social Foundations of Thought and Action:
sion need to disengage, future research may aim at A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice-Hall.
investigating moral disengagement among recipients Bandura A. 1990. Selective activation and disengagement of moral
of aggressive behavior. control. J Soc Issues 46:27–46.
There were some limitations in this study. The Bandura A. 1999. Moral disengagement in the perpetration of
inhumanities. Pers Soc Psychol Rev 3:193–209.
cross-sectional design of the research made it Bandura A. 2002. Selective moral disengagement in the exercise of
difficult to interpret the causal relation between moral agency. JME 31:101–119.
moral disengagement and bullying; it is impossible Bandura A, Underwood B, Fromson ME. 1975. Disinhibition of
to determine whether bullying precedes moral aggression through diffusion of responsibility and dehumaniza-
disengagement or vice versa. Future research will tion of victims. J Res Pers 9:253–269.
Bandura A, Barbarnelli C, Caprara GV, Pastorelli C. 1996. Mechan-
need to aim at longitudinal investigations of the isms of moral disengagement in the exercise of moral agency. J Pers
theoretically assumed reciprocal interplay between Soc Psychol 71:364–374.
moral disengagement and bullying over time for a Bandura A, Caprara GV, Barbarnelli C, Pastorelli C, Regalia C.
clarification of the causal effects. 2001. Sociocognitive self-regulatory mechanisms governing
Moreover, when interpreting the results of the transgressive behavior. J Pers Soc Psychol 80:125–135.
Bar-Tal D. 2000. Shared Beliefs in a Society: Social Psychological
study, the possible biases involved in the self- Analysis. Thousand Oaks, CA: Sage.
reported as well as the peer-reported measures must Batson CD, Kobrynowitcz D, Dinnerstein J, Kampf HC, Wilson AD.
be considered possible limitations. Social desirability 1997. In a very different voice: Unmasking moral hypocrisy.
and other self-report biases may influence the J Pers Soc Psychol 72:1335–1348.
so-called outsiders, making them underreport their Batson CD, Thompson ER, Seuferling G, Whitney H, Strongman JA.
1999. Moral hypocrisy: Appearing moral to oneself without being
actual disengagement acceptance as well as their
so. J Pers Soc Psycholy 77:525–537.
actual involvement in active bullying and victimiza- Batson CD, Thompson ER, Chen H. 2002. Moral hypocrisy:
tion. Likewise, the peer nomination measure may be Addressing some alternatives. J Pers Soc Psychol 83:330–339.
biased by reputations, likes, and dislikes, as well as Baumeister RF, Vohs KD. 2004. Four roots of evil. In: Miller AG
the wish not to speak ill of one’s friends. (ed.). The Social Psychology of Good and Evil. New York:
Guilford Press, pp 85–101.
In summary, this study contributes significantly to
Boulton MJ, Underwood K. 1992. Bully/victim problems among
the field of research on moral disengagement and middle school children. Br J Educ Psychol 62:73–87.
bullying. It is the first published study on moral Caprara GV, Pastorelli C. 1993. Early emotional instability,
disengagement that considers discrepancies between prosocial behaviour, and aggression: Some methodological
self-reported and peer-nominated bullying. The aspects. Eur J Psychol 7:19–36.
Caprara GV, Pastorelli C, Bandura A. 1995. La misura del
findings indicate that a person’s social reputation
disimpegno morale in età evolutiva (Measuring age differences
has a more direct influence on moral disengagement in moral disengagement). Età Evol 51:18–29.
than so far theorized. Carmák I, Blatný M. 1995. Personality indicators of aggression and
The findings of the study further point to the need moral disengagement. Studia Psychologica 37:199–201.
for future research to consider using a combination Cohen JW. 1988. Statistical Power Analysis for the Behavioural
Sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates.
of peer nominations and self-reports when investi-
Copes H. 2003. Societal attachments, offending frequency, and
gating moral disengagement in relation to peer techniques of neutralization. Deviant Behav 24:101–127.
aggression and bullying, as the two methods offer Cromwell P, Thurman Q. 2003. The devil made me do it: Use of
slightly different contributions. neutralizations by shoplifters. Deviant Behav 24:535–550.

Aggr. Behav.
Moral Disengagement in Bullying 143

Evans RD, Porche DA. 2005. The nature and frequency of Minor W. 1984. Neutralization as a hardening process: Considera-
medicare/medicaid fraud and neutralization techniques among tions in the modelling of change. Social Forces 62:995–1019.
speech, occupational, and physical therapists. Deviant Behav Monroe KR. 2008. Cracking the code of genocide: The moral
26:253–270. psychology of rescuers, bystanders, and Nazis during the
Gini G. 2006. Social cognition in bullying: What’s wrong? Aggr holocaust. Polit Psychol 29:699–736.
Behav 32:528–539. Nansel TR, Overpeck M, Pilla RS, Ruan WJ, Simons-Morton B,
Graham S, Bellmore A, Juvonen J. 2007. Peer victimization in Scheidt P. 2001. Bullying behaviors among US youth: Prevalence
middle school: When self- and peer views diverge. In: Zins JE, and association with psychosocial adjustment. J Am Med Assoc
Elias MJ, Maher CA (eds). Bullying Victimization, and Peer 285:2094–2100.
Harassment. A Handbook of Prevention and Intervention. Olweus D. 1991. Bully/victims problems among school children:
New York: The Hayworth Press, pp 121–141. Basic facts and effects of a school bases intervention program. In:
Grussendorf J, McAlister A, Sandström P, Udd L, Morrison TC. Pepler D, Rubin K (eds). The Development and Treatment of
2002. Resisting moral disengagement in support for war: Use of Childhood Aggression. Hillsdale, NJ: Lawrence Erlbaum Associ-
the ‘‘Peace Test’’ scale among student groups in 21 nations. ates, pp 411–448.
J Peace Psychol 8:73–83. Olweus D. 1993. Bullying at School: What We Know and What We
Haslam N. 2006. Dehumanization: An integrative review. Pers Soc Can Do. Malden, MA: Blackwell.
Psychol Rev 10:252–264. Olweus D. 1996. The Revised Olweus Bully/Victim Questionnaire.
Hymel S, Rocke-Henderson N, Bonanno RA. 2005. Moral disen- Mimeo.Bergen, Norway: Research Centro for Health Promotion
gagement: A framework for understanding bullying among (HEMIL center) University of Bergen.
adolescents. J Soc Sci 8:1–11. Osofsky MJ, Bandura A, Zimbardo PG. 2005. The role of moral
Juvonen J, Nishina A, Graham S. 2001. Self-views versus peer disengagement in the execution process. Law Hum Behav
perceptions of victims status among early adolescents. In: 29:371–393.
Juvonen J, Graham S (eds). Peer Harassment in Schools. The Österman K, Björkqvist K, Lagerspetz KMJ, Kaukiainen A,
Plight of the Vulnerable and Victimized. New York: The Huesmann LR, Fraczek A. 1994. Peer and self-estimated
Guilford Press, pp 105–124. aggression and victimization in 8-year-old children from five
Kelman HC. 1976. Violence without restraint: Reflections on the
ethnic groups. Aggr Behav 20:411–428.
dehumanization of victims and victimizers. In: Kren GM,
Paciello M, Fida R, Tramontano C, Lupinetti C, Caprara GV. 2008.
Rappoport LH (eds). Varieties of Psychohistory. New York:
Stability and change of moral disengagement and its impacts
Springer, pp 282–314.
on aggression and violence in late adolescence. Child Dev
Kelman HC, Hamilton VL. 1989. Crimes of Obedience: Toward a
79:1288–1309.
Social Psychology of Authority and Responsibility. New Haven,
Peets K, Kikas E. 2006. Aggressive strategies and victimization
CT: Yale University Press.
during adolescence: Grade and gender differences, and cross-
Khoo ACE, Oakes PJ. 2000. The variability of the delinquent self:
informant agreement. Aggress Behav 32:68–79.
Anti-authority attitudes and endorsement of neutralizations
Pellegrini AD. 2001. Sampling instances of victimization in
techniques among incarcerated delinquents in Singapore. Asian
middle school. A methodological comparison. In: Juvonen J,
J Soc Psychol 3:125–132.
Graham S (eds). Peer Harassment in Schools. The Plight of the
Kyriakides L, Kaloyirou C, Lindsay G. 2006. An analysis of the
revised Olweus bully/victim questionnaire using the Rasch Vulnerable and Victimized. New York: The Guilford Press,
measurement model. Br J Educ Psychol 76:781–801. pp 125–144.
Lagerspetz K, Björkqvist K, Berts M, King E. 1982. Group Perry DG, Kusel SJ, Perry LC. 1988. Victims of peer aggression. Dev
aggression among school children in three schools. Scand J Psychol 24:807–814.
Psychol 23:45–52. Pershing JL. 2003. To snitch or not to snitch? Applying the concept
Malti T, Gasser L, Buchmann M. 2009. Aggressive and prosocial of neutralization techniques to the enforcement of occupational
children’s emotion attributions and moral reasoning. Aggr Behav misconduct. Soc Perspec 46:149–178.
35:90–102. Piquero NL, Tibbetts SG, Blankenship MB. 2005. Examining
McAlister A. 2000. Moral disengagement and opinions on war with the role of differential association and techniques of neu-
Iraq. Int J Public Opin Res 12:191–198. tralization in explaining corporate crime. Deviant Behav
McAlister A. 2001. Moral disengagement: Measurement and 26:159–188.
modification. J Peace Res 38:87–99. Pelton J, Gound M, Forehand R, Brody G. 2004. The moral
McAlister AL, Bandura A, Owen SV. 2006. Mechanisms of moral disengagement scale: Extension with an American minority
disengagement and support for military action: The impact of sample. J Psychopathol Behav Assess 26:31–39.
Sept. 11. J Soc Clin Psychol 25:141–165. Pornari CD, Wood J. 2010. Peer and cyber aggression in secondary
Menesini E, Camodeca M. 2008. Shame and guilt as behaviour school students: The role of moral disengagement, hostile
regulators: Relationships with bullying, victimization and proso- attribution bias, and outcome expectancies. Aggr Behav 36:
cial behaviour. Br J Dev Psychol 26:183–196. 81–94.
Menesini E, Eslea M, Smith PK, Genta ML. 1997. Cross-national Rapoport DC, Alexander Y. 1982. The Morality of Terrorism: Religious
comparison of children’s attitudes towards bully/victim problems and Secular Justification. Elmsford, New York: Pergamon Press.
in schools. Aggr Behav 23:245–257. Roland E, Idsøe T. 2001. Aggression and bullying. Aggr Behav
Menesini E, Sanchez V, Fonzi A, Ortega R, Costable A, Feudo GL. 27:446–462.
2003. Moral emotions and bullying: A cross-national comparison Shafer JL, Graham JW. 2002. Missing data: Our view of the state of
of differences between bullies, victims and outsiders. Aggr Behav the art. Psychol Methods 7:147–177.
29:15–530. Solberg ME, Olweus D. 2003. Prevalence estimation of schools
Milgram S. 1974. Obedience to Authority. An Experimental View. bullying with the Olweus bully/victim questionnaire. Aggr Behav
New York: Harper & Row. 29:239–268.

Aggr. Behav.
144 Obermann

Strohmeier D, Spiel C, Grandinger P. 2008. Social relationships in in elementary schools: A comparison of bullies, victims, bully/
multicultural schools: Bullying and victimization. Eur J Dev victims and uninvolved preadolescents. Dev Psychol 41:
Psychol 5:262–285. 672–682.
Sykes GM, Matza D. 1957. Techniques of neutralization: A theory Wolke D, Woods S, Stanford K, Schulz H. 2001. Bullying and
of delinquency. Am Sociol Rev 22:664–670. victimization of primary school children in England and Germany:
Tsang JA. 2002. Moral rationalization and the integration of Prevalence and school factors. Br J Psychol 92:673–696.
situational factors and psychological processed in immoral Zebel S, Zimmermann AJ, Viki GT, Doosje B. 2008. Dehumaniza-
behavior. Rev Gen Psychol 6:25–50. tion and guilt as distinct but related predictors of support for
Valdesolo P, DeSteno D. 2007. Moral hypocrisy. Social groups and reparation polices. Polit Psychol 29:193–219.
the flexibility of virtue. Psychol Sci 18:689–690. Zimbardo PG. 2004. A situationist perspective on the psychology of
Veenstra R, Lindenberg S, Oldehinkel AJ, de Winter AF, evil. In: Miller A (ed.). The Social Psychology of Good and Evil.
Verhulst FC, Ormel J. 2005. Bullying and victimizations New York: The Guilford Press, pp 51–84.

Aggr. Behav.

Das könnte Ihnen auch gefallen