Motiv Emot (2011) 35:251266
DOI 10.1007/s11031-010-9175-8
ORIGINAL PAPER
Assessing implicit motives with an online version
of the picture story exercise
Katharina Bernecker Veronika Job
Published online: 13 August 2010
Springer Science+Business Media, LLC 2010
Abstract An online version of a Picture Story Exercise
(PSE) was administered to 180 participants to assess
implicit motives for achievement, affiliation, and power.
First, data were analyzed with respect to different aspects
of participant behavior (dropout, writing time, story length,
setting). Second, online data were compared with data
collected in the laboratory (n = 93). Results regarding
motive density, internal consistency, picture profiles, and
position effects were comparable among samples and
consistent with previous research (Pang and Schultheiss in
J Person Assess 85:280294, 2005; Schultheiss and Brun-
stein in J Person Assess 77: 7186, 2001). In addition,
online participants completed self-report measures assess-
ing explicit motives and the Big Five personality factors.
Explicit motives showed no substantial overlap with PSE
motive scores, replicating previous research. Personality
factors revealed gender differences in correlates: men with
a high implicit power motive reported less agreeableness
and conscientiousness. Results suggest that administering
the PSE online yields reliable and valid data.
Keywords Implicit motives
Picture story exercise

Internet
Assessment
K. Bernecker (&)
Department of Psychology, University of Erfurt,
Postfach 900221, 99105 Erfurt, Germany
e-mail: katharina.bernecker@stud.uni-erfurt.de
K. Bernecker
V. Job
Department of Psychology, Stanford University,
Stanford, CA, USA
Introduction
In the past few years, the Internet became an important tool
for research in psychology, as indicated by a rapid
expansion of web-based research.1 The Internet offers
researchers the opportunity to gather information from a
larger number of individuals, without expensive lab
assessment sessions that require laboratory space, equip-
ment, and personnel hours from the researchers, and time
and travel from participants (Birnbaum 2004a; Buchanan
2000; Reips 2002a). Particularly psychologists involved in
the assessment of personality attributes have profited from
Internet administration of questionnaires. It is easy to post
items from personality inventories on web-pages to be
completed anonymously and in a variety of settings con-
venient to participants all around the world (Buchanan
et al. 2005; Buchanan 2001; Gosling et al. 2004).
An open question remains whether the Internet can be a
valuable platform to measure implicit motives, aspects of
personality, which cannot be assessed using self-report
measures. Instead implicit motives are usually measured by
Picture Story Exercise (PSE; as the Thematic Apperception
Test, TAT; Murray 1943) a projective measure which asks
participants to write imaginative stories as open-ended
responses to a set of picture cues.
Implicit motives
Implicit motives are defined as enduring preferences for
specific classes of affective incentives (McClelland 1985;
Schultheiss 2008). They provide the motivational resource
1
For a recent list of studies conducted online see the American
Psychological Society (APS) experiment list maintained by John
Krantz: http://psych.hanover.edu/research/exponnet.html.
123
252
to strive for motive-congruent goal states as they orient,
select, and energize behavior directed to a motive specific
end state and a corresponding affective experience. Three
motives came to be seen as primary and were called by
McClelland (1985) the big three: the achievement
motive, the affiliation motive, and the power motive. The
achievement motive represents the need (n) for the expe-
rience of accomplishing something difficult, attaining a
high standard, or excelling (nAchievement). The affiliation
motive is the need for the establishment and maintenance
of positive relationships with others (nAffiliation). The
power motive in the need for the experience of having an
impact on other people by influencing and directing their
behavior (nPower). Implicit motives develop very early in
the preverbal phase of life via affective, not verbally pro-
cessed experiences (McClelland and Pilon 1983), which is
why they are not accessible to introspection.
Implicit motives prompt spontaneous, self-initiated
behavior and, therefore, predict long-term behavioral pat-
terns (Brunstein and Maier 2005; McClelland et al. 1989;
Spangler 1992). Engagement in behavior that is congruent
with implicit motives leads to positive affective experi-
ences which, in turn, may accumulate to a higher level of
emotional well-being. Accordingly, people who progress
towards goals congruent with their implicit motives report
higher levels of emotional well-being than people whose
goal striving is not in line with their implicit motives
(Brunstein et al. 1998). Chronic frustration of implicit
motives has been shown to constitute a stressor which is
associated with negative affect and which has negative
effects on physical health (e.g., McClelland and Jemmott
1980).
In the last two decades, researchers became increasingly
interested in the relation of implicit motives and explicit
motives (exAchievement, exAffiliation and exPower)
which are assessed via self-report measures (McClelland
et al. 1989; Spangler 1992). There is abundant evidence
that implicit motives are largely independent from explicit
motives (e.g., deCharms et al. 1955; King 1995; Schul-
theiss et al. 2009; Spangler 1992; but see Emmons and
McAdams 1991; Thrash et al. 2007, for correlation
depending on the assessment procedure). Empirical find-
ings suggest that explicit motives, as opposed to implicit
motives, shape short-term respondent behavior that is
performed self-consciously in response to the extrinsic
demands of a given situation. The low correlation between
implicit and explicit motives and the different ways they
shape behavior led to the development of a two-systems
model of motivation which defines implicit and explicit
motives as two distinct constructs with different predictive
validity (McClelland et al. 1989). Discrepancies between
implicit and explicit motives (e.g., someone high on nPo-
wer but low on exPower) opened a new field of research.
123
Motiv Emot (2011) 35:251266
Implicit/explicit motive discrepancies were found to be
related to negative outcomes in terms of health and well-
being (Baumann et al. 2005), self-regulation (Kehr 2004),
and health behavior (Job et al. in press).
Research on implicit motives, and hence the use of PSE
as a diagnostic tool, has experienced a renaissance during
the last 10 years (Schultheiss and Brunstein 2010). The
role of implicit motives has been investigated in fields such
as affective neuroscience and hormone release (Schultheiss
et al. 2008b; Stanton and Schultheiss 2009), memory
(Woike et al. 2001; Woike et al. 2003), health and health
behavior (Baumann et al. 2005; Job et al. in press), and
personal goal striving (Brunstein et al. 1998).
Assessment of implicit motives with the PSE
The PSE is a projective measure in which participants write
imaginative stories in response to a set of usually 48
picture cues. The pictures show people in different situa-
tions, for instance a ship captain talking to a passenger or
an architect working in an office. Each picture is presented
separately and participants are instructed to make up a
short story about each of them within a certain amount of
time. The PSE was traditionally administered individually
or in small groups using a paperpencil format. During the
session an experimenter had to be present to show the
pictures and control writing time. Today computer-based
administration is more common with no experimenter
being present during the session. In this case, stories are
typed directly into the computer via keyboard.2 Stories are
later coded for specific content which indicate a persons
implicit motive disposition.
In the past, there has been a great deal of debate
regarding the psychometric properties of the PSE. Past
research found the instructions to be a crucial factor to
increase the PSEs testretest reliability (Lundy 1981,
1985; Winter and Stewart 1977). In a recent meta-analysis,
the average testretest reliability of the PSE was satisfac-
tory and the rate of decrease with increasing retest intervals
was similar to that observed for questionnaire trait mea-
sures (Schultheiss and Pang 2007). Picture motive profiles,
which is the amount of achievement, affiliation and power
motive imagery for an individual picture cue, has also been
demonstrated to be stable between two testing sessions
spaced 2 weeks apart (Schultheiss et al. 2008a).
2
Typed stories were found to be by a third longer than handwritten
ones, but when corrected for word count the two methods of
administration produce comparable motive scores (Blankenship and
Zoota 1998; Schultheiss et al. 2008a).
Motiv Emot (2011) 35:251266
The PSE on the Internet
Researchers on implicit motives generally face a variety of
costs and hurdles accompanying the application of PSE.
Although computer-based administration in the laboratory
decreases time for otherwise necessary transcription and
word counting tasks, the administration of the PSE is
nevertheless time-consuming for researchers and partici-
pants. This aspect might be particularly disadvantageous
when non-student populations are desired, such as man-
agers or middle-aged adults, leading researchers to choose
more economical procedures than the PSE (e.g., Job et al.
in press; Kehr 2004).
Although administration of the PSE on the Internet does
not eliminate all costs involved in PSE assessment (i.e.,
participants still have to write stories, which still have to be
manually coded) the Internet can substantially reduce some
costs and offer new benefits, particularly with respect to
limit the need for laboratory sessions, as well as to provide
access to larger and non-student populations. For instance,
researchers could administer the PSE online before par-
ticipants are invited to the lab, which leaves more time for
manipulation and measurements and allows for the poten-
tial to invite only participants who score high on certain
motive dispositions.
However, Internet assessment of implicit motives with
the PSE also raises a variety of challenges. The most
serious problem involves the lack of experimental control
over the testing situation. Since implicit motives are
aroused by and therefore sensitive to incentives in the
surrounding environment, a standardized, neutral testing
situation was often seen to be a precondition for the valid
assessment of implicit motives (Klinger 1968; Schultheiss
and Pang 2007; Smith 1992). Further, distractions or
motivational issues (Buchanan and Smith 1999) can neg-
atively affect the flow of imagery and participants
involvement in story writing. Finally, since the PSE is time
consuming and requires high amounts of cognitive effort
and involvement from the participant, participants who do
not want to put forth cognitive effort may self-select out of
the study. This could occur in two ways. First, people could
decline to participate in such a long study. Second, par-
ticipants could decide to drop out over the course of the
procedure (Birnbaum 2004b). Therefore, participants
behavior in online PSE assessment and properties of the
resulting PSE data have to be examined before implicit
motives can be reliably assessed on the Internet.
Overview of this study
The aim of the present research was to create a web-version
of the PSE and compare data collected via the Internet with
data collected in the laboratory. To make the two settings
253
comparable lab participants completed the PSE on the
computer in separated cubicles and with no experimenter
present. Comparing the two data sets we focused on the
following questions: participant behavior and setting (e.g.,
drop-out rate), motive density (amount of motive imagery
in relation to story length), internal consistency (stability of
motive imagery across picture cues), picture motive pro-
files (amount of nAchievement, nAffiliation and nPower
imagery for an individual picture cue), and picture position
effects (effects due to position of the picture within the set).
The online sample additionally completed self-report
measures assessing explicit motives and the Big Five per-
sonality traits to compare correlations with the PSE motive
scores with those found in previous research (Pang and
Schultheiss 2005; Schultheiss and Brunstein 2001).
Participant behavior and setting. In contrast to partici-
pants in the lab, online participants can drop out from a
study more easily, simply by closing the browser window.
This makes online research more sensitive to dropout and
sampling bias (Birnbaum 2004b; Frick et al. 2001; Reips
2000; Reips and Bosnjak 2001). Web experiments that take
longer than a few minutes might cause psychological
reactance and data quality may suffer from high dropout
rates (Reips 2002a). Hence, we were interested in how
sensitive the rather time consuming PSE would be to
dropouts. Another question was at which point people
would stop participating and whether specific picture cues
lead to more dropouts than others. As opposed to usual PSE
administration in the lab online participants were given the
opportunity to proceed to the next picture before response
time for story writing was over, hence, it was further
possible to look at writing times for each picture in the
online study and to compare story lengths with the lab
sample. Since participants in the online sample chose the
setting themselves, we wanted to investigate whether spe-
cific characteristics of the surroundings, for instance the
presence of other people, would affect PSE motive scores.
Motive density of written stories. The uncontrolled
environment might distract participants of the online study
in a number of ways. It is further possible that participants
online might look at pictures and instructions less closely
which might lead them to write stories that are not com-
parable to lab-written stories regarding codable motive
content. One indicator to check for comparability is motive
density, which means the amount of motive imagery pro-
duced per word count (Winter 1994). If stories written in
the lab and online were comparable the type of study
(online vs. lab) should not predict overall motive imagery
after controlling for word count.
Previous research with PSE consistently showed that
there is a positive relationship between story length and
motive imagery. The longer stories participants write the
more motive content they usually produce (Schultheiss
123
254 Motiv Emot (2011) 35:251266

et al. 2008a; Smith et al. 1992). It is not self-evident that cues did not differ in motive profiles with regard to their
this relationship will hold for stories written online. There position in the picture sequence. Therefore, our study
might be more variation in online stories regarding motive aimed to compare online and lab study data in position
density which would be reflected in lower correlations effects on story length, motive imagery, and on motive
between word count and motive imagery. Therefore, we profiles of individual picture cues.
compared whether the type of study moderates the word Correlations between PSE motive scores and self-report
count motive imagery relationship. measures. Past research consistently found little overlap
Internal consistency. Previous research reported low between implicit motives and explicit motives (e.g. King
scores for internal consistency between picture cues esti- 1995; Thrash et al. 2007; Schultheiss et al. 2009). Others
mated by Cronbachs a (-0.02 to 0.43) (Schultheiss and studied the relationship of implicit motives and traits
Pang 2007; Schultheiss et al. 2008a). Atkinson (1981) measured with self-report scales such as the Big Five
argued that measures of reliability are not as applicable for personality factors and did not find substantial overlap
the assessment of implicit motives, because they are either (Schultheiss and Brunstein 2001; Winter et al. 1998).
dynamic need states that depend on environmental incen- As previous research has already addressed these questions
tives. People might have learned which situations allocate with lab administered PSEs, we were interested how
motivational incentives; hence behavior may differ implicit motive scores were related to self-report scales in
between these situations. Similarly, peoples stories vary in our online study.
motive imagery between different picture cues. Although
we did not expect internal consistency to be high in either
sample, we explored whether Cronbachs a would be
Methods
comparable between the lab and online assessment.
Picture motive profiles. Previous research has repeatedly
Designing an online PSE measure
found that different picture cues elicit different amounts of
nAchievement, nAffiliation, and nPower in participants
We programmed the online PSE measure in WEXTOR, a
stories (Langan-Fox and Grant 2006; Pang and Schultheiss
web-based tool for Internet experiments created by Reips
2005; Schultheiss and Brunstein, 2001). For instance,
and Neuhaus (2002), which includes features that meet
women in laboratory constantly elicited high amounts of
general methodological requirements of web research (e.g.,
nAchievement and nPower imagery but almost no nAffil-
generation of a session ID to detect multiple submissions
iation imagery.
by the same participant). As described below, we used
The different environments of online participants might
several techniques to address methodological challenges of
not only contain possible distractions but a variety of
Internet research (Reips 2002a, b) and specifically for an
random incentives which might interact with picture cues
online PSE assessment.
as well as peoples motive dispositions, and so affect
To reduce dropout we used two techniques that were
motive imagery. The comparison of picture motive profiles
used in previous web experiments (Musch and Klauer 2002;
between the two samples can give insight into the presence
Reips 2000, 2001; Reips and Morger 2001): warm-up and
of such random factors. If the picture motive profiles from
seriousness check. The warm-up technique was developed
the online sample are less pronounced than the motive
on the basis of the observation that most participants drop
profiles obtained in the lab this could indicate the impact of
out in the beginning of a study. Therefore, Reips (2002a)
additional incentives. Therefore, the present research tested
recommends placing the main measurement several pages
whether picture profiles from the lab and from previous
into the survey. Before participants see the first PSE picture
research can be replicated in the online sample.
cue on the eighth page, they have already answered ques-
Picture position effects. Pang and Schultheiss (2005)
tions on demographic information and filled in a seriousness
first explored effects of picture position on story length,
check, which is a single question asking, How do you
motive imagery, and motive profiles of individual picture
intend to browse the Web pages of this study? with two
cues. By randomizing the order of picture cues for each
response choices: I would like to seriously participate
participant, they found that participants increased the
now. or I would like to look at the pages only. Data is
length of their stories in the first few pictures, and main-
recorded in both cases, but only data from persons who
tained this length to the last picture. Achievement motive
expressed a serious motivation to participate were analyzed.
imagery remained relatively constant, whereas affiliation
First participants read the standard instructions adapted
imagery dropped in the middle of the picture sequence and
from Smith (1992):
power imagery showed the reverse pattern. However, the
dynamic effects of both motives were modest and as far as On the upcoming pages we will show you different
we know have not been replicated yet. Individual picture pictures, each for about 20 s and ask you to make up

123
Motiv Emot (2011) 35:251266
a story about each of them. Try to imagine what is
happening. Then write down what is going on now,
what happened before, what the people are thinking
and feeling, and what they are going to do. Write a
story that is completea story that has a beginning, a
middle, and an end. There are no right or wrong
stories, so write whatever story you think of when
you look at the picture. Spelling, punctuation, and
grammar are not at all important. What is important is
to write whatever story you think of as you imagine
what is going on in each picture.
The four picture cues (ship captain, architect at desk,
women in laboratory, and trapeze artists) were presented in
randomized order, in the middle of the screen, and
formatted to fit screens with 17 diagonal or larger. Each
picture was headed by a short instruction saying: Look at
the pictureThe program will automatically turn to the
next page within 20 s. On the next page a text field was
presented with six reminders about what to include in the
story (e.g. What is happening?, Who are the people?).
We limited the text field to 3000 words to prevent
participants from pasting long texts into the survey. The
text field was presented for 5 min before the program
automatically turned to the next page. As the program
recorded response times (in milliseconds) for all pages we
were able to control for the time participants spent on story
writing, and therefore allowed participants to move onto
the next picture whenever they were ready.
Participants
The sample consisted of 180 participants for the online
study (97 women, 56 men, 6 not indicated, 21 dropped out
before indicating their gender) and 93 participants (58
women, 26 men, 9 not indicated) for the lab study. The two
samples did not differ in terms of gender composition and
the number of non-indications, v2 (2, N = 252) = 4.41, ns.
Participants in the online study were significantly older
(M = 37.01, SD = 9.34) than participants in the lab study
(M = 29.75, SD = 11.83), t(210) = -4.97, p \ 0.001
(two-tailed), d = 0.70. As previous research found that
older participants express significantly less achievement
motive imagery (Pang and Schultheiss 2005), we con-
trolled for age in all further analysis to rule out age dif-
ferences as an explanation for the results.
Procedures
Online study. Participants were recruited from a nationwide
online pool. New pool members are recruited via adver-
tisements on a website that provides local sales and job
forums throughout various cities of the US. Registered
255
members receive an email once a week informing them of
studies available to them that week. They can then log in
and follow the link to participate in those studies.
In the present study, participants were invited to take
part in a 60-min online study on Relationships, Leisure,
and Work that would ask participants to write short
stories about people in different situations (i.e., refer-
encing the PSE) and to answer questions about different
areas of your everyday life (i.e., referencing the other
personality measures administered after the PSE).3 The
description further informed potential participants that they
would need to provide their email address to receive a $12
gift card for compensation. When following the link to the
survey, participants first answered the seriousness check.
Next they were informed about the purpose and duration of
the study, before they provided consent and verified their
age. The main study began with the PSE assessment of
implicit motives. In the second part of the study partici-
pants were asked control questions about the kind of place
(i.e. private, semi-public, public) where they had com-
pleted the PSE, if they were surrounded by other people
and if they had communicated with them as they completed
the measures. Then participants completed a standard
measures of explicit motives and the Big Five personality
factors as well as additional measures of interest for a
larger research project. After the study participants were
automatically guided to a separate online survey, where
they were asked to provide their email address in order to
receive a gift card for compensation. This approach
allowed us to save email addresses separately from the rest
of participants data securing anonymity. We also provided
a link to a website offering easy creation of compostable
email addresses (ones that expire after a given time of use)
in case participants did not want to provide a private email
address.
Lab study. Most participants were recruited by flyers,
ads on a website that provides local sales and job forums,
and from the participant pool of the psychology department
at Stanford University. Participants took part individually
in lab sessions in a university building. They were seated in
a separate cubicle in front of a computer. To make the
settings comparable lab participants also completed the
PSE on the computer and without any experimenter being
present during the session. Hence, the 60-min session
consisted of two parts with the PSE measure administered
first. Instructions and pictures of the PSE measure were
identical to the online measure; except participants were
not allowed to move onto the next picture until 5 min had
elapsed and the program would lead them there.
3
The PSE assessment was part of a questionnaire for a larger
research project on long-term relationships.
123
256 Motiv Emot (2011) 35:251266

Coding of PSE stories a = 0.57). As internal consistency for openness to expe-

Stories were coded for motive-specific content using
Winters (1991) Manual for Scoring Motive Imagery in
Running Text assessing nAchievement, nAffiliation, and
nPower. According to the manual, nAchievement was
coded for story contents expressing concern for a standard
of excellence, e.g. indicated by adjectives that positively
evaluate performances or positive evaluated goals and
performances. Story content coded for nAffiliation was that
indicating the establishment, maintenance, or restoration of
friendly relations among persons including a warm and
friendly quality to the relationship, e.g. indicated by
expressions of positive or intimate feelings toward others.
Finally, nPower was scored for content indicating impacts
or influence on other persons or groups, e.g. by strong,
forceful actions or eliciting strong emotions in others. Each
story was coded by two coders who reached an overall
agreement of Cohens Kappa of 0.85.
Additional measures in online study
Explicit motives. To assess explicit motives, we used the
three subscales for exAffiliation, exAchievement, and
exPower (ex stands for explicit) of McClellands Personal
Values Questionnaire (PVQ, McClelland 1991). Each scale
consists of ten items representing personal values, e.g.
Maintaining close relationships with the people I really care
about (exAffiliation, M = 4.56, SD = 0.75, a = 0.85),
Continuously new, exciting, and challenging goals
and projects (exAchievement, M = 4.61, SD = 0.80,
a = 0.90), and Important positions and projects that can
give me recognition (exPower, M = 3.72, SD = 1.12,
a = 0.92). Participants were asked to rate the personal
importance of each item on a fully labelled scale from 1
(not important to me) to 7 (extremely important to me).
Personality scales. We assessed the Big Five personality
factors, neuroticism, extraversion, openness to experience,
conscientiousness, and agreeableness, using a 10-item short
version of the Big Five Inventory (Rammstedt and John
rience was low (a = 0.34), we excluded it from further
analysis.
Results
Participants behavior in the online assessment
Dropout. Out of 180 participants who followed the link to
our online survey, 137 (76.1%) completed the PSE mea-
sure. We identified a group of participants (11.7%) for
whom the program did not collect response times for the
very first page and concluded that those were not able to
start the survey probably due to technical problems like
interaction of certain web browser versions and incom-
patible elements on web pages (Schwarz and Reips 2001).
These participants do not count as dropouts since they did
not choose to not continue the survey (Reips 2002a). Of the
159 participants who actually started the survey 13.8%
dropped out, which is about the same rate of 13.2% found
by Frick et al. (2001), who offered financial incentives at
the beginning of the survey and asked for personal infor-
mation at the end of it, similar to our procedure. All par-
ticipants who dropped out did so during the first nine
pages; none did later. Most drop outs quit after the first text
field (n = 9), presented on the ninth page of the survey.
There was no effect of the picture cue presented on page
eight on drop outs, v2 (3, n = 147) = 3.32, ns. Figure 1
shows the dropout and remaining sample size over the
course of the survey.
Of participants who finished the PSE, 8.0% did not write
stories but instead wrote short descriptions of the pictures
or wrote about something that was not related to the pic-
ture. Because we doubted whether these participants
understood the instructions, we excluded their data from
25 100 Percentage of remaining participants
Absolute number of dropout

Dropout
95
20 Sample size
2007). Participants rated their agreement on each item on a
fully labelled scale from 1 (strongly disagree) to 5 90
15
(strongly agree). Sample items included, for neuroticism 85
I see myself as someone who gets nervous easily 10
(M = 2.77, SD = 0.97, a = 0.64), for extraversion, e.g. 80

I see myself as someone who is outgoing, sociable 5

75
(M = 3.20, SD = 0.86, a = 0.43), for openness to expe-
rience, e.g. I see myself as someone who has few artistic 0

interests (reverse, M = 3.42, SD = 0.82, a = 0.34), for

St

In r

Co

In nt

In

In

Pi

Te 1

Pi eld

e2

Th
em
ar

str

str

str

str

ct

ct
xt

an
ns

ur

ur 1
t

og

k
1

fi
e

conscientiousness, e.g. I see myself as someone who does

Web page
a thorough job (M = 3.84, SD = 0.92, a = 0.69) and for
agreeableness, e.g. I see myself as someone who tends Fig. 1 Dropout and sample size for online study tracked over pages
to find fault with others (reverse, M = 3.58, SD = 0.84, of the PSE measure

123
Motiv Emot (2011) 35:251266
the coding procedure, resulting in a sample of 125 for the
final analysis.
Writing time and story length. We were interested in
how long participants in the online study actually spent
writing stories, as they could proceed to the next picture at
any time. Therefore, we averaged writing times for the four
stories (recorded response times for the four pages
including the text field) and found an average time of
M = 683.55 (SD = 349.87) seconds (about 11 min). This
was about half the time participants were allowed to write
the four stories.
Individual picture cues did not differ in writing time,
F(3, 134) = 0.80, ns. Next, we tested for a trend in writing
time and conducted a repeated measures ANOVA with
writing time at each position (one to four) as within-subjects
variable and found a significant linear trend, F(3, 123) =
6.67, p \ 0.01, n2 = 0.05. Writing time decreased over the
four stories.
Next we examined the relation of writing time and
motive imagery and found a significant correlation between
writing time and raw motive scores for all three motives,
r(125) [ 0.32, p \ 0.05. The longer participants wrote the
more motive imagery was expressed.
As participants in the lab were not allowed to proceed to
the next picture until 5 min had elapsed for each picture,
we could not compare the writing time in the two studies.
Instead we looked at differences in story length counting
words for all four pictures. Stories written in the lab were
significantly longer (M = 529.54 words; SD = 190.59)
than stories written online (M = 307.06 words; SD =
171.76), F(1, 216) = 81.44, p \ 0.01, partial n2 = 0.27,
which is not surprising since participants in the online
study were allowed to proceed to the next picture. Since we
aimed to compare motive scores that are corrected for story
length, we examined whether story length fully accounted
for differences in writing time in the online study. Hence,
we ran a linear regression to see whether writing time still
predicted raw motive scores when controlling for story
length. For all three motives the analysis revealed that
writing time was no longer a significant predictor for
motive scores when story length was entered into the
regression equation, t(121) \ 0.30, ns. Therefore, control-
ling for story length in further analyses is sufficiently
accounting for the differences in writing time between the
two studies.
Setting. In the online study 90.7% of participants indi-
cated that they had completed the PSE measure in a private
place, 6.2% in a semi-public place, and 3.1% in a public
place. We combined the latter two and compared them with
the private group regarding writing time, story length,
and motive imagery using two-tailed t tests. The private
group spent marginally more time writing, t(120) = 1.70,
p = 0.09, d = 0.54, but did not produce longer stories,
257
t(120) = 0.88, ns. The stories of the private group con-
tained significantly more nAchievement motive imagery
observing raw scores, t(120) = 2.34, p \ 0.05, d = 0.74,
but did not differ in nAffiliation or nPower imagery,
-0.23 \ t(120) \ 0.92, ns.4
Second, we tested if being among other people (reported
by 29.5% of participants) related to writing time, story
length, or motive imagery. We conducted two-tailed t tests
to compare this group with participants who indicated they
were alone when they completed the measures. There
was no difference in writing time, t(120) \ 1.11, ns,
story length, t(120) \ 0.80, ns, or raw motive scores,
-0.66 \ t(120) \ 1.36, ns.
Motive density
One way to check for comparability in motive density is to
analyze whether the study type (online vs. lab) predicts
overall motive imagery after controlling for word count.
Hence, we ran a hierarchical regression analysis predicting
overall motive imagery by word count in a first step adding
study type (online = 0; lab = 1) in a second step. This
analysis revealed both a significant main effect of word
count, R2 = 0.48, b = 0.70, t(1, 216) = 14.25, p \ 0.001,
and of study type, b = -0.14, t(1, 216) = -2.41,
p \ 0.05, DR2 = 0.01, p \ 0.05. There was a tendency for
stories written online to be higher in motive density than
stories written in the lab. We ran the same regression
analysis for each motive separately. In a hierarchical
regression we predicted the raw motive scores by word
count and study type (online = 0; lab = 1). For nAffilia-
tion we found a significant main effect of setting, b =
-0.23, t(2, 215) = -3.34, p \ 0.01. However, there was
no effect of study type predicting raw scores for
nAchievement, b = -0.09, t(2, 215) = -1.25, ns, or for
nPower, b = 0.01, t(2, 215) = 0.12, ns. Online stories
were denser specifically for nAffiliation.
In a last step we wanted to see whether the online study
differed from the lab study in the variation in motive
density. Therefore, we compared whether the type of study
moderates the word countmotive imagery relationship.
Hierarchical regression analyses tested whether the study
type interacted with word count to predict motive scores
revealed no interaction for any of the three motives,
t(3,214) \ -1.32, ns. Hence, we found no evidence for a
difference between online and lab written studies in the
relationship between word count and motive imagery.
To control for the effects of story length, raw motive
scores are often corrected using a formula introduced
by Winter (1994): (motive score/word count) 9 1,000.
4
Six participants did not provide stories to all four picture cues and
were therefore not included in the analysis for overall word count.
123
258
Table 1 shows raw motive scores and word corrected
motive scores for each picture cue separately for both
studies. Two-tailed t tests showed that after word correc-
tion stories in the online sample included more motive
imagery in nAffiliation, t(216) = -3.56, p \ 0.001, d =
-0.49, and in nAchievement, t(216) = -2.00, p \ 0.05,
d = -0.27. For nPower the trend was in the same direction
but the effect was not significant, t(216) = -1.14, ns.
Internal consistency
We estimated Cronbachs a for nAchievement, nAffilia-
tion, and nPower using the raw motive scores for all four
pictures and found somewhat higher reliability scores in
the lab study (aAch = 0.55, aAff = 0.50, aPow = 0.53) than
in the online study (aAch = 0.27, aAff = 0.48, aPow =
0.45). For word corrected motive scores internal consis-
tency was still somewhat higher in the lab study (aAch =
0.20, aAff = 0.32, aPow = 0.52) than online study (aAch =
0.15, aAff = 0.29, aPow = 0.002), meaning that although
pictures in the online study were on average denser in
motive imagery and individuals stories differed more in
motive imagery across picture cues.
Picture profiles
We tested the difference in motive profiles between indi-
vidual picture cues and between studies in a repeated
measures ANOVA using picture (one to four) and motive
(nAchievement, nAffiliation, and nPower) as within-sub-
jects variables and study type (online = 0, lab = 1) as a
between-subjects variable predicting word corrected
motive scores. Replicating the results of previous research
the picture cues differed in their motive profiles, as the
Picture 9 Motive interaction was highly significant,
F(2, 215) = 51.64, p \ 0.001, n2 = 0.19. The studies
slightly differed in picture profiles as the three-way inter-
action for Picture 9 Motive 9 Study Type was significant,
F(6, 211) = 4.98, p \ 0.001, n2 = 0.02, although the
effect was small.5
Figure 2 shows motive profiles for individual picture
cues for both studies for raw motive scores and word
corrected motive scores. Motive profiles in the online study
showed a greater range compared to the lab profiles. For
instance, architect at desk showed the highest pull for
nAffiliation in both studies but the difference between
nAffiliation and nAchievement was larger in the online
study. Similarly, women in laboratory showed the highest
5
When we used raw motive scores we found the same effects: both
the Picture 9 Motive interaction, F(2, 215) = 70.28, p \ 0.001,
n2 = 0.25, and the three-way interaction were significant, F(6,
211) = 2.34, p \ 0.05. n2 = 0.01.
123
Motiv Emot (2011) 35:251266
pull for nAchievement in both studies, but the difference
with nPower was again larger in the online study. Motive
profiles for ship captain and trapeze artists were similar in
pull and range though for the two studies. To test these
differences for statistical significance we conducted repe-
ated measures ANOVAs separately for picture cues and
entered motive as a within-subjects variable and study type
as a between-subjects variable. For architect at desk
the interaction for Motive 9 Study Type was significant,
F(2, 216) = 11.31, p \ 0.001, n2 = 0.05. Post hoc t-tests
confirmed that architect at desk had a significantly higher
pull for affiliation imagery online than in the lab, t(218) =
3.73, p \ 0.01, d = 0.51, while there was no difference
between online and lab study for nAchievement and nPo-
wer. For women in laboratory the Motive 9 Study Type
interaction was smaller but significant, F(2, 216) = 3.34,
p \ 0.05, n2 = 0.02. Post hoc t-tests confirmed the higher
pull for achievement imagery for women in laboratory in
the online study, t(218) = 2.45, p \ 0.05, d = 0.35. There
was no difference between online and lab study for
nAffiliation and nPower. For ship captain and trapeze
artists the Motive 9 Study Type interaction was not sig-
nificant, F(2, 216) = 1.55, ns, and F(2, 215) = 0.98, ns,
respectively.
Table 1 illustrates similarities and differences in
absolute pull of pictures for motive imagery: High pull
represents when at least 50% responded with at least one
instance of codable motive imagery to the picture (Schul-
theiss and Brunstein 2001). For both studies, architect at
desk pulled high for nAffiliation, women in laboratory high
for nAchievement and trapeze artists high for both
nAchievement and nPower, which is consistent with find-
ings from previous studies using this indicator of picture
profiles (Pang and Schultheiss 2005; Schultheiss and
Brunstein 2001). But only for the lab study the pictures
ship captain and women in laboratory pulled high for
nPower, and replicated previous findings.
Picture position effects
As we randomized the order of picture cues in both studies
we were able to look at the effects of picture position on
story length and motive imagery. Pang and Schultheiss
(2005) found word count to increase with later position for
PSEs administered in the lab and using six picture cues. To
compare position effects in word count for the two studies
we entered study type (online = 0, lab = 1) as between-
subjects variable in a repeated measures ANOVA pre-
dicting word count for the four positions. There was a
marginally significant Position 9 Study Type interaction,
F(3, 208) = 2.23, p = 0.08, n2 = 0.01, meaning there
were different trends for word count in the two studies.
Therefore, we conducted the repeated measures ANOVAs
Motiv Emot (2011) 35:251266 259

Table 1 Means and standard deviations of raw and word corrected scores across motives and picture cues for online and lab study
Picture nAchievement nAffiliation nPower Words
M SD M SD M SD M SD

Raw scores
Ship captain
Online-study 0.12 0.41 0.21 0.60 0.57 0.78 100.20 50.52
Lab-study 0.32 0.64 0.15 0.53 1.09 1.25 129.71 49.62
Difference -0.36** 0.10 -0.48** -1.19**
Architect at desk
Online-study 0.14 0.37 1.14 1.00 0.34 0.66 109.45 57.66
Lab-study 0.32 0.53 1.16 1.08 0.61 0.75 138.84 56.03
Difference -0.39** -0.02 -0.37** -1.08**
Women in laboratory
Online-study 0.95 1.03 0.14 0.41 0.51 0.80 100.44 51.08
Lab-study 1.11 1.05 0.20 0.56 0.91 0.92 127.48 51.46
Difference -0.15 -0.14 -0.47** -1.14**
Trapeze artist
Online-study 0.67 0.74 0.48 0.77 0.70 0.79 103.68 53.22
Lab-study 0.95 1.00 0.67 0.91 0.79 0.88 132.34 51.89
Difference -0.32* -0.23 -0.11 -1.09**
Total
Online-study 1.88 1.60 1.96 1.85 2.12 1.89 307.06 171.76
Lab-study 2.70 2.19 2.18 2.04 3.41 2.49 529.54 190.59
Difference -0.41** -0.12 -0.57** -1.24**
Word corrected scores
Ship captain
Online-study 1.69 6.13 2.56 7.38 7.25 10.28
Lab-study 2.36 4.81 1.07 3.63 8.02 9.16
Difference -0.12 0.27? -0.08
Architect at desk
Online-study 1.62 4.88 14.56 14.07 3.77 7.69
Lab-study 2.30 4.26 8.57 7.90 4.39 5.64
Difference -0.15 0.54** -0.09
Women in laboratory
Online-study 12.97 15.33 1.98 7.01 6.73 12.04
Lab-study 8.69 8.15 1.46 4.13 7.15 7.35
Difference 0.36* 0.09 -0.04
Trapeze artist
Online-study 9.66 12.06 5.81 10.21 10.25 13.20
Lab-study 7.42 7.64 4.68 6.15 6.33 7.88
Difference 0.21 0.13 0.37*
Total
Online-study 6.27 5.65 6.27 5.71 7.24 5.80
Lab-study 5.04 3.42 4.06 3.41 6.43 4.74
Difference 0.27* 0.49**
Note: Difference = (Cohens d) differences derived from t tests (two-tailed) between online and lab study within picture and coding category.
Underlined motive scores indicate that more than 50% of participants in the total sample or subsample (online or lab study) have responded with
at least one instance of codable motive imagery to the picture cue
Online: n = 125; Lab study: n = 93
?
p \ 0.10; * p \ 0.05; ** p \ 0.01

123
260 Motiv Emot (2011) 35:251266

Online Study Lab Study

2 2
nAchievement nAchievement
nAffiliation nAffiliation

Raw motive scores

1.5 1.5 nPower
nPower

1 1

0.5 0.5

0 0
Ship Architect Women in Trapeze Ship Architect Women in Trapeze
captain at desk laboratory artist captain at desk laboratory artist

30 30
Word corrected motive scores

nAchievement nAchievement
25 25 nAffiliation
nAffiliation
nPower 20 nPower
20

15 15

10 10

5 5

0 0
Ship Architect Women in Trapeze Ship Architect Women in Trapeze
captain at desk laboratory artist captain at desk laboratory artist

Fig. 2 Picture profiles for online study and lab study for raw motive scores (above) and word-corrected motive scores (below)

separately for the studies and found word count consis-
tently increased in the lab study, F(1, 91) = 9.91,
p \ 0.01, n2 = 0.10, replicating the findings by Pang and
Schultheiss (2005). However, in the online study we did
not find any linear, F(1,119) = 0.01, ns, or quadratic trend,
F(1, 119) = 0.44, ns.
Next, we tested whether studies differed in the effects of
position on motive imagery. Therefore, we created twelve
variables each representing word corrected scores for
nAchievement, nAffiliation, and nPower at each of the four
positions. We entered those variables in a repeated mea-
sures ANOVA with motive (nAchievement, nAffiliation,
and nPower) and position (one to four) as within-subjects
variable and study type (online = 0, lab = 1) as between-
subjects variable. We found that motive imagery did not
change over time for any of the motives, as the main effect
for position, F(3, 208) = 0.25, ns, and the Posi-
tion 9 Motive interaction were both not significant, F(6,
205) = 0.44, ns. The studies did not differ in this regard as
the three-way interaction for Position 9 Motive 9 Study
Type was also not significant, F(6, 205) = 1.28, ns.
Last, we tested position effects for individual picture
profiles and conducted MANOVAs predicting picture-
specific motive scores in nAchievement, nAffiliation, and
nPower with picture position as quantitative predictor and
study type (online = 0, lab = 1) as categorical predictor
variable. For none of the picture cues position affected any
of the motive imagery, as all Position 9 Study Type
interactions were not significant, F(1, 214) \ 2.34, ns.
Correlations for PSE motive scores and self-report
measures
Correlations among PSE motive scores. In the online study
we found significant positive correlations for raw scores
between all three motives, r(125) [ 0.25, p \ 0.01. How-
ever, for word corrected motive scores the correlations
were not significant, r(125) B |0.11|, ns. In the lab study
only raw scores for nAchievement and nPower were sig-
nificantly correlated, r(92) = 0.38, p \ 0.001, which is
consistent with findings from Schultheiss and Brunstein
(2001). However, for word corrected motive scores the
correlations were again not significant, r(92) B |0.16|, ns.
Explicit motives. Analyzing word corrected PSE scores
we did not find any significant correlation between implicit
and explicit motives, r(121) B |0.13|, ns, replicating results
from previous research (e.g. King 1995; Schultheiss et al.
2009). However, when dividing the sample by gender for
men there was a marginally significant negative correlation
between nPower and exAffiliation (r = -0.29, p = 0.07)

123
Motiv Emot (2011) 35:251266 261

Table 2 Correlations between word corrected implicit motives, explicit motives and the big five personality factors for men and women
Gender exAch exAff exPow N E A C

Men
n Achievement 0.19 0.23 0.27? -0.20 0.14 0.12 0.17
n Affiliation -0.15 -0.07 -0.05 -0.03 0.11 0.17 0.20
n Power -0.11 -0.29? -0.01 0.16 -0.12 -0.49** -0.26?
Women
n Achievement 0.06 0.05 -0.05 -0.09 0.02 0.11 0.25*
n Affiliation -0.08 0.01 -0.13 -0.12 0.12 -0.07 -0.03
n Power 0.03 -0.02 -0.10 -0.20 0.04 0.03 0.09
Note: ExAch, explicit Achievement motive; exAff, explicit Affiliation motive; exPow, explicit Power motive; N, Neuroticism; E, Extraversion;
A, Agreeableness; C, Conscientiousness
Men: n = 83; Women: n = 43
?
p \ 0.10; * p \ 0.05; ** p \ 0.01

and a marginally significant positive correlation between
nAchievement and exPower (r = 0.27, p = 0.08). Table 2
shows correlation coefficients between word corrected PSE
scores and explicit motives separately for men and for
women.
Personality variables. In the online sample we found
word corrected PSE scores for nAchievement to be sig-
nificantly correlated with self-reported conscientiousness,
r(125) = 0.21, p \ 0.05. and nPower scores to be signifi-
cantly negatively correlated with agreeableness, r(125) =
-0.20, p \ 0.05.
Interestingly, there were gender differences in the rela-
tion of implicit motives and the Big Five personality fac-
tors. Table 2 shows correlation coefficients between word
corrected PSE scores and the Big Five personality factors
separately for men and women.
Among men nPower was negatively correlated to
agreeableness and conscientiousness, which was highly
significant and marginal significant, respectively. For
women both correlations were near zero. However, among
women nPower was negatively correlated to neuroticism,
whereas for men this correlation was positive, although not
significant. Among both sexes nAchievement was posi-
tively correlated to conscientiousness, but the relation was
only significant among women.
To see whether gender moderates the relationships
between personality variables and motive scores we con-
ducted hierarchical regression analyses. We predicted word
corrected nPower scores by gender (male = 0; female = 1)
and agreeableness in a first step and entered their interaction
term in a second step. The Gender 9 Agreeableness inter-
action got significant, DR2 = 0.09, b = -0.53, t(121) =
3.49, p \ 0.01. Similar hierarchical regression models
predicting nPower revealed a significant Gender 9 Consci-
entiousness interaction, DR2 = 0.04, b = 0.40, t(121) = 2.13,
p \ 0.05, and a marginal significant Gender 9 Neuroticism
interaction, DR2 = 0.03, b = -0.36, t(121) = -1.86,
p = 0.07. Last, we regressed word corrected nAchievement
on gender, conscientiousness, and their interaction term,
but the interaction with gender got not significant, DR2 =
0.001, b = -0.07, t(121) = -0.36, p [ 0.05.
Discussion
The aim of the present research was to investigate whether
it is possible to measure implicit motives with an Internet
version of the PSE, which asks participants to write short
stories as response to a set of picture cues. First, we doc-
umented how participants processed the online PSE as
reflected by dropouts, story writing time, story length, and
whether there were effects regarding the surrounding set-
ting on motive imagery. Second, we compared stories
written online with stories written in the lab regarding
motive density, internal consistency, picture profiles, and
position effects. Our basic goal was to document whether
the data collected online were comparable with data col-
lected in the lab and whether both were consistent with
previous research (Pang and Schultheiss 2005; Schultheiss
and Brunstein 2001).
Participant behavior in the online study
Dropout. The dropout rate in the online study was similar
to those found in previous online studies using comparable
procedures (Frick et al. 2001). The standard techniques
incorporated in the online PSE measure successfully
encouraged early dropouts and led to few dropouts later in
the questionnaire. The fact that no participant dropped out
after completion of the first story suggests that the PSE can
be assessed online despite its time consuming and rather
demanding procedure.

123
262
Nevertheless, 8.0% of participants who completed the
PSE wrote about something that was not related to the
picture or provided a brief description of it. Since we had to
exclude these participants from the coding procedure,
participants time and expenses for compensation were
wasted. However, unfeasible responses are problem of web
research in general since participants do not have the
chance to ask the experimenter questions about the
instructions (Birnbaum 2004a; Reips 2002a). Since we
used the standard instructions for the PSE problems of
understanding were minimized but cannot be eliminated
completely. Another explanation for single-sentence
responses might be that participants were distracted by the
single-line format of the text field. Numerous studies show
how changes in answer formats influence results obtained
(Birnbaum 2001; Dillman and Bowker 2001; Reips 2002c).
Future studies might choose a multiple-line text box to
avoid these misunderstandings.
Further we cannot rule out the possibility that the
incorporated techniques to reduce dropout affected motive
content of the stories. However, the questions asking par-
ticipants to indicate the seriousness of their participation
and their demographic information should be relatively
innocuous and likely do not induce additional motive
arousal.
Writing time and story length. Participants in the online
sample, where they were allowed to proceed to the next
picture as soon as they liked, wrote for about half the time
usually required for story writing. However, our analyses
showed that controlling for story length can fully account
for the variance due to differences in writing time, which
backs up the use of word corrected motive scores for
comparing our two studies. Schultheiss and colleagues
(2008a) showed that typed stories are overall substantially
longer than handwritten stories. Therefore they suggested
that a writing time of 5 min, as usually prescribed for each
PSE story, might be unnecessary for obtaining comparable
motive scores from typed stories. Also, group administra-
tions of the PSE often do not perfectly hold constant
writing time. Some participants stop writing earlier while
others write longer than prescribed (Schultheiss and Pang
2007). In an early study on computer assessment of the
PSE, Blankenship and Zoota (1998) compared computer
vs. handwritten stories in an untimed vs. timed condition
(both factors manipulated between subjects), found no
significant difference in Power motive imagery between
the four conditions. They therefore recommended the use
of untimed procedures to allow for participants various
levels of typing ability. Finally, PSE was successfully
administered in previous research using take-home pack-
ages, which also do not control writing time (Job and
Brandstatter 2009; Thrash et al. 2007). Allowing partici-
pants to move forward when they like might be
123
Motiv Emot (2011) 35:251266
advantageous as people substantially differ in speed of
typing and fluency of imagination. Waiting for the time to
pass might even interrupt motive imagery flow and, spe-
cifically in online settings, might produce unnecessary
dropouts or create distraction, e.g. tempt participants to
browse another webpage meanwhile. For these reasons, we
suggest that the option to let participants continue to the
next picture is an acceptable deviation from traditional PSE
administration, particularly useful for the online assess-
ment of implicit motives.
Motive density
When participants write PSE stories online there is no
control over their environment and it is much more likely
that there are distractions present, making it unclear whe-
ther participants are able to be as involved with story
writing as they would be in the controlled, quiet setting of
the lab. Hence, one question of major importance was
whether the stories written online contained the same
amount of codable motive imagery per word count as
stories written in the lab. Our results show that these
concerns were unsubstantiated. In fact, stories written
online turned out to be denser in motive imagery then
stories written in the lab, particularly in affiliation and
achievement imagery. One reason could be that the setting
which most participants indicated to be a private place
allowed them to fantasize more freely and to get more
strongly involved in the task.
It is further important to notice that there was no dif-
ference in variation in motive density between the online
and the lab study, since study type did not moderate the
relationship between story length and motive imagery.
Participants in the online sample seemed to get consistently
engaged in their story writing, producing comparable
amounts of motive content overall and per word.
Internal consistency
In both studies we found low internal consistency between
picture cues. Thereby, the internal consistency in the online
study tended to be somewhat lower than in the lab study.
However, this finding is not evidence against the online
administration of the PSE. Researchers working with the
PSE contend that the common criteria for internal consis-
tency are not applicable for the PSE (Atkinson 1981;
Reuman 1982; Schultheiss and Pang 2007). As opposed to
personality inventories, the PSE does not assess the same
construct in multiple ways, but forces the participant to
react to different items in order to trigger all motives
sequentially over the course of the test. Hence, inconsistent
reaction to different cues does not mean that the motive
itself is inconsistent. Drawing the same conclusion
Motiv Emot (2011) 35:251266
Schultheiss and colleagues (2008a) interpreted low internal
consistencies as stable ifthen consistencies with reference
to cognitive-affective system theory of personality (Mi-
schel and Shoda 1995). Thus they argue that motive dis-
positions are manifested in the stable construction of
specific situations but not in the construction of all situa-
tions in the same way. In fact, early studies showed that
PSE motive scores with lower or even negative internal
consistencies had better predictive validity (Reuman 1982)
a finding that has to be replicated for online PSE scores.
Picture profiles
The present research essentially replicated picture motive
profiles reported in previous research conducted in the lab
(Pang and Schultheiss 2005; Schultheiss and Brunstein
2001) in both studies, for example the picture architect at
desk elicited high levels of nAffiliation motive imagery and
trapeze artists elicited high levels of nAchievement and
nPower imagery. These results indicate that overall the
pictures are perceived similarly and lead to comparable
motive imagery whether the stories are written in a con-
trolled laboratory setting or via Internet at random loca-
tions. However, the pictures ship captain and women in
laboratory had been found to pull high for nPower in
previous research which was replicated in our lab study,
but not in the online study. One reason for this difference
between the online and lab administration of the PSE could
be that in the lab the experimenter gives instructions and
guides participants through the procedure. He or she could
be perceived as an authoritative person, which arouses
participants power motive. Early studies on the effects of
administration showed that when the experimenter was an
authoritative person versus a fellow student reduced the
validity coefficients of the PSE (Lundy 1980, 1981).
Although there was no experimenter present during the
PSE session in the lab, participants still were welcomed
and seated by an experimenter. Therefore, we cannot
completely rule out the possibility of an experimenter bias
here. Reduction of experimenter bias is a major advantage
of online research in general (Birnbaum 2004a, b). But it
might be particularly advantageous for the assessment of
implicit motives, which seems to be especially sensitive to
this kind of bias.
Further, picture motive profiles in the online sample
turned out to be more differentiated than those from the lab
sample, meaning that they were more likely to contain
higher amounts of one (typical) motive imagery. Hence, in
the online study we found the range between highest and
lowest word corrected motive score to be significantly
larger for architect at desk and marginally larger in women
in the lab. However, this result is not evidence that the
presence of random incentives in the surroundings interact
263
with picture cues, producing more diverse and atypical
motive imagery than in the lab. Rather, it seems that the
familiar, less artificial environment in which people prob-
ably completed the online PSE administration might have
alleviated spontaneous writing that causes typical motive
profiles. In fact, from the very beginning of the assessment
of implicit motives via PSE, Murray (1943) highlighted the
need for an informal and relaxing environment for the PSE
assessment, where participants feel natural and spontane-
ous. Our results strengthen Murrays point and suggest that
a familiar environment might allow people to freely
express in their stories what is typically triggered by the
picture cues.
Position effects
In line with previous research we found that story length
increased over the picture sequence in the lab study (Pang
and Schultheiss 2005). In the online study, word count
remained on a constant level over the picture sequence,
although writing time tended to decrease from the first to
the last picture. An explanation for this pattern is that
participants over time get more fluent (Pang and Schul-
theiss 2005), which might be both in imagination and
writing. This makes them more likely to produce longer
stories in the lab and to get to an end of their stories faster
when the PSE is completed online.
We did not find the position of pictures to affect overall
amount of nAffiliation, nAchievement, and nPower imag-
ery expressed and studies did not differ in this regard. In
contrast, Pang and Schultheiss (2005) in their first inves-
tigation of position effects found stories to be saturated
with nAffiliation imagery in the beginning and with nPo-
wer in the middle of the sequence. In addition, we did not
find any effect of picture position on motive profiles of the
individual picture cues in either of the studies, which is in
line with the findings by Pang and Schultheiss (2005).
Altogether, our findings support the conclusion that posi-
tion effects are negligible for PSE administration (Schul-
theiss and Pang 2007).
Correlations for PSE motive scores and self-report
measures
In the online sample we did not find any significant cor-
relation between PSE motive scores and explicit motive
scores, which is consistent with numerous previous studies
and the conception of two distinct motivational systems
(e.g., Brunstein and Hoyer 2002; deCharms et al. 1955;
King 1995; Schultheiss et al. 2009; Spangler 1992; Thrash
et al. 2007; see also Emmons and McAdams 1991).
We further measured the Big Five personality traits and
found gender to moderate the relationship between PSE
123
264
scores in nPower and agreeableness and neuroticism.
Whereas for men high scores in nPower were significantly
related to lower scores in agreeableness, there was no
relationship for women. In contrast, for women nPower
was marginally significantly related to lower scores in
neuroticism, whereas for men this relation was positive,
although not significant. These findings are not consistent
with the findings of Schultheiss and Brunstein (2001), who
found no relation between PSE motive scores and per-
sonality measures for both sexes. However, Winter (2010)
recoded TAT stories from 45 young men at age 23 col-
lected in a longitudinal study (Block 1985) and found
similar relations for nPower to personality variables
assessed with the California-Q-Sort (CAQ, a procedure in
which each participant is rated by the project staff on 100
phrases). The nPower motive was significantly correlated
to CAQ-phrases that Winter (2010) summarized as
Immoral, dehumanized view of other people and
Aggression, as well as Paranoia and Negative affect, Summary
lack if capacity for pleasure. In line with our results,
Winter (2010) reported these relations to be much stronger
for men. The present results have to be replicated before
allowing for proper interpretation. However, they can
provide new insights to the previously investigated ques-
tion about gender differences in the expression of the
implicit power motive (e.g., Winter 1988).
Sex differences in motive expression might be explained
according to social role theory differences in social
behavior and personality originate in the contrasting dis-
tributions of men and women into social roles (Eagly 1987,
1997). Sex role stereotypes might allow men to express
their power motive in less agreeable ways than women and
higher power status in society offer them more opportu-
nities to do so. For instance, for men nPower has been
shown to predict profligate impulsive behavior, like
drinking, aggression sexual exploitation (Winter 1988,
1992; Winter and Stewart 1978). In contrast, feminine
people were more likely than masculine or androgynous
people to report using subtle methods to influence others,
like tears and emotional alteration (Falbo 1977). Impul-
sivity is one facet of neuroticism and might therefore
explain why men high in nPower showed a small positive
correlation to this trait. Similarly, Winter (2010) suggests
that gender differences in the expression of the power
motive might be due to a stronger responsibility sociali-
zation in womenthrough childrearing or caring for
younger siblingswhich has been shown to be a moderator
of motive expression for both sexes.
Limitations and future directions
It is important to note that a limitation of the present
research is the fact that participants were not randomly
123
Motiv Emot (2011) 35:251266
assigned to online vs. lab conditions but were recruited
separately. Therefore, differences in the findings may not
be caused by differences in the method of administration
but could be due to the differences in samples. However, to
back up the comparison and to validate our findings we
compared them to those from previous research (Pang and
Schultheiss 2005; Schultheiss and Brunstein 2001; Schul-
theiss and Pang 2007; Schultheiss et al. 2008a) and repli-
cated previous findings from studies conducted in the lab.
Future research should investigate online PSE adminis-
tration with comparative validity studies. Participants
should be randomly assigned to conditions of administra-
tion and some criterion variables known to be predicted by
implicit motives (e.g., achievement task performance)
should be collected in each condition in order to further
examine the validity of implicit motive scores assessed on
the Internet.
The present study yields promising results for the future
administration of the PSE via the Internet by comparing
data collected online and in the lab. Overall online stories
were comparable with stories written in the lab and with
previous research regarding motive density, internal con-
sistency, picture profiles, and position effects. In a next
step, research has to provide additional evidence for the
predictive validity of online administered PSE data.
However, the present research represents a first step into
the direction of making the numerous advantages of web
research (e.g., low costs, no constraints in time and place,
no experimenter bias, and access to wider samples)
accessible for research on implicit motives using PSE
procedures.
Acknowlegments This research was supported in part by the
Studienstiftung des deutschen Volkes with a scholarship awarded to
the first author. We thank Alena Friedrich, Yousuf Haq, and Rebecca
Wheeler, for their help in collecting and coding the data, Carol Dweck
and Ulf Reips for their support in conducting this research, and
Gregory Walton and David Yeager for helpful comments on the
manuscript.
References
Atkinson, J. W. (1981). Studying personality in the context of an
advanced motivational psychology. American Psychologist, 36,
117128.
Baumann, N., Kaschel, R., & Kuhl, J. (2005). Striving for unwanted
goals: Stress-dependent discrepancies between explicit and
implicit achievement motives reduce subjective well-being and
increase psychosomatic symptoms. Journal of Personality and
Social Psychology, 89, 781799.
Birnbaum, M. H. (2001). Introduction to behavioral research on the
Internet. Upper Saddle River, NJ: Prentice Hall.
Motiv Emot (2011) 35:251266
Birnbaum, M. H. (2004a). Methodological and ethical issues in
conducting social psychology research via the Internet. In C.
Sansone, C. C. Morf, & A. T. Panter (Eds.), Handbook of
methods in social psychology (pp. 359382). Thousand Oaks,
CA: Sage.
Birnbaum, M. H. (2004b). Human research and data collection via the
internet. Annual Review of Psychology, 55, 803832.
Blankenship, V., & Zoota, A. L. (1998). Comparing power imagery in
TATs written by hand or on the computer. Behavior Research
Methods, Instruments, & Computers, 30, 441448.
Block, J. (1985). A longitudinal study of ego and cognitive
development. Research grant application. Berkeley: University
of California Department of Psychology.
Brunstein, J. C., & Hoyer, S. (2002). Implizites und explizites
Leistungsstreben: Befunde zur Unabhangigkeit zweier Motiva-
tionssysteme [Implicit versus explicit achievement strivings:
Empirical evidence of the independence of two motivational
systems]. Zeitschrift fur Padagogische Psychologie, 16, 5162.
Brunstein, J. C., & Maier, G. W. (2005). Implicit and self-attributed
motives to achieve: Two separate but interacting needs. Journal
of Personality and Social Psychology, 89, 205222.
Brunstein, J. C., Schultheiss, O. C., & Grassmann, R. (1998). Personal
goals and emotional well-being: The moderating role of motive
dispositions. Journal of Personality and Social Psychology, 75,
494508.
Buchanan, T. (2000). Potential of the Internet for personality
research. In M. H. Birnbaum (Ed.), Psychological experiments
on the Internet (pp. 121140). San Diego: Academic Press.
Buchanan, T. (2001). Online personality assessment. In U.-D. Reips
& M. Bosnjak (Eds.), Dimensions of internet science (pp.
5774). Lengerich, Germany: Pabst Science Publishers.
Buchanan, T., Johnson, J. A., & Goldberg, L. R. (2005). Implement-
ing a five- personality inventory for use on the internet.
European Journal of Psychological Assessment, 21, 115127.
Buchanan, T., & Smith, J. L. (1999). Using the internet for
psychological research: Personality testing on the world-wide
web. British Journal of Psychology, 90, 125144.
deCharms, R., Morrison, H., Reitman, W., & McClelland, D. (1955).
Behavioral correlates of directly measured and indirectly mea-
sured achievement motivation. In D. McClelland (Ed.), Studies
in motivation. New York: Appleton-Century-Crofts.
Dillman, D. A., & Bowker, D. K. (2001). The Web questionnaire
challenge to survey methodologists. In U.-D. Reips & M.
Bosnjak (Eds.), Dimensions of internet science (pp. 159178).
Lengerich, Germany: Pabst Science Publishers.
Eagly, A. H. (1987). Sex differences in social behavior: A social-role
interpretation. Hillsdale, NJ: Erlbaum.
Eagly, A. H. (1997). Sex differences in social behavior: Comparing
social role theory and evolutionary psychology. American
Psychologist, 52(12), 13801383.
Emmons, R. A., & McAdams, D. P. (1991). Personal strivings and
motive dispositions: Exploring the links. Personality and Social
Psychology Bulletin, 17(6), 648654.
Falbo, T. (1977). Relationships between sex, sex role, and social
influence. Psychology of Women Quarterly, 2(1), 6272.
Frick, A., Bachtiger, M. T., & Reips, U.-D. (2001). Financial
incentives, personal information, and drop-outs in online studies.
In U.-D. Reips & M. Bosnjak (Eds.), Dimensions of internet
science (pp. 209219). Lengerich: Pabst Science Publishers.
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004).
Should we trust web-based studies? A comparative analysis of
six preconceptions about internet questionnaires. American
Psychologist, 59, 93104.
Job, V., & Brandstatter, V. (2009). To get a taste of your goals:
Creating motive-goal congruence by affect-focus goal-fantasy.
Journal of Personality, 77, 15271560.
265
Job, V., Oertig, D., Allemand, M., & Brandstatter, V. (in press).
Motive-discrepancies as antecedents of unhealthy eating behav-
ior. Journal of Personality.
Kehr, H. M. (2004). Implicit/explicit motive discrepancies and
volitional depletion among managers. Personality and Social
Psychology Bulletin, 30, 315327.
King, L. A. (1995). Wishes, motives, goals, and personal memories:
Relations of measures of human motivation. Journal of Person-
ality, 63, 9851007.
Klinger, E. (1968). Short-term stability and concurrent validity of
TAT and need scores: Achievement, affiliation and hostile press.
In Proceedings of the 76th Annual Convention of the American
Psychological Association, American Psychologist, 23, 878.
Langan-Fox, J., & Grant, S. (2006). The thematic apperception test:
Towards a standard measure of the big three motives. Journal of
Personality Assessment, 87, 277291.
Lundy, A. C. (1980). Conditions of test administration and TAT
validity. Unpublished manuscript, Harvard University, Depart-
ment of Psychology and Social Relations.
Lundy, A. C. (1981). Situationally evoked schema sets in TAT-based
research: A cognitive explanation for differential TAT validity.
Unpublished doctoral dissertation, Harvard University.
Lundy, A. C. (1985). The reliability of the thematic apperception test.
Journal of Personality Assessment, 49, 141145.
McClelland, D. C. (1985). Human motivation. Glenview, IL: Scott,
Foresman.
McClelland, D. C. (1991). The personal value questionnaire. Boston:
McBer & Company.
McClelland, D. C., & Jemmott, J. B. (1980). Power motivation, stress
and physical illness. Journal of Human Stress, 6, 615.
McClelland, D. C., Koestner, R., & Weinberger, J. (1989). How do
self-attributed and implicit motives differ? Psychological
Review, 96, 690702.
McClelland, D. C., & Pilon, D. A. (1983). Sources of adult motives in
patterns of parent behavior in early childhood. Journal of
Personality and Social Psychology, 44, 564574.
Mischel, W., & Shoda, Y. (1995). A cognitive-affective system theory
of personality: Reconceptualizing situations, dispositions,
dynamics, and invariance in personality structure. Psychological
Review, 102, 246268.
Murray, H. A. (1943). Thematic apperception test manual. Cam-
bridge: Harvard University Press.
Musch, J., & Klauer, K. C. (2002). Psychological experimenting on the
World Wide Web: Investigating content effects in syllogistic
reasoning. In B. Batinic, U.-D. Reips, & M. Bosnjak (Eds.), Online
social sciences (pp. 181212). Gottingen, Germany: Hogrefe.
Pang, J. S., & Schultheiss, O. C. (2005). Assessing implicit motives in
U.S. college students: Effects of picture type and position,
gender and ethnicity, and cross-cultural comparisons. Journal of
Personality Assessment, 85, 280294.
Rammstedt, B., & John, O. P. (2007). Measuring personality in one
minute or less: A 10-item short version of the Big Five Inventory
in English and German. Journal of Research in Personality, 41,
203212.
Reips, U.-D. (2000). The web experiment method: Advantages,
disadvantages, and solutions. In M. H. Birnbaum (Ed.), Psycho-
logical experiments on the Internet (pp. 89117). San Diego:
Academic Press.
Reips, U.-D. (2001). The web experimental psychology lab: Five
years of data collection on the internet. Behavior Research
Methods, Instruments, & Computers, 33, 201211.
Reips, U.-D. (2002a). Standards for internet experimenting. Exper-
imental Psychology, 49(4), 243256.
Reips, U.-D. (2002b). Internet-based psychological experimenting:
Five dos and five donts. Social Science Computer Review,
20(3), 241249.
123
266
Reips, U.-D. (2002c). Context effects in web surveys. In B. Batinic,
U.-D. Reips, & M. Bosnjak (Eds.), Online social sciences.
Seattle, WA: Hogrefe & Huber.
Reips, U.-D., & Bosnjak, M. (2001). Dimensions of internet science.
Lengerich, Germany: Pabst Science Publishers.
Reips, U.-D., Morger, V., & Meier B. (2001). Funfe gerade sein
lassen: Listenkontexteffekte beim Kategorisieren [Letting five
be equal: List context effects in categorization]. Unpublished
manuscript.
Reips, U.-D., & Neuhaus, C. (2002). WEXTOR: A Webbased tool for
generating and visualizing experimental designs and procedures.
Behavior Research Methods, Instruments, and Computers, 34,
234240.
Reuman, D. A. (1982). Ipsative behavioral variability and the quality
of thematic apperceptive measurement of the achievement
motive. Journal of Personality and Social Psychology, 43(5),
10981110.
Schultheiss, O. C. (2008). Implicit motives. In O. P. John, R.
W. Robins, & L. A. Pervin (Eds.), Handbook of personality:
Theory and research (3rd ed., pp. 603633). New York: Guilford.
Schultheiss, O. C., & Brunstein, J. C. (2001). Assessment of implicit
motives with a research version of the TAT: Picture profiles,
gender differences, and relations to other personality measures.
Journal of Personality Assessment, 77, 7186.
Schultheiss, O. C., & Brunstein, J. C. (Eds.). (2010). Implicit motives.
New York, NY: Oxford University Press.
Schultheiss, O. C., Liening, S., & Schad, D. (2008a). The reliability of
a picture story exercise measure of implicit motives: Estimates
of internal consistency, retest reliability, and ipsative stability.
Journal of Research in Personality, 42, 15601571.
Schultheiss, O. C., & Pang, J. S. (2007). Measuring implicit motives.
In R. W. Robins, R. C. Fraley, & R. Krueger (Eds.), Handbook of
research methods in personality psychology (pp. 322344). New
York: Guilford.
Schultheiss, O. C., Wirth, M. M., Waugh, C. E., Stanton, S. J., Meier,
E., & Reuter-Lorenz, P. (2008b). Exploring the motivational
brain: Effects of implicit power motivation on brain activation in
response to facial expressions of emotion. Social Cognitive and
Affective Neuroscience, 3, 333343.
Schultheiss, O. C., Yankova, D., Dirlikov, B., & Schad, D. (2009).
Are implicit and explicit motive measures statistically indepen-
dent? A fair and balanced test using the picture story exercise
and a cue- and response-matched questionnaire measure. Journal
of Personality Assessment, 91, 7281.
Schwarz, S., & Reips, U.-D. (2001). CGI versus JavaScript: A web
experiment on the reversed hindsight bias. In U.-D. Reips & M.
Bosnjak (Eds.), Dimensions of internet science (pp. 7590).
Lengerich, Germany: Pabst Science.
123
Motiv Emot (2011) 35:251266
Smith, C. P. (1992). Motivation and personality: Handbook of
thematic content analysis. New York: Cambridge University
Press.
Smith, C. P., Feld, S. C., & Franz, C. E. (1992). Methodological
considerations: Steps in research employing content analysis
systems. In C. P. Smith (Ed.), Motivation and personality:
Handbook of thematic content analysis (pp. 515536). New
York: Cambridge University Press.
Spangler, W. D. (1992). Validity of questionnaire and TAT measures
of need for achievement: Two meta-analyses. Psychological
Bulletin, 112, 140154.
Stanton, S. J., & Schultheiss, O. C. (2009). The hormonal correlates of
implicit power motivation. Journal of Research in Personality,
43, 942949.
Thrash, T. M., Elliot, A. J., & Schultheiss, O. C. (2007). Method-
ological and dispositional predictors of congruence between
implicit and explicit need for achievement. Personality and
Social Psychology Bulletin, 33, 961974.
Winter, D. G. (1988). The power motive in women and men. Journal
of Personality and Social Psychology, 54, 510515.
Winter, D. G. (1991). Manual for scoring motive imagery in running
text. Unpublished manuscript, University of Michigan.
Winter, D. G. (1992). Power motivation revisited. In C. P. Smith
(Ed.), Motivation and personality: Handbook of thematic content
analysis (pp. 311324). Cambridge: University Press.
Winter, D. G. (1994). Manual for scoring motive imagery in running
text (4th edn). Unpublished manuscript, Department of Psychol-
ogy, University of Michigan, Ann Arbor.
Winter, D. G. (2010). Power in the person: Exploring the psycho-
logical underground of power. In A. Guinote & T. Vescio (Eds.),
The social psychology of power. New York: The Guilford Press.
Winter, D. G., John, O. P., Stewart, A. J., Klohnen, E. C., & Duncan,
L. E. (1998). Traits and motives: Toward an integration of two
traditions in personality research. Psychological Review, 105,
230250.
Winter, D. G., & Stewart, A. J. (1977). Power motive reliability as a
function of test-retest instructions. Journal of Consulting and
Clinical Psychology, 45, 436440.
Winter, D. G., & Stewart, A. J. (1978). The power motive. In H.
London & J. Exner (Eds.), Dimensions of personality (pp.
391447). New York: Wiley.
Woike, B., Lavezzary, E., & Barsky, J. (2001). The influence of
implicit motives on memory processes. Journal of Personality
and Social Psychology, 81, 935945.
Woike, B., McLeod, S., & Goggin, M. (2003). Implicit and explicit
motives influence accessibility to different autobiographical
knowledge. Personality and Social Psychology Bulletin, 29,
10461055.