A Critical Appraisal of Demand Artifacts in Consumer Research

Author(s): Terence A. Shimp, Eva M. Hyatt and David J. Snyder

Source: Journal of Consumer Research, Vol. 18, No. 3 (Dec., 1991), pp. 273-283
Published by: Oxford University Press
Stable URL: https://www.jstor.org/stable/2489339
Accessed: 15-03-2020 12:24 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

Oxford University Press is collaborating with JSTOR to digitize, preserve and extend access
to Journal of Consumer Research

This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
 A Critical Appraisal of Demand Artifacts
in Consumer Research

TERENCE A. SHIMP
EVA M. HYATT
DAVID J. SNYDER*

Especially since the publication of Sawyer's instructive article on the topic, consumer
researchers have been concerned that demand artifacts significantlyvcompromise
the validity and generalizability of experimental findings. In this article we provide
an overview of the issues surrounding the demand-artifacts controversy, evaluate
the preconditions for demand artifacts, and offer a critique of suppositions about
the consequences and appropriate control of demand artifacts. Kellaris and Cox's
critique of Gorn's well-known classical conditioning experiments provides the back-
drop for much of the discussion.

A long with other social scientists (e.g., Orne 1962),
consumer researchers have been apprehensive that
demand artifacts may compromise the validity and
generalizability of experimental results. Concern be-
came especially pronounced with the appearance of
Sawyer's (1975) article at a time when the emerging
discipline of consumer research was just beginning to
deal with sophisticated reliability and validity issues.
Although Sawyer's treatise was an intelligent treatment
of the subject in step with conventional wisdom in other
disciplines, a reappraisal of demand artifacts in con-
sumer research is now needed. In this article we consider
tacit but widespread views among consumer researchers
about the prevalence, consequences, and control of de-
mand artifacts. Our argument is as follows.
1. Consumer researchers regard demand artifacts as
a commonplace if not inevitable characteristic of ex-
perimental research. This view follows from the con-
viction that (1) demand cues persist in experiments and
promote hypothesis guessing, (2) hypothesis guessing
alters subjects' natural (unbiased) response tendencies,
and (3) aggregate results are biased in a hypothesis-con-
firming (i.e., theory-supporting) direction.
*Terence A. Shimp is professor of marketing at the University of
South Carolina, Columbia, SC 29208. Eva M. Hyatt is assistant pro-
fessor of marketing at Appalachian State University, Boone, NC
28608, and David J. Snyder is an assistant professor of marketing at
Illinois State University, Normal, IL 61761. Our sincere appreciation
is extended to three reviewers for their challenging and valuable com-
ments, to editors Rich Lutz and Kent Monroe for their helpful input,
and to the following individuals for their reactions to various drafts
of the manuscript: Chris Allen, Bill Bearden, David Brinberg, Daniel
Feldman, Morris Holbrook, Donnie Lichtenstein, Tom Madden, Paul
Miniard, Randy Rose, Alan Sawyer, and Joe Urbany.
273
2. Although consumer researchers are justifiably
concerned that demand artifacts may compromise the
validity and generalizability of experimental results, we
believe concern about demand artifacts is typically
based on two equivocal premises: (1) that hypothesis
guessing equates with demand artifacts and (2) that de-
mand artifacts necessarily represent a source of system-
atic (vs. random) error.
3. Given concerns about the real or presumed un-
desirable consequences of demand artifacts, consumer
researchers often conduct postexperimental inquiries
to assess hypothesis guessing and then delete guessers
from subsequent statistical analyses. This approach, we
argue, is based on the dubious assumption that subjects
necessarily are demand biased because they appear to
have guessed the experimental hypothesis.
This article is organized as follows. In the next section
we clarify terminology and explore why the issue of
demand artifacts is so troubling to researchers. We then
specify the necessary conditions for demand artifacts
and, in this context, introduce a well-known experiment
in consumer behavior as a concrete illustration for
much of the remaining analysis. Finally, in separate
sections, we evaluate three major suppositions about
the consequences and control of demand artifacts.
THE NATURE OF DEMAND
ARTIFACTS
It is important to make clear distinctions between
the related terms demand characteristics and demand
artifacts. Demand characteristics refer here only to the
cues present in an experiment that may convey an ex-
perimental hypothesis to subjects. For example, an ex-
perimenter's verbal instructions are a potential demand
?3 1991 by JOURNAL OF CONSUMER RESEARCH, Inc. * Vol. 18 * December 1991
All rights reserved. 0093-5301/92/1803-0002$2.00

This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
 274 JOURNAL OF CONSUMER
cue. Demand artifacts, on the other hand, refer to the
bias itself, the "error of inference regarding the cause
of an observed effect" (Kruglanski 1975, p. 103). Here-
after, demand artifacts and demand bias are used in-
terchangeably but separately from demand character-
istics and demand cues.
The Troublesome Nature of
Demand Artifacts
Because behavioral laboratories are pristine and ster-
ile in comparison with natural environments, experi-
mental treatments typically are prominent. As such,
subjects can be expected to make causal judgments
about the purpose of manipulations and their role in
an experiment. Indeed, in all capacities of life, people
tend to ascribe causes to effects, or, more technically,
to make judgments of probable cause (Einhorn and
Hogarth 1986). It is precisely this tendency that makes
demand artifacts so troubling to behavioral scientists.
On the one hand, the experimenter (E) wishes to make
a scientific causal claim along the lines, "Treatment
variable X has Z amount of effect on criterion variable
Y." On the other hand, the subject (S) has reasons to
make his or her own causal judgment, albeit an informal
one involving subvocal queries along the lines: "Why
am I here? What am I supposed to be doing? What does
this person want from me?" If S judges that E's apparent
hypothesis is supposed to cause how he or she is to be-
have, then S may act on the hypothesis, behaving dif-
ferently than s/he otherwise might and thereby biasing
the data on which E relies to make his or her causal
claim.
The credibility of E's preferred explanation, or causal
conclusion, therefore hinges on discounting the alter-
native explanation that subjects discerned the experi-
mental hypothesis (i.e., judged it to be the intended
cause of their behavior) and complied with it. Unfor-
tunately, from E's perspective, this threat is not easy to
dismiss if an experimental treatment really does affect
S's causal judgment because it stands out and represents
a difference-in-a-background factor (Einhorn and Ho-
garth 1986). E's judgment about a treatment variable's
cause is thereby reduced by the gross strength of an
alternative demand-artifacts explanation.' That is,
stronger the gross strength of a demand-artifacts expla-
nation, the weaker is the net strength of E's preferred
"X causes Y" explanation. Given doubts about the net
strength of X-causes- Y claims, critics often challenge
experiments on grounds that putative cause-effect
claims are indefensible.
While such challenges are necessary to ensure that
researchers are not facile in drawing conclusions about
'A potential cause, or cue to causality, has both gross and net
strengths. Gross strength is its strength in the absence of other cues
to causality, whereas its net strength is reduced by the gross strength
of other cues (Einhorn and Hogarth 1986).
This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
RESEARCH
the effect of one theoretical construct on another, the
preconditions necessary for demand artifacts limit the
likelihood that any given subject will be demand biased.
Indeed, the original argument on which the accusation
of widespread demand artifacts is based-that the role
expectations of experimenter and subject are well de-
fined, that student subjects have a strong desire to see
experiments they participate in make a scientific con-
tribution, and that they are motivated to figure out and
validate the experimental hypothesis (Orne 1962)-is
itself founded on a set of questionable and unsupported
premises (cf. Berkowitz and Donnerstein 1982, p. 250).
NECESSARY CONDITIONS FOR
DEMAND ARTIFACTS
Three conditions must exist for a subject to be de-
mand biased.2 First, the subject must encode a demand
cue that would alert him or her to the research hypoth-
esis. Second, s/he must discern the correct research hy-
pothesis or guess another hypothesis that is incidentally
correlated with the true research hypothesis.3 Third,
s/he must act on the hypothesis by conforming to a cer-
tain role that leads to biased responses on dependent-
variable measures. In equation form,
Pr(Bi) = Pr(E1) X Pr(D1 I Ei) X Pr(AI IDi), (1)
where Pr(Bi) = probability that the ith subject will be
demand biased; Pr(Ei) = probability of encoding a de-
mand cue; Pr(D1/Ei) = conditional probability of dis-
cerning the true experimental hypothesis or a correlated
hypothesis; and Pr(A1l/Di) = conditional probability of
acting on the hypothesis.
This multiplicative function stresses that all three
conditions must apply for a subject to be demand
biased. Moreover, it shows that the overall probability
of demand bias is constrained by the probability of the
least likely event-as we shall see, a very important
consideration indeed.
2The first two preconditions are related to steps two (receptivity)
and three (motivation) in a model of the psychological process un-
derlying demand artifacts (Rosnow and Aiken 1973; Rosnow and
the
Davis 1977).
3Although an incorrectly conjectured hypothesis might be inde-
pendent of the content of the actual research hypothesis, this incorrect
hypothesis may be problematic, nonetheless, if the direction of the
response suggested by the incorrect hypothesis is correlated with that
suggested by the research hypothesis itself. For example, in a classical
conditioning experiment, the correct research hypothesis (i.e., the
hypothesis predicted by theory) is that the repeated pairing of con-
ditioned and unconditioned stimuli will promote positive attitude
formation toward the neutral conditioned stimulus by virtue of the
contingency relation established. However, subjects in a classical
conditioning experiment may conjecture, incorrectly, that they are
to evaluate an attitude object favorably simply because of its repeated
occurrence (i.e., the incorrect hypothesis corresponds to the scientific
notion of a mere exposure effect). Hence, although these hypotheses
are independent of one another in content, both predict response
behavior in the same direction.
 CRITICAL APPRAISAL OF
We elaborate on these conditions in the following
sections. To facilitate the discussion, we use as a ref-
erence point an experiment by Gorn (1982) that has
been subjected to charges of demand artifacts on two
occasions in JCR (Allen and Madden 1985; Kellaris
and Cox 1989).4 In brief, in a first experiment Gorn
presented subjects with a slide of a light blue or beige
writing pen (the conditioned stimulus [CS]) while si-
multaneously playing background music (the uncon-
ditioned stimulus [US]). Music from the movie Grease
served as a positive US, and classical Indian music pro-
vided a negative US. After exposure to the pen and mu-
sic combination, subjects selected either a light blue or
beige pen as a gift for having participated in the exper-
iment; this choice measured the conditioned effect.
Congruent with conditioning expectations, 79 percent
of the subjects picked the pen color associated with the
positive music, whereas only 30 percent picked the pen
color associated with the negative music.
There are several reasons for suspecting that Gorn's
results are vulnerable to demand artifacts (cf. Allen and
Madden 1985; Kellaris and Cox 1989). First, subjects
were run in two groups, and the first group may have
discussed the study between classes with the second
group. Second, because the study was rather straight-
forward (no confederates, no complicated procedure,
etc.), subjects may have suspected that the music was
intended to influence their choice. Third, subjects'
choice of pen color may have been biased because of
the obtrusiveness of the choice task. Finally, as college
students in a classroom, subjects might have been ap-
prehensive or inclined to play the good-subject role,
affirming the experimenter's hypothesis. We note before
continuing that these claims are plausible but not nec-
essarily correct.
Condition 1: Encoding a Demand Cue
The initiating condition for demand bias is that a
subject must encode a demand cue that provides him
or her with a basis for discerning the experimental hy-
pothesis. In general, demand-cue encoding is the most
probable of the three conditions. Subjects are likely to
detect demand cues in sterile laboratory environments.
In Gorn's ( 1982) experiments, it is virtually certain that
most subjects would have encoded the simultaneous
display of the pen and rendition of the music. Gorn's
cover story may have heightened this awareness because
he told subjects that the study dealt with the role of
music in advertising.
Condition 2: Discerning the Hypothesis
Having encoded a demand cue, or cue to causality,
a subject must use this perception to guess the experi-
4Subsequent discussion will focus only on Kellaris and Cox's (1989)
challenge to Gorn because Allen and Madden's (1985) creative re-
search employed humor as the unconditioned stimulus (US) and,
therefore, lacks direct comparability with Gorn's use of music as US.
This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
DEMAND ARTIFACTS 275
mental hypothesis. Undoubtedly, subjects do formulate
and act on some hypothesis about an experiment's pur-
pose, but the conjectured hypothesis is highly idiosyn-
cratic and, typically, incorrect (Carlopio et al. 1983).
However, as in the case of encoding, the probability
that subjects will discern the hypothesis is a function
of the particular environment. Simplistic experiments
and transparent instructions or procedures increase the
odds that subjects will guess the hypothesis correctly.
In classical conditioning experiments involving multiple
trials, for example, subjects are most likely to guess the
experimental hypothesis when no filler material is in-
terspersed between trials to confuse hypothesis guessing
(see Stuart, Shimp, and Engle 1987).
In Gorn's classical conditioning work and in related
experiments in consumer research involving a single-
trial condition, evidence suggests that subjects rarely
discern the experimental hypothesis. In their thorough
attempt to ascertain the amount of hypothesis guessing
Allen and Madden (1985) found "little indication that
participants recognized the hypothesized relationship
between the affective valence of the humorous material
and their pen choices" (p. 309). Kellaris and Cox (1989)
determined that the percentage of subjects who correctly
discerned the experimental hypothesis (i.e., that music
would affect the choice of pen color) ranged from a low
of 0.3 percent in experiment I to a high of 13.6 percent
in experiment 3. These low values support the claim of
Carlopio et al. (1983) that the ventured hypothesis is
typically incorrect.
Condition 3: Acting on the Hypothesis
Demand bias cannot materialize unless subjects act
on the surmised experimental hypothesis. Four types
of role enactment have been identified, the first two of
which are in a hypothesis-confirming direction: good
subjects attempt to confirm the hypothesis so as to ad-
vance knowledge or to pursue other noble goals; ap-
prehensive subjects do the same to avoid appearing wit-
less or to create a positive impression; negative subjects
attempt to disconfirm the hypothesis; and faithful sub-
jects act on their own inclinations rather than alter be-
havior in response to a surmised hypothesis (Sawyer
1975).
Although much has been written about the various
subject roles (see, e.g., Berkowitz and Donnerstein
1982), there is limited evidence that these roles are
widely enacted. Orne's (1962) claim that subjects act
out the good-subject role by behaving in a way that
supports the experimenter's hypothesis is basically un-
supported (Kruglanski 1975; Silverman 1977; Weber
and Cook 1972). Apprehensive role playing clearly is
the most frequently enacted of the four roles (Weber
and Cook 1972), but, even so, the evidence is mixed
regarding this role's prevalence (e.g., see Burkart 1975;
Christensen 1982; Henchy and Glass 1968; Innes and
Young 1975; Minor 1970; Rosenberg 1969; Silverman
 276 JOURNAL OF CONSUMER
1977; Turner and Simons 1974; Weber and Cook 1972).
Documentation concerning negative role playing is in-
consistent and statistically suspect (Greenberg and Fol-
ger 1988); this role is of limited interest anyway because
claims of demand artifacts are typically predicated on
the assumption that subjects act in a theory-supporting
direction, that is, either as good or apprehensive sub-
jects.
Again returning to Gorn's (1982) experiment, one
cannot plausibly argue that his business-student subjects
selected pen colors to advance science or that they felt
compelled to make a good impression on the experi-
menter. Likewise, one would have difficulty arguing that
Allen and Madden's (1985) and Kellaris and Cox's
(1989) nonsupportive results were due to subjects acting
in a contrary, disconfirming manner.
COMMON SUPPOSITIONS ABOUT
DEMAND ARTIFACTS
Having identified the fundamental issues involved in
the demand-artifacts controversy, we now appraise
three suppositions that extend, if only tacitly, from the
view that demand artifacts are prevalent in experimental
consumer research: (1) that failure to replicate prior
experimental findings may indicate demand artifacts
are responsible for the initial findings; (2) that putative
theory-supporting evidence actually may represent little
more than false-positive readings due to the presence
of demand artifacts; and (3) that demand artifacts can
be controlled by conducting postexperimental inquiries
and deleting hypothesis guessers.
Supposition 1: Failure to Replicate Indicates
Demand Artifacts
When an experiment is replicated, chances are that
the exact or even similar findings will not be obtained
(cf. Epstein 1980). This can be explained by at least five
factors, including improper replication (e.g., nonequiv-
alent manipulations), nonrobust theory, the presence
of demand artifacts in the original or replicated data,
sampling effects, and laboratory effects.5 These multiple
possibilities notwithstanding, consumer researchers
tend to attribute failed replications to demand artifacts
or theory frailties while overlooking quirky sampling
and laboratory effects.
Kellaris and Cox's (1989) challenge to Gorn (1982)
is a typical demand-artifacts explanation for nonequiv-
alent results between original and replicated experi-
ments. In an attempt to replicate Gorn's findings (or,
alternatively, to examine the robustness of his results),
5Laboratory effects involve differences across experiments in such
context-specific factors as physical and temporal conditions, exper-
imenters, and procedures. It is hard to control or even assess the
magnitude of laboratory effects (cf. Epstein 1980; see also Rosenthal
and Rubin 1980).
This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
RESEARCH
Kellaris and Cox conducted two experiments (experi-
ments 1 and 3) and a nonexperiment (experiment 2).6
Although their two experiments, especially experiment
3, replicated the procedures in Gorn's first experiment
with only minor modifications, neither replication sup-
ported Gorn's finding that music conditioned pen-color
preference. However, subjects in the nonexperiment
who were asked to imagine themselves listening to "a
very pleasant, likable piece of upbeat music taken from
a popular film" were significantly more likely to select
the advertised pen color than were subjects who imag-
ined themselves listening to "a rather unpleasant piece
of Indian classical music that most people would dis-
like." Does the fact that Kellaris and Cox could not
replicate Gorn's findings in their experiments but could
do so in a nonexperiment suggest the possibility that
Gorn's results may have been due to demand artifacts
and not to conditioning?
This certainly is what these authors have insinuated
and apparently what others have gleaned from their ar-
ticle (e.g., see n. 4 in Bruner 1990). In the following we
challenge Kellaris and Cox's demand-artifacts inter-
pretation of Gorn's results and argue more generally
against the tendency to accept too readily demand-ar-
tifact explanations for nonreplicated results.
First, although Kellaris and Cox were conscientious
and clever in their effort to replicate Gorn's study, their
music selections serving as US were not identical to
Gorn's. Although we cannot identify any obviously
crucial differences, basic research has established that
slight changes in CS-US combinations can alter the
likelihood, rate, and magnitude of conditioning (e.g.,
Domjan and Burkhard 1986; Swartz 1989).
Second, the fact that Kellaris and Cox's nonexperi-
ment supports Gorn's findings is not in itself convincing
evidence that Gorn's results were due to demand arti-
facts. It is well-known that nonexperiments produce a
special mental set that is unique to role-playing subjects
(Kruglanski 1975). That Kellaris and Cox's nonexper-
iment obtained results congenial with Gorn's original
findings may simply establish that the nonexperiment
worked as it should and that subjects readily surmised
that the experimenter expected them to select (not se-
lect) the pen color associated with the appealing (un-
appealing) music. Another possibility is that the subjects
in the nonexperiment merely predicted their own be-
havior without any attempt at guessing what the ex-
perimenter expected. In either case, Kellaris and Cox's
6In a nonexperiment subjects listen to a description of an experi-
mental treatment and then are asked how they would respond to the
treatment. Nonexperiments implicitly encourage subjects to surmise
the experimental hypothesis and act on it. The purpose is to ascertain
whether an actual experiment might be susceptible to demand arti-
facts. If a nonexperiment fails to replicate a true experiment, demand
artifacts can be ruled out as a cause for the true experiment's results;
yet, if a nonexperiment replicates a true experiment, this does not
necessarily mean that the experiment includes a demand artifact. For
another application of nonexperiments, see Lim and Summers (1984).
 CRITICAL APPRAISAL OF DEMAND ARTIFACTS 277

nonexperimental results are not probative regarding the TABLE 1

matter of whether Gorn's findings are demand biased. DATA AND ESTIMATES BASED ON KELLARIS
A final challenge can be mounted on the basis of the AND COX'S EXPERIMENT 2
logic Kellaris and Cox employ when speculating that
Gorn's findings reflect demand artifacts rather than Pen choice (B)

conditioned behavior. They argue that the absence of

Choice of Choice of
significance in their two actual experiments indicates
advertised nonadvertised
that pen-color preference was not conditioned by music Music appeal (A) pen (B1i) pen (B2) Total
and that the significant relationship in their nonexper-
iment between the music description and pen-color A. Actual data:a

choice had to be due to demand artifacts. They therefore Liked (Al) 45 25 70

Disliked (A2) 24 42 66
go on to suggest that Gorn's significant results were
themselves probably due to demand artifacts. We argue, Total 69 67 136
on the other hand, that, if their replications are not
B. Estimated data:b
demand biased, then Gorn's results probably are not
Liked (Al) 45 25 70
demand biased either since, as already noted, Kellaris Disliked (A2) 24 42 66
and Cox's experiments are close approximations of
Gorn's. Total 69 67 136
Consider first Kellaris and Cox's experiment 1, in
aData are from Ke
which a postexperimental inquiry found that only one bData are based on Kellaris and Cox's measure of 29 percent hypothesis
of 299 subjects guessed the experimental hypothesis. guessers and the assumption that 100 percent of guessers enacted a hypothesis-
The fact that fewer than one-half of 1 percent of the confirming role; X2 = 10.63, p <.05.

subjects discerned the research hypothesis is proof pos-

itive that demand artifacts could not have influenced
findings obtained in this initial experiment. However,
a more appropriate comparison is experiment 3, which
more closely matched Gorn's experiment. In this ex-
periment, in which Gorn's own cover story was used,
Kellaris and Cox determined that 13.6 percent of the
subjects (12 of 88) guessed the hypothesis. This, of
course, simply establishes that about one of seven sub-
jects may have been demand biased. Even for these
subjects bias ultimately depends on their enacting hy-
pothesis-supporting behavior in the role of good or ap-
prehensive subjects. By making assumptions about the
probability of each of the three demand-biasing events
(i.e., Ei, D1/Ei, A1/Di; see Eq. 1), it is possible to re-
construct Kellaris and Cox's data in experiment 3 and
ascertain what their results would have been if all hy-
pothesis guessers had acted on the hypothesis and be-
come demand biased.
Building Demand Bias into Kellaris and Cox's Data.
In justifying assumptions about these probabilities, it
is useful first to examine data from their nonexperiment
(experiment 2). Part A of Table 1 presents actual data
from Kellaris and Cox's nonexperiment. Part B, which
for reasons explained below is identical to part A, es-
timates what the data would have looked like if all hy-
pothesis guessers had been demand biased. The recon-
structed data (part B) are based on two assumptions
and one fact: (1) the probability is 1 that subjects would
have encoded a demand cue in this intentionally trans-
parent nonexperiment; (2) the probability is .29 that
subjects discerned the hypothesis-a fact based on Kel-
laris and Cox's own data indicating that 29 percent of
their nonexperimental subjects guessed the hypothesis;
and (3) the probability is 1 that subjects would have
acted on the hypothesis as good or apprehensive sub-
jects. Taken together, this yields a probability of .29
that subjects would have been demand biased (i.e.,
Pr(Bi) = 1 X .29 X 1). If this probability is applied to
all 136 subjects who participated in experiment 2, up
to 39 could have been demand biased (i.e., .29 X 136).
On the basis of the row marginals in the actual data
(i.e., 70 subjects exposed to the "liked" and 66 to the
"disliked" music; part A), it can be deduced that 20 of
the 39 demand-biased subjects would have been exposed
to the liked and 19 to the disliked music. Additionally
assuming that the remaining 50 liked and 47 disliked
nondemand-biased subjects made pen choices at ran-
dom (since they were not demand biased and could not
have been conditioned by the nonexperiment), we ob-
tain the frequency counts in part B by adding the 20
and 19 demand-biased subjects to the numbers of non-
biased subjects in the good-subject-congruent cells (i.e.,
Al B 1 and A2B2). That is, good and apprehensive sub-
jects exposed to the liked (disliked) music should have
chosen the advertised (nonadvertised) pen color. Given
these assumptions, the data in part B perfectly replicate
the actual data in part A.
Our logic thus provides an appropriate estimate of
the maximum number of subjects who could have been
demand biased in experiment 3. Part A of Table 2 re-
produces Kellaris and Cox's results, while part B esti-
mates what the results would have looked like under
the assumption that 100 percent of hypothesis guessers
were indeed demand biased. The data in part B are
based on the same two assumptions made above; how-
ever, the probability of discerning the hypothesis (i.e.,
Pr(Dl/Ei)) now is set at the lower level of .136, which
is the percent of hypothesis guessers in the relevant

This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
 278 JOURNAL OF CONSUMER RESEARCH

TABLE 2 TABLE 3

DATA AND ESTIMATES BASED ON KELLARIS DATA AND ESTIMATES BASED ON GORN'S EXPERIMENT 1
AND COX'S EXPERIMENT 3

Pen choice (B)

Pen choice (B)
Choice of Choice of
Choice of Choice of advertised nonadvertised
advertised nonadvertised Music appeal (A) pen (B1) pen (B2) Total
Music appeal (A) pen (B1) pen (B2) Total
A. Actual data:a
A. Actual data:a Liked (Al) 74 20 94
Liked (Al) 22 19 41 Disliked (A2) 30 71 101
Disliked (A2) 23 24 47
Total 104 91 195
Total 45 43 88
B. Estimated data:b
B. Estimated data:b Liked (Al) 61 33 94
Liked (Al) 25 16 41 Disliked (A2) 44 57 101
Disliked (A2) 20 27 47
Total 105 90 195
Total 45 43 88
aData are from Gor
aData are from bData are based on Kellaris and Cox's measure of
Kellaris 13.6 percent hypothesis
and Cox
NS. guessers and the assumption that 100 percent of guessers enacted a hypothesis-
bData are based on Kellaris and Cox's measure of 13.6 percent hypothesis confirming role; x2 = 8.90, p < .01.
guessers and the assumption that 100 percent of guessers enacted a hypothesis-
confirming role; x2 = 2.97, p < .1.

evidence anything other than demand artifacts. More

Gorn-procedure condition of Kellaris and Cox's ex-
periment 2. If the probability of the other two demand-
biasing events (i.e., Pr(Ei) and Pr(Al/Di)) is assumed to
be 1, then Pr(Bi) is .136.
With this estimate, a maximum of 12 subjects (.136
X 88) could have been demand biased, with six subjects
exposed to the liked music version and six to the disliked
music version. Under the additional assumption that
some subjects may have been influenced by the exper-
imental treatments, the 76 remaining nondemand-
biased subjects are allocated to the four cells exactly in
proportion to what Kellaris and Cox's actual data (part
A) reveal. For example, of the 35 unbiased subjects who
were exposed to liked music, 54 percent, or 19 subjects,
should have chosen the advertised pen. Adding the six
demand-biased subjects to cell Al B 1 (the demand-con-
gruent cell) results in an estimate of 25 subjects who
would have selected the advertised pen after listening
to appealing music. Similarly, adding the six demand-
biased subjects in the disliked music condition who,
acting as good or apprehensive subjects, would have
selected the nonadvertised pen results in an estimate of
27 subjects in cell A2B2.
On the basis of these- assumptions, part B shows a
marginally significant relationship between music ap-
peal and pen choice (X2 = 2.97; p < .1). This demand-
biased result indicates, by implication, that Kellaris and
Cox's actual data from experiment 3 also would have
yielded a statistically significant relationship had those
data been demand biased to the magnitude indicated
by the proportion of hypothesis guessers. Thus, the fact
that the actual data (Table 2, part A) show no significant
relationship indicates that Kellaris and Cox's results
will be said about this shortly.
Removing Demand Bias from Gorn's Data. Re-
versing the above process, we can investigate what
Gorn's (1982) results would have looked like if the pu-
tatively demand-biased subjects in his frequency table
were removed from the demand-congruent cells. The
data from his first experiment are displayed in part A
of Table 3. Because Gorn did not report the specific
incidence of hypothesis guessing, we assume that per-
cent to be the same as in Kellaris and Cox's experiment
3 in which Gorn's cover story was used, namely, 13.6
percent.
With this assumption, part B of Table 3 shows that
potentially 27 subjects would have been demand biased
(i.e.,.136 X 195). Proportionately, 13 would have been
exposed to the liked music and 14 to the disliked music.
Removing these demand-biased subjects from the de-
mand-congruent cells (Al B 1 and A2B2) yields a fre-
quency distribution that remains statistically significant
(x2 = 8.90, p < .0 1). In other words, even allowing the
possibility that 27 of Gorn's subjects chose pen colors
acting as good or apprehensive subjects, the results are
nevertheless statistically significant, because sufficient
numbers of nondemand-biased subjects responded in
a conditioning-congruent fashion. Thus, Gorn's original
conclusion that his data evidence classical conditioning
is sustained.
Section Summary. These reconstructed results
show that statistically significant relations are obtained
after building demand artifacts into the data from Kel-
laris and Cox's experiment 3. Thus, their original data
would themselves have been statistically significant had

This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
 CRITICAL APPRAISAL OF
their experiments contained demand-biased subjects to
the extent suggested by their postexperimental evidence
of hypothesis guessing. Furthermore, on removing po-
tentially demand-biased subjects from Gorn's data
through the reverse process, we show that his results
still warrant a nonartifactual, classical conditioning ex-
planation.
The fact that Kellaris and Cox's results are not sta-
tistically significant indicates (1) that pen choice in their
experiments was not influenced by music appeal and,
hence, failed to evidence conditioning and (2) that de-
mand artifacts did not bias their results. Hence, because
Kellaris and Cox's replications reveal no evidence of
demand artifacts, it is only logical to conclude that
Gorn's findings, too, are probably more attributable to
successful classical conditioning than to demand arti-
facts.7 It is beyond the scope of this article to explain
why conditioning occurred in Gorn's experiments but
not in Kellaris and Cox's. As noted earlier, basic dif-
ferences in the choice of US may account for the fact
that near-equivalent experiments sometimes do not
yield comparable results. Also, the possibility remains
that Gorn's results are artifactual even though, as we
have shown, Kellaris and Cox's research does not sus-
tain this conclusion.
More generally, this article shows that it is easier to
assert than to justify a demand-artifacts-based indict-
ment of a nonreplicated experiment. Furthermore, us-
ing demand artifacts to explain nonreplicated results
may amount to supplanting a more probable sampling-
or laboratory-effects explanation with a less probable
demand-artifacts explanation. The issue of sampling ef-
fects requires no commentary here, but some brief re-
marks about laboratory effects are in order. In partic-
ular, no matter how similar two experiments may
appear, the fact always remains that differences between
experimenters, laboratory conditions, time of day, time
of semester, time of year, and so on can dramatically
affect experimental outcomes. In addition, experimental
findings are inherently unstable when results are based
on single items of data, as is the case with the pen-color
choice measure in Gorn and Kellaris and Cox's exper-
7Because Gorn employed what, on the surface, appears to be a
more obtrusive measure of pen selection than did Kellaris and Cox
in their replication, it could be claimed that his experiment is more
susceptible to demand bias than our position has acknowledged.
However, we would argue that the pen-selection procedure in Gorn's
experiment may have been less, not more, obtrusive than Kellaris
and Cox's procedure. Whereas Gorn's subjects left their seats en masse
(thereby providing a sense of obscurity) and selected a preferred pen
color from one of two boxes located on opposite sides of the room,
Kellaris and Cox's subjects individually selected a preferred pen color
from their seats as a box containing both pen colors was passed around
the room. It therefore is possible that Kellaris and Cox's subjects
were more likely than Gorn's to think that their choice was being
observed and recorded. As such, Kellaris and Cox's observation pro-
cedure actually may have been more reactive, more likely to prompt
evaluation apprehension, and hence more biasing than Gorn's (cf.
Kazdin 1982).
This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
DEMAND ARTIFACTS 279
iments, as well as much other consumer research. Ep-
stein (1980, p. 793) cogently expresses the problem as
follows: "Behavior observed on a single occasion,
whether in the laboratory or elsewhere, is apt to be so
situationally unique as to be incapable of establishing
reliable generalizations that hold over even the most
minor variations in the situation. To the extent that
this is true, it follows that the laboratory study as nor-
mally conducted is often unreplicable." This assertion
has received support from related work in consumer
research (Lastovicka and Joachimsthaler 1988). The
implication is clear: single measures of complex behav-
iors are as unreliable as single-item measures of attitudes
and other psychological constructs.
Supposition 2: Demand Artifacts Create
False-Positive Readings
Measurement errors, both random and systematic,
are pervasive in behavioral research (cf. Cote and
Buckley 1987, 1988). Although both forms of error af-
fect conclusions regarding the true relation between
theoretical constructs, systematic errors are more prob-
lematic in that they attenuate or inflate estimates of
relationships, whereas random errors only attenuate
observed relations (Cote and Buckley 1988). Thus,
whether demand bias represents random or systematic
error holds important implications for how journal re-
viewers and other critics apply claims of demand arti-
facts in judging theory tests.8
It has been conventional to see demand cues as sys-
tematically inflating observed scores on dependent
variables in a theory-supporting direction.9 However,
the notion that demand characteristics produce sys-
tematic bias necessarily implies that subjects guess the
same, or a similar, hypothesis and enact the same hy-
pothesis-supporting role(s), thus ultimately affecting
aggregate behavior in a uniform direction (Kruglanski
1975). For subjects to discern the same hypothesis and
act the same way is, as previously noted, improbable;
indeed, the direction of bias is inconsistent even in
studies in which subjects are given the true hypothesis
(Kruglanski 1975). Yet experimental data are likely to
be compromised by systematic demand effects when
cues are present that both (1) arouse concern about how
one is to be appraised and (2) provide hints about ap-
propriate responses for obtaining favorable evaluations
from experimenters (Fromkin and Streufert 1976).
Arousal cues are present when subjects sense that
their personalities, intelligence, or judgment is being
appraised, as is the case when subjects are run individ-
8We note that the term demand bias suggests by definition that the
error is systematic; however, this usage is a matter of tradition and
is not necessarily appropriate.
9The theory-supporting qualification is made here on the premise
that consumer researchers are more concerned with preventing false-
positive than false-negative results.
 280 JOURNAL OF
ually or given the impression that their individual per-
formance is important. Appropriate-response cues are
most common when frequent or continuous feedback
is given based on subject performance (cf. Gergen 1965)
or when the experimenter makes the purpose of the
experiment transparent.
In cases in which both arousal and appropriate-re-
sponse cues are present, some subjects can be expected
to respond in a uniform fashion, thereby producing a
systematic response bias between treatment groups if
cues are correlated with treatment conditions. However,
when only an arousal cue is present, subjects must gen-
erate their own notions of an appropriate response.
Since subsets of subjects would formulate different no-
tions of an appropriate response and since errors across
subsets would generally offset one another, the resulting
overall response error would be random rather than
systematic.
It is most appropriate, then, to invoke demand ar-
tifacts to account for theory-supporting results in cases
in which it is likely that a treatment group's scores erred
systematically because all or, more reasonably, some of
the subjects enacted a hypothesis-confirming role. But
this, unfortunately, is not easy to demonstrate. The im-
plicit assumption that hypothesis guessing necessarily
results in demand artifacts is problematic, for all hy-
pothesis guessers do not enact the same role. It may be
that theory-supporting experiments should not be the
only, or even primary, target of consumer researchers'
apprehension about demand artifacts. Rather, in view
of the well-known bias against the null hypothesis (At-
kinson, Furlong, and Wampold 1982; Greenwald 1975),
concerns about demand artifacts might more properly
be directed at file drawers of unsubmitted studies in
which the theory was not supported because demand-
based random errors attenuated observed relations be-
tween theoretical constructs.
Lest we be misinterpreted as suggesting that theory-
supporting results can never be ascribed to demand ar-
tifacts, let us note that, while it is unlikely that large
numbers of subjects are demand biased in competently
conducted experiments, the possibility remains that a
small proportion of demand-biased subjects may sway
experimental findings in a theory-supporting direction.
Unfortunately, this recognition brings no solution with
it, for no simple solution is possible. The canonical pro-
cedure for eliminating demand bias in consumer re-
search has been postexperimental measurement and
removal of hypothesis guessers, but this approach is
predicated on the dubious assumptions that hypothesis
guessing can be measured validly and that hypothesis
guessers necessarily enact hypothesis-confirming roles.
Supposition 3: Deleting Hypothesis Guessers
Controls the Problem
The most common method in consumer research for
dealing with demand artifacts is to perform postexper-
This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
CONSUMER RESEARCH
imental inquiries (PEIs) and delete subjects who appear
to have surmised the experimental hypothesis. It seems
to be widely accepted that postexperimental measure-
ment of hypothesis guessing controls the problem of
demand artifacts. However, as noted above, this as-
sumes that hypothesis guessing can be measured validly
and that hypothesis guessing necessarily leads to hy-
pothesis-confirming role playing. Both assumptions are
questionable in that the measurement of hypothesis
guessing is tricky and potentially invalid (see Gorn, Ja-
cobs, and Mana 1987), and role playing, if undertaken
at all, need not materialize in a hypothesis-confirming
direction.
Thus, our position is that the PEI, rather than elim-
inate bias, may in fact introduce another form of bias
that is systematically related to the dependent variable.
Specifically, hypothesis-guessing subjects may system-
atically differ from nonguessers in being more alert, in-
telligent, and sophisticated, in having a higher need for
cognition, and so on. Therefore, in a between-subjects
design, subjects in the treatment group who remain after
hypothesis guessers have been removed may be fun-
damentally different (less alert, etc.) than subjects in
the control group against whom they are compared,
since, by definition, only subjects in experimental
groups have an opportunity to guess the experimental
hypothesis and be deleted. Stated alternatively, the pool
of treatment-group subjects on whose responses statis-
tical analyses are performed (i.e., the nonguessers) may
not be representative of the population to which the
experimenter generalizes findings. Consequently, re-
moving hypothesis guessers may lead to systematic dif-
ferences between comparison groups along the lines of
the selection-bias threat to internal validity noted by
Cook and Campbell (1979, p. 5 3). In addition to threat-
ening internal validity, removing hypothesis guessers
may also adversely affect an experiment's external va-
lidity and in turn compromise its construct validity (cf.
Lynch 1982).
Because it can never be known with certainty whether
the subset of data without hypothesis guessers is more
or less biased than the full data set, albeit in different
ways, it is probably most prudent to analyze experi-
mental data both with and without hypothesis guessers.
In the event that statistical conclusions are incongruous,
it would then be appropriate for consumer researchers
to note the differences. Readers could draw their own
conclusions regarding the possible reasons for and im-
port of the inconsistencies.
DISCUSSION
The publication process invariably involves some
friction between authors and the editors and reviewers
who are the gatekeepers responsible for assuring that
authors' theory-supporting results do not have alter-
native explanations. This interaction is healthy insofar
as it leads to published articles that are more circum-
 CRITICAL APPRAISAL
spect than they otherwise might be. However, demand
artifacts are often accentuated as a source of error to
the exclusion of the myriad other errors that are just as
likely, or perhaps more likely, to jeopardize the validity
and generalizability of experimental results. For ex-
ample, idiosyncratic subject states (e.g., mood) and ex-
perimental contextual conditions, or laboratory effects
(e.g., time of day, day of week, laboratory atmospherics),
undoubtedly affect how subjects respond to experimen-
tal stimuli; however, these factors are typically assumed
away.
Another potentially unhealthy offshoot of the em-
phasis on demand artifacts is that otherwise meritorious
studies are challenged on grounds that they are fatally
flawed by demand artifacts. Gatekeepers sometimes as-
sert that manuscripts are unacceptable because of de-
mand artifacts, without articulating exactly how such
artifacts might have occurred, and it is easier for critics
to raise the specter of demand bias than it is for re-
searchers to counter such claims. For example, an
anonymous reviewer of a different paper commented
that "those of us who believe in practicing safe science
would contend that it is prudent to place the onus on
the experimenter to show that [results are not demand
biased]." Although on the surface this comment is ir-
refutable (who does not want to practice safe science?),
it imposes an impossible task on experimenters. How
is one to prove that his or her results are not demand
biased? Certainly, as we have argued, conducting PEIs
and deleting hypothesis guessers does not solve the
problem, because the measurement of hypothesis
guessing has questionable validity and the act of hy-
pothesis guessing, even if measured validly, is not
equivalent to being demand biased. Moreover, deleting
hypothesis guessers may itself create a form of system-
atic bias that is more insidious than the presumed bias
thought to be associated with demand artifacts. Al-
though consumer researchers believe they are doing the
right thing when they conduct PEIs and remove hy-
pothesis guessers, this behavior may merely serve to
give the impression of safe science and thereby assuage
peer criticism. 1
Demand Artifacts beyond the Laboratory
The concept of demand artifacts has its roots in the
domain of laboratory experimentation, and it is here
where consumer researchers' primary concerns remain.
It is important to recognize, however, that biasing akin
to demand artifacts is possible in any form of data col-
lection that both arouses respondents' concerns for how
they are being evaluated and provides cues to appro-
priate response behavior.
"0This is not to suggest that PEIs have no role in consumer research.tifacts. At the same time it is important to caution
Indeed, the PEI is invaluable at the pretesting and pilot testing stages
of the research process (Lim and Summers 1984), and in the final
experiment it can offer reassuring results by establishing that subjects tended ways. Specifically, it would be incorrect to find
did not guess the experimental hypothesis.
This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
OF DEMAND ARTIFACTS 281
The wording of questions and the context in which
questions appear can affect the responses to question-
naires and interviews (Bradburn 1982). Respondents
may sense what the researcher is getting at and alter
their responses to accommodate what they think the
researcher expects. In-depth interviews in the interpre-
tivist tradition are particularly susceptible to demand-
type artifacts. The same individual often interviews
numerous participants, and though in strict naturalistic-
inquiry terms the researcher has no a priori agenda or
hypotheses (Lincoln and Guba 1985), it is only human
to have expectations about the research outcome or to
form expectations during the course of the, research.
Strong rapport often develops between a researcher and
informant during an in-depth interview, rapport that
may encourage the informant to be especially concerned
with the impression s/he conveys to the researcher.
When informants say what they think the researcher
wants to hear, the bias threatens an ethnographic study's
integrity every bit as much as demand artifacts jeop-
ardize the validity and generalizability of experimental
results in the laboratory (see Wallendorf and Belk 1989).
The observation method also is susceptible to de-
mand-type bias, particularly when observations are ob-
trusive (Kazdin 1982). Obtrusive observation evokes
reactivity since people sometimes alter their behavior
when aware that they are being observed. Just as in the
case of experimental demand artifacts, obtrusive ob-
servation can influence apprehensive subjects to alter
their behavior so as to be viewed in a more socially
desirable light (Kazdin 1982).
The point here is to emphasize that something par-
allel to demand artifacts is a fact of research life re-
gardless of which data-collection method is used. In
keeping with our earlier probable-cause (Einhorn and
Hogarth 1986) account of demand artifacts in the lab-
oratory, we do not mean to suggest now that the po-
tential for demand artifacts is equivalent within and
outside the laboratory; the point, instead, is that many
of the issues are similar. Perhaps it is time for researchers
to accept the fact that experiments are not all that dif-
ferent from other data-collection methods; indeed, de-
mand-cue encoding, hypothesis conjecturing, and role
enacting are universal phenomena.
Some Final Caveats
In pointing out that demand-type biasing crosses all
modes of data collection we do not mean to imply that
researchers need not be concerned with demand arti-
facts in experimental consumer research. Rather, the
purpose of this article has been to confront tacit beliefs
among consumer researchers regarding the supposed
consequences and appropriate control of demand ar-
against any tendency to use our arguments in unin-
here a claim that demand artifacts are unimportant. On
 282 JOURNAL OF CONSUMER
the contrary, any aspect of experimentation that in-
creases measurement error, random or systematic, is a
matter of concern. Experimenters must attempt to
minimize the likelihood that subjects will discern and
act on transparent hypotheses. Various procedures for
accomplishing this have been discussed and need not
be repeated here (e.g., Rosenthal and Rosnow 1984, pp.
106-107; Sawyer 1975).
On the other hand, consumer researchers could ap-
propriately use our arguments to fend off unjustified
accusations when critics (1) challenge an experiment
on grounds that it is demand biased without articulating
how it is biased, (2) discount theory-supporting results
as actually due to demand artifacts, or (3) claim that
an experiment is fatally flawed science just because a
PEI has not been performed and hypothesis guessers
removed. Although it would be imprudent for consumer
researchers to summarily dismiss demand artifacts and
assume that they never threaten the validity of exper-
imental findings, accusations of demand artifacts must
be made judiciously to ensure that the presumption of
demand bias does not overrule reason.
[Received March 1990. Revised April 1991.]
REFERENCES
Allen, Chris T. and Thomas J. Madden (1985), "A Closer
Look at Classical Conditioning," Journal of Consumer
Research, 12 (December), 301-315.
Atkinson, Donald R., Michael J. Furlong, and Bruce E.
Wampold (1982), "Statistical Significance, Reviewer
Evaluations, and the Scientific Process: Is There a (Sta-
tistically) Significant Relationship?" Journal of Coun-
seling Psychology, 29 (March), 189-194.
Berkowitz, Leonard and Edward Donnerstein (1982), "Ex-
ternal Validity Is More Than Skin Deep: Some Answers
to Criticisms of Laboratory Experiments," American
Psychologist, 37 (March), 245-257.
Bradburn, N. M. (1982), "Question Wording Effects in Sur-
veys," in Question Framing and Response Consistency,
ed. Robin M. Hogarth, San Francisco: Jossey-Bass, 65-
76.
Bruner, Gordon C. (1990), "Music, Mood, and Marketing,"
Journal of Marketing, 54 (October), 94-104.
Burkart, Barry R. (1975), "Apprehension about Evaluation,
Paralanguage Cues, and the Experimental-Bias Effect,"
Psychological Reports, 39 (August), 15-23.
Carlopio, James, John G. Adair, R. C. L. Lindsay, and Barry
Spinner (1983), "Avoiding Artifact in the Search for Bias:
The Importance of Assessing Subjects' Perceptions of the
Experiment," Journal of Personality and Social Psy-
chology, 44 (April), 693-701.
Christensen, Larry (1982), "Examination of Subject Roles:
A Critique of Carlston and Cohen," Personality and So-
cial Psychology Bulletin, 8 (September), 579-583.
Cook, Thomas D. and Donald T. Campbell (1979), Quasi-
Experimentation: Design and Analysis Issues for Field
Settings, Boston: Houghton Mifflin.
Cote, Joseph A. and M. Ronald Buckley (1987), "Estimating
Trait, Method, and Error Variance: Generalizing across
This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
RESEARCH
70 Construct Validation Studies," Journal of Marketing
Research, 24 (August), 315-318.
and M. Ronald Buckley (1988), "Measurement Error
and Theory Testing in Consumer Research: An Illustra-
tion of the Importance of Construct Validation," Journal
of Consumer Research, 14 (March), 579-582.
Domjan, Michael and Barbara Burkhard (1986), The Prin-
ciples of Learning and Behavior, Monterey, CA: Brooks/
Cole.
Einhorn, Hillel J. and Robin M. Hogarth (1986), "Judging
Probable Cause," Psychological Bulletin, 99 (1), 3-19.
Epstein, Seymour (1980), "The Stability of Behavior. II. Im-
plications for Psychological Research," American Psy-
chologist, 35 (September), 790-806.
Fromkin, Howard L. and Siegfried Streufert (1976), "Labo-
ratory Experimentation," in Handbook of Industrial and
Organizational Psychology, ed. Marvin D. Dunnette,
Chicago: Rand McNally, 415-465.
Gergen, Kenneth J. (1965), "The Effects of Interaction Goals
and Personalistic Feedback on the Presentation of Self,"
Journal of Personality and Social Psychology, 1 (5), 413-
424.
Gorn, Gerald J. (1982), "The Effects of Music in Advertising
on Choice Behavior: A Classical Conditioning Ap-
proach," Journal of Marketing, 46 (Winter), 94-10 1.
, W. J. Jacobs, and Michael J. Mana (1987), "Obser-
vations on Awareness and Conditioning," in Advances
in Consumer Research, Vol. 14, ed. Melanie Wallendorf
and Paul F. Anderson, Provo, UT: Association for Con-
sumer Research, 415-416.
Greenberg, Jerald and Robert Folger (1988), Controversial
Issues in Social Research Methods, New York: Springer.
Greenwald, Anthony G. (1975), "Consequences of Prejudice
against the Null Hypothesis," Psychological Bulletin, 82
(January), 1-20.
Henchy, Thomas and David C. Glass (1968), "Evaluation
Apprehension and Social Facilitation of Dominant and
Subordinate Responses," Journal of Personality and So-
cial Psychology, 10 (December), 446-454.
Innes, John M. and Roger F. Young (1975), "The Effect of
Presence of an Audience, Evaluation Apprehension, and
Objective Self-Awareness on Learning," Journal of Ex-
perimental Social Psychology, 11 (January), 35-42.
Kazdin, Alan E. (1982), "Observer Effects: Reactivity of Di-
rect Observation," in Using Observers to Study Behavior,
ed. Donald P. Hartmann, San Francisco: Jossey-Bass, 5-
19.
Kellaris, James J. and Anthony D. Cox (1989), "The Effects
of Background Music in Advertising: A Reassessment,"
Journal of Consumer Research, 16 (June), 113-118.
Kruglanski, Arie W. (1975), "The Human Subject in the Psy-
chology Experiment: Fact and Artifact," in Advances in
Experimental Social Psychology, Vol. 8, ed. L. Berkowitz,
Orlando, FL: Academic Press, 101-147.
Lastovicka, John L. and Erich A. Joachimsthaler (1988),
"Improving the Detection of Personality-Behavior Re-
lationships in Consumer Research," Journal of Consumer
Research, 14 (March), 583-587.
Lim, Jeen-Su and John 0. Summers (1984), "A Non-exper-
imental Investigation of Demand Artifacts in a Personal
Selling Experiment," Journal of Marketing Research, 21
(August), 251-258.
 CRITICAL APPRAISAL OF
Lincoln, Yvonna S. and Egon G. Guba (1985), Naturalistic
Inquiry, Beverly Hills, CA: Sage.
Lynch, John G., Jr. (1982), "On the External Validity of Ex-
periments in Consumer Research," Journal of Consumer
Research, 9 (December), 225-239.
Minor, Marshall W. (1970), "Experimenter-Expectancy Effect
as a Function of Evaluation Apprehension," Journal of
Personality and Social Psychology, 15 (August), 326-
332.
Orne, Martin T. (1962), "On the Social Psychology of the
Psychological Experiment: With Particular Reference to
Demand Characteristics and Their Implications," Amer-
ican Psychologist, 17 (November), 776-783.
Rosenberg, M. J. (1969), "The Conditions and Consequences
of Evaluation Apprehension," in Artifact in Behavioral
Research, ed. R. Rosenthal and R. L. Rosnow, New York:
Academic Press, 280-350.
Rosenthal, Robert and Ralph L. Rosnow (1984), Essentials
ofBehavioral Research: Methods and Data Analysis, New
York: McGraw-Hill.
and Donald B. Rubin (1980), "Comparing Within-
and Between-Subjects Studies," Sociological Methods
and Research, 9 (August), 127-136.
Rosnow, Ralph L. and Leona S. Aiken (1973), "Mediation
of Artifacts in Behavioral Research," Journal of Exper-
imental Social Psychology, 9 (May), 181-201.
This content downloaded from 88.255.99.25 on Sun, 15 Mar 2020 12:24:48 UTC
All use subject to https://about.jstor.org/terms
DEMAND ARTIFACTS 283
and Daniel J. Davis (1977), "Demand Characteristics
and the Psychological Experiment," Et cetera, 34 (Sep-
tember), 301-313.
Sawyer, Alan G. (1975), "Demand Artifacts in Laboratory
Experiments in Consumer Research," Journal of Con-
sumer Research, 1 (March), 20-30.
Silverman, I. (1977), The Human Subject in the Psychological
Laboratory, New York: Pergamon.
Stuart, Elnora W., Terence A. Shimp, and Randall W. Engle
(1987), "Classical Conditioning of Consumer Attitudes:
Four Experiments in an Advertising Context," Journal
of Consumer Research, 14 (December), 334-349.
Swartz, Barry (1989), Psychology of Learning and Behavior,
New York: Norton.
Turner, Charles W. and Lynn Stanley Simons (1974), "'Effects of
Subject Sophistication and Evaluation Apprehension on Ag-
gressive Responses to Weapons," Journal of Personality and
Social Psychology, 30 (September), 341-348.
Wallendorf, Melanie and Russell W. Belk (1989), "Assessing
Trustworthiness in Naturalistic Consumer Research," in In-
terpretive Consumer Research, ed. Elizabeth C. Hirschman,
Provo, UT: Association for Consumer Research, 69-84.
Weber, Stephen J. and Thomas D. Cook (1972), "Subject Effects
in Laboratory Research: An Examination of Subject Roles,
Demand Characteristics, and Valid Inference," Psycholog-
ical Bulletin, 77 (April), 273-295.