Empirical Accounting Research Design for Ph. D.

Students
Author(s): William R. Kinney, Jr.
Source: The Accounting Review, Vol. 61, No. 2 (Apr., 1986), pp. 338-350
Published by: American Accounting Association
Stable URL: http://www.jstor.org/stable/247264 .
Accessed: 09/05/2014 16:35

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

American Accounting Association is collaborating with JSTOR to digitize, preserve and extend access to The
Accounting Review.

http://www.jstor.org

This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM

All use subject to JSTOR Terms and Conditions
THE ACCOUNTING REVIEW
Vol. LXI, No. 2
April 1986

EDUCATION RESEARCH
Frank H. Selto, Editor

Empirical Accounting Research Design

For Ph.D. Students
WilliamR. Kinney, Jr.

ABSTRACT: This paper discusses an approach to introducing empirical accounting research

design to Ph.D. students. The approach includes a framework for evaluating accounting ex-
periments as well as studies based on passive observation of subjects or data. Alternative
methods of isolating the effect of the "independent" variable of interest from effects of prior-
to-the-study-period variables and contemporaneous variables are discussed along with the
advantages and limitations of each method. Also discussed is the relationship between type I
and type 11error risks, sample size, and research design. The importance of research design,
including theory development and means for mitigating the effects of extraneous variables, is
emphasized as perhaps the only practical way to achieve research objectives in empirical re-
search in accounting.

A FREQUENTLY encountered problem The approach is generic-it is not tied to

in accounting Ph.D. programs is
that first-year students do not have
background in empirical research in ac-
counting. Few B.B.A., M.B.A. or M.Acc.
programs include courses in empirical re-
search and many students have not seri-
ously considered its nature. Yet, such an
introduction is necessary if Ph.D students
are to efficiently relate other courses to
substantive problems in accounting and
be able to take full advantage of account-
ing workshops.
The purpose of this paper is to show
how a basic framework for evaluating
empirical research in accounting can be
obtained in a short introduction. This can
be done at the start of the first term course
and provides a context for further work in
philosophy of science and statisticaldesign
as well as substantive areas of accounting.
338
an area of accounting and doesn't depend
on prior knowledge of a particular para-
digm.'
' Illustrations and extensions from applied areas of
accounting are also helpful. Good sources for financial
accounting are Ball and Foster [1982], Lev and Ohlson
[1982], and Abdel-khalikand Ajinkya [1979].Good sources
for behavioral work are Ashton [1982] and Libby [1981].
I would like to acknowledge the helpful comments and
suggestions of Vic Bernard, Dan Collins, Grant Clowery,
Bob Libby, JerrySalamon and two anonymous reviewers.
An earlier version of this paper was presented at the
American Accounting Association's Doctoral Consor-
tium in Toronto, Ontario, in August 1984.
William R. Kinney, Jr. is Price Water-
house Auditing Professor at the Univer-
sity of Michigan.
Manuscript received September 1984.
Revisions received April 1985 and August 1985.
Accepted September 1985.

This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM

All use subject to JSTOR Terms and Conditions
Kinney
The generic approach focuses attention
on the essence of scientific inquiry in ac-
counting. Many of the problems faced by
accounting experimenters who can man-
ipulate some (but not all) of the levels of
variables to be studied are similar to those
faced by "passive observers" of the levels
of all variables as set by Nature.2 Thus,
the generic approach may help to avoid
premature specialization [Boulding,
1956, p. 199].
Section I presents a definition of empir-
ical accounting research and theory, hy-
pothesis, and fact. It also defines "depen-
dent," "independent," and prior and
contemporaneous influence variables.
Section II discusses alternative means for
separating the effects of prior and con-
temporaneous influence variables from
the independent variable(s), and in Section
III the interrelationships among signifi-
cance, power, and research design are
explored. Section IV gives a summary and
conclusions.
I. A FRAMEWORK
FOREMPIRICAL
RESEARCH IN ACCOUNTING
Research is a purposive activity and its
purpose is to allow us to understand, pre-
dict, or control some aspect of the envi-
ronment. Research will be defined here
as the development and testing of new
theories of 'how the world works' or refu-
tation of widely held existing theories.
For accounting research, the theories con-
cern how the world works with respect to
accounting practices. Watts and Zimmer-
man [1984, p.1], state: "The objective of
accounting theory is to explain and predict
accounting practice." This positive, how-
the-world-is approach is in contrast with
the more traditional normative view that
accounting "theory" is concerned with
what accounting practices ought to be.
Empirical accounting research (broadly
considered) addressesthe question: "Does
how we as a firm or as a society account
This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM
All use subject to JSTOR Terms and Conditions
339
for things make a difference?"' Clearly,
the accounting for items affecting tax pay-
ments makes a difference in our individ-
ual and collective lives. But does the ac-
counting for, say, depreciation in internal
or external reports affect decisions within
a business firm or affect stock prices? If
it does, then the size of the effect and why
it occurs are important follow-up ques-
tions. Additionally, the accounting re-
searcher must separate the underlying
economic event (or state) from the account-
ing report of the event. Thus, while a fi-
nance researcher may be concerned only
with firm characteristics, the accounting
researcher must also be concerned with
the costs and benefits of alternative ac-
counting reports of those characteristics.4
In essence, empirical research involves
theory, hypothesis, andfact. "Facts" are
states or events that are observable in the
real world. A "theory" offers a tentative
explanation of the relationship between
or among groups of facts in general. "Hy-
potheses" are predictions (or assertions)
about the "facts" that will occur in a par-
ticular instance assuming that the theory
is valid. Finally, observing "facts" con-
sistent with the prediction or assertion
2 The problems are not identical. For example, while
experimenters have the advantage of being able to specify
the values of some variables, they also face the risk of
choosing values that are too close together (or too far
apart) to allow precise estimates of treatment effects or to
allow generalization of conclusions to the real world.
3 Within this definition, relevant questions for
auditing include, "Does how precisely we audit and
report the state of a firm make a difference?" and,
"How can audits of a given precision be conducted ef-
ficiently?" The first auditing question is related to the
accounting question through the concept of materiality.
Parallel questions involving the design of accounting
systems also could be developed.
4Accounting professors may conduct research in
finance, economics, behavioral science, or statistics. If
the accounting question is not addressed, however, the
accounting professor may face the disadvantage of being
undertrained relative to other researchers. Also, he or she
ignores a comparative advantage in the knowledge of
accounting institutions and the sometimes subtle role of
information.
340
made in the hypothesis lends credibility to
the theory.
Ordinarily, researchbegins with a real-
world problem or question. One thinks
about or studies the problem, reads about
seemingly similar problems in other areas
or disciplines such as economics, psy-
chology, organizational behavior, or po-
litical science. By immersing himself or
herself in the problem, the researcher
may, either by genius or by adapting a
solution from another area, develop a
general theory to explain relationships
among facts [Simon, 1976, Chapter 7 and
Boulding, 1956]. From this statement of
the general relationship among facts, hy-
potheses about what should be observed
in a particular situation can be derived.
An experiment or passive observation
study then can be designed to support or
deny the hypotheses.
For example, suppose it is observed
that a stock price increase usually follows
the announcementthat a firm has changed
from straight-line to accelerated depre-
ciation. Why should a mere bookkeep-
ing change seem to lead to an increase
in firms' values? An explanation might
be that market participants believe that
eventsleadingto such an accountingchange
also typically lead to better prospects for
the firm in the future. With development
and elaboration of such a theory, the
researchermight develop a passive obser-
vation study of past changes or an exper-
iment to test hypotheses based on the pro-
posed explanation.
Theories are usually stated in terms
of theoretical variables or "principals"
while empirical measurement requires ob-
servable variables. The difference between
the principal and real-world observable
variables presents difficulties for account-
ing researchers since accounting mea-
surements may be either surrogates for
some underlying principal of interest or
may be the principal itself. For example,
This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM
All use subject to JSTOR Terms and Conditions
The Accounting Review, April 1986
if firm "performance" is the theoretical
principal and earnings is chosen as the
surrogate measure of firm performance,
then straight-line depreciation is a com-
ponent of the surrogate.
As a surrogate, straight-line deprecia-
tion contains two sources of potential
error that may require consideration by
the researcher. One is the surrogation
error due to the fact that straight-line de-
preciation does not "correctly" reflect
the relevant performance of the firm for
the purpose at hand. The other is applica-
tion error due to mistakes or imprecision
in applying straight-line depreciation.
On the other hand, in evaluating possi-
ble determinants of managerial behavior,
audited earnings using straight-linedepre-
ciation may be specified in a contract and
may serve as a principal. For example, if a
manager is to receive a bonus or profit
share of one percent of audited earnings,
then audited earnings is the principal.
Surrogation error, and any application
error not detected and corrected by the
auditor, is ignored for contract purposes.
The same number used as a measure of
firm performance will likely contain both
surrogation and application error.5
To add credibility to a theory, one must
not only be able to show hypothesis test
resultsthat are consistent with the theory's
predictions, but also have a basis to rule
out alternativeexplanationsof the observed
facts. This requiresconsideration of a rea-
sonably comprehensive list of alternative
explanations. Again, knowledge of related
disciplines is useful in generating alterna-
tive explanations for accounting-related
"facts." Some possible explanations can,
of course, easily be ruled out as being of
5 Accounting systems designers and financial
accounting standards setters can control the first type of
measurement error, while auditors and auditing stan-
dards setters can control the second. Problems relating to
the interaction of accounting and auditing standards
setters, users, and auditors is, of course, a matter for
accounting research.
Kinney
likely negligible effect, but others will re-
quire attention.
To be more specific, let Y denote the
"dependent" variable to be understood,
explained, or predicted. Variables caus-
ing Y(or at least related to Y) can be clas-
sified into three broad groups as follows:
X=the "independent" variable that
the proposed theory states should
effect Y,
Vs = prior-influence (prior-to-study
period) factors that may affect Y,
and
Zs= contemporaneous factors (other
than X) that may affect Y.6
That is, Y=f(X, V,, V2, . ..,ZIZ2, ...
A common Yfor addressingan account-
ing question is the change in a firm's stock
price. Others are a manager's act or deci-
sion. A common X is a change or differ-
ence in accounting method, whether by
management's choice or by a regulatory
directive. Prevalent Vs are the firm's prior
period state variablessuch as profitability,
leverage, liquidity, and size. For tests of
theories about decision-making behavior
of particular human subjects, relevant
Vs often include the subject's personality
traits, mathematical ability, education,
training, age, firm association, and exper-
ience.
The most common Z factor in account-
ing research studies involving stock prices
is the market return (Rm). Another com-
mon group of Zs for external reporting
and managerial performance studies is
the unexpected portion of contemporan-
eous accounting measures for other firms
or other divisions. Finally, since the ac-
counting researcher is concerned with the
effect of accounting reports, Zs may be
underlying characteristics of the "true
state" of the firm at the time of the study
as measured by contemporaneous nonac-
counting reports about the firm.
This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM
All use subject to JSTOR Terms and Conditions
341
DISENTANGLINGTHE EFFECTS FROM VS
AND ZS FROMTHE EFFECT OF X
For simplicity, assume that X is mea-
suredat only two levels. Eitherthe observed
Yis from the "control" group that receives
no "treatment" or from the treatment
group that receives the treatment. Alter-
natively, the two groups could simply be
different on some relevant dimension
(e.g., to test theories about accelerated
depreciation, the control group might be
defined as those firms that use straight-
line and the treatment group as those that
use accelerated).
Also for simplicity, assume that there is
a singleprior-influencefactor Vthat effects
Yand Vhas the same effect on Ywhether
the subject is from the control or treat-
ment group. Furthermore, there are no
contemporaneous Zs that affect Y and
the model determining Y is:
Yij=Bo+BX,+B2Vj+ ej,, (1)
where Yij is the value of the dependent
variable for the "j"th subject in the "i"th
treatment, Bo is the intercept for the con-
trol group, Xi is an indicator variable
(zero for the control group and one for
treatment), Bo+ B1 is the intercept for the
treatment group (that is, B1is the effect of
treatment), B2is the regression coefficient
relating Vto Y, and eij is a random error
term. The eij term will include the effects
of other Vs and Zs that are here assumed
to be negligible and randomly distributed
between the two groups, and eijis assumed
to have expectation zero and be uncorre-
lated with either X or V.
For the simple model of equation 1, a
plot of the expected values of Y given V
for both groups will be parallel lines with
possibly different intercepts. The differ-
ence in intercepts is the effect of the treat-
ment (B1). Figure 1 shows the components
6 Some Zs may be expectations,at the time of the
study,of still futurevaluesof X, Y, V, and otherZs.
342 The Accounting Review, April 1986

FIGURE 1
Y VALUES GROUPSUNDERANOVA, ANCOVA, ANDMATCHING
FORNONEQUIVALENT

I ~~~~~~~~~~~~Y=00+
X+02V

X
ANOVA ;
/- | andMatching
~~~NCOVA V

~~~~~~~~~~I I

lI _~~~~~~ I I
I I
V. V Vs

Matches

of equation 1 along with ellipsesthat ap-
proximatethe locus of membersof the
two groups.
An experimentermay ignore V and
may randomlyassignsubjectsto groups.
On average,the groupswill be equivalent
on V. For small samples,however,there
is nontrivialrisk that such a procedure
may assign to the treatmentgroup, say,
those with high values of V and to the
controlgroupthose withlow V. In analy-
zingresults,the effect of the high Vvalues
(i.e., B2VJ) will be mixed with the treat-
ment effect. An experimentermay rule
out the possible effect of V by random
assignmentof subjects measuredon V

This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM

All use subject to JSTOR Terms and Conditions
Kinney
between groups. The sample subjects are
measured on V, matched into pairs ac-
cording to their V values, and one from
each pair is randomly assigned to treat-
ment. Thus, even for small samples the
two groups will be approximately equiva-
lent on V.
The passive observerhas no opportunity
to randomlyassignsamplesubjectsto treat-
ment. Even experimenters may have dif-
ficulty developing a satisfactory random-
ized design due to having too many
potentially important Vsand Zs that must
be simultaneously matched. Thus, in gen-
eral, researchersface the problem of treat-
ment groups that are not equivalent with
respect to V.
For nonequivalent groups, there are
three basic ways in which the researcher
can mitigate the possible effects of the V
factor in the model of equation 1. These
are:
1. ignoring V(i.e., assuming or hoping
that Vis randomized with respect to
X),
2. matching on Vex post (i.e., match-
ing after X has been chosen by the
subject or assigned by Nature), and
3. using covariance analysis to statis-
tically estimate and remove the ef-
fect of V.
The first approach ignores V, and results
can be analyzed with a single-factor anal-
ysis of variance (ANOVA). The second
approach physically equates the treatment
groups with respect to V, and results can
be analyzed using a randomized block de-
sign. The third approach "statistically"
equates the groups, and results can be
analyzed using analysis of covariance
(ANCOVA).
Each of these approaches is discussed in
turn, along with some of the advantages
and limitations of each for experimenters
and passive observers.
This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM
All use subject to JSTOR Terms and Conditions
343
Ignoring V
As discussed above, ignoring a poten-
tial V is generally inadvisable due to the
unknown effect of V. A negligible effect
is the hopedfor result for any unmatched,
unmeasured, or unknown Vsor Zs. How-
ever, most real-worldevents have multiple
causes and a negligible overall effect is
unlikely. Furthermore, larger samples
will not help in researchdesignsthat ignore
systematic effects of V.
While expost matching and covariance
analysis can't account for all possible Vs
and Zs, they can reduce the risk that some
potentially important Vs and Zs disguise
the true effect of X. Figure 1 shows the
relevant sampling distributions for the
three approaches applied to the example.
As shown in the relativelyflat distributions
on the left margin, ANOVA is based on
the marginal distribution of Y with no
consideration of V.
Ex Post Matching
In many situations, the researcher se-
lects a sample after the phenomenon of
interest has taken place. Often, the re-
searcher selects a sample of subjects from
the treatment group and then selects a
subject from the control group with V
equal or similar to V for each treatment
subject. This ex post matching on V is
probably the most commonly used design
for passive observational studies in ac-
counting.
For ex post matching, the model as-
sumed to determine Y is:
m
Y, = Bo +BIX,+ E BjMj +eij, (2)
j2
where Bo is the overall mean of Yplus the
effect of (arbitrarilydesignated) match 1,
Bj (for j> 1) is the differential effect of
match compared to match 1, Mj is an in-
dicator variable (equal to one if the sam-
ple subject is a member of match j and
344
zero otherwise), and m is the number of
matches. '
For passive observational studies, it is
impossible to randomly assign subjects to
treatments since by definition the subjects
have already either "self-selected" into
treatmentgroups, or have been so selected
by Nature. It is possible that there will be
few or even no matches. For example, all
the firms using straight-line depreciation
may be small firms and all firms using ac-
celerated depreciation may be large. A
firm's choice of accounting method (or
decision to change methods) may merely
reflect its V value. Figure 1 illustrates
such a possibility in that only the bottom
half of group 1 can be matched with a
member of group 0 due to the difference
in V for the groups.
Even experimenters using ex ante
matching with random assignment of
subjects often face the lack-of-matches
problem for at least some Vsand Zs. Sup-
pose, for example, that a researcherbelieves
that an auditor's response in a profession-
al task experiment may be related to the
auditor's professional training (X) after
accounting for his or her mathematical
abilities ( V). It may be difficult to match
subjects from different firms based on
mathematical abilities. This is because
firms may hire and thus train (or students
may choose firms and be trained) on the
basis of mathematical ability.8
As will be discussedbelow, the efficiency
of matching may be less than that for co-
varianceanalysis. However, expost match-
ing is likely to be superior to covariance
analysis when the functional form of the
YlIV relationshipis nonlinearor unknown.
Given that the treatment effect is not cor-
related with V, matching can be used for
any functional form of Y and V (known
or not) and analyzed using a blocked de-
sign.
Covariance Analysis
A researcher using analysis of covari-
This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM
All use subject to JSTOR Terms and Conditions
The Accounting Review, April 1986
ance (ANCOVA) statistically estimates
the effect of V on Y and removes it.
ANCOVA can be viewed as the result
of projecting observed Y values along
the regression line to a common point
on the V axis, such as V, to yield the
conditional distributions as shown in Fig-
ure 1.1
Figure 1 shows that part of the differ-
ence between the marginal distributions
of Yoand Y1(as estimated using ANOVA)
is due to a larger Vfor the treatmentgroup.
Matching (equation 2) accounts for the
difference by subtracting B O+1 BjMj
from Y for each subject, and ANCOVA
accounts for the difference by subtracting
B0+ B2 Vijfrom Y for each subject. Thus,
both matching and ANCOVA are seen to
mitigate the differential effect of V. Fig-
ure 1 also shows that control group sub-
jects with relatively high Vfor the control
group are matched to subjects with rela-
tively low Vfor the treatment group. For
matched designs, all other potential sample
subjects must be omitted due to lack of
matches.
Matching and ANCOVA yield more
efficient (more precise) estimates of the
treatment effect than does ANOVA. In
general, however, it is unclear which of
the two will be more precise. This is due to
the fact that while the difference in Voand
V, reduces the precision of ANCOVA,
the reduction in sample size due to lack of
matches reduces precision for matching.
' The matches may be by individual subject ("pre-
cision" or "caliper" matching) as discussed above, or by
frequency distribution (e.g., equal mean and variance
with respect to V for both groups). A test of equality on V
is often used as a justification for ignoring V in the
statistical analysis.
I An alternative design is to limit all subjects in an
experiment to a fixed level of V. This equalizes the effect
of V but greatly reduces the generalizability of results
over the range of reasonable V values that might occur.
I The sampling distributions for matching and
ANCOVA are shown as the same in Figure 1 since the
expectations of estimates of the treatment effects are the
same for both. As discussed, however, their standard
errors will differ.
Kinney
It is often difficult to predict which will be
the greater problem. 10
Equation 1 and Figure I present a very
simple situation even for a single V. For
example, the YI Vrelationship may differ
depending on whether X is at level zero or
one. Furthermore, the occurrence of a
given level of Vat time t-1 may have a di-
rect effect on Y at time t but may also
affect the level of X at time t which, in
turn, affects Y at t. Thus, there may be
two paths by which V affects Y. A com-
plete approach would include a model of
the "selection" process by which V affects
X as well as the direct effects of X and V
on Y."1
III. ALPHA, BETA, SAMPLE SIZE
AND RESEARCH DESIGN
Planning researchto disentangle X, Vs,
and Zs involves four related factors.
These are alpha (ce),beta (@),sample size,
and what will be called the "research de-
sign" factor (denoted D). In a given situa-
tion, setting any three of them sets the
fourth. The statistical factors of a (the
probability of a type I error or incorrectly
rejectinga true null hypothesis of no treat-
ment effect), f3(the probability of a type
II error or not rejecting the null hypothe-
sis when, in fact, there is a treatment
effect), and sample size are well known.
The research design factor is the ratio of
two subfactors. Its numerator is the hy-
pothesized magnitude of the (X) treat-
ment effect (denoted 6), and its de-
nominator is the standard deviation of
the residuals in the equation used to esti-
mate B1 (denoted a). Thus, D = 6/a. The
numerator depends on the researcher's
theory, and the denominator depends on
how the researcher disentangles the Vs
and Zs and the inherent variability in the
phenomenon under examination.
The required sample size is a decreasing
function of a, 3,6 and an increasing func-
tion of a. Therefore, for a given a and 3,
the required sample size will be small if
This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM
All use subject to JSTOR Terms and Conditions
345
the proposed theory implies a large effect
on Yand/or the researcheris clever in de-
signing a plan to disentangle the effects of
the Vsand Zs. For example, usingANOVA
(no matching) in a single test with target
ac=.05, 03=.1, and D=6/a=.5, the re-
quired sample size is 70 for each of the two
groups. If the researcher has a theory
yielding a larger 6 that would increase D
to .75, then the sample would be 32 each,
and if D is 1.0 then the sample size is 18
per group. Alternatively, for D=.5 and
holding 6 constant, if matching is used
and the Y., Y, correlation is .25 then
the required sample is approximately 57
pairs."3 If the Yo, Y1 correlation is .5
(implying a reduction in aof about 18 per-
cent), then the required sample is 36 pairs.
The four factors and their implications
for accounting research will be discussed
throughtwo subtopics.These are: 1) power,
and 2) prejudice against the null hypoth-
esis.
Power
Consider a researcherwho has a theory
that the treatment effect (B1) is positive,
and who therefore is interested in testing
the (null) hypothesis that the true effect
of treatment is less than or equal to zero
against the alternative that the true effect
'0 ANCOVA will usually be less biased, however (see
Cook and Campbell [1979, pp. 177-182], and Cochran
[1983, pp. 127-128]).
" See Cochran's comments on R. A. Fisher's advice to
"Make your theories elaborate. " According to Cochran,
Fisher meant that when "constructing a causal hypoth-
esis one should envisage as many different consequences
of its truth as possible, and plan observational studies to
discover whether each of these consequences is found to
hold" [Rosenbaum, 1984, p. 43]. This advice is consis-
tent with Boulding's exhortation to develop and test
theories that are at the level of the real world.
12 The required sample size is:
n =2[(t., ) 2
2X-2 + 6, 2,-2 (1 /D)]
See Ostle [1963, p. 553] for a table.
'" The required sample size is:
n= + to55]) (f aD)]b'
(1963,p
See Ostle [1963, p. 55 1] for a table.
346
is greaterthan zero. Assume that a sample
of treatment subjects has been matched
or "paired" on V with control subjects.
Also, based on an assessment of the ap-
propriatesignificance level for the issue at
hand, the researcherhas set a at .05, and
the researcher has a research design in
mind.
What the researcher may not consider
at the planning stage is the magnitude of
the hypothesized effect (i.e., a particular
6 for the alternative hypothesis) and the
allowable 3 for that 6. 14 A 6 may not be
considered since most theories suggest
only the direction of an effect and not its
magnitude, and f3 is not considered be-
cause no particular 6 is specified. The re-
searcher may proceed to testing with little
consideration of whether the planned test
has an adequate chance to reject the null
hypothesis even if it is false.
To illustrate, consider the sampling dis-
tributions in Figure 2. For both panels of
Figure 2, the left-hand distribution is for
the mean of the paired differences if the
null hypothesis (B1=0) is true, and the
right-hand distribution applies if the par-
ticular alternative (B1= 6) is true. Also for
both panels, k is the point that yields
a =.05 or five percent of the area to the
right of the point under the left-hand dis-
tribution (i.e., Ho). In panel a, the re-
search design and sample size yield a
sampling distribution with a large area
(1 - 3)to the right of k under the alterna-
tive hypothesis. Thus, there is high proba-
bility of rejecting Ho when the alternative
hypothesis is true. In other words, the
power of the test (1 - f) is high.
In Figure 2, panel b, 6 is the same as in
panel a, but the sampling distributions
are much flatter due either to small sam-
ple sizes or a large standard deviation due
to remaining effects of Vsand Zs. Rather
than the relatively high power test of
panel a, the researcher faces a low power
test. At a(=.05, 13for the simple alterna-
This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM
All use subject to JSTOR Terms and Conditions
The Accounting Review, April 1986
tive hypothesis is greater than .5, and
power is less than .5. Even if Ho is false
(i.e., B = 6). and thus the researcher's
theory of a positive treatment effect is
correct, the researcherhas a less than even
chance of rejecting it!
Suppose that the researcher in panel b
observes a test statistic that is almost sig-
nificant, and the sample estimate of the
treatment effect is equal to 5. He or she
then decides to take a follow-up sample.
The follow-up sample is also likely to in-
dicate nonrejection due to its small size
[Tversky and Kahneman, 1971, p. 107].
The real culprit, of course, is the low
power of the test. If the low power is anti-
cipated at the planning stage, an attempt
can be made to mitigate its negative effects
or else abort the project. In general, power
can be increased by 1) increasing the sam-
ple size, or 2) increasing the design factor
D by developing better theories (yielding
larger 6) or by making better use of agiven
sample size and theory by careful atten-
tion to the Vs and Zs (yielding smaller a).
As a practical matter, improved design
is often the only alternative in accounting
research since the size of samples in ac-
counting frequently is effectively fixed.
For experiments, the pool of available au-
ditors, accountants, financial statement
users, and even students is effectively lim-
ited to fairly small numbers. Subjects' time
is not free, and the supply is not inexhaust-
ible. For passive observation, the number
of firms for which particular accounting
and other required economic data are
available may be relatively small. Thus,
accounting researchers need to be aware
of a variety of analytical techniques appli-
4 See Tversky and Kahneman [19711. This is in
contrast to classical or normal distribution theory-based
audit sampling where, in addition to setting a to control
the risk of incorrect rejection, the auditor sets i to con-
trol the risk of incorrect acceptance and sets 6 based on
"intolerable" error (materiality). The auditor then
selects an estimator and calculates the minimum sample
size subject to the target a, j, and 6.
Kinney 347

FIGuRE2
SAMPLINGDIsRIuBUTIONSOF d * FORHIGH AND Low POWER TESTS

a. HighPower I I

OI _ _ _ _ _

Rt
b. LowPower

l~~~~~~~~~~~ l.0
l l~~~~~~~~
RejectHo~ Reec
I
l~~~~~~~~~~~~~Rc Ho
lY~o)M l
jlal

cable to a variety of research problems. 15 degreeof owner versusmanagercontrol

Furthermore, for a given research par-
adigm, the problem of low power is likely
to become more difficult over time. Other
things equal, as knowledge of the effects
of accounting expands, the likely size of
the effect of each new or more refined
theory (B1)will tend to have less addition-
al explanatory power. As knowledge ex-
pands, the best potential Xs are investi-
gated and become Vsor Zs. For example,
early studies tested hypotheses about the
as an X that affectedaccountingchoices.
Laterstudieshave usedthe samevariable
as a Vor Z. Absentdevelopmentsthat re-
structurethe way a particularproblem
is addressed, future researcherswill be
faced with discoveringnew Xs that have
less potentialexplanatorypower.
's In debate on the preferability of parametric vs. non-
parametric statistics in research, the ability of parametric
methods to accommodate more Vs and Zs through co-
variance analysis is an often overlooked advantage.

This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM

All use subject to JSTOR Terms and Conditions
348
In fact, it may be unreasonableto expect
that a particulartheory based on account-
ing methods will yield a true differential
effect that is very large relative to the vari-
ance of Y. How things are accounted for
simply can't be expected to explain a large
portion of stock price variability or man-
agerial or investor behavior. Under some
conditions, the sample size required to
yield reasonable power exceeds the size of
the known population!
A researcher can get some protection
by making a tentative calculation of power
before investing in expensive data collec-
tion or in experimentation. For example,
passive observers of accounting changes
and stock returns may be able to make
reasonable estimates of the standard devi-
ation of return residuals and might make
power estimates for various levels of 6. 16
If the estimated power is inadequate even
for the maximum 6 that might reasonably
exist, then the research can be redesigned
or aborted. Experimenters are perhaps
more familiar with prospective power cal-
culations and frequently use a pilot sam-
ple to assist with the sample size and re-
search design development.
Prejudice Against the Null Hypothesis
A theory usually specifies the direction
of the treatment effect and a researcher
generally sets out to reject hypotheses
based on the assumptionthat the treatment
effect is zero or in the opposite direction
from what the theory predicts. The focus
on rejecting the null hypothesis is the
source of a number of "biases" against
the null that may lead to dysfunctional
consequences. Greenwald [1975] lists
eight such consequences; four that seem
most important for accounting research-
ers are discussed in order to be better able
to avoid them.
1. A paper will not be submitted for
publication consideration unless the
This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM
All use subject to JSTOR Terms and Conditions
The Accounting Review, April 1986
results against the null are "signifi-
cant." Especially interesting or in-
novative results may be submitted
on higher than .05 significance (or
alternatively, the probability at
which the results are significant are
reported), but rarely does an editor
see results with significance levels
above .15. This prejudice need not
exist if not rejecting the null gives
reasonable credibility to the null."7
2. Ancillary hypotheses will be ele-
vated in the exposition of results.
Secondary hypotheses that are sig-
nificant ex post will receive more
attention than other secondary hy-
potheses and perhaps the primary
hypotheses. Suggestions will be
made that these results warrant
further study, when in fact one
would expect about one in ten non-
sense relationships to be significant
at the .10 level.
3. Alternative operationalization of
variables or their functional form
will be conducted only if "prelimi-
nary" results are insignificant. The
extent to which this search activity is
justified is open to debate since
most theories don't imply a single
measurement or functional form.
4. The search for errors will be asym-
metric. Outliers that impede re-
jection of the null hypothesis will
tend to receive more diligent atten-
tion than those that favor rejecting
the null. If significant results are
16 The choice of 6 is somewhat arbitrary, but in plan-
ning it is useful to consider reasonable or plausible values
for the true treatment effect of X. Alternatively, one
might choose the smallest effect that informed persons
would agree is empirically "important" and therefore
worth knowing about if it exists, or the largest amount
that one could reasonably expect.
" In classical statistics, not rejecting the null is not
equivalent to accepting the null. However, non-rejection
by a reasonably powerful test or series of tests does
increase one's subjective degree of belief in the null.
Kinney
obtained on the first analysis of a
problem, the neophyte researcher
may not consider a search for out-
liers or for other violations of sta-
tistical assumptions underlying the
analysis. Nonrejection may lead one
to consider such explanations and
to search for programming errors
and data coding errors.
V. SUMMARY AND CONCLUSIONS
In this paper we have stressed consider-
ation of planning for Vsand Zs to be able
to isolate the effect of X as a potential ex-
planation of differences in Y. This consid-
eration may allow increased power in tests
and may allow more to be learned from a
given sample. Planning for Vs and Zs can
reduce the risk of not rejecting the null
hypothesis when it is false. Such planning
may also allow a basis to argue that non-
rejection of the null hypothesis may sup-
port acceptance of the null. That is, if the
treatment has an important effect, then it
should be revealedby the test. Thus, some-
thing may be learned whether results are
statistically significant or not. This should
increase the objectivity of the researcher,
since the work is valuable whatever the
empirical results.
There are at least two ways in which the
approach discussed in this paper can be
useful to Ph.D. students. One is in evalu-
ating the researchdesign of others, and the
other is in planning the student's disserta-
tion. 18 Students must evaluate the work
of others whether in published articles,
working papers, or accounting research
workshops. A student applying the ap-
proach to the work of others might try to
answer the following questions: What is
the Y and what is the X? What Vs and Zs
This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM
All use subject to JSTOR Terms and Conditions
349
are considered? Are there better ways to
account for the effects of Vs and Zs?
What are other Vsand Zs that might have
important effects?
The same approach can be applied by
the student to his or her own dissertation
proposal. While the basic development of
a research proposal is the responsibility of
the student, there is much to warrantearly
faculty discussion of planned dissertation
research. That is, the faculty can evaluate
a proposal by considering the reasonable-
ness of the theory and the adequacy of
control of potential Vs and Zs. The
faculty should be asked: Is the magnitude
of the hypothesized effect plausible? Are
all importantcompetingexplanationslisted
and adequately dealt with in the plan?
Will the proposed tests likely uncover evi-
dence of a difference equal to 6 if it exists?
Will nonrejection lend credibility to the
null?
Faculty approval of planned disserta-
tion research reduces the student's risk by
1) ruling out potential topics that have
little chance of successful completion, 2)
gathering the right data on the first at-
tempt, 3) eliminatingoutcome dependence
(thus reducing moral hazard for the stu-
dent), and 4) reducing the temptation of
the student (and faculty) to pursue num-
erous tangents that may come to light as
the research progresses.
1 In planning research or evaluating the research of
others, a useful practice is to give early attention to the
purpose of the research through preparation of a three-
short-paragraph abstract, synopsis, or working model of
the research. The first paragraph answers the question
"What is the problem?" The second asks, "Why is it an
important problem?" and the third, "How will it be
solved?" Alternatively, the questions might be: "What
are you (or the researcher) trying to find out?",
"Why?", and "How will it be done?"
350 The Accounting Review, April 1986

REFERENCES
Abdel-khalik, A. R. and B. B. Ajinkya, Empirical Research in Accounting: A Methodological Viewpoint,
Accounting Education Series No. 4 (American Accounting Association, 1979).
Ashton, R. H., Human Information Processing in Accounting, Accounting Research Study, No. 17
(American Accounting Association, 1982).
Ball, R. and G. Foster, "Corporate Financial Reporting: A Methodological Review of Financial
Research," Studies in Current Research Methodologies in Accounting: A Critical Evaluation," Journal
of Accounting Research (Supplement 1982), pp. 161-234.
Boulding, K. E., "General Systems Theory-The Skeleton of Science," Management Science (April 1956),
pp. 197-208.
Cochran, W. G., Planning and Analysis of Observational Studies, edited by L. E. Moses and F. Mosteller
(John Wiley & Sons, Inc. 1983).
Cook, T. D. and D. T. Campbell, Quasi-Experimentation Design & Analysis Issues for Field Settings
(Houghton-Mifflin Company, 1979) especially chapters 3 and 4.
Greenwald, A. G., "Consequences of Prejudice Against the Null Hypothesis," Psychological Bulletin
(January 1975), pp. 1-20.
Lev, B. and J. A. Ohlson, "Market-Based Empirical Research in Accounting: A Review, Interpretation,
and Extension," "Studies in Current Research Methodologies in Accounting: A Critical Evaluation,"
Journal of Accounting Research (Supplement 1982), pp. 249-322.
Libby, R., Accounting and Human Inlormation Processing: Theory and Applications (Prentice-Hall,
Inc., 1981).
Ostle, B., Statistics in Research, 2nd Edition (The Iowa State University Press, 1963).
Rosenbaum, P. R., "From Association to Causation in Observational Studies: The Role of Tests of
Strongly Ignorable Treatment Assignment," Journal of the American Statistical Association (March
1984), pp. 41-48.
Simon, J. L., Basic Research Methods in Social Science: The Art of Empirical Investigation, 2nd Edition
(Random House, Inc., 1978) especially chapters 3, 7, and 11.
Tversky, A. and D. Kahneman, "Belief in the Law of Small Numbers," Psychological Bulletin (August
1971), pp. 105-1 10.
Watts, R. L. and J. L. Zimmerman, Positive Accounting Theory (Prentice-Hall, 1986).

This content downloaded from 169.229.32.138 on Fri, 9 May 2014 16:35:44 PM

All use subject to JSTOR Terms and Conditions