00 positive Bewertungen00 negative Bewertungen

18 Ansichten7 Seitendetect fraud

Dec 05, 2017

© © All Rights Reserved

PDF, TXT oder online auf Scribd lesen

detect fraud

© All Rights Reserved

Als PDF, TXT **herunterladen** oder online auf Scribd lesen

18 Ansichten

00 positive Bewertungen00 negative Bewertungen

detect fraud

© All Rights Reserved

Als PDF, TXT **herunterladen** oder online auf Scribd lesen

Sie sind auf Seite 1von 7

May 2005

pp. 7781

Benfords Law to Detect Fraud:

The Dangers of Type I Errors

Richard Cleary and Jay C. Thibodeau

SUMMARY: At first glance the application of digital analysis using Benfords Law holds

great promise as a fraud detection process. However, a closer look at the underlying

statistical assumptions reveals that auditors seeking to use Benfords Law must be

aware of the costs of the potential Type I errors that can occur during the analysis stage.

For example, statistical considerations indicate that there is a far greater chance of

making a Type I error if the Benfords Law analysis is completed on a digit-by-digit

basis, as compared to the test-by-test basis typically employed by statisticians. In this

paper, we explain the merits of each choice in terms of statistical concepts and practical

audit process considerations.

Keywords: digital analysis; Benfords Law; fraud detection; Type I Errors; and Com-

puter Assisted Auditing Techniques (CAATs).

INTRODUCTION

I

n recent years, auditors have employed more sophisticated statistical techniques, such as digital

analysis using Benfords Law, as part of their fraud detection processes. Benfords Law proposes

a probability distribution for first, second, and other digits of numbers in data sets describing the

sizes of similar phenomena as long as the sizes span multiple orders of magnitude. There is much

empirical evidence suggesting that the frequencies of first and second digits of a data set that

contains credible numbers will indeed correspond to a Benfords Law probability distribution (e.g.,

Nigrini 1996; Nigrini and Mittermaier 1997). Given its potential to identify data points (e.g., transac-

tion amounts) that contain characteristics associated with fraudulent activity, digital analysis using

Benfords Law holds great promise as a fraud detection process (Coderre 1999).

The increased use of sophisticated statistical tools, like digital analysis using Benfords Law, has

been facilitated by the increased use of Computer Assisted Auditing Techniques (CAATs) as part

of the audit process. For example, an auditor seeking to apply the Benford command using ACL

(http://www.acl.com) need only identify the appropriate data field (e.g., invoice amount) within the

appropriate data file (e.g., client accounts receivable file) in order to successfully run the command.

The auditor would then have to consider whether additional audit procedures should be conducted

Richard Cleary and Jay C. Thibodeau are Associate Professors at Bentley College.

The authors gratefully acknowledge the comments from two anonymous reviewers and Bill Messier (the editor). We also

thank Jim Hunton and participants at the 2004 AAA Midyear Auditing Conference, the University of Florida Statistics

Seminar, and the University of Massachusetts Statistics Seminar for their helpful comments and suggestions.

Submitted: December 2003

Accepted: June 2004

77

78 Cleary and Thibodeau

on any field that did not conform to the Benfords Law probability distribution. It is precisely at this

stage of the process where the auditor must be aware of the costs of possible Type I errors, especially

considering the nature of a standard audit software package such as ACL.1

Standard audit software packages that feature digital analysis using Benfords Law may be

based on statistical assumptions that dramatically increase the likelihood of a Type I error. Given

their potential impact on the audit decision process, these assumptions should be more transparent.

For example, in ACL, the Benford command automatically produces an analysis completed on a

digit-by-digit basis rather than on a test-by-test basis. The need for greater transparency is

particularly salient when considering the statistical literature, which suggests that the chance of

making a Type I error is significantly greater when the Benfords analysis is completed on a digit-

by-digit basis. This is very similar to the multiple comparisons problem encountered in traditional

statistical methods such as Analysis of Variance (Ott 1993).

The purpose of this paper is to illuminate the statistical assumptions that are germane to an

auditors decision process when completing a digital analysis using Benfords Law. To do so, we first

provide a brief description of Benfords Law and a review of the literature seeking to apply Benfords

Law in various auditing contexts. We then provide a description of the Benfords Law analysis stage,

including a discussion of the possibilities for Type I errors. In this section, we also describe the

various statistical assumptions and their importance to the audit process. The final section provides

conclusions and implications to practicing auditors.

BACKGROUND LITERATURE

Benford (1938) became convinced that more numbers have small leading digits, like 1 or 2, than

large leading digits. He researched this assertion by studying many lists of data, such as the areas of

rivers, and the atomic weights of the elements. These empirical studies led Benford to propose that in

many real world applications the first digits d follow the probability distribution:

P(d = m) = log10 ((m + )/ m), for any d in the set {1, 2, 3,,9}.

This probability distribution gives P(d = 1) = log 2 = .301, P(d = 2) = log(3/2) = .176, on up to

P(d = 9) = log(10/9) = .046. In addition to Benfords empirical work, there is a sound theoretical

basis for this distribution (Hill 1995).

Benfords law does not apply universally. For instance, when data are all about the same

magnitude, Benfords distribution will not hold. The heights of adults, measured in any units, will not

follow Benfords law because very few adults are two times as tall as any other, and none are ten

times as tall as another. For example, if the units used were inches, the first digits would nearly all be

fives, sixes, and sevens (but the choice of unit is not important). The incomes of individuals,

however, and many other financial characteristics will span many orders of magnitude so that the

largest is perhaps thousands of times as big as the smallest. Importantly, Benfords Law is not likely

to apply to measurements where human intervention has occurred (e.g., rent expense based on a

contractual lease) or where random numbers have been applied (e.g., phone numbers). In general, for

data fields that cover several orders of magnitude, it is reasonable to expect that Benfords Law will

apply.

In auditing, Benfords law has been shown to be applicable in a number of auditing contexts,

including external (Nigrini and Mittermaier 1997; Nigrini 1999a; Tapp and Burg 2001), internal

(Nigrini 1999b), and governmental (Nigrini 1996; Wallace 2002) auditing contexts. To date, the

extant literature has primarily focused on properly identifying and defining the data sets that would

be appropriate candidates for a digital analysis using Benfords Law. There has been very little

1 Among internal auditors, ACL is the current market leader in fraud detection/prevention and continuous monitoring audit

software (McCollum and Salierno 2003). In addition, each of the Big 4 audit firms uses ACL in their fraud practices.

Applying Digital Analysis Using Benfords Law to Detect Fraud: The Dangers of Type I Errors 79

discussion however, about the importance of analyzing the statistical output from a properly

conducted digital analysis using Benfords Law. This apparent void in the literature is surprising

because the underlying statistical assumptions made in an application of Benfords law are important

in determining the correct inference. A closer look at the analysis stage is therefore warranted.2

At the analysis stage of a simple first digit analysis using Benfords Law, an auditor would

essentially be testing the following statistical null hypothesis:

H0: First digits in a data set are distributed according to Benfords Law.

The corresponding practical statement is that no fraud has taken place. Rejecting this null

hypothesis would mean that the digits are not behaving as one might expect. There are at least four

possible explanations:

1. The data really do follow Benfords Law, but due to random chance this particular set of

observations does not. This is the classical Type I error.

2. The assumption of many orders of magnitude is not met.

3. There is a reasonable explanation for an overabundance of some particular first digit. For

instance, if a firm has an arrangement to make a daily transfer to a vendor of $450, the first digit

4 might be overrepresented.

4. Some of the entries are fraudulent. This is the practical alternative hypothesis.

As such, when we reject the statistical null, the presence of fraudulent entries is just one of the

possible explanations. From an audit perspective, rejecting H0 means that additional work needs to

be completed in order to determine which explanation is appropriate, given the client context.

Of particular interest in the present case is the test statistic, or set of test statistics, that should be

used to carry out the test of the null hypothesis. Statisticians presented with this case would likely

select the classical Chi-squared (2) Goodness of Fit test (Keller and Warrack 2003). In this case, we

can compute the number of observations that we expect for each first digit within a data set of this

size if the null hypothesis is true, and compare these to the number actually observed, summarizing

the results in the calculated 2 statistic:

2 calc = ((observed expected)2/expected),

where the sum is over each of the admissible digits or digit-combinations. Large values of this

statistic lead us to reject the null since the observed values are then far from what we expected. We

compare this calculated value to tabulated values to find the p-value for the test (i.e., the probability

of a 2 calc value at least this large if the null hypothesis is in fact true). We reject the null if this p-

value is smaller than some designated probability of type I error; usually if p < .05.

Notice that this case illustrates an overall test of the fit of the data to the distribution proposed by

Benfords Law. It corresponds to checking for deviation from Benfords Law with a single test for

the entire data set. Rejecting the null does not say which digits are overrepresented or underrepresented.

An alternative testing approach would be to examine the digits; one at a time. For first digit testing,

this yields a set of nine different tests. For each d in the set {1,2,3,,9} we test:

H0,d: The first digit d appears as often as we would expect according to Benfords Law.

These tests obviously carry far more detailed information than the overall test. However, there is

a potential consequence in that repeating a testing procedure nine separate times greatly increases the

probability of at least one Type I error, when the data actually follow the Benfords Law distribution.

2 The possibility of Type I errors and the resulting audit consequences has been studied previously in a generalized manner

(e.g., Elliott and Rogers 1972; Beck and Solomon 1985).

80 Cleary and Thibodeau

In fact, if the test for each individual digit is carried out at significance level of 0.05 (so that the null

is rejected when the p-value is less than 0.05), then the probability of at least one Type I error in a

battery of nine tests is about 1 (.95)9 = .37.3 An auditor using this digit-by-digit approach will

thus see a false alarm when searching for fraud roughly seven times more often than an auditor

using only the overall test. An important (and potentially scary) byproduct of frequent false alarms is

that the output of a digital analysis may lose relevance for practicing auditors if it almost never

uncovers an actual fraudulent entry.

We suggest the following two courses of action for practitioners. First, consider doing both a

test-by-test and a digit-by-digit analysis when using Benfords Law. The Chi-squared goodness of fit

test is standard in many spreadsheet and statistical packages and the output from ACLs Benford

command could easily be adapted and tested in this manner. Second, on large data sets, consider

doing a Benford analysis of both the first digit, and the first two digits, of a particular set of data.

While examining the results of the 90 statistical tests produced in the two-digit analysis means an

even greater chance of discovering a Type I error than in the one-digit test, the resulting detail will

allow a practitioner to more easily determine which frequently occurring numeric amount is the

cause for the deviation from the expected Benford distribution, where the actual proportion exceeds

the expected proportion.

The situation faced by practicing auditors in this case is very similar to what is faced by

statisticians using an Analysis of Variance (ANOVA) to decide if the mean value of some variable is

the same across several different populations. The overall ANOVA null hypothesis, carried out using

an F-statistic, is that all of the means are equal (Ott 1993). Rejecting this null does not give any

indication of which populations have the largest or smallest mean values, or which pairs of popula-

tions have significantly different means. In the ANOVA setting, statisticians have a choice of several

methods for making these important distinctions. Some are conservative, and carefully control the

risk of Type I error. Others more aggressively look for differences and thus increase the chance of

Type I error. Auditors testing a distribution should be aware of the merits of the testing method

employed and choose accordingly.

The importance of this issue is heightened when considering that the Benford command in

certain auditing software packages (i.e., ACL) automatically supplies the digit-by-digit analysis

without even providing an option for an overall chi-squared test. The individual hypotheses H0,d are

tested using a normal distribution approximation (z-test). Auditors using Benfords Law or other

statistical tests to find evidence of fraudulent entries should be aware of the benefits of the test-by-

test and digit-by-digit approaches. Using an overall test-by-test approach makes it relatively

easy to control the probability of Type I error, but the results may not be as informative in the case

where fraud actually has occurred. Practitioners interested in applying a single test should consult a

standard business statistics book for detailed guidance (e.g., Keller and Warrack 2003, 535). Using a

digit-by-digit approach increases the chances of a Type I error, but also increases the chances of

finding actual fraudulent entries. Perhaps the most prudent approach would be to begin the analysis

stage with an overall analysis using a Chi-squared test, and then follow up with a digit-by-digit

analysis only if there is an indication of possible fraud in the overall analysis.

3 This calculation is not exact because it assumes that each of the nine tests is independent of the others. The authors have

completed a simulation study suggesting that the calculation is not very sensitive to this assumption.

Applying Digital Analysis Using Benfords Law to Detect Fraud: The Dangers of Type I Errors 81

REFERENCES

Beck, P., and I. Solomon. 1985. Sampling risks and audit consequences under alternative testing approaches.

The Accounting Review 60 (October): 714723.

Benford, F. 1938. The law of anomalous numbers. Proceedings of the American Philosophical Society 78:

551572.

Coderre, D. 1999. Computer-assisted techniques for fraud detection. The CPA Journal 69 (August): 5759.

Elliott, R., and J. Rogers. 1972. Relating statistical sampling to audit objectives. Journal of Accountancy 134

(July): 4655.

Hill, T. 1995. The significant digit phenomenon. American Mathematical Monthly 102 (4): 322327.

Keller, G., and B. Warrack. 2003. Statistics for Management and Economics. Pacific Grove, CA: Brooks/Cole-

Thomson.

McCollum, T., and D. Salierno. 2003. Choosing the right tools. Internal Auditor 60 (August): 3243.

Nigrini, M. 1996. A taxpayer compliance application of Benfords Law. The Journal of the American Tax

Association 18 (Spring): 7291.

, and L. Mittermaier. 1997. The use of Benfords Law as an aid in analytical procedures. Auditing: A

Journal of Practice & Theory 16 (2): 5267.

. 1999a. Ive got your number. Journal of Accountancy 187 (May): 7983.

. 1999b. Adding value with digital analysis. Internal Auditor 56 (February): 2123.

Ott, R. 1993. An Introduction to Statistical Methods and Data Analysis. Belmont, CA: Duxbury Press.

Tapp, D., and D. Burg. 2001. Using technology to detect fraud. Pennsylvania CPA Journal 71 (Winter): 2023.

Wallace, W. 2002. Assessing the quality of data used for benchmarking and decision making. Journal of

Government Financial Management 51 (Fall): 1622.

## Viel mehr als nur Dokumente.

Entdecken, was Scribd alles zu bieten hat, inklusive Bücher und Hörbücher von großen Verlagen.

Jederzeit kündbar.