F

Analysis of Estimating Accuracy

F.1 Overall Estimating Accuracy

The following numerical example is presented to illustrate how a construction company can analyze the overall accuracy of its cost estimating. When making such a check, comparisons are made between estimated and actual total costs of completed projects. Only relatively recent projects should be considered because estimating personnel and pricing behavior have a tendency to change with time. Enough projects should be included to provide a statistically signicant sample. A minimum number of approximately 20 past projects might be required, with larger samples providing somewhat more reliable results. As an example, suppose that a construction contracting rm has completed 68 contracts during the past ve years. A ratio, R, is computed for each of these projects. R is equal to the actual total project cost divided by the estimated total cost. Cost in each instance includes all project expense, including eld overhead, but not prot. These values of R are then grouped together into equal intervals, say 0.02 or 2 percent. Larger or smaller intervals can be used depending somewhat on the size of the sample chosen. In the example, assume that the distribution of R-values shown in columns 1 and 2 of Figure F.1 is obtained for the 68 completed jobs. Using the data given in columns 1 and 2, the histogram in Figure F.2 is plotted. A frequency polygon is actually more useful and is drawn as dashed lines between the midpoints of the horizontal bars of the histogram. The
373

374

F Analysis of Estimating Accuracy

Frequency of Intervals of R Projects fi (1) (2) R<0.88 0.88<R<0.90 0.90<R<0.92 0.92<R<0.94 0.94<R<0.96 0.96<R<0.98 0.98<R<1.00 1.00<R<1.02 1.02<R<1.04 1.04<R<1.06 1.06<R<1.08 1.08<R<1.10 1.10<R<1.12 1.12<R Totals
R=

Interval Midpoint Ri (3) 0.89 0.91 0.93 0.95 0.97 0.99 1.01 1.03 1.05 1.07 1.09 1.11

f i x Ri (4) 0.00 0.89 0.00 2.79 5.70 11.64 15.84 14.14 9.27 3.15 2.14 1.09 1.11 0.00 67.76

Percentage Interval Cumulative Variance Cumulative Frequency Frequency 2 F (Ri - R) x fi F n x 100 (5) (6) (7) 0 1 1 4 10 22 38 52 61 64 66 67 68 68 0 1.47 1.47 5.88 14.71 32.35 55.88 76.47 89.71 94.12 97.06 98.53 100.00 100.00 0.0000 0.0113 0.0000 0.0133 0.0130 0.0084 0.0007 0.0026 0.0101 0.0086 0.0108 0.0087 0.0129 0.0000 0.1004
2

0 1 0 3 6 12 16 14 9 3 2 1 1 0 n = 68

Actual Project Cost Estimated Project Cost 67.76 68 = 0.9965 s = Standard Deviation = s=

(Ri R) x fi

n 1
0.1004 = 0.039 681

Average Value of R = R =

Figure F.1 Analysis of R-values

frequency polygon is now smoothed by sketching in a smooth curve that closely approximates the linear segments of the polygon. The resulting curve affords a quick and nonquantitative grasp of the companys bidding accuracy over the past ve years, as will now be explained. It is reasonable to assume that estimating errors should be essentially random in nature and conform to the well-known normal distribution. Another way of saying this, perhaps, is that there should be an equal tendency to overprice and underprice a construction estimate. Consequently, it is suggested that a good estimating system is one whose ratios, R, are normally distributed with a mean value of 1.0 and a small range of larger and smaller values. If the smoothed polygon obtained from the histogram of R-values resembles a normal distribution of small variance and with its mode at or very near a value of 1.0, such as that shown by curve A of Figure F.3, the overall bidding accuracy of the contractor has been good. Although curve B has its mode at 1.0, the large dispersion of values indicates loose estimating with a serious tendency to both underestimate and overestimate. A biased distribution such as curve C is indicative of systematic estimating errors or procedures that result in a pronounced

F.1 Overall Estimating Accuracy

18 16 14 12 Number of projects 10 8 6 4 2 0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 0

375

R=
Figure F.2 Histogram of R-values

Actual project cost Estimated project cost

Number of projects

C B

1.00
R
Figure F.3 Bidding accuracy distribution

376

F Analysis of Estimating Accuracy

70 60 50 40 38 30 20 10 0 1.12 1.14 0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10

Cumulative frequency, F

R=
Figure F.4 Cumulative frequency of R-values

Actual project cost Estimated project cost

tendency to consistently either underestimate or overestimate a majority of the jobs bid. If a smoothed polygon were to be sketched onto Figure F.2, the resulting curve would closely resemble a normal distribution with its mode at or very near a value, R, of 1.0. Another plot of interest is cumulative frequency versus values of R. The values of cumulative frequency are computed in Figure F.1 and are plotted in Figure F.4. Figure F.4 is actually a less-than curve. To illustrate, this gure discloses that 38 of the 68 projects had an R-value of less than 1.00. This means that on 38 of the completed projects, the actual costs were less than those estimated.

F.2 Evaluation of Accuracy

With regard to the evaluation of estimating accuracy, there are two measures of interest. One is the average or mean value of R. In this case, the average can be computed with satisfactory accuracy by the procedure shown in Figure F.1. As can be seen, the average value of the ratio of actual costs to estimated costs over the past ve years has been very close to the value of

F.2 Evaluation of Accuracy

377

1.0. The signicance of this is that, despite the fact that more jobs were overestimated than underestimated (38-30), there has been no consistent tendency to estimate too high or too low insofar as R-values are concerned. The other measure of accuracy has to do with the dispersion or scatter of values of R. In our case, there were no values of R less than 0.88 nor larger than 1.12. This indicates, in a general fashion, that during the past years, projects have been both underestimated and overestimated by as much as approximately 12 percent. These values are indicative of extreme range, however, and do not tell the entire story. A much better measure of dispersion is the standard deviation that is computed in Figure F.1 and has a value of 0.039. The signicance of this value will be discussed subsequently. Another way discrete data can be checked for normal distribution is to plot them on normal probability paper. This is a special graph paper available at technical bookstores and similar outlets. Figure F.5 shows a plot on normal probability paper of percentage cumulative frequency verus the ratio, R. The values of percentage cumulative frequency have been calculated in Figure F.1. The closer the points plot to a straight line, the better the normality of a set of data. R-values deviating from a straight

98 Percentage cumulative frequency 95 90 80 70 60 50 40 30 20 10 5 2 1 0.5 0.90 0.92 0.94 0.961 16 84

s 0.039 s 0.039

Mean 1.0

0.96

0.98

1.00

1.02

1.039 1.04 1.06

1.08

R=
Figure F.5 Normal probability plot

Actual project cost Estimated project cost

1.10

378

F Analysis of Estimating Accuracy

Estimating Accuracy Estimating Costs Job # Labor 200001 $242,146 200002 $186,184 200003 $12,463 200004 $144,128 200005 $278,934 200006 $577,590 200007 $52,534 200008 $182,890 200009 $161,492 200010 $452,127 Total $2,290,488 Actual Costs Job # Labor 200001 $258,975 200002 $192,812 200003 $12,394 200004 $151,118 200005 $286,577 200006 $586,600 200007 $52,376 200008 $185,780 200009 $161,169 200010 $466,143 Total $2,353,945 R Values Job # 200001 200002 200003 200004 200005 200006 200007 200008 200009 200010 Average R Std. deviation

Equipment Material $57,028 $237,922 $78,867 $370,277 $3,884 $26,018 $71,165 $52,934 $21,034 $310,225 $194,280 $605,308 $36,950 $76,541 $86,416 $215,880 $71,074 $220,315 $129,532 $415,658 $750,230 $2,531,078

Overhead $41,338 $64,066 $11,825 $50,751 $80,288 $104,596 $41,080 $66,014 $71,949 $112,371 $644,278

Equipment Material $56,030 $228,643 $78,173 $361,279 $3,794 $24,683 $71,983 $51,510 $20,923 $306,037 $195,601 $582,367 $36,780 $75,500 $85,336 $212,966 $71,458 $219,456 $128,677 $404,477 $748,755 $2,466,918

Overhead $38,986 $62,022 $12,824 $52,502 $79,622 $100,600 $40,550 $67,922 $76,129 $111,382 $642,540

Labor 1.070 1.036 0.995 1.049 1.027 1.016 0.997 1.016 0.998 1.031 1.023 0.024

Equipment 0.983 0.991 0.977 1.012 0.995 1.007 0.995 0.988 1.005 0.993 0.995 0.011

Material 0.961 0.976 0.949 0.973 0.987 0.962 0.986 0.987 0.996 0.973 0.975 0.015

Overhead 0.943 0.968 1.085 1.035 0.992 0.962 0.987 1.029 1.058 0.991 1.005 0.045

Figure F.6 Estimating accuracy of project components

F.3 Estimating Accuracy of Project Components

379

line indicate where the distribution is departing from normal. Figure F.5 shows that the distribution of R-values is close to normal except for the two extreme end points. The standard deviation of the R-values is found by the dashed line construction shown in Figure F.5, starting from the ordinate values of 84 and 16. The mean value of R is found from the ordinate value of 50. The data plot in Figure F.5 shows a mean value of very close to 1.00 and a standard deviation of 0.039, the same values as previously calculated in Figure E.1. Having established that the R-values are very close to being normally distributed, the following observations can be made concerning the companys long-term bidding accuracy using well-known properties of the normal distribution curve. First, about 68 percent of the projects bid lie within plus or minus a standard deviation from the mean. Having established that the mean is essentially 1.0, this means that approximately 68 percent of the projects have a ratio, R, of between 0.961 and 1.039. Another way of saying this is that about two-thirds of the projects have a bidding error no larger than 3.9 percent of the total project cost. In this regard, the error can be either plus or minus. It is also true that about 95 percent of the projects are located within plus or minus two standard deviations from the mean. This says that the maximum bidding error for 95 percent of the jobs bid is no larger than plus or minus 7.8 percent. What constitutes acceptable values of standard deviation is now a matter for management decision.

F.3 Estimating Accuracy of Project Components

The discussion of estimating accuracy presented in this appendix has been based on a comparative analysis of actual and estimated total project costs. Although a study of this kind provides the contractor with very valuable insight into the reliability of its overall estimating procedures, it does not provide information regarding the estimating accuracy of job components such as labor, equipment, materials, and project overhead. Figure F.6 presents the results of a detailed estimating accuracy study made of ten construction projects that our contractor estimated and constructed during the year 2000. This gure presents, for each project, the R-values for labor, equipment, materials, and project overhead. In each case, R-values are obtained by dividing actual cost by estimated cost. Thus, R-values greater than 1.0 indicate underestimating and less than 1.0 indicate overestimating. The R-values contained in Figure F.6 give the contractor a detailed and reliable indication of its recent estimating performance. Inspection of the data contained in Figure F.6 discloses that this contractor is more apt to underestimate labor and project overhead and overestimate equipment and materials.