ASTM Precision and Bias Statements

The precision and bias section is the curriculum vitae of the test method. It provides validation for the test
method and ensures that it can do what it purports.

The manual Form and Style for ASTM Standards defines precision as the closeness of agreement among
test results obtained under prescribed conditions and bias as a systematic error that contributes to the
difference between the mean of a large number of test results and an accepted reference value. In short,
precision addresses how close test results are to each other and bias addresses how close test results are to
the truth. They are two different things!

In the evaluation of a test method, two different measurements of precision are used: single operator
precision (also called repeatability) and multilaboratory precision (also called reproducibility). The former
deals with variability between multiple test results gathered from within a single laboratory and the latter
deals with variability among single test results gathered from multiple laboratories. Both are calculated as
a standard deviation of test results. Together, repeatability and reproducibility establish upper and lower
bounds for the precision of a test method.

The precision statement in an ASTM specification includes the standard deviations (sometimes denoted
by 1s) and 95% limits on the difference between two test results (sometimes denoted by d2s). The
95% limits are numerically equal to 2.8 times the respective standard deviations for data that are known to
be normally distributed. That same value is approximately correct for most other (non-normal) data.

The single-operator (repeatability) limits represent the greatest acceptable difference between results from
two tests conducted on the same material by the same operator. The multilaboratory (reproducibility)
limits represent the greatest acceptable difference between results from two tests conducted on the same
material by two different operators at two different laboratories.

A typical precision statement for a test to measure the strength of concrete cylinders might be:

The single-operator standard deviation (1s limit) has been found to be 160 psi throughout the
range 2300 to 4300 psi. Therefore, results of two properly conducted tests by the same operator
should not differ by more than 453 psi (d2s limit).

The multilaboratory standard deviation (1s limit) has been found to be 235 psi throughout the
range 2300 to 4300 psi. Therefore, results of two properly conducted tests from different
laboratories should not differ by more than 665 psi (d2s limit).

So what? Well, if your lab technician breaks two concrete cylinders and one breaks at 400 psi less than
the other, you cant conclude that one material is weaker than the other because the difference could be
due to random test error. On the other hand, if one breaks at 600 psi less than the other, the material
probably is weaker. This kind of information is important when writing acceptance specifications.

In cases where the test results can be compared to an accepted reference value, you can also write a bias
statement that states the amount by which the test over- or underestimates the correct answer. The bias is
expressed as a 95% confidence interval on the difference between the test results and the actual value. For
example the bias of the test method was found with 95% confidence to lie between 0.0062 and 0.0071.

It is important to remember that if an accepted reference value is not available, then the bias cannot be
established. That fact should be stated in the test specification (rather than simply omitting the statement).