Quality Engineering, 23:212216, 2011

Copyright # Taylor & Francis Group, LLC

ISSN: 0898-2112 print=1532-4222 online
DOI: 10.1080/08982112.2011.557345

Statistical Standards and ASTM, Part 3

Thomas D. Murphy1,
Thomas J. Bzik2,
Neil R. Ullman3
1

Murphy Statistical Consulting,

Morristown, New Jersey
2
Air Products, Allentown,
Pennsylvania
3
Independent Consultant,
Florham Park, New Jersey

In this article we continue to review standards published by ASTM

International originating from Committee E11 on Quality and Statistics.
These standards appear in Volume 14.02 of the Book of ASTM Standards
(2010), which is reissued annually. This article will discuss the suite of standards that are created and supported by ASTM Subcommittee E11.20 on Test
Method Evaluation and Quality Control. In the following discussion of these
standards, the scope (indicated by numbering 1.1, 1.2, etc.) found in each
standard is presented to give the reader an idea of what each standard is
intended to address. This is followed by some brief remarks for each standard covered. We end with a short outline of current and future work.
The scope of Subcommittee E11.20 is To develop and maintain standards
and to advise ASTM committees and others in the evaluation and statistical
quality control of test methods. The 10 standards listed in this article under
the jurisdiction of this subcommittee consist of five practices and five guides
and are designed to provide statistical support to ASTM International test
methods developed by its over 130 technical committees. The second
number in the standard designation is the year of the last revision.

E177-10 STANDARD PRACTICE FOR USE OF THE TERMS

PRECISION AND BIAS IN ASTM TEST METHODS
1.1. The purpose of this practice is to present concepts necessary to the
understanding of the terms precision and bias as used in quantitative
test methods. This practice also describes methods of expressing precision and bias and, in a final section, gives examples of how statements
on precision and bias may be written for ASTM test methods.

Edited by Stephen Luko, Hamilton

Sundstrand Corporation, Windsor
Lock, Connecticut
Address correspondence to Thomas
D. Murphy, T. D. Murphy Statistical
Consulting LLC, 33 Junard Drive,
Morristown, NJ 07960-2501. E-mail:
tmurphy33@aol.com

Part A of the Blue Book, Form and Style for ASTM Standards (2008),
requires that all test methods include statements of precision and bias
(P&B statements). This practice discusses sources of variation and further
defines concepts of repeatability precision (short-term within-laboratory
precision) and reproducibility precision (interlaboratory precision) for use
in commerce and laboratory management. Current work is underway to
further develop the concept of intermediate precision (long-term withinlaboratory precision). The practice also discusses the estimation of test
method bias compared with a referee test method or a certified reference
standard. Current developmental work on this standard is twofold: (1) reorganizing the order of presentation of the material, which was just completed
212

in 2010, and (2) updating the content on precision

and bias and updating the examples of precision
and bias statements for ASTM test methods with
respect to their ease of use and relevance.

E691-09 STANDARD PRACTICE FOR

CONDUCTING AN INTERLABORATORY
STUDY TO DETERMINE THE
PRECISION OF A TEST METHOD
1.1. This practice describes the techniques for
planning, conducting, analyzing, and treating
the results of an interlaboratory study (ILS) of a
test method. The statistical techniques described
in this practice provide adequate information
for formulating the precision statement of a test
method.
1.2. This practice does not concern itself with the
development of test methods but rather with
gathering the information needed for a test
method precision statement after the development stage has been successfully completed.
The data obtained in the interlaboratory study
may indicate, however, that further effort is
needed to improve the test method.
1.3. Because the primary purpose of this practice
is the development of the information needed
for a precision statement, the experimental
design in this practice may not be optimum for
evaluating materials, apparatus, or individual
laboratories.
1.4. Field of applicationThis practice is concerned
exclusively with test methods that yield a single
numerical figure as the test result, although the
single figure may be the outcome of a calculation from a set of measurements.
1.4.1. This practice does not cover methods in
which the measurement is a categorization,
such as a gono-go allocation (two categories) or a sorting scheme into two or more
categories. For practical purposes, the discontinuous nature of measurements of these
types may be ignored when a test result is
defined as an average of several individual
measurements. Then, this practice may be
applicable, but caution is required and a
statistician should be consulted.
213

This practice deals with the design, conduct, and

data analysis of an ILS for estimating the precision
part of the P&B statement and is a companion
standard to E177. Future work will address the analysis of data described in 1.4.1 in the above scope.

E1169-07 STANDARD PRACTICE FOR

CONDUCTING RUGGEDNESS TESTS
1.1. This practice covers conducting ruggedness tests.
The purpose of a ruggedness test is to identify
those factors that strongly influence the measurements provided by a specific test method and to
estimate how closely those factors need to be
controlled.
1.2. This practice restricts itself to designs with
two levels per factor. The designs require the
simultaneous change of the levels of all of
the factors, thus permitting the determination
of the effects of each of the factors on the
measured results.
Ruggedness tests, originating from W. J. Youden
of the National Bureau of Standards (now NIST),
determine the sensitivity of a test method to small
deviations from the test procedures and are basically designed experiments known as screening
designs. These can also be useful in designing a
test method as well as evaluating a test method
in use.

E1323-09 STANDARD GUIDE FOR

EVALUATING LABORATORY
MEASUREMENT PRACTICES AND THE
STATISTICAL ANALYSIS OF THE
RESULTING DATA
1.1. This guide covers key elements of an evaluation of
a laboratorys measurement practices and the
statistical analysis of the resulting data. This guide
addresses an evaluation that covers a broad range
of in-house quality measurements, some
of which may be directly related to accreditation
requirements.
1.2. This guide describes what to look for as documentation in order to verify the operation of
Statistical Standards, Part 3

the practices and what parts of the data to test

and interpret in order to verify the quality of data
being generated by the laboratory.
1.3. This guide does not specify or provide guidance
for the establishment or assessment of a quality
program.

E2489-06E1 STANDARD PRACTICE FOR

STATISTICAL ANALYSIS OF
ONE-SAMPLE AND TWO-SAMPLE
INTERLABORATORY PROFICIENCY
TESTING PROGRAMS

This guide originated in Committee E36 on

Accreditation and Certification and jurisdiction was
transferred to E-11 in 2002. It is intended to provide
guidance for an assessor to evaluate measurement
practices of laboratories, their protocol for statistically analyzing the resulting data from these practices, and their statistical results from these
practices.

1.1. This practice describes methods for the statistical

analysis of laboratory results obtained from interlaboratory proficiency testing programs. As per
Practice E 1301, proficiency testing is the use of
interlaboratory comparisons for the determination of laboratory testing or measurement performance. Conversely, a collaborative study (or
collaborative trial) is the use of interlaboratory
comparisons for the determination of the performance characteristics of a method, as covered
by Practice E 691.
1.1.1. Method A covers testing programs using
single test results obtained by testing a single
sample (each laboratory submits a single test
result).
1.1.2. Method B covers testing programs using
paired test results obtained by testing two
samples (each laboratory submits one test
result for each of the two samples). The two
samples should be of the same material or
two materials similar enough to have approximately the same degree of variation in test
results.
1.2. Methods A and B are applicable to proficiency
testing programs containing a minimum of 10
participating laboratories.
1.3. The methods provide direction for assessing
and categorizing the performance of individual
laboratories based on the relative likelihood of
occurrence of their test results and for determining estimates of testing variation associated with
repeatability and reproducibility.

E1488-09 STANDARD GUIDE FOR

STATISTICAL PROCEDURES TO USE IN
DEVELOPING AND APPLYING TEST
METHODS
1.1. This guide identifies statistical procedures for
use in developing new test methods or revising
or evaluating existing test methods or both.
1.2. This guide also cites statistical procedures
especially useful in the application of test
methods.
This guide describes various statistical methodologies for use in designing, developing, validating, and
evaluating test methods, and provides linkage to the
other E11.20 standards.

E2282-09 STANDARD GUIDE

FOR DEFINING THE TEST RESULT OF A
TEST METHOD
1.1. The purpose of this standard is to provide
guidelines for identifying the elements that
comprise the test result of a test method and
to illustrate how these elements combine into
the test result.
This guide provides information on constructing
the test result for a test method.
T. D. Murphy et al.

Proficiency testing (PT) programs are interlaboratory collaborative studies designed to estimate individual laboratory performance with respect to a test
method. Laboratories are rated as to their agreement
with other laboratories in the studies. This standard
advocates the use of robust statistics such as medians
and interquartile ranges for use in data analysis.
Graphical techniques are also described.
214

E2554-07 STANDARD PRACTICE FOR

ESTIMATING AND MONITORING THE
UNCERTAINTY OF TEST RESULTS OF A
TEST METHOD IN A SINGLE
LABORATORY USING A CONTROL
SAMPLE PROGRAM
1.1. This practice describes techniques for a laboratory to estimate the uncertainty of a test
result using data from test results on a control
sample.
1.2. Uncertainty as defined by this practice applies
to the capabilities of a single laboratory. Any
estimate of uncertainty determined through
the use of this practice applies only to the
individual laboratory for which the data are
presented.
1.3. The laboratory uses a well-defined and established test method in determining a series of test
results. The uncertainty estimated using this
practice only applies when the same test
method is followed. The uncertainty only
applies for the material types represented by
the control samples, and multiple control samples may be needed, especially if the method
has different precision for different sample types
or response levels.
1.4. The uncertainty estimate determined by this
practice represents the intermediate precision
of test results. This estimate seeks to quantify
the total variation expected within a single laboratory using a single established test method
while incorporating as many known sources of
variation as possible.
1.5. This practice does not establish error estimates
(error budget) attributed to individual factors
that could influence uncertainty.
1.6. This practice describes the use of control charts
to evaluate the data obtained and presents a special type of control chart to monitor the estimate
of uncertainty.
This practice describes the use of control charts
for monitoring results of a control sample program used for estimating uncertainty within a
laboratory.

215

E2655-08 STANDARD GUIDE FOR

REPORTING UNCERTAINTY OF TEST
RESULTS AND USE OF THE TERM
MEASUREMENT UNCERTAINTY IN
ASTM TEST METHODS
1.1. This guide provides concepts necessary for
understanding the term uncertainty when
applied to a quantitative test result. Several measures of uncertainty can be applied to a given
measurement result; the interpretation of some
of the common forms is described.
1.2. This guide describes methods for expressing test
result uncertainty and relates these to standard
statistical methodology. Relationships between
uncertainty and concepts of precision and bias
are described.
1.3. This guide also presents concepts needed for a
laboratory to identify and characterize components of method performance. Elements that
an ASTM method can include to provide guidance to the user on estimating uncertainty for
the method are described.
Part A of the Blue Book, Form and Style for ASTM
Standards (2008), introduces the concept of measurement uncertainty as an optional estimate of laboratory variation. Preparation of uncertainty estimates
is a requirement for laboratory certification under
ISO 17025. This guide describes some of the types
of data that the laboratory can use as the basis for
reporting uncertainty.

E2782-10 STANDARD GUIDE FOR

MEASUREMENT SYSTEMS ANALYSIS
1.1. This guide presents terminology, concepts, and
selected methods and formulas useful for
measurement systems analysis (MSA). Measurement systems analysis may be broadly described
as a body of theory and methodology that
applies to the nondestructive measurement of
the physical properties of manufactured objects.
This is a new guide for analysis of variability in
nondestructive testing and has just been approved.

Statistical Standards, Part 3

CURRENT ACTIVITY

ABOUT THE AUTHORS

Work items for revision of three E11.20 standards are

underway. WK24224 on E177 revision and WK28944
on E1488 revision, both dealing with content changes,
are in progress and should go to ballot early in 2011.
WK28947 on E2489 revision has been completed and
is going through the subcommittee ballot stage.

Thomas Murphy is an independent statistical consultant to the pharmaceutical, chemical, and consumer products industries. He is currently chair of
Subcommittee E11.20 and is a past chair of Committee E11. He is a fellow of ASTM International and
ASQ and is a member of ASA, AIChE, and the U.S.
TAG to ISO=TC 69.
Thomas Bzik is an industrial statistician with Air
Products Corporation in Allentown, Pennsylvania.
He is currently vice chair of Subcommittee E11.20
and ASA representative to E11 and is a past chair
of the Statistical Methods Task Force in SEMI
standards.
Neil Ullman is an independent industrial consultant. He is past chairman of Subcommittee E11.20
and Committee E11. He is a fellow of ASTM International and ASQ and is a recipient of the ASTM
Harold F. Dodge award. He is also a member of
ASA and the U.S. TAG to ISO=TC 69.

LONG-RANGE PLAN FOR

FUTURE WORK
In addition to continuous improvement of our
current standards, there are other evident needs to
address in the support of test methods. Some of these
have been identified for future standards and are as
follows:
.

Methodology for accepting equivalence of new test

methods, apparatus, or suppliers, and in the laboratory transfer of methods, using two one-sided t tests
(TOST) for single materials (Limentani et al. 2005),
and the Iglewicz-Tan (Tan and Iglewicz 1999)
procedure for multiple ranges of materials.
Statistical methods in calibration and evaluating
calibration time stability.
Evaluating homogeneity and stability of standard
reference materials.
Evaluating long-term laboratory performance in
proficiency testing.

E11.20 is always looking for additional contributors

to its work. If you are interested in participating,
please contact us.

T. D. Murphy et al.

REFERENCES
Form and Style for ASTM Standards. (2008). West Conshohocken, PA:
ASTM International.
General Methods and Instrumentation: Volume 14.02. Annual Book
of ASTM Standards. (2010). West Conshohocken, PA: ASTM
International.
Limentani, G., Ringo, M. C., Ye, F., Bergquist, M. L., McSorley, E. O.
(2005). Beyond the t-test: Statistical equivalence testing. Analytical
Chemistry, 221A226A. American Chemical Society.
Tan, C., Iglewicz, B. (1999). Measurement-methods comparisons and
linear statistical relationship. Technometrics, 41(3):192201.

216

Copyright of Quality Engineering is the property of Taylor & Francis Ltd and its content may not be copied or
emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.
However, users may print, download, or email articles for individual use.