Sie sind auf Seite 1von 4

HowTo

Fact
vs. THE TRUTH ABOUT PRECISION
:
AND BIAS /by Pat Picariello/

EVERY ASTM TEST METHOD


REQUIRES A P R E C I S I O N
AND B I A S SECTION. WHAT
IS IT? HOW DO I CREATE ONE?
READ ON.
HAVE YOU HEARD ANY OF THESE TERMS BEFORE— in 1956–where is the precision and bias section?!!”
precision, bias, repeatability, reproducibility, interlab- All levity aside, these are the members for which to
oratory, intralaboratory? Have they sent shivers run- be thankful. They understand that the backbone of the
ning up and down your spine? When the call goes out at ASTM test method is the precision and bias section.
a committee meeting for volunteers to participate in a This section is the curriculum vitae of the test method.
ruggedness test or round robin, do you suddenly find It is the pedigree, the provenance, the very foundation
your hands glued to the table? upon which the method rests. It provides validation for
If you answered yes to any of these questions, you’re the test method and ensures that it can do what it pur-
reading the right article (and you’re also the typical ports.
ASTM member).
MARCH 2000 // ASTM STANDARDIZATION NEWS

Anyone who has helped create, revise, distribute, or WHAT IS PRECISION?


catalog an ASTM test method has encountered the The Form and Style for ASTM Standards defines pre-
world of precision and bias. Every committee has dealt cision as follows:
with members whose detailed knowledge of Form and Precision—the closeness of agreement among test
Style for ASTM Standards (the Blue Book) has generat- results obtained under prescribed conditions.
ed negative votes because, “The precision and bias sec- A precision statement allows potential users of the
tion is insufficient per Section A21 of the Blue Book,” test method to assess, in general terms, its usefulness in
“The five-year window for a sole reference to repeata- proposed applications. A precision statement is not in-
bility data has long since closed,” or (the granddaddy of tended to contain values that can be duplicated in every
all negatives) “This is a test method that was developed user’s laboratory. The statement offers guidelines re-

16
ToolBox
Guide for the Format of a Research Report:
Go to www.astm.org (click on Technical Committees/Membership,
then on Standards Development Tools) or contact your staff manager.

ASTM Standard E 691, Practice for Conducting an Interlaboratory


Study to Determine the Precision of a Test Method and accompa-
nying software:
Go to www.astm.org (click on ASTM Store) or call Customer Service
at 610/832-9585. Your staff manager can lend a copy as well.

Your Peers:
Many ASTM members have successfully created valid P&B state-
ments. Network with your fellow committee members and your staff
manager for references. In addition, your committee may have devel-
oped its own standard for development of P&B statements.

garding the type of variability that can cannot be established. However, if the run the test(s) described in the pro-
be expected among test results when bias is unknown but the direction or posed or existing method and all re-
the method is used in one or more bounds of the bias can be estimated, sults will ultimately be compared on
reasonably competent laboratories. this information should be included an interlaboratory basis.
There are two measurements that in the bias statement. There are multiple schools of
serve to express precision in the eval- thought regarding the exact number
uation of a test method. They are THE RUGGEDNESS TEST of participating laboratories neces-
commonly referred to as “repeatabili- The determination of the vari- sary to perform a statistically mean-
ty” and “reproducibility” and provide ables associated with the performance ingful round robin. For example, the
the boundaries between which preci- of a test method is critical to the ulti- ASTM standard E 691, Practice for
sion exists. Simply put, repeatability mate accuracy of the precision of that Conducting an Interlaboratory Study
addresses variability between inde- method. The ruggedness test is de- to Determine the Precision of a Test
pendent test results gathered from signed to highlight these variables and Method, recommends that a mini-
within a single laboratory (otherwise allow them to be tightened if neces- mum of six laboratories supply test
known as intralaboratory testing). sary. If a ruggedness test is not per- results for a precision statement.
Reproducibility addresses variability formed prior to the actual interlabora- When making this determination, it is
among single test results gathered tory study, the participants in the important to remember to include
from different laboratories (other- study may never be able to determine sufficient laboratories to establish a
wise known as interlaboratory test- if or where any problems exist. Unlike reasonable cross-section of the labo-
ing). Due to the fact that repeatability the interlaboratory study, the rugged- ratory population, to ensure that the
tends to produce nominal variability ness test should be confined to a single poor performance of one or more
and reproducibility tends to produce laboratory to maximize the visibility laboratories will not skew the study,
appreciable variability, the bound- of any variables. Examples of variables and to guarantee that the repro-
aries for the precision are clearly es- to consider include, but are not limit- ducibility data is meaningful. A good
tablished. ed to, sampling, conditioning, proce- rule of thumb is that some precision
dure, pressure, temperature, and rela- information is always preferable to an
WHAT IS BIAS? tive humidity. absence of precision information.
The Form and Style for ASTM There is an ASTM Standard—E At the completion of the round
Standards defines bias as follows: 1169 Guide for Conducting Rugged- robin and after the formal balloting of
MARCH 2000 // ASTM STANDARDIZATION NEWS

Bias—a systematic error that con- ness Tests—that was specifically de- the data into the test method, a final
tributes to the difference between the signed to direct you through the steps report on the interlaboratory study
mean of a large number of test results necessary to complete this process. shall be provided to ASTM to be re-
and an accepted reference value. tained on file as a research report.
When included in a test method, HOW IS THE DATA GENERATED? Upon receipt of the data, ASTM will
this statement describes the bias and The interlaboratory study (ILS) assign a research report number and a
the methods utilized to provide cor- or “round robin” is utilized to gener- note giving the research report num-
rected test results. It is important to ate the final data used to support the ber will be editorially inserted into the
remember that if an accepted refer- statistical validity of the test method. precision and bias section of the test
ence value is not available, the bias Typically, a series of laboratories will method. It is important to remember
(CONTINUED ON PAGE 19)
17
FAQs
What is the cost of the E 691 software? How long can we wait after developing a test
ASTM will provide on loan a copy of the E 691 soft- method to include a P&B statement?
ware to an ASTM member for the purpose of work- If a committee determines that a delay is necessary
ing on a precision statement of a test method. to conduct a round robin, a temporary statement
ASTM does require that when the committee’s addressing repeatability conditions is permitted for
work with the software is completed, the software five years. Included in that statement shall be lan-
be returned. If a person, lab, etc., wishes to pur- guage regarding the date of actual availability of re-
chase E 691 to keep as a permanent resource, the producibility data.
cost of the E 691 software is $99 and the 22-page
hard or electronic copy of the standard is $40.
PRECISION AND BIAS
What should I do with the round robin data?
Does ASTM keep it on file? Does ASTM require strict compliance with E 691?
Once the round robin has been completed and the ASTM requires strict compliance with Form and
information has been balloted into the test method, Style for ASTM Standards, which mandates that a
the data should be submitted to ASTM to be kept precision and bias section be included in all test
on file as a research report. A number will be as- methods. A means toward achieving that end can
signed to the research report and a note will appear be E 691, or a standard based upon E 691 that has
in the precision and bias section of the test method been developed by an ASTM committee.
referencing the specific research report. The Guide
for the Format of a Research Report is located on the Can a committee publish a P&B statement if E
ASTM Web site (www.astm.org) as a standards 691 wasn’t used?
development tool. Yes. Section A21.2.7 of the Form and Style for ASTM
Standards indicates that a precision statement shall
Who absorbs the cost of round robin testing? be developed in accordance with the test program
In most cases, industry will assume the cost of prescribed in E 691 or by an interlaboratory test
round robin testing. The laboratories that volunteer program that yields equivalent information.
to perform the testing to generate precision and
bias data will do so with all time, effort, and materi- What happens if we find an old test method
als offered on a pro bono basis. without a P&B statement? Is it grandfathered?
The Form and Style for ASTM Standards mandates
Who can I contact at ASTM Headquarters if I that all test methods contain a precision and bias
need help understanding E 691 or using the statement. If a test method is discovered that does
software? not contain this section, an evaluation should be im-
The staff manager for your technical committee mediately performed to determine the utility of the
serves as the initial point of contact for all inquiries. document and steps should be undertaken to ad-
The manager will be able to refer you to the appro- dress precision and bias.
priate resource to answer your question. You will
often be referred to a member of the committee that What happens if we can’t produce the six labs
developed E 691—Committee E11 on Quality and that E 691 requires? Will the software still
Statistics. work?
The E 691 software is designed to work optimally
MARCH 2000 // ASTM STANDARDIZATION NEWS

Can a single company with multiple labs con- when data from a minimum of six laboratories is en-
sider each lab as a separate testing site to reach tered—the greater the number of participating lab-
the required number of labs in E 691? oratories, the smaller the chance that lab-to-lab
Yes. As long as each data set is generated by a lab- variability will be an issue. However, it is possible to
oratory with its own set of conditions, calibrations, generate a precision statement with fewer than six
and equipment condition and is then compared to participating laboratories; the bottom line is that
other laboratories with their own set of variables, some precision information is preferable to no pre-
the separate testing site requirement is met. cision information.

18
that a research report should be orga- robin and ultimately generating a HOW TO
nized in a specific format. See the precision statement. The answer to
ToolBox on how to acquire a Guide for this question is yes, such a document Methods for Analysis and Testing of
the Format of a Research Report. exists. In fact, multiple standards that Industrial Chemicals), D22 on Sam-
specifically address this issue have pling and Analysis of Atmospheres
WHY THE SPECIFIC RESEARCH been developed by ASTM technical (D 3670, Guide for Determination of
REPORT FORMAT? committees. Precision and Bias of Methods of
Due to the volume of test meth- The most widely applied of these Committee D-22), D13 on Textiles
ods that have been produced by documents is the ASTM standard E (D 2906, Practice for Statements on
ASTM (approximately 5,000), a con- 691, Practice for Conducting an In- Precision and Bias for Textiles), and
sistent format for the cataloging of terlaboratory Study to Determine the C09 on Concrete and Concrete Ag-
each research report is a necessity. Precision of a Test Method (see Tool- gregates (C 670, Practice for Prepar-
That format is detailed in the Guide Box). This standard was developed in ing Precision and Bias Statements for
for the Format of a Research Report. 1979 (and has been continuously up- Test Methods for Construction Ma-
This document identifies sections dated) by Committee E11 on Quality terials).
common to many interlaboratory and Statistics. According to its scope, The point to remember when
studies. All technical committees are E 691 “describes the techniques for searching for the appropriate stan-
encouraged to include as many of the planning, conducting, analyzing, and dard to aid in the determination of
sections relevant to each interlabora- treating the results of an interlabora- precision and bias is that choices do
tory study. The nine sections of the tory study (ILS) of a test method. The exist. A thorough search of the stan-
guide, with a brief description of their statistical techniques described in this dards database located on the ASTM
content are as follows: practice provide adequate informa- Web site (www.astm.org) will pro-
1. TITLE—A description of the tion for formulating the precision vide many options. If questions arise,
nature of the study. statement of a test method.” contact your staff manager. The staff
2. INTRODUCTION—Similar Additionally, E 691 has a compan- manager will be able to put you in
to the scope of a standard, this should ion Windows-based software package touch with the right technical contact
include the purpose of the study. that will aid the member in the calcu- to answer your specific questions and
3. TEST METHOD—A copy of lation and generation of
the method. a precision statement. The point to remember when
4. LIST OF PARTICIPATING This software is available searching for the appropriate
LABORATORIES—A list of all par- from ASTM for pur-
ticipating labs. It is important to note chase or can be loaned
standard to aid in the
that prior to the distribution of the to a member to assist in determination of precision and
data, each participating lab will re- standards development bias is that choices do exist.
ceive a code (Lab A, Lab B, etc.) that activities. Upon comple-
will allow them to identify only their tion of that activity, ASTM requires direct you toward the most applicable
data. that the software be returned. Stan- standard.
5. INTERLABORATORY TEST dards that further clarify the issues As precision and bias are manda-
PROGRAM INSTRUCTIONS—All discussed in E 691 include E 177, tory sections of an ASTM test meth-
instructions/guidelines provided to the Practice for Use of the Terms Preci- od, the utmost care should be taken to
participants of the testing program. sion and Bias in ASTM Test Methods, ensure that the final data, as well as
6. QUESTIONNAIRES—If and E 456, Standard Terminology Re- the steps that were taken to generate
questionnaires were distributed, in- lating to Quality and Statistics. the data, are as precise and accurate as
clude a copy with a summary of re- Several committees have chosen possible. A test method that is inca-
sponses. to develop statistical standards that pable of doing what it purports can be
7. DATA REPORT FORMS— specifically reflect issues faced by misleading. Quality precision and
All report forms returned by partici- their particular industry or technol- bias statements will help strengthen
pating labs. ogy. Most of these documents are the validity of the test method and
8. STATISTICAL DATA SUM- based on the principles found in E provide the user of the document
MARY—A summary of data re- 691, but have then been tailored to with the confidence to know that the
turned by participating labs. provide a greater degree of relevance test will work. After all, isn’t that the
MARCH 2000 // ASTM STANDARDIZATION NEWS

9. RESEARCH REPORT SUM- for their technical field. Committees point? //


MARY—Summary of results, signifi- that have chosen this path include
cance, and conclusions. D02 on Petroleum Products and Lu-
bricants (D 6300, Practice for Deter-
STANDARDS AS A RESOURCE: mination of Precision and Bias Data PAT PICARIELLO is a
E 691 AND OTHERS for Use in Test Methods for manager in the
A question that is frequently Petroleum Products and Lubri- Technical Committee
asked by the ASTM member is cants), E15 on Industrial and Spe- Operations Division.
whether a document exists to aid a cialty Chemicals (E 180, Practice for
committee in conducting a round Determining the Precision of ASTM

19