MEASURE
FOR MEASURE
Calibration: What Is It?
by Graeme C. Payne
W
hen members of just about
any profession talk among
themselves, they usually use
specialized terms they all under-
stand—jargon. There is usually con-
sensus among them about the meaning
of the terms, but they often forget new-
comers or interested parties from other
fields may not have the same under-
standing. There are also cases in which
the common use of a word is different
from or even opposite its technical def-
inition. “Metrology” and “calibration”
are two such words.1
The International Vocabulary of
General and Basic Terms in Metrology
(VIM) is an internationally accepted
document that provides technical def-
initions of metrology, calibration and
many other measurement related
terms.2
Why do we need to refer to this tech-
nical glossary instead of a common
desktop dictionary? One reason is the
VIM is a listed authoritative reference
in ISO 9000 and ISO 10012. Another
reason is technical dictionaries and
glossaries define the accepted technical
meanings of terms, while common dic-
tionaries merely record the ways
words are used or misused in a lan-
guage and most of their meanings.
According to the VIM, metrology is
“the science and practice of measure-
ment.”3 Metrology is important in some
way to every human endeavor. More
specifically, it is critical to all the physi-
cal, chemical and biological sciences
and the technologies and manufactur-
ing processes that flow from them.
In daily life, metrology affects us in
commerce and law enforcement and
in regulated industries such as health-
care and aviation. Whenever you pur-
chase a gallon of gasoline or a pound
of onions, drive by an officer with a
radar gun on the highway or have an
electrocardiograph taken, you have
directly interacted with metrology by
means of a calibrated measuring
instrument. One practical application
of metrology—a subset of the whole
field—is the system of ensuring mea-
72 I MAY 2005 I www.asq.org
surements conform to certain defined
relationships.
A Process of Comparison
Broadly speaking, calibration is the
process of determining the relation-
ship between the readings obtained
by a measuring instrument or system
The common use
of this word is quite
different from its
technical definition.
and the applicable units of some
defined system of measurement.
According to records uncovered by
archaeologists, people have been
doing this for at least 5,000 years. At
first, units of measure were often
based on things such as the volume of
grain that could be held in two hands
(cup) or the distance between the
point of the Pharaoh’s elbow and
longest fingertip plus the width of his
palm (cubit). Now, our defined set of
measurement references is known as
the International System of Units (SI).4
Calibration quantifies the relation-
ship between the readings of a microm-
eter, voltmeter, thermometer, weighing
balance, mass spectrometer or graduat-
ed cylinder and the relevant units in
the SI system. Paraphrasing the formal
definition in the VIM, the instrument’s
readings are compared to the values of
a measurement standard under con-
trolled and specified conditions.5
Each measurement result can be
related to the SI units by the property
of traceability, which accounts for the
known or estimated uncertainty of the
measurement process.6 The measure-
ment standard has, in most cases, gone
through the same process. Calibration
is repeated at regular intervals to pro-
vide continued assurance the instru-
ment’s performance is suitable for its
use.
Calibration is essentially a process
of comparison. An instrument is used
to measure or is measured by a cali-
bration standard, and the result is
compared to two things: the known
value and uncertainty of the standard
and the performance specifications
required by the customer. The concept
is simple, but the work is in the
details. Some of the details include:
• The assigned value of the mea-
surement standard, which is usual-
ly determined from its calibration
history.
• The known uncertainty of the
standard, which comes from sev-
eral places, including the historical
reports of calibration and the
internal statistical process control
(SPC) methods many calibration
labs have for their measurement
systems. Labs that have an effec-
tive measurement SPC system
know how their systems perform
in that location, so their uncertain-
ty values are likely to be more
realistic—not always better, just
more realistic.
• The environment of the calibration
activity, which almost always
includes temperature and relative
humidity. Depending on the mea-
surement, other influences such as
absolute barometric pressure, the
local gravitational vector, electro-
magnetic fields or building vibra-
tion may also have measurable
effects.
• The methods and equipment used
to make the comparisons.
• The uncertainty of the measure-
ment system relative to the pub-
lished performance specifications of
the item being calibrated or the cus-
tomer’s requirements, if different.
All the details have to be evaluated
to provide a measure of the relation-
ship between the measurements made
with the instrument and the reference
SI values. That relationship is the
uncertainty, and its documentation
defines traceability, which is the char-
acteristic of the measurement result

 MEASURE
FOR MEASURE
that demonstrates its accuracy in
terms of the SI. Taken together, the
relationship and its documentation
indicate the quality of the measure-
ments that can be made with that
instrument when it is used correctly.
A calibration is performed using a
calibration procedure, which is a doc-
umented, validated and controlled
method for making the comparisons.
The procedure may be a written paper
or electronic document, or it may be a
particular test program on an auto-
mated calibration system.
Many calibration procedures are
written in conformance to the guide-
lines in Recommended Practice 3:
Preparation of Calibration Procedures
(NCSL RP-3), which defines the proce-
dures’ purpose and content.7
The purpose of a calibration proce-
dure is to determine and document
the measurement relationships of the
item being calibrated. It should define
the parameters to be measured, the
measurement standards to be used
74 I MAY 2005 I www.asq.org
and the data to be collected. It should
also list any safety precautions and
preliminary steps and make note of
the particular method of the compar-
isons, the calibration environment and
anything else that is important for the
items covered by the procedure.
A calibration procedure is written
with the fundamental premises that
the item being calibrated is in good
working order and the person per-
forming the calibration is trained and
qualified and understands the scien-
tific and physical principles of the
measurements.8
What About Adjustment?
Up to this point, I haven’t said any-
thing about adjustment. That’s
because it’s not part of the formal def-
inition of calibration, nor is it part of
the description of a calibration proce-
dure in NCSL RP-3. The results (data)
of a calibration procedure may indi-
cate a need for adjustment or other
repair, but taking such action is a sep-
arate process. After the adjustment or
repair is complete, the calibration pro-
cedure should always be repeated to
verify the proper measurement rela-
tionship has been re-established.
There are two reasons adjustment is
not part of the formal definition of
calibration:
1. The historical calibration data on
an instrument can be useful when
describing the normal variation
of the instrument or a population
of substantially identical instru-
ments. That information can also
be used for process improvement.
For example, it can be used to
evaluate the reliability of the
instruments and change the recal-
ibration interval. If the instru-
ment is adjusted before the data
from a full calibration run have
been collected, then there is no
historical value, and it cannot be
used to improve the system.
2. Any set of similar measurements
may be considered a statistical

 MEASURE
FOR MEASURE

process, and a single measure- GRAEME C. PAYNE is the president of GK

ment from that process is a ran- Systems Inc., a technical consulting company
dom sample from the probability near Atlanta. A Senior Member of ASQ, Payne
density function that describes it. has been working in electronic calibration and
product testing since 1981. He is a certified
Without other knowledge, there is
quality engineer, calibration technician and
no way to know if the sample is quality technician. He is also the chair-elect of
within the normal variation lim- the Measurement Quality Division and a
its. The history gives us that infor- member of NCSL International.
mation. If the measurement is
within the normal variation and
not outside the specification lim-
its, there is no reason to adjust it.
In fact, making an adjustment Please
could just as likely make it worse comment
as it could make it better. W.
Edwards Deming discusses the If you would like to comment on this
problem of overadjustment in article, please post your remarks on
chapter 11 of Out of the Crisis.9 the Quality Progress Discussion
The most common uses of the word Board at www.asq.org, or e-mail
“calibration” outside the metrology them to editor@asq.org.
community include the concept of
adjusting the instrument, and most cus-
tomers expect it. In addition, many
manufacturers have calibration proce-
dures in their manuals that are not per-
formance comparisons but, instead, are
the test, alignment or adjustment pro-

Customer
cedures used for a new or repaired unit
that is in an unknown condition.
Remember, a true calibration proce-
dure assumes the instrument is in good
working order. So, while acknowledg-
Care Center
ing the formal technical definition, cali-
bration providers also have to
recognize the practical realities of busi- ASQ’s customer care center is
ness and work with the common defin-
staffed with representatives to
ition as well.
personally answer each incom-
REFERENCES AND NOTES ing phone call and e-mail
1. This article is the first in a series of three.
The July column will examine some of the differ- during its hours of operation,
ent views of calibration, and the September col-
umn will discuss the importance of calibration. which are Monday through
2. International Vocabulary of Basic and General
Terms in Metrology, second edition, International Thursday from 7 a.m. to 6 p.m.
Organization for Standardization, 1993. It is com-
monly referred to as the VIM, an acronym taken
and Friday from 7a.m.to 5 p.m.,
from the French title, Vocabulaire International de
Termes Fondamentaux et Généraux de Métrologie.
3. VIM, section 2.2, see reference 2.
Central Standard Time.
4. SI is an acronym taken from the French
name, Le Systéme International d'Unités.
5. VIM, section 6.11, see reference 2. 800-248-1946 or
6. VIM, section 6.10, see reference 2.
7. Recommended Practice 3: Preparation of 414-272-8575
Calibration Procedures, NCSL International (for-
merly National Conference of Standards help@asq.org
Laboratories), 1990.
8. Ibid.
www.asq.org.
9. W. Edwards Deming, Out of the Crisis,
Massachusetts Institute of Technology, 1982.

76 I MAY 2005 I www.asq.org