1960 IRE TRANSACTIONS ON INSTRUMENTATION 73

A New Space Age Challenge Standards and

Electronic Measurements*
LLOYD B. WILSONt, MEMBER, IRE

ANYONE involved in some way in the design, pro- which we can use to measure accuracy-i.e., a meter
duction, calibration or use of instruments should which has a scale marked off in accuracy values so that
have a sense of personal pride in the fact that we can look at it and see if we have 5, 1, or 0.0001 per
he is participating in a critical field of work at the time
of our entry into the space age. Mankind's greatest ad-
venture planetary escape is about to take place. Man
himself has not yet achieved planetary escape, but his
instruments have already done so in the various rockets
and satellites which have been launched to date. This is
all very glamorous and fascinating to contemplate.
However, we know that behind these spectacular efforts
are a grueling and exhaustive series of measurements
and tests of materials, components, assemblies and
systems. The tests are complicated, but at the same
time interesting, because of the variety and ranges of
environmental conditions which will be encountered in
space cosmic radiation, temperature extremes, etc.'
In our adventure into space, we are depending upon
instruments to give us accurate information. This ap-
plies not only to instruments in rockets, satellites and
space craft, but also to those used in laboratories and
shops to develop and produce space age hardware. The
accuracy of these instruments is very important because
it has a direct effect on the amount of information which
can be communicated to others or which we gain for our
own use.
ACCURACY
Here we come face to face with the challenige of
standards and electronic measurements namely, how
reliable are the accuracy figures we use and quote? If we
assume that the accuracy of a measurement is better
than it really is, we may be in for a hard time in analyz-
ing and explaining the results of our measurements.
A logical question is, how do we determine accuracy?
Can we measure it directly, or if not, can we determine
it by some other method? First, consider the matter of
whether or not we can measure accuracy.2 Note that we
said "measure accuracy," not "accuracy of measure-
ment." Unfortunately, it is difficult, if not impossible, to
mneasure accuracy directly. We do not have a meter
*Received by the PGJ, June 22, 1960. Presented at the 1960 Con-
ference on Standards and Electronic Measurements as paper 1-1.
t Sperry Gyroscope Co., Great Neck, N. Y.
1 H. A. Manoogian, "The challenge of space," Electronics, vol. 32,
pp. 65-80; April 24, 1959. See especially pp. 67-68.
2 C. W. Churchman, "Why measure?" in "Measurement Defini- Product Engrg., vol. 31, p. 24, 25; May 23, 1960. (Interview with NBS
tions and Theories," C. W. Churchman and P. Ratoosh, Eds., John Director Dr. Allen V. Astin by Product Engineering's \Vashington
WRiley and Sons, Inc., New York, N. Y. p. 92; 1959. editor.)
cent accuracy.
If we cannot measure accuracy by some simple proc-
ess such as with a meter, how do we go about determin-
ing it? As with almost everything we do, there are right
and wrong ways of determining accuracy. At the top of
the list of the wrong ways is the matter of just having
blind faith3 in the readings of instruments. An example
of this is the tendency to take a manufacturer's specifica-
tion of accuracy at face value, even when the accuracy
quoted is better than the National Bureau of Standards
can supply. A similar example is the high degree of confi-
dence often placed in a manufacturer's specified accu-
racy, even after the instrument in question has been
used for several years without having been calibrated
since it was received from the manufacturer. In other
cases it is not unusual for people to assume that accuracy
is equal to the readability of the instrument being used,
or that accuracy is equal to the precision or degree of re-
peatability obtained when a measurement is repeated
several times.
None of these methods is the way to determine or
even to estimate the accuracy of a measurement. They
do have some effect on accuracy but other important
factors must also be considered. For example, in this
day and age we depend principally upon a hierarchical
system for determining the accuracy of the instruments
we use. In other words, in practically all cases today the
accuracy of an instrument or a standard must be deter-
mined by comparison with a standard which is higher in
the over-all hierarchy of standards. In the United States
this higher level standard is either one which has been
calibrated at the National Bureau of Standards or at
least one whose calibration can be traced to NBS. The
figures we quote for the accuracy of a measurement or a
calibration tell the limits of probable deviation of the re-
sults from the NBS national standard for this particular
measurement quantity.
Eventually we may have a better system whereby
each of us can determine accuracy in our own labora-
tory by comparison with independently determinable
standards, without going through a hierarchy of stanld-
3"What has the missile age meant to the Bureau of Standards?"

Authorized licensd use limted to: Imperial Coleg Lond. Downlade on June 07,21 at 19:35 UTC from IE Xplore. Restricon aply.
 76 IRE TRANSACTIONS ON INSTRUMENTATION September
ards to NBS. One of the few examples of such an inde- know that these are areas where NBS offers no calibra-
pendently determinable standard which is practical and tion service, except to a limited extent for X-band bo-
in use today is the atomic beam type of frequency lometer mounts. However, there are many other meas-
standard. Independently determinable standards would urement categories almost as deficient. The answers to
be acceptable substitutes for the NBS national stand- the above questions would probably have to be that we
ards becatuse they would represent the true or abso- calibrate these items or these quantities indirectly. For
lute value of each of the quantities being standard- example, we may calibrate a water flow type of micro-
ized. The only use for the hierarchical system in such wave calorimeter by a derived technique which depends
a utopian age of the future would be for the academic upon flow rate and temperature rise of the water. In the
satisfaction to be gained from using it as a cross check. case of power measurements with bolometer mounts
However, today such independently determinable we may not worry about the efficiencies of the mounts
standards are the exception rather than the general rule, we use, but instead, rely on a frequency conversion
and until such standards are developed for a large num- technique wherein dc or low-frequency audio power is
ber of measurement quantities we must resort to the substituted via a bridge circuit in an amount equal to
hierarchical approach to determine the accuracy of our the microwave power being measured.
measurermenlts and calibrations.
meAcurenacyt and caliberaetiolns.led byblndfathin LOCAL COMPATIBILITY vs
Accuracy cannot be determined by blind faith in NATIONAI COMPATIBILITY

instrumenit readings, by readability, resolution or sensi- We hope the preceding sections have proved that it
tivity of instruments, nor by repeatability of instrument is not too hard to find examples where companies are
readings. Instead, accuracy must be determined in establishing many of their own standards indirectly by
terms of the accuracy of a standard against which a means of derived calibration techniques or frequency
calibrationi is made, coupled with a proper analysis and conversion techniques. We are now faced with the ques-
evaluation of the various sources of error which may tion of whether this in itself is anything to worry about.
have arisen in the course of making the calibration. After all, some people may even prefer this approach to
Accuracy implies compatibility or agreement of re- calibrations-particularly the derived type of calibra-
sults in an organized system of measurements. An in- tion which sometimes is claimed to be a "primary cali-
teresting light is shed on this matter of compatibility by bration" in the sense that it is made in terms of quan-
the calibration and standards surveys made within the tities other than the quantity being standardized. This
past several years-surveys such as that made by the might be well and good if there were reasonable assur-
Aerospace Industries Association (AIA),4 or surveys ance that all companies making, e.g., a derived calibra-
made by the military services. Many of us have prob- tion of a water flow microwave calorinmeter in terms of
ably helped answer questionnaires or at least have seen flow rate and temperature rise of the water, were using-
reports of one or more of these surveys. The most the same procedure and the same types of equipment.
startling conclusion from these surveys is that the needs Yet, many differences can arise here. One company
of industry for calibration services are, in many cases, might measure the temperature rise by the use of ther-
so far beyond the capabilities of NBS that individual mometers, another company might use a calibrated
companies are establishing their own standards. The thermopile, while a third company might measure the
accuracy and therefore the compatibility of these stand- 60-cps ac power substituted into a heater coil in series
ards with standards of other companies and of the mili- with the water load of the calorimeter. And no mention
tary services is very questionable! has even been made yet of the additional possibilities of
error arising from mismatch of the calorimeter water
load or the inefficiency of the water load. About the best
We do not have to look far to see that companies are that can be hoped for under these circumstances is that
establishing many of their own standards in this way. In there will be a certain degree of local compatibility of
fact, most of us can probably cite examples from the calibrations and measurements within a given group, de-
calibration laboratories or engineering departments of partment, or perhaps even an entire company. The
our own companies. For example, how does your com- amount of this local compatibility would be primarily a
pany make calibrations of microwave power in any fre- function of the use of identical calibration procedures
quency bands except X band? How are calibrations and equipment throughout the particular group, depart-
obtained for your microwave calorimeters, your bolom- ment, or company.
eter mounts? Or, how do you calibrate microwave When we consider the hundreds of companies which
peak power? How do you calibrate pulse voltage? These make derived and frequency conversion calibrations, it
examples have been chosen intentionally because we is hard to see how there can be any reasonable degree of
national compatibility with a system such as this. How-
4F. McGinnis, H. Martin, J. Novotny, and L. Wilson, "Aero- ever, national compatibility of measurements is par-
space Industries Association Industry Calibration Survey," AIA ticularly important these days in view of the complex
Quality Control Committee report published by the project sponsor,
Sperry Gyroscope Co., Great Neck, N. Y.; 1959. nature of military weapons systems which are being de-

Authorized licensd use limted to: Imperial Coleg Lond. Downlade on June 07,21 at 19:35 UTC from IE Xplore. Restricon aply.
 1960 Wilson: Standards and Electronic Measurements 77
veloped. The materials and components which comprise each and every one of us in the taxes we pay to the fed-
such a systemii are rarely the exclusive product of one eral government every year. In addition, even with this
manufacturer alone. The prime contractor for a typical huge expenditure of money, we are barely able to main-
weapon system uses products from a large number of tain our pace as we run neck and neck in the missile and
subcontractors anid vendors. In addition, once the prod- satellite race with Russia.6
uct has been delivered to the military service which has So much for the results obtained to date. Perhaps in-
contracted for it, the product is likely to be used by vari- stead of worrying about them we should take a quick
ous military groups at widely different geographical lo- look to see if the prospects for the future are any better.
cations throughout the world. For this reason, we can no The only thing to do here is to try to evaluate the ef-
longer afford to regard local compatibility as being suffi- forts being expended to improve these situations. In do-
cient to assure accuracy of measurement. Dr. Allen V. ing this, we feel competent only to evaluate the calibra-
Astin, the director of the National Bureau of Standards, tion and measurement efforts. There are other efforts
said in anl interview reported recently in a national mag- of various types which will also contribute to the final
azine,3 "It is important to develop increased apprecia- results we may expect in the future, but the measure-
tion of the necessity of relating measuring instruments ment and calibration efforts should not be overlooked.
to national standards whenever measurement data For example, one of the conclusions which was quite
taken at one spot must be compatible with measure- obvious from the first AIA Calibration and Standards
ments in other locations." Survey was that many companies could not show trace-
SPACE AGE NEEDS FOR IMPROVED STANDARDS, ability of their calibrations to NBS. The situation is
complicated by the fact that NBS does not provide
CALIBRATIONS AND MEASUREMENTS many needed calibration services and as a result many
Earlier we used the word startling to describe the con- companies have been forced to establish their own
clusion that individual companies are establishing their standards on a local compatibility basis.
own standards although their compatibility with
standards of other companies and of the militarv serv- EFFORTS BY THE NATIONAL BUREAU OF STANDARDS
ices is questionable. Perhaps the word shocking or even AND BY THE \4LITARY SERVICES
the word frightening would have been more appropriate. In evaluating the efforts being expended to improve
At this time in the history of mankind, when we are these situations, we should also take into account what
relying upon new and dramatic scientific achievements NBS is doing and has been doing while these problems
to uniock the doors of the space age, it is frightening to were developing. The people at NBS were aware of these
think that we are building our new space age technolo- problems, in a general way, even before the rash of sur-
gies on such unsubstantial foundations. This is parti- veys made during the past two years, and they have
cularly true in the electrical, radio frequency and micro- been doing the best they can with the facilities and
wave measurement categories in which so many of to- within the financial limitations under which they oper-
day's calibration problems exist and which we are dis- ate. An example of these limitations is that this year
cussing at this Conference. It is doubly significant for they have essentially the same staffing level as they had
these measurement categories because it is by electrical, in t950.3 However, the indications are that NBS needs
radio frequency and microwave devices that we will three to five times the facilities it now has, with a pro-
control, acquire and analyze much of the data which portionate increase in its staff to work with these fa-
will supply basic information about our experiments in cilities.7 About all we can conclude here is that NBS is
the exploration of space.5 forging ahead slowly over a broad front, doing its best
At this point we might logically ask: If calibrations against overwhelming odds. Unless drastic changes oc-
and stanidards are that important in supporting our new cur, it cannot hope to catch up with all the requirements
space age technologies, why are we so far behind? Why for increased accuracies, increased ranges of measure-
isn't more being done right now? Do we really have ment, and new categories of measurement.
calibration and measurement problems, or are we mak- The military services have been increasingly con-
ing mountains out of molehills? An analysis of the re- cerned with the need for improvements in calibrations
sults of our progress into the space age to date shows and standards. The effort here is evident in the form of
that we still have reliability problems, we still have rather elaborate standards and calibration programs set
schedule delays because many problems are solved on a up within their own organizations. The Navy has its
crisis b)asis, and we still have constantly increasing costs. calibration program affecting contractors on certain
All of these things are directly reflected in our huge major NTavy projects, as described in two papers given
national budget of approximately $40 billion a year for
the military services. This, in turn, is directly affecting 6L. V. Berkner, "IRE enters space,"' PROC. IRE, VOl. 47, PP.
1048-1052; June, 1959. See p. 1052.
5C. H. Hoeppner, "Space electronics," PROC. IRE, vol. 48, pp.
7"Proceedings of the Forum, 15th Annual Meeting, Standards
and Metrology Division, American Ordnance Association," Redstone
435-437; April, 1960. See p. 436. Arsenal, Huntsville, Ala., p. 5; January 22, 1960.

Authorized licensd use limted to: Imperial Coleg Lond. Downlade on June 07,21 at 19:35 UTC from IE Xplore. Restricon aply.
 78
at the last Conference on Electronic Standards and
Measurements held at the NBS Boulder Laboratories
in August, 1958.8,9 In addition, all of the military serv-
ices have requested their resident inspectors to check
more carefully on contractor efforts in providing ade-
quate calibration of instruments used in connectioi with
militry contracts.
THE CALIBRATION SURVEYS OF 1959 AND 1960
None of the above reasons really tell why we are so
lacking in support of our new space age technologies, nor
do they tell how much we are suffering from lack of
proper calibrations and standards in this work. It was
to try to probe deeper and to answer these questions
that the AIA surveys and other similar surveys were
made. The years 1959 and 1960 may well go down in the
history of metrology as the years of the calibration sur-
veys; not that there were no surveys ever made before
this to determine calibration and standards needs, but
during these two years a number of surveys were made.
This paper refers to the AIA calibration and standards
surveys a number of times because these are the ones
with which the author has been most intimately asso-
ciated, having helped to formulate a number of the
questions, to prepare the reply of the Sperry Gyroscope
Company to the questionnaires, and to analyze and pre-
pare the AIA reports on the results from the question-
naires. However, similar surveys were made by the
Electronic Industries Association and by the National
Security Industrial Association. In addition to the sur-
veys made by these associations, teams of Air Force and
NBS personnel visited many companies working on
defense contracts to gain first-hand information of cali-
bration problems and needs.
A major step forward in initiating these surveys was
taken in November, 1958, when Colonel J. G. Schneider
and Colonel Richard Stolle of the Air Force Head-
quarters Air Materiel Command Quality Control group
at Wright Field requested the AIA Quality Control
Committee to question its member companies regard-
ing their needs for improved calibrations and standards.
In December, 1958, and February, 1959, two question-
naires were sent out by ALA, and the resulting informa-
tion was made public at a press conference held at the
Sperry Gyroscope Company in Great Neck, N. Y. on
August 25, 1959. The AIA report on these surveys, as
well as the individual replies, were studied carefully by
NBS. The information was found to be extremely help-
ful and, as a result, work was started in some of the
problem areas.
However, NBS felt that it was important to obtain
additional information about the exact nature of cer-
tamn indicated problem areas and the degree of urgency
TRANS. ON' INSTRUJMENTATION, VOl. 1-7, PP. 357-360; December,clbainsrie a enaalbei ubn oa
1958.
patM.nL. Stcroggls, "InstBrument cfalibrratiaontpCrsogram wilthintThe De-
ON INSTRUMENTATION, VOl. 1-7, PP. 360-363; December, 1958.
Authorized licensd use limted to: Imperial Coleg Lond. Downlade on June 07,21 at 19:35 UTC from IE Xplore. Restricon aply.
IRE TRANSACTIONS ON INSTRUMENTATION September
associated with them. Therefore, in April, 1960, the AIA
again questioned its Quality Control Committee mem-
ber companies in a survey of priorities of industry cali-
bration needs. The results of this are shown in the bar
graphs of Fig. 1. These bar graphs cover only the electri-
cal, radio frequency, microwave and time portions of
this survey. Actually, the survey covered all major
measurement categories, including categories such as
dimensional, infrared, temperature, humidity, pressure,
optics, etc. As we can see from Fig. 1, there are a large
number of companies which have problems in the
electrical, radio frequency, microwave and time cate-
gories of measurements.
The next step in obtaining additional information
about the exact nature of certain indicated problem
areas and the degree of urgency associated with them
was the planning of a series of meetings between NBS
and industry personnel to discuss these problem areas.
The official title given to these meetings is the Measure-
ments Research Conference series, and the first two
meetings have already been held on the subjects of
microwave power and microwave attenuation measure-
ments. These meetings were held at the NBS Boulder-
Laboratories on May 24 and 25, 1960, and were at-
tended by twenty industry representatives and by a
number of NBS personnel. The objectives of these
meetings were as follows:
1) To present and discuss with NBS personnel the
needs of defense industries for new types of micro-
wave power and attenuation calibration services
where NBS provides no services.
2) To present and discuss with NBS persoiinel the
needs of defense industries for microwave power
and attenuation calibration services of increased
accuracy and/or increased magnitude and fre-
quency ranges where NBS now provides only
limited services.
3) To recommend courses of action for NBS, industry
and others which will result in long-range as well
as interim solutions to meet these needs.
Several interesting results came out of these meetings.
The first is that there definitely are urgent needs for
better calibrations in the microwave power and micro-
wave attenuation categories. For example, one of the
speakers at the first Measurements Research Conference
meeting on microwave power pointed out that a saving
of several months of schedule time and several hundred
thousand dollars could have been achieved in the de-
velopment of a high-power klystron tube if proper NBS
power calibration services had been available. At the
second M4icrowave Research Conference meeting on
microwave attenuation another speaker said that almost
a million dollars might have been saved if attenuation
accuracy of a few tenths of a db for a range of 60 to 80
db. If such calibrations had been possible, a computer
could have been used to calculate the design of anl an-
1960 Wilson: Standards and Electronic Measurements 79

ELECTRICAL CATEGORY
DC RESISTANCE 4
DC VOLTAGE - DIGITAL 2 1_ 0
VOLTMETERS
A C VOLTAGE 6 9

DC VOLTAGE RATIO

PULSE VOLTAGE

CAPACITANCE

RADIO FREQUENCY CATEGORY

POWER

FREQUENCY 7

ATTENUATION

IMPEDANCE

PHASE

VOLTAGE

CURRENT

FIELD STRENGTH 6

MICROWAVE CATEGORY
UNMODULATED AND AVERAG
PULSED POWER
PULSED POWER, PEAK

FREQUENCY

ATTENUATION
VSWR

PHASE

NOISE
CALIBRATION OF FIELD
INTENSITY DEVICES..
PROPERTIES OF MATERIALS |_

TIME CATEGORY
TIME INTERVAL _
FREQUENCY

LEGEND:
HIGH MEDIUM LOW Q NuMBER OF REPLIES

Fig. 1-Results of AIA survey of priorities of industry calibration needs.

Authorized licensd use limted to: Imperial Coleg Lond. Downlade on June 07,21 at 19:35 UTC from IE Xplore. Restricon aply.
80 IRE TRANSACTIONS ON INSTRUMENTATION September
tenna radome instead of having to resort to a trial and use the services of NBS to establish better direct trace-
error approach which involved several million dollars of ability of our calibrations, i.e., calibrations performed in
tooling costs. Another speaker pointed out that from a terms of the same units as the quantities being stand-
company primary standards laboratory point of view, ardized. In addition, for those measurement categories
it was necessary to have increased accuracy in micro- where NBS cannot yet provide direct calibration serv-
wave attenuation measurements because manufacturers ices, we must work hard to achieve an interim solution
of conmmercial instruments are specifying, and in some by establishing a nationally compatible system of in-
cases even guaranteeing, accuracies for microwave at- directly traceable calibrations. In other words, we must
tenuators and directional couplers which are equal to or expand our present system of local compatibility to
better than those NBS will certify. One of the results make it a nationally compatible system for calibrations
of the first two meetings is that a series of recommenda- which must be obtained by derived techniques or by
tions was prepared and is Inow beinig reviewed for ac- frequency conversion techniques.
tions to be taken by NBS, the military services, and in- The third and final part of the challenge is that we
dustry to obtain improved calibrations for standards of must close the loop by checking on the end result we
microwave power and microwave attenuation. seek-accurate measurements to see how well we are
achieving that goal. This can be done by a measure-
CoNcL.ITSJON ment-agreemenit type of checking operation. Such a sys-
By now I think that we can start to see the over-all tem already has proven practical in company opera-
picture the real challenge taking shape. It consists tions; therefore, I see no reason why it should not be
of three parts. First, the necessity for action must be practical on a national scale. For example, the General
brought out into the open and made known. Here we do Electric Company does this throughout its departments
not look to NBS for the needs; they must come from and divisions spread all over the United States by what
each and every one of us in industry and in the military is called an interlaboratory comparison of standardizing
services. One of the things which I feel very strongly facilities.10The Sandia Corporation determines quality of
about, and which was discussed as one of the recom- work done in Atonmic Energy Commission secondary
mendations at the first two Measurements Research calibration laboratories by means of what are called
Conference meetiings, is the need for improved com- technical audits." Establishment of a national system
munications withini industry to pass on the need for of checking similar to these would show whether or not
better calibrationis to NBS. Few, if any, companies have we are getting accurate and compatible measurements
a central clearinig house for information on calibration and, if not, it would show where effort is needed most.
needs. Eveni company standards laboratories which Each of us can help to meet this new space age chal-
might normally be expected to have this information lenge the challenge of accurate and compatible stand-
readily available do not seem to have it. This is im- ards and measurements. Onily through our efforts can
portant not only in providing proper calibration services the need for better standards and measurements be
within a company, but also in passing appropriate parts brought to light and passed on to the National Bureau
of this needs information on to NBS. If the needs are of Standards. And only through our cooperaiion can
there but are not passed on to those who can take action there be a measurement-agreement type of checking
to meet thenm, then there is very little chance that any- operation on a national scale to tell how well we are
thing will be done until a crisis arises. In the case of cali- meeting the challenge. The end results will be improved
brations aind measurements, the crisis approach is being quality, improved reliability and reduced cost for space
relied on too frequently to solve problems, and, as we age products.
all know, the crisis approach to solving calibration and
measurement problems involves not only extra money
costs but also schedule delays. 10 S. C. Richardson, "Reference Level Standards in a Large Elec-
trical Manufacturing Company," presented at the 14th Annual In-
The second part of the challenge is that once these strument-Automation Conf. and Exhibit, Chicago, Ill., p. 9-59-4;
needs are made known, action must be taken to meet September 21-25, 1959. (Instrument Society of America Conf. Pre-
print No. 9-59.)
them. NBS must be the focal point in this effort, but 11 H. C. Biggs, "Physical and electrical standardization program
364-370;AEC,"
for the IRE TRANS. ON INSTRUMENTATION, VOl. 1-7, pp.
this is not a problem for NBS alone. We must learn to December, 1958. See p. 365.

Authorized licensd use limted to: Imperial Coleg Lond. Downlade on June 07,21 at 19:35 UTC from IE Xplore. Restricon aply.