11/9/2017 Statistics in Analytical Chemistry: Part 8—Calibration Diagnostics | American Laboratory

Statistics in Analytical Chemistry: Part 8—Calibration Diagnostics

Pos ted: Nov em ber 2 1 , 2 0 0 3 Share

Dav id Colem an

Ly nn Vanatta

The past sev en articles hav e prov ided the fundam entals of calibration, including a protocol for designing a calibration
study . Once such a study has been perform ed (i.e., the chosen standards hav e been analy zed in replicate), the data are
av ailable for constructing a calibration curv e. (See installm ents 5 and 6 in American Laboratory June 2 003 and July
2 003 , respectiv ely , for details on calibration design.) It is tem pting sim ply to fit a straight line (SL) using ordinary least
squares (OLS) and to ev aluate the SL m odel using only the correlation coefficient, or its square, R2 . Note that
R2 represents the proportion of the v ariation in the response (y-v alue) that is “explained” by the calibration. If R2 is high
enough (ty pically 0.9 9 or better), the curv e is deem ed adequate.

Howev er, m ore rigorous testing should be done to ensure that the selected m odel and fitting technique are adequate.
Statistics can prov ide the necessary tools for this task. This article is the first of three that will discuss the process in
detail. The ev aluation inv olv es sev en basic steps, all of which are easy to perform with the help of statistical software.
The steps are:

Plot response v ersus true concentration

Determ ine the behav ior of the standard dev iation of the response
Fit the proposed m odel and ev aluate R2 adj
Exam ine the residuals for nonrandom ness
Ev aluate the p-v alue for the slope (and any higher-order term s)
Perform a lack-of-fit test
Plot and ev aluate the prediction interv al.

Before these steps are discussed in detail, the concept of p-v alue (“p” stands for “probability ”) m ust be introduced. This
statistic will be used throughout the diagnosis process to guide decisions about plausible hy potheses. Most statistical
ev aluations of data inv olv e hy pothesis testing. A proposed assum ption (called the null hy pothesis) is m ade for the data; a
contrasting assum ption is called the alternativ e hy pothesis. When the statistical test is perform ed, the question being
asked is, “What are the odds of getting this set of data (or data at least this unusual, as defined by the alternativ e
hy pothesis), purely by chance, if the null (or starting) hy pothesis is true?” The p-v alue is that probability . If the odds are
low (ty pically less than 1 % or 5%, depending on the test being used), then the null hy pothesis is rejected and the
alternativ e is accepted. Note that one can nev er prove a hy pothesis to be false; one can only decide based on the weight of
the ev idence, m uch as in a court of law.

Step 1: Plot response versus true concentration

As m ight be expected, the first step in constructing a calibration curv e is to plot the response data v ersus the respectiv e
true concentrations. Ev en without hav ing a m odel fitted to the points, this plot is usually inform ativ e. Any data that are
possible outliers (or ty pos in the data table) m ay be obv ious. Suspect points should be inv estigated to see if they are
correct and if they belong with the rest of the data set. However, “errant” (or inconv enient) data should not be
perm anently excluded unless a sound phy sical reason can be found for taking such action. As m uch as possible, any
calibration experim ent should capture the v ariability that will likely enter into the ty pical analy sis process. (The topic of
outliers will be discussed in m ore detail in later installm ents.) If there is a high degree of curv ature, this situation m ay
also be detectable.

Step 2: Determine the behavior of the standard deviation of the response

http://www.americanlaboratory.com/913-Technical-Articles/1703-Statistics-in-Analytical-Chemistry-Part-8-Calibration-Diagnostics/ 1/2
11/9/2017 Statistics in Analytical Chemistry: Part 8—Calibration Diagnostics | American Laboratory

The im portance of this second step cannot be ov erem phasized. If the com puted standard dev iation (SD) of the responses
changes sy stem atically (e.g., increases or decreases) with concentration, then ordinary least squares is not the
appropriate fitting technique. Recall from installm ent 3 (January 2 003 ) that one of the assum ptions behind OLS is that,
“The standard dev iation of the responses does notchange ov er the range of x v alues for which the m odel will be applied.
Howev er, in analy tical chem istry , this assum ption does not alway s hold; the v ariability of the response will often
increase with increasing concentration.”

To perform this behav ior analy sis, the SD of the responses is com puted separately at each concentration (hence, true
replicates m ust be run). These v alues are plotted v ersus true concentration. A straight line (using, for exam ple, OLS) is
fitted through the points, resulting in an equation of the form (g + hx). Note that in this series of articles, the (a + bx)
form of the straight-line equation will be reserv ed for situations in which the instrum ental response is being plotted
v ersus true concentration. For m odeling standard dev iations (g + hx) will be used.

Associated with the slope (h) is a p-v alue, which allows the analy st to decide if the slope is significant. In general, if this p-
v alue is less than 1 % (possibly reported by software as 0.01 ), then the slope is considered to be significant and the
SD is declared to change with concentration.

Because OLS assum es a constant SD, all data points are allowed to influence the regression line equally ; in other words,
each point carries a “weight” of 1 . Howev er, if som e data are noisier than others (i.e., the standard dev iation is not
constant), then the m ore v ariable points should not be allowed to hav e as m uch influence. To incorporate this
nonconstant SD into the regression process, a generalization of OLS is used instead: weighted least squares (WLS).

As is indicated by the nam e, weighted least squares, weights are inv olv ed. Various form ulas hav e been used by different
authors to calculate these weights. Howev er, a robust procedure uses the form ula that results from fitting a SL to the SD
data (see abov e discussion). The basic equation for the weight is the reciprocal squared of the estim ated standard
dev iation:

weight @ x = (g + hx) –2

To enable the calculation of root m ean square error in original response units, this weight should be norm alized by
div iding by the m ean of all the reciprocal-squared v alues:

{(g + hx) –2 }/{Av g [(g + hx) –2 ]}

This form ula is ev aluated for each concentration and applied to the corresponding data to “weight” them . The result is
that the noisy responses hav e less influence on the calibration curv e than do the precise v alues.

While step 2 m ay seem a com putational annoy ance, ignoring a SD that changes with concentration will hav e two
negativ e results. First, the m odel’s coefficient estim ates (g and h) will be noisy . Second (and possibly m ore im portant),
the prediction interv al will be too wide in the well-behav ed-data region and too narrow in the noisy -data region.

Steps 3 through 7 will be discussed in the next two installm ents. Following the details of calibration diagnostics will be a
series of articles using these procedures to diagnose real calibration data sets.

Mr. Coleman is an Applied Statistician, Alcoa Technical Center, MST-C, 100 Technical Dr., Alcoa Center, PA 15069, U.S.A.;
e-mail: david.coleman@alcoa.com. Ms. Vanatta is an Analytical Chemist, Air Liquide-Balazs™ Analytical Services, Box
650311, MS 301, Dallas, TX 75265, U.S.A.; tel: 972-995-7541; fax: 972-995-3204; e-mail: lynn.vanatta@airliquide.com.

http://www.americanlaboratory.com/913-Technical-Articles/1703-Statistics-in-Analytical-Chemistry-Part-8-Calibration-Diagnostics/ 2/2