Statistical Data

Treatment and
Evaluation contd
Lecture 4
Nykieta James

Q-test

Outliers are the result of gross errors.

When a set of data contains an outlying result that differ

significantly from the mean, the decision must be made
whether to retain or reject it and this decision can be made
based on the results obtained from the Q-test.

In this test, the value of the difference between the

questionable result xq and its nearest neighbour xn is
divided by the spread w of the entire set to give the quantity
Qexp

Qexp=| xq - xn| / w

Detection of Gross Errors

Asetofresultsmaycontainanoutlyingresult
outoflinewiththeothers.
Shoulditberetainedorrejected?
Thereisnouniversalcriterionfordecidingthis.
OnerulethatcangiveguidanceistheQtest.

Detection of gross errors: Rejection of aberrant data

: the Q-test
Q = gap / range
If Qobserved > Q critical, discard the questionable point

Critical Values for the Rejection Quotient, Q*

Qcrit (Reject if Q> Q crit)
Number of
Observations

90% confidence

95% confidence

99%confidence

0.941

0.970

0.994

0.765

0.829

0.926

0.642

0.710

0.821

0.560

0.625

0.740

0.507

0.568

0.680

0.468

0.526

0.634

0.437

0.493

0.598

10

0.412

0.466

0.568

Example :

gap =0.11

12.47 12.48 12.53 12.56

12.67

Range = 0.20
Q = 0.11 /0.20 = 0.55 < 0.64 (table value, =0.10)
12.67 should be retained.

Values of Q for rejection of data

Q (90% confidence)
Number of observation

0.94
3

0.76
4

0.64

0.56

0.51

0.47

0.44

0.41

10

QexpisthencomparedtoasetofvaluesQcrit:
Qcrit (reject if Qexpt > Qcrit)
No. of observations

90%

95%

99%

3
0.941
0.970
0.994
4
0.765
0.829
0.926
5
0.642
0.710
0.821
6
0.560
0.625
0.740
7
0.507
0.568
0.680
8
0.468
0.526
0.634
9
0.437
0.493
0.598
10
0.412
0.466
0.568
RejectionofoutlierrecommendedifQexp>Qcritforthedesiredconfidencelevel.
Note:1.

Thehighertheconfidencelevel,thelesslikelyis
rejectiontoberecommended.
2.Rejectionofoutlierscanhaveamarkedeffectonmean
andstandarddeviation,esp.whenthereareonlyafew
datapoints.Alwaystrytoobtainmoredata.
3.Ifoutliersaretoberetained,itisoftenbettertoreport
themedianvalueratherthanthemean.

Q Test for Rejection

of Outliers

The following values were obtained for

the concentration of nitrite ions in a sample
of river water: 0.403, 0.410, 0.401, 0.380 mg/l.
Should the last reading be rejected?

Qexp 0.380 0.401 ( 0.410 0.380) 0.7

But Qcrit = 0.829 (at 95% level) for 4 values
Therefore, Qexp < Qcrit, and we cannot reject the suspect value.
Suppose 3 further measurements taken, giving total values of:
0.403, 0.410, 0.401, 0.380, 0.400, 0.413, 0.411 mg/l. Should
0.380 still be retained?

Qexp 0.380 0.400 ( 0.413 0.380) 0.606

But Qcrit = 0.568 (at 95% level) for 7 values
Therefore, Qexp > Qcrit, and rejection of 0.380 is recommended.
But note that 5 times in 100 it will be wrong to reject this suspect value!
Also note that if 0.380 is retained, s = 0.011 mg/l, but if it is rejected,
s = 0.0056 mg/l, i.e. precision appears to be twice as good, just by
rejecting one value.

Dixons Q test for rejections of outliers.

A null hypothesis: the data are not significantly different.
Q10(expt) = (xn - xn-1)/(xn - x1).

(n = 3 7)

Q11(expt) = (xn - xn-1)/(xn - x2).

(n = 8 10)

Q21(expt) = (xn - xn-2)/(xn - x2).

(n = 11 13)

Q22(expt) = (xn - xn-2)/(xn - x3).

(n = 14 25)

Qcrit from table for the relevant n and CL

xn - the point being considered
xn-1 - the point closest to xn
xn-2 - the point next closest to xn, etc.
Sort the data and then apply the test (in ascending or descending order)

EXAMPLE 1
Cr(VI) determined by colourimetry gave the following data: 0.0893, 0.1439,
0.0809, 0.1035, 0.1042, 0.1062, 0.1037, 0.1034, 0.1073, 0.0968. Should
the smallest value be rejected? Should the largest be rejected?
a)

Sort the data: 0.0809, 0.0893, 0.0968, 0.1034, 0.1035, 0.1037, 0.1042,
0.1062, 0.1073, 0.1439.
xn = 0.0809, 10 datum points.
Q11(expt) = (xn - xn-1)/(xn - x2)
= (0.0809 - 0.0893)/(0.0809 - 0.1073) = 0.318.
Qcrit at the 95% confidence level = 0.477.
Qexpt < Qcrit retain the null hypothesis and retain the datum point.

b) For xn = 0.1439,

xn-1 = 0.1073, x2 = 0.0893.

Qexpt = 0.670
Qcrit = 0.521 reject the point.

EXAMPLE 2