Validation of Test methods

General principles and concepts

(In cooperation with EAL, European
cooperation for
Accreditation of Laboratories)
EL1545/96
VALIDATIO O! TE"T #ET$OD"
GEE%AL &%I'I&LE" AD 'O'E&T"
December 1996
2
!O%E(O%D
EAL and Eurolab have a Permanent Liaison Group PLG!" #hich is a
$orum #here EAL and Eurolab are discussin% matters o$ mutual
interest& 'he PLG consists o$ (ve members $rom each or%ani)ation&
'his document has been prepared in the PLG and endorsed b* both
or%ani)ations&
'he document is intended to %ive %eneral vie#s on certain issues
related to the validation o$ test methods and should be seen as a
common understandin% and position o$ EAL and Eurolab& +n order to
de(ne and describe the activities behind the concept ,-alidation o$ test
methods, more detailed %uidance documents are needed& 'his
document should be seen as a basis $or such %uidance documents&
.
VALIDATIO O! TE"T #ET$OD"
)* Introduction
'he de(nition used $or /validation/ in the +01 standard 2432 is
/con(rmation b* e4amination and provision o$ ob5ective evidence that
the particular re6uirements $or a speci(c intended use are $ul(lled/&
'his de(nition %ives the impression o$ con(ned and #ell7de(ned
e4act! operations& 'est methods are normall* developed $or an
intended application ran%e& 'he re$erence to particular re6uirements
must in man* cases be interpreted in a 8e4ible #a* as the
re6uirements can be o$ %eneral nature&
9oth standardi)ed and non7standardi)ed test methods are covered&
'he* can be e4act or associated #ith lar%e uncertainties& Even novel
test methods #ill be considered& 'he validation o$ a test method
becomes in this conte4t a #a* o$ demonstratin% that the method is (t
$or its intended purpose& 'he (tness $or purpose includes an
assessment and a balancin% o$ technolo%ical possibilities" ris:s and
costs&
'here are ver* $e# papers in the open literature dealin% #ith the
%eneral principles o$ test method validation& 1n the other hand" a lot o$
detailed descriptions o$ the validation o$ speci(c test methods are
available& A brie$ overvie# o$ the concepts" aims and procedures in
validation is %iven in this document&
+* General principles to be used in ,alidation
+n the validation process an estimate is made o$ the
representativeness" repeatabilit* and reproducibilit* o$ the test method& 'he
de(nitions are %iven in anne4 1&
+n the validation process the ultimate aim is to secure that the test
methods are %ood enou%h #ith respect to representativeness"
reproducibilit* and repeatabilit*& ;o# much e<ort should be spent on
validation must be decided on a case b* case basis& +$ lar%e economic
values as #ell as considerable health" sa$et* and environmental issues
are involved" much more emphasis must be paid to the validation o$
the test methods& 'he $re6uenc* o$ use o$ the test method should also
be considered #hen determinin% the e4tent o$ validation& 'he total
conse6uences o$ #ron% results are o$ course lar%er $or methods in
e4tensive use than $or test methods used occasionall*&
'he validation o$ test methods covers to a lar%e e4tent the uncertaint*"
repeatabilit* and reproducibilit* o$ the test method& As the $actors
a<ectin% the results and contributin% most to the uncertaint* chan%e
$rom one technical sector to another or even $rom one test method to
4
another" a universal solution cannot be %iven& Guidance on the
e4pression o$ uncertainties can be $ound $or e4ample in the
international ,Guide to the e4pression o$ uncertaint* in measurement,
and EAL %uidance document ,E4pression o$ uncertaint* in 6uantitative
testin%,&
0tandardi)ed test methods should be considered validated $or their
intended application ran%e and thus %ood enou%h $or that purpose
althou%h their repeatabilit* and reproducibilit* are not :no#n in detail&
'he testin% laborator* must" ho#ever" chec: that the* appl* the
method correctl*& =or non7standardi)ed test methods it is up to the
testin% laboratories to determine ho# $ar the* %o in de(nin% the level
o$ repeatabilit* and reproducibilit*&
'o develop a representative test method" ade6uate :no#led%e is
re6uired o$ the practical use o$ the test results and o$ the real service
conditions o$ the ob5ect o$ the test& 9ased on such :no#led%e" the
/representative/ properties to be determined b* the test ma* be
identi(ed&
'he $actors a<ectin% the test results and their uncertaint* ma* be
%rouped into three main cate%ories>
+nstrumental and technical $actors
7 samplin%
7 homo%eneit*
7 test method
7 e6uipment
;uman $actors
Environmental $actors
7 testin% environment
Instrumental and technical factors are related to the
constructional and $unctional characteristics o$ the test and
measurement e6uipment" as #ell as to other technical operations
involved in the test e&%& samplin%" preparation o$ samples" test ob5ect
homo%eneit*!& 'heir e<ect ma* be minimi)ed and :ept under control
b* the $ollo#in% provisions>
7 de(ne the e6uipment as precisel* as necessar*
7 provide a clear description o$ the test procedure as #ell as the
e6uipment operation
7 establish procedures $or operational control and calibration
7 ensure #here applicable traceabilit* o$ measurements to the 0+
units&
?henever practical" the above provisions should be included in the
description o$ the test method& @e$erences to internal procedures or
applicable standards should be included&
5
$uman factors are related to the competence o$ the sta< and ma*
be controlled throu%h>
7 education/basic :no#led%e
7 on 5ob trainin%/practical e4perience&
'he 6uali(cation re6uired $or the personnel emplo*ed $or a %iven test
ma* be speci(ed in the test method or re$erence can be made to the
applicable internal procedures&
En,ironmental factors are associated to the environment #here the
test is per$ormed& Amon% others the e<ect o$ the $ollo#in% parameters
must be assessed and properl* controlled>
7 atmospheric conditions temperature" pressure" humidit*!
7 pollution/contamination
7 other environmental characteristics e&%& EAB!&
'he e<ect o$ the above parameters should be described in the test
method or re$erence to other applicable documents should be made&
;o#ever" $or ne# test methods this in$ormation is o$ten not available&
+n some cases the data base $or method validation is so lar%e that
statistical methods should be applied&
'he validation process must consider the e4pected or re6uired
uncertaint* o$ the test results and their intended use&
Britical threshold values e&%& in health and environment! cannot
%enerall* be technicall* 5usti(ed #ith a small uncertaint*& ;o#ever" i$ a
le%al limit is set" there must be test methods suited $or the purpose&
@e$erence is made to a recent +LAB Guide&
'he re6uired depth o$ the validation process depends also on the
maturit* o$ the test method and the prevalence o$ its use& 1ne can
distin%uish bet#een the $ollo#in% cate%ories>
7 novel methods
7 methods used b* several laboratories
7 modi(cation o$ established methods
7 standardi)ed methods&
'he #a*s" in #hich the validation is per$ormed in the di<erent cases"
need not be clearl* di<erentiated& +$ the (tness $or purpose concept is
maintained" it is o$ten possible to validate at reasonable cost but #ith a
hi%her de%ree o$ uncertaint*&
'he novel methods are (rst developed in one sin%le laborator*" o$ten
on the basis o$ a special re6uest $rom a customer or on ideas created in
the laborator*& 'hat customer cannot pa* $or a #ide ran%e validation
nor can the laborator* itsel$& 'he aim o$ the validation o$ test methods
must al#a*s be to demonstrate that the method is (t $or the intended
purpose and that the results have an acceptable uncertaint*& +t is
important that the rules o$ validation o$ test methods do not prevent
6
the natural technolo%ical development $rom ta:in% place& 'he
laborator* does not e4pect althou%h it does #ant! outside (nancial
help $or validation o$ novel methods and in man* cases tries to protect
its ne# development $rom %oin% to its competitors or $rom becomin%
%enerall* available to all&
?hen a certain number o$ laboratories #or: in the same area"
cooperation and inter7laborator* comparisons can be arran%ed& 'he
coordination o$ such activities is an e4tra economic burden& +n order to
speed up the process" e4ternal (nancin% is needed&
'he testin% laboratories need to update their e4istin% test methods&
'he 8e4ible scope o$ accreditation as a%reed bet#een EAL and Eurolab
#as also intended to allo# modi(cations to be made to accepted
accreditation covered! test methods& 'his re6uires validation
procedures applicable to method modi(cations& +t is up to the
laboratories to describe their procedures $or validatin% modi(ed test
methods&
'he most thorou%h validation procedure is re6uired $or test method
standardi)ation purposes& 'he #or: needed is considerable and covers
pro(cienc* testin%" the determination o$ $actors a<ectin% the
uncertaint*" measurin% ran%e" etc& 'he (nancial burden cannot be laid
on the laboratories but on the standardi)ation or%ani)ations&
0tandardi)ed test methods must be considered suCcientl* validated
$or their intended application ran%es& +$ the* are not" the* should be
#ithdra#n&
'he validation o$ test methods consists o$ t#o interrelated steps>
i! suitabilit* o$ the test to solve the problem customer
needs!
ii! demonstration o$ the technical capabilit* o$ the test
method #ithin the speci(ed test ran%e
i&e& measurin% the ri%ht properties #ith a suCcientl* reliable method&
'he suitabilit* or representativeness o$ a test method is in man* cases
an attribute #hich is diCcult to de(ne especiall* $or tests related to
product acceptance& 'he test methods must be such that the results
obtained correlate #ith the per$ormance characteristics and
operational e4perience o$ the product&
-* Validation procedure
9oth testin% laboratories and accreditation bodies are loo:in% $or
procedures and %uidelines $or plannin% and controllin% the test method
validation process&
;o#ever" the discussion above has clearl* indicated that one sin%le
procedure cannot be developed& Bonse6uentl*" a palette o$ di<erent
choices o$ validation techni6ues has to be developed& ;o# detailed the
validation #ill be" depends on the circumstances needs" costs"
possibilities" ris:s" etc&!&
D
'he validation o$ the test methods is" o$ course" o$ interest also to the
accreditation bodies& 'he principle to be applied should be that the
laborator* describes the #a* it is validatin% the test methods and the
accreditation bod* should ma:e the 5ud%ement i$ the procedure used is
acceptable in that case& 'he di<erent validation possibilities are built
up around
7 utili)ation o$ calibration
7 intercomparisons includin% the use o$ re$erence materials and
re$erence methods
7 #ell 6uali(ed sta< and their pro$essional 5ud%ement
7 simulation and modellin%
7 other approaches&
Aethod validation is o$ten based on the combined use o$ validation
procedures& 'he validation used can be /direct/ or comparative& 'he
selection o$ the validation procedures should also be 5usti(ed on a
cost7bene(t basis as lon% as the (tness7$or7purpose is maintained&
=ocusin% the e<ort on the most critical $actors a<ectin% the test
method #ill lead to a di<erent solution $or the validation o$ /e4act/
ph*sical and chemical test methods as compared to that $or product
or sub5ective testin%& =or e4ample" in the validation o$ er%onomics and
sensor* test methods not all possibilities are applicable&
As said above di<erent validation procedures ma* be $ollo#ed" their
e<ectiveness and applicabilit* dependin% on the t*pe o$ test
considered& 'he* can be characteri)ed as /scienti(c/ or
/comparative/>
2
."cienti/c approach.
+n the scienti(c approach the assessment o$ the representativeness"
repeatabilit* and reproducibilit* o$ the method is per$ormed #ith re$erence
to the di<erent constitutive elements and $eatures& Evidence should
describe the representativeness o$ the selected properties and the
associated uncertaint*& 'his can be based on in$ormation published in the
scienti(c and technical literature or on ad hoc investi%ations per$ormed b* the
laborator* developin% the method& 'he laborator* shall
demonstrate that relevant in8uencin% $actors instrumental and
technical" human" environmental! have been anal*)ed and that the*
are under control #ithin the uncertaint* associated #ith the
method&
.'omparati,e approach.
'he test method is assessed b* comparin% its results to those
obtained b* means o$ another alread* validated test method" #hich has
been developed $or the same purposes& +$ this is not possible" the
per$ormance characteristics o$ the method ma* be assessed throu%h
interlaborator* comparisons& 'he method is /valid/ i$ the results obtained b* the
di<erent laboratories $all #ithin the e4pected uncertaint* limit&
Deviations be*ond such limits ma* indicate e&%& a lac: o$ control o$ the
in8uencin% parameters& 'he causes o$ this behaviour should be clari(ed
and the method is to be rede(ned accordin%l*& 'he interlaborator*
comparison does not al#a*s provide a comprehensive validation o$ the
representativeness o$ the method" #hich ma* be accurate and stable"
thou%h ph*sicall* /#ron%/&
'he acceptance procedure $or ne# or modi(ed test methods is either
i! determined internall* in the laborator* ii! a%reed upon bet#een the
customer and the laborator* or iii! accepted b* the authorities and/or
accreditation bodies& A hi%her de%ree o$ reliance is needed #hen
sa$et*" health and lar%e economic values are involved& Balibration has
been emphasi)ed as an important element in the method validation
process" but it is not necessaril* the most dominatin% $actor& 'he
understandin% o$ the testin% method #ith its s*stematic and random
errors is crucial& A scienti(c approach to anal*)e sources o$ error as
#ell as the competence o$ the personnel doin% that 5ob is o$ %reat
importance&
'he laborator* should al#a*s describe the #a* the validation o$ test
methods is done and this description should be a part o$ the 6ualit*
s*stem/manual #hen appropriate&
As simpli(ed validation procedures /$ast/ validation methods! must be
used in man* cases" the capabilit* to use pro$essional 5ud%ement in
assessin% #hether the validation is comprehensive enou%h becomes
pronounced& ;o#ever" even #hen tal:in% about simpli(ed or $ast
validation procedures" the validation must be done #ith such a depth
that the method is (t $or the intended use and acceptable to the
9
customer and/or authorities& +t is clear that the de(nition o$ the use
and scope o$ the method and assumption o$ uncertaint* should not be
misleadin% and too optimistic&
?hen the use o$ ne# test methods becomes more e4tensive" #or:
describin% the e<ect o$ chan%es in test parameters can be initiated in
order to sho# the robustness o$ the method& Prenormative research
should also be initiated&
'he need $or ne# or improved test methods arises #hen #e lac:
methods or the e4istin% ones are not complete" %ood or eCcient
enou%h& 'here is no need $or the laborator* communit* to develop ne#
methods i$ e4istin% ones can be considered ade6uate&
13
AEEEF 1
DE!IITIO"
%epeatabilit0 (of results of measurement)
Bloseness o$ the a%reement bet#een the results o$ successive
measurements o$ the same measurand carried out under the same
conditions o$ measurement& -+A!
Eotes
1& 'hese conditions are called repeatabilit* conditions
2& @epeatabilit* conditions include>
7 the same measurement procedure
7 the same observer
7 the same measurin% instrument" used under the same
conditions
7 the same location
7 repetition over a short period o$ time
.& @epeatabilit* ma* be e4pressed 6uantitativel* in terms o$
the dispersion characteristics o$ the results&
%eproducibilit0 (of results of measurements)
Bloseness o$ the a%reement bet#een the results o$ measurements o$
the same measurand carried out under chan%ed conditions o$
measurement& -+A!
Eotes
1& A valid statement o$ reproducibilit* re6uires speci(cation o$
the conditions chan%es&
2& 'he chan%es conditions ma* include
7 principle o$ measurement
7 method o$ measurement
7 observer
7 measurin% instrument
7 re$erence standard
7 location
7 conditions o$ use
7 time&
.& @eproducibilit* ma* be e4pressed 6uantitativel* in terms o$
the dispersion characteristics o$ the results&
4& @esults are here usuall* understood to be corrected results&
11
1ncertaint0 (of measurement)
Parameter" associated #ith the result o$ a measurement" that
characteri)es the dispersion o$ the values that could reasonabl* be
attributed to the measurand& 9+PA/+EB/+=BB/+01/+GPAB/+GPAP/1+AL!&
Eotes
1& 'he parameter ma* be" $or e4ample" a standard deviation
or a %iven multiple o$ it!" or the #idth o$ a con(dence
interval&
2& Gncertaint* o$ measurement comprises" in %eneral" man*
components& 0ome o$ these components ma* be evaluated
$rom the statistical
distribution o$ the results o$ series o$ measurements and can
be
characteri)ed b* e4perimental standard deviations& 'he
other components" #hich can also be characteri)ed b*
standard deviations" are evaluated $rom assumed probabilit*
distributions based on e4perience or other in$ormation&
.& +t is understood that the result o$ the measurement is the
best estimate o$ the value o$ the measurand" and that all
components o$ uncertaint*" includin% those arisin% $rom
s*stematic e<ects" such as components associated #ith
corrections and re$erence standards" contribute to the
dispersion&
Validation
Bon(rmation b* e4amination and provision o$ ob5ective evidence that
the particular re6uirements $or a speci(c intended use are $ul(lled +01
2432!&
Eotes
1& +n desi%n and development" validation concerns the process
o$ e4aminin% a product to determine con$ormit* #ith user
needs&
2& -alidation is normall* per$ormed on the (nal product under
de(ned operatin% conditions& +t ma* be necessar* in earlier
sta%es&
.& 'he term /validated/ is used to desi%nate the correspondin%
status&
4& Aultiple validations ma* be carried out i$ there are di<erent
intended uses&
12
Veri/cation
Bon(rmation b* e4amination and provision o$ ob5ective evidence that
speci(ed re6uirements have been $ul(lled +01 2432!&
Eotes
1& +n desi%n and development" veri(cation concerns the
process o$ e4aminin% the result o$ a %iven activit* to
determine con$ormit* #ith the stated re6uirements $or that
activit*&
2& 'he term /veri(ed/ is used to desi%nate the correspondin%
status&
1.
AEEEF 2
%E!E%E'E" AD 2A'3G%O1D #ATE%IAL
+01 5D25 1926!> Precision o$ test methods 7 Determination o$
repeatabilit* and reproducibilit* $or a standard test method b* inter7
laborator* tests&
+01 2432 1994!> Hualit* mana%ement and 6ualit* assurance 7
vocabular*&
9+PA/+EB/+=BB/+01/+GPAB/+GPAP/1+AL 1995!> Guide to the e4pression
o$ uncertaint* in measurement&
EAL7G 2. 1996!> E4pression on uncertaint* in 6uantitative testin%&
EAL7G 12 1995!> 'raceabilit* o$ measurin% and test e6uipment to
national standards&
=orstIn Jarl 1991!> A vie# on the assessment o$ the technical
competence o$ testin% laboratories& Eordtest 'echnical @eport 149"
46p&
+LAB 1996!> Guide an assessment and reportin% o$ compliance #ith
speci(cation" based on measurements and tests in a laborator*&
-+A 199.!> +nternational vocabular* o$ basic and %eneral terms in
metrolo%*&
14
;>Kte:stit1KvalidatK.&12&1996
15