JOURNAL OF CLINICAL MICROBIOLOGY, Mar. 2003, p. 1080–1086 Vol. 41, No.

3
0095-1137/03/$08.00⫹0 DOI: 10.1128/JCM.41.3.1080–1086.2003
Copyright © 2003, American Society for Microbiology. All Rights Reserved.

International Proficiency Study of a Consensus L1 PCR Assay for the

Detection and Typing of Human Papillomavirus DNA: Evaluation
of Accuracy and Intralaboratory and Interlaboratory Agreement
Janet R. Kornegay,1 Michel Roger,2,3 Philip O. Davies,4 Amanda P. Shepard,1 Nayana A. Guerrero,5
Belen Lloveras,5 Darren Evans,4 and François Coutlée2,3*
Roche Molecular Systems Inc., Alameda, California1; Département de Microbiologie et Infectiologie, Hôpital Notre-Dame du
Centre Hospitalier de l’Université de Montréal,2 and Département de Microbiologie et Immunologie, Université de
Montréal,3 Montréal, Québec, Canada; BMI, London, United Kingdom4; and Department of Pathology,
CSU Bellvitge/Lab. Recerca Translacional, Institut Catala d’Oncologia,
Barcelona, Spain5
Received 24 May 2002/Returned for modification 26 August 2002/Accepted 9 December 2002

The PGMY L1 consensus primer pair combined with the line blot assay allows the detection of 27 genital
human papillomavirus (HPV) genotypes. We conducted an intralaboratory and interlaboratory agreement
study to assess the accuracy and reproducibility of PCR for HPV DNA detection and typing using the PGMY
primers and typing amplicons with the line blot (PGMY-LB) assay. A test panel of 109 samples consisting of
29 HPV-negative (10 buffer controls and 19 genital samples) and 80 HPV-positive samples (60 genital samples
and 20 controls with small or large amounts of HPV DNA plasmids) were tested blindly in triplicate by three
laboratories. Intralaboratory agreement ranged from 86 to 98% for HPV DNA detection. PGMY-LB assay
results for samples with a low copy number of HPV DNA were less reproducible. The rate of intralaboratory
agreement excluding negative results for HPV typing ranged from 78 to 96%. Interlaboratory reliability for
HPV DNA positivity and HPV typing was very good, with levels of agreement of >95% and kappa values of
>0.87. Again, low-copy-number samples were more prone to generating discrepant results. The accuracy varied
from 91 to 100% for HPV DNA positivity and from 90 to 100% for HPV typing. HPV testing can thus be
accomplished reliably with PCR by using a standardized written protocol and quality-controlled reagents. The
use of validated HPV DNA detection and typing assays demonstrating excellent interlaboratory agreement will
allow investigators to better compare results between epidemiological studies.

Human papillomavirus (HPV) infection is a strong indepen-
dent predictor for squamous intraepithelial lesions (SIL) and
invasive cancer of the uterine cervix (10, 34). Women with
persistent HPV infection are at highest risk for development of
cervical SIL or evolution from low-grade SIL to higher-grade
disease of the uterine cervix (8, 24). The causal association
between HPV and cervical cancer was demonstrated by epide-
miological investigations that used PCR, the most sensitive
tool for HPV detection and typing, in cells collected from the
uterine cervix (5).
Because of the genetic polymorphism of HPV, consensus
PCR assays have been utilized to amplify in one reaction the
majority of known, as well as novel, anogenital HPV geno-
types. The most widely used primer sets, the MY09/MY11/
HMB01, GP5⫹/GP6⫹, PGMY09/PGMY11, and SPF1/SPF2
consensus primers, target conserved sequences in the HPV L1
gene (1, 7, 12, 15, 18, 20, 23). To render the consensus PCR
assays more feasible for large-scale testing and clinical appli-
cation, convenient assays for the detection and typing of HPV
have been developed for all primer sets. HPV amplicons gen-
erated by PGMY primers can be easily detected and typed by
* Corresponding author. Mailing address: Département de Micro-
biologie et Infectiologie, Hôpital Notre-Dame du Centre Hospitalier
de l’Université de Montréal, 1560 Sherbrooke est, Montréal, Québec
H2L 4M1, Canada. Phone: (514) 890-8000, ext. 25162. Fax: (514)
412-7512. E-mail: francois.coutlee@ssss.gouv.qc.ca.
a nonisotopic reverse hybridization assay, the line blot (LB)
assay (4, 13). Recently, a colorimetric microtiter plate-based
enzyme immunoassay was also reported for screening of the
broad spectrum of HPV amplified by the PGMY primers using
a generic probe mix (21).
The proficiency of microbiology laboratory testing is gener-
ally monitored by proficiency-testing programs that also allow
the determination of test variability between laboratories (32).
Implementation of proficiency testing panels is essential for
unregulated molecular diagnostic tests. Proficiency panels have
been developed for the molecular diagnosis of several infec-
tious agents (25, 32) and in research settings to monitor the
performance of molecular virology laboratories (17). Thus far,
no proficiency panel has been constructed and made available
to the general research community for HPV testing. The va-
lidity of HPV DNA detection and typing with PCR assays has
not been thoroughly assessed.
Epidemiological studies and vaccine clinical trials require
the reliable and reproducible identification of genital HPV
infection. Several studies have evaluated the intermethod vari-
ation of HPV DNA detection (2, 11, 14, 19, 22, 26, 28, 30, 33).
However, few studies have evaluated the intralaboratory and
interlaboratory reproducibility of L1 consensus PCR assays
although these assays have been widely used (6, 16, 19). The
latter studies were conducted with in-house reagents and pro-
tocols. Although the consensus L1 PCR assays are not com-
mercially available, standardized reagents and protocols for

1080
VOL. 41, 2003 INTERLABORATORY STUDY OF A CONSENSUS L1 PCR FOR HPV
the PGMY-LB assay are available from Roche Molecular Sys-
tems for research purposes. The use of standardized and re-
producible protocols for HPV detection and typing could fa-
cilitate the comparison of results between studies on HPV
infection.
In order to assess the accuracy and reproducibility of the
PGMY-LB assay, three laboratories were invited to participate
in a collaborative study to compare their abilities to detect the
presence of and to genotype HPV DNA in blinded specimens.
We report here the estimate of intralaboratory and interlabo-
ratory reproducibility of the PGMY-LB assay for HPV DNA
detection and typing on 109 specimens tested in triplicate by
three independent laboratories under standardized conditions.
This information is useful in view of the wider use of the
PGMY-LB assay by numerous research groups, as well as its
potential application in diagnostic laboratories for HPV detec-
tion and typing.
MATERIALS AND METHODS
Selection of samples for the test panel. Roche Molecular Systems established
the test panel of samples with the PGMY-LB assay. The quality control set
included 30 synthetic samples: negative buffer samples devoid of HPV DNA
sequences (n ⫽ 10), samples containing 0.063 ␮g of human genomic DNA per 5
␮l (catalog no. 1691112; Roche Molecular Biochemicals, Indianapolis, Ind.)
spiked with 4,000 copies of HPV-16 DNA plasmid per 5 ␮l (n ⫽ 5), human DNA
spiked with 400 copies of HPV-16 DNA plasmid per 5 ␮l (n ⫽ 5), human DNA
spiked with 4,000 copies of HPV-45 DNA plasmid per 5 ␮l (n ⫽ 5), and human
DNA spiked with 400 copies of HPV-45 DNA plasmid per 5 ␮l (n ⫽ 5). The
panel also included 79 anonymous cervical samples collected with cytobrushes
from 79 women attending clinics at the BMI hospitals in London. The latter
samples came from a set of 90 samples that had been screened initially with the
PGMY-LB assay at the BMI Health Services Laboratory and stored frozen.
Roche Molecular Systems retested with the PGMY-LB assay the 90 clinical
samples in duplicate to ensure the presence of HPV genotypes detected initially.
Confirmation of initial PCR results was not obtained for 11 of the 90 samples (8
generated discordant results between PCR runs, 2 had been mislabeled, and 1
was only tested once at Roche), leaving 79 clinical samples (19 HPV negative and
60 HPV positive) in the panel.
Of the 109 samples included in the proficiency-testing panel, 29 were HPV-
negative specimens (10 buffer controls and 19 genital samples) and 80 samples
contained HPV DNA. The samples containing HPV DNA included 47 genital
samples with one HPV type, 10 synthetic samples with low HPV DNA copy
numbers, 10 synthetic samples with high HPV DNA copy numbers, 3 synthetic
samples containing multiple HPV types, and 10 genital samples containing, in
addition to the genotype(s) consistently detected in repeated PGMY-LB assay
runs, at least one type that was not consistently detected. The latter 10 samples
were tested four times by Roche Molecular Systems: an additional type was
detected in one out of four runs (two samples), two out of four runs (seven
samples), or three out of four runs (two samples). One sample contained two
HPV types that were inconsistently detected in the four runs. HPV genotypes
that were inconsistently detected in these 10 samples included types 42 (in three
samples), 31 and 58 (in two samples each), and 16, 35, 33, and 84 (in one sample
each).
In addition to the 10 samples containing HPV-16 plasmids and 10 samples
containing HPV-45 plasmids, there were a variety of HPV genotypes represented
among the 60 HPV-positive samples. They included types 16 (in 10 samples), 18
and 51 (in 5 samples each), 31, 45, 52, 59, and 66 (in 4 samples each), 33, 39, 42,
and 54 (in 3 samples each), 56, 68, 83, and 6 (in 2 samples each), and 35, 53, 58,
82, 84, and 73 (in 1 sample each). Of the three samples with multiple types, one
contained types 16 and 31, one contained types 16, 18, 52, and 73, and another
one contained types 16, 35, 42, and 82. The results from the detection of ␤-globin
were not considered and compared in this work.
Study design. Three laboratories with different levels of experience with mo-
lecular diagnostic methods participated in the study. Laboratory A had experi-
ence in human genetic testing but was unfamiliar with molecular virology tests.
Laboratory B was a diagnostic molecular microbiology laboratory specifically
trained to perform the PGMY-LB assay. Laboratory C was a diagnostic cytopa-
thology laboratory familiar with commercialized molecular diagnostic techniques
1081
but unfamiliar with the PGMY-LB assay. Standardized reagents comprising the
PCR master mix, LB strips, and reagents were sent to each participating center
along with a written standard operating procedure for the PGMY-LB assay.
Each participating center was asked to follow the protocol without modification.
Three 20-␮l aliquots from each panel sample were coded and distributed on
dry ice by Roche Molecular Systems to each of the three independent labora-
tories. In the test panel, the HPV-negative or buffer controls were distributed
randomly among the HPV-positive specimens. Five microliters of each sample
was used for HPV testing with the PGMY-LB assay. Each laboratory tested, in
different PCR runs, each of the 109 samples three times, blinded to the results
obtained from Roche Molecular Systems, from the other laboratories, and from
previous runs. Laboratories A and B also tested PGMY amplicons with the
generic probe microplate assay. The study testing was conducted between June
and August 2000.
PGMY-LB assay. HPV DNA was amplified in each center under standard
conditions with the L1 consensus HPV PGMY09/PGMY11 primer set, as pre-
viously described (12). The amplification mixture contained 4 mM MgCl2, 50
mM KCl, 7.5 U of AmpliTaq Gold DNA polymerase (Applied Biosystems), 200
␮M concentrations each of dATP, dCTP, and dGTP, 600 ␮M dUTP, 100 pmol
of each biotinylated PGMY primer pool, and 5 pmol each of the 5⬘-biotinylated
␤-globin primers GH20 and PC04. HPV was amplified with the ultrasensitive
profile that consisted of activation of AmpliTaq Gold at 95°C for 9 min, dena-
turation for 1 min at 95°C, annealing for 1 min at 55°C, and extension at 72°C for
1 min for a total of 40 cycles. Amplification was followed by a 5-min terminal
extension step at 72°C. Laboratory A used a Biometra Uno II thermocycler,
laboratory B used a TC 9600 thermocycler, and laboratory C used an MJ Re-
search PTC-1 thermocycler. Measures to avoid false-positive reactions due to
contamination were strictly adhered to. HPV genotyping was performed with the
reverse LB detection system as previously described (13). PCR products were
denatured in 0.4 N NaOH and hybridized to an immobilized probe array con-
taining probes for 27 HPV genotypes (types 6, 11, 16, 18, 26, 31, 33, 35, 39, 40,
42, 45, 51, 52, 53, 54, 55, 56, 57, 58, 59, 66, 68, 73, 82, 83, and 84). Positive
hybridization was detected by streptavidin-horseradish peroxidase-mediated
color precipitation on the membrane at the probe line.
Generic probe microplate assay. Generic probe detection reactions were per-
formed by using reagents from a PCR-ELISA DIG detection kit (Roche Mo-
lecular Biochemicals) as described previously (21). Twenty microliters of dena-
tured PCR products was added to the streptavidin-coated microtiter wells,
followed by the addition of 200 ␮l of hybridization buffer provided by Roche and
20 ␮l of the denatured generic probe pool. Roche synthesized the digoxigenin-
labeled generic probe pool by amplification of DNA from HPV types 11, 16, 18,
and 51 as described previously (21). Hybridization was performed at 37°C for 1 h.
Following color development, absorbance was measured at 405 nm and the
background, defined as the average value of blank cells containing no PCR
product, was subtracted from all values. A specimen was considered positive if
the corrected A405 was greater than 0.5, negative if the value was less than 0.2,
and indeterminate in the range between 0.2 and 0.499.
Statistical methods. To ensure an independent evaluation of the reproduc-
ibility of the assay, statistical analyses were performed by a scientist (F.C.)
without financial interests in Roche Molecular Systems. Results from the three
laboratories were imported into a common database for comparison (Microsoft
Excel). Results obtained by Roche Molecular Systems (the reference laboratory)
were considered the “gold standard” (HPV DNA reference standard). Agree-
ment for overall HPV positivity (HPV DNA positive irrespective of types iden-
tified) and for type-specific positivity was calculated as the percentage of runs
with identical results for the presence or absence of HPV. Cohen’s unweighted
kappa statistic was calculated to adjust for chance agreement between sites or
between sites and HPV DNA reference standard results (9). In general, a kappa
value of ⬎0.75 indicates excellent agreement beyond chance, a kappa value
between 0.40 and 0.75 indicates fair to good agreement, and a kappa value of
⬍0.40 represents poor agreement.
The reproducibility of repeated PCR assays (intralaboratory reproducibility)
was calculated for each site by comparing results from triplicate runs and calcu-
lating the crude percent agreement and the percent agreement considering only
HPV-positive results. The intralaboratory reproducibility for the presence or
absence of each of 27 genotypes was first calculated by using crude typing results.
Since 10 samples contained HPV types that were not consistently detected
between runs by the reference laboratory, agreement was recalculated by not
considering the presence or absence of these additional types (see above).
Interlaboratory agreement (interlaboratory reproducibility) was assessed by
pairwise comparisons of test results from the three laboratories calculating the
crude percent agreement and the kappa statistic. Results obtained by the refer-
ence laboratory were not considered in these comparisons. To avoid combining
1082 KORNEGAY ET AL.
TABLE 1. Intralaboratory reproducibility of PGMY-LB for HPV DNA detection and HPV typing on triplicate testing of
109 samples by three laboratories
No. of assays with following results in runs 1/2/3a:
Test and laboratory
⫹/⫹/⫹ ⫹/⫹/⫺
HPV DNA detectionc
A 65 11
B 77 3 0
C 78 2 0
HPV typingd
A 69 17 (14)e
B 90 5 (3)e
C 84 8 (2)e
a
Triplicate testing of 109 samples was done by each laboratory.
b
Two measurements of percent agreement were calculated: one in which HPV-positive and HPV-negative results were considered (all results) and one in which
specimens that had negative results in all triplicates were excluded.
c
Results for the detection of HPV DNA irrespective of types detected.
d
Typing results for 27 genotypes were considered.
e
In parentheses are numbers of assays not taking into consideration types that were inconsistently detected by the reference laboratory.
intralaboratory and interlaboratory variability in the comparisons of the three
laboratories, two strategies of analysis were considered. Laboratories were first
compared by considering the HPV DNA and typing results obtained on the
initial run for each sample in each laboratory. This strategy allows evaluation of
interlaboratory reproducibility when samples are only tested once, as is often the
case. Laboratories were also compared by using a consensus definition of HPV
results: the final HPV result for a specimen was that obtained from at least two
out of three runs for the presence of HPV DNA and HPV typing. This definition
favors a higher degree of concordance between laboratories but takes into
account triplicate results for each sample.
Accuracy of HPV DNA detection and HPV typing was assessed by pairwise
comparisons in contingency tables of each laboratory with the HPV DNA ref-
erence standard results, using the two-sided McNemar chi-square analysis for
matched-pair data. Agreement was determined first by considering results of the
first aliquot for each sample and then by considering the consensus HPV results
for each sample as explained above. Since the reference laboratory did not
consistently detect a type in 10 samples, agreement was recalculated by not
considering the presence or absence of these additional types as a discordant
result. Results from laboratories A and B for the generic probe microplate assay
were compared for HPV DNA positivity with results obtained with the LB assay.
Indeterminate results were excluded from the calculation of agreement.
RESULTS
Results from triplicate runs with the PGMY-LB assay were
compared in each laboratory to first assess the reproducibility
of HPV DNA detection irrespective of the type(s) detected.
The intralaboratory agreement for HPV DNA detection of the
PGMY-LB assay was high for each site (Table 1). The agree-
ment between triplicates for each center was lower when only
the HPV-positive results were considered. Two laboratories
had intralaboratory levels of agreement of ⬎97% for HPV
DNA detection when all results were considered. A significant
difference was found in the number of discordant triplicates for
HPV DNA detection among the three participating laborato-
ries (Table 1): 15 (13.8%), 3 (2.8%), and 2 (1.8%) of 109
samples generated discordant triplicates for HPV DNA in
laboratories A, B, and C, respectively (P ⫽ 0.005; Pearson
chi-square test). When laboratories were compared two by
two, significant differences were found, after the Bonferroni
correction for a total of three comparisons, in the proportion
of discordant triplicates between laboratories A and B and
between laboratories A and C (P ⫽ 0.009 and 0.003, respec-
tively; Pearson chi-square test). The difference in the number
J. CLIN. MICROBIOL.
Agreement (%)b
HPV⫹
⫹/⫺/⫺ ⫺/⫺/⫺ All results
results
4 29 86.2 81.3
29 97.3 96.3
29 98.2 97.5
6 (5)e 2,858 99.2 (99.4)e 75.0 (78.4)e
3 (1)e 2,845 99.7 (99.9)e 91.8 (95.8)e
6 (5)e 2,845 99.5 (99.8)e 85.7 (92.3)e
of discordant triplicates between laboratories B and C did not
reach statistical significance (P ⫽ 0.65; Pearson chi-square
test).
An intralaboratory agreement analysis stratified by the
amount of HPV DNA introduced in the sample for laboratory
A that had the highest number of discordant triplicates re-
vealed that the presence of a low copy number of HPV DNA
had an influence on reproducibility. In this laboratory, 7
(70.0%) of the 10 samples containing small amounts of HPV
DNA generated discordant triplicates for HPV DNA detec-
tion, as opposed to 8 of the 70 samples with clear signals in the
PGMY-LB assay or high copy numbers (P ⫽ 0.002; Fisher’s
exact test).
In order to estimate the intralaboratory agreement for HPV
DNA typing, we compared the results of triplicate testing of
109 samples for 27 genotypes (2,943 type-specific results per
center) (Table 1). Although the number of discordant tripli-
cates was greater for typing results than HPV DNA detection
in all laboratories, agreement reached at least 99% when all
specimens were considered (Table 1). The high number of
HPV type-specific negative results explains the greater intral-
aboratory agreement obtained with HPV typing results than
with generic HPV DNA detection despite a greater number of
discordant triplicates. When concordant negative triplicates
were excluded from the analysis, the level of agreement de-
creased as shown in Table 1. We then considered in our eval-
uation the fact that 10 samples contained HPV types that were
not consistently detected between runs by the reference labo-
ratory. In laboratory C, 7 (50%) of the 14 discordant triplicates
contained HPV types not uniformly detected by the reference
laboratory, thus leaving only 7 truly discordant triplicates for
HPV typing (values in parentheses in Table 1). In laboratory B,
four (50%) of the eight discordant triplicates contained HPV
types not uniformly detected by the reference laboratory, leav-
ing only four truly discordant triplicates for HPV typing. In
laboratory A, 4 (17.3%) of the 23 discordant triplicates con-
tained HPV types not uniformly detected by the reference
laboratory, leaving 19 truly discordant triplicates for HPV typ-
ing. When only truly discordant triplicates were considered,
the intralaboratory agreement for HPV typing increased
VOL. 41, 2003 INTERLABORATORY STUDY OF A CONSENSUS L1 PCR FOR HPV 1083

TABLE 2. Interlaboratory agreement of PGMY-LB results for TABLE 3. Interlaboratory agreement of PGMY-LB results for
HPV DNA detection and HPV typing obtained on initial testing of HPV DNA detection and HPV typing obtained in three laboratories
109 samples by three laboratories on triplicate testings of 109 samples using the consensus definition
for HPV results
No. of concordant
results % Agreement No. of concordant
Test and Kappa
(no. identical/ results % Agreement
laboratory pair HPV⫹ HPV⫺ value Test and Kappa
no. tested) (no. identical/
(n ⫽ 80) (n ⫽ 29) laboratory pair HPV ⫹
HPV ⫺ value
no. tested)
(n ⫽ 80) (n ⫽ 29)
HPV DNA
detection HPV DNA
A-B 73 29 94 (102/109) 0.85 detection
B-C 79 29 99 (108/109) 0.98 A-B 76 29 96 (105/109) 0.91
A-C 72 29 93 (101/109) 0.83 B-C 80 29 100 (109/109) 1.00
A-C 76 29 96 (105/109) 0.91
HPV typing
A-B 68 29 95 (97/109) 0.75 HPV typing
B-C 71 29 93 (101/109) 0.81 A-B 75 29 95 (104/109) 0.87
A-C 69 29 90 (98/109) 0.78 B-C 80 29 100 (109/109) 1.00
A-C 76 29 96 (105/109) 0.91

slightly when all results were included in the calculation of

agreement and ranged from 78.4 to 95.8% when only HPV-
positive results were used (Table 1).
When the numbers of truly discordant triplicates for HPV
DNA typing were compared between participating sites, sig-
nificant differences were found, after the Bonferroni correction
for a total of three comparisons, between laboratories A and B
and between laboratories A and C (P ⫽ 0.006 and 0.054,
respectively; Pearson chi-square test). Considering typing re-
sults from laboratory A that showed the greatest intralabora-
tory variability, 8 of 10 samples with low copy numbers of HPV
DNA generated discordant triplicates, as opposed to 15 of 70
samples with clear signals or high HPV copy numbers (P ⫽
0.005; Fisher’s exact test). This relationship was not found for
laboratories B and C, which had a higher level of agreement
between triplicates (data not shown). Excluding specimens
with low copy numbers of HPV types 45 and 16, discordant
triplicates were distributed equally across several types. Repro-
ducibility was perfect for the three samples with multiple HPV
type infections for two laboratories (data not shown). One
laboratory failed to detect an HPV type in one of the triplicates
for two of the latter samples (data not shown).
Interlaboratory reproducibility of the PGMY-LB assay was
first assessed by considering only the first PCR run in each
laboratory. Laboratory pairwise comparisons are shown in Ta-
ble 2 for HPV DNA detection and typing. Agreement for HPV
DNA positivity between laboratory pairs A and B, B and C,
and A and C reached 94% (102 of 109 samples, kappa value of
0.85), 99% (108 of 109 samples, kappa value of 0.98), and 93%
(101 of 109 samples, kappa value of 0.83), respectively. Agree-
ment for HPV typing results between laboratory pairs A and B,
B and C, and A and C reached 94% (102 of 109 samples, kappa
value of 0.81), 97% (106 of 109 samples, kappa value of 0.88),
and 93% (101 of 109 samples, kappa value of 0.80), respec-
tively.
Interlaboratory reproducibility of the PGMY-LB assay was
then assessed by using a consensus definition of HPV positivity
(two of three runs with identical results), as explained in Ma-
terials and Methods. Laboratory pairwise comparisons are
shown in Table 3 for HPV DNA detection and typing. Inter-
laboratory reliability for HPV DNA positivity and HPV typing
was excellent, with levels of agreement of ⬎95% and kappa
values of ⬎0.87, including perfect kappa values for HPV DNA
detection and HPV typing between laboratories B and C. Of
the four samples testing negative in laboratory A but positive
in the two other laboratories, two contained low copy numbers
of HPV-16 or -45. Low-copy-number samples were more
prone to generate discordant results in laboratory A, since 2
(20%) of 10 low-copy-number samples versus 2 (3%) of 70
other HPV-positive samples were misclassified by laboratory A
(P ⫽ 0.07; Fisher’s exact test).
The accuracy of the PGMY-LB assay was measured by com-
paring results obtained from each laboratory with those ob-
tained at Roche Molecular Systems (HPV DNA reference
standard) for 109 samples. Additional HPV types inconsis-
tently detected by Roche in multiple testing in 10 samples were
not considered in this analysis. For the 29 HPV-negative sam-
ples, only 2 (0.8%) of 261 assays scored positive for HPV
DNA. One specimen that tested falsely positive for HPV-51
was preceded by an HPV-51-positive sample, while HPV-31
was responsible for the other false-positive result. Since sam-
ples were tested in triplicate, results from the first run and from
a consensus definition of HPV positivity were compared sep-
arately to the HPV DNA reference standard results, as shown
in Table 4. The percentages of agreement between each labo-
ratory and the HPV DNA reference standard varied from 91 to
100% for HPV DNA positivity and from 90 to 100% for HPV
typing. Laboratory B correctly identified all HPV-positive sam-
ples using results from the first assay or the consensus defini-
tion of HPV positivity. HPV types were identified correctly in
all samples when the consensus definition of HPV positivity
was used and in 79 (98.8%) samples when the first run only was
considered. Similarly, laboratory C correctly identified 99% of
HPV-positive samples and correctly identified HPV types for
nearly all samples except three (3.8%) and one (1.3%) when
the first run and the consensus definition of HPV positivity
were used, respectively. When the first sample was considered,
laboratory A failed to identify 7 (9%) of 80 HPV-positive
samples and failed in HPV typing for 8 (10%) for 80 HPV-
positive samples. Four (57.1%) of the seven HPV-positive
samples that were misclassified by laboratory A in the first run
contained low copy numbers of HPV DNA (data not shown).
The agreement for HPV-positive samples, which can be con-
1084 KORNEGAY ET AL.
TABLE 4. Accuracy of HPV DNA detection and typing results obtained with PGMY-LB for 109 samples by three laboratories
Laboratory and HPV
No. of samples
standard
1st assay
A
HPV positive 80 73 (91)
HPV negative 29 29 (100)
All samples 109 102 (94)
B
HPV positive 80 80 (100)
HPV negative 29 29 (100)
All samples 109 109 (100)
C
HPV positive 80 79 (99)
HPV negative 29 29 (100)
All samples 109 108 (99)
sidered the sensitivity of the PGMY-LB assay compared to the
HPV DNA reference standard, ranged from 95 to 100% for
HPV DNA detection and typing using the consensus definition
of HPV results and from 90 to 100% using results from the first
test (Table 4). Samples containing multiple HPV types or with
high copy numbers of HPV DNA were all appropriately clas-
sified by each site (data not shown).
In laboratories A and B, amplicons generated by PGMY
primers were tested with the LB assay and also with the generic
probe microplate assay. Since two laboratories tested 109 sam-
ples in triplicate, 654 assay results could be compared. Inde-
terminate results for the generic probe microplate assay were
obtained for 59 samples (30 HPV-positive samples and 29
HPV-negative samples). When these indeterminate results
were excluded, 420 assays were HPV positive by both tests, 149
were negative by both tests, 11 were HPV positive with the
generic probe microplate assay only, and 15 were HPV-posi-
tive with the LB assay only. The agreement between the tests
for detection of HPV amplicons was 95.6% (569 of 595 re-
sults), for an excellent kappa value of 0.89.
DISCUSSION
Interlaboratory agreement is a concern for any HPV DNA
detection method that has not undergone extensive compara-
tive field testing. The HPV detection and typing method also
has to be accurate. We evaluated the intralaboratory and in-
terlaboratory reproducibility of the PGMY-LB assay per-
formed by three laboratories with different levels of experience
with PCR testing. Our study demonstrates the high reliability
of the PGMY-LB assay when a laboratory is well trained to
perform the assay. Laboratories without experience with in-
house PCR tests can also use the PGMY-LB assay, since lab-
oratory C generated reproducible and accurate results. The
use of standardized reagents and clearly written protocols was
instrumental in obtaining these excellent results with the
PGMY-LB assay. The lowest intralaboratory, interlaboratory,
and accuracy levels were found in a laboratory with experience
with PCR testing. That laboratory had determined in previous
experiments that the PGMY-LB assay with their thermocycler
J. CLIN. MICROBIOL.
No. of correct PGMY-LB results (% accuracy)
HPV DNA detection HPV typing
Consensus 1st assay Consensus
76 (95) 72 (90) 76 (95)
29 (100) 29 (100) 29 (100)
105 (96) 101 (93) 105 (96)
80 (100) 79 (99) 80 (100)
29 (100) 29 (100) 29 (100)
109 (100) 108 (99) 109 (100)
80 (100) 77 (96) 79 (99)
29 (100) 29 (100) 29 (100)
109 (100) 106 (97) 108 (99)
required higher concentrations of ␤-globin primers in the PCR
master mix (data not shown). It was demonstrated previously
that coamplification of ␤-globin with HPV DNA can result in
competition and impede PGMY-LB assay performance, ex-
plaining in part the discordant results obtained with low-copy-
number synthetic samples (3, 4). The importance of utilizing
the same instruments and parameters set by validation studies
of a test is emphasized by these results. Each of the three
laboratories used a different make and model of thermocycler,
while the PGMY-LB assay protocol was developed and opti-
mized for only one, the TC 9600 from Perkin-Elmer. As an
alternative explanation, specimen heterogeneity may have ac-
counted for the difference in performance between laborato-
ries.
We demonstrated on synthetic samples that specimens with
small amounts of HPV DNA were prone to generate discor-
dant results between triplicates. Since synthetic samples do not
contain the amplification modifier found in clinical specimens,
we also included in the proficiency panel anonymous cytobrush
samples. Ten clinical samples contained at least one HPV type
not consistently detected by the reference laboratory. For all of
these samples, the type detected intermittently by the refer-
ence laboratory was also detected by at least one of the par-
ticipating laboratories. This phenomenon could result from the
presence of a low copy number of HPV DNA or inhibitors and
illustrates the difficulty of using clinical samples in a proficiency
test.
The intralaboratory agreement was excellent for all three
laboratories. It increased when we did not consider variable
HPV results obtained in the certification panel by the refer-
ence laboratory. Some groups have reported greater reproduc-
ibility of PCR for HPV testing (22, 27). In those studies, du-
plicates instead of triplicates were tested, only one laboratory
was evaluated, and the panel included more HPV-negative
samples than did our study. Our level of intralaboratory agree-
ment for HPV DNA detection and typing corresponds to a
previous evaluation by Daniel et al. (6). They also reported
that the reproducibility of HPV typing with MY09/MY11 in-
creased in the presence of greater HPV loads. We found that
for the laboratory with several discordant triplicates, low-viral-
VOL. 41, 2003 INTERLABORATORY STUDY OF A CONSENSUS L1 PCR FOR HPV
load synthetic samples were prone to generating discordant
results. As reported by others, we found a lower agreement for
HPV typing than for HPV DNA detection (6, 19). We calcu-
lated not only overall agreement but also agreement with re-
gard only to HPV-positive results because of the large number
of concordant HPV-negative assays that artificially increased
the level of agreement for HPV typing. As previously reported,
agreement was strong for samples with multiple HPV types,
although we only analyzed three such specimens (6, 19).
Two studies reported the interlaboratory agreement of PCR
for HPV detection using the MY09/MY11 primer pair (16, 22).
In one study, 33 samples were tested in two laboratories, while
in the other study, 70 anal samples were analyzed in two lab-
oratories. Our evaluation of agreement was consistent with the
estimates from these previous reports for HPV DNA positivity
(83 and 96%) and HPV typing (90 and 88%).
The accuracy of the PGMY-LB assay was excellent. How-
ever, specimens may better agree with a reference standard
when the same test is used to validate and test the panel. The
low rate of false-positive results in the 29 HPV-negative sam-
ples suggests a low rate of contamination in our study. One of
the false-positive specimens was preceded by an HPV-51-pos-
itive sample. Contamination may have occurred after amplifi-
cation during the hybridization of amplicons with the LBs,
since the same amplicons hybridized with the generic probe
tested negative. Another study evaluated the accuracy of the
GP5⫹/GP6⫹ L1 consensus primer pair and reported excellent
accuracy for 50 samples tested in four laboratories, ranging
from 92 to 100% for detection of HPV DNA, a rate similar to
our results (19). That study also reported that most discrepant
results were false-negative results from HPV-positive samples
(19). The reproducibility of the PGMY-LB assay in our study
was also similar to that of the FDA-approved hybrid capture
assay from Digene (29). The accuracy of hybrid capture in
detecting high-risk HPV types ranged for three laboratories
from 88 to 92%, and interlaboratory agreement reached 90%.
Nonetheless, our results should be interpreted cautiously,
with the following caveats. Few samples with multiple type
infections were included in our evaluation. These samples have
been demonstrated to generate intralaboratory variability in
one study (6). Future evaluations of the PGMY-LB assay
should include multiple type infections with some types in
small amounts. Our clinical samples were obtained without
knowledge of cytopathological diagnoses. Assay reproducibil-
ity and accuracy may differ when samples from different pa-
tient populations are selected, especially since viral load is
related to cervical disease status and assay reproducibility (31).
We demonstrated here that there is a very strong interlabo-
ratory agreement when the PGMY-LB assay is used. HPV
testing can thus be reliably accomplished with PCR by using a
validated and clearly written protocol with quality-controlled
reagents. Although the assay procedure is well standardized,
an initial certification panel to ensure the reliability of results
would be desirable before a laboratory performs the
PGMY-LB assay on a regular basis. Our study stresses the
importance of using standardized reagents and protocols and
the establishment of proficiency panels for the comparability of
results between studies. More studies on performance stan-
dards that include samples with multiple type infections, with
inhibitors and low viral load infections, need to be performed.
1085
ACKNOWLEDGMENTS
We thank Diane Gaudreault and France Dion for testing samples
with the PGMY-LB assay, Pierre Forest for training participants, and
BMI Health Services for the provision of clinical samples. Roche
Molecular Systems supplied reagents for the PGMY-LB assay.
F.C. and M.R. hold a clinical career award supported by the Fonds
de Recherche en Santé du Québec.
REFERENCES
1. Bauer, H. M., Y. Ting, C. E. Greer, J. C. Chambers, C. J. Tashiro, J. Chi-
mera, A. Reingold, and M. M. Manos. 1991. Genital human papillomavirus
infection in female university students as determined by a PCR-based
method. JAMA 265:472–477.
2. Cope, J. U., A. Hildesheim, M. H. Schiffman, M. M. Manos, A. T. Lorincz,
R. D. Burk, A. G. Glass, C. Greer, J. Buckland, K. Helgesen, D. R. Scott,
M. E. Sherman, R. J. Kurman, and K. L. Liaw. 1997. Comparison of the
Hybrid Capture tube test and PCR for detection of human papillomavirus
DNA in cervical specimens. J. Clin. Microbiol. 35:2262–2265.
3. Coutlée, F., P. Gravitt, J. Kornegay, C. Hankins, H. Richardson, N.
Lapointe, H. Voyer, The Canadian Women’s HIV study Group, and E.
Franco. 2002. Use of PGMY primers in L1 consensus PCR improves detec-
tion of human papillomavirus DNA in genital samples. J. Clin. Microbiol.
40:902–907.
4. Coutlée, F., P. Gravitt, H. Richardson, C. Hankins, E. Franco, N. Lapointe,
H. Voyer, and The Canadian Women’s HIV Study Group. 1999. Nonisotopic
detection and typing of human papillomavirus DNA in genital samples by
the line blot assay. J. Clin. Microbiol. 37:1852–1857.
5. Coutlée, F., M. H. Mayrand, D. Provencher, and E. Franco. 1997. The future
of HPV testing in clinical laboratories and applied virology research. Clin.
Diagn. Virol. 8:123–141.
6. Daniel, R. W., L. Ahdieh, D. Hayden, S. Cu-Uvin, and K. V. Shah. 2000.
Intra-laboratory reproducibility of human papillomavirus identification in
cervical specimens by a polymerase chain reaction-based assay. J. Clin. Virol.
19:187–193.
7. de Roda, H., J. M. Walboomers, A. J. van den Brule, C. J. Meijer, and P. J.
Snijders. 1995. The use of general primers GP5 and GP6 elongated at their
3⬘ ends with adjacent highly conserved sequences improves human papillo-
mavirus detection by PCR. J. Gen. Virol. 76:1057–1062.
8. Ferenczy, A., and E. L. Franco. 2002. Persistent human papillomavirus in-
fection and cervical neoplasia. Lancet Oncol. 3:11–16.
9. Fleiss, J. L. 1981. Statistical methods for rates and proportions. John Wiley
and Sons Inc., New York, N.Y.
10. Franco, E. L. F. 1996. Epidemiology of anogenital warts and cancer, p.
597–623. In R. Reid and A. T. Lorincz (ed.), Human papillomavirus I. W. B.
Saunders, Philadelphia, Pa.
11. Gravitt, P., A. Hakenewerth, and J. Stoerker. 1991. A direct comparison of
methods proposed for use in widespread screening of human papillomavirus
infection. Mol. Cell. Probes 5:65–72.
12. Gravitt, P., C. L. Peyton, T. Q. Alessi, C. Wheeler, F. Coutlée, A. Hildesheim,
M. Schiffman, D. R. Scott, and R. J. Apple. 2000. Improved amplification of
genital human papillomaviruses. J. Clin. Microbiol. 38:357–361.
13. Gravitt, P., C. L. Peyton, R. J. Apple, and C. Wheeler. 1998. Genotyping of
27 human papillomavirus types by using L1 consensus PCR products by
single-hybridization, reverse line blot detection method. J. Clin. Microbiol.
36:3020–3027.
14. Guerrero, E., R. W. Daniel, F. X. Bosch, X. Castellsagué, N. Munoz, M. Gili,
P. Viladu, C. Navarro, M. L. Zubiri, N. Ascunce, L. C. Gonzales, L. Tafur, I.
Izaraugaza, and K. V. Shah. 1992. Comparison of Virapap, Southern blot
hybridization, and polymerase chain reaction methods for human papillo-
mavirus identification in an epidemiological investigation of cervical cancer.
J. Clin. Microbiol. 30:2951–2959.
15. Hildesheim, A., M. H. Schiffman, P. E. Gravitt, A. G. Glass, C. E. Greer, T.
Zhang, D. R. Scott, B. B. Rush, P. Lawler, M. E. Sherman, R. J. Kurman,
and M. M. Manos. 1994. Persistence of type-specific human papillomavirus
infection among cytologically normal women. J. Infect. Dis. 169:235–240.
16. Hsing, A. W., R. D. Burk, K. L. Liaw, C. J. Chen, T. Zhang, M. Schiffman,
C. E. Greer, S. L. You, C. Y. Hsieh, T. W. Huang, T. C. Wu, T. J. O’Leary,
J. D. Seidman, and M. M. Manos. 1996. Interlaboratory agreement in a
polymerase chain reaction-based human papillomavirus DNA assay. Cancer
Epidemiol. Biomed. Prev. 5:483–484.
17. Jackson, J. B., J. Drew, H. J. Lin, P. Otto, J. W. Bremer, F. B. Hollinger,
S. M. Wolinski, The ACTG PCR Working Group, and The ACTG PCR
Virology Laboratories. 1993. Establishment of a quality assurance program
for human immunodeficiency virus type 1 DNA polymerase chain reaction
assays by the AIDS clinical trials group. J. Clin. Microbiol. 31:3123–3128.
18. Jacobs, M. V., P. J. F. Snijders, A. J. C. van den Brule, T. J. M. Helmerhorst,
C. Meijer, and J. Walboomers. 1997. A general primer GP5⫹/GP6⫹-medi-
ated PCR-enzyme immunoassay method for rapid detection of 14 high-risk
and 6 low-risk human papillomavirus genotypes in cervical scrapings. J. Clin.
Microbiol. 35:791–795.
1086 KORNEGAY ET AL.
19. Jacobs, M. V., P. J. F. Snijders, F. J. Voorhorst, J. Dillner, O. Forslund, B.
Johansson, M. V. K. Doeberitz, C. J. L. M. Meijer, T. Meyer, I. Nindl, H.
Pfister, E. Stockfleth, A. Strand, G. Wadell, and J. M. M. Walboomers. 1999.
Reliable high risk HPV DNA testing by polymerase chain reaction: an
intermethod and intramethod comparison. J. Clin. Pathol. 52:790.
20. Kleter, B., L. J. van Doorn, L. Schrauwen, A. Molijn, S. Sastrowijoto, J. ter
Schegget, J. Lindeman, B. ter Harmsel, M. Burger, and W. Quint. 1999.
Development and clinical evaluation of a highly sensitive PCR-reverse hy-
bridization line probe assay for detection and identification of anogenital
human papillomavirus. J. Clin. Microbiol. 37:2508–2517.
21. Kornegay, J. R., A. P. Shepard, C. Hankins, E. Franco, N. Lapointe, H.
Richardson, The Canadian Women’s HIV Study Group, and F. Coutlée.
2001. Nonisotopic detection of human papillomavirus DNA in clinical spec-
imens using a consensus PCR and a generic probe mix in an enzyme-linked
immunoassay format. J. Clin. Microbiol. 39:3530–3536.
22. Kuypers, J. M., C. W. Critchlow, P. E. Gravitt, D. A. Vernon, J. B. Sayer,
M. M. Manos, and N. B. Kiviat. 1993. Comparison of dot filter hybridization,
Southern transfer hybridization, and polymerase chain reaction amplification
for diagnosis of anal human papillomavirus infection. J. Clin. Microbiol.
31:1003–1006.
23. Manos, M. M., Y. Ting, D. K. Wright, A. J. Lewis, T. Broker, and S. M.
Wolinski. 1989. Use of the polymerase chain reaction amplification for the
detection of genital human papillomaviruses. Cancer Cells 7:209–215.
24. Nobbenhuis, M. A., J. M. Walboomers, T. J. Helmerhorst, L. Rozendaal,
A. J. Remmink, E. K. Risse, H. C. van der Linden, F. J. Voorhorst, P.
Kenemans, and C. J. Meijer. 1999. Relation of human papillomavirus status
to cervical lesions and consequences for cervical-cancer screening: a pro-
spective study. Lancet 354:20–25.
25. Noordhoek, G. T., A. H. Kolk, G. Bjune, D. Catty, J. W. Dale, P. E. Fine, P.
Godfrey-Faussett, S. N. Cho, T. Shinnick, and S. B. Svenson. 1994. Sensi-
tivity and specificity of PCR for detection of Mycobacterium tuberculosis: a
blind comparison study among seven laboratories. J. Clin. Microbiol. 32:277–
284.
J. CLIN. MICROBIOL.
26. Peyton, C. L., M. Schiffman, A. T. Lorincz, W. C. Hunt, I. Mielzynska, C.
Bratti, S. Eaton, A. Hildesheim, L. A. Morera, A. C. Rodriguez, R. Herrero,
M. E. Sherman, and C. M. Wheeler. 1998. Comparison of PCR- and hybrid
capture-based human papillomavirus detection systems using multiple cer-
vical specimen collection strategies. J. Clin. Microbiol. 36:3248–3254.
27. Qu, W., G. Jiang, Y. Cruz, C. J. Chang, G. Y. Ho, R. S. Klein, and R. D. Burk.
1997. PCR detection of human papillomavirus: comparison between MY09/
MY11 and GP5⫹/GP6⫹ primer systems. J. Clin. Microbiol. 35:1304–1310.
28. Schiffman, M. H., H. M. Bauer, and A. T. Lorincz. 1991. Comparison of
Southern blot hybridization and polymerase chain reaction methods for the
detection of human papillomavirus DNA. J. Clin. Microbiol. 29:573–577.
29. Schiffman, M. H., N. B. Kiviat, R. D. Burk, K. V. Shah, R. W. Daniel, R.
Lewis, J. Kuypers, M. M. Manos, D. R. Scott, M. E. Sherman, R. J. Kurman,
M. H. Stoler, A. G. Glass, B. B. Rush, I. Mielzynska, and A. T. Lorincz. 1995.
Accuracy and interlaboratory reliability of human papillomavirus DNA test-
ing by hybrid capture. J. Clin. Microbiol. 33:545–550.
30. Smits, H. L., L. J. M. Bollen, S. P. Tjong-A-Hung, J. Vonk, J. van der Vendel,
F. J. W. Ten Kate, J. A. Kaan, B. W. Mol, and J. ter Schegget. 1995.
Intermethod variation in detection of human papillomavirus DNA in cervical
smears. J. Clin. Microbiol. 33:2631–2636.
31. Swan, D. C., R. A. Tucker, G. Tortolero-Luna, M. F. Mitchell, L. Wideroff,
E. R. Unger, R. A. Nisenbaum, W. C. Reeves, and J. P. Icenogle. 1999.
Human papillomavirus (HPV) DNA copy number is dependent on grade of
cervical disease and HPV type. J. Clin. Microbiol. 37:1030–1034.
32. Tang, Y. W., and D. H. Persing. 1999. Molecular detection and identification
of microorganisms, p. 215–244. In P. R. Murray, E. J. Baron, M. A. Pfaller,
F. C. Tenover, and R. H. Yolken (ed.), Manual of clinical microbiology, 7th
ed. American Society for Microbiology, Washington, D.C.
33. Vernon, S. D., E. R. Unger, and D. Williams. 2000. Comparison of human
papillomavirus detection and typing by cycle sequencing, line blotting, and
Hybrid Capture. J. Clin. Microbiol. 38:651–655.
34. Villa, L. L. 1997. Human papillomaviruses and cervical cancer. Adv. Cancer
Res. 71:321–341.