Vaccine 26 (2008) 38703879

Contents lists available at ScienceDirect

Vaccine
journal homepage: www.elsevier.com/locate/vaccine

Indirect foot-and-mouth disease vaccine potency testing based on a serological alternative

Nesya Goris a, , Tom Willems a , Vyacheslav I. Diev b , Petra Merkelbach-Peters c,1 , Tine Vanbinst a , Yves Van der Stede d , Horst-Peter Kraft e,2 , Valery M. Zakharov b , Vladimir V. Borisov b , Hans J. Nauwynck f , Bernd Haas g , Kris De Clercq a
a

Veterinary and Agrochemical Research Centre, Virology Department, Section of Epizootic Diseases, Groeselenberg 99, 1180 Brussels, Belgium Federal Governmental Institution Federal Centre for Animal Health, Biological and Technical Control Department, 600901, Yurevets, Vladimir, Russia c Bayer HealthCare, Animal Health, Osterather Strasse 1a, 50739 Kln, Germany o d Veterinary and Agrochemical Research Centre, Coordination Centre for Veterinary Diagnostics, Groeselenberg 99, 1180 Brussels, Belgium e Bayer HealthCare, Animal Health, Osterather Strasse 1a, 50739 Kln, Germany o f Ghent University, Faculty of Veterinary Medicine, Department of Virology, Parasitology and Immunology, Laboratory of Virology, Salisburylaan 133, 9820 Merelbeke, Belgium g Friedrich-Loefer-Institute, Federal Research Institute for Animal Health, S dufer 10, 17493 Greifswald Insel Riems, Germany u
b

a r t i c l e

i n f o

a b s t r a c t
Foot-and-mouth disease (FMD) vaccine potency testing has historically been performed by experimentally infecting vaccinated cattle. A few alternative approaches to the in vivo challenge test based on the correlation between serum titres of primo-vaccinated cattle and protection against infection have been proposed, but none have been accepted by the European Pharmacopoeia (Ph.Eur.) due to the lack of statistical power and the pooling of data over time. The present study addresses these issues and presents data of 150 cattle vaccinated according to Ph.Eur. standards. Four laboratories took part in the serological testing and different serological assays were used, including virus neutralisation assays and ELISA formats. Models correlating specic anti-FMD virus antibody titres to protection were built using logistic regression followed by Receiver Operating Characteristic (ROC) analysis. The best models accurately predicted the in vivo protection status in 80.0% of the cases. Although differences were observed between laboratories and assays used, the majority of antibody pass-levels, determined using ROC analysis, corresponded to at least 75.0% probability of protection. The indirect potency assessment procedure proposed is at least as precise (repeatability = 65.8%, reproducibility = 60.7%) as the in vivo test, can be standardised and results in a quantitative PD50 value. The validity of the procedure was also demonstrated. 2008 Elsevier Ltd. All rights reserved.

Article history: Received 14 January 2008 Received in revised form 30 April 2008 Accepted 7 May 2008 Available online 28 May 2008 Keywords: Foot-and-mouth disease Vaccine potency Serology Alternative potency test

1. Introduction Foot-and-mouth disease (FMD) is the most contagious and infectious disease of cloven-hoofed animals and constitutes a substantial impediment to international trade [1]. Worldwide, good quality, reliable vaccines are one of the essential tools in controlling the spread of the causative virus. Although within the European Union (EU) prophylactic vaccination was prohibited as of 1 January 1992 (Council Directive 90/423/EEC), the change in Community legislation on measures for the control of FMD facilitates the use of

Corresponding author. Tel.: +32 2 3790400; fax: +32 2 3790666. E-mail address: negor@var.fgov.be (N. Goris). 1 Current address: Intervet International GmbH, Betriebsstatte Koln, Osterather Strasse 1a, 50739 Koln, Germany. 2 Current address: Bayer HealthCare, Product Supply, 51368 Leverkusen, Germany. 0264-410X/$ see front matter 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.vaccine.2008.05.013

emergency vaccination in case of future FMD incursions (Council Directive 2003/85/EC). At present, the potency of FMD vaccines, a major determinant in vaccine choice, is assessed in vivo by challenging fully susceptible vaccinated cattle. In the EU, FMD vaccine potency testing must comply with the requirements of the 50% protective dose (PD50 ) test laid down in Monograph 04/2005:0063 of the European Pharmacopoeia (Ph.Eur.) [2], whereas the Protection against Podal Generalisation test is the prescribed test for FMD potency assessment in South America [3]. However, in vivo FMD vaccine potency estimation is extremely expensive due to the high containment bio-security animal facilities that are needed to safeguard the environment from subsequent infection. Consequently, the smallest permitted number of cattle is used, leading to imprecision and a lack of statistical power [4,5]. Additionally, in PD50 potency tests, like in any extinction point test, approximately 50% of the animals that are not protected against FMD virus (FMDV) infection will

N. Goris et al. / Vaccine 26 (2008) 38703879

3871

suffer the painful clinical manifestations of the disease (in casu fever, hoof lesions, lameness, etc). Even the majority of protected animals will show primary lesions at the site of challenge resulting in severe salivation, loss of appetite and weight loss [6]. In light of the 3R (Renement, Reduction, Replacement) concept on animal use for regulatory testing [7], FMD researchers over the last 40 years have been investigating alternative FMD vaccine potency tests that are more acceptable from an animal welfare point of view and offer greater statistical reliability [6,822]. Among these methods, serological assays correlating a specic antibody response post vaccination to in vivo protection against challenge have several advantages. For instance, the minimal discomfort of vaccination and blood sampling, multivalent vaccines can be tested simultaneously, animal carcasses do not require rendering and their products can be processed for the national market which minimises costs. More importantly, like the prescribed in vivo test, serological alternatives also take the biological variation of animals to vaccination into account. Even though scientic evidence to replace the in vivo FMD vaccine potency test by a serological approach seems overwhelming, Ahl and colleagues [23] have urged caution when interpreting data obtained by pooling serological results obtained over the course of time with different assay methods and/or cell lines. Others have criticised any and all serological alternative [24 and reviewed by 2526]. They argued that virus neutralisation tests and ELISAs merely measure the capacity of antibodies to neutralise the virus in cell cultures and/or to interact with it. Therefore, these tests constitute no measure of the protective immune response against FMDV, in which an important role is attributed to innate immunity as well. In light of the controversy surrounding in vitro FMD vaccine potency testing based on serology, and the time and effort it takes to establish a statistically sound correlation between in vivo protection by a certain vaccine and serum titres measured in a particular test, the Ph.Eur. monograph on FMD vaccine includes a vague, indicative batch potency test in which a satisfactory pass level for a given antigen should be established, but where the term correlation was deliberately avoided [27]. The present paper will address some of the criticisms concerning serological approaches to FMD vaccine potency testing by presenting indirect potency estimation models obtained by four different laboratories on serological results of 10 replicate Ph.Eur. FMD PD50 tests using the same commercial vaccine batch against FMDV strain O1 Manisa. 2. Materials and methods 2.1. Animals, vaccines and potency tests The animals (n = 170), the vaccine batch and PD50 vaccine potency test were previously described [4]. Briey, steers of the Friesian Holstein breed were vaccinated according to Ph.Eur. standards [2] with a commercial, puried, double-oil emulsion, inactivated vaccine against FMDV strain O1 Manisa (produced by Bayer HealthCare (BHC), Koln, Germany). In total, 10 replicate PD50 tests were performed resulting in an assigned potency of 9.99 PD50 for the above-mentioned vaccine with a 95% condence interval (95% CI) ranging from 7.45 to 13.27 PD50 . In each PD50 test and at 21 days post vaccination (dpv) all 15 vaccinated animals [3 groups of 5 animals were vaccinated with either the full (2 ml), a 4th (0.5 ml) or a 16th (0.125 ml) of the vaccine dose] and 2 unvaccinated control cattle were challenged intradermolingually with 104 ID50 (50% bovine infectious dose) of homologous FMDV strain O1 Manisa. All animals were observed for 8 days during which time the general health status, rectal temperatures, food intake, nasal discharge, salivation, tongue lesions and sensitivity

of the hoofs was recorded daily. Eight days post challenge (dpc), the PD50 content of the vaccine was determined according to the method described by Karber [28]. All animals were kept for an additional 4 days. Blood samples were taken prior to vaccination (0 dpv), 21 dpv, 8 dpc and 12 dpc. 2.2. Serum sample preparation and participating laboratories Blood was allowed to clot at room temperature and serum was collected and immediately divided into four equal fractions. Each fraction was kept at 20 C until further use. In total, four laboratories took part in the serological testing. Each serum fraction was transported to these laboratories following International Air Transport Association rules and custody clearance. Furthermore, procedures were put in place to ensure adequate refrigerated conditions during shipment. The participating laboratories included (i) the Veterinary and Agrochemical Research Centre (VAR, Brussels, Belgium), (ii) BHC, (iii) the Friedrich-Loefer-Institute (FLI, Insel-Riems, Germany) and (iv) the Federal Governmental Institution Federal Centre for Animal Health (FGI-ARRIAH, Vladimir, Russia). 2.3. Serological tests Five different serological assays were used: the virus neutralisation test (VNT) against FMDV O1 Manisa on baby hamster kidney (BHK-21) cells, the liquid-phase blocking ELISA (LPBE) using homologous virus, the FMDV O1 Manisa solid-phase competition ELISA (SPCE), the Ceditest FMDV type O (Cedi-O, Prionics AG, SchlierenZurich, Switzerland) and the Ceditest FMDV-NS (Cedi-NS, Prionics AG, Schlieren-Zurich, Switzerland). The VNTs and LPBEs were performed following the World Organisation for Animal Health (OIE) Manual of Diagnostic Tests and Vaccines for Terrestrial Animals [29], whereas the SPCE was done according to the previously described procedure [30]. The Cedi-O and Cedi-NS were carried out in keeping with the manufacturers instructions. The Cedi-O detects anti-FMDV serotype O antibodies elicited either by vaccination, infection or both; whereas the Cedi-NS is designed to differentiate between FMDV vaccinated and infected animals as it specically detects antibodies to the non-structural proteins (NSP) of the virus which result from infection only and not from vaccination when a sufciently puried FMD vaccine is used which is a requirement under Ph.Eur. regulations [2]. The VAR performed the VNT, LPBE, SPCE, Cedi-O and Cedi-NS; BHC used the VNT, whereas FLI and FGIARRIAH made use of the VNT and LPBE to measure anti-FMDV O1 Manisa antibodies. Serum samples were 1:2 serially titrated in each assay and 50% endpoint serum titres were calculated according to the Reed-Munch method [31]. 2.4. Control charts In each assay performed by VAR, negative and positive control samples (i.e. working standards) were included to guarantee the quality of the assay results and to monitor the performance of the different tests in time. The ELISA charts were built using the MultiQC programme (http://www.multiqc.com/) as described by Goris and De Clercq [32]. For the VNT, the virus titre and the titre of a positive working control were charted, monitored and compared to their predetermined values. 2.5. Logistic model building The 50% endpoint serum titres at 21 dpv calculated for each serum sample and in each serological assay were transformed onto a logarithmic scale (base 10) (log10 ) to account for the distributional

3872

N. Goris et al. / Vaccine 26 (2008) 38703879

asymmetry caused by the exponentional serum dilution effect. The resulting values together with their associated binary protection status (1 = protected against challenge; 0 = not protected against challenge) were used as input in the model. The generalised linear model was based on logistic regression implemented in the R programme (http://www.r-project.org/) [33]. The model related the predicted probability of protection (pi ) (i.e. dependent variable and ranging from 0 to 1) to the log10 serum titre (ti ) of an individual animal (i.e. independent variable) according to the following formula [34]: pi = ea+bti 1 + ea+bti (1)

too many falsely predicted protected animals. In other words, the overall sensitivity and specicity of the serological potency model is maximised when using tC/O as a cut-off (C/O) point in the protection decision making process. The tC/O was determined for each serological model. 2.7. External model validation Serum samples originating from a FMD vaccine potency test performed in 2004 by FGI-ARRIAH for BHC following Ph.Eur. guidelines with a FMDV O1 Manisa vaccine batch of 10.56 PD50 , unrelated to the one used to build up the serological model, were 1:2 serially titrated as described-above and the corresponding log10 serum titres were used to determine the validity of the proposed serological model for indirect potency assessment. Animals having a log10 serum titre equal to or greater than tC/O were assumed to be protected against in vivo live virus challenge, whereas animals with a log10 serum smaller than tC/O were assumed not to be protected. In other words, for each vaccinated animal, regardless of the vaccine dose group to which it belonged, a protection status of 0 or 1 could be assigned and the total number of predicted protected animals per vaccine dose was obtained. Subsequently, the serologically expected PD50 estimate was calculated according to Karber [28] and compared to the previously in vivo determined PD50 value (in casu 10.56 PD50 ). The serological procedure was repeated 10 times to assess vaccine accordance (VACC or intra-potency test repeatability) and vaccine concordance (VCON or inter-potency test reproducibility) of the indirect FMD vaccine potency assessment method as described by Goris et al. [4]. 3. Results 3.1. Serological results In general, a limited number of test runs were repeated due to MultiQC violations (data not shown). All serum samples taken prior to vaccination (n = 170) tested negative in the VNT procedures used by the participating laboratories, in the VAR SPCE and in the Cedi-O. In total, 3 of the 170 animals sampled prior to vaccination scored doubtful in the VAR LPBE, whereas all were negative in the FLI and FGI-ARRIAH LPBEs. One animal had a borderline positive antibody response of 53% inhibition (PI) (C/O = 50 PI) in the Cedi-NS when sampled prior to vaccination, but tested negative for anti-FMDV NSP antibodies in the same test at 21 dpv. No other cattle scored positive in the Cedi-NS assay at 0 dpv or 21 dpv which is a clear sign of absence of past or present FMDV infection. Table 1 summarises the obtained mean log10 serum titres and standard deviations for all vaccinated cattle (n = 150) at 21 dpv, 8
Table 1 The serological mean log10 serum titre and standard deviation of all vaccinated cattle (n = 150) sampled 21 days post vaccination, 8 and 12 days post challenge Assay VNT Laboratory VAR BHC FLI FGI-ARRIAH VAR FLI FGI-ARRIAH VAR VAR VAR 21 dpv 1.21 0.92 1.19 0.55 1.72 0.47 1.59 0.38 1.85 0.52 2.33 0.39 1.61 0.42 1.43 0.59 1.10 0.74 All negative 8 dpc 3.17 0.43 2.84 0.52 n.a. n.a. 2.99 0.56 n.a. n.a. 3.24 0.50 2.73 0.45 0.46 0.59 12 dpc 3.40 0.42 2.97 0.48 n.a. n.a. 3.20 0.39 n.a. n.a. 3.35 0.49 2.83 0.38 1.25 0.78

in which a and b are the intercept and slope parameters, respectively, of the logistic model. The intercept yields the predicted probability of being protected when no measurable serum titres are detected in the serological assay (animals that do not serologically respond to vaccination or non-responders); whereas the slope adjusts for how quickly pi changes with varying ti . The 95% CI around pi were estimated using the two-tailed Students tdistribution with n-1 degrees of freedom in which n is the number of individual ti . The dataset used to construct the model comprised 150 individual ti corresponding to serum samples taken at 21 dpv from 50 animals vaccinated with the full vaccine dose, 50 animals receiving a 4th vaccine dose and 50 animals vaccinated with a 16th vaccine dose. The 20 unvaccinated control animals were excluded for the purpose of model building. One model was built for every serological test used per participating laboratory. Hence, nine different logistic models were obtained. Cedi-NS was only used to explore possible virus replication post challenge. It was not considered for model building procedures. The logistic model t was judged by use of the generalised Akaike Information Criterion (AIC) which shows how closely the tted pi tends to the true value. A value nearing 0 indicated a good t of the model to the data after having penalised for the number of parameters that are estimated [35]. AIC is given by: AIC = 2 loge L(b) + 2p (2)

where loge L(b) is the log-likelihood estimate and p is the number of parameters estimated in the model. AIC was thus used to screen for outliers and as such to improve the overall model t. Subsequently, in each model, the SomersD rank correlation, which is a measure (ranging from 0 to 1, with a value of 1 indicating a perfect correlation) to associate between a continuous and binary variable, was used to quantify the correlation of ti to the binary protection status [36]. 2.6. Logistic model characteristics A Receiver Operating Characteristic (ROC) analysis was carried out in the R programme to determine the log10 serum titre for which the probability of predicting in vivo protected animals as protected based on their serological response (i.e. true positives or TP) was maximised and the level of falsely predicted protected animals (i.e. in vivo unprotected animals) was minimised (i.e. false positives or FP). The ROC curve plots the TP rate (TPR or sensitivity) against the FP rate (FPR or 1-specicity) of the indirect vaccine potency test for any given log10 serum titre. Each point on the ROC curve thus represents a TPR/FPR pair with an associated level of accurate predictions (ACC). The point (i.e. tC/O ) closest to the upper left corner (100% TPR and 0% FPR) results in the highest level of accurate predictions (i.e. accuracy or ACC) given the data at hand. Hence, tC/O can be regarded as a trade-off point where serologically detecting in vivo protected animals is maximised without resulting in

LPBE

SPCE Cedi-O Cedi-NS

dpv: days post vaccination; dpc: days post challenge; n.a.: not available.

N. Goris et al. / Vaccine 26 (2008) 38703879

3873

Table 2 The Ceditest FMDV-NS ELISA mean log10 serum titre and standard deviation of vaccinated-protected and vaccinated-unprotected cattle sampled 8 and 12 days post challenge and divided into the corresponding vaccine dose groups Vaccine dose group 8 dpc Unprotected 2 ml group 0.5 ml group 0.125 ml group dpc: days post challenge. All negative (n = 4) 0.65 0.72 (n = 12) 0.61 0.64 (n = 26) Protected 0.47 0.59 (n = 46) 0.32 0.57 (n = 38) 0.48 0.53 (n = 24) 12 dpc Unprotected 1.68 0.72 (n = 4) 1.86 0.33 (n = 12) 1.71 0.51 (n = 26) Protected 0.99 0.78 (n = 46) 1.02 0.85 (n = 38) 1.24 0.72 (n = 24)

and 12 dpc using the VNT, LPBE, SPCE, Cedi-O and Cedi-NS tests. On average, signicantly higher serum titres were observed in all assays following the challenge boost than at 21 dpv (p < 0.01). Serum titres continued to increase until 12 dpc although they were not signicantly different from those observed at 8 dpc. At 8 dpc, half of the vaccinated-unprotected animals had no detectable serum titres using the Cedi-NS test; whereas all were Cedi-NS positive 4 days later. Vaccinated-protected animals were Cedi-NS positive in merely 38.3% of the cases at 8 dpc. At 12 dpc, however, only 29.9% of the animals remained Cedi-NS negative. The difference in Cedi-NS reactivity between vaccinated-protected and vaccinated-unprotected animals was further reected in the height of the serum antibody response with vaccinated-unprotected animals showing, on average, a higher log10 serum titre than vaccinated-protected animals, except for the 2 ml group at 8 dpc which included only four vaccinated-unprotected animals (Table 2). Additionally, half of the unvaccinated control animals were Cedi-NS negative at 8 dpc, but all were Cedi-NS positive 4 days later.

3.2. Logistic models One outlier was identied based on AIC (in casu animal with ear-tag number 1803). This animal received a full cattle vaccine dose which resulted in a clearly positive antibody response in all structural protein assays performed, yet the animal was found not to be protected against live virus challenge at 21 dpv. Hence, it was excluded from subsequent analysis. In total, nine generalised linear models based on logistic regression were built using the log10 transformed serum titres and binary protection statuses of the remaining 149 vaccinated animals, each representing a different serological assay performed by the participating laboratories. Fig. 1 represents the four logistic regression models correlating virus neutralising log10 serum titres to the probability of protection; Figs. 24 depict the logistic models based on the LPBE, SPCE and Cedi-O ELISA log10 serum titres, respectively. The 95% CI around the probability of protection is illustrated by a dotted line. All models clearly display a sigmoid response curve,

Fig. 1. Logistic models (full line) for virus neutralisation tests used by VAR (A), BHC (B), FLI (C) and FGI-ARRIAH (D). The dotted line represents the 95% condence interval around the predicted probability of protection.

3874

N. Goris et al. / Vaccine 26 (2008) 38703879

Fig. 2. Logistic models (full line) for the liquid-phase blocking ELISA used by VAR (A), FLI (B) and FGI-ARRIAH (C). The dotted line represents the 95% condence interval around the predicted probability of protection.

apart from the VAR VNT and Cedi-O models for which the sensitivity was insufcient to detect lower levels of specic anti-FMDV serotype O antibodies in the serum of vaccinated animals. For these assays, non-responders still had a probability of protection of 0.3000.400 (Figres 1A and 4). The best association between log10 serum titre and probability of protection is observed for models built for three different sero-

logical assays, namely the BHC VNT, the VAR SPCE and the VAR LPBE with a SomersD rank correlation ranging from 0.801 to 0.765 (Table 3). For the remaining models, the SomersD correlation factor was lower and varied from 0.575 to 0.702. This was further conrmed by poorer AIC values for the latter logistic models compared to the BHC VNT, VAR SPCE and VAR LPBE models. This indicates an inferior degree of model t to the in vivo data (Table 3).

Fig. 3. Logistic model (full line) for VAR solid-phase competition ELISA. The dotted line represents the 95% condence interval around the predicted probability of protection.

Fig. 4. Logistic model (full line) for Cedi-O ELISA performed by the VAR. The dotted line represents the 95% condence interval around the predicted probability of protection.

N. Goris et al. / Vaccine 26 (2008) 38703879 Table 3 Logistic model parameters for all 9 different serological assays used Assay VNT Laboratory VAR BHC FLI FGI-ARRIAH VAR FLI FGI-ARRIAH VAR VAR a 0.281 3.508 3.742 5.220 6.547 6.163 3.920 4.254 0.741 b 1.300 4.022 2.900 4.148 4.178 3.172 3.254 3.875 1.906 AIC 141.0 112.1 143.3 131.6 124.7 145.9 140.3 110.1 130.7 SomersD 0.598 0.801 0.6178 0.671 0.765 0.575 0.630 0.788 0.702

3875

3.3. Logistic model characteristics In order to determine a suitable antibody or log10 serum titre pass-level, a ROC analysis was performed (Fig. 5). Table 4 outlines the sensitivity (TPR) and specicity (1 FPR) of the VAR SPCE model. Each point on the ROC curve of Fig. 5(II) represents aTPR/FPR pair which is associated with a certain level of ACC in predicting truly protected animals as protected and truly unprotected animals as unprotected based on their antibody response (i.e. log10 serum titre). A SPCE log10 serum titre of 1.45 (i.e. tC/O ) corresponds to a probability of protection of 0.795 [95% CI: 0.7010.865] (i.e. pC/O ) and a highest achievable level of ACC of 80.2%. At this ACC level, the FPR (22.0%) and false negative rate (17.6%) virtually level each other out. The area under the curve was found to be 0.894 (Fig. 5.II).

LPBE

SPCE Cedi-O

a: intercept of the logistic regression curve; b: slope of the logistic regression curve; AIC: Akaike Information Criterion.

Fig. 5. (I) The ROC curves for the indirect FMD vaccine potency test based on the virus neutralisation test logistic regression models for VAR (A), BHC (B), FLI (C) and FGI-ARRIAH (D); (II) the ROC curve for the indirect FMD vaccine potency test based on the solid-phase competition ELISA logistic regression model for VAR.

3876 Table 4 The ROC analysis output for the solid-phase competition ELISA pi 0.406 0.547 0.687 0.795 0.876 0.928 0.958 0.976 0.986 0.992 ti 1.00 1.15 1.30 1.45 1.60 1.76 1.90 2.05 2.20 2.35 FPR 0.634 0.439 0.317 0.220 0.098 0.024 0.024 0.000 0.000 0.000 TPR 0.991 0.981 0.880 0.824 0.676 0.500 0.315 0.194 0.065 0.028

N. Goris et al. / Vaccine 26 (2008) 38703879

3.4. External model validation

ACC (%) 67.8 77.1 78.1 80.2 78.9 73.8 64.5 59.7 53.2 51.4

pi : probability of protection at ti ; ti : log10 serum titre at pi ; TPR: true positive rate or sensitivity of the indirect FMD vaccine potency test; FPR: false positive rate or 1 specicity of the indirect FMD vaccine potency test; ACC: accuracy of predictions.

Table 5 Logistic model characteristics determined by ROC analysis at the highest level of accuracy for all serological assays used Assay VNT Laboratory VAR BHC FLI FGI-ARRIAH VAR FLI FGI-ARRIAH VAR VAR tC/O 1.68 1.30 1.66 1.51 1.95 2.35 1.65 1.45 1.30 pC/O 0.870 0.849 0.745 0.736 0.834 0.784 0.812 0.795 0.850 FPR 0.244 0.073 0.244 0.317 0.122 0.171 0.390 0.220 0.122 TPR 0.778 0.694 0.796 0.833 0.806 0.611 0.833 0.824 0.704 ACC (%) 76.7 81.1 77.6 75.8 84.2 72.0 73.6 80.2 79.0

LPBE

SPCE Cedi-O

pC/O : probability of protection at tC/O ; tC/O : log10 serum titre at pC/O ; C/O: ROC analysis cut-off point where ACC is highest; FPR = false positive rate or 1 specicity of the indirect FMD vaccine potency test; TPR: true positive rate or sensitivity of the indirect FMD vaccine potency test; ACC: accuracy of predictions.

The same ROC analysis was performed for the remaining models and results are summarised in Table 5. Three models reach ACC levels superior to 80.0%, namely the BHC VNT, the VAR SPCE and the VAR LPBE model, whereas the FLI LPBE model with a sensitivity of 61.1% and a specicity of 82.9% only accurately predicts the animals protection status in 72.0% of the cases. Not surprisingly, the latter model also displays the lowest value of SomersD and the highest AIC (Table 3) indicating the models inferior suitability for use as an indirect vaccine potency test.

The validity of the VAR SPCE logistic model was challenged externally using a tC/O of 1.45. Using this value as antibody passlevel for protection means that primo-vaccinated animals with a log10 serum titre of at least 1.45 at 21 dpv are regarded as protected against live virus challenge, while animals displaying a log10 serum titre inferior to 1.45 are regarded as unprotected. Based on 1:2 serially titrated serum samples obtained at 21 dpv from a Ph.Eur. FMDV O1 Manisa vaccine potency test performed in 2004, which was unrelated to the one used to build up the model, log10 serum titres were determined for all 15 vaccinated animals and serologically predicted binary protection statuses were assigned. Table 6 depicts the in vivo and serologically assigned protection statuses for the 15 vaccinated animals. SPCE titrations were performed on 10 different occasions to assess VACC and VCON of the proposed indirect FMD vaccine potency test. As apparent from Table 6, 11 out of the 15 vaccinated animals were found to be protected against signs of generalised FMD in vivo resulting in a vaccine potency of 10.56 PD50 with the 95% CI ranging from 4.27 to 21.82 PD50 based on Goris et al. [4]. Based on serology and using the previously developed SPCE logistic model, a range of 7 to 10 out of 15 vaccinated animals are predicted to be protected which corresponds to PD50 values ranging from 3.48 [95% CI: 1.9310.43] to 8.00 [95% CI: 3.3118.72]. On average, 8.2 out of the 15 vaccinated animals were found to be protected resulting in an average vaccine potency of 4.86 PD50 [95% CI: 2.6612.00] using the indirect vaccine potency method based on the SPCE logistic model. Animal 1 was falsely predicted as unprotected in one of the 10 SPCEs performed. Animals 6 and 8 had serum titres below the predetermined pass-level in all SPCEs, whereas animal 11 was only correctly classied as protected in 20% of the cases and animal 13 was falsely predicted as protected on one occasion. VACC and VCON for the indirect vaccine potency test based on the SPCE logistic regression model were found to be 65.8% [95% CI: 57.474.3] and 60.7% [95% CI: 52.369.2], respectively. Similar results were obtained for the remaining assays (data not shown). 4. Discussion Alternatives to the Gold Standard live virus challenge methods (e.g. Ph.Eur. PD50 test) to asses the potency of FMD vaccines based on serology have been proposed previously [6,813,1622]. The majority of these methods converted either group mean antibody titres or individual antibody titres (neutralising or ELISA titres)

Table 6 External indirect vaccine potency assessment based on the solid-phase competition logistic model using sera from a European Pharmacopoeia vaccine potency test with an FMDV strain O1 Manisa vaccine batch unrelated to the one used to build up the serological model Full vaccine dose 1 In vivo SPCE 1 SPCE 2 SPCE 3 SPCE 4 SPCE 5 SPCE 6 SPCE 7 SPCE 8 SPCE 9 SPCE 10 1 1 1 1 1 0 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 3 1 1 1 1 1 1 1 1 1 1 1 4 1 1 1 1 1 1 1 1 1 1 1 5 1 1 1 1 1 1 1 1 1 1 1 4th vaccine dose 6 1 0 0 0 0 0 0 0 0 0 0 7 1 1 1 1 1 1 1 1 1 1 1 8 1 0 0 0 0 0 0 0 0 0 0 9 1 1 1 1 1 1 1 1 1 1 1 10 0 0 0 0 0 0 0 0 0 0 0 16th vaccine dose 11 1 0 1 0 1 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 13 0 0 1 0 0 0 0 0 0 0 0 14 1 1 1 1 1 1 1 1 1 1 1 15 0 0 0 0 0 0 0 0 0 0 0

1: protected against live virus challenge; 0: not protected against live virus challenge; SPCE: solid-phase competition ELISA.

N. Goris et al. / Vaccine 26 (2008) 38703879

3877

of vaccinated cattle sampled at 21 dpv to a percentage protection value and not to a PD50 estimate; the most favoured statistical regression model used being based on probit analysis [e.g. 18]. One noted exception was the serological FMD vaccine potency model proposed by Barnett et al. [22] in which a logistic regression model was applied to determine the probability that a particular vaccine batch had a potency equal to or greater than a pre-specied PD50 level (e.g. 6 PD50 ). However, even this serological approach is unable to predict if the tested vaccine with a potency of, for instance, at least 6 PD50 has an actual potency of 6, 10 or even 32 PD50 . The distinction between potent (6 PD50 ) and highly potent (10 PD50 ) vaccines is nonetheless of paramount importance in light of emergency FMD vaccination control policies as studies have demonstrated that highly potent vaccines more rapidly induce protection [37], protect against clinical signs [38], reduce FMDV replication and excretion [39], and thereby minimise virus transmission [40]. Furthermore, highly potent vaccines increase the likelihood of cross-protection against clearly heterologous challenge strains [41], which is of particular relevance in outbreak scenarios where no closely matched vaccine strains are available. Serum samples acquired at 21 dpv from 10 replicate Ph.Eur. FMD vaccine potency tests using the same FMDV strain O1 Manisa vaccine were utilised in the present study to propose an indirect FMD vaccine potency test that directly results in an estimate of the true PD50 content of the vaccine batch. Hence, the quantitative aspect of the FMD vaccine potency test is preserved, which is in line with the recommendations of the XIVth Conference of the OIE Permanent Commission on FMD [42]. Serum titres were correlated to a probability of protection based on logistic regression without the need to pool data obtained from different vaccine batches over time, since the samples represented 108 vaccinated-protected animals and 41 vaccinated-unprotected animals and varying serum titres were observed covering a broad range of the sigmoid dose-response curve (lower and upper plateaus and linear phase). The proposed alternative FMD vaccine potency test based on a serological, logistic regression model fulls all requirements of Ph.Eur. Monograph 01/2008:50300 Statistical analysis of results of biological assays and tests [43] since (i) cattle were randomly assigned to the vaccine dose groups, (ii) the serological responses were normally distributed [30] and (iii) a typical nonlinear, sigmoid dose-response curve was observed. Furthermore, the sensitivity, specicity and ACC of the indirect FMD vaccine potency test could be determined since the true protection status of the animals was known and ROC analysis using the in vivo protection status as Gold Standard was applied. Where necessary the characteristics of the indirect FMD vaccine potency test can be altered if a higher/lower sensitivity or specicity is required in certain situations (t-for-purpose). Moreover, the repeatability (i.e. VACC = 65.8%) and reproducibility (i.e. VCON = 60.7%) of the vaccine potency test based on the proposed serological approach at least matches that of the Gold Standard in vivo test (i.e. 67.6% and 58.8%, respectively) [4]. Furthermore, based on the SPCE tC/O the potencies of the 10 replicate Ph.Eur. vaccine potency tests were serologically predicted which resulted in a range of 4.59 to 13.93 PD50 . In vivo a much broader range of 4.59 to 24.25 PD50 was observed for this particular vaccine batch with an average potency of 9.99 PD50 [4]. Including and monitoring of working standards leads to an increased level of condence in the results obtained in the serological assays and, thus, in the overall potency test system, a procedure which is not feasible for the prescribed in vivo test. Differences in antibody pass-level or log10 serum titre were observed between different serological assays as well as between different laboratories using the same serological test, which conrms the observation made by Barnett et al. [22] that signicant

differences in the model parameters exist between different laboratories. For instance, the antibody pass-level for logistic regression models based on neutralising antibody titres varied from 1.30 to 1.68 with an associated probability of protection ranging from 0.736 to 0.870 (Table 5). The same is observed for the LPBE models although antibody pass-levels were generally higher, a phenomenon also observed by Hamblin et al. [13]. In general, however, the antibody pass-level corresponded to a probability of protection of at least 0.750 (Table 5), a value which corresponds to the 75% expected probability of protection proposed as serological passlevel in South America [29]. Furthermore, based on the logistic model parameters (AIC and SomersD rank correlation) a distinction between suited and less suited serological tests for indirect potency assessment is feasible. For instance, the FLI LPBE seems to be the less suited test to build potency assessment models due to a lack of sensitivity in predicting in vivo protected animals as protected (61.1%), whereas the BHC VNT, VAR SPCE and VAR LPBE are well suited to correlate serology to protection and their respective logistic regression models lead to highly accurate predictions (> 80.0%) (Tables 3 and 5). When using the SPCE log10 serum titres of 15 cattle from a different Ph.Eur. FMDV strain O1 Manisa PD50 tests, the validity and precision, in terms of VACC and VCON, of the SPCE indirect potency test were demonstrated. Even if the serologically predicted average potency is lower than the in vivo recorded PD50 value, these differences were not signicant as the 95% CI were highly overlapping. It is, nonetheless, proposed to either use a series of serological assays in parallel to increase the condence in predicting the potency of a particular vaccine batch or to combine for instance a serological indirect potency assessment method (e.g. BHC VNT) with a correlated method based on innate immune responses following vaccination (e.g. interferon-gamma proliferation assay [44]) and/or based on the 140S content of the nal vaccine batch [14,18,26,45]. For instance, using the very specic BHC VNT model (FPR = 0.073) in parallel with the sensitive and relatively specic VAR LPBE model (TPR = 0.806 and FPR = 0.122) results in an increase of sensitivity (TPR = 0.941) and ACC (i.e. 87.7%) in predicting the protection status of the vaccinated animals, although a slight decrease in specicity (FPR = 0.186) of the combined test system was noted. Even if two models with identical sensitivities and specicities in predicting protection were used in parallel, which would not necessarily lead to improved levels of ACC, condence in the obtained PD50 estimate would still be increased by at least 25%, since the 95% CI around the potency value can then be calculated based on twice as many data. Using three models would increase condence by approximately 40.0%. Regardless of the indirect tests used, a statistically sound logistic regression model (low AIC) correlating well to protection (high SomersD value) combined with a ROC analysis to determine the sensitivity, specicity and ACC of the prediction model provides a valid and accurate alternative to live virus challenge methods for FMD vaccine potency testing in cattle. The proposed model is at least as precise as the in vivo potency test, and is more acceptable from an animal welfare point of view. It must, however, also be remembered that the protection status measured (i.e. absence of secondary lesions on the feet) and the manner by which the animals are challenged (i.e. virulent needle challenge in the tongue) in FMD vaccine potency tests at present do not correspond to the denition of a FMDV infected animal as mentioned in the OIE Code (e.g. presence of anti-FMDV NSP antibodies) [46] or to eld outbreak situations where infection is usually contracted via direct contact through infected animals, materials and/or uids. We have clearly demonstrated that the majority of vaccinated-protected animals have detectable anti-NSP antibodies at 12 dpc. Even as early as 8 dpc, anti-NSP antibodies

3878

N. Goris et al. / Vaccine 26 (2008) 38703879 [8] Stellmann C, Bornarel P, Lang R, Terre J. Contr le quantitatif du vaccin antiapho teux Analyse statistique de la relation liant les titres danticorps neutralisant au pourcentage de protection bovine. Rec Med Vet (Alfort) 1968;144:325 51. [9] Sutmoller P, Vieira A. The relationship of neutralizing antibody titers for footand-mouth disease virus and the protection of cattle. Bltn Centro Panamericano Fiebre Aftosa 1980;3940:5762. [10] Ahl R, Lorenz RJ, Wittman G. A trial to estimate the potency of FMD vaccines by means of serum antibody assays with sera from groups of vaccinated cattle. In: Report of the Session of Research Group of the Standing Technical Committee of the European Commission for the Control of Foot-and-Mouth Disease. 1983. p. 5661. [11] Amadori M, Barei S, Bugnetti M, Panina GF. Correlation between antibody level and response to challenge in FMD vaccinated cattle: results for A and C types. In: Report of the Session of Research Group of the Standing Technical Committee of the European Commission for the Control of Foot-and-Mouth Disease. 1985. p. 329. [12] van Bekkum JG. Potency testing of FMD vaccines. In: Report of the Session of Research Group of the Standing Technical Committee of the European Commission for the Control of Foot-and-Mouth Disease. 1985. p. 104. [13] Hamblin C, Kitching RP, Donaldson AI, Crowther JR, Barnett IT. Enzyme-linked immunosorbent assay (ELISA) for the detection of antibodies against foot-andmouth disease virus. III. Evaluation of antibodies after infection and vaccination. Epidemiol Infect 1987;99(3):73344. [14] Hingley PJ, Pay TW. Sources of variability in foot and mouth disease vaccine potency estimates based on serum neutralizing antibody assay. J Biol Stand 1987;15(2):12742. [15] Pay TW, Hingley PJ. Correlation of 140S antigen dose with the serum neutralizing antibody response and the level of protection induced in cattle by foot-and-mouth disease vaccines. Vaccine 1987;5(1):604. [16] Van Maanen C, Terpstra C. Comparison of a liquid-phase blocking sandwich ELISA and a serum neutralization test to evaluate immunity in potency tests of foot-and-mouth disease vaccines. J Immunol Methods 1989;124(1):1119. [17] Bolwell C, Parry NR, Rowlands DJ. Comparison between in vitro neutralizing titres and in vivo protection against homologous and heterologous challenge induced by vaccines prepared from two serologically distinct variant of footand-mouth disease virus, serotype A22. J Gen Virol 1992;73:72731. [18] Pay TW, Hingley PJ. Foot and mouth disease vaccine potency tests in cattle: the interrelationship of antigen dose, serum neutralizing antibody response and protection from challenge. Vaccine 1992;10(10):699706. [19] Pay TW, Hingley PJ. A potency test method for foot and mouth disease vaccine based on the serum neutralizing antibody response produced in cattle. Vaccine 1992;10(10):70713. [20] Robiolo B, Grigera PR, Periolo OH, Seki C, Bianchi T, Maradei E, et al. Assessment of foot and mouth disease vaccine potency by liquid-phase blocking ELISA: a proposal for an alternative to the challenge procedure in Argentina. Vaccine 1995;13(14):134652. [21] Dus Santos MJ, Wigdorovitz A, Maradei E, Periolo O, Smitsaart E, Borca MV, et al. A comparison of methods for measuring the antibody response in mice and cattle following vaccination against foot and mouth disease. Vet Res Commun 2000;24(4):26173. [22] Barnett PV, Statham RJ, Vosloo W, Haydon DT. Foot-and-mouth disease vaccine potency testing: determination and statistical validation of a model using a serological approach. Vaccine 2003;21(23):32408. [23] Ahl R, Haas B, Lorenz RJ, Wittman G. Alternative potency test of FMD vaccines and results of comparative antibody assays in different cell systems and ELISA. In: Report of the Session of Research Group of the Standing Technical Committee of the European Commission for the Control of Foot-and-Mouth Disease. 1990. p. 514. [24] McCullough KC, Bruckner L, Schaffner R, Fraefel W, Muller HK, Kihm U. Relationship between the anti-FMD virus antibody reaction as measured by different assays, and protection in vivo against challenge infection. Vet Microbiol 1992;30(23):99112. [25] McCullough KC, De Simone F, Brocchi E, Capucci L, Crowther JR, Kihm U. Protective immune response against foot-and-mouth disease. J Virol 1992; 66(4):183540. [26] Doel TR. Natural and vaccine induced immunity to FMD. Curr Top Microbiol Immunol 2005;288:10331. [27] Castle P. Current regulatory background. In: Proceedings of the International Symposium on Foot and Mouth Disease organised by the European Directorate for the Quality of Medicines, Council of Europe, Strasbourg. 2003. p. 58. [28] Karber G. Beitrag zur kollektiven Behandlung pharmakologischer Reihenversuche. Arch Exp Pathol Pharmakol 1931;162:4803. [29] OIE (World Organisation for Animal Health). Foot and mouth disease. OIE Standards Commission. Manual of diagnostic tests and vaccines for terrestrial animals. 5th ed. Paris, France: Ofce International des Epizooties; 2004 [chapter 2.1.1]. [30] Goris N, De Clercq K. Quality assurance/quality control of foot and mouth disease solid phase competition enzyme-linked immunosorbent assay. Part I. Quality assurance: development of secondary and working standards. Rev Sci Tech Off Int Epiz 2005;24(3):9951004. [31] Reed LJ, Munch H. A simple method of estimating fty percent endpoints. Am J Hyg 1938;27:4937. [32] Goris N, De Clercq K. Quality assurance/quality control of foot and mouth disease solid phase competition enzyme-linked immunosorbent assay. Part II.

were observed in nearly 40.0% of the vaccinated protected animals conrming the experimental data analysis performed by Brocchi et al. [47]. One may, thus, question whether absence of clinical signs at the feet at 8 dpc is the best manner by which protection should be demonstrated, especially when relatively high anti-NSP antibodies are noted and low anti-FMDV serum titres are observed at 21 dpv. It is not unthinkable in those instances that small foot lesions have gone undetected or have not yet developed by the time the animals are clinically scored. Vice versa, animals with high anti-FMDV antibodies and low anti-NSP antibodies, indicating a low level of FMDV replication, but having one small foot lesion are in vivo classied as unprotected, whereas their probability of protection based on serological regression models is high. For instance, animal 1803 (outlier) with a small lesion on the left hind foot would have been serologically predicted as protected in 7 out of the 9 models developed. Moreover, even at 12 dpc this animal had a relatively low log10 anti-NSP serum titre (i.e. 0.85) which is more indicative for vaccinated-protected animals than for vaccinated-unprotected animals. Considering that some authors question whether generalised FMD can be demonstrated based on the detection of a single foot lesion which could have been contracted as a primary infection due to a small foot injury [18], the actual level of correlation between the log10 serum titre and protection may thus be even better than the approximately 80.0% correspondence demonstrated here. In conclusion, clear evidence to replace the current Ph.Eur. vaccine PD50 potency test for ofcial FMD vaccine batch release by a valid, alternative, indirect PD50 test has been provided. The proposed method is sensitive, specic, accurate and precise (VACC and VCON), and can easily be standardised. Including reference sera increases the condence in the obtained results even further. Moreover, the important quantitative aspect of the FMD potency test is preserved. However, additional studies are needed to examine the validity the approach for FMDV serotypes, and possibly certain subtypes, other than serotype O. Acknowledgements The support of DG Research of the European Commission is greatly acknowledged. The study was funded by the FMD ImproCon project (http://www.fmdimprocon.org) under the Sixth Framework Programme (grant SSPE-CT-2003-503603) and by the Federal Public Service Health, Food Chain Safety and Environment (grant RT-05/06-ALTANDI-2). Special thanks go to Ina Musch for her invaluable technical assistance and to the animal attendees of FGIARRIAH. References
[1] Leforban Y. Prevention measures against foot-and-mouth disease in Europe in recent years. Vaccine 1999;17(1314):17559. [2] European Pharmacopoeia. Foot-and-mouth disease (ruminants) vaccine (inactivated) 04/2005:0063. 2006, 5th ed., version 5.5. [3] Centro Panamericano de Fiebre Aftosa. Requisitos para acreditacion de sistemas nacionales ociales para el control de calidad de vacuna antiaftosa. 2002, version 1.0. [4] Goris N, Merkelbach-Peters P, Diev VI, Verloo D, Zakharov VM, Kraft HP, et al. European Pharmacopoeia foot-and-mouth disease vaccine potency testing in cattle: between test variability and its consequences. Vaccine 2007;25(17):33739. [5] Goris N, Maradei E, DAloia R, Fondevila N, Mattion N, Perez A, et al. Footand-mouth disease vaccine potency testing in cattle using homologous and heterologous challenge strains: precision of the Protection against Podal Generalisation test. Vaccine 2008;26:34327. [6] Pay TWF, Hingley PJ. The use of serum neutralizing antibody assay for the determination of the potency of foot and mouth disease (FMD) vaccines in cattle. Dev Biol Stand 1986;64:15361. [7] Hendriksen CF. Renement, reduction and replacement of animal use for regulatory testing: current best scientic practices for the evaluation of safety and potency of biologicals. ILAR J 2002;43(Suppl:):S438.

N. Goris et al. / Vaccine 26 (2008) 38703879 Quality control: comparison of two charting methods to monitor assay performance. Rev Sci Tech Off Int Epiz 2005;24(3):100516. The R Foundation for Statistical Computing, R Version 2.5.1. ISBN 3-900051-003, 2007. Agresti A. Categorical data analysis. New Jersey: Wiley-Interscience; 2002. Berkvens D, Speybroeck N, Praet N, Adel A, Lesaffre E. Estimating disease prevalence in a Bayesian framework using probabilistic constraints. Epidemiology 2006;17(2):14553. Somers RH. A new asymmetric measure of association for ordinal variables. Am Sociol Rev 1962;27:799811. Cox SJ, Parida S, Voyce C, Reid SM, Hamblin PA, Hutchings G, et al. Further evaluation of higher potency vaccines for early protection of cattle against FMDV direct contact challenge. Vaccine 2007;25(44):768795. Cox SJ, Voyce C, Parida S, Reid SM, Hamblin PA, Hutchings G, et al. Effect of emergency FMD vaccine antigen payload on protection, sub-clinical infection and persistence following direct contact challenge of cattle. Vaccine 2006;24(16):318490. Cox SJ, Voyce C, Parida S, Reid SM, Hamblin PA, Paton DJ, et al. Protection against direct-contact challenge following emergency FMD vaccination of cattle and the effect on virus excretion from the oropharynx. Vaccine 2005;23(9):110613. Orsel K, de Jong MC, Bouma A, Stegeman JA, Dekker A. The effect of vaccination on foot and mouth disease virus transmission among dairy cows. Vaccine 2007;25(2):32735.

3879

[33] [34] [35]

[36] [37]

[38]

[39]

[40]

[41] Brehm KE, Kumar N, Thulke H-H, Haas B. High potency vaccines induce protection against heterologous challenge with foot and mouth disease virus. Vaccine 2008;26(13):16817. [42] OIE (World Organisation for Animal Health). Recommendations of XIV Conference of the Commission on Foot and mouth disease. Bull Off Int Epizoot 1975;83:5579. [43] European Pharmacopoeia. Statistical analysis of results of biological assays and tests 01/2008:50300, version 6.0. [44] Parida S, Oh Y, Reid SM, Cox SJ, Statham RJ, Mahapatra M, et al. Interferongamma production in vitro from whole blood of foot-and-mouth disease virus (FMDV) vaccinated and infected cattle after incubation with inactivated FMDV. Vaccine 2006;24(7):9649. [45] Rweyemamu MM, Black L, Boge A, Thorne AC, Terry GM. The relationship between the 140S antigen dose in aqueous foot-and-mouth disease vaccines and the serum antibody response of cattle. J Biol Stand 1984;12(1):11120. [46] OIE (World Organisation for Animal Health). Foot and mouth disease. OIE Standards Commission. Terrestrial animal health code. 16th ed. Paris, France: Ofce International des Epizooties; 2007 [chapter 2.2.10, Article 2.2.10.1]. [47] Brocchi E, Bergmann IE, Dekker A, Paton DJ, Sammin DJ, Greiner M, et al. Comparative evaluation of six ELISAs for the detection of antibodies to the nonstructural proteins of foot-and-mouth disease virus. Vaccine 2006;24(4748): 696679.