Using Tolerance Intervals for Setting Process

Validation Acceptance Criteria

Richard K. Burdick —Amgen, Inc. (CO)

Graybill Conference
June, 2008
 Using Tolerance Intervals for Setting Process
Validation Acceptance Criteria

“A worn-out academician’s adventure in the ‘real

word’"
Outline

 Life at Amgen
 Nonclinical statistics
 Definitions for Process Characterization and
Validation
 Statistical Methods for Setting Process Validation
Acceptance Criteria
 Future Opportunities

Operational Excellence 3
 Amgen: A Biotechnology Pioneer

 Founded in 1980, Amgen was

one of the first biotechnology
companies to successfully
discover, develop and make
protein-based medicines
 Today, we’re leading the
industry in its next wave of
innovation by:
– Developing therapies in
multiple modalities
– Driving cutting-edge research
and development
– Continuing to advance the
science of biotechnological
manufacturing

Operational Excellence 4
Research and Development at Amgen

Guiding Principles
 Focus on serious illness
 Be modality independent
 Assess efficacy in patients
 Seamless integration from
research through
commercialization
Therapeutic Areas
 Inflammation
 Oncology
 Hematology
 Metabolic and bone disease
 Neuroscience

Operational Excellence 5
Nonclinical Statistics

 Chemistry, Manufacturing, Controls (CMC)

development establishes the process of
manufacturing drug product to meet clinical
requirements.
 Work in both research and development and
manufacturing.

Operational Excellence 6
Nonclinical statisticians involved in…
 R&D with
– Assay validation
– Process validation
– Method transfer
– Stability studies (storage conditions, shelf-life, expiry extensions)
– DOE for process characterization
– Establishment of specifications and process validation acceptance
limits.
 Manufacturing with
– Maximization of yields
– Control charting
– Support in non-conformance reports (identification of assignable
causes)
– Raw materials inspection

Operational Excellence 7
Timeframe of Characterization and Validation
Activities Relative to Clinical Trials

End of Phase II End of Phase III

Clinical Trial Clinical Trial and
Commit to File

Characterization Validation Update CV

documents

Operational Excellence 8
Very Simple Process Diagram

(Upstream)

Diafiltered Medium (DFM)

(Downstream)

Operational Excellence 9 Filtered Purified Bulk (FPB)

Process Characterization

 Process Characterization is a precursor to

process validation and is comprised of a set
of documented studies in which operating
parameters (inputs) are purposely varied to
determine the effect on product quality
attributes (outputs) and process performance.
 Employs Failure Modes and Effects Analysis
(FMEA) and Experimental Design

Operational Excellence 10
Process Validation
 Process validation provides the documented evidence that
the process, when operated within established limits, can
perform effectively and reproducibly to produce an
intermediate, active pharmaceutical ingredient (API) or drug
product meeting predetermined criteria and quality attributes.

 Final drug product and API have specifications that must be

met based on standards mandated by safety concerns and
other factors.
 However, intermediate process steps (which do not have
mandated standards) have a number of acceptance criteria
that must be met to demonstrate process consistency and
the ability to meet final specifications.

Operational Excellence 11
Process Validation Acceptance Criteria

 Process Validation Acceptance Criteria (PVAC)

A set of numerical limits that when exceeded,
signals a significant departure from operating
conditions or product quality.
 Set prior to initiation of the validation
campaign.
 Establishing PVAC is one of the greatest challenges
in the development of a commercial
biopharmaceutical manufacturing process.

Operational Excellence 12
Definitions

 Operating Parameter (OP): Parameter that can be

directly manipulated (input)
 Performance Parameter (PP): In-process parameter or
measurement used for process performance
evaluation (output)
 Normal Operating Range (NOR): A range for an
operating parameter that is listed in the
Manufacturing Procedure. Frequently based on
equipment and/or process capability.

Operational Excellence 13
Setting PVAC-A personal history

 My involvement with the ACO process development

(PD) group began as a discussion concerning
analysis of one-off studies conducted at 3 times
outside the NOR.
 Questions concerned how to determine the operating
parameters (OPs) that were most important in the
process.
 I helped them analyze the data in a manner they were
comfortable with, and gained their confidence so that
I could work with them on future projects.

Operational Excellence 14
Setting PVAC-A personal history

 My involvement with the ACO process development

(PD) group began as a discussion concerning
analysis of one-off studies conducted at 3 times
outside the normal operating range (NOR).
 Questions concerned how to determine the operating
parameters (OPs) that were most important in the
process.
 I helped them analyze the data in a manner they were
comfortable with, and gained their confidence so that
I could work with them on future projects.
Lesson 1: Sometimes it is best to answer the client’s
question instead of telling them what they are doing wrong.
Operational Excellence 15
 When the discussion of setting PVAC came up, I
researched the history of setting PVAC at ACO:
– There was some sentiment for “3 sigma” rules
– JMP Prediction Profiler at the extremes of the NOR had been
used with previous projects (these limits are actually the
confidence intervals on the average for a given value of the
OP).
– Data sets from robustness and edge of range studies were
not being combined. In some cases, only centerpoints were
being used to determine PVAC.

Operational Excellence 16
  When the discussion of setting PVAC came up, I
researched the history of setting PVAC at ACO:
– There was some sentiment for “3 sigma” rules
– JMP Prediction Profiler at the extremes of the NOR had been
used with previous projects (these limits are actually the
confidence intervals on the average for a given value of the
OP).
– Data sets from robustness and edge of range studies were
not being combined. In some cases, only centerpoints were
being used to determine PVAC.

Lesson 2: Find out why certain methods were used in the past.
Can you use these approaches as a starting point,
and demonstrate continuous improvement?
Operational Excellence 17
Construction of PVAC

 I suggested we use tolerance intervals for defining

PVAC because they describe the long range expected
behavior of the process.
 Bench data derived from process characterization
experimental design studies can be combined with
large-scale runs to compute tolerance intervals at set-
point conditions (or any other point in the NOR)
centered at either commercial or clinical scale.

Operational Excellence 18
TI Depends on OP
Assumed distribution
of PP for given OP is
normal. Regression
PP Line

99% of PP
values
in this range
when OP=+1

99% of PP
values
in this range
when OP=-1

OP=-1 OP=0 OP=+1

Operational Excellence 19
Type of TIs

 If all OPs are fixed effects, then exact one-sided

tolerance intervals can be constructed based on the
non-central t distribution
– See, e.g., Graybill (1976, pages 270-275)

 Exact two-sided tolerance intervals are available

(Eberhardt, Mee, and Reeve, 1989), but
computationally complex.
– Various two-sided approximations have been suggested
• Weissberg, A. and G. H. Beatty (Technometrics,1960)
• Lee, Y. and T. Mathew (JSPI, 2004)
• Liao, C. T., Lin, T. Y., and Iyer, H. (Technometrics, 2005).

Operational Excellence 20
One other refinement

 Many times, the PC models involve random effects

such as the raw materials that feed into a process
step.
 In this case, the fixed effect methods can not be
applied for computing tolerance intervals.
 Generalized Inference provides an approach for
computing tolerance intervals with a random effect.
• Liao, C. T., Lin, T. Y., and Iyer, H. (Technometrics, 2005)
• Based on generalized fiducial intervals

Operational Excellence 21
One other refinement

 Many times, the PC models involve random effects

such as the raw materials that feed into a process
step.
 In this case, the fixed effect methods can not be
applied for computing tolerance intervals.
 Generalized Inference provides an approach for
computing tolerance intervals with a random effect.
• Liao, C. T., Lin, T. Y., and Iyer, H. (Technometrics, 2005)
• Based on generalized fiducial intervals

Lesson 3: Continue to make improvements and

demonstrate you are willing to continually improve your work.

Operational Excellence 22
Example—Purification Column

 Purification is used in a biopharmaceutical product to

separate desired protein from unwanted materials.
 This example considers one such column where the
response is modeled as a function of a fixed OP
(coded -1 to +1) and the random effect feed material.
 Response is a purity measure in %.

Operational Excellence 23
 Scatterplot of PP vs OP
93

92

91

90

89
PP

88

87

86

85

84
2 4 6 8 10 12
OP

Operational Excellence 24
 Summary of Fit Parameter Estimates
RSquare 0.363075 Term Estimate Std Error DFDen t Ratio Prob>|t|
RSquare Adj 0.354105 Intercept 90.725619 0.826556 48.6 109.76 <.0001*
Root Mean Square Error 1.391341 OP -0.201453 0.092223 68.09 -2.18 0.0324*
Mean of Response 88.86986
Observations (or Sum Wgts) 73

REML Variance Component Estimates

Var
Random Effect Var Ratio Component Std Error 95% Lower 95% Upper Pct of Total
Column Feed Source Lot 0.4137555 0.8009605 0.6543545 0.256919 11.161442 29.266
Residual 1.9358307 0.3336 1.4182415 2.8007496 70.734
Total 2.7367912 100.000
-2 LogLikelihood = 263.85628974

Operational Excellence 25
 Using the GCI approach, the computed
tolerance interval for the OP=0 (setpoint
condition) is from 83.4-95%

Operational Excellence 26
Plot of Tolerance Intervals and Runs with
OP = 0

95.0 95

92.5
Response (%)

90.0

87.5

85.0

83.4

Operational Excellence 27
Future Opportunities

 FDA initiative for Quality by Design.

 ICH Q8 Appendix on movement within the proven
acceptable range (PAR)—also referred to as “Design
Space”.

Operational Excellence 28
Design Space (ICH Q8)
Risk Assessment to
Prioritize Investigation

Explored
Explored SpaceSpace
Unexplored Space
Unexplored Space DOE
DOE

Modeling
Modeling

Prior Knowledge
Prior Knowledge

First Principles
First Principles
Knowledge Space
Knowledge Space
Control Strategy
Specifications
“ Design
Design ” Space
Space Tolerances

PAR
PAR
(Proven Acceptable Range)
NOR NOR
Explored with
Acceptable
Performance
NOR
(Normal Operating Range)
Operating Strategy
based on Business/Equipment Requirements

Operational Excellence 29
Operational Excellence 30