Process & Product Performance

Review & Continuous Improvement

Presented by Linda Ambrose

ISPE Indonesia, May 2015
 Today…..

 Quality Systems review

 Quality Metrics
 PQR (Bonus slides)
Quality Systems Review
 QMS review
Develop a Project to :
 Measure &
 Monitor
the performance of QMS considering both:
 Prevention or
 Correction
Where’s your focus?
 QMS Review
Consider infrastructure of your QMS?
 Science & Patient Focus (or more simply
“compliance” based)?
 Metrics established – clear & effective for
Continuous Improvement?
 Management review
 Goals and Initiatives
 Quality Plans
 Trend Monitoring
 Quality Investigation
 QMS Review - PREVENTION

 Company Culture - emphasis on learning

from mistakes or punishment & blame?
 Frequency & Intent of Management
reviews (e.g compare with monitoring of
financial & sales data) i.e. what you
measure & monitor matters
 How are Internal (& external audits)
viewed – as a valued resource or a
nuisance?
 QMS Failure - PREVENTION
 Planning & monitoring “Quality” across the
product lifecycle - need discipline and
senior level buy-in
 Responses to regulatory inspections -
fixing observation or preventing the next
one that you haven’t received?
 Metrics – how will you measure and
monitor the status of QMS?
 Are Quality Improvement plans in place?
 Quality Metrics
 Metrics are a good guide
 Be guided by the data rather than ruled
by it!!
 The cross-functional
“rich discussion” is
always the best part of
the management review
i.e. what does the data mean?
 ISPE Quality metrics
Quality Metric Summit, You can’t improve
what you can’t measure

See more at:

http://blog.ispe.org/?p=2256#sthash.U8azHeaZ.dpuf
Quality Metrics
 Quality Performance (1)
Quality performance indicators should be:
 Specific
 Easy to collect
 Easy to understand
 Translatable into costs where appropriate
and/or practicable
 Quality Performance (2)
Quality performance indicators should be:
 Easy to trend
 Capable of identifying improvement
opportunities and tracking improvements.
 Capable of driving the required behaviour
 Visible to everybody
 Quality Performance (3)
Each quality performance indicator
should have specified & documented:
 Scope of the measure/process that is
being measured
 Method of calculation
 Frequency of measurement
 Owner of the measure
 Quality Performance (4)
Each quality performance indicator
should have specified & documented:
 Reporting mechanism
 Recipients of the reported measures
 Frequency of reporting the measures
 Review process for reported measures
 Quality Performance (5)
When defining quality performance
indicators consider:
 What are the quality critical operations
 Who are their customers and
 What measures would be key to driving
quality improvement ?
Defined measures should include both
predictive leading indicators and reactive
lag indicators.
 Quality Metrics
Typical balanced set of measures:
 Measures for assessing achievement of
strategic business objectives and
customer satisfaction
 Measures of process and product
performance.
 Quality Metrics
Examples:
 Actual vs planned achievement of business
quality objectives & improvement targets
 Risk mitigation - actual vs predicted level of
quality and compliance risk reduction
 Overdue regulatory audit actions
 Customer complaint trends
 Stock Outs due to quality problems
 Returns reports relating to order processing or
transportation or distribution related problems
 Process Quality Measure
Process quality performance measures:
 Process capability (CpK)
 Yield
 Cycle time and throughput time
 Amount of waste in process
 Product Quality Measure
Product quality performance measures:
 Number of deviations
 Percent of batches produced right first
time (RFT)
 Number of batch rejects and reworks
 Cost of batch rejects and rework
 Customer complaint rate by product
 Recalls and withdrawals
 Supplier Quality Measure
Supplier Quality performance measures:
 Number of batches rejected/ returned vs
number of batches received
 Number of batches received from a
supplier right first time vs total number of
batches supplied
 Quality System Measures
Measures of the overall effectiveness of
the quality system:
 Cost of quality.
 Compliance with quality policies and
regulatory standards.
 Number and criticality of adverse findings in
regulatory inspections, level 2 and 3 audits
and self- inspections.
 Self assessments against quality excellence
models
 Quality Metrics
 Cross functional Quality Improvement targets
should be defined - not only the quality
organisation.
 Quality performance indicators must be tracked
& trended to verify improvement target
attainment and improvement sustainability.
 Specified key quality performance indicators
should be regularly reviewed for their suitability
for purpose and capability to drive
improvement.
 QMS review
Develop a Project to:
 Measure &
 Monitor
the performance of QMS considering both:
 Prevention or

 Correction
Where’s your focus?
 QMS Review - CORRECTION

 Review changes not anticipated

 Regulatory
 Deviations
 Unexpected performance changes identified
during review and not through working systems
 Review recurring Issues
 Line clearance
 Complaints
 Deviations
 QMS Review - CORRECTION

 Review critical failures

 Recalls
 Batch Rejections
 Regulatory certificates of Non-Compliance
 QMS Review - CORRECTION
 Audit Observation Responses - “band-aid”
approach or really Root Cause Analysis &
prevention?
 Exception reports have a lot of “retrain the
operator”
 Complaints are not confirmed since…
 Training records are only pretty paperwork
systems
 Understanding of Industry trends
 Formal PQR – why?

 Regulatory
Requirement
(Ch 1.10)

 Good Business
Sense
 Formal PQR – why?
Useful to have
 Standardised approach for evaluating
product review data
 Identified statistical or quality tools to
analyse the data
 Record the resultant analysis, make
observations, formulate conclusions
about the product or process
 Formal PQR – why?
 Regulatory requirement to assess the
product performance in terms of process
driven control limits and registered
product specs and to compare
performance to previous review period
results (historical performance)
 To assess the significance of identified
process performance and/or process
capability changes
 PQR – Regulatory View (1)
GMP Ch 1.10 Product Quality Review requires
“Regular periodic or rolling quality reviews….”
Objectives:
 Verifying the consistency of the existing process
 Appropriateness of current specifications - starting
materials & finished product
 Highlight any trends and to identify product and
process improvements.
 PQR – Regulatory View (2)
Review & document (normally annually) :
i. Starting materials including packaging materials
used in the product,
ii. Critical in-process controls and finished product
results.
iii. All batches that failed to meet established
specification(s) and their investigation.
iv. Significant deviations or non-conformances, &
related investigations, and effectiveness of
corrective and preventive actions taken.
 PQR – Regulatory View (3)
v. Changes to the processes or analytical
methods.
vi. Marketing Authorisation variations submitted,
granted or refused, including those for third
country (export only) dossiers.
vii. Results of the stability monitoring programme
& adverse trends.
viii. Quality-related returns, complaints & recalls &
investigations.
 PQR – Regulatory View (4)
ix. For new marketing authorisations and
variations to marketing authorisations, a review
of post-marketing commitments.
x. Qualification status of relevant equipment and
utilities, e.g. HVAC, water, compressed gases,
etc.
xi. Contractual arrangements as defined in
Chapter 7 to ensure that they are up to date
 EU Guide – Annex 15
5.31. Ongoing process verification should be conducted under an
approved protocol or equivalent documents and a corresponding
report should be prepared to document the results obtained.
Statistical tools should be used, where appropriate, to support any
conclusions with regard to the variability and capability of a given
process and ensure a state of control.

5.32. Ongoing process verification should be used throughout the

product lifecycle to support the validated status of the product as
documented in the Product Quality Review. Incremental changes
over time should also be considered and the need for any
additional actions, e.g. enhanced sampling, should be assessed.
 Formal PQR or APR
 What is the difference between Annual
Product Review (APR) and Product
Quality Review (PQR)?
 Principle objective same.
 Details specified for PQR in WHO, PIC/S
& EU GMP are more extensive than that
in APR as represented in US guidelines
 See “bonus” slides at the end for more
detail).
 Warning on Software
 Ensure that any software used, is
validated for GMP use
 Trend Plots
 Going beyond “compliance with specification”
 Look for patterns (not simply “Compliance with
Spec”)
 Control Charts have two broad classifications,
variables and attributes
 Cumulative Sum (CuSum) Analysis - a
graphical and statistical methodology used to
identify significant changes and trends.
 PQR Conclusions
Review all data analyses, summarise
conclusions, make recommendations

Summarise and document all conclusions such

that the conclusion may be effectively converted
into recommendations.
 Take Home messages
Prevention is Better than CURE!! So be
proactive in considering the health of QMS

Ensure Regulatory expectations are met.

Go beyond “Compliance” to “Assurance”

Use appropriate statistical tools and

validated software
And Finally…
Contact me………

Linda @Qualitymindset.com
 Reference/acknowledgement

Some slide builds were inspired by materials from a ppt

given by Zena Kaufman Divisional Vice President Global
Pharmaceutical Operations – Abbot labs.
Freely available on the Internet
 PQR Bonus Slides…

More “Take Away”

detailed suggestions
are given in the
following slides for
each PQR
Regulatory
requirement
 Starting Materials & FP
 To confirm that quality of key input
materials has not materially changed.
 To check for any correlation between
major processing issues/ product quality
problems with input material quality.
 To confirm key input material supplier
performance is not posing any risk to
product quality
 Critical Process Equipment
• To demonstrate the qualified status and
satisfactory performance of process
equipment.
• To confirm that planned validation and re-
validations were completed and
acceptance criteria met.
• To assess the cumulative effect of any
equipment changes and modifications on
process performance and product quality.
 Critical Utilities
 To note any adverse trends, action and
alert limit excursions which may need to
be taken into consideration in the related
product review.
 IPC
Check for emerging trends to confirm:-
 Compliance with in-process specifications
 Effectiveness of in-process controls
 Appropriateness of action and alert limits.
 Review variation of in-process control data
& check correlation to final product quality.
 Assess impact of changes to IPControls
and specifications on process performance
and product quality.
 Yield/Reconciliation
Comparison of actual yield with expected,
theoretical or standard
 To validate the yield range for future
manufacture.
 To identify where there may be significant
material losses and highlight required
improvements.
 Lab data
 Assess trends.
 Assess process capability.
 Assess level of out of specification (OOS)
results for the product.
 Assess level of results outside the typical
quality ranges for the product.
 Changes to analytical
methods and specifications
 Assess impact of any specification
changes on process performance and
product quality and stability.
 Identify any changes to methodology
including the date of implementation of
the change and to then assess the impact
of these changes on the batch analytical
test data and compliance to specification
 Changes to analytical
methods and specifications
 Confirm the validated status of analytical
methods used to test product.
 Assess the robustness of analytical
methods and identify any method
improvements or training requirements
for specific method
 Deviations to standard
laboratory practice
 Summarise all laboratory deviations and
associated corrective and preventive
actions that may impact the analytical
test results for the product under review.
 Assess any trends or underlying issues
that may require additional
corrective/preventive action to be taken
to prevent recurrence
 Impact of unplanned process
deviations
 Summarise discrepancies from standard
manufacturing & material handling
procedures for the product under review.
Corrective and preventive actions should
also be summarised and included.
 Assess any trends or underlying issues
that may require additional
corrective/preventive action to be taken
to prevent recurrence
 Impact of planned process
deviations (Change Control)
 Identify all changes made during the
review period and use this information in
the product review to assess the
cumulative and measurable effects of
these changes on process performance,
product compliance to specification and
product stability
 Registered manufacturing
details
 To provide assurance that product
manufacturing activities conformed to
regulatory requirements current at time of
manufacture for all markets receiving the
product
 Registered specs for materials,
components & product
 Provide assurance that product and input
material specifications used to test and
release batches of input materials and
product conformed to the regulatory
requirements current at time of
manufacture for all markets receiving the
product.
 Expiry Dates
• To confirm that correct expiry/use before
dates are being applied to product for all
markets supplied.
 Batches with valid
OOS results
• To trend and analyse the main/common
reasons for product failures.
 Rework/ Reprocessed
Batches
 Document level of rework/reprocessing
 Evaluate effectiveness of actions taken to
reduce frequency/% of product failures.
 Identify further opportunities to reduce the
frequency/% of reworking/reprocessing.
 Evaluate the effectiveness of the
rework/reprocessing procedures to yield
product that complies with specification.
 Batch Rejects
 Document the reject level for the product.
 Evaluate the effectiveness of actions
taken to reduce frequency/% of product
failures.
 Identify further opportunities to reduce the
reject level for the product.
 Process validation (including
cleaning)/ revalidation status

 To confirm that planned validations, re-

validations and validation reviews were
completed and the acceptance criteria
met
 Stability Data
 Confirm that product stability studies
support the approved expiry/use before
dates for all markets where product is
supplied.
 Review any deviations from normal
product stability profile and actions
taken.
 Retention Samples
 Note any observed changes in retained
samples and the implications of these
observations.
 Document if there are no visible signs of
deterioration as this will provide
supporting data to the commercial stability
programme.
 Storage & distribution
 Summarise any discrepancies from approved
storage and distribution conditions or
procedures relevant for the product under
review. Corrective and preventive actions should
also be summarised and included.
 Assess any trends or underlying issues that may
require additional corrective/preventive action to
be taken to prevent recurrence.
 Customer Complaints
 Review complaint trends for the product.
 Confirm that corrective action has been
taken.
 Evaluate the effectiveness of corrective
actions.
 Returns
 Review the level of returns for the
product and the reasons for the returns.
 Review level of waste generated and
opportunities for reduction.
 Demonstrate that returned goods were
correctly assessed and dispositioned.
 Recalls
 Review any product recalls.
 Review the effectiveness of corrective
actions to prevent recurrences.
 Warning on Software
 Ensure that any software used, is
validated for GMP use
 Trend Plots
 Going beyond “compliance with specification”
 Look for patterns (not simply “Compliance with
Spec”)
 Control Charts have two broad classifications,
variables and attributes
 Cumulative Sum (CuSum) Analysis - A
graphical and statistical methodology used to
identify significant changes and trends.
 Conclusions
Review all data analyses, summarise
conclusions, make recommendations

Summarise and document all conclusions such

that the conclusion may be effectively converted
into recommendations.