Risk Assessment Principles for the Product Quality Initiatives

H. Gregg Claycamp, Ph.D., CHP Center for Veterinary Medicine Office of New Animal Drug Evaluation hclaycam@cvm.fda.gov June 25, 2003

Outline

Overall Premise and Questions Basics of Risk Analysis Possible Stages of Risk Assessments for PQ Initiatives Risk Ranking Model for a PQ Initiative? Pilot Scale Conclusions
The opinions and ideas presented here are those of the author and do not represent policy or opinion of the FDA. This material is intended for discussion purposes only.
2

Premise: Links Among Process (GMP) Risks and Patient Risks are Lost
cGMP Patient

Process Inspection Risks

Correlations? Models?

Quality (Patient) Factors

RISK

RISK
3

Goal: Re-Link cGMP (PQ) Risks with Actual Risks to the Patient
cGMP Patient

Process Inspection Risks

RISK

Quality (Patient) Factors

RISK
4

The Question

Can Risk Management theory, tools, practice and philosophy be employed to re-link risks to the patient with the risks identified, perceived or otherwise implicated in product quality terms?
How can we share a common language about risk, risk management,and science-based decision making so that we can focus on developing a high-quality risk management model for product quality?

Getting Started

What theories, tools and lessons learned in risk analysis can help address these questions? Given the need for a significant shift in the approach to risk management, how do we begin the change process?
Are

there off-the-shelf models and tools that might be used, i.e., at a pilot-scale? kinds of RM processes can be used to foster changes needed both the regulatory and industrial spheres?
6

What

Basic Risk Analysis

Starting with the Some Basics

Risk is intuitive and familiar to everyone, yet few among us define risk carefully and formally enough for complex risk analysis.

Risk = exposure to a chance of loss

(or, Risk = chance of losing something we value)

Risk = Hazard x Exposure

RiskConsequence = Hazard x Exposure
9

Contemporary Risk Analysis

Includes four major activities: Hazard Identification Risk Assessment Risk Management Risk Communication

10

Risk Assessment Precedes Risk Management

Risk assessment is not a single process, but a systematic approach to organizing and analysing scientific knowledge and information to support a risk decision. NRC (1994)
Various paradigms exist for the execution of a risk assessment in public health; however, all paradigms have in common fundamental scientific principles.

11

Risk Assessment Asks:

What can go wrong?

What is the likelihood it would go wrong?

What are the consequences?

12

Risk Management Asks:

What can be done?

What options are available? What are risk trade-offs in terms of risks, benefits and costs? What are the impacts of current management decisions on future options?
13

Roles/Tasks (--short list)

Risk Managers

Risk Assessors

Pose the risk question. Charge the Risk Assessors with the Risk Assessment Task. Convene stake holders. Analyze decision options.

Identify data and gather information on the nature, extent, magnitude and uncertainty of the risk. Write the Risk Assessment. Recommends changes to RM questions.

Make/recommend the decision.

14

Risk Assessments

Regulatory Policy Risk Assessment: (e.g., Biotechnology RA to determine the need for risk management regulation.) Applied Risk Assessment: To determine compliance with a regulation or policy. Safety Assessments: Highly defined risk calculations. Usually under a bright line safety policy.
15

Safety vs. Risk

Safe Unsafe

e.g., Declared unsafe

Safety Limit

Estimates of risk

e.g., Limit exceeded 10 times in 100

Risk Limit
16

Risk Analysis in a Democracy

Risk assessments provide the facts for risk analysis.

Risk

Risk
Risk Risk Risk Assessment Risk Risk

Risk Risk

Risk

Risk

Risk

Risk Risk
17

Worst

Risk

Risk Analysis in a Democracy

The risk management decisions about which risks to manage are value-laden decisions.
Risk Management Rank Risk

Values

Risk
Risk

Worst

Risk
Risk
Risk Management

Risk
Risk Risk Risk

Risk Risk Risk

Costs

Risk
Risk
18

Translating Risk Analytic Paradigms

Risk Analysis

Hazard Identification
Risk Risk Assessment Assessment
Release Assessment PQ Failures Exposure Exposure Assessment Assessment Consequence Consequence Assessment Assessment Risk Risk Estimation Estimation
What can go wrong? What are the consequences?

What is the likelihood that it would go wrong?

What can be done? What are the trade-offs in terms of costs, benefits and risks? What is the impact of decisions on future RM options?

Risk Management
Risk Communication

19

Possible Stages of Risk Assessment for Work Planning

20

Hazard Identification

What can go wrong?

Identify

hazards: events Identify hazardous agents (chemical, biological, physical)

How severe are the potential consequences?

Given

the event occurs, is the consequence catastrophic? Mildly annoying?

How likely are the events to occur?

Essentially

a crude risk estimate for initial prioritization

purposes.
21

Exposure Assessment

Release Assessment: How much of the

hazardous event occurs?
Example:

Does a non-sterile event involve 1 or 10,000 vials?

Pathway analysis: If the hazardous event occurs, what pathways are there that expose humans to the hazard?
Extent of exposure: If a hazardous event occurs, how many people are potentially exposed?
22

GMP Failure (Release) Assessment

How frequent are the identified PQ events (hazards)? Boundary of release? Process line, plant, warehouse, distributor? Release rates (PQ Faults) are obtained in fault tree assessments, empirically, historical data, expert analyses.
Example:

FMEA
23

Consequence Assessment*

Given exposure to the hazardous event/agent, what is the likelihood of harm under a predefined endpoint? Endpoint examples:
Death Illness Worry OAI

*A.K.A. Dose-Response Assessment (see next slide)

24

Consequence Assessment
100% Proportion of exposed persons who become ill

50%

0%

ED50

Quantity of contamination (non-sterility) i.e., in bacteria counts per vial

25

Qualitative Consequence Assessment

Relative Effect/Impact High Medium

Quantitative relationships known in few cases

Low Low Medium

(Exposure or Dose Metric)
26

High

Risk Estimation

Bring together the information about the hazard, the extent of exposure to the hazard, the consequences of exposures, and then estimate the risk. Includes a critical analysis of uncertainty in both the data and risk assessment models.
27

Uncertainties in Risk Assessment

Knowledge
Data Parameters Model

Variability
Temporal Spatial Inter-individual

28

Conceptual Models for RM in PQ Initiative

29

The PQ Risk Management Problem

Diverse PQ failure (hazards) are identified.

Wide-ranging risk (= chance that exposure to the hazard will result in harm [adverse outcome]). How can we objectively rank Wide-ranging consequences (death to worry). apples and oranges among the Quantitativepotatoes risk analysis hazard-by-hazard too and beans? vast an undertaking. Ranking of risks for re-linking worst PQ risks with worst health risks, etc.
30

From the Beginning

Is risk analysis for each hazard independentlyfeasible?

31

Fault Trees for

Bulb Fails

each process?

No electricity

Glass Broken

Filament Broken

Vacuum Leak

Power Plant Fails

Power Line Fails

Connector Corroded

Impurities

Vibrations

Wind Breaks Line

Tree Breaks Line

32

Faults Magnified N-fold for a Simple Manufacturing Process

33

Decision Analyses for Each Hazard Multiplies Complexity!

e.g.,

34

Solution? A Multifactor Approach to PQ Risk Management

Multifactor methods already exist.

Some tools (software) already developed. Appropriately-scaled approach to
the the

question, data quality,

the
the

nature of the decision, and

understanding of the overall process.
35

State the Assumptions

E.g., assume that health risks were linked to PQ compliance risks previously, i.e., the historical basis of regulation. Historically based assumption: compliance Health risk quality Given the assumption, can GMP compliance risk be modeled as a surrogate of health risk?

36

Identify the PQ Failures (Hazards)

What can go wrong?

Top level organization of hazards:

Health

| Compliance | Resources | Sociopolitical

(microbial contamination) (formulation) (chemical contamination) hazards (physical contamination/defect)
37

Second level (detail) organization:

Sterility Dose Toxicity Physical

Fine detail: risk factor event descriptors.

Sort the Hazards/Risks by Major Categories

Start with assumptions. State questions to be answered. Sort under the questions. Re-sort if new patterns emerge.

For example, (next slide)

38

Organizing a Multi-factorial Risk Model

Risk
Health Compliance Resource

SocioPolitical
Human

Public

Death

VAI

Chronic Illness

OAI

Inspection $

Industry

Acute Illness

The Hill

Mental Health
39

Focused Multi-factorial Risk Model

Risk Health Compliance
Death Chronic Illness Acute Illness Mental Health VAI OAI

Example health risk endpoints

Example compliance risk endpoints

40

Risk factors for a given endpoint

Health
Death
Sterility

Compliance
OAI
Sterility

Lyophilization Final Sterility etc.

Lyophilization Final Sterility etc.

41

Estimate the Prevalence

The prevalence of inspection findings for a given type of event are initial estimates of probabilities necessary for risk management modeling. Failure analysis in plant. Failure in compliance inspections.

Human adverse events.

42

For each hazard

Health
Endpoint

Probability of Occurrence Very Low

Medium

Low
Medium

Medium
High

High
High

Very High
High

Death
Chronic Illness Acute Illness Worry

Low
Low Low

Medium
Low Low

Medium
Medium Low

High
Medium Medium

High
High Medium

43

The modelers view (for example)

Health
Endpoint

Probability of Occurrence Very Low

5

Low
4

Medium
3

High
2

Very High
1

Death
Chronic Illness Acute Illness Worry

6
7 8

5
6 7

4
5 6

3
4 5

2
3 4

44

For each hazard

Compliance

Prior History of Actions

Never Violations
Medium Low Low

Endpoint

Few Viol.
Medium Low Low

Average Viol.
High Medium Low

Some Viol.
High High Medium

Many Viol.
High High High

OAI
VAI Other?

45

Scoring, then prioritize multiple hazards

Probability of Occurrence Ve ry Lo w Endpo Me int diu m Death Lo w Chron ic Lo Illness w Acute Illness Lo w Worry Ver Probability of Occurrence y Medi Hi Hig um gh h

Scored and Prioritized

1. PQ Fault A
Ver y Hig h Hig h

Endpo int

Death

Low Ve ry Lo Med w ium Me Endpo diu Med int m ium Lo Death w Low Chron ic Lo w Illness Low Acute Lo Illness w

Chron ic Illness Acute Illness

Low High Ve ry Med Lo ium Medi w um

Ver y Probability of Medi HiOccurrence Hig Hi Hig um gh h gh h

High Hi Low gh

Me Med Medi Endpo Med diu Me ium ium um Medi m int diu um m

Worry

LoDeath Med Medi Low Me w ium um Low diu m Chron Lo Loic Low Low Low w w Illness LoAcute Low w Illness Lo w

Ver y Hi Probability Hig Medi Hi of Occurrence Hig gh h Hig um gh h h Ve ry Hi Medi Hig Hi Hi Hig Lo High gh Low gh h um h w Hig gh h Me Me Med Hig Hi High Hig Hi diu Medi diu ium gh um gh h h m Med m ium Me Med Me Medi Hig Medi Med diu ium diu um h um ium m m Low Low Hi gh

2. PQ Fault T 3. PQ Fault C 4. PQ Fault D 5. PQ Fault X 6. PQ Fault M

Hig h

Worry

Me Me Medi Med diu diu um ium m m Me diu m

Hig h

Worry

Lo w

Low

Low

Med ium

46

Risk Ranking & Filtering Model

Compliance Health
(Risk Ranking and Filtering)

Scored and Prioritized Under Multiple Criteria

1. PQ Fault M 2. PQ Fault T 3. PQ Fault C

Other

4. PQ Fault D 5. PQ Fault X

6. PQ Fault A

47

Risk Analysis Cycle

Start cGMP/Compliance Inspections

Risk Assessment Assessments (Data Bases)

Risk Management Work Planning

Multi-Factorial Risk Model

Risk Ranking and Filtering

48

Conclusions

Risk Assessment provides a process for organizing information in support of risk-based decision making. Risk assessment is one of the tools available for Risk Management, the activity in which the options for controlling risks are examined in light of costs, benefits and risk trade-offs. Multifactor Risk Ranking and filtering approach might be robust enough to employ in the GMP Initiative.
49