VALIDATING RAPID MICROBIOLOGICAL METHODS

Method validation is the process used to confirm that an analytical procedure employed for a specific test is reliable,
reproducible and suitable for its intended purpose. All analytical methods need to be validated prior to their
introduction into routine use, and this is especially true for novel technology platforms, such as rapid microbiological
methods (RMMs).
Because many RMM technologies consist of a combination of instrumentation, software, consumables and reagents,
in addition to specific detection, quantitative or identification methodologies, it is important to develop a
comprehensive and holistic approach to the validation process to ensure that the entire RMM system is suitable for its
intended use. The following sections provide an overview of how to design a meaningful validation program in order
to effectively demonstrate that the new RMM is suitable for its intended use and is equivalent to, or better than, the
existing method you intend to replace.
The majority of the guidance provided on this page has been excerpted from Dr. Michael J. Millers Training
Course on RMM validation and implementation, which is now aligned with the newly revised PDA Technical Report
No. 33 (Revised 2013).

INITIAL ACTIVITIES
Prior to purchasing and validating a RMM, there are a number of due diligence activities that should be undertaken.
These can include understanding and identifying the scientific and technical benefits the RMM possesses as
compared with the existing method, regulatory impact, financial advantages (e.g., return on investment), and the
capabilities and role of the RMM supplier in terms of providing support during the initial assessment, validation, and
most importantly, after the system has been placed in service for routine use. Each of these considerations are
discussed in greater detail below.
From a scientific perspective, it is important to understand what technical capabilities are required, including, but not
limited to, method sensitivity and specificity (e.g., detection levels and for what types of microorganisms), sample
throughput, sample type, automation, data handling and archiving, report management, if the system needs to meet
21 CFR Part 11 expectations, and the required degree of operator training.
Next, proof-of-concept or feasibility testing can be performed to determine if incompatibilities exist between the RMM
and the intended product or test sample(s). These types of studies can also be performed in the event the RMM
supplier has little or no data on testing similar product or test materials. This can be accomplished using a rental or
loaner instrument, or by sending samples directly to the RMM instrument supplier for evaluation. The data obtained
from these initial studies will help with the decision to purchase the RMM and proceed with formal validation activities.
The due diligence process also involves a review of existing regulatory commitments and whether implementing the
RMM will result in significant changes that will require a formal submission to relevant regulatory bodies. Additionally,
a financial assessment of the costs (and cost savings) associated with the purchase, validation and implementation of

the RMM should be performed. Information on both of these topics can be reviewed by visiting
the Regulatory and ROI (return on investment) pages.
Finally, the selection of a rapid method supplier is just as important as the technology itself, and it is important to have
a thorough understanding of the supplier's technical capabilities and their ability to support each phase of the
validation process as well as continuing assistance once the RMM is placed into service. When deciding on a RMM
and a RMM supplier, some points to consider may include the following:

Does the supplier have a robust quality, change control and manufacturing system in place?

Do they have appropriate documentation with regard to the design and manufacture of their instrumentation?

Has the supplier been audited by other companies or regulatory agencies?

Are they financially secure?

Are they the sole provider of the RMM consumables, reagents, supplies or replacement parts?

Do they provide training programs for the end-user?

Do they provide on-site technical services, calibration and preventive maintenance programs? Can they respond
to technical issues in a timely manner?

Does the supplier provide validation protocols or similar documentation?

Have they published results of their own testing or have they submitted a Drug Master File of similar document
to a regulatory agency?

How does the supplier manage software updates and notification to the end-user?

Suppliers should be assessed to determine if they can meet your requirements. This can be accomplished through a
formal audit or supplier questionnaire.
In summary, the initial assessment of a RMM should include a comprehensive scientific, regulatory and business due
diligence review, in addition to matching the appropriate technology with the desired microbiology application. It is not
uncommon for companies that have purchased a RMM system to spend considerable time, resources, and expense
in validating the instrumentation and method, only to find at a later date that the technology is incompatible with the
process and/or product being evaluated, or that the sensitivity and/or specificity of the system is not what was
originally anticipated. Therefore, careful planning and fact finding during the due diligence phase is critical to a
successful RMM validation and implementation program.

THE VALIDATION STRATEGY

In order to design a holistic approach to RMM validation, it is necessary to develop a comprehensive strategy that
includes qualifying the RMM instrumentation, software and the analytical method. The validation plan can be
comprised of a number of process steps, which are outlined below and discussed in greater detail in the subsequent
sections.

Risk Assessment

Validation Planning

User Requirements Specification (URS)

Design Qualification (DQ)

Functional Design Specification (FDS)

Requirements Traceability Matrix (RTM)

Technology Training and Standard Operating Procedures (SOPs)

System Integration

Installation Qualification (IQ)

Operational Qualification (OQ) and Computer System Validation

Performance Qualification (PQ)

Method Validation

Method Suitability

Ongoing Maintenance and Periodic Reviews

RISK ASSESSMENT
Quality risk management (QRM) is an important part of science-based decision making. The ICH Q9 guideline,
Quality Risk Management, defines QRM as a systematic process for the assessment, control, communication and
review of risk to the quality of drug product across the product lifecycle. Similarly, the FDA Final Report for
Pharmaceutical cGMPs for the 21st Century - A Risk-Based Approach, states that using a scientific framework to find
ways of mitigating risk while facilitating continuous improvement and innovation in pharmaceutical manufacturing is a
key public health objective, and that a new risk-based pharmaceutical quality assessment system will encourage the
implementation of new technologies, including RMMs, to facilitate continuous manufacturing improvements via
implementation of an effective quality system.
A risk assessment should be performed prior to the start of any RMM validation activity. Identified risks will vary
depending on the RMM technology and the RMM supplier, the method the RMM is intended to replace, the product or
sample(s) for evaluation, whether the new measurements are qualitative or quantitative and if the resulting data are
significantly different from the existing method, method variability, method robustness, pharmacopeial equivalence,
regulatory acceptance, and other attributes.
First, the user should identify the hazards (i.e., what might go wrong when implementing the RMM), the likelihood of
occurrence, severity of harm and the ability to detect the hazard. Next, the user will analyze the risk against
predefined criteria, and determine how the risks will be addressed. Tools such as Failure Modes and Effects Analysis
(FMEA) or Hazard Analysis and Critical Control Points (HACCP) may be utilized in assessing the potential risks when
implementing the RMM.

THE VALIDATION PLAN

A validation plan should be followed which will provide the roadmap for all of the activities that will be required to
demonstrate that the RMM is validated and suitable for its intended use. The plan should include the overall project
deliverables, the organizations or individuals that are responsible for each phase of test execution, review and
approval requirements, and the documentation required to satisfy the expectations of the validation strategy. At the
end of each phase of the validation plan, a summary of the results, whether the acceptance criteria have been met,
and any deviations from the test plan should be documented and approved prior to initiating the following phase,
unless it is acceptable to run phases in parallel.

USER REQUIREMENTS SPECIFICATION (URS)

When choosing a RMM, the end-user must first establish the basic expectations that the system must meet. For
example, the system may have to detect and enumerate bacteria, fungi and spores, have a sensitivity level of a
single viable cell, process at least 80 samples within an 8-hour shift, and show (at least) equivalent results to the
current method. From here, the user can develop specific requirements for the entire RMM system, including the
instrumentation, software and the analytical method, all of which will demonstrate that the system performs as
expected. The document that describes the functions and characteristics that the RMM system must be capable of
performing is known as the URS. The requirements specified in the URS will also form the basis for all of the
validation testing requirements, test scripts/protocols and acceptance criteria. Examples of what may be included in
the URS are as follows:

The purpose of the RMM system

The scope or application of how the RMM will be used

Desired technical performance and functionality characteristics

Excluded characteristics

Hardware and software requirements

Computer system communication, data management and report needs

Safety requirements

Engineering and physical requirements

Preventive maintenance and calibration expectations

Operator qualification and training

Economic and regulatory considerations

Supplier requirements

DESIGN QUALIFICATION (DQ)

Design Qualification (DQ) is documented verification that the proposed design of the equipment or system is suitable
for the intended purpose. Because most RMMs are commercial off-the-shelf systems (COTS), DQ is accomplished
by verifying that the supplier's design specifications meet the design requirements as specified in the URS. This
activity can be completed prior to purchasing the RMM system or can be incorporated into the formal validation plan.

FUNCTIONAL DESIGN SPECIFICATION (FDS)

The FDS is the document that describes all of the functions and requirements for the RMM system and what will be
tested to ensure that the system performs as specified in the URS. The FDS can be quite extensive, covering system
functionality, configuration, input/outputs, environment, utilities, architecture, interfaces, data and security.
Additionally, the FDS will point to specific test scripts/protocols where each requirement will be evaluated and verified
against pre-established acceptance criteria. These test scripts/protocols are normally contained within the Installation,
Operational and Performance Qualification documents. Example sections and requirements within the FDS may
include, but are not limited to the following:

Purpose, scope and description of the RMM

Documentation

User manuals

Guidelines

Standards

SOPs

Physical specifications

Size

Electrical power

Voltage frequency

Operating temperature

Environmental requirements

Utility requirements

Computer system specifications

Processor, hard drive, RAM and video graphics

Network address and connections

Operating software

Printer ports

Software and algorithms

Databases

Security specifications

User ID and password

Access to data

Record retention

Audit trail

Administrative control

Data view and print reports

Data transfer to a dedicated server

21 CFR Part 11

Functional specifications

Accuracy

Precision

Specificity

Limit of detection

Limit of quantification

Linearity

Range

Ruggedness

Robustness

Equivalency

Functions that will not be used (or tested)

System customization

Alarm configuration and error handling

REQUIREMENTS TRACEABILITY MATRIX (RTM)

The RTM is a document that provides traceability that all the requirements listed in the FDS have been verified and/or
tested. Think of the RTM as a checklist of the validation process. The document identifies the test script or protocol
where a function or requirement will be tested, such as the Installation, Operational and Performance Qualification
(IQ, OQ and PQ, respectively) protocols. The RTM also specifies which SOPs and other documentation that needs to
be in place in order to satisfy the criteria for meeting a specific function or requirement. The RTM is a living document
during the execution of the validation test scripts/protocols.

TECHNOLOGY TRAINING AND SOPS

Training with the RMM supplier and the proper qualification of analysts are required for the effective execution of the
testing protocols and are critical to the success of the overall validation plan. Training may be scheduled during initial
proof-of-concept or feasibility testing, either in-house or at the supplier's facility. Many suppliers will also offer training
as part of the initial installation and commissioning of RMM instrumentation.
SOPs that facilitate the proper execution of the RMM instrumentation, as well as those that are required to be in place
as specified in the URS and FDS should be written and approved prior to the execution of the validation plan.
Examples of SOPs that may be required in order to execute the RMM validation plan may include:

System operation

Training

Calibration

System and software security

Data management, backup and recovery

Corrective and Preventive Actions (CAPA)

Preventive maintenance

Business contingency plan

Change control

SYSTEM INTEGRATION
System integration brings together all of the component subsystems into a single, operating system and ensures that
all of the components function appropriately. An example may include setting up the RMM to communicate with a
Laboratory Information Management System (LIMS). Many RMM systems are not required to be connected to an
external server or IT platform; however, if this is required by a firms IT organization, system integration testing may
be necessary.

FAT AND SAT

In some instances, it may be appropriate (or required by the end-users company) to conduct Factory Acceptance
Testing (FAT) or Site Acceptance Testing (SAT) prior to accepting the RMM instrumentation and initiating the IQ. A
FAT is performed at the RMM supplier's facility to ensure that the system meets certain design criteria prior to the
system being shipped to the end-user's site. FAT is appropriate when the end-user cannot test certain requirements
at their own facility, when custom made systems have been manufactured, or when the safety of the end-user may
be at risk. A SAT may be performed when the system arrives at the end-user's facility to ensure that the system
operates properly after shipping. In some cases, the RMM supplier may conduct initial calibration and commissioning
activities, as well as their own IQ procedures.

INSTALLATION QUALIFICATION (IQ)

The IQ establishes that the equipment is received as designed and specified, that it is properly and safely installed
with the correct utilities in the selected environment, and that the environment is suitable for the operation and use of
the equipment. Simply put, the IQ verifies that the equipment was received and meets the design specifications for
the equipment that was ordered. The IQ can be carried out by the RMM supplier or by the end-user. Examples of
what may be verified or tested during the IQ include:

Equipment delivery verification

Equipment descriptions and registration

Equipment installation

Environmental conditions

Preventive maintenance

SOPs for the operation, maintenance and calibration of the equipment

Establishment of an equipment log book

Safety checks

Required utilities

Power and wiring

Computer system capabilities

Secure server installation and communication

Computer access

Firmware and software installation and access

Data backup and recovery

OPERATIONAL QUALIFICATION (OQ)

The OQ provides documented verification that the equipment, as installed in the selected environment, performs
effectively and reproducibly as intended throughout the anticipated or representative operational ranges, defined
limits and tolerances. The OQ is also the focal point for the majority of the computer system, software and security
validation activities.
Computer system validation encompasses both hardware and software functionality and security, and demonstrates
that these components of the RMM system operate accurately and reliably. Depending on the complexity of the RMM
technology and the end-user's company policies, CSV can be quite extensive. Examples of what may be verified or
tested can include the following:

Administrator control and operator access

User ID and password set up

User and system lockout

Data archiving and access

Audit trails

Report generation

Data transfer and server communication

Electronic signatures, 21 CFR Part 11

Data backup and recovery

Database management and integrity

Interference (radio frequency, electromagnetic, wireless)

PERFORMANCE QUALIFICATION (PQ)

The PQ confirms that the instrumentation, as installed, consistently performs in accordance with predetermined
criteria and thereby yields correct and appropriate results. Two very significant phase of the validation plan are
performed during the PQ: validation of the microbiological method and method suitability testing.
Validation of the microbiological method first involves the use of an appropriate selection of standardized microbial
suspensions in a suitable diluent and/or other target material in order to demonstrate that specific validation criteria
can be met. Separately, the RMM is demonstrated to be equivalent to the existing method using actual test samples
or product. Method suitability demonstrates that the presence of product, test material or the sample matrix does not
significantly interfere with the performance of the RMM. Examples of interference may include background noise,
false positives and/or false negatives.
When validating the method using standardized microorganisms, guidance on test procedures and acceptance
criteria have historically come from three documents:

PDA Technical Report No. 33, Evaluation, Validation and Implementation of Alternative and Rapid
Microbiological Methods

United States Pharmacopoeia Informational Chapter <1223>, Validation of Alternative Microbiological Methods

European Pharmacopoeia Chapter 5.1.6, Alternative Methods for Control of Microbiological Quality

The most up-to-date validation guidance may be found in PDA TR No. 33. Revised in 2013, TR33 incorporates the
latest information on testing strategies, the use of statistics, acceptance criteria and industry best practices. Both the
USP and Ph. Eur. chapters are undergoing revisions, with drafts expected sometime in 2014; however, until the
drafts are available for public review and comment, it is not possible to determine how similar (or different) the
chapters will be as compared with PDA TR33. We will update this page as the compendia drafts are made
available.
A brief review of the validation criteria from each guidance document is provided below. Please note that there are
differences in how the validation criteria and acceptance criteria are employed in EACH document, and we expect the
USP and Ph. Eur. requirements will significantly change with their next revision. In the meantime, many companies
now follow the revised PDA TR33 to meet their internal and external regulatory expectations when validating RMMs
(REMINDER: always check with the relevant regulators before embarking on a validation program, as appropriate).
Validation Criteria for Quantitative Tests

PDA TR33

USP <1223>

Ph. Eur. 5.1.6

Revised 2013 Under Revision Under Revision

Accuracy
Precision
Specificity
Limit of Detection
Limit of Quantification
Linearity
Range
Ruggedness
Robustness
Equivalence or
Comparative Testing

Validation Criteria for Qualitative Tests

PDA TR33

USP <1223>

Ph. Eur. 5.1.6

Revised 2013 Under Revision Under Revision

Accuracy
Precision
Specificity
Limit of Detection
Ruggedness
Robustness
Equivalence or
Comparative Testing

Validation Criteria for Microbial Identification Tests

PDA TR33

USP <1223>

Ph. Eur. 5.1.6

Revised 2013 Under Revision Under Revision

Accuracy
Precision
Robustness
Equivalence or
Comparative Testing

The following definitions and recommendations are aligned with PDA TR33. Similarities may exist with the USP and
Ph. Eur. chapters.
Accuracy is the closeness of the actual test results obtained by the new method to the actual test results obtained by
the existing method (e.g., plate count). Accuracy is demonstrated across the practical range of the test, and is usually
expressed as the percentage of recovery of microorganisms. Accuracy is usually assessed for quantitative methods,
although microbial identification systems requires Accuracy testing as well (see below).
Precision is the degree of agreement among individual test results when the procedure is applied repeatedly to
multiple samplings of the same suspension of microorganisms and using different suspensions across the range of
the test. Precision is associated with the use of the method within the same laboratory over a short period of time
using the same analyst with the same equipment (also referred as repeatability, within-run variability or intra-assay
precision). Precision is usually expressed as the standard deviation or coefficient of variation of a series of
measurements.
Specificity is the ability to detect a range of microorganisms which demonstrates that the method is fit for its
intended purpose. Specificity is usually assessed during testing of the relevant validation criteria (e.g., Accuracy) for
both quantitative and qualitative methods. Inclusivity and exclusivity testing may also be included for this methods
that are designed to specifically detect a target organism and exclude all others. Organisms may be sourced from
culture collections (e.g., ATCC), environmental or facility isolates, in-process or sterility failure isolates, slow-growing,
fastidious or anaerobic strains, and/or clinically relevant cultures.
Limit of Detection (LOD) is lowest concentration of microorganisms in a test sample that can be detected, but not
necessarily quantified, under the stated experimental conditions. The test determines the presence or absence of
microorganisms in the original sample (e.g., sterility test).
Limit of Quantification (LOQ) is the lowest number of microorganisms in a test sample that can be enumerated with
acceptable accuracy and precision under the stated experimental conditions.

Linearity is the ability to elicit results that are proportional to the concentration of microorganisms present in the
sample within a given range, where accuracy and precision are demonstrated .
Range is the interval between the upper and lower levels of microorganisms that have been demonstrated to be
determined with Accuracy, Precision and Linearity.
Ruggedness is the degree of intermediate precision or reproducibility of test results obtained by assessing the same
samples under a variety of normal test conditions, such as different analysts, different instruments, different lots of
reagents or on different days. Intermediate precision is performed within the same laboratory, and reproducibility is
performed between laboratories. Ruggedness is best suited to be determined by the supplier of the test method who
has easy access to multiple instruments and batches of components; however, similar studies may be conducted by
the end user.
Robustness is measure of a methods capacity to remain unaffected by small but deliberate variations in method
parameters and provides an indication of its reliability during normal usage. Robustness is best suited to be
determined by the supplier of the test method who has easy access to multiple instruments and batches of
components; however, similar studies may be conducted by the end user.
Equivalence or Comparative Testing is the level of agreement in accuracy, precision, specificity, limit of detection,
limit of quantification, linearity and/or range between the existing and the new method. This is initially demonstrated
using standardized microbiology cultures (see above) and separately using actual product and other sample matrices
that will be routinely tested using the new method once it is validated and implemented. However, prior to testing with
actual product or test samples, these materials should be assessed for their potential to cause background noise,
interference, false positive or false negative results (see Method Suitability below). Test samples should be identified
that are expected to contain microorganisms in order to demonstrate that the new method will detect microorganisms
similarly as the existing method. Furthermore, low levels of microorganisms may need to be inoculated into test
samples in order to conduct the evaluation for equivalency (e.g., when using a sterile product to demonstrate the
LOD for a rapid sterility test). The new method should be run in parallel with the existing method for a specified period
of time or number of product batches or test samples (the end-user should determine the most appropriate strategy
for the duration and extent of these studies).
NOTE TO READERS:
Detailed validation procedures, specific acceptance criteria and recommended statistical models are NOT
provided on this webpage.
Readers should consult with a relevant guidance document (PDA TR33, USP and/or Ph. Eur.) for this
information.
Method Suitability
Each test material should be evaluated for the potential to produce interfering or abnormal results, such as false
positives (a positive result when no viable microorganisms are present) or false negatives (a negative result when
microorganisms are present). This may also include evaluating whether cellular debris, dead microorganisms or

mammalian cell cultures have any impact on the ability of the new method to operate as it is intended to. If false
positives or negatives cannot be resolved, then the test sample may be incompatible with the new method.
Validation of Microbial Identification Systems
New or rapid microbial ID systems are usually tested for Accuracy and Precision (Repeatability). The end-user should
establish suitable acceptance criteria for each, taking into account the ID methods specific capabilities. For example,
the ID system may not be able to identify an isolate because the organism is not included in the database, the system
parameters are not sufficiently comprehensive to identify the organism, the isolate may be nonreactive in the system,
or the species may not have been taxonomically described.

ONGOING MAINTENANCE AND PERIODIC REVIEWS

Following equipment verification, software validation and method validation as described above, procedures should
be established to maintain the RMM system in a validated state. This wold also include a periodic review of system
performance and compliance with cGMPs.

IMPLEMENTATION AND SECONDARY SITE QUALIFICATION

Once the validation plan has been executed and approved, the RMM may be implemented for routine use. In the
case where the RMM will be implemented at a location other than where it was originally validated, it is usually not
necessary to repeat the same VMP at the secondary site. In this case, a copy of the original validation package can
be provided, and a reduced testing plan developed for the installation of the identical RMM system at the secondary
site.
A standard equipment IQ and OQ can be performed that includes basic functionality and computer system testing.
The original validation package can be used as a reference/template for what is expected during the IQ and OQ for a
particular system. Furthermore, the original hardware/software security configuration testing may not need to be
repeated unless there will be systems in use at a secondary site that were not evaluated in the initial test plan. An
example might be the use of a different data handling or archiving platform (e.g., LIMS, LAN server, external drive)
than what was originally qualified. Each new installation, or technology transfer, must be separately evaluated to
determine the extent of additional IQ and OQ testing that may need to be performed.
Because an exhaustive microbiological testing plan will be completed during the PQ at the initial qualification facility,
it is not necessary for the secondary facility to repeat this testing in its entirety when a like-for-like instrument is
installed. The original validation package can be used to support this strategy; however, there should be limited
microbiological challenges performed to demonstrate that the system is performing as intended. For example, a few
reference organisms, identical to what was used during the original qualification, may be used to confirm basic
functionality and demonstrate that key qualification requirements are met (e.g., Accuracy and Precision).
A review of locally recovered microbial isolates should indicate if the reference strains used during the original PQ are
representative of the isolates recovered at that site. If it is determined that they are not, then the local qualification

plan should include a list of microbial isolates to be evaluated and what qualification requirements they will be tested
against.
Next, if the originating qualification did not include the actual product and/or process material that the site will be
evaluating routinely, then these materials must be evaluated during the local PQ where the impact of sample material
on the test method is assessed (i.e., during Method Suitability studies). The site will then determine the nature of the
test plan to provide meaningful data about the ability of the RMM system to operate as it is intended. This may
include some required number of batches, replicates, location of sampling points, length of study, etc. Additionally,
sufficient data will be required to evaluate whether the data from the RMM is statistically greater than the method
being replaced, and whether the data warrants a modification to baseline acceptance levels or product specifications
(see additional information on changing acceptance criteria).

ASSISTANCE IN DEVELOPING YOUR RMM VALIDATION PLAN

The development of a meaningful and comprehensive validation strategy can be a significant undertaking. For
assistance in designing your own RMM validation plan, ourConsulting Team is available to support your company's
needs, in addition to providing in-house training sessions on all aspects of RMM validation and implementation.