Acceptance of Commercial Grade

Computer Code for Use as a

Basic Component in Safety
Related Applications

Marc Tannenbaum
Project Manager
NUPIC Vendor Meeting
June 15, 2011
 Background

• EPRI is currently developing guidance in response to

NRC comments at NUPIC meetings and ASQ meetings
– Verification and validation is no longer enough
– Dedication methodology should be applied
• The Technical Advisory Group includes:
– Auditors (NUPIC)
– Utility procurement engineers (JUTG)
– Utility software experts (NITSL)
– ASME NQA-1 Software Subcommittee
• EPRI is working with NRC through NEI

© 2011 Electric Power Research Institute, Inc. All rights reserved. 2

 Background

• Existing EPRI reports do not address dedication of

computer code that is not integral to plant structures,
systems, and components (SSCs)
– EPRI NP-5652
– EPRI TR-102260
– No content specific to software

ASC American Society for Quality

EPRI Electric Power Research Institute
JUTG EPRI Joint Utility Task Group
NEI Nuclear Energy Institute
NRC U.S. Nuclear Regulatory Commission
NUPIC Nuclear Procurement Issues Committee
SSC Systems, Structures, Components

© 2011 Electric Power Research Institute, Inc. All rights reserved. 3

 Background

• Several existing EPRI reports do address acceptance of

commercial grade digital devices integral to plant SSCs
– EPRI TR-106439, Guideline on Evaluation and
Acceptance of Commercial Grade Digital Equipment for
Nuclear Safety Related Applications (October 1996)
– EPRI TR-107339, Evaluating Commercial Digital
Equipment for High-Integrity Applications: A
Supplement to EPRI Report TR-106439 (December
1997)
• Methodology is helpful as digital devices include computer
programs

© 2011 Electric Power Research Institute, Inc. All rights reserved. 4

 Background

• Focus of EPRI guidance currently in development

– Acceptance of computer code (not integral to SSCs
installed in the plant)
• Objectives
– Provide guidance on how to perform safety
classification of software
– Provide guidance on how to accept computer code
using methodology that finds basis in commercial grade
item dedication methodology
– Obtain regulatory acceptance

© 2011 Electric Power Research Institute, Inc. All rights reserved. 5

 Target Audience for EPRI Guidance

• Utilities
– Many accept software using a full-scale software quality
assurance (SQA) program that may be more robust
than the commercial grade dedication process
– Change may be necessary in light of NRC’s position

• Suppliers

• Auditors

© 2011 Electric Power Research Institute, Inc. All rights reserved. 6

 Commercial Grade Dedication Fundamentals

Commercial Grade Item

Technical Evaluation
+
Acceptance Process

Process to ensure Process to produce

technical objective evidence
requirements for the providing reasonable
item are “specified” assurance the
correctly commercial grade
item received is the
(Design was already item specified
completed and
qualified)
© 2011 Electric Power Research Institute, Inc. All rights reserved. 7
 Commercial Grade Dedication Fundamentals

Commercial Grade Item

Technical Evaluation
+
Acceptance Process

Together, these two processes contribute

to assuring the purchased item will
perform its safety-related function(s)
as defined by the dedicating entity

© 2011 Electric Power Research Institute, Inc. All rights reserved. 8

 Technical Evaluation for Items

• Identify Application / End Uses

• Perform safety classifications
– Identify Function(s)
– Failure modes and effects analysis
• Identify Critical Characteristics
– Physical (dimensions, materials, configuration, etc.)
– Performance (resistance, closing time, input/output, etc.)
• Select Acceptance Methods and Criteria
– Test & Inspect, Survey, Surveillance, Historical
Performance

© 2011 Electric Power Research Institute, Inc. All rights reserved. 9

 Pop Quiz!

• How many of you use software to design or analyze

safety-related SSCs?

• How many of you have a formal process for accepting the

computer code for use?

• How many of you employ “verification” and “validation”

(V&V) in your acceptance process for computer code?

• What fundamental part of the commercial grade

dedication process might need more attention in your
process?

© 2011 Electric Power Research Institute, Inc. All rights reserved. 10

 Commercial Grade Dedication Fundamentals

Commercial Grade Item

Technical Evaluation
+
Acceptance Process

Item purchased as a
basic component from a
Dedicated supplier with a QA program
Commercial
Grade Item = meeting the requirements
(In quality) of10CFR50, Appendix B or
ANSI/ASME NQA-1

© 2011 Electric Power Research Institute, Inc. All rights reserved. 11

 Technical Evaluation

• Identify software being procured

– What software am I acquiring?

• Identify end use application(s)

– How will the software be used?
• What types of calculations / modeling is being
performed
• Can it impact safety related SSCs?

© 2011 Electric Power Research Institute, Inc. All rights reserved. 12

 Technical Evaluation

• Determine if the computer code performs a safety function

– What happens if it fails? (Failure modes and effects)
– Can failure impact the safety function(s) of the SSCs?
• Can information obtained from the computer code
result in failure of the SSC to perform its safety
function(s)
– Is it the sole basis for design/analysis decisions?
• Identify safety function(s) of the computer code
• What does the computer code have to do to ensure
associated SSCs will be capable of performing their
safety related function(s)
© 2011 Electric Power Research Institute, Inc. All rights reserved. 13
 6WDUW


6RIWZDUHLV6DIHW\5HODWHG
,VVRIWZDUHLQWHJUDOWRDVDIHW\ 6RIWZDUHLV1RQVDIHW\5HODWHG
<HV )ROORZJXLGDQFHLQ75WR
UHODWHG66&" 'RFXPHQW5HVXOWVRI(YDOXDWLRQ
GHGLFDWHGLJLWDOGHYLFHV

Computer 1R
1R

Code Safety 

&DQVRIWZDUHLPSDFWVDIHW\ 1R

,VVRIWZDUH
XVHGLQDZD\WKDWVXSSRUWV

Classification
UHODWHG66&V" TXDOLW\SURJUDPUHTXLUHPHQWV"
EXWLVQRWDEDVLF
FRPSRQHQW

Process <HV

 
<HV

,VVRIWZDUHXVHGWR 6RIWZDUHLV1RQVDIHW\5HODWHGRU
GHVLJQRUDQDO\]H66&VLQD $UHWKHUHVXOWVGHULYHGXVLQJ $XJPHQWHG4XDOLW\,PSRUWDQWWR3ODQW
<HV <HV
ZD\WKDWFRXOGLPSDFW66& VRIWZDUHLQGHSHQGHQWO\ 6DIHW\ 'RFXPHQW5HVXOWVRI
VDIHW\IXQFWLRQV YHULILHGIRUHYHU\XVH" (YDOXDWLRQ

1R



,GHQWLI\VDIHW\UHODWHGIXQFWLRQVRIWKH 1R
DVVRFLDWHG66&V


3RVWXODWHIDLOXUHVRIWKH6RIWZDUHWKDW
FRXOGSUHYHQWDVVRFLDWHG66&VIURP
SHUIRUPLQJWKHLUGHVLJQHGVDIHW\
UHODWHGIXQFWLRQV

1R
 
&RXOGIDLOXUHRIWKH ,VVRIWZDUH <HV
VRIWZDUHDGYHUVHO\LPSDFWWKH XVHGWRDVVHVVWKHDELOLW\RI
1R <HV
DELOLW\RIWKH66&WRSHUIRUPLWV 66&VWRSHUIRUPWKHLUVDIHW\
VDIHW\IXQFWLRQ" UHODWHGIXQFWLRQ"

1R
<HV


6RIWZDUHLV6DIHW\5HODWHG ,VVRIWZDUH
'RFXPHQW5HVXOWVRI(YDOXDWLRQ XVHGWRPRQLWRURSHUDWLRQDQG
FRQWUROIXQFWLRQVRI66&V"
 Technical Evaluation

• Identify Critical Characteristics for Acceptance

– Important characteristics of the software that once
verified will provide reasonable assurance that the
computer code will perform its intended safety
function(s)
• Critical characteristics must be based upon the
application(s) and safety function(s) documented in the
technical evaluation
• Currently available references:
– EPRI TR-107339
– EPRI TR-106439 (www.epri.com)

© 2011 Electric Power Research Institute, Inc. All rights reserved. 15

 Technical Evaluation

• Physical Characteristics (Examples)

– Media type

• Performance Characteristics (Examples)

– Consistency
– Accuracy
– Compatibility with operating environment

• Dependability Characteristics (Examples)

– Built in Quality
– Quality of Design

© 2011 Electric Power Research Institute, Inc. All rights reserved. 16

 Technical Evaluation

• Identification
– “Part” number
– Version/build number

• Establish Boundaries
– Acceptance must envelope scope of use
– Applicable functions, environments, etc.

© 2011 Electric Power Research Institute, Inc. All rights reserved. 17

 Technical Evaluation

• Select Acceptance Methods and Tolerances

– Test and Inspection
– Commercial Grade Survey
– Source Surveillance
– Historical Performance (restrictions in GL 89-02)

• Ensure acceptance bounds intended use

– Acceptance must envelope scope of use
– All applicable functions and range of input variables
– Environments, etc.

© 2011 Electric Power Research Institute, Inc. All rights reserved. 18

 Technical Evaluation

•Document the technical evaluation

End-Use
Applications
SSCs Critical

Safety Function Characteristics

FMEA Acceptance
Methods

Results of
Acceptance
Activities

© 2011 Electric Power Research Institute, Inc. All rights reserved. 19

 

,GHQWLI\VRIWZDUHEHLQJSURFXUHG
GHVLJQDQGTXDOLILFDWLRQDFWLYLWLHVDUH
FRPSOHWH

 
'RHVVRIWZDUHSHUIRUPDVDIHW\ 3URFXUHVRIWZDUHQRQVDIHW\UHODWHG
1R
IXQFWLRQ"

<HV

 

7HFKQLFDO(YDOXDWLRQ
'RHVVRIWZDUHPHHWWKH 3URFXUHVRIWZDUHDVDEDVLF
1R
GHILQLWLRQRIDFRPPHUFLDOJUDGH FRPSRQHQW
LWHP"

Computer Code <HV

Acceptance 

,GHQWLI\&ULWLFDO&KDUDFWHULVWLFVIRU

Process $FFHSWDQFH



'RFXPHQW6DIHW\)XQFWLRQ)DLOXUH
0RGHVDQG(IIHFWV$QDO\VLV&ULWLFDO
&KDUDFWHULVWLFV



$FFHSWDQFH
6HOHFW$FFHSWDQFH0HWKRGV

   

0HWKRG 0HWKRG 0HWKRG 0HWKRG

6SHFLDO7HVWVDQG,QVSHFWLRQV 6XUYH\RI&RPPHUFLDO9HQGRU 6RXUFH9HULILFDWLRQ ,WHP9HQGRU3HUIRUPDQFH
5HVWULFWLRQVLQ*/



&RQGXFW$FFHSWDQFH$FWLYLWLHVDQG
HYDOXDWHUHVXOWV
 Relationship of Design and Acceptance for
Software

Qualification Technical N
Design of TIO &
evaluation
A
design DIC
acceptance
E
D
activities

Figure 1 - Typical process flow for hardware items

Requirements

Install/Check

Maint,/Op’s
Technical evaluation and

Retirement
Concept

Design

acceptance activities ON
Test

Design, ATI
I C
qualification of design, DED
maintenance & updates

Figure 3
2– Overlaying
- Typical dedication
process techniques
flow (lifecycle) for to accept software
software

© 2011 Electric Power Research Institute, Inc. All rights reserved. 21

 Relationship of Design and Acceptance for
Software

Requirements

Install/Check

Maint,/Op’s

Retirement
Concept

Design

Test
Figure 2 - Typical process flow (lifecycle) for software
Requirements

Install/Check

Maint,/Op’s
Technical evaluation and

Retirement
Concept

Design

acceptance activities
Test

Design,
qualification of design,
maintenance & updates

Figure 3 – Overlaying dedication techniques to accept software

© 2011 Electric Power Research Institute, Inc. All rights reserved. 22

 Selection of Acceptance Methods

High

Access to Design Process and Info

More reliance on Balanced reliance on
CG survey & Testing & inspection
source inspection and
Less reliance on CG survey &
testing & Inspection source inspection

More reliance on
Seek Alternate
testing & inspection
Computer Code
Less reliance on
(Reasonable assurance
CG survey &
can not be achieved)
Source inspection
Low
Ability to Test and Inspect
Low High
© 2011 Electric Power Research Institute, Inc. All rights reserved. 23
 Questions?

© 2011 Electric Power Research Institute, Inc. All rights reserved. 24