WANO REPORT

RPT ǀ 2013-5 July 2013

Analysis of Emergency Diesel Generator
Events

LIMITED DISTRIBUTION
This page is left blank intentionally
LIMITED DISTRIBUTION WANO RPT 2013-5

APPLICABILITY

THIS WANO REPORT APPLIES TO ALL REACTOR TYPES

LIMITED DISTRIBUTION

Confidentiality notice
Copyright 2013 World Association of Nuclear Operators (WANO). All rights reserved. Not for sale. This document is protected as an unpublished work under the
copyright laws of all countries which are signatories to the Berne Convention and the Universal Copyright Convention. Unauthorised reproduction is a violation of
applicable law. Translations are permitted. This document and its contents are confidential and shall be treated in strictest confidence. In particular, except with the
prior written consent of the WANO London, this document shall not be transferred or delivered to any third party and its contents shall not be disclosed to any third
party or made public, unless such information comes into the public domain otherwise than in consequence of a breach of these obligations.

Liability disclaimer notice

This information was prepared in connection with work sponsored by WANO. Neither WANO, WANO members, nor any person acting on the behalf of them (a) makes
warranty or representation, expressed or implied, with respect to the accuracy, completeness, or usefulness of the information contained in this document, or that use
of any information, apparatus, method or process disclosed in this document may not infringe on privately owned rights, or (b) assumes any liabilities with respect to the
use of, or for damages resulting from the use of any information, apparatus, method, or process disclosed in this document.

WWW.WANO.ORG i
LIMITED DISTRIBUTION WANO RPT 2013-5

Report ǀ RPT 2013-5

Revision History

Author Date Reviewer Approval

Reason for Changes:

WWW.WANO.ORG ii
LIMITED DISTRIBUTION WANO RPT 2013-5

REPORT ǀ RPT 2013-5

Analysis of Emergency Diesel Generator Events

CONTENTS
Analysis of Emergency Diesel Generator Events 2
Introduction 2
Organisation and administration 7
Equipment performance 8
Engineering 8
Maintenance 9
Examples of Events 10
Related to Design/Modification issues 10
Related to Maintenance Issues 13
Relate to OE not Properly Applied 17
Related to Common Cause Failure 18
Examples of Events with Improper Operation or Component Failures 18
Gaps in Excellence 20
Recommendations 22
References 23

WWW.WANO.ORG 1
LIMITED DISTRIBUTION WANO RPT 2013-5

REPORT ǀ RPT 2013-5

Analysis of Emergency Diesel Generator Events

Introduction

A review of emergency diesel generator (EDG) events was initiated in 2013 because of a noticeable decline
in EDG performance. Out of the 878 WANO event reports (WERs) classified as Significant, Noteworthy or
Trending since 1 July 2012, 56 events (6.37%) were related to EDGs performance and/or reliability. The
number of EDG-related events reports increased from 31 in 2010 to 48 in 2012 and the first six months of
2013 showed a further increase in event numbers. In addition, a review of peer review areas for
improvement (AFIs) showed EDG vulnerability to failures was apparent at most stations reviewed and EDGs
were consistently ranked as one of the most risk-significant components at the stations. A noticeable
decline in EDG performance was also identified by INPO and subsequently IER L4-13-21 was issued, based
on information obtained during emergency diesel generator review visits (EDGRVs) initiated in 2011. This
information, indicating an adverse event trend, vulnerability to failure and decline in performance,
prompted the WANO operating experience central team (OECT) to further analyse EDG-related events and
performance.

The objectives of the analysis of EDG performance and related operating experience were to identify gaps
in excellence and make recommendations to correct these identified gaps.

Industry Event Analysis

Emergency diesel generator related events reported to WANO from 2009 to June 2013 were analysed. Only
events classified as Significant, Noteworthy or Trending and affecting an EDG’s availability or reliability
were analysed. The EDG-related events include:

 failure of the EDG to start on demand or test

 failure of an EDG while running on demand or during testing

 required to stop an EDG while running (during actual demand or testing)

 deficiencies revealed during EDG inspection

 deficiencies revealed while addressing EDG operating experience lessons learned

Figure 1 and Table 1 on the next page, shows the number of EDG-related events (Significant, Noteworthy or
Trending) reported by regional centre. The ‘year’ shown in the figure is the year when the event occurred.
The number of events reported to WANO has increased from 30 reports in 2009 to 48 in 2012. In the first
six months of 2013, 26 events were reported, nearly equalling the 27 events reported in the last six months
of 2012.

WWW.WANO.ORG 2
LIMITED DISTRIBUTION WANO RPT 2013-5

Figure-1 Emergency Diesel Generator Events

60

50
Number of Events

40
TYO
30
PAR
20 MOW
ATL
10

0
2009 2010 2011 2012 2013
Years Event Occurred

Year/Centre AC MC PC TC Year Total

2009 7 4 12 7 30

2010 10 3 13 5 31

2011 20 5 11 11 47

2012 23 3 11 11 48

June 2013 13 1 9 3 26

Centre Total 73 16 56 37 182

Table 1

WWW.WANO.ORG 3
LIMITED DISTRIBUTION WANO RPT 2013-5

Figure 2 presents the percentage of (Significant, Noteworthy and Trending) EDG events reported to WANO,
against the total number of events similarly classified. The ‘year’ shown in the figure is the year when the
event occurred. The percentage of EDG events reported to WANO increased from 4.08% in 2009 to 6.40%
in the first half of 2013.

Figure- 2 Emergency Diesel Generator Events

2013

2012
year

2011

2010

2009

0 1 2 3 4 5 6 7

Percentage of total events

Information from the OE database shows that from 1 July 2012 to 30 June 2013, 83 events reported were
related to EDG.

Figures 3 to 7 represent the distribution of these 83 events by:

 category

 consequences

 components failed

 direct cause

 root cause

WWW.WANO.ORG 4
LIMITED DISTRIBUTION WANO RPT 2013-5

Figure 3 shows the distribution of categories under which the events were reported. Of these, 24% were
due to safety system malfunction or improper operation, 22% due to maintenance practices and 21% due
to deficiencies of design, fabrication, construction and installation.

Severe or unusual
Figure-3 Category of Events station transient
3%

Other events
Deficiencies observed involving station
Safety system
safety or reliability
during analysis
17% malfunctions or
4%
improper operations
Deficiencies observed 24%
in procedures, testing
or training
7%
Major equipment
damage
Deficiencies Deficiencies
in of
2%
design,
maintenance fabrication,
22% construction and
installation
21%

Figure 4 shows the distribution of the consequences resulting from the EDG events. The consequence for
46% of the events was degradation of safety systems, either directly or indirectly. A few of the events
required a lengthy outage to repair the damaged EDG.

Station
transient Figure-4 Conseqences of the Events
4% Equipment damage
2%

Non-consequential or
near miss
33%
Degradation of safety
systems, such as
emergency power
46%
Other
15%

WWW.WANO.ORG 5
LIMITED DISTRIBUTION WANO RPT 2013-5

Figure 5 shows the distribution of the failed or affected components involved in causing an event. Of these,
51% of the events were due to a mechanical component problem and 41% were the result of problems
with electrical components.

Figure-5 Components Failed

computer Instrument
1% 7%

electrical
41%
mechanical
51%

Figures 6 and 7 illustrate the distribution of direct and root causes for a reported event (note that the
causal factors are incorporated with the root causes). Each cause code was assigned according to the
coding system provided in the WANO Operating Experience Programme Reference Manual.

Figure-6 Distribution of Direct Causes

Environment Not identified
5% 1% Direct Cause and
Frequency
Mechanical deficiency (36)

Hydraulic and
pneumatics Human factor Electrical deficiency (24)
4% 14%
Mechanical Instrumentation and
deficiency control (5)
42%
Hydraulic and pneumatics
(4)
Human factor (12)

Environment (4)
Electrical deficiency
Instrumentation 28%
and control Not identified (1)
6%

WWW.WANO.ORG 6
LIMITED DISTRIBUTION WANO RPT 2013-5

Figure-7 Distribution of Root Causes

Root Cause and
Frequency
Human performance
(26)

Equipment Management-
performance related (5)
28%
Human performance Equipment design
29% configuration (7)

Equipment
Maintenance/testing specification (13)
/surveillance
16% Management-related Maintenance/testing
5% /surveillance (14)
Equipment
specification Equipment design Equipment
14% configuration performance (25)
8%

Recent WANO peer reviews have identified AFIs based on EDG performance and condition. While some
problems identified in AFI are station- or unit-specific, other problems have broader implications across the
industry, relating to failure or reduced availability of EDGs. Examples of AFI-related causes identified from
peer review reports are as follows:

Organisation and administration

 Management personnel tolerate degraded station equipment. Some deficiencies have existed since
the start of commercial operation without being effectively addressed and create a greater potential
for contributing to or causing events.

 Personnel exhibit a lack of questioning attitude toward existing and emergent equipment issues or
adverse plant conditions. This has led to a delay in identifying the root causes and corrective actions
for problems, such as one EDG that had a startup time longer than specified during some of the tests
performed.

 Personnel are sometimes not sensitive to restoring important equipment to service in a timely
manner. In one situation, EDG testing extended over a six-day period during a seven-day limiting
condition for operation (LCO), without an owner for the activity being assigned.

 Personnel did not recognise the risk of performing a surveillance test on equipment that rendered one
EDG inoperable while the other EDG was unavailable. If the second EDG had become unavailable
because of the testing, outage risk would have changed from the lowest risk classification to the
highest risk classification without contingency plans in place.

WWW.WANO.ORG 7
LIMITED DISTRIBUTION WANO RPT 2013-5

Equipment performance

 Emergency diesel generators experience a high start and run failure rate because of age-related
subcomponent equipment failures. This has resulted in unplanned unavailability of EDGs, additional
start demands on the redundant EDG unit and increased vulnerability for a unit shutdown.
Contributing is a lack of management oversight to ensure timely implementation of EPG improvement
projects and interim reliability strategies.

 Reliability of EDGs has degraded. One station experienced six unplanned LCO entries in an
18-month period because of failures during EDG testing. In three cases, an EDG failed to start or
tripped during a routine test. In two cases, an EDG was declared unavailable because of emergent
maintenance work. In one case, an EDG tripped during a routine test and the backup EDG also tripped.

 Failures of EDGs have occurred during testing and in response to a valid demand signal and managers
did not request that reasons for their failure or the underlying fundamental cause for the failure to be
determined.

 Some stations are not testing for biodiesel at the point of delivery or in storage systems, potentially
adversely affecting the performance of the EDG’s fuel system.

Engineering

 A long-standing issue with unavailability or procurement delays for some key spare parts, equipment
and technical services has not been successfully resolved. This issue is one of the major contributors to
degraded equipment condition and unavailability. Reliability of EDGs has also degraded because of the
unavailability of technical services from the vendor for overhaul and refurbishment.

 Core engineering functions, such as trending important system parameters, conducting effective
walkdowns and maintaining system health reports, are not being completed for some important
safety-related systems. This deficiency contributed to an EDG trip because of water in the lubricating
oil, a problem that should have been identified through monitoring and trending of lube oil
parameters.

 Some stations have not implemented a comprehensive strategy for identifying and maintaining critical
spare parts for large equipment, such as EDGs. This includes diesel engine parts, such as crankshafts,
power assemblies and camshafts.

 Safety equipment status reports, trending and obsolescence are not effectively tracked and
monitored, which has resulted in safety systems or components, such as EDGs exceeding unavailability
criteria.

 Station personnel have not been active in owners groups (OGs) related to EDG issues. This represents
a missed opportunity to share current operating experience, pool resources to address common
issues, network with experienced EDG personnel and build strong relationships with various the EDG
vendors.

 Inaccurate technical evaluation assumptions and acceptance of degraded conditions for EDGs
contributed to several instances of inoperability. For example, an engineering evaluation of an
auxiliary switch for an EDG was incorrect, resulting in diesel generator additional inoperability time.
Contributing was that the engineering manager did not challenge some important evaluations to
ensure high quality and standards are maintained for engineering judgments and assumptions.

WWW.WANO.ORG 8
LIMITED DISTRIBUTION WANO RPT 2013-5

 In many cases, important on-line work is not ready for execution when scheduled. This has resulted in
unavailability and rework of safety systems being extended. Contributing are insufficient walkdowns
and lack of preparation by the work groups involved. For example, during an EDG outage, an
inadequate walkdown contributed to personnel not identifying that the heat exchanger end bell would
not align with the associated piping. As a result, the scheduled duration for the safety system window
outage was exceeded by 14 hours.

 No evaluation on seismic concern has been performed on the temporary cooling units for EDG control
panels.

Maintenance

 Two EDG failures occurred in 2010 because of foreign material in the diesels. In one case, a nut was
found in the cylinder head and, in the other case, a piece of cloth was found in the oil pan.

 Several major components, such as EDG generators and turbochargers are not included in the PM
programme or are not sufficiently addressed.

 Electricians and instrument and control technicians often do not use proper verification practices to
validate correct performance of work when lifting leads and manipulating electrical components. For
example, when replacing an EDG fuel oil reed switch, it was soldered to the wrong side of the relay
coil. When the circuit was energised, the associated fuses blew, resulting in an additional six hours of
EDG inoperability while the connection was reworked.

WWW.WANO.ORG 9
LIMITED DISTRIBUTION WANO RPT 2013-5

REPORT ǀ RPT 2013-5

Examples of Events

Related to Design/Modification issues

Station: Laguna Verde 2

Reference: WER ATL 12-0720

Event date: 12 September 2012

While performing a surveillance test on an EDG, alarms were received in the control room indicating
problems in the EDG. Control room operators generated an emergency trip signal but the generator failed
to stop. An auxiliary operator then cut the fuel supply by closing the fuel supply valves and causing the EDG
to trip. White smoke was also present in the diesel generator room; therefore, the fire brigade was
activated. A subsequent investigation revealed extensive damage to the EDG.

Contributor: There was a total loss of lubrication to bearings because of blockage to the piston holes. The
blockage was caused by the wearing of the piston bearing’s silver coating. As the silver coating degraded,
the lubricating veins also degraded, resulting in a lack of proper lubrication.

Lesson Learned

 The piston bearing wear can be monitored by the quantity of silver found in the lubricating oil, but the
station lacked a process to perform monitoring and trend analysis of the silver concentration in the
lubricating oil.

 Operating an unloaded EDG contributed to the degradation of the power assembly. The vendor
manual and operating procedures both have cautions stating not to operate the machine unloaded for
more than 15 minutes and instructions to apply at least 25% of its nominal load as soon as possible.
These recommendations were not considered in the station’s special instructions.

 According to the analysis, the lubricating oil used in EDGs from both units had a zinc concentration of
between 0 and 10ppm. Zinc chemically reacts with the silver coating of the piston bearing, causing it to
degrade.

 If an EDG high crankcase pressure emergency trip signal had existed, the machine would have stopped
automatically before the failure progressed to the point where major damage occurred.

WWW.WANO.ORG 10
LIMITED DISTRIBUTION WANO RPT 2013-5

Station: Cruas 4

Reference: EAR PAR 11-0007

Event date: 28 November 2010

During a periodic test of an emergency diesel generator, an abnormal mechanical noise was heard just
after its connection to the 6.6kV switchboards. A large amount of smoke was produced and technicians
immediately evacuated the room. The room was re-entered and the fire that caused the smoke was quickly
brought under control. A follow-up inspection showed major damage to the diesel engine, the generator
and the other nearby equipment. The unit was required to shut down for approximately one month, for
repairs.

Contributor: The root cause of the failure was a design deficiency that resulted in a defective bearing shell.
Specifically, the metal thickness of the bearing was inadequate. This led to its fusion and the release of
smoke. It should be noted that the bearing had been changed following the Chinon B incident in 2009 in
which a defect was also found in the bearing shell.

Lesson Learned
The operating experience from a known problem at another station was not properly used to prevent the
fusion and seizure of the shaft in an EDG, which caused severe damage.

Station: Tricastin 3

Reference: EAR PAR 11-013

Event date: 16 February 2011

Following failures of EDGs at Blayais and Cruas because of fast degradation of bearing shells manufactured
by MIBA, inspections were performed in the fleet. These inspections identified that 25 EDGs have a similar
type of bearing shell. For example, all the EDGs at Tricastin units 3 and 4 were found to be equipped with
the same type of bearing shells, creating the potential for a common mode failure of both emergency
power trains, should a loss of grid event occur.

Contributor: The cause of the fast degradation of bearing shells is a high under-thickness of the anti-friction
layer of the bearings that could lead to localised fusions and subsequent deterioration. These MIBA second
generation bearing shells have replaced MIBA first generation shells during preventive maintenance, which
started at the end of 2009.

Lesson Learned
Fleetwide changes to EDG parts or component design must be thoroughly reviewed and verified to avoid
creating a potential for widespread common mode failures.

Station: Loviisa 1&2

Reference: MER MOW 11-073

Event date: 14 January 2011

Urgent information received from the Finnish service company responsible for overhauling the station EDG
led to one Loviisa diesel generator being dismantled for bearing inspections and to replace the rod bearings
due to minor damages in bearing surfaces.

WWW.WANO.ORG 11
LIMITED DISTRIBUTION WANO RPT 2013-5

Contributor: Direct cause was impurities in the bearings.

The bearing had a cavitation-related defect in the lubrication groove section. This is typical for bearings
with lubrication grooves.

The top halves of the bearing had some polished sections. The B-half had two scratches and both bearings
had sporadic point-sized defects. All these defects were most likely caused by impurities in the bearings.

Lesson Learned

Similar bearing damages had been experience previously at other stations, but this information was not
used effectively and the problem was not well disseminated to other WANO members.
Station: EDF 1300 MW Unit

Reference: MER PAR 12-023

Event date: 09 November 2011

The investigation revealed that the ventilation system of the EDG buildings on the 1300MW units were not
seismically qualified. Failure of the diesel building ventilation system creates the risk for loss of the two
EDG sets at each unit.

Contributor: Design analysis deficiency was the primary root cause with design not being properly
evaluated for externally damaging conditions.

Lesson Learned

All supporting systems for EDGs should be checked to determine if they are seismically qualified.

Station: Cooper 1

Reference: WER ATL 13-0066

Event date: 30 October 2012

During post-maintenance testing of an EDG, the newly installed static excitation system failed. The
excitation system was recently designed and supplied to the plant.

Contributor: Follow-up tests identified unanticipated high voltages on the terminals of the control windings
of the affected transformers.

Lesson Learned

Reinforce the importance of conducting adequate post-maintenance testing following modification or

maintenance to EDGs.

WWW.WANO.ORG 12
LIMITED DISTRIBUTION WANO RPT 2013-5

Station: Bohunice 4

Reference: MER MOW 10-0048

Event date: 13 January 2010

With the unit in operation and EDG testing in progress, the area filled with smoke after a blast sound was
heard. The operator immediately shut down the diesel generator.

Contributor: A defect was discovered in the manufacture of parts. Specifically, flake graphite was found in
the grey cast iron used for some parts.

Lessons Learned
Non-destructive testing, using the dye penetrant inspection method, should be incorporated into the
technological procedure describing routine repairs on the critical starting air system.

Related to Maintenance Issues

Station: Cernavoda 1

Reference: MER ATL 11-505

Event date: 16 February 2011

During periodic testing, a standby EDG experienced a catastrophic failure of the number three bearing. The
event resulted in the prolonged unavailability of the EDG.

Contributor: The cause was determined to be the poor condition of the bearings overall, particularly the
high rates of wear and overlay loss. Also, the bearings were found to be dirty and contaminated with
debris, possibly from previous work on the EDG or with a contaminated lubricating oil supply.

Lesson Learned

 One possible entry point for dirt and debris was from filters when the elements were changed. The oil
analysis did not include determining the metallic particulate content.

 The organisational factors that contributed to the failure are related to ineffective communication
with the EDG supplier, regarding operating experience.

WWW.WANO.ORG 13
LIMITED DISTRIBUTION WANO RPT 2013-5

Station: Oskarshamn 1

Reference: WER PAR 13-0016

Event date: 14 November 2012

With the unit in cold shutdown, two out of three starting attempts on an EDG failed during surveillance
testing. A subsequent investigation determined that there was a risk of common cause failure; therefore,
the redundant EDG was also considered inoperable. In addition, it was determined that the EDG had a
longer starting time than expected.

Contributor: The root cause was that grease had been applied to the incorrect location in the starter
motor.

Grease

Piston

Contact
Point with
the washer

Lesson Learned

Testing in accordance with Institute of Electrical and Electronics Engineers (IEEE) standards did not
guarantee that some deficiencies, such as prolonged starting time, would be identified, because IEEE
standards do not consider time-dependent failures.

Station: Gentilly 2

Reference: WER ATL 12-0454

Event date: 17 August 2011

With the unit operating at 88% power and EDG testing in progress, the EDG failed to start correctly. The
failure was due to an excessive startup time (over 15 seconds). The test was repeated and the EDG failed
the test a second time for the same reason.

Contributor: During a previous maintenance activity, the wrong type of grease was used to lubricate the
starters.

Lesson Learned

Reinforce the use of the equipment vendor manual.

WWW.WANO.ORG 14
LIMITED DISTRIBUTION WANO RPT 2013-5

Station: Prairie Island 1

Reference: WER ATL 12-0531

Event date: 14 August 2012.

During a monthly surveillance run of an EDG, a small candle-sized flame was identified at the exhaust
manifold, resulting in the EDG being shut down. In order to determine if common cause failure existed, the
second EDG was tested. The same problem occurred with the second EDG, resulting in both EDGs being
declared inoperable.

Contributor: The likely problem was a gasket leak on the turbocharger because of loose bolting that
resulted in the insulation being soaked with oil.

Lesson Learned
Review the frequency of equipment (exhaust manifold) preventive maintenance and the operating
environment in which the equipment is used.

Station: Narora-1

Reference: WER TYO 12-0173

Event date: 30 April 2012

With the unit operating at 70% power, a fire alarm signal occurred in the control room. A field investigation
revealed smoke coming from an EDG’s exhaust vertical line expansion joint, which was running in iso-mode
on its bus. The EDG was immediately stopped and the fire extinguished.
Contributor: The EDG exhaust line has male-female slip type expansion joint with asbestos sealing. The oil
coming out from the expansion joint soaked the thermal insulation. The high temperature of the exhaust
line caused heating of the oil soaked insulation, resulting in heavy smoke. Sealing rope used at the
expansion joint was not a sufficient method for preventing oil leaking, due to the size of the expansion
joint’s radial gap (2-5mm).
Lesson Learned
Check the vulnerable points, such as expansion joints, that may be potentially affected by oil leaks.
Station: Browns Ferry 1

Reference: MER ATL 11-381

Event date: 02 May 2011

The site experienced multiple tornados, severely damaging all seven of the plant's 500kv transmission lines
and one of the two 161kv lines. This resulted in a loss of off-site power, simultaneous scrams on all three
units. Five days after this event, the "A" EDG's output breaker tripped, resulting in the "A" 4KV shutdown
board being de-energised. As a result, shutdown cooling, spent fuel pool cooling pumps, reactor water
cleanup system pumps and control rod drive system pumps were lost for 54 minutes. The temperature of
the reactor coolant system rose approximately 25°F (about -3.9°C) before electrical loads were restored,
utilising 161kv off-site power. The spent fuel pool temperature did not change.

Contributor: The direct cause of the EDG output breaker tripping was an inadvertent actuation of the over
speed trip limit switch because the switch was misaligned. Contributing causes were inadequate
maintenance instructions for verifying the position of the over speed trip limit switch arm and internal
inspection of the over speed trip lever assembly and lack of engineering oversight.

WWW.WANO.ORG 15
LIMITED DISTRIBUTION WANO RPT 2013-5

Lesson Learned

Maintenance instructions/procedures should have clear instructions for the positioning of the over speed
switch and an independent inspection performed after installation.

Station: Susquehanna 1, 2

Reference: MER ATL 12-113

Event date: 05 June 2012

During surveillance testing, the EDG lost firing on one cylinder. An investigation revealed the delivery valve
spring was installed incorrectly in the fuel injector pump for that cylinder. It was later found that 12 of the
16 delivery valve springs were either broken or in a failed state. The event resulted in a determination that
for approximately a three-month period, EDGs had been unavailable, unknowingly impacting probabilistic
risk evaluations for the station.

Contributor: The broken springs were the result of an inverted spring and stop installation and the failure
to fire on one cylinder was believed to be the result of foreign material (spring parts) migrating to the
injector nozzle.

The root causes are as follows:

 The work package provided for field use contained insufficient instructions.

 The work crew proceeded with the work using an inadequate work package.

 Quality control activities were insufficient to prevent the incorrect reassembly of the fuel injector
pump components.

Lesson Learned
Ensure the work packages have all information for installation in the field and a quality check programme is
in place to ensure the quality of critical work activities in the field.
Station: Darlington 0

Reference: MER ATL 11-426 Event date: 23 June 2011

Standby EDGs are designed to trip upon high differential pressure across the air intake filters. The outdoor
instrument tubing for detecting the ambient air pressure has been found to be vulnerable to getting
plugged with insect debris.

Contributor: Obstruction/blockage due to the presence of foreign material was the direct cause of the
event. Initial design inadequacy, inadequate preventive maintenance and inadequate component
monitoring were identified as the casual factors for the event.

Lesson Learned

Reinforce in the design, the environmental working conditions of the equipment.

WWW.WANO.ORG 16
LIMITED DISTRIBUTION WANO RPT 2013-5

Station: Monju 1

Reference: EAR TYO 11-002

Event date: 28 December 2010

During a monthly routine load test of an EDG, an operator heard an abnormal noise and found exhaust gas
coming from the top of cylinder head #8. The operator stopped the EDG. Several cracks were identified in
the liner body and the lip of the liner was fractured into six parts. Cylinder #2, which is located at the
symmetrical position of #8, was overhauled at the same time as #8 and was found undamaged.

Contributor: The damage was mainly attributed to the excessive tension to the tension bolts.

Lesson Learned

Oil pressure must be controlled when putting tension to the tension bolts. A penetration test should be
conducted on the cylinder rim after disassembling a cylinder.

Relate to OE not Properly Applied

Station: KORI 1

Reference: EAR TYO 12-0002, SER 2012-3

Event date: 09 February 2012

During an outage and testing in progress on main generator protection relays, the unit experience a loss of
off-site power and the only available EDG failed to start.

Contributor: The cause of EDG’s failure was a malfunction of starting air solenoid valves, making it unable
to open on demand. A similar event had occurred in 2005, but recommended corrective action to add a
second solenoid valve in parallel was not completed.

Lesson Learned

Delays in the implementation of corrective actions could have later consequences.

Station: Forsmark 2

Reference: MER PAR 11-089

Event date: 09 March 2011

During testing of an EDG, smoke was observed leaking from the exhaust stack just below the upper
mechanical support. An inspection of its stack and the stacks of other EDGs found them severely corroded
and, in several cases, cracked. The stacks on the north side of the buildings, were found to be corroded to
the extent that they were beginning to collapse. This condition could render one or more of the EDGs
inoperable and unavailable, should it be required for emergency power.

Contributor: Weather factors have played a considerable role in the deterioration process. Preventive
maintenance programmes have not been in place to monitor the condition of the upper parts of the EDG
stacks. Inadequate use of operating experience was also identified as a causal factor of the event.

Lesson Learned
Reinforce in the design and the preventive maintenance programmes, the effect of the environment in
which the equipment is expected to perform.

WWW.WANO.ORG 17
LIMITED DISTRIBUTION WANO RPT 2013-5

Related to Common Cause Failure

Station: Doel 3

Reference: WER PAR 12-0105

Event date: 22 July 2010

With the unit operating at full power, all three supply fans providing combustible air for starting the EDGs
starting were out-of-service. These circumstances resulted in the simultaneous unavailability of several
EDGs.

Contributor: One fan was undergoing maintenance, a second fan tripped on spurious actuation of thermal
protection during testing and the third fan tripped due to short circuit in the motor.

Lesson Learned
The supporting equipment required for reliable operation of EDGs should be identified and its preventive
maintenance strategy and ageing management should be improved.

Station: Chalk River

Reference: MER ATL 11-357

Event date: 23 August 2011

The reactor was operating at normal high power when a thunderstorm in the area resulted in a complete
failure of off-site power at the site, leading to a reactor trip. The standby EDG started automatically on loss
of off-site power, but tripped immediately on low coolant level. When the EDG failed to start, the exhaust
air filtration system fans became unavailable. The loss of the fans resulted in failure to maintain minimum
fan configuration requirements.

Contributor: The standby EDG remains a single point of failure for three credited safety systems and has no
backup during off-site power failures.

Lesson Learned

System vulnerabilities, due to failure of single diesel generator, should be identified and addressed.

Examples of Events with Improper Operation or Component Failures

Station: Tihange 2

Reference: EAR PAR 12-016

Event date: 22 February 2012

The reserve EDG was being re-qualified via startup signal from the safety injection system controls. While
the EDG was coming up to speed, the smoke box exploded and the EDG shut down. Considerable damage
was caused to the room as a result of the explosion and the EDG was extensively damaged. No one was
injured during the incident.

WWW.WANO.ORG 18
LIMITED DISTRIBUTION WANO RPT 2013-5

Contributor: The cause of the explosion was ignition of unburned elements in the chimney that had
accumulated, resulting from misfires during startup. Insufficiently high-temperature water would have
caused the misfires at startup. The root cause of the low temperature water was a shorter pre-heating time
because of a malfunctioning thermostatic valve. An inadequate and/or insufficiently long startup speed also
influenced the compression rate and the mixing temperature.

Lesson Learned

Reinforce the use of the previous events that are similar, or anomalies in reports related to the same
equipment.

Station: Tihange 2

Reference: EAR PAR 12-024

Event date: 14 March 2012

The time delays for the EDGs fundamental protection was set to a value that did not comply with the
technical specifications. This finding highlighted the failure to take this parameter into account when
carrying out the mandatory monitoring measure for all the EDGs and resulted in all the EDGs being
declared unavailable.

Contributor: The event was caused by a lack of questioning attitude and procedures that were technically
incomplete.

Lesson Learned

Apply human performance error reduction tools.

Station: Browns Ferry 3

Reference: WER ATL 13-0200

Event date: 09 January 2013

An EDG was declared inoperable due to failure of the inboard bearing for the generator blower. When
inspected later, grease and metal shavings were found under the inboard bearing for the generator blower.
The inboard (drive end) bearing catastrophically failed due to age-related breakdown of the grease.

Contributor: The EDG blower shielded bearings were not adequately assessed on a component level to
identify potential failure modes and operability impacts. Standard vibration data did not identify that
lubrication in the blower shielded bearings had degraded.

Lesson Learned

Ensure sealed and shielded bearings in safety-related and risk-significant equipment (pumps, motors,
blowers, fans and so forth) are assessed at the component level to identify potential failure modes and
operability impacts.

Ensure that procedures for monitoring component vibrations, during diesel generator surveillances, alert
personnel that vibration monitoring data may not detect bearing degradation because of diesel generator
masking effects.

WWW.WANO.ORG 19
LIMITED DISTRIBUTION WANO RPT 2013-5

REPORT ǀ RPT 2013-5

Gaps in Excellence
Six major gaps in excellence were identified that require industry attention to further improve EDG
performance. The gaps involve EDG problem resolution, preventive maintenance, life cycle management,
spare parts, work management and monitoring.

1. EDG Problem Resolution

Managers did not use an equipment reliability approach that leverages the strength of the entire
organisation to resolve EDG issues. Instead, required EDG improvements were addressed piecemeal,
resulting in some EDG problems not being resolved in a timely manner and others not resolved at all.
In many cases, the totality of required work, to ensure high EDG reliability and availability, was
extensive and complex; yet system engineers often undertook resolving the work without help from
others within the organisation, such as projects, design engineering and work management personnel.
Comprehensive improvement plans were not developed and appropriate station personnel were not
involved in the developing or implementing of improvement initiatives. An additional impact was that
the ability of EDG system engineers to perform their core business was often hindered and on-line
work processes designed for preventive and corrective maintenance were unduly burdened.

2. EDG Preventive Maintenance (PM)

A gap analysis of the EDG preventive maintenance (PM) programme against vendor and owners group
recommendations was not performed. As a result, numerous stations were not performing some
recommended vendor and owners group PM tasks. In other situations, unnecessary PM activities were
being performed that resulted in additional exposure to human errors and needless system
unavailability.

Single component vulnerability studies of EDG systems were not performed at several stations. As a
result, mitigating strategies such as increased preventive maintenance activities, or modifications to
make the designs more robust, were not implemented. In other cases, critical EDG components and
parts did not have unique identifiers and thus were not considered by the PM programme.

3. EDG Life Cycle Management

Either a life cycle management strategy was not developed or it was ineffective in addressing
component obsolescence, particularly for generators, exciters and voltage regulators. Originally
installed parts have become obsolete; however, this fact was only identified when procurement
organisations attempted to order an item. Unavailability of spare parts challenges planning and
scheduling EDG work. Similar to PM programmes, current life cycle programmes are not adequately
addressing component obsolescence or availability of parts.

4. EDG Spare Parts

Critical spare parts are not identified or effective strategies developed for known shortfalls in critical
spare parts. Manufacturers of many original EDG components and parts are no longer in business;
therefore, many parts are becoming difficult to obtain as spare part inventories diminish. For example,
a spare generator is not available or a strategy for replacing a generator has not been developed.
Additional examples of parts that are needed, but are unavailable, include:

 voltage regulators and voltage regulator circuit boards

 current transformers

WWW.WANO.ORG 20
LIMITED DISTRIBUTION WANO RPT 2013-5

Some spare parts were not in a serviceable condition because they were not properly tested upon
receipt, maintained or properly stored. This often affected completion of scheduled work, requiring
work to be either delayed or cancelled. In other cases, station personnel were slow to return spare
EDGs to a serviceable condition after parts had been cannibalised for use on in-service EDGs.

5. EDG Work Management for Optimising Availability

Ineffective work implementation contributed to additional EDG unavailability. For example, ineffective
coordination when installing clearance tags or restoring the EDG to an operational line-up.
Inefficiencies in work schedules and inadequate turnovers between shift crews also resulted in added
EDG unavailability.

6. EDG Monitoring

Monitoring programmes did not sufficiently consider direct and indirect EDG parameters. In some
cases, personnel only monitored indirect EDG parameters, such as reliability and availability. In other
cases, personnel monitored direct data, such as temperatures, pressures and voltages obtained during
EDG runs, but only reviewed the data to ensure it was meeting specifications. Long-term trends were
not adequately evaluated. In addition, action levels were not established for some parameters and few
(if any) actions were taken when values deviated from the normal range. Several stations relied on
EDG vendors for obtaining performance data taken during testing, but the information received was
often inadequate for identifying long-term trends.

WWW.WANO.ORG 21
LIMITED DISTRIBUTION WANO RPT 2013-5

REPORT ǀ RPT 2013-5

Recommendations
1. Implement an equipment reliability approach, for addressing EDG performance problems, that
leverages the strength of the entire station (for example, maintenance, operations, engineering and
procurement). Develop a comprehensive improvement plan if performance and/or self-assessments
reflect multiple problems.

2. Perform an EDG PM gap analysis that compares station PMs to vendor and owners group
recommendations for preventive maintenance. Also, perform an EDG single component vulnerability
review to ensure that the PM strategy addresses this component group appropriately. Identify critical
components that currently do not have a unique identifier for inclusion in the PM programme. Close
any identified gaps.

3. Implement a life cycle management strategy to address obsolete EDG components, with a focus on the
generator, exciter and voltage regulator. The life cycle strategy should include a long-term plan
approved by senior station managers.

4. Implement a strategy for obtaining and storing critical EDG spare parts. Implement actions to ensure
that critical spares have been properly tested, stored and maintained in a serviceable condition for
immediate use.

5. Take actions to reduce unnecessary EDG unavailability by improving the management of work during
an EDG workweek. Ensure that work schedules include sufficient details for all activities. Minimise the
time between the hanging of a clearance tag and the beginning of work on an EDG. Also, minimise the
time between work completion and tag removal. Ensure that maintenance personnel walk down EDG-
related work packages. Verify availability of required parts before the EDG is removed from service.

6. Improve the monitoring of key engine, generator and support equipment parameters. Ensure that
parameters are analysed for long-term trends and that actions are taken when the trends indicate a
performance problem. Clearly discuss adverse EDG parameters in EDG health reports and share the
information with senior station managers.

WWW.WANO.ORG 22
LIMITED DISTRIBUTION WANO RPT 2013-5

REPORT ǀ RPT 2013-5

References
1. INPO IER L4-13-21, Emergency Diesel Generator (EDG) Review Visit Summary

2. WER ATL 2013-0273, Emergency Diesel Generator (EDG) Review Visit Summary

3. WER ATL 2012-0713, Emergency Diesel Generator Leaks (INPO IER L4-12-82)

4. WER ATL 12-0720, Failure of Emergency Diesel Generator Division III

5. EAR PAR 11-0007, Emergency Diesel Generator Fire Damage Caused by Bearing Shell Failure

6. EAR PAR 11-013, Risk of Common Mode Failure of Emergency Diesel Engines due to
Potential Fast Degradation of Bearing Shells

7. MER MOW 11-073, Bearing Problems of the Emergency Diesel Generator

8. MER PAR 12-023, Absence of Seismic Qualification of Emergency Diesel Generator Building Ventilation

9. WER ATL 13-0066, Replacement Excitation System Failures for Emergency Diesel Generator During
Post-Installation Testing

10. MER MOW 10-0048, Diesel Generator 2QX Engine Failure Identified during 3-Minute Trial Run

11. MER ATL 11-505, Unavailability of Standby Diesel Generator Due To Bearing Failure

12. WER PAR 13-0016, Failure Start of Emergency Power Diesel Engines 660 GA1 and GB2

13. WER ATL 12-0454, Emergency Diesel Generator Failing to Start

14. WER ATL 12-0531,Technical Specification Required Shutdown Based on both Emergency Diesels Being
Declared Inoperable

15. WER TYO 12-0173, Smoke Observed from Diesel Generator Exhaust Vertical Line Expansion Joint in
NAPS-1

16. MER ATL 11-381, Diesel Generator Output Breaker Trip Results in Loss of Shutdown
Cooling and Spent Fuel Pool Cooling

17. MER ATL 12-113, Diesel Generator Fuel Injection Pump Delivery Valve Springs/Stops
Installed Incorrectly Resulting in D/G Failure

18. MER ATL 11-426, Standby Generator Intake Filter Instrument Line Partially Blocked by
Insects

19. EAR TYO 11-002, Emergency Diesel Engine Failure

20. EAR TYO 12-0002, Loss of Offsite Power and Emergency Diesel Engine Failure

21. SER 2012-3, Station Blackout and Loss of Shutdown Cooling Event Resulting from
Inadequate Risk Assessment

22. MER PAR 11-089, Emergency Diesel Exhaust Stacks, Severe Corrosion and Cracking Challenges
Mechanical Integrity in all Eight Stacks in Two NPP Units

WWW.WANO.ORG 23
LIMITED DISTRIBUTION WANO RPT 2013-5

23. WER PAR 12-0105,Tripping of VK fans

24. MER ATL 11-357, Loss of Grid Results in Scram - Diesel Generator Fails to Operate

25. EAR PAR 12-016, Explosion in a diesel Generator smoke box during initial start-up

26. EAR PAR 12-024, Failure to check The Time Delays of Emergency Diesel Generators

27. WER ATL 13-0200, Emergency Diesel Generator 3D Blower Bearing Failure

28. SOER 2002-2, Emergency Power Reliability

29. WANO GL 2007-02,Guidelines for On-The-Job Training and Evaluation

30. WANO GL 2001-02, Guidelines for the Conduct of Operations at Nuclear Power Plants

31. JIT-019, Emergency Diesel Generator (EDG) Maintenance (Rev.1)

32. JIT-094, Emergency Diesel Generator (EDG) Testing

33. JIT-136, Diesel Generators – Planning maintenance and Testing

34. JIT-145, Emergency Generator Maintenance – Foreign Material Controls

35. INPO TR10-73, Emergency Diesel Generator (EDG) Start, Load, and Run Failures and Events Affecting
Power Generation (January 2007 – June 2010)

36. INPO TR7-60, Emergency Diesel Generator Demand and Run Failures (2002– 2006)

37. INPO IER L3-13-32, Emergency Diesel Generator Supply Air Blower Bearing Issue

WWW.WANO.ORG 24
This page is left blank intentionally
This page is left blank intentionally
 ATLANTA
LONDON & HONG KONG
MOSCOW
PARIS
TOKYO
WORLD ASSOCIATION OF NUCLEAR OPERATORS

www.wano.org & www.wano.info