TMR Control System Design

-Procedure and A Design Example-

Fault-Tolerant Control Systems
Definition of a Fault-Tolerant System
A fault-tolerant system is one that can continue to
correctly perform its specified tasks in the presence of
hardware failures and software errors, [Johnson 1989]
Objectives
Long Life Applications
Critical Computation/Control Applications
Maintenance Postponement Applications
High Availability Systems
Safety Critical Applications
Design Criteria
Reliability, Availability, Maintainability,
Safety (RAMS)
Testability, Performability, Dependability
(TDP)
Others: Survivability, Diagnosability,
Programmability etc.
Ways to Fault Tolerance
Analytical Method
Estimation/Detection/Self-diagnosis
Application dependency
Physical Method (Redundancy)
Hardware Redundancy
Software Redundancy
Information Redundancy
Time Redundancy
Design Procedure
Start
SimplexDesign
TMRSpecific
Design
FeasibilityTest
Technology
Pool
Requirements
Analysis
SystemEvaluation
Pass?
Designfor Implementation
Yes
No
DefineTheProject
Detail Design
Testing
End
Define Project
System Specifications
Target Plant
I/Os and Actuators
Required Control Performance etc.
Required Fault Tolerance Criteria
RAMS & TDP
Mission-based or Non-Mission-based
Feasibility Test
Trial Design from Technology Pool
Evaluation for the Requirements
Sensitivity Analysis
Performance vs Cost Evaluation
Decision on the Final Candidate

I&C-KERI ISIM2000-8
Detail Design
System Structure
Real-time Characteristic
Voting Mechanism
Power Supply and Communication Protocol

Evaluation
Mathematical Models
RBD, Markov, Petri-Net etc.
Tools
MATLAB, Mathematica etc.
RelCalc
SHARPE, SURE, HiReL, SPNP, IsoGraph,
UltraSan etc.
Implementation and Tests
Physical Implementation
Hardware Connections, Protections, Installation
Tests
Functional Tests
Environmental Tests
Field Test
Design Example
Digital Excitation System
for 500MW-class Thermal Power Plants
Regulation Error: within 0.5%
104 DIO & 68 AIO
50 critical, 122 non-critical
Actuator: Phase-controlled rectifiers (Static)
Reliability Requirement: Higher than 0.99
for Two Years Continuous Operation
Excitation System Overview
Excitation and Excitation Control

Regulator Exciter Generator
Power
System
Excitation
Excitation Control
Simplex Digital Controller
COTS-based Pool
VMEbus System with RTOS (VxWorks
tm
)
Configuration
1 CPU board and 1 A/D board (32 points)
2 Digital Inputs boards (62 points)
1 Digital Output board (40 points)
1 Counter/Timer board (for PCR firing)
1 Communication board (2 RS-422, 2 RS-485)
MTTF and Failure Rate
BOARD NAME
MTTF
(HOURS)
FAILURE RATE
(FPMH)
A/D 75,000 13.333
CPU 75,000 13.333
Timer/Counter 75,000 13.333
Power Supply 100,000 10.000
I/O 11,000 90.909
Voter 199,436 5.0141
PCR 50,000 20.000
First Impression
Quick Evaluation
Reliability over 0.99: Only for 61 hours
Why TMR?
Fault Containment and Recovery
Hot/Cold Swapping Capability
Efficient Voting Mechanism
Flexible Reconfiguration
Standby Mechanism
Degradation and Replacement
Design of TMR controller
Asynchronous TMR structure
No synchronizing hardware signal
Synchronization by Counter/Timer board
Loosely coupled synchronous system
Majority Voting and Heartbeat signal
Intelligent voting

TMR Control System
Controller
#1
D/O Voter
Gate Pulse
Voter
PCR
Controller
#2
D/O Voter
Gate Pulse
Voter
PCR
Controller
#3
D/O Voter
Gate Pulse
Voter
PCR
Hardwired 2-out-of-3 Logic
Digital Output
Evaluation (RBD)
| |
(

|
|
.
|

\
|

|
|
.
|

\
|
=

i
v
i
v
i
p
i
c
i
c
i
N
t R t R
i
t R
t R t R
i
t R
)) ( 1 )( (
3
)) ( 1 ( 1
)) ( 1 )( (
3
) (
3
1
0
2
3
2
0
I/O & CPU
I/O & CPU
I/O & CPU
Voter & PCR
Voter & PCR
Voter & PCR
+ G
Power
Supply
Power
Supply
1-of-3 1-of-2 2-of-3
Evaluation (Markov Model)
) (
1
0 2 ) 1 ( 2 0
0 ) ( 0 2 0
0 0 ) 2 ( 0 ) 1 ( 3
0 2 3
0 0 0 3
) (
1
t P
c c c
D
c
c c c
D
c
c
C
c c
D
c
c c
C
c c
D
c
c c
dt
t dP
(
(
(
(
(
(

+
+

=





Values for Simulation
c
D 0.9
p
D
0.9
v
D
0.1
c
C
1/24
p
C
1/24
v
C
1/24
c

0.9
p

0.9
v

0.5
Evaluation Result
More than 2.5 Years with the reliability higher than 0.99
For 0.999 Reliability requirement: 4000 hours
Overall System Reliability
0 1 2 3 4 5 6 7 8 9 10
x 10
5
0.7
0.75
0.8
0.85
0.9
0.95
1
time[hours]
r
e
l
i
a
b
i
l
i
t
y
--*-- : Dv=0.1
--o-- : Dv=0.5
--x-- : Dv=0.9
Testing and Field Application
Standards (11 items used)
IEC
NEMA
ANSI/IEEE standards
5 Sets for 500MW-class Thermal Power
Plants

TMR Controller
Digital Exciter
Conclusions
A design procedure of a fault-tolerant control system for
thermal power plants were introduced.
Proposed procedure successfully worked for our project even
though trial-and-error and rule-of-thumb approaches are
needed in some steps.
It still turns out to be very difficult to establish an intuitive
design procedure for fault-tolerant systems as it has been
thought so far.
Redundancy can be an option to realize the design of highly
reliable systems from COTS-based component pool.
More detailed and formalized approach may be necessary to
develop a design procedure for ultra-reliable systems.