Automatiea, Vol. 7, pp. 637-640. Pergamon Press, 1971. Printed in Great Britain.
Correspondence Items
An Innovations Approach to Fault Detection and Diagnosis
in Dynamic Systems*
Une approche d'innovation/t la d6tection d'erreurs et au diagnostic
dans les syst~mes dynamiques
Neugestaltetes Vorgehen zur Fehlererkennung und Diagnose in
dynamischen Systemen
Ho~xo~ 06HOB~eH~ r O6Hapy~eHH~OOmH60r 14 K ~lHarHo3y
B ~ i 4 u a M w - i e c r i , i x CI,I C T e M a x
R. K . M E H R A ~ f a n d J. P E S C H O N ~ -
Sumnml'y--A general approach to fault detection, diagnosis
and prognosis in systems describable by mathematical models
is outlined. It is based on System Theory and Statistical
Decision Theory. The special case of linear dynamic
systems with Gaussian random inputs is considered and it is
shown how the statistical properties of the innovation
process can be used for fault detection and diagnosis.
1. Introduction
THE PROBLEM of fault detection and diagnosis in engineering
systems is a common one. The present approach relies
heavily on the experience of the operating staff with the
physical system. Often, this experience is epitomized in
rules of thumb which are used by the operating staff under
emergency conditions. In general, fault detection requires
continuous or periodic monitoring of the output variables
(observables) of the plant. Under normal conditions, these
observables follow certain known time histories, but under
faulty conditions, they deviate from their nominal histories.
Fault detection is mostly performed by spotting these
deviations in the plant observables.
The problem of fault diagnosis and prognosis is more
difficult than the detection problem. It often requires the use
of special test signals to isolate the fault. For prognosis, an
understanding of the future behavior of the plant under
faulty conditions is required.
In the present article, the detection, diagnosis and pro-
gnosis functions are unified using System Theory and
Statistical Decision Theory. The historical background of
this approach is difficult to trace because of its numerous
special applications. The formulation here elaborates on the
work of PESCHON [1] MEHRA [2] and WILLNER [3]. The
organization of the paper is as follows. Section 2 outlines
the general method of approach. Section 3 specializes it to
stochastic linear dynamic systems. Section 4 proposes some
applications. The conclusions are stated in section 5.
2. Method of approach
The general approach consists of the following steps:
2.1. Development o f a model. A mathematical model is
developed for the system or the plant based on physical
* Received 22 January 1971; revised 19 March 1971.
This article was not presented at any I F A C meeting. It
was recommended for publication in revised form by Asso-
ciate Editor K. AstrOm.
t Systems Control, Inc., 260 Sheridan Avenue, Palo Alto,
California.
637
information and statistical data. This model can be static or
dynamic, linear or non-linear, continuous or discrete and
deterministic or stochastic. The input and the output
variables of the system are clearly defined and all the relevant
parameters are identified. The model describes the behavior
of the system under normal operating conditions. It also
specifies the statistics of the measurement noise in the output
variables.
2.2. Generation of an error signal or the innovation
process. The error signal or the innovation process is defined
as the difference between the actual system output and the
expected output based on the model and the previous output
data. The latter is generated directly by the model if the
system is deterministic or by a staticical filter if the system is
stochastic, i.e. subject to random inputs and variations. It
is called the innovation process since it represents the new
information brought by the latest observation. Under
normal conditions, the error signal is "small" and corresponds
to random fluctuations in the output since all the
systematic trends are eliminated by the model. However,
under faulty conditions, the error signal is "large" and
contains systematic trends because the model no longer
represents the physical system adequately.
2.3. Statistics of the error signal under normal conditions.
In deterministic systems, the random fluctuations in the error
signal are due to measurement noise in the output variables.
Their statistics are obtained as part of the system description
in step 2.1. In stochastic systems, the statistics of the error
signal are obtained from the filter which is used to predict the
output of the system. For linear dynamic systems with
Gaussian random inputs, a Kalman filter [4] generates both
the error signal and its statistics. For these systems, it is
known that under normal conditions, the error signal or the
innovation process is a zero mean Gaussian white noise
process [2, 5].
2.4. Fault detection via hypothesis testing. The problem
of fault detection is easily formulated as a problem in
Hypothesis Testing [6] by regarding the normal operation of
the system as the null hypothesis. The actual error signal
from the system is tested against this hypothesis at a certain
level of significance. For example, if the system is described
by a set of linear differential equations and a Kalman filter
is used to generate the innovation process, the null hypo-
thesis consists of testing the innovation process for zero
mean, whiteness and a given covariance.
2.5. Diagnosis and prognosis via system identification. If a
fault is detected in the system, the present model for the
638 Correspondence items

system no longer applies. In order to diagnose the fault and The random vectors wi and t:~ arc Gaussian and ~xhil¢.
prognose the future behavior of the system, it is necessary to Their mean and covariance are
develop a new model for the system. If it can be postulated
that only some of the parameters of the system have changed, E{w,}=~,, E{(w,-~,)(w~-~j)'}=Q,% 13i
an estimation procedure can be used to obtain new values
for these parameters. Alternately, if the system can occupy E{v,} = ~,, E{(~, - ~,)(~,j - ~j)~l = R,6,j (:4)
only a finite number of states, multiple hypothesis testing can
be used to ascertain the most likely state of the system. In e{(v~- ~,,)(w.~- ~/} =o (5~
many cases, it might be necessary to use special test signals to
improve the accuracy of system identification. where 3~1 denotes the Kronecker delta
3. Linear dynamic systems
In this section, the general approach of section 2 is 1, i =.j
specialized to systems describable by a set of linear difference 6ii= o l i~j
equations. Continuous-time linear systems can be treated in
the the same way. (See Fig. 1.) Static systems can be and E( • } denotes the expectation operator.
regarded as special cases of the dynamic systems. The The initial conditions Xo are also assumed to be random.
approach can also be carried over to nonlinear dynamic Their distribution is Gaussian with mean and covariance
systems.
Consider a discrete-time linear dynamic system whose
model is e{Xo}=~o, e{(Xo- ~o)(Xo- ~o) T} = e o
Xi + 1 = lYPiXi+ Giui + FiW i O) e{(Xo-~o)(W,-~y}=o

z i = H i x i + v~, i = 0, 1, 2 . . . . (2) e{(Xo-~o)O,,-~J}=O

where f o r i = 0 , 1, 2 . . . .

x~ is n x 1 vector of state variables (subscript i denotes

the time-instant)
m is p × 1 vector of control variables
w~ is q × 1 vector of random forcing functions
z~ is r x 1 vector of output variables (or observables)
v~ is r × I vector of random measurement errors
q~ is n × n state-transition matrix
G~ is n × p input distribution matrix
F, is n ×q noise distribution matrix
and
Hi is r × n output matrix.
The more general case of correlated random forcing
functions and correlated measurement errors can be reduced
to the above case by augmenting the state vector x¢ [7].
The D D P function (detection, diagnosis and prognosis)
is performed as follows:
3.1. Development of the model. This consists of identifying
matrices dPi, Gi, Fi, Q~, Rt and the order n of the system
under normal operating conditions. It is mostly done by
using a combination of physical information and statistical
data on the system. Various method for system identification
and model validation are useful at this stage [8, 9].

DDP Function

System Kalman Filter

x state of the system K Kalman gain

u input noise z output observation
Z summer 2 estimate of the state x of the system
I~ integrator v innovation process
F, G, H parameters of the mathematical model (.) time derivative

FIG. 1. Detection, diagnosis and prognosis for systems describable by linearl equations.
 Correspondence items 639

3.2. Generation of innovation sequence. The innovation
sequence v, is defined as
Vi = Zi -- ~i/i- I
where z~/~/l denotes the unbiased minimum.variance
estimate of z~ based on observations up to ( i - 1 ) , i.e. based
on the set {zo . . . . z H }. If it is assumed that all the system
parameters and statistics are known exactly, the innovation
sequence can be generated by a Kalman filter of the following
form [41
Xi + l/i = (~[Xi/i- 1 "~-K i v J + Giui + Fiwi
201-~ = x o
vi=zi - - H i J ~ i l i - 1 - - Vi
where ~ denotes the sample mean
^ 1 N
~ = ~ r i ~ 1 q~. (17)
(6)
It can be shown the ~ is an asymptotically unbiased
and consistent estimate of ck [11]. Under the null hypo-
thesis, t3e, k = l , 2 . . . . are asymptotically independent
and normal with zero mean and covariance of I/N. Thus
they can be regarded as samples from the same normal
distribution and must lie in the band + 1"96/.,/N more
than 95 per cent of the times for the null hypo-
thesis [11]. Another statistic than can be used for testing
(7)
indepedence between the components of the innovation
vector is the sample correlation coefficient defined as
(8)
N

Ki = P~/i- xH f ( H t P t / - l H i T + Ri) - 1 (9) P="- J " ' = ' r., ,7 (18)

P, +1/, = * , P M + r,o.,rl
P i/i = ( I - K i H i)Pi/i_ ~
where
~/~ is the unbiased minimum variance estimate of x~
based on observations up to time j.
K~ is n × r Kalman gain matrix
P~/~ is the error covariance of .~/~
i.e.
Pilj = E { ( x i - Xilj)(xi - xi/j)T}
3.3. Statistics of the innovation sequence. It is well known
that the innovation sequence v~ is a zero mean Gaussian
white noise sequence with covariance
(HiPi/i- 1H~ + Ri).
(1 O)
(11)
where the subscripts a and fl indicate the components of
the vector it. ANDERSON [10] shows that under the null
hypothesis the distribution of p,a is
r[½(u- 1)]
1-
where F[.] denotes the gamma function. This statistic is
invariant with respect to the mean and the covariance of
(12) rn. It is particularly useful in the present ease since the
true mean and covariance of r/~ are unknown. This
statistic can also be used for testing whiteness by defining
p=z for different lags.
(b) Tests of mean. These tests check whether the observed
innovation sequence is zero mean or not. The mean of
the innovation sequence is estimated as

For Hypothesis Testing purposes, it is more convenient to

o
consider the Standardized Innovation Sequence L
1=1
rh = (H~Pi/i- 1 n f + R t ) - ~vi (13)
where N is the sample-size and ~ denotes the true mean.
where ( ' ) - t denotes the square root of the inverse of a Under the null hypothesis, ~ has a Gaussian distribution
matrix. Then with zero mean and covariance

e{qitl T} = IOij (14)

E { ~ r} = I / N . (19)
where I denotes the identity matrix.
3.4. Fault detection via hypothesis testing. Different kinds Therefore at the 5 per cent significance level, the null
of faults can develop in the system. Some of these are (a) hypothesis is rejected whenever
bias errors in instruments, (b) noisy instruments, (c) change
in system parameters, (d) change in level of input noise, (e) If~l > 1"96 l / x / N . (20)
change in the structure of the system, etc. All these faults
make the standardized innovation t/, depart from their zero
mean, unit variance and whiteness properties. Therefore it The above test suffers from the fact that the covariance
is useful to perform the following statistical tests: of q~ is assumed known. A better test is the T2-test
which use the T2-statistic
(a) Tests of whiteness. The most important property of
the innovation sequence is whiteness or independence at T 2 = Jv
~,a,,-
r/Co 1r/~ (21)
different time instants. Most of the tests of independence
are based on the autocorrelation function ce of a station- where ~o is the sample covariance. This test is uniformly
ary process for lag k = 1, 2 . . . . as follows: most powerful among all the tests for zero mean which are
invariant with respect to scaling (or covariance). For
Ck= E { ( r h - ?/)(q,-k -- F/)T} (15) details, see ANDERSON [10].

where ~ denotes the mean of q,. (c) Tests ofcovariance. The covariance of the innovation
c~ is often estimated as sequence is estimated as

1 N ,, 1 N ~
~k=-~. Z (?h--~])(l~i--k--~l)T (16) ~o = ~ 1~1 (r/i - q)(qi - r/) r . (22)
640 Correspondence
Under the null hypothesis, ~o has a WISHART Distribution
[I0]. The trace of ~'o has a Chi-Square distribution with
( N - l ) r degrees of freedom. Thus ~ o can be tested for its
null hypothesis covariance equal to an identity matrix.
Both the tests of mean and covariance assume that the
innovation sequence is white. Therefore, it is important
to test the innovation sequence for whiteness first,
especially using tests which are invariant with respect to
the mean and covariance of the distribution.
It should also be mentioned that none of the tests given
above are true likelihood ratio tests. The evaluation of
likelihood ratio is very complicated since it involves
obtaining the maximum of the likelihood function with
respect to all the system parameters. The latter involves
the solution to a complicated nonlinear program [12].
3.5. Fault diagnosis and prognosis via system identification.
The statistics of the innovation sequence were estimated
above for hypothesis testing. The same can be used here
along with the autocorrelation function of the output z~
to identify the new structure of the system [2, 9]. The
estimates of the parameters are obtained via a maximum
likelihood procedure since it is known to provide efficient
estimates [12, 13]. The details of these algorithms are given
in Refs. [2, 8, 9, 11, 12-14]. Useful information for fault
diagnosis is also provided by the Tests of Mean and Covari-
ance, e.g. if the innovations have nonzero mean, this in-
dicates the possibility of bias in the instruments. Special
system characteristics can often be used to simplify diag-
nostic procedures [3].
4. Applications
Possible applications of the above method are:
1. Gyro, E M Log and Accelerometer Failures in Inertial
Navigation Systems [2].
2. Aircraft Instrument Malfunction Detection Using
Aircraft Equations of Motion and Instrumentation
Redundancy [14].
3. Anomaly Detection in Power Systems for line faults,
noisy instruments, noisy communication channels, etc.
[31.
4. Fault Detection in Aircraft Jet Engines.
In most of these applications, special system characteristics
can be used along with the general approach to simplify the
tasks of fault detection, diagnosis and prognosis.
5. Conclusions
A general approach to fault detection, diagnosis and
prognosis using System Theory and Hypothesis Testing is
given. It is shown how this approach can be applied to
linear dynamic systems with Gaussian random inputs.
References
[1] J. PESCHON: Data processing structures for failure
detection, diagnosis and prognosis by indirect measure-
ments. Sixth Quarterly Meeting Mechanical Failures
Working Group, Office of Naval Research, New York
(December 1968).
[2] R. K. MErmA: On the identification of variances and
adaptive kalman filtering. IEEE Trans. Aut. Control
(April 1970).
[3] D. WmLNER: New developments in system state
estimation and anomaly detection. Ed. L. P. Hajdu,
SCI Final Report to BPA (October 1969).
items
[4] R. E. KALMAN: New methods and results in linear
prediction and filtering theory. Proe. Syrup. on Engineer-
ing Applications of Random Function Theory and
Probability. John Wiley, New York (1961).
[5] T. KMLATH: An innovations approach to least-squares
estimation, Part I, IEEE Trans. Aut. Control AC-13,
646-655 (1968).
[6] M. G. KENDALL and A. S~UAal-: The Advanced Theoo
of Statistics, Vol, 2, Hafner, New York (1961).
[7] A. E. BRYSON and Y. C. Ho: Applied Optimal Control.
Blaisdell, Waltham, Mass. (1969).
[8] P. EAURRE and J. P. MARMORAT: Une algorithme de
realization stochastique. C. R. Acad. Sci. Paris 168,
978-981 (1969).
[9] R. K. MEHRA: On-line identification of linear dynamic
systems with applications to Kalman filtering. Joint
Automatic Control Conference, Atlanta, Georgia,
June 22q26 (1970). (Also IEEE Trans. AC, February,
1971).
[10] T. W. ANDERSON: An Introduction to Multivariate
Statistical Analysis. John Wiley, New York (1958).
[11] G. M. JENKIN and D. G. WAaq'S: Spectral Analysis and
Its Applications, Holden Day, San Francisco 0968).
[121 R. K. MEHRA: Identification of stochastic linear
dynamic systems. IEEE 1969 Syrup. on Adaptive
Processes (November 1969).
[13] K. J. ASTR6M and S. WENMARK: Numerical identi-
fication of stationary time series. Sixth International
Instruments and Measurement Congress, Stockholm,
Sweden (September 1964).
[14] L. MEIER, D. W. ROSS and M. B. GLASER: Evaluation
of the feasibility of using internal redundancy to detect
and isolate on-board control data instrumentation
failures. T R AFFDL-TR-70-172, Stanford Research
Institute, Menlo Park, Calif. (January 1971).
R6sum6--L'article d6crit une approche g6n6rale ~t la d6tec-
tion d'erreurs, au diagnostic et aux prognostics dans des
syst6mes pouvant ~tre d6crits ~t l'aide de mod61es math6-
matiques. Elle est bas6e sur la Th6orie des Syst6mes et la
Th6orie des D6cisions Statistiques. L'article consid6re le
cas particulier des syst6mes dynamiques lin6aires ~ entr6e
al6atoires gaussiennes et montre comment les propriet6s
statistiques du processus d'innovation peuvent 6tre utilis6es
pour la d6tection d'erreurs et le diagnostic.
Zusammenfassung--Ein allgemeines Verfahren zur Fehlerer-
kennung, Diagnose und Prognose in Systemen, die durch
mathematische Modelle beschreibbar sind, wird skizziert.
Es ist auf Systemtheorie und statistische Entscheidungs-
theorie gegriindet. Der Spezialfall linearer dynamischer
Systeme mit Gauss'schen Zufallsprozessen als Eingangs-
grOBen wird betrachtet und es wird gezeigt, wie die statisti-
schen Eigenschaften des neugestalteten Prozesses zur Feh-
lererkennung und Diagnose benutzt werden k6nnen.
Pe$~oMe--CTaTba onnebmaer o6ma~ noa~xojl K o6napy-
g e n n m omK6or, K ~aaruo3y ~ nporno3y B ci~creMax
ornlcbIBaeMbiX MaTCMaTHqeCKtiMH Mo]/eJI~IMH. OH OCHOBaH
Ha Teopnn CHCTeM H ua Teopar~ CTaTHCTHqeCKHX PemeHHfl.
CTaTb~I paccMaTpnBaeT qaCTHbn~lc.qyqafi ytnrte~'lnbix ,HMHaM-
riqeCKHX CVICTeM C cylyqa~inbiMH FayceOBCKHMrt BXO~HbIMH
Koop/IHHaTaMtt H nol~a3blBaeT KaK CTaTHCTHtleCKHeCBOI~CTBa
npouecca O6HOBYleHHII MOFyT 6bl'rb Hcno.rlb3OBarlbl ~Ylfl
06napy~renH~ omn6o~¢ a ~n~ ~Ir~arHo3a.