A Metric in Angularly

Quantized Polar Domain for

Pole-Based Classification of
Stochastic Models and
Dynamic Systems EGNIYA
Technical Report

August 2016

EGNIYA
 A Metric in Angularly
Quantized Polar Domain for
Pole-Based Classification of
Stochastic Models and
Dynamic Systems
Abstract Autoregressive (AR) models are
used in a variety of applications and an AR
model can be represented by the poles
corresponding to describing AR coefficients.
Similarly, the behavior of linear dynamic single-
input-single-output (SISO) systems can be
described as a function of the system poles,
which are directly estimated from the given data
and represent a system as a set of poles without
any identities, which is analogous to the nature
of association-free multi-target tracking and
corresponding application of set distances
known as optimal subpattern assignment
(OSPA) distance. In this work, we define a new
metric in terms of sums and differences of
magnitude and angles of poles in pole-space and
provide a measure of the distance between AR
processes, or linear dynamic systems
represented by poles.
Index TermsAutoregressive (AR) models,
classification, stochastic models, metric, distance
measures, poles, linear dynamic systems
I. INTRODUCTION
In many applications, autoregressive (AR)
models are increasingly popular. In radar signal
processing, the properties of the Gaussian clutter
can be analyzed by modeling it by an AR process.
In biomedical applications, AR models can be
used to classify signals of patients with a specific
pathology from signals recorded with healthy
people. In speech processing such as speech
analysis and Kalman-filter based enhancement,
sets of AR parameters { } estimated by
different methods, are often compared one
another. [10] Another area is classification of
systems described by poles which includes
several similar but slightly different topics.
Change detection considers the problem of
finding modifications to the system based on
observations. Anomaly detection is concerned
with finding elements or behavioral patterns
deviating from elements or patterns that are
defined as normal. Outliers detection addresses
the problem of finding elements of a set that
deviate markedly from other set members. Fault
detection is concerned with the problem of
recognizing system behavior or states that
exceed a permitted region of operation. Concept
drift addresses the very slow and gradual
changes in processes. Novelty detection emerged
from the field of machine learning and has its
focus on discerning incoming information with
respect to its coverage by already learned
models and pattern. In all those areas, a form of
model/system comparison (based on either
dissimilarity/distance/divergence or contrast)
and consequent classification is required. [9]
Commonly used dissimilarity measures between
AR-model pairs include the Itakura divergence,
the Itakura-Saito divergence, the log-spectral
distance or Jeffreys divergence (JD) which are
based on spectral characteristics of the
processes. [9],[6] One common distance
measure between systems is based on the
spectral characteristics, namely a distance
between the cepstra, the inverse Fourier
transform of the logarithm of the power
spectrum. The ARMA distance, a metric for
ARMA processes as the distance of two cepstra,
can be calculated by a function of the system
poles. [4] Another distance measure between
systems is association-free distance measure,
namely the optimal subpattern assignment
(OSPA) distance (and a variant of it called as
MAX-OSPA), the system distance as a distance
between their sets of poles representing the
spectral characteristics of the system. [1], [2], [9]
The key to comparison and classification of
models or systems represented by poles is a set
of representative and intuitive parameters, (in
order to define a distance measure as a basis for
decision-making) which can be the set of the AR
parameters or system coefficients, the set of
poles or the corresponding PSDs. [9], [10] In this
work, the main contribution is a new approach
and derivation of a novel metric based on poles
which can be used for classification of AR
processes, or classification, identification or
change detection in linear dynamic systems
represented by system poles.
The remainder of this paper is structured as
follows. First, in Sec. II we give a formal
description of the problem along with
parametric models, pole estimation and existing
approaches. Sec. III introduces the classification
of stochastic models and linear dynamic systems
in pole-space. We derive an efficient novel
metric in pole-space in Sec. IV. A numerical
example along with a comparison of the
proposed metric is presented in Sec. V. Finally,
Sec. VI concludes the work.
1
 II. PROBLEM STATEMENT D. Existing Approaches

A. Mathematical Model There are several proposals in literature using

information on the positions of poles,
The distance between a reference model amplitudes of characteristic frequencies and
() ()
and a test model calculated on basis of subspace formulations [10], which constitutes
() () different metrics and distance measures used for
their parameter vectors and , which are
classification of models and systems. We are
usually estimated from the input and observed briefing some popular ones and shortly
output data, mentioning about similarities and limitations of
() () () ()
others next in Section III.
= ( , ) = ( , ) (1)
III. POLE-BASED DISTANCE MEASURES
where the superscripts () and () represents
the reference and test models respectively,
which are either two models in AR models space A. Base Distance of Poles
or two systems in linear time-varying systems
space. [9] The base distance (. , . ), with : + as a
function satisfying identity, symmetry, and the
B. Parametric Models triangle inequality. The base distance is the
()
distance between two complex poles and
Autoregressive (AR), moving-average (MA) and ()
The plane, in which the poles of a
autoregressive moving average (ARMA) models discrete-time system are defined, can be
are useful time-domain models for the interpreted as a specific Mobius transformation
representation of discrete-time signals. The time from the more intuitive plane of a
series is called as an ARMA(, ) model if continuous-time system that transforms the

Cartesian coordinate system of the plane into
= = + = (2) a polar coordinate system.[9] Let and be two
poles of a continuous-time system in the
where is uncorrelated noise of variance . In plane. The base distance (, ) between
domain the system function is and is defined as

=
( /)
() = = =
(3) (, )
= =( /)
= [() ()] + [() ()]
where = = , the s are the AR
coefficients, s are the MA coefficients, and (4)
are the poles and zeros of the model which is the Euclidian distance between and
respectively. [4] in the complex plane. Let and be two poles of
a discrete-time system in the plane. The base
C. Pole Estimation distance (, ) between and is defined as

There are different approaches for the (, ) = [() ()] + [
] (5)

estimation of the poles of a system but our main
focus in this work is providing a metric for a where is the sampling period, , and , are
given set of system or model poles rather than polar coordinates of and respectively. [9]
estimation or statistical properties of the
estimator, so we assume we have different sets B. OSPA and MAX-OSPA Distance
of poles obtained using a proper pole estimation
method. The optimal subpattern assignment (OSPA)
distance between two sets of poles
However, in systems with real valued output, () () () () () ()
poles occur either real or in complex conjugate { , , , } and { , , , } each
pairs and a known proportion of real and comprising elements and given by the ordered
complex poles can be assumed. Since an a prior () () () ()
vectors = [ , , , ] and
estimate is easy to attain from a small number of () () () ()
data samples, this assumption is not unrealistic. = [ , , , ] is defined by

2
 () () () ()
( , ) ( , )
/
= =( () () () ()
) = =( )
() () = [ () () () ()
]
= [ ( ( , () ) )] = =( ) = =( )

= (9)
()
() ()
with , ( , ) is the base distance which is in the form of finite products in pole
() () domain. [4]
between the two elements and and
describes all permutations of the set {, , , }. E. Limitations & Requirements
()
The notation () represents the -th element of Most of the existing or proposed metrics or
()
the permutation of which is generated by measures use either norms for distances
between poles in domain or transformed
reordering the vector. Given two sets of poles by
() () poles in domain assisted with some
the two vectors and , the MAX-OSPA weighting schemes. The pole representation in
distance is defined by either in domain or in domain is a two
dimensional representation and any distance
() () () () measure based solely on this two dimensional
( , ) = =,,.. ( ( , () )) representation (which corresponds to a pattern
(7) with two features forming a two dimensional
where is the optimal OSPA permutation given feature vector) is far from measuring the
by distance between corresponding
models/systems. Besides a distance measure in
() () pole-space must consider not only the distance
= [ =,,.. ( ( , () ))]
between poles of reference and test models but
(8) also the distance among the poles (intra-pole
distance) of each model/system. So the distance
So, the maximum base distance between two among poles of a model is a factor as well, which
() () might be as important as the closeness of a pole
single elements of a pair and is chosen,
while using the optimal subpattern assignment to the unit circle. In other words, the metrics or
over the maximum norm ( ) resulted in distance measures based solely on two
minimized maximal base distance. dimensional representation might suffer from
loss of available information. ARMA metric can
The OSPA distance finds the optimal assignment be thought as a nonlinear transformation of two
by minimizing over the sum of distances dimensional space into a higher dimensional
between poles. The MAX-OSPA chooses an space and might not have suffering from loss of
optimal assignment by minimizing the maximum information.
distance between two poles.
The main requirement can be defined as reliable,
In OSPA and MAX-OSPA a weighting scheme is analytically tractable, physically meaningful and
generally used (usually as the exponential computationally efficient metric to measure
weighting) in order to take closeness of poles to distance between models or systems in their
unit circle (or imaginary axis in domain) into associated spaces. We are outlining a new
account [9] approach and a new metric based on this
requirements avoiding majority of the
C. ARMA Metric limitations next in Section IV.

ARMA metric is one of the popular metrics IV. PROPOSED APPROACH & METRIC
whose derivation is based on cepstrum domain
processing or homomorphic processing. For two A. Preliminaries
()
stable AR models a reference model and a
()
test model with orders and two sets of A model/system can be represented by poles,
poles
() () ()
{ , , , } and
() () ()
{ , , , } coefficients or PSD. Although there is not a one-
to-one correspondence, pole representation can
the ARMA metric is defined as; be thought as a vector quantized form, or low
rank approximation or projection onto a lower
dimensional space, in other words it contains all

3
the information contained by PSD in a lower () () ()
positions of both and components in
dimensional space. Our aim is extracting this ()
is based on angular quantization and the
and
available information contained in poles to
value of ).
measure the distance between two
models/systems.
This operation (angular quantization and
obtaining model/system vectors) can be thought
B. Approach
as transforming the original pole vectors to
vectors by a rank matrix which contains
Consider two (as pole vectors, consisting of () ()
poles as components of the vector) (or ) as components in the
() () () () respective diagonal elements. (and also provides
= [ , , , ]
a link to cepstrum, FFT and PSD domains)
() () () ()
= [ , , , ] (10)

() ()
where and represents the pole vectors
of reference and test models/systems
() ()
respectively. and are the poles which
can be written as
() () ()
=
() () ()
= (11)
in polar domain. The key idea is extracting
information contained in poles by using a
transformation to a higher dimensional space in
such a way that each pole (actually magnitude of
each complex pole) corresponds to a coordinate
in the new transformed space and each Figure 1. Selection of angular
model/system can be represented as a point quantization parameter (number of
(which is defined by the coordinates of the angular sectors) based on minimum
model/system) in this high dimensional space, angular separation between poles
(which can be thought as a model/system space Although we will not use angular sectorization
where magnitudes of poles forms the and high dimensional space in final metric
coordinates as aligned vectors with a subgroup definition, it is useful in derivation. Selection of
of the axes of new space) . is important and one condition on that comes
from representability of each model/system as a
Assume that polar domain is quantized angularly point in the new high dimensional space, which
(with angular sectors, each angular sector means each angular sector must contain only
width is /, = / and they are one pole of either reference or test model
numbered from 1 to in counterclockwise otherwise none. This can be satisfied by
direction starting from (0,1) point of complex selecting considering the minimum angular
plane) as shown in Fig.1, each non-empty (not separation between poles of either test or
containing any poles) angular sector contains reference model, whichever is smaller. (This
either one pole of reference model or one pole of operation takes the distance among poles of
test model. model/system (intra-poles distance) argument
in previous section into account)
If we think each angular sector as an axis of the
new space (which is the high dimensional C. Proposed Metric
transform space with dimension ), then each
model/system can be uniquely represented as Since we have -dimensional vector space and
dimensional vectors () and
() which are models/systems represented as points in this

space as

() = [, , . . () , ()

()
, , , , ]

() = [, , . . () , ()

()
, , , , ]


() = [, , . . () , ()

()
, , , , ] (12)

() = [, , . . () , ()

()
, , , , ] (13)

now model/system vectors for reference and
test models/systems respectively.(note that, the

4
then we can use familiar vector space properties weighting is applied as each weight defined as
and define the metrics to measure distance the angular separation between corresponding
between the models/systems. poles,
() ()
= | | | | (15)
Although usual vector space metrics ( and
or -norms for example) can be used to measure Actually this weighting scheme can be
distance between two models represented by considered as the corollary of angular
vectors () and () , this would be a coarse sectorization as radial quantization, for each

distance measure (does not take angular sector radial quanta each pole angle represents one
index or angular separation into account, and coordinate in the axes of the high dimensional
results in virtual equidistant models) and space formed by radial quantization and weights
requires formation of those -dimensional as pole angle differences are the absolute value
vectors implying realization of angular metrics on this new high dimensional space
sectorization by search methods or matrix constructed by radial quantization. Then
multiplication which we avoid due to weighted distance measure can be written as;

computational efficiency considerations. To be
able to take effect of angular separation into () ,
= ( () ) = |
() ()
() ()|

account and in fact to be able to provide a solid =
link to cepstrum, FFT and PSD domains a form (16)
of weighting should be applied as well. where is the diagonal weighting matrix
Weighting can be applied either in complex with each diagonal element is in the form of
domain as pre-weighting (which is a transform multiples of ,
() ()
of vectors and by a real valued

diagonal weighting matrix) or in high
. .
dimensional transform space as post weighting =[ ] (17)
.
(which is a transform of vectors () and
.
()
by a real valued diagonal weighting

matrix). In first case since original poles move In limiting case when any two poles are too
through radial directions the distances between close, and , which allows us to
models/systems are preserved to a possible write Eq.16 as

scale factor. In latter case, the post weighting
matrix transforms dimensional linear vector () ,
( () ) ()
= (| | | |)( + )
() () ()

space and distances are defined on =
hyperellipsoids, in other words the distances can (18)
be called as well-known Mahalonabis distances. However, although defined for , the
() ()
Before going back to dimensional pole space distance is valid for to satisfy
and defining the metric in that space we initial assumptions that each angular sector
continue with derivations in dimensional contains only one pole of either test or reference
space. model or not any poles. Using similar arguments
for radial quantization as used in angular
When the minimum angular separation is sectorization gives the final form of the metric in
chosen as minimum angular separation between pole space which we call as (PSM stands
the angularly nearest two poles, the respective for pole space metric) and defined as;
() () ()
positions of both and components in
() ()
and () is different and absolute value metric = ( , )

()
can be written as sum of components and () () () ()
() = |( )(| | + | |)
.
=

= () ,
( () ) = [| () () () ()|]
() () () ()
+ (| | | |)( + )|
=
=
() () (19)
= ( + ) In vector notation, it can be rewritten as;
=
(14) () ()
As mentioned, this would be a coarse distance ( , ) = | + |
measure producing virtual equidistant models where
() () () ()
and to take effect of angular separation a form of = [( ), , ( )]

5
 () ()
() ()
= [( + ), , ( + )]
() () = . / = .
() () () ()
= [(| | | |), , (| | | |)]
()
() () () ()
and a test model with same poles except and
= [[(| | + | |), , (| | + | |)]] ()
complex pole pair, which are the only
(20)
difference between reference and test models
It should be noted that this form of the metric is
and those two complex pole pair move radially
association-free or valid for unlabeled or non- () ()
ordered pole pairs of test and reference models (while phase kept same, magnitudes and
which avoids permutations and also performs changes between 0 and 1) in first scenario and
well for both higher order models/systems or moves angularly (while magnitude kept same,
() ()
applications requiring more than two model phases and changes between 0 and 2)
classification. in second scenario.

As seen from Eq.19, computational load is In first scenario, since the reference model has
relatively low, can be computed by () ()
poles at = . / and = . /
multiplications and ( + ) additions, and can when the radially moving complex pole pair of
be used for high order model classification test model coincides with those then the
problems as well. We are dealing with a distance becomes zero, as the complex poles pair
numerical example next in Section V. moves away from that point the distance
between models increase as expected, as shown
V. NUMERICAL ANALYSIS in Fig 2.

There might be some applications where pole

dominancy or movement of some poles might be
important as in change detection or system
identification problems. We use pole movement
to illustrate the numerical evaluation of the
metric and provide a comparison with ARMA
and OSPA metrics.

() ()
The ARMA metric ( , ) used in
comparison simulations and graphs are actually
squared form of the ARMA metric given in Eq (9)
so can be written as;

() ()
( , )
Figure 2. Behavior of Proposed Metric PSM
= =( () () () ()
) = =( )
= [ ] and comparison with ARMA & OSPA metrics
() () () ()
= =( ) = =( ) for radially moving complex pole pair

() ()
The PSM pole space metric ( , ) is In second scenario, since the reference model
used as it is; ()
has poles at = . / and =
()

/
() () . when the angularly moving complex
( , ) = | + | pole pair of test model coincides with those then
the distance becomes zero, as the complex poles
and the OSPA metric is used without number of pair moves away from that point the distance
poles normalization with = as; between models increase as expected, as shown
in Fig 3.

() () () ()
( , ) = [ ( ( , () ))]

=

Consider a reference model with

() ()
= . / = . /
() ()
= . / = . /
() ()
= . / = .

6
 Figure 3. Behavior of Proposed Metric PSM
and comparison with ARMA & OSPA metrics
for an angularly moving complex pole pair
In both scenarios, the proposed metric
behaves properly as expected and this behavior
is very similar to ARMA metric, those two
scenarios summarize the overall picture of
classification of models or systems based on
their pole characterization, test model/system
poles move away or get closer to reference
model/system poles in time varying systems,
and at a distance in static model/system
classification, so the PSM metric can be
used in a great range of problems, especially for
the ones with computational sensitivities.
Figure 4. Computational requirements of
ARMA and PSM metrics in terms of
multiplications
Figure 5. Computational requirements of
ARMA and PSM metrics in terms of additions
The computational comparison in terms of
number of multiplications and number of
additions are given in Figure 4 and Figure 5 for
PSM and ARMA metrics. The reason OSPA metric
is not included in that plots is due to huge
number of both multiplications and additions
required by OSPA metric due to permutation
operation which makes computational load
proportional to ! . Just to show effect of this
permutation operation on computational load
we provided another plot which shows the
number of multiplications required by each
metric for number of poles up to ten in Figure 6.
In all those three graphs the axis showing the
required number of computations is logarithmic.
Figure 6. Computational requirements of
ARMA,OSPA and PSM metrics in terms of
multiplications
As seen from the graphs, the PSM metric is the
least demanding in terms of both multiplications
and additions and does not require
permutations for non-ordered set of poles,
which makes it suitable for
classification/identification of state changes in
7
linear dynamic systems as well as applications [7] K. De Cock, B. De Moor, Subspace angles
involving high model orders and more than two between ARMA models, Systems Control Lett. 46
model classification requirements. (4) (July 2002) 265270.

VI. CONCLUSIONS [8] G.H. Golub, C.F. Van Loan, Matrix

Computations, The Johns Hopkins University
We have described a new approach for pole Press, Baltimore, 1996.
based distance measure and derived a
computationally efficient ( multiplications [9] C. Chlebek and U. D. Hanebeck, Pole-based
and (( + ) additions), analytically tractable Distance Measure for Change Detection in Linear
and physically meaningful pole-based metric. Dynamic Systems, in Proc. 2014 European
Control Conference (ECC 2014), Strasbourg,
The metric can be used in classification of France, June 2014.
models and systems which has a pole-space
representation and also in detection, estimation [10] C. Magnant, E. Grivel, A. Giremus, L. Ratton,
and classification of change in model parameters B. Joseph, Classifying Autoregressive Models
or system state changes in time varying AR Using Dissimilarity Measures: A Comparative
models or linear time varying dynamic systems. Study, in 23rd European Signal Processing
Conference(EUSIPCO)
Our main focus was on stable AR models and
systems represented by poles, however the
approach can be extended to involve other
models and might work for non-stable non-
minimum phase systems or the systems
represented by entities both inside and outside
of the unit circle of complex plane.

REFERENCES
[1] D.Schuhmacher, B.-T. Vo, and B.-N. Vo, A
Consistent Metric for Performance Evaluation of
Multi-Object Filters, IEEE Trans. Signal
Processing, Vol. 56, No. 8, pp. 3447-3457, Aug.
2008

[2] B. Ristic, B.-N. Vo,D. Clark and B.-T. Vo, A

Metric for Performance Evaluation of Multi-
Target Tracking Algorithms, IEEE Trans. Signal
Proc., Vol. 59, No. 7, pp. 3452-3457, July. 2011

[3] H. W. Kuhn, The Hungarian Method for the

Assignment Problem, Naval Research Logistics
Quarterly, 2:8397, 1955..

[4] R.J. Martin, A metric for ARMA processes,

IEEE Trans. Signal Process. 48 (2000) 1164
1170.

[5] P. Van Overschee and B. De Moor., Subspace

identification for linear systems: Theory
Implementation-Applications. Kluwer Academic
Publishers, 1996.

[6] M. Basseville, Distance measures for signal

processing and pattern recognition, Signal
Processing 18 (4) (December 1989) 349369.