SET MEMBERSHIP ESTIMATION THEORY

Michele TARAGNA
Dipartimento di Automatica e Informatica
Politecnico di Torino
michele.taragna@polito.it

III level Course 01LCPIU

Experimental modeling: model building from experimental data

Academic Year 2010

 Politecnico di Torino - DAUIN M. Taragna

Example #1: estimation of a resistance value

N voltage-current measurements are performed on a real resistor, assuming that:

its static characteristic is linear the device model is given by the Ohms law
vR = R iR
the measurements are corrupted by an unknown noise
T
e = [e1 , . . . , eN ]
The following system of linear equations is derived:


vR,1 = R iR,1 + e1



vR,2 = R iR,2 + e2
..

.



vR,N = R iR,N + eN

Experimental modeling: model building from experimental data 1

 Politecnico di Torino - DAUIN M. Taragna

In matrix terms:

vR,1 iR,1 e1

vR,2 iR,2 e2

. = . [R ] + ..
.. ..
.

|{z}
vR,N iR,N o eN
| {z } | {z } | {z }
y e

is in the standard form: y = F (o ) + e

|{z} | {z } |{z}
known data known function unknown noise

F (o ) = o = linear function of the unknown parameter o

Goal: find an estimate R of R by means of an estimation algorithm (estimator)

applied to the data vector y :
R = (y)
=R

Experimental modeling: model building from experimental data 2

 Politecnico di Torino - DAUIN M. Taragna

Example #2: parametric estimation of an ARX (na , nb ) model

yj a1 yj1 a2 yj2 . . . ana yjna =
| {z }
autoregressive part (AR)
b1 uj1 + b2 uj2 + . . . + bnb ujnb +ej , j = 1, . . . , N
| {z }
exogenous part (X)

Pna Pnb m

yj = i=1 ai yji + b u
i=1 i ji + ej , j = 1, . . . , N
 T
o = a1 a2 ana b1 b2 bnb Rn , n = na + nb
 T
y = yna +1 yna +2 yN RN , N = N na

y = o + e
" #
yna y1 una una +1nb
.. .. .. .. .. ..
= . . . . . . RN n
yN 1 yN na uN 1 uN nb
Experimental modeling: model building from experimental data 3
 Politecnico di Torino - DAUIN M. Taragna

Least squares estimation errors

o : true parameters that generated the data vector y

Due to measurement noise (or uncertainty) e, it results that: y = o + e 6= o
using the least squares algorithm as estimator:

1 T T T

1 T
= y= (o + e) =
T

1 T T

1 T T

1 T
= o + e = o + e
| {z }
I

T

1 T
o = e = estimation error
e is not exactly known, but different assumptions may be made on e :
- random variable statistical
(or classical) estimation
- componentwise bounded
Set Membership estimation
- energy bounded

Experimental modeling: model building from experimental data 4

 Politecnico di Torino - DAUIN M. Taragna

Unknown But Bounded (UBB) errors

e Be = uncertainty set
 

Be = e R : |ei | , i = 1, . . . , N = e RN : kek = max |ei |
N
i=1,...,N
  ( s )
N
P PN
2 N T 2 2 N
Be = e R : e e = ei = e R : kek2 = e2i
i=1 i=1

e2 e2
2 %e %
% %e2

2
e1 e1

e3 e3
Assumption: Be is symmetric with respect to the origin of RN
Experimental modeling: model building from experimental data 5
 Politecnico di Torino - DAUIN M. Taragna

Problem: how to evaluate the uncertainty on induced by the uncertainty set Be ?

T

1 T
A= = least squares operator : RN Rn
| {z } |{z}

measurement space parameter space
1
o = T T e = Ae
o = Ae
o EU S = A[Be ] = AyA[Be ] = A [y Be ] = Estimate Uncertainty Set
y2 2

T)-1T 2
A = (
2
y

^
e
o
o 2

y Be T)-1T
A = ( EUS
y1 1
Parameter space n
y3 Measurement space N
Note that o EU S and that the distance between o and y is not greater than
Experimental modeling: model building from experimental data 6
 Politecnico di Torino - DAUIN M. Taragna

The EU S volume gives an idea of the estimate quality and, in particular,

the Estimate Uncertainty Intervals EU Ij , j = 1, . . . , n, provide this measure:
h i h m Mi
EU Ij = min j , max j = j , j R
EU S EU S
| {z } | {z }
m M
j j
the range of the j -th component of the estimate is such that:
m M
j [o ]j j
an upper bound on the estimation error of the j -th component is:
  M m
.
  2
j [o ]j  (j j ) 2
EUS
2
^
M
is the symmetry center of EU S ,

EUI2

^
2
^
o
because EU S is the image of a 2m
^

symmetric set under a linear mapping

1m ^1 1M 1
^ ^

EUI1
Experimental modeling: model building from experimental data 7
 Politecnico di Torino - DAUIN M. Taragna

Evaluation of EUS

The uncertainty set is a cube in RN centered in the origin:

 N

Be = e R : |ei | , i = 1, . . . , N
y = o + e
the set of any possible measurement (called Measurement Uncertainty Set)
is a cube in RN whose symmetry center is the data vector y :

MUS
=y Be = y R : |yi yi | , i = 1, . . . , N RN
N

the vertices of M U S
= 1, . . . , 2N
are denoted by yk , k

Theorem: EU S = A[M U S ] = conv Ayk , k = 1, . . . , 2 Rn
N
y2 y7 y8
conv{ 1 , . . . , p } : y6 y5 T)-1T 2
A = (
EUS
convex hull of the set { 1 , . . . , p} y
Ay6
Ay2

^

is the smallest convex polyhedron y4 Ay3 Ay7

Ay8
y3 Ay4
Ay5
Ay1
(polytope) containing 1 , . . . , p y2 y1
y Be 1
y1 Parameter space n
y3 Measurement space N

Experimental modeling: model building from experimental data 8

 Politecnico di Torino - DAUIN M. Taragna

Evaluation of EUIj
h m M
i
Theorem: EU Ij = j , j R 2
m
where j =
N
P 

ajk yk sign ajk ,
EUS
2M
^

EUI2
k=1

^
2
^
M m
j = 2 j j , o
2m
^
 
A = ajk = (T )1 T ,
h i
= j = Ay
1m ^1 1M 1
^ ^
m
= min j = min (Ay)j =
Proof: j
EU S yM U S EUI1
N
P N
P N
P
= min ajk yk = min ajk yk = min ajk yk
y:|yi yi |k=1 y:yi yi k=1 y:yi yi yi +k=1
i=1,...,N i=1,...,N i=1,...,N
= yk if ajk > 0, or by yk = yk + if ajk < 0.
and such a minimum is achieved by yk
Since M U S = y Be is symmetric with respect to the data vector y , then

EU S = A [M U S ] is symmetric with respect to the estimate = Ay and then:

m M
. M m
j = ( j + j ) 2, j = 1, . . . , n j = 2j j , j = 1, . . . , n

Experimental modeling: model building from experimental data 9

 Politecnico di Torino - DAUIN M. Taragna

Description of ellipsoids
Let x be an ellipsoid in RN centered in xo :
n o
T
x = x RN : (x xo ) 1
x (x x o
)1
The form matrix x RNN is symmetric and positive definite it is invertible
The directions of the main axes of x are given by the eigenvectors ui of x ,
which are orthogonal because x is positive definite
p
The lengths of the semi-axes of x are given by i (x ),
where i (x ) is the i-th eigenvalue of x
u1
O1
x2 O2
u2 u1
u2

xo
:x

x1

Experimental modeling: model building from experimental data 10

 Politecnico di Torino - DAUIN M. Taragna

Linear transformation of ellipsoids

Let x be an ellipsoidnin RN centered in xo : o

o T
x = x RN : (x x ) 1
x (x x o
) 2

and consider the linear transformation:

z = P x Rn , with P RnN , n < N
Theorem: if rank (P ) = nn, then o
T
z = P [x ] = z Rn : (z z o ) 1
z (z z o
) 2

z o = P xo Rn , z = P x P T Rnn
x2
x P z2

xo z
Pxo

x1 z1
x3 Space N Space n , n<N

Experimental modeling: model building from experimental data 11

 Politecnico di Torino - DAUIN M. Taragna

2
Evaluation of EUS

The uncertainty set is a sphere in RN centered in the origin:

2
 N T 2

Be = e R : e e
y = o + e
the set of any possible measurement (called Measurement Uncertainty Set)
is a sphere in RN whose symmetry
n center is the data vector y : o
T
M U S 2 = y Be2 = y RN : (y y) (y y) 2 RN
  n o
Theorem: EU S 2 = A y Be2 = Rn: ( )T T ( ) 2 Rn
is an ellipsoid in Rn with = Ay as symmetry center and (T )1 as form matrix
y2

T)-1T
A = ( 2
2
^
M EUS 2
y
EUI22
^

2
^
o
o
y Be2 T)-1T
A = (
2m
^

1m ^1 1M 1
^ ^
y1
y3 Measurement space N
EUI12
Parameter space n

Experimental modeling: model building from experimental data 12

 Politecnico di Torino - DAUIN M. Taragna

  n o
Theorem: EU S 2 = A y Be2 = Rn: ( )T T ( ) 2 Rn
is an ellipsoid in Rn with = Ay as symmetry center and (T )1 as form matrix
y2

T)-1T
A = ( 2
2
^
M EUS 2
y

EUI22

^

2
^
o
o
y Be2 T)-1T
A = (
2m
^

1m ^1 1M 1
^ ^
y1
y3 Measurement space N
EUI12
Parameter space n
Proof: by definition, EU S 2 is the linear mapping of M U S 2
= y Be2 through the matrix A:
2
 2
 n n T T 1 2
o
EU S = A y Be = R : ( Ay) [AA ] ( Ay)
But Ay = , A = (T )1 T and then:
n o
AAT = (T )1 T [(T )1 T ]T = (T )1 T [(T )1 ]T =
= (T )1 T [(T )T ]1 = (T )1
| {z }
I

Experimental modeling: model building from experimental data 13

 Politecnico di Torino - DAUIN M. Taragna

2
Evaluation of EUIj
 h m Mi 
Theorem: EU Ij2 = j j , j + j = j , j R
| {z } | {z }
m M
j j
rh i
1
j = (T )
jj
2
2M EUS 2
^
EUI22

2
^
o

2m
^

1m ^1 1M 1
^ ^

EUI12
Experimental modeling: model building from experimental data 14
 Politecnico di Torino - DAUIN M. Taragna

Optimal (with minimal uncertainty) estimates

Is the EU S the smallest set containing the true parameter o ?
Are the EU Ij the smallest possible uncertainty intervals?
Does the LS estimator provide the minimal uncertainty intervals?
To answer all these questions, it is necessary to analyze the set of all the parameters
that are consistent with both the data and the available information on noise

Definition: a parameter is said to be feasible (or consistent) if (y ) Be

n o
FPS = Rn : (y ) Be = Feasible Parameter Set =
= set of all the parameters consistent with both the data and
the prior information on noise and on the estimation problem
F P S is independent of the estimation algorithm
If data are generated by the true parameter o , then o is feasible; in fact:
y = o + e, e Be y o = e Be o F P S

Experimental modeling: model building from experimental data 15

 Politecnico di Torino - DAUIN M. Taragna

Relationship between FPS and EUS

Theorem:
F P S EU S
F P S , then
Proof: if h i
(y ) Be y Be A A [y Be ] = EU S
h i
But A = (T )1 T = and then EU S .

The Parameter Uncertainty

h Intervals P U Ij , j = 1, . . . , n are defined as:
i h i
P U Ij = min j , max j = jm , jM R
F P S F P S
| {z } | {z }
jm jM
from the above theorem:
P U Ij EU Ij , j = 1, . . . , n

m M
j jm M
[o ]j j j

Experimental modeling: model building from experimental data 16

 Politecnico di Torino - DAUIN M. Taragna

Evaluation of FPS and PUIj
 N

If F P S , then (y

) Be = e R : |ei | , i = 1, . . . , N
Ti : i-th row of
   
   T 
 (y )i = y
 i i  , i = 1, . . . , N
n   o
F P S = Rn : yi Ti  , i = 1, . . . , N
 

i.e., F P S is a polytope (a convex polyhedron) generated by linear inequalities:

 
 T  T T
y
 i i  yi i yi i yi +
h i h i
Moreover, P U Ij = min j , max j = jm , jM R
F P S F P S
| {z } | {z }
jm jM
m M
with j and j solutions of linear programming problems of the standard form:
minx cT x with the constraint: Ax b

Experimental modeling: model building from experimental data 17

 Politecnico di Torino - DAUIN M. Taragna

2 2
Evaluation of FPS and PUIj
n   o
Theorem: F P S 2 = Rn : ( )T T ( ) 2 2
2 = (y )T (y ) = k y k22 2
= fitting error between measured outputs and estimated outputs
a greater fitting error a smaller F P S 2 a lower uncertainty on parameters
h p p i h i
Moreover, P U Ij2 = j j 2 2 , j + j 2 2 = jm , jM R
| {z } | {z }
jm jM T2
q EUS 2
j = [(T )1 ]jj TM

PUI22
EUI22
T T
2 ^

T2
^
o

T 2m FPS 2

T1m T^1 T1M T1

PUI12
EUI12

Experimental modeling: model building from experimental data 18

 Politecnico di Torino - DAUIN M. Taragna

Optimal estimates
Definition: given an estimate , the estimate error E() is given by:
E() = sup k k
F P S
opt
Definition: an estimate is optimal if:
opt
E( ) E(), Rn
Central estimate:
C
h Ci C
.
= j , where j = (jm + jM ) 2, j = 1, . . . , n
the central estimate is optimal both if Be = Be2 and if Be = Be , since:
 C
 .
M
jm ) 2, j = 1, . . . , n
 
[
 oj ] j  ( j

LS
2
if Be = Be , the least squares estimate = (T )1 T y is central
LS
is optimal if Be = Be2 , but in general it is not optimal if Be = Be

Experimental modeling: model building from experimental data 19

 Politecnico di Torino - DAUIN M. Taragna

Example: parametric estimation of a position transducer model

Position transducer
8

2
Voltage Vz in V

8
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045
Position z in m

The static characteristic of the position-voltage transducer is nearly linear in the range
between 1.3 e 3.5 cm the characteristic can be linearly approximated by:
Vz = Kt z + Vo
Experimental modeling: model building from experimental data 20
 Politecnico di Torino - DAUIN M. Taragna

In the linearity interval between 1.3 e 3.5 cm:

Vz = Kt z + Vo
|{z} |{z}
unknown unknown
The most relevant error occurs in the position z measurement and it is not greater
than 0.5 mm to account for this error, the model equation can be rewritten as:
1 Vo
z= Vz +e
Kt Kt
where the unknown parameters are:
1 Vo
1 = , 2 =
Kt Kt
The N measurements taken in the linearity interval form a system of equations:
z1 = Vz,1 1 + 2 + e1
z2 = Vz,2 1 + 2 + e2
..
.
zN = Vz,N 1 + 2 + eN
Vz,i : voltage provided by the transducer when the position value is zi
Experimental modeling: model building from experimental data 21
 Politecnico di Torino - DAUIN M. Taragna

In matrix form:
z V 1 e1
1 z,1
1 1 e2

z2 Vz,2
.. = .. .. + ..
. . . 2 .
zN Vz,N 1 eN
i.e., the estimation problem is in the standard form:
y =+e
where y RN , RN 2 , e RN and the unknown is R2
Using the Least Squares estimation algorithm:
T 1 T
= A" y , #with A
" = ( ) #
1 1.8194 103
= =
2 2.2791 102

1 2
Kt = = 549.62 V/m, Vo = = 12.526 V
1 1
Experimental modeling: model building from experimental data 22
 Politecnico di Torino - DAUIN M. Taragna

Evaluation of the Estimate Uncertainty Intervals EUIj

N
 N N

e Be= e R : |ei | , i = 1, . . . , N , = 5 104
h m M
i
EU Ij = j = min j , j = max j , j = 1, 2
EU S EU S
m PN
k=1 ajk [yk sign (ajk )]
j = min j =
EU S
M PN m
j = max j = k=1 ajk [yk + sign (ajk )]
= 2 j j
EU S
h m Mi
1 , 1 = [1.6996 103 , 1.9392 103 ]
h m Mi
2 , 2 = [2.2291 102 , 2.3291 102 ]
h i h M m
i
Ktm , KtM = 1/1 , 1/ 1 = [515.67, 588.36] V/m
h i h M m m M
i
Vom , VoM = 2 /1 , 2 /1 = [13.703, 11.495] V

Experimental modeling: model building from experimental data 23

 Politecnico di Torino - DAUIN M. Taragna

Envelope of the static characteristics of models whose parameters are taken as the
extremes of the Estimate Uncertainty Intervals EU Ij , j = 1, 2
Position transducer
10
Experimental data (z, Vz)
z min(M * V , m * V ) + m
8 1 z 1 z 2
z = 1 * Vz + 2
z max(M 1
* Vz , m
1
* Vz) + M
2
6

2
Voltage V in V
z

linearity interval
10
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04
Position z in m

Experimental modeling: model building from experimental data 24

 Politecnico di Torino - DAUIN M. Taragna

Evaluation of the Parameter Uncertainty Intervals PUIj
n   o
dim() 
F P S = R : yi [ ]i  , i = 1, . . . , N

h i
P U Ij = min j , max j EU Ij , j = 1, 2
F P S F P S
The extremes of P U Ij , j = 1, 2, are solutions of the linear programming problems

min j = min cT
" # " #
F P S M b
y
M= , b= + , c = j -th column of I22
max j = min (c)T y
F P S M b

h i
m
1 = min ,
1 1 M
= max 1 = [1.7909 103
, 1.8484 103
]
F P S F P S
h i
m
2 = min 2 , M
2 = max 2 = [2.2596 102
, 2.2807 102
]
F P S F P S
 m M h i
Kt , Kt = 1/ M 1 , 1/ m
1 = [541.01, 558.38] V/m
 m M h M m i
Vo , Vo = 2 /1 , m 2 / M
1 = [12.735, 12.225] V

Experimental modeling: model building from experimental data 25

 Politecnico di Torino - DAUIN M. Taragna

Envelope of the static characteristics of models whose parameters belong to the

Feasible Parameter Set F P S
Position transducer
10
Experimental data (z, V )
z
z = 1 * Vz + 2
8

2
Voltage V in V
z

linearity interval
10
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04
Position z in m

Experimental modeling: model building from experimental data 26

 Politecnico di Torino - DAUIN M. Taragna

Feasible Parameter Set F P S (continuous line) and set of estimates given by the
extremes of Parameter Uncertainty Intervals P U Ij , j = 1, 2
Feasible Parameter Set FPS
0.0229

0.0228

0.0228
LS
(y)

0.0227
2

0.0227

0.0227

0.0226

0.0226
1.79 1.8 1.81 1.82 1.83 1.84 1.85
3
1 x 10

Experimental modeling: model building from experimental data 27

 Politecnico di Torino - DAUIN M. Taragna
" #   " #
C min 1 + max 1 2
C 1 F P S F P S
1.8196 103
Note that: =
C
=    =


2 min 2 + max 2 2 2.2702 102
F P S F P S
. .
KtC = 1 C 1 = 549.56 V/m, V o
C
= C
2 C
1 = 12.476 V

Feasible Parameter Set FPS
0.0229

0.0228

0.0228
LS
(y)

0.0227
2

0.0227
C(y)

0.0227

0.0226

0.0226
1.79 1.8 1.81 1.82 1.83 1.84 1.85
3
1 x 10

Experimental modeling: model building from experimental data 28

 Politecnico di Torino - DAUIN M. Taragna

Essential references
F. C. Schweppe, Uncertain Dynamics Systems. Englewood Cliffs, NJ: Prentice Hall, 1973.
M. Milanese, R. Tempo, A. Vicino (editors), Robustness in Identification and Control. New York:
Plenum Press, 1989.
M. Milanese, A. Vicino, Optimal estimation theory for dynamic systems with set membership
uncertainty: an overview, Automatica, vol. 27, no. 6, pp. 9971009, 1991.
Special Issue on System Identification for Robust Control Design, IEEE Transactions on
Automatic Control, vol. AC-37, no. 7, pp. 899974, 1992.
R. S. Smith, M. Dahleh (editors), The Modeling of Uncertainty in Control Systems, vol. 192 of
Lecture Notes in Control and Information Sciences. London, UK: Springer-Verlag, 1994.
M. Milanese, J. Norton, H. Piet-Lahanier, E. Walter (editors), Bounding Approaches to System
Identification. New York: Plenum Press, 1996.
J. R. Partington, Interpolation, Identification, and Sampling, vol. 17 of London Mathematical
Society Monographs New Series. New York: Clarendon Press - Oxford, 1997.
A. Garulli, A. Tesi, A. Vicino (editors), Robustness in Identification and Control, vol. 245 of
Lecture Notes in Control and Information Sciences. Godalming, UK: Springer-Verlag, 1999.
J. Chen, G. Gu, Control-Oriented System Identification: An H Approach. New York: John
Wiley & Sons, Inc., 2000.

Experimental modeling: model building from experimental data 29