SDRE-1

State-Dependent Riccati Equation (SDRE) Design for Nonlinear Control Systems

A systematic design technique for nonlinear control systems. The SDRE strategy is nothing but pointwisely apply the LQR scheme at every nonzero state.

SDRE-2

Outline

Linear Quadratic Regulator (LQR) Algebraic Riccati Equation (ARE) State-Dependent Riccati Equation (SDRE)

SDRE-3

The LQR Design

Consider the following linear control system and performance index:
x = A x + Bu R1 and J = 0 (xQ 0 x + uR0 u)dt

where x 2 R n ; u 2 R p ; Q 0 0 & R0 > 0 : Objective: To organize a control u such that the origin of the closed-loop system is stable and the cost J is minimum.

SDRE-4

LQR solution: If (A,B) is stabilizable and (A; Q 0 ) is detectable, then the optimal control is known to be
1 T u = R B Px 0

where P = P T 0 is the solution of the following ARE:

1 T B P + Q 0 = 0: ATP + PA PBR 0

Question: Does the solution of the ARE stated above always exist?

SDRE-5

Algebraic Riccati Equation (ARE)

Lyapunov equations are useful in system analysis, while AREs are useful in control system synthesis. Recall the ARE has the form:
1 T B P + Q 0 = 0: ATP + PA PBR 0 1 T B Let X = P; Q = Q 0 & R = BR 0

ATX + XA XRX + Q = 0 :

(ARE)

where R = RT 0 & Q = Q T 0 :

SDRE-6

If n = 1, the ARE becomes rx 2 + 2 ax + q = 0

p ) a rx = a2 + rq )x=
a

a2+rq

Note that, at most one x such that (a rx ) C Question: Under what conditions does the solution X of the ARE exist and satisfy (i) real and symmetric. (ii) ATX + XA XRX + Q = 0 : 1 T B P ) C : (iii) (A RX) = (A BR 0

SDRE-7

Associated with the ARE ATX + XA XRX + Q = 0 ; where A; Q; R 2 R n n ; R = RT 0 & Q = Q T 0 ; we introduce the following Hamiltonian matrix
H := A R Q AT 2 R 2 n 2 n

Question: What properties does the Hamiltonian matrix have?

SDRE-8

Lemma 1: The spectra of H, (H); are symmetric about the real and the imaginary axes. That is,
2 (H) ) ; 2 (H): 0 In ) 2 = I2 n ) 1 = : Proof: Let := I 0 n

) 1 H = H = HT

) H & HT are similar.

Thus, 2 (H) , 2 (H): Moreover, 2 (H) , 2 (H); because H is a real matrix. This completes the proof.

SDRE-9

Corollary 2: Suppose that H has no imaginary eigenvalues. Then there are exactly n eigenvalues on C and n eigenvalues on C + Question: When does H have no imaginary eigenvalues? To be discussed later.

SDRE-10

Question: How to solve and what are the solutions of the ARE? Theorem 3: Let V C 2 n be an invariant subspace of H and
V = Im

X1 X2

, where X1 ; X2 2 C n n and X1 is invertible.

Then

1 (i) X := X2 X is a solution of the ARE. 1

(ii) (A RX) = (HjV ): (iii) The solution X is independent of the choice of the basis of V: That is, X is unique whenever the invariant subspace V is given.

SDRE-11

Proof: To prove (i): V is an invariant subspace of H

) 9 2 C n n s:t:

X1 A R X1 = X2 X2 Q AT A R 1 In = In X X1 (post-multiply X ) (1) ) 1 1 T 1 Q A X X A R Q AT

Pre-multiply Eq. (1) by ( X; In ) we have

( X; In ) In X = ATX XA + XRX Q = 0 :

) X is a solution of the ARE.

SDRE-12

To prove (ii): By the first n equations of (1) we have

1 A RX = X1 X 1

(2)

) (A RX) = () = (HjV ):

To prove (iii): Any basis spanning V can be represented as

X1 X2 M= X1 M X2 M

for some nonsingular matrix M.

1 ; ) X = (X2 M)(X1 M)1 = X2 X 1

which is independent of the matrix M.

In Corollary 4: If Im X is an invariant subspace of H, where X 2 C n n : Then (i) ATX + XA XRX + Q = 0 :

SDRE-13

(ii) H In = In (A RX):
X X

Proof: (i) follows from Theorem 3 by setting X1 = In & X2 = X; while (ii) follows from Eqs. (1) and (2).

SDRE-14

3 2 ; R= 0 0 Example 1: Let A = 2 1 0 1 Eigenvalues and eigenvectors:

& Q=

0 0 0 0

(H) = f 1 ; 1 ; 1 ; 1 g :

The eigenvector and generalized eigenvector associated with = 1 are v1 = (1; 2; 2; 2)T & v2 = ( 1; 1:5; 1; 0)T The eigenvector and generalized eigenvector associated with = 1 are v3 = (1; 1; 0; 0)T & v4 = (1; 1:5; 0; 0)T

All solutions of the ARE: X1 = f v1 ; v2 g span f v1 ; v2 g is H-invariant. Let X2

1 ) X = X2 X = 1

10 6 6 4

is a solution & (A RX) = f1; 1g

SDRE-15

span fv3 ; v4 g is H-invariant. Let

1 =0 ) X = X2 X 1

X1 X2

= f v3 ; v4 g

is a solution & (A RX) = f 1; 1g

2 2 2 2
X1 X2 = f v1 ; v3 g

span fv1 ; v3 g is H-invariant. Let

1 ) X = X2 X = 1

spanfv1 ; v4 g; spanfv2 ; v3 g; and spanfv2 ; v4 g are not Hinvariant. One can show that the matrices constructed from those vectors are not solutions of the ARE.

(A RX) = f 1 ; 1 g :

is a solution and

Note that, there is only one solution such that (A RX) C :

SDRE-16

Remark 1: When H has no imaginary eigenvalues, then there always exist two real matrices X1 ; X2 2 R n n such that X1 and (Hj ) C : Moreover, X1 can be V = Im V
X2

chosen to be nonsingular if there exists a X1 2 C n n, stated in Theorem 3, which is nonsingular.

Hint: If X1

its real and imaginary parts. (iii) Use the fact (x + i y; x i y) = (x; y) 1
i

2 C 2 n n & X1 is nonsingular, then X2 (i) Columns of X1 are either real or appear in X2 complex conjugate. 1 (ii) Construct a new real matrix from X by taking X2

and the last 2 2 matrix is nonsingular.

1 i

SDRE-17

Definition: Let H R 2 n 2n be the set containing the Hamiltonian matrices with the following properties: (i) Each H 2 H has no imaginary eigenvalues. (ii) The real matrix X1 of the n-dimensional stable
X

H-invariant subspace basis X1 is nonsingular. 2 We define the mapping Ric : H ! R n n ; Ric(H) = X; where X is the associated ARE solution. Note that, from (iii) of Theorem 3, the mapping Ric is well-defined. That is, 8H 2 H; 9!X s:t: Ric(H) = X:

SDRE-18

Theorem 5: Suppose that H 2 dom (Ric) & X = Ric(H): Then (i) X is real and symmetric; (ii) ATX + XA XRX + Q = 0 (iii) (A RX) C : Proof: From Remark 1, X can be chosen to be real. It remains to show that X is symmetric. From (ii) of Corollary 4
In ; where () 2 C : X T 0 In ; Pre-multiply In ; = I 0 X n T T ) In H In = In In X X X X H = In X

on both sides

SDRE-19

Note that, the LHS is symmetric because H is symmetric, so is the RHS. Thus,
T T In In = T In T In X X X X

or T(X XT) + (X XT) = 0

() C This is a Lyapunov function with

) X XT = 0 , the result then follows.

SDRE-20

Question: Suppose that H has no imaginary eigenvalues. Under what conditions is X1 stated in Remark 1 nonsingular? Theorem 6: Suppose that H has no imaginary eigenvalues and R 0 or R 0: Then H 2 dom (Ric) , (A; R) is stabilizable. Proof: " ) " This is trivial, because by Theorem 5 This implies that (A,R) is stabilizable.
H 2 dom (Ric) ) (A RX) C :

SDRE-21

"("

To show that H 2 dom (Ric) , by definition, we have to show that X1 is nonsingular, i.e., N (X1 ) = f 0 g : First, we note that H has n eigenvalues in C
)9 X1 X2 & with rank X1 X2 = X1 X2 X1 X2 = n & () C s:t:

(3)

SDRE-22

T Next, we show that X2 X1 is a symmetric matrix:

2 T T X1 X1 X1 X1 ) H = X2 X2 X2 X2 where = 0 In In 0 * H is symmetric ) LHS of (4) is symmetric

T T T T transpose ) (XT X X X ) = ( X X X X) 1 2 2 2 1 1 2 1 This is a Lyapunov function and () C

1 Pre-multiply (3) by X X

(4)

) RHS of (4) is symmetric. Expand RHS and its

This proves the claim.

T ) (XT X X X )=0 2 1 2 1

SDRE-23

Third, we show that N (X1 ) is -invariant: Suppose that x 2 N (X1 ) and pre-multiply (3) by[In ; 0] Pre-multiply (5) by xTXT and post-multiply by x 2
T T T T T is symmetric * XT X ) x X X x = x X2 X1 x = 0 1 1 2 2 T T RX x = x X2 X1 x ) xTXT 2 2

AX1 RX2 = X1

(5) (6) (7)

RX2 x = 0 ) RX2 x = 0 : (6) ) xTXT 2

( Since R is semidefinite)

Post-multiply (5) by x ) X1 x = 0 ) x 2 N (X1 ) This proves the claim.

SDRE-24

Finally, we show that X1 is nonsingular: Suppose, on the contrary, that X1 is singular This together with N (X1 ) is -invariant implies that jN( X 1) has an eigenvalue and associated eigenvectors. Pre-multiply (3) by [0 ; In ]
) QX1 ATX2 = X2
T

) N (X1 )6=f 0 g

i:e:; 9 2 C & x 2 N (X1 )nf 0 g s:t: x = x

(8)

This completes the proof.

Stabilizability of (A,R ) implies that X2 x = 0 X1 X1 loses rank, a contradiction. ) x=0)

X2 X2

(7)(8) ) x

Post-multiply (8) by x ) (AT + I)X2 x = 0 (* x 2 N (X1))

(A XT 2 . + I . . R) = 0 :
(* 2 C )

SDRE-25
1 T B and the matrix R0 > 0 : Show Homework 1: Let R = BR 0 that (A,R) is stabilizable if and only if (A,B) is stabilizable. Hint: By PBH test.

Corollary 7: Suppose that H has no imaginary eigenvalues 1 T B Then and R = BR 0 H 2 dom (Ric) , (A; R) is stabilizable. , (A; B) is stabilizable. Question: When does H has no imaginary eigenvalues?

SDRE-26

Theorem 8: Suppose that H has the form

H= A BB CTC AT
T

; B 2 R n p & C 2 R qn

(9)

(i) If (A,B) is stabilizable, then H has no imaginary eigenvalues if and only if (A,C) has no unobservable modes on the imaginary axis. (ii) H 2 dom (Ric) , (A,B) is stabilizable and (A,C) has no unobservable mode on the imaginary axis. (iii) H 2 dom (Ric) ) X = Ric(H) 0 : (iv) Suppose that H 2 dom (Ric) & X = Ric(H): Then N(X)={0} (i.e., X is nonsingular or X > 0) if and only if (A,C) has no stable unobservable mode.

SDRE-27

Proof: To prove (i): Suppose (A,C) has unobservable mode on imaginary axis
A j!In x = 0 : ) 9 j! & x6=0 s:t: C n )x ) (H j!I2 n ) x = (A j!I = 0: T 0 C Cx
")"

) H has imaginary eigenvalues.

"(" Let j!; x be an eigen-pair of H z (A j!In )x = BBTz x x = j! or ) H (A j!In )H z = CTCx z z

(10)

SDRE-28

) zH (A j!In )x 2 R

zH (A j!In )x = zH BBTz = jjBTzjj2 xH (A j!In )H z = xH CTCx = jjCxjj2

) jjBTzjj2 = zH (A j!In )x = xH (A j!In )H z = jjCxjj2

) BTz = 0 & Cx = 0 (A j!In )x = 0 (10)(11) ) (A j!In )H z = 0 . A j!In x = 0 (11)(12) ) zH (A j!In . . B) = 0 & C (Since (A,B) is stabilizable) )z=0 x ( * is an eigenvector) ) x6=0 z

(12)

& j! is an unobservable mode of (A,C).

SDRE-29

To prove (ii): Follows directly from Theorem 6, Corollary 7 and (i) of this theorem. To prove (iii): Note that ATX + XA XBBTX + CTC = 0

* (A BBTX) C R 1 ( ABBTX ) Tt T T ( ABBTX ) t )X= 0 e (XBB X + C C)e dt 0 ;

) (A BBTX)TX + X(A BBTX) + XBBTX + CTC = 0

(13)

because XBBTX + CTC 0 :

SDRE-30

To prove (iv): Suppose that (A,C) has a stable unobservable mode Pre-multiply the ARE by xH and post-multiply by x
) 2Re( )xH Xx xH (XBBTX)x = 0 (*Re( ) < 0) ) xH X x = 0 ) X is singular. (*xH Xx = jjX1 = 2 xjj2 = 0)
")"

) 9x6=0 s:t: Ax = x & Cx = 0 :

SDRE-31

To show that N (X) N (C) : (14) Suppose that x 2 N (X): Since X is a solution of the ARE

"("

) xT(ATX + XA XBBTX + CTC)x = 0 ) Cx = 0 ) x 2 N (C)

To show that N(X)={0}: Suppose N (X)6=f 0 g ) 9x6=0 s:t: Xx = 0 : Pre-multiply (13) by xT and post- multiply by x ) Cx = 0 Post-multiply Eq. (13) again by x ) XAx = 0 ) N(X) is A-invariant. Since N(X) and nontrivial and A-invariant
) 90 6=y 2 N (X) s:t: y = Ay = (A BBTX)y & Cy = 0

(by (14))

) is a stable unobservable mode, because 2 (A BBTX) C :

SDRE-32

N (X)6=f 0 g ) 9x6=0 s:t: Xx = 0 : Suppose that Since X is a solution of the ARE

"("

) xT(ATX + XA XBBTX + CTC)x = 0 ) Cx = 0

) N (X) N (C)

(14)

Pre-multiply (13) by xT and post-multiply by x ) Cx = 0 Post-multiply Eq. (13) again by x ) XAx = 0 ) N(X) is A-invariant. Since N(X) and nontrivial and A-invariant
) 90 6=x 2 N (X) s:t: x = Ax = (A BBTX)x & Cx = 0

(by (14))

) is a stable unobservable mode, because

(A BBTX) C & Re( ) < 0 :

SDRE-33

From Theorem 8, we have the next result: Corollary 9: Suppose that (A,B) is stabilizable and (A,C) is detectable (resp., observable). Then the Riccati equation has a unique positive semidefinite (resp., positive definite) solution. Moreover, the solution is stabilizing.
ATX + XA XBBTX + CTC = 0 :

SDRE-34

Remark 2: Observability of (A,C) is not necessary for the existence of a positive definite stabilizing solution. For instance, let A =
1 0 0 2 ; B= 1 & C = (0 ; 0) 1

Then (A,B) is stabilizable, but (A,C) is NOT observable. However, X =

18 24 24 36 > 0 is the stabilizing solution.

SDRE-35

MATLAB code for ARE: Step 1: Construct the associated Hamiltonian matrix H. Step 2: Do Schur factorization for H:
H = UTHU = 11 0 12 22

where
U=

1 Step 3: X = U21 U 11

U11 U21

U12 U22

; (11 ) C & (22 ) C +

SDRE-36

The SDRE Design

The SDRE strategy is nothing but pointwisely apply the LQR scheme at every nonzero state. A excellent survey paper: T. Cimen, ``State-dependent Riccati equation (SRDE) control: a survey, Proceedings of the 17th IFAC World Congress, pp. 3671-3775, 2008.

SDRE-37

The capabilities of the SDRE design: The capability to directly specify and affect performance through the selection of the state-dependent state and control weighting matrices Q(x) and R(x), respectively. The capability to impose the hard bounds on the control of the control rate. The capability to satisfy state constraints and combined state and control constraints. The capability to directly handle unstable, non-minimum phase systems. The capability to preserve beneficial nonlinearities.

SDRE-38

The capability to utilize the extra design degrees of freedom that are available in the non-uniqueness of the state-dependent coefficient matrix to enhance the performance of the systems. (Please refer to J.R. Cloutier and D.T. Stansbery, ``The capabilities and art of state-dependent Riccati equationbased design, Proceedings of the American Control Conference, Anchorage, USA, pp. 86-90, 2002.)

SDRE-39

Consider a class of nonlinear control system and a quadratic performance index:

x = f (x) + B(x)u R1 and J = 0 [xQ (x)x + uR(x)u]dt

where x 2 R n & u 2 R p denote the system state and the n n p f ( x ) 2 R ; B ( x ) 2 R ; f (0 ) = 0 ; control input, respectively,
Q T(x) = Q (x) 0 & RT(x) = R(x) > 0 :

Note that, the weighting matrices Q(x) and R(x) are in general state-dependent.

SDRE-40

Procedure of the SDRE scheme:

Factorize f(x) into the SDC matrix representation form as f(x)=A(x)x, where A(x) 2 R nn: Symbolically check stabilizability of (A(x),B(x)) and observability (resp., detectability) of (A(x),C(x)) to ensure the existence of a unique positive definite (resp., positive semi-definite) solution of the following SDRE:
AT(x)P (x) + P (x)A(x) P (x)B(x)R1 (x)BT(x)P (x) + Q (x) = 0 ;
C(x) 2 R q n

has full rank and satisfies Q (x) = CT(x)C(x):

Solve for P(x) to produce the SDRE controller

u = R1 (x) BT(x)P (x)x: