Automatica, Vol. 29, No. 1, pp.

225-228, 1993

0005-1098/93$6.00+ 0.00
~) 1992PergamonPress Ltd

Printedin GreatBritain.

Brief Paper

Control for Discrete-time Systems

Optimality and Robustness*t
P. L. D. PERES* and J. C. GEROMEL*
Key Words--Discrete-time systems; ~2-optimal control; convex programming; linear optimal control;
robustness; robust control.

incorporated in the algorithm. To support this claim, we

solve completely the discrete-time linear system stabilizability problem subject to actuators failure. In this case,
since only one feedback gain" must be stabilizing for all
prespecified contingencies, we provide an upper bound for
each closed-loop transfer function ~z norm.

Airliner--This paper proposes a new approach to determine

optimal control for discrete-time linear systems, based on
convex programming. It is shown that all stabilizing state
feedback control gains belong to a certain convex set,
well-defined in a special parameter space. The Linear
Quadratic Problem can be then formulated as the
minimization of a linear objective over a convex set. The
optimal solution of this convex problem furnishes, under
certain conditions, the same feedback control gain which is
obtained from the classical discrete-time Riccati equation
solution. Furthermore, the method proposed can also handle
additional constraints, for instance, the ones needed to
assure asymptotical stability of discrete-time systems under
actuators failure. Some examples illustrate the theory.

2. Preliminaries

Let us consider a discrete-time linear system whose

dynamic behavior is given by the following difference
equations
x~+ 1 = A x k + B l w k + B 2 u k,
u~ = - K x k ,

(1)

z k = Cxt + DUk,

1. Introduction

THE THEORY OF discrete-time linear systems has been

developed, historically, as an extension of previous results
concerning continuous-time systems. In the 1960s, the Linear
Quadratic Problem--LQP has been exhaustively studied for
both continuous-time and discrete-time systems, becoming
a well-known technique for control design (Anderson and
Moore, 1971). The discrete-time optimal state feedback
solution can be calculated from the solution of an associated
Riccati equation. It exhibits many good robust properties as
for the continuous-time case. However, the control gain
depends directly on the matrices model. This fact is one of
the main reasons for the difficulties to generalize the discrete
LQP to take into account additional requirements as, for
instance, parameter uncertainties, output feedback, decentralized control or sensors/actuators failure. Other
difficulties stem from the fact that neither the set of
stabilizing controllers nor the objective function are convex.
This paper presents a convex approach to solve the
discrete-time LQP. It is formulated as a ~2 optimal state
feedback control problem. First, we define a parameter space
into which a convex set generates all the stabilizing state
feedback control gains. An optimization problem is then
formulated whose solution furnishes the state feedback gain
that minimizes the ~2 norm of a closed-loop transfer
function. Under certain conditions (to be defined in the
sequel) this control gain equals the one provided by the
classical LQP approach, that is, the associated Riccati
equation solution.
Furthermore, additional requirements can be easily

where Xk ~ " is the state variable, uk ~ " is the control

variable, Wk ~ t is the external disturbance and zk ~tq is
the output variable. Matrices A, B~, B 2, C and D have
appropriate dimensions and are supposed to be known.
Without loss of generality, the usual orthogonality hypothesis
is also made, that is C ' D = O and D ' D > O . Defining the
closed-loop matrices A a = A - B 2 K and Col = C - D K and
supposing that a state feedback gain K is calculated in such a
way that Act is asymptotically stable, the closed-loop transfer
function from w to z is given by
H ( z ) ~ C,[zl - A d I - 1 B , .

(2)

The ~2 norm for a stable transfer matrix H ( z ) can be

defined as
1 f*=
Ilnll 2 =~-~ J_~ Tr {H(e-J')'H(eJ')} dto,

(3)

and can also be calculated from the discrete-time associated

Gramians. Let Lc be the controllability Gramian of (Ad, Bt)
and Lo the observability Gramian of (Cot, Ac0. Then,
AdL~A' I - Lc + B i B ~ = 0,

(4)

A'ILoA d - L o + C'lCct = 0.

(5)

The ~2 norm is given by

IIHII2 = Tr (CdL~C~O = Tr ( B ~ L o B z ) .

(6)

If we denote by ~ the set of all stabilizing state feedback

control gains K e ~t"n, the problem of ~z optimal control
can be stated as follows:

*Received 17 July 1991; received in final form 18

December 1991. This paper was not presented at any IFAC
meeting. This paper was recommended for publication in
revised form by Guest Editor P. Dorato.
i" This research has been supported in part by grants from
"Fundaq,~o de Amparo [1 Pesquisa do Estado de S~o
PauI(>--FAPESP" and "Conselho National de Desenvolvimento Cientlfico e Tecnol6gico----CNPq", Brazil.
:~LAC-DT/Faculty of Electrical Engineering, UNICAMP,
CP 6101, 13081, Campinas, SP, Brazil.

(P1)

min (IIHII~:K ~}.

(7)

Examined in the state feedback parameter space, this

problem may have a very complicated geometry. In fact,
neither the objective function nor ~r are convex with respect
to the elements of the control gain K. However, it is
well-known (Anderson and Moore, 1971; Dorato and Levis,
1971) that the optimal solution of (P1) is given by
K = (B~PB2 + D'D)-tB~PA,

225

(8)

226

Brief Paper

where, under controllability and observability assumptions,

P ~ " " is the unique symmetric positive definite solution of
the discrete-time Riccati equation

K ~m, as well as a symmetric positive definite matrix

Lce ~ , x , , Lc > 0, satisfying
(18)

ActLcA'ct- L,: = - B t B ~.

(9)

A'PA-P-A'PB2(B~PBz+D'D)-~B~PA+C'C=O.

From (8), we note that the optimal feedback gain K depends

directly on the system matrices A and B 2. Obviously
(Kwakernaak and Sivan, 1972) this solution can also be
regarded as the optimal one for the following discrete-time
LQP:
min ~ (x't,C'Cxk + u't,D'Ouk),
"k k=0

(P2)

As a consequence, there also exists W -> Lc such that

Keeping in mind that Act = A - B2K and developing (19), it

gives, for any x ,~"
x'[AWA' - W - B2KWA' - AWK'B6
+ B2KWK'B~ + BtB~]x <-0,

(10)

It is important to remark that, although the optimal control

gain does not depend on the initial condition x 0, the minimal
value of the quadratic criterion is such that
(11)

J* = x;,Pxo.

(20)

v ' = I x ' ! 0].

x 0 given.

xk+ 1 = A x k + B2Uk,

(19)

A d W A " I - W + B I B ~~ O.

If we rearrange equation (9) in a closed-loop form, keeping

in mind that C ' D = O, we have

Since Vv--/=O~V:G'v = 0 , v = Ix' ~0], it is easy to verify

from (20) that the ~" matrix
~/.

W
=[KW

WK'
K W K ' ] >'0'

(21)

belongs to %, proving thus the necessity.

Now, the sufficiency. From the fact that every v ~
O ~ P : G ' v = O has the form v ' = [ x ' ~ O ] , with x 4 : 0 ~ " ,
taking W' % partitioned as in (17), we have

(A - B 2 K ) ' P ( A - B.,K) - P + (C - O K ) ' ( C - D K ) = O,

v'[F#/'F ' - ~ ' +

(12)
and we conclude from (5) that P = L o and IIHIbZ=J *
provided we choose B ~ = x o. Note that the necessary
optimality conditions for (P1) are given by equations (4)-(5)
and
[(B~LoBz + D ' D ) K - B~LoA]L~ = 0,

3. Main results

= x ' [ ( a - B 2 W ; W ? ' ) W , ( A - B..,W~W; ')'

- W I + B1B~]x

(22)

+ x'[Bz(W3 - W~W;'Wz)B~]x.

Since ~V" %, we have

v'[F'WF' - 'W + Q]v<-O

Vv .~P q=O:G'v = 0

(23)

~V'.>-O~ W~ >" W ~ W ~ l W z

Consequently, from (22) and (23), we conclude that

x'[(A - B z W ~ W ~ I ) W I ( A - B2W~W~I) ' - W, + B,B;]x <-0.

In this section, we propose a new convex programming

problem which is equivalent to (P1) in a sense to be defined
in the sequel. From the above discussion, this is necessary in
order to circumvent the non convexity of 9g. First of all, let
us.introduce the following extended matrices FE~.~ pxp,
p ~ m + n and G ~ p x m

o:[0,]

.4)

as well as the symmetric matrices Q ~pp, R ~'~PP

o0o]

.,,

and the set ~ :

% --~ {'W= W" ~-O:v'[F~/F' - ' W + Q]v <-0,
VvS,~tP--/=O:G'v=O},

(16)

where ~ / ~ ~vp is symmetric and is partitioned as

w~

~'=[wW'~ W3],

(24)
The

last

inequality evidences that the control gain

K = W~W~ 1 is a stabilizing one for the pair (A, B2). The
proof of part (b) then follows. Part (c) is obvious, since qg2 is
a convex set (Luenberger, 1973).

At this point, it is important to notice, from the previous

theorem, that there exists a one-to-one relationship between
the elements of ~ (the set of all stabilizing control gains) and
the ones of the convex set qg2. Indeed, for each ~" %z,
K = W~W~ ~ Y{ and, conversely, for each K ~, there
exists a matrix ~v" given by (21) which belongs to qg2 (see
Fig. 1).
Now, with the results of Theorem 1, it is possible to handle
the convex set q~2 in order to achieve stabilization, avoiding
the use of set Y/. The next theorem guarantees the optimal
characteristics to a particular W" qg2, which furnishes the
desired optimal control gain.
Theorem 2. The optimal solution of (P1) is obtained by
solving the following convex problem:

(17)

with W~ e ~lt"" being positive definite.

Theorem 1. The following statements hold:
(a) ~ is a convex set.
(b) The pair (.4, Bz) is stabilizable by a linear state feedback
if and only if ~ q : ~ . In the affirmative case, V~t/" qg2, a
stabilizing control gain is given by K = W ~ W ~ 1.
(c) For a 'lip ~ % , there exists an hyperplane that separates
W'* from %.

Part (a) follows directly from the fact that ~ is

defined as the intersection of an uncountable number of
linear constraints. Let us prove part (b). Suppose the pair
(,4, Bz) is stabilizable; then there exists a control gain
Proof.

- A W z B ~ + B2W~B ~ + BIB;]x

(13)

being thus related to the solution of (P2). Indeed, from (13),

if L~ is non singular then the unique solution is given by (8),
which is the optimal solution for (P2) and does not depend
on the initial condition x o. This fact will always occur in case
(A~,B~) is observable or rank ( B 0 = n
(a sufficient
condition). This is assumed throughout this paper.

Q]v

= x'[AW, A ' - W, - B2W~A'

(a3)

min {Tr (R1') : ~v" qg2}.

(25)

Being ~ * its optimal solution, then K* = W ~ W ~ 1 solves

(P1) (and consequently solves also (P2)) and Tr (RAN'*) = J*.
Proof. From Theorem

Non convex

1, K* = W ~ W { l ~ ,

w=w/( I
o')= t KW KWX" ]
r

being so a

Convex

FIG. 1. Relationship between sets Y{ and %.

Brief Paper

227

stabilizing control gain. We just have to prove that this state

feedback is indeed the optimal one. First, let us prove that
for K = W ~ W ~ 1, VOW%, we have I[HII~<Tr(ROW). To
this end, note that for any OWbelonging to qg2, Wl is positive
definite and

other situation can be taken into account by simple including

the case in ~2 and defining properly the parameter M (see
the example in the next section).

(26)

given by (16) with F replaced by F// (defined as in (14)),

i = 1 M. The optimal solution /Cryof

(A - B2K)WI(A - B 2 K )' - W 1 + B I B~ <- O,

then we conclude from (4) that W1->L. Now, using

equation (6), the fact that OW->0 implies W3 -> W ~ W { tW2 and
the orthogonality condition C ' D = 0, we get
IIHI[2 = T r ((C - DK)L(C - Dr)'}

DK)W~(C - DK)'}
( C W ~ C ' + DW:~W(IW2 D'}

<--Tr ((C -Tr

Tr (ROW), VOW qg2.

(27)

The above inequality is a very important result; actually it

holds in particular for the optimal solution of (P1), implying
that J*~Tr(ROW), VOW% and, as can be verified, the
equality holds for

Theorem 3. Define the convex set cg2/= i=('~1qg~, where q~2iis

(P4)

L c K ' ] <g2,
KLeK' J

Proof. Since ~ belongs to %f, it also belongs to each one

of the sets %~, i = l - . . M ;
then, we have Vve~Rp#:
O:G'v =0,
v'[FiowfF:-OW/ + Q]v <- 0,

min {Tr (ROW): OW cg2}= Tr (ROW*)

= Tr (CdLC'O = J*.

(29)

The proof of Theorem 2 is completed.

The above results deserve some remarks. The first one is
related to the uniqueness of solution of (P3). Being convex
(not strictly), it would be possible that its optimal solution
was not unique in the sense that different state feedback
gains could be generated. Fortunately, this fact does not
occur. To prove that, suppose OW*4:OW c2 generate two
different state feedback gains, being such that Tr {R(OWOW*)} = 0. With K associated to OW, determining the transfer
function H(z), using (27) and taking into account that (P1)
admits only one solution, we obtain
J* = min { IIHI[2: K e Y(},

< IIHII,:- Tr (ROW),

(32)

implying (see the proof of Theorem 1), with ~ partitioned

as in (17), that
-

I
--1
B2iW~W
I )WI(A

I
--1 t
B2iW~W,
)

W,

B,B~ <-0,

Vi=l.-.M.

(33)

It is then clear that KI = W ' ~ W ~ guarantees asymptotical

stability for all B 2 f~2, taking into account all feasible
actuator's failure. On the other hand, since W1 is unique for
all the above i = 1 M inequalities, using (27) we get
Tr (ROW:) _A_#/_> IIHII~, VB~,
consequently,
completed.

#:->llHll2z, V B 2 ~ z

i = 1 . . . M,
and

(34)

the proof is
[]

4. Numerical procedure and examples

This section is in part devoted to solve the convex problem
(P4), that is

rain (Tr (ROW): OW %/).

(35)

Note that this problem is convex and reduces to (P3) in case

M = 1. Using the previous results, for each OWe %p it is a
simple task (Geromel et al., 1991) to determine a matrix
~(OWe) and a scalar fl(OWe), which define a separating
hyperplane from OWeto % r Then, it is possible to apply the
following outer linearization algorithm which converges to
the global optimal solution (Bernussou et al., 1989;
Luenberger, 1973):

(30)

which is an impossibility, since by assumption J * =

Tr (ROW*). The second remark concerns the geometry of
(P3). Indeed, we have proved that the discrete-time LQP is
equivalent to a convex problem defined on a special
parameter space (the elements of matrix OW~ %). This fact is
of great importance since (P3) can be solved by means of
efficient numerical procedures available in the literature (see
for instance a convex-based algorithm proposed in Geromel
et al., 1991).
Another important characteristic of (P3) is that further
convex constraints can be easily added to it. For instance,
suppose we want to solve the problem of finding a control
gain that guarantees closed-loop stability and minimal
suboptimality in case of actuator failure. This problem can
be stated as follows: suppose the model (1) is such that the
input matrix B 2 is not exactly known but belongs to the set
~2 ~ {B2~: i = 1 M}. Find (if one exists) a state feedback
matrix gain K/ and a positive parameter #: (as small as
possible) such that:
A - B2K: is asymptotically stable VB 2 E ~2.
: IIHII~<#:, WB2E~ 2.
Note that, in the above problem formulation, we are
representing a particular actuator failure (the ith component
of the control variable u is reduced to zero) by the equality
B2u = B n u . In this sense, the set ~2 is composed by all
matrices obtained from B 2 by zeroing its ith column,
i = 1 m. In fact, this is the case when only one actuator is
supposed to fail at a time. However, it is obvious that any
AUTO 29:1-P

Vi = 1 . - . M,

(28)

where K = ( B ~ L o B 2 + D ' D ) - t B ~ L o A ,
being P = L o the
definite positive solution of the discrete-time Riccati
equation (9) and Lc being the solution of the controllability
Gramian (eguation (4)). Now, it is simple to see that
K* = W ~ W ~ ~ provides the optimal gain and

(31)

is such that K / = W ~ W ~ l and #f=Tr(ROW:) solve the

actuator's failure problem stated before.

(A
OW*= [ Lc
KL~

min {Tr (ROW): OWe cg2r),

OWe+I= arg min (Tr (ROW): OW cg~+l},

(36)

cg2t:+l= cg2trN (OW:fl(OWe)+ (ol(owe), OW) -< - e } ,

(37)

:go

given,

where e > O is a sufficiently small parameter used to

approximate q ~ by a closed convex set and ~2:'~ ~ [ .
To illustrate the proposed method, let us consmer the
following discrete-time system:
A=

-0.2113 0.0087 0.4524]

0.0824 0.8096 0.8075|,
0.7599 0.8474 0.4832_1

C=

1 ,

B2=

D=

[0.6135
0.2749
0.8807

0.6538]
0.4899[,
0.7741 ]

0,

0
0
and B I = I . Since the eigenvalues of matrix A are
(0.3827, -0.4919, 1.6133), it is open-loop unstable. Using the
discrete-time Riccati equation, we get
KRiccati=[0.2968 0.3758
L0.2302 0.4953

0.3114]
0.4997J'

and IIH112~=5.2448. With the proposed algorithm, the

228

Brief Paper

optimal solution of (P3) is calculated to be

F

LQR w i t h folLure a t k : 6

2I

1.4582

-0.3930

-0.0926

0.2597

0.0918q

1-0.3930

1.6316

-0.1254

0.4629

0.6612 /

o.e

~t~'*= / - 0 . 0 9 2 6

-0.1254

1.1972

0.3013

0.51511,

0.6

1.0

0.2597

0.4629

0.3013

0.3489

0.4411 /

0.4

0.0918

0.6612

0.5151

0.4411

0.60841

0.2

0.2749 0.48991,
0.8807 0.7741 /
B23 =

B22 =

0.4899 | .
0.7741_1

Kr=L0.4823

s'J"

, . . . . ~:: : : : : = % - ' ' " '

-o.s

IO

20

30

40

50

60

/r

FIG. 2. Impulse response----Kruc~ti.

R o b u s t c o n t r o l w i t h faiLure o t k : 6
i.o
o.6

"!

o.6

0.4 ii
0 :

"~

:'

-0.2
i(~
-0,4

Considering the set ~2 and solving (P4), we have found

/~I = 11.8127 and the control gain
[0.2966

.'"'""

"

I
/"

-0,4

L !1
0.2749
0.8807

-0.2

0.3803 0.3148]
0.4989 0.5002 J'

and IIHll 2 = 5.2452. Comparing both numerical solutions, we

verify relative errors of about 0.8% in the control gain norm
and about 0.007% in the ~g2 norm.
Now, let us suppose that one of the actuators may fail.
Obviously, the optimal solution of the discrete-time LQP
provides no guarantee for stability in this case. However,
Theorem 3 can easily handle this possibility by defining
----&-(B21 , B22 , B23 } where
B21 =

.,'
..'

]i?.
:'...~,~.
"'.

providing the control gain

K.=[0.3006
0.2292

.'
.-

-0.6
-0.8

-i.o

0.6356 0.7405]
0.6592 0.4925J'

IO

20

30

40

50

60

that guarantees the closed-loop asymptotical stability under

the actuators failure previously defined. Furthermore, for
each matrix B2~, i = 1, 2, 3, the associated ~2 norm are given
by

FIG. 3. Impulse response--Kf.

B21 : IIHII2 = 7.0560,

B22 : IIHll 2 = 8.6285,
B23 : IIHII2 = 8.4779,
making evident that /~r is a ~ - n o r m upper bound indeed.
Table 1 shows the closed-loop eigenvahies in two situations,
namely B2 = B21 (the nominal one) and B2 = B22 (actuator-2
failure). It is easy to see that under the last contingency the
closed-loop system with gRieeati becomes unstable, and the
same does not occur for the closed-loop system with K;. In
Figs 2 and 3, we show the impulse response (and unitary
impulse has been applied to the second component of w) of
the closed-loop system with KRieeat i and Kr, respectively,
supposing that actuator 2 (corresponding to the second
component of u) fails at k = 6 and remains inactive for k > 6.
The unstable behavior is obvious when the Riccati control
gain is used.

5. Conclusion
In this paper we have proved that the optimal state
feedback solution of the discrete-time LQP can be
determined by means of a convex problem. This is an
important result, mainly due to two facts. First, for a given
TABLE 1. CLOSED-LOOP EIGENVALUES
KRiccat i

gf

Both actuators

-0.4826
0.2085
0.4385

0.4153
-0.5326
-0.4069

Only first actuator

-0.4861
0.3347
1.0959

0.4047
0.5973
-0.5068

Contingencies

pair (,4, B2), all stabilizing state feedback gains can be

parametrized over a convex set. Second, the convexity of the
YCz discrete-time control problem allows to solve it by using
the most powerful mathematical programming methods. As a
by-product of this fact, we claim that additional structural
constraints can be easily handled. For instance, we defined
and solved a problem involving actuator's failure whose
solution, to our knowledge, was not available until this time
in the literature. In this sense we want to emphasize that the
numerical procedure proposed here is specially addressed for
solving "non-classical" ~
optimal control problems,
including additional convex constraints (actuators/sensors
failure, robustness, uncertain systems control . . . . ) which
cannot be solved by means of the algebraic Riccati equation.

References
Anderson, B. D. O. and J. B. Moore (1971). Linear Optimal
Control. Prentice Hall, Englewood Cliffs, NJ.
Bernussou, J., P. L. D. Peres and J. C. Geromel (1989). A
linear programming oriented procedure for quadratic
stabilization of uncertain systems. Systems and Control
Letters, 13, 65-72.
Dorato, P. and A. H. Levis (1971). Optimal linear
regulators: the discrete time case. IEEE Trans. on Aut.
Control, 16, 613-620.
Geromel, J. C., P. L. D. Peres and J. Bernussou (1991). On
a convex parameter space method for linear control design
of uncertain systems. SIAM J. on Control and Optimiz.,
29, 381-402.
Kwakernaak, H. and R. Sivan (1972). Linear Optimal
Control Systems. John Wiley, New York.
Lucnberger, D. G. (1973). Introduction to Linear Programruing. Addison-Wesley, Reading, MA.