Linear Quadratic Gaussian Control and Model

Predictive Control (LQG and MPC)

Sachin C. Patwardhan
Dept. of Chemical Engineering,
I. I. T. Bombay, Powai, Mumbai 400 076
Email: sachinp@iitb.ac.in

Dynamic Model for Controller Synthesis

Let us consider a stochastic process described by the following linear discrete state space
model
x(k + 1) =

x(k) + u(k)+w(k)

(1)
(2)

y(k) = Cx(k)+v(k)

where x 2 Rn represents state variables, u 2 Rm represents manipulated inputs, y 2 Rr

represents measured outputs and w 2 Rn and v 2 Rr represent unmeasured disturbances
(state noise) and measurement noises, respectively. Here the vectors w(k) and v(k) are
assumed to be zero mean white noise sequences such that
R1 = E w(k)w(k)T

(3)

R12 = E w(k)v(k)T

(4)

R2 = E v(k)v(k)T

(5)

Such a model can be derived using either from linearization of a of rst principles (or grey
box model or state realization of a time series model developed from input-output data.

1.1

Linearization of First Principles / Grey Box Model

Linear discrete perturbation model of the form
x(k + 1) =

x(k) + u(k)+

y(k) = Cx(k)+v(k)
1

d d(k)

(6)
(7)

can be developed in the neighborhood of an operating point starting from a nonlinear rst
principles (or grey box) model. Here d 2 Rd represents vector of (physical) unmeasured
such as uctuations in feed concentrations. If it is further assumed that d(k) with known
covariance matrix, say Q; then we have
w(k) =
E [w(k)] = 0 ;

(8)

d d(k)

Cov [w(k)] = R1 =

dQ

T
d

(9)

When a state space model derived from rst principles is used for inferential control, the set
of controlled outputs will dier from the set of measured output. We represent controlled
outputs yr 2 Rr as follows
yr (k) = Cr x(k)
Example: Consider a CSTR in which the state variable vector is perturbations in reactor
concentration and reactor temperature
h

CA

Cr =

x=

iT

If reactor temperature is measured and it is desired to control reactor concentration, then

h
i
C= 0 1

and

1.2

1 0

State Realization of Time Series Model

The innovations form of state space model of the form

x(k + 1) =

x(k) + u(k) + L1 e(k)

y(k) = Cx(k) + e(k)

(10)
(11)

can be obtained from time series models (ARX/ARMAX) models identied from inputoutput data. Here, the innovations (or residuals) fe(k)g are a zero mean Gaussian white
noise sequence with covariance matrix Ve and K represents the corresponding steady state
Kalman gain. The above model can be re-written as
x(k + 1) =

x(k) + u(k) + w(k)

y(k) = Cx(k) + v(k)

(12)
(13)

where the vectors w(k) 2 Rn and v(k) 2 Rr are zero mean white noise sequences such that
R1 = E w(k)w(k)T = L1 VeT LT1

R12 = E w(k)v(k)T = L1 Ve
R2 = E v(k)v(k)T = Ve

(14)
(15)
(16)

It may be noted that, when a state space model is identied from data, typically we have
Cr = C, i.e., the set of outputs that can be controlled is identical to the set of measured
outputs.

Quadratic Optimal Control

The above model is used as a basis for design of a state feedback controller. Steps involved
in design of any state feed-back controller are as follows
1. Solve regulator / controller design problem under the assumption that full state is
available for feedback
2. Design state estimator and implement control law using estimated states.
In this section, we rst describe the method for designing optimal state feedback control
law and later show how control law can be implemented using optimal state observer (Kalman
Filter).

2.1

Linear Quadratic Optimal Regulator Design

We rst discuss the state regulator design problem, where it is desired to bring the system
form non-zero initial state to zero initial state (the origin of state space). Non-zero initial
state can result from impulse like disturbances, which are su ciently spaced in time. We
want to device a state feedback regulator of type
u(k) =

Gx(k)

which will bring the system to the origin in a optimal fashion. For designing the regulator,
we only consider the deterministic part of the model, i.e.
x(k + 1) =

x(k) + u(k)

y(k) = Cx(k)

(17)
(18)

The regulator design problem is posed as an optimization problem where it is desired to

determine control sequence u(0); u(1):::::u(N 1) such that objective function
(N 1
)
X
J =E
x(k)T Wx x(k) + u(k)T Wu u(k) + x(N )T WN x(N )
(19)
k=0

is minimized. Here E(:) represents the expectation and x(N ) represents the state at nal
time N T . Wx ; Wu and WN are symmetric positive denite matrices. This optimization
problem is solved using the Bellmans method of dynamic programming. In order to derive
the control law, we start optimization from time k = N and work backwards in time. Let
us dene S(N ) = WN and
(N 1
)
X
J(k) =
min
E
x(i)T Wx x(i) + u(i)T Wu u(i) + x(N )T WN x(N )
(20)
u(k):::::u(N 1)

i=k

For k = N;
J(N ) = x(N )T WN x(N )
Then, for k = N

J(N

(21)

1;

1) = min

u(N 1)

x(N

1)T Wx x(N

1) + u(N

1)T Wu u(N

1) + J(N )

(22)

But

J(N ) = x(N )T WN x(N )

= [ x(N

1)]T S(N ) [ x(N

1) + u(N

1) + u(N

1)]

(23)

Thus,
(

x(N 1) Wx + S(N ) x(N 1) + x(N 1)

S(N ) u(N 1)
T
T
T
T
u(N 1)
+u(N 1)
S(N ) x(N 1) + u(N 1)
S(N ) + Wu u(N 1)
(24)
T
T
Note that the rst term x(N 1) Wx + S(N ) x(N 1) in J(N 1)cannot be inuenced by u(N 1). We solve the problem of minimizing the last three terms in J(N 1)
by method of competing squares. In order to see how this can be achieved, consider a scalar
quadratic function
J(N 1) = min

F (u) = uT Au + zT u + uT z
= uT Au + zT u + uT z + zT A 1 z
=

u + A 1z

A u + A 1z
4

(25)
zT A 1 z
zT A 1 z

(26)
(27)

where A is a positive denite matrix. The rst term on the right hand side in the above
equation is always non-negative. This implies that minimum of F (c)with respect to u is
attained at
u = A 1z
(28)
and its minimum value is
Fmin (u) =

zT A 1 z

(29)

In the minimization problem at hand, with some rearrangement, we have

T

(30)

S(N ) + Wu

S(N ) x(N

1)

(31)

1)

(32)

and the optimal solution can be expressed as

1) =

u(N

1) =

G(N

1)x(N

G(N
Wu +

S(N )

S(N )

(33)

1)

(34)

which gives minimum loss function,

J(N

1)T S(N

1) = x(N

1)x(N

where,
S(N

1) =

S(N ) + Wx

G(N

1)T Wu +

S(N )

G(N

(35)

1)

Similar arguments give,

J(N

2) = min

u(N 2)

x(N

2)T Wx x(N

2)T Wu u(N

2) + u(N

2) + J(N

1)

(36)

This is same as the earlier optimization problem, but with the time argument shifted. The
procedure can be repeated backward in time. The equation,
S(k) = [

G(k)]T S(k + 1) [

G(k)] + Wx + G(k)T Wu G(k)

(37)

is called the discrete time Riccati equation. The matrices S(N ) = WN and Wu are assumed
to be positive denite and symmetric. This implies that S(k) is positive denite/semi-denite
and symmetric and this condition guarantees optimality at each stage. When horizon N
becomes large, S(k) tends to a constant matrix S(k) ! S1 which can be computed by
solving the algebraic Riccati equation (ARE)
G 1 = Wu +

S1

S1

(38)

S1 = [

G1 ]T S1 [

G1 ] + Wx + GT1 Wu G1

(39)

This ARE has several solutions. However, if ( ; ) is reachable and if ( ; ) is observable

pair, where Wu = T , then there exists a unique, symmetric, non-negative denite solution
to the ARE. The corresponding state feedback control law can be formulated as
u(k) =

(40)

G1 x(k)

Further, when ( ; ) is controllable and objective function is symmetric and positive definite, the LQ controller will always give asymptotically stable closed loop behavior. By
selecting Wx and Wu appropriately, it is easy to compromise between speed of recovery and
magnitude of control signals.

2.2

Stability of LQ controller

Theorem 1 (Stability of the closed loop system) : Consider the time invariant system
given by equation (1) the loss function for the optimal control is given by equation (19).
Assume that a positive-denite steady state solution S1 exists for Riccati equation (39).
Then the steady state optimal strategy
u(k) =

G1 x(k) =

Wu +

S1

S1 x(k)

gives an asymptotically stable closed-loop system

x(k + 1) = (

G1 )x(k)

Proof: Theorem A.3 in Appendix can be used to show that the closed loop system is
asymptotically stable (see Appendix for denitions of stability). It is su cient to show that
the function
V (x(k)) = xT (k)S1 x(k)
is a Lyapunov function. V is positive denite and

4V (x(k)) = xT (k + 1)S1 x(k + 1)

= xT (k)(
=

xT (k)S1 x(k)

G1 )T S1 (

G1 )x(k)

xT (k)S1 x(k)

xT (k)[Wx + LT Wu L]x(k)

Because Wx + GT1 Wu G1 is positive denite, 4V is negative denite. The closed loop

system is thus asymptotically stable.
6

The poles of the closed loop system can be obtained in several ways. When the design is
completed, the poles are obtained from
det( I

+ G1 ) = 0

It is possible to show that the poles are the n stable eigenvalues of the generalized
eigenvalue problem.
("

I
Wx

0
T

"

Wu
I

#)

=0

This equation is called the Euler equation of the LQ problem. Theorem 1 shows that LQ
controller gives a stable closed loop system, i.e. all poles of
G1 are inside unit circle.

2.3
2.3.1

Linear Quadratic Gaussian Control

State Estimation

The model (1-2) can be used to develop the optimal state predictor as follows
e(k) = y(k)
b(k + 1jk) =
x

Cb
x(kjk

b(kjk
x

1) +

(41)

1)
u u(k)

+ Lp e(k)

(42)

where Lp is the solution of the steady state Riccati equation

Lp =
P1 =

P1 CT + R12
P1

CP1 CT + R2

Lp CP1 CT + R2 LTp

+ R1

(43)
(44)

Here, matrix P1 denotes steady state covariance of error in state estimation. It can be shown
that the residual (or innovation) fe(k)g is a zero mean Gaussian white noise process with
covariance matrix V1 . The state feedback LQ control law designed above is implemented
together with the Kalman predictor using estimated states as follows
u(k) =

b(kjk
G1 x

(45)

1)

Alternatively, model (1-2) can be used to develop the optimal current state estimator as
follows
e(k) = y(k)
b(kjk
x

1) =

Cb
x(kjk

b(k
x

b(kjk) = x
b(kjk
x

jk

1) +

u u(k

1) + Lc e(k)
7

(46)

1)
1)

(47)
(48)

where Lc is the solution of the steady state Riccati equation

P1;1 =

P0;1

P1;1 CT + R12

Lc =
P0;1 =

[I

(49)

+ R1
CP1;1 CT + R2

(50)
(51)

Lc C] P1;1

We can then use Kalman ler (current state estimator) to implement control law as follows
u(k) =
2.3.2

Separation Principle

b(kjk)
G1 x

(52)

To assess the nominal stability of the closed loop generated by observer - regulator pair, the
observer and the plant dynamics has be considered together. For example, the closed loop
system with Kalman predictor can be described by following set of equations
x(k + 1) =

(53)

x(k) + u(k) + w(k)

(54)

y(k) = Cx(k) + v(k)

b(k + 1jk) =
x

b(kjk
x

1) + u(k) + Lp [y(k)

"(kjk

1) = x(k)

Cb
x(kjk

1)]

(55)

By introducing state estimation error

b(kjk
x

(56)

1)

the above set of equations can be rearranged as

"
#
"
#"
x(k + 1)
[
G1 ]
G1
x(k)
=
"(k + 1jk)
[0]
[
Lp C]
"(kjk
" #
"
#
I
[0]
+
w(k) +
v(k)
I
Lp

1)

#
(57)

Thus, dynamics of LQG controller is determined by dynamics of LQ controller (i.e. eigen

values of [
G1 ]) and dynamics of the state observers (i.e. eigen values of [
Lp C] ).
Thus, designing state feedback controller and state observer to be individually stable ensures
stability of the closed loop (separation principle). As a consequence, if the system under
consideration is observable and controllable, then the resulting closed loop is guaranteed
to be stable if weighting matrices in LQ formulation are chosen to be positive denite /
semi-denite. It is straight forward to show that separation principle also holds in this case.

2.4
2.4.1

Linear Quadratic Optimal Controller

State Augmentation for Unmeasured Disturbance Modeling

Linear quadratic regulator designed above can generate oset if (a) the unmeasured disturbances are non-stationary, i.e. they have slowly drifting behavior (b) mismatch exists
between the plant and the model. In order to deal with such a situation and introduce
integral action in the controller in the face of plant-model mismatch, the state space model
(1-2) is augmented with extra articial states as follows
x(k + 1) =

x(k) +

u u(k)

(k) + w(k)

(58)

(k + 1) =

(k) + w (k)

(59)

(k + 1) =

(k) + w (k)

(60)

y(k) = Cx(k) + C

(61)

(k) + v(k)

where 2 Rs and 2 Rt are articially introduced input and output disturbance vectors
while vectors w 2 Rs and w 2 Rt are zero mean white noise sequences with covariances Q
and Q ; respectively. The model coe cient matrices ( , C ) and noise covariances matrices
(Q , Q ) are treated as tuning parameters, which can be chosen to achieve the desired closed
loop disturbance rejection characteristics. Note that the total number of extra states cannot
exceed the number of measurements due to the requirement that the additional states should
be observable. Typical choices of the articial state variables and the corresponding coupling
matrices are as follows
Output bias formulation: A simple approach is to view the drifting disturbances
as causing a bias in the measured outputs, i.e., we can choose
= [0] ; Q = [0] ; C = I ;

Q =

Input Bias Formulation: The elements of vector

can be viewed as bias in r manipulated inputs. When number of manipulated inputs equals the number of measured
outputs (r = m), then we can choose
=

;Q =

I ; C = [0] ;

Q = [0]

If number of manipulated inputs exceeds the number of measurements, then r linearly

independent columns of u can be selected as :
Disturbance bias formulation: When the state space model is derived from rst
principles, it is possible to choose
=

;Q =

provided number of disturbance variables (d) = r: If d > r; then r linearly independent

columns of d can be chosen as :
In all the above cases, 2 is treated as a tuning parameter. The above set of equations
can be combined into an augmented state space model of the form
xa (k + 1) =

a xa (k)

ua u(k)

(62)

+ wa (k)

(63)

y(k) = Ca xa (k) + v(k)

where

3
2
3
x(k)
w(k)
6
7
6
7
xa (k) = 4 (k) 5 ; wa (k) = 4 w (k) 5
k)
w k)
2
3
"
#
[0]
6
7
u
[0] 5 ;
a = 4 [0] I
ua =
0
[0] [0] I
h
i
Ca =
C [0] C
R1a

R12a

R1 [0]
6
T
= E wa (k)wa (k) = 4 [0] Q
[0] [0]
"
#
R
12
= E wa (k)v(k)T =
[0]

3
[0]
7
[0] 5
Q

R2a = E v(k)v(k)T = R2
This augmented model can be used for developing a Kalman predictor of the form
ea (k) = y(k)
ba (k + 1jk) =
x

Cb
xa (kjk

ba (kjk
ax

(64)

1)

1) +

ua u(k

1) + La ea (k)

(65)

where the steady state Kalman gain is obtained by solving the corresponding steady state
Riccati equations
La =
Pa1 =

T
a Pa1 Ca
a Pa1

T
a

+ R12a

+ R1

Ca Pa1 CT
a + R2

T
La Ca Pa1 CT
a + R2 La

(66)
(67)

In order to maintain the observability of the articially introduced states, the number of additional states introduced in the augmented model should not exceed the number of measured
outputs . When the state space model (10-11) is observable and stable with no integrating
modes, the augmented state space model will be observable (detectable) in most of the cases.
10

2.4.2

Quadratic Optimal Servo Control

The problem of tracking a setpoint is solved by modifying the regulatory control law as
follows
us (k) =

u(k)

G [x(k)

u(k) = us (k)

(68)

xs (k)]

G [x(k)

(69)

xs (k)]

where xs (k) represent the nal steady state target corresponding to the setpoint, say r(k);
and us (k) represents the steady state input necessary to reach this steady state target. At
any instant k;these vectors are computed by solving steady state equation
xs (k) =

xs (k) +

u us (k)

r(k) = Cr xs (k) + C

(70)

(k)

(71)

(k)

Eliminating xs (k) from above equations, we have

r(k)= Ku us (k) + K

(k) + C

(72)

(k)

where steady state gain matrices Ku and K are dened as follows

Ku = Cr (I

K = Cr (I

When number of inputs (m) equals the number of controlled outputs (r), we can re-arrange
above equations as follows
us (k) = Ku 1 [r
xs (k) = (I

K
)

(k)
u Ku

(k)]

[r(k)

(73)
(k)] +

u Ku

(k)

(74)

When number of the manipulated inputs (m) is not equal to number of controlled outputs
(r), matrix Ku 1 in the above expression should be replaced by Kyu ; i.e., pseudo-inverse of
the steady state gain matrix Ku : For the case m = r; two special cases of quadratic optimal
tracking control law are as follows
Output bias formulation: In this case we have
= [0] ; C = I; which implies
that K = [0] and computation for xs (k); us (k) reduces to
us (k) = Ku 1 [r(k)
xs (k) = (I

11

(75)

(k)]
u Ku

[r(k)

(k)]

(76)

Input Bias Formulation: In this case we have

= u ; C = [0]; which implies
that K = Ku and computation for xs (k); us (k) reduces to
us (k) = Ku 1 r(k)
xs (k) = (I

(77)

(k)
1

u Ku

(78)

r(k)

Model Predictive Control

LQG formulation described above provides a systematic approach to designing a control law
for linear multi-variable systems. However, main di culty associated with the classical LQG
formulation is inability to handle constraints explicitly. Operating constraints, such as limits
on manipulated inputs, limits on controlled outputs arising out of product quality or safety
considerations are commonly encountered in any real problem. Also, it is di cult to deal
with plant-model mismatch in classical LQG framework.
Model Predictive Control (MPC) refers to a class of multi-variable control algorithms
developed in process industry to deal with operating constraints together with multi-variable
interactions. MPC can be viewed as modied versions of LQ (or LQG) formulation, which
can deal with plant-model mismatch and operating constraints in a systematic manner. This
approach was rst proposed independently by two industrial groups
Dynamic Matrix Control (DMC): Proposed by Cutler and Ramaker from Shell, USA
in 1978.
Model Algorithmic Control (MAC): proposed by Richalet (1978, France)
These initial versions were based on nite impulse models (FIR) models. In this section,
we rst present DMC formulation starting from equations (17-18). We later proceed to
develop the recent version of MPC formulation based on Kalman lter, which is better
suited for dealing with unmeasured disturbances.

3.1

Model Based Prediction of Future Behavior

The main component of DMC is the model describing deterministic contributions to process
dynamics given by equations (17-18). This model is used on-line to perform future predictions
of plant dynamics. When the system under consideration is open loop stable, this model can
be used to develop an open loop state observer as follows
b(kjk
x

1) =

b(k
x

1jk
12

2) +

u u(k

1)

(79)

b(kjk 1) denotes estimate of state x(k) based on information up to time instant

where x
(k 1).
In a typical DMC formulation, at each sampling instant, the state estimator (79) is used
for predicting future behavior of the plant over a nite future time horizon of length p (called
as the prediction horizon) starting from the current time instant k: Let us assume that at
any instant k; we are free to choose only q future manipulated input moves
fu(kjk); u(k + 1jk)::::::u(k + q

1jk)g

with the following constraints on the remaining of the future input moves
u(k + qjk) = u(k + q + 1jk) = :::::: = u(k + p

1jk) = u(k + q

1jk)

(80)

where q is called as the control horizon. The predicted estimates of the state variables can
be generated by recursively using the state estimator (41-42) as follows
b(k + 1jk) =
x

b(k + j + 1jk) =
x

b(kjk
x

1) +

b(k + jjk) +
x

u u(kjk)
u u(k

+ jjk)

(81)
(82)

A predictive control formulation based on the above open loop observer predictions can
pose practical di culties in the presence of mismatch between plant and the model. Since
the model is identied once in the initial phase of the MPC implementation, discrepancies
between plant and model parameters can arise over the period of time due to shift in the
operating point, changes in the disturbance characteristics etc. Thus, there is a need to
introduce some mechanism to account for the plant model mismatch, which, in turn, introduces integral action in the controller formulation. In DMC formulation, this is achieved by
correcting the future predictions as follows
b(k + j + 1jk) = Cr x
b(k + jjk) + b (k + jjk)
y
b (k + j + 1jk) = b (k + jjk)
b (kjk) = y(k)

(for j = 1::::p)

Cb
x(kjk

1)

(83)
(84)
(85)
(86)

Note that vector b (k) contain information about unknown / unmeasured disturbances as
well as plant model mismatch. It is easy to see that this mismatch compensation strategy is
equivalent to the output bias formulation discussed in the previous section.

13

3.2

Future Setpoint Trajectory Generation

In addition to predicting the future output trajectory, at each instant, a ltered future
setpoint trajectory is generated using a reference system of the form
xr (k + j + 1jk) =

r xr (k

+ jjk) +

[r(k)

yr (k)]

yrf (k + j + 1jk) = y(k) + Crf xr (k + j + 1jk)

for j = 0; 1; ::::p

(87)
(88)

with initial condition xr (k) = 0. Here, r(k) 2 Rr represents the setpoint vector. The
coe cient matrices of the reference system are tuning parameters which can be selected
to achieve the desired closed loop tracking performance. In order to ensure the free servo
responses for step changes in the setpoint, the coe cient matrices of the reference system
should be selected such that the steady state gain of the reference system is equal to the
identity matrix, i.e.
1
Crf (I
r)
r = I
Typically, the reference system is selected such that its transfer function matrix is diagonal
with unit gain rst (or higher) order low pass lters on the main diagonal.

3.3

Base Dynamic Matrix Control (DMC) Formulation

Given future desired setpoint trajectory fyrf (k + jjk) : j = 1; 2; ::::pg at instant k; the model
predictive control problem at the sampling instant k is dened as a constrained optimization
problem whereby the future manipulated input moves u(kjk); u(k + 1jk)......u(k + q 1jk)
are determined by minimizing an objective function dened as
=

p
X

ef (k + jjk) we ef (k + jjk) +

j=1

q 1
X

u(k + jjk)T wu u(k + jjk)

(89)

j=0

where
ef (k + jjk) = yrf (k + jjk)
j = 1; 2; ::::p

b(k + jjk)
y

(90)
(91)

represents future predicted error and

u(k + jjk) = u(k + jjk)
j = 1; :::q
u(kjk) = u(kjk)
14

u(k + j

1jk)

(92)

1
u(k

1)

(93)

The above minimization problem is subject to following constraints

yL

yH

bc (k + jjk)
y

(94)

j = 1; 2; ::::p
uL
u

uH

u(k + jjk)

uf (k + jjk)

j = 0; 1; 2; ::::q

(95)
u

(96)

Here, we represents positive denite the error weighting matrix and wu represents positive
semi-denite the input move weighting matrix. The closed loop stability and the desired
closed loop performance can be achieved by judiciously selecting the prediction horizon p;
control horizon q. Typically, prediction horizon is selected close to the open loop settling
time while control horizon (q) is chosen signicantly smaller (say between 1 to 5). The
weighting matrices we and wu are typically chosen diagonal and can be eectively used to
specify relative importance of errors and manipulated input moves. The resulting constrained
optimization problem can be solved using any standard nonlinear programming method such
as SQP (Sequential Quadratic Programming). For e cient on-line implementation, it is
possible to re-arrange above problem as a QP problem.
The controller is implemented in a moving horizon frame work. Thus, after solving the
optimization problem, only the rst move uopt (kjk) is implemented on the plant, i.e.
u(k) = uopt (kjk)
and the optimization problem is reformulated at the next sampling instant based on the
updated information from the plant. Note that the use of open loop observer limits applicability of DMC to open loop stable systems. Thus, if a process has unstable modes, then
these modes are stabilized rst by introducing a local feedback loop and DMC is applied as
a master controller in a cascade conguration.

3.4

Unconstrained DMC Control Law

b
Dening the future input vector Uf (k) and the predicted output vector Y(k)
over the future
horizon as
h
iT
(97)
Uf (k) =
u(kjk)T u(k + 1jk)T ::::::: u(k + q 1jk)T
h
i
b
Y(k)
=
(98)
b(k + 1jk)T y
b(k + 2jk)T ::::: y
b(k + pjk)T
y

the above prediction model, together with the constraints (118) on the future inputs, can be
expressed as
b
b(k=k 1) + Su Uf (k) + S b (k)
Y(k)
= Sx x
(99)
15

where

6
6
Sx = 6
4

C u
C
:::::
C q
C q
:::::
C p

6
6
6
6
6
6
Su = 6
6
6
6
6
4

C
C 2
::::::
C p
[0]
C u
:::::
C q
C q
::::::
C p

3
7
7
7
5

2
1

[0]
[0]
::::
::::
:::
::::
:::

u
u

6
6
S =6
4
::::
::::
:::
:::
::::
::::
:::

I
I
::::::
I

3
7
7
7
5

(100)

[0]
[0]
[0]
C u
C( + I) u
::::
C( p q + :::: + I)

7
7
7
7
7
7
7
7
7
7
7
5

(101)

Here, matrix Su is called dynamic matrix of the system. Dening the future reference
trajectory vector R(k) as
R(k) =

yrf (k + 1jk)

yrf (k + 2jk)

::::: yrf (k + pjk)

the prediction error vector E(k) at instant k can be computed as

E(k) = R(k)

iT

(102)

b
Y(k)

(103)

The unconstrained version of the DMC control problem stated above can be re-cast as follows
min E(k)T WE E(k) +

Uf (k)

Uf (k)T WU

(104)

Uf (k)

where WE and WU represents error weighting and input move weighting matrices, respectively, and are dened as
h
i
WE = diag we we :::: we
(105)
h
i
WU = diag wu wu :::: wu
(106)
Here,

Uf (k) is dened as
2

6
6
Uf (k) = 6
4

u(k=k) u(k 1)
u(k + 1=k) u(k=k)
::::::
u(k + q 1=k) u(k + q

16

3
2=k)

7
7
7=
5

Uf (k)

0 u(k

1)

(107)

where

6
6
=6
4

[0] [0]
I I [0]
::: ::: :::
[0] ::::
I

[0]
[0]
:::
I

3
7
7
7
5

6
6
=6
4

I
[0]
:::
[0]

3
7
7
7
5

(108)

This unconstrained problem can be solved analytically to compute a closed form control law.
The unconstrained optimization problem can be reformulated as
1
min Uf (k)T HUf (k) + Uf (k)T z(k)
Uf (k) 2

(109)

where
H = 2(STu WE Su + T WU )
h
b(k=k 1)
z(k) =
2 (R(k) Sx x

Sd b (k)) WE Su + (

0 uk 1 )

WU

The least square solution to above minimization problem is

(110)
(111)

[Uf (k)]opt = H 1 z(k)

Since only the rst input move is implemented on the process
uopt (kjk) =

T
0 Uf (k)

T
1
0 H z(k)

With some algebraic manipulations, the above control law can be rearranged as follows
uopt (kjk) = Gu u(k

1)

b(kjk
Gx x

1) + G

(kjk) + Gr R(k)

From the above rearranged equation, it is easy to see that the unconstrained DMC control
law is a special type of state feedback controller.

3.5

QP Formulation

The base constrained DMC formulation stated in Section 3.3 can be re-cast as follows
min E(k)T WE E(k) +

Uf (k)

Uf (k)T WU

Uf (k)

(112)

subject to the following constraints

YL
UL
UL

b
Y(k)

Uf (k)
Uf (k)
17

YH

(113)

UH

(114)
UH

(115)

Note that above formulation has a quadratic objective function and linear constraints. Thus,
for improving the computational e ciency, the above problem can be transformed into an
equivalent quadratic programming (QP) formulation as follows
1
min Uf (k)T H Uf (k) + zT Uf (k)
Uf (k) 2
Subject to AUf (k)
where

3
Iqm
6
7
6 Iqm 7
6
7
6
7
7 ;
A=6
6
7
6
7
6
7
4 Su
5
Su

6
6
6
6
b =6
6
6
6
4

(116)
(117)

UH
UL
UH + 0 uk 1
UL
0 uk 1
b
b(k=k 1) Sd d(k)
Sx x
+ YH
b
b(k=k 1) + Sd d(k)
Sx x
YH

3
7
7
7
7
7
7
7
7
5

Here, Iqm represents identity matrix of dimension q m: QP formulation is more suitable

for on-line implementation as e cient algorithms exist for solving QP.

3.6

MPC Formulation Based on Kalman Filter

The DMC controller discussed above gives good setpoint tracking performance. However.
the main limitation of DMC formulation arises from poor handling of unmeasured disturbance and plant model mismatch. Thus, the research eorts over last decade were directed
toward improving disturbance handing capabilities of MPC formulations. This section illustrates how MPC can be formulated using Kalman predictor to improve disturbance rejection
capabilities of MPC.
The main component of the improved MPC formulation is the model describing deterministic and stochastic contributions to process dynamics (equations 1-2), which is used to
develop the Kalman predictor (equations 41-44). At each sampling instant, the Kalman state
estimator is used for predicting future behavior of the plant over a nite future time horizon
of length p (called as the prediction horizon) starting from the current time instant k: Let us
assume that at any instant k; we are free to choose only q future manipulated input moves
fu(kjk); u(k + 1jk)::::::u(k + q

1jk)g

with the following constraints on the remaining of the future input moves
u(k + qjk) = u(k + q + 1jk) = :::::: = u(k + p

18

1jk) = u(k + q

1jk)

(118)

where q is called as the control horizon. As the expected value of future innovations is zero,
the optimal predicted estimates of the state variables can be generated by recursively using
the state estimator (41-42) as follows
b(k + 1jk) =
x

b(kjk
x

b(k + j + 1jk) =
x

1) +

b(k + jjk) +
x

(for j = 1::::Np

u u(kjk)
u u(k

+ Le(k)

+ jjk)

(119)
(120)

1)

A predictive control formulation based on the above optimal predictions can pose practical
di culties in the presence of mismatch between plant and the model. Since the model is
identied once in the initial phase of the MPC implementation, discrepancies between plant
and model parameters can arise over the period of time due to shift in the operating point,
changes in the disturbance characteristics etc. Thus, there is a need to introduce some
mechanism to account for the plant model mismatch, which, in turn, introduces integral
action in the controller formulation. This can be achieved by augmenting the state space
model with articially introduced input and / or output disturbance variables, which behave
as integrated white noise sequences. The resulting augmented model can be written as
x(k + 1) =

x(k) +

u u(k)

(k) + w(k)

(121)

(k + 1) =

(k) + w (k)

(122)

(k + 1) =

(k) + w (k)

(123)

y(k) = Cx(k) + C

(124)

(k) + v(k)

where 2 Rs and 2 Rt are articially introduced input and output disturbance vectors
while vectors w 2 Rs and w 2 Rt are zero mean white noise sequences with covariances
Q and Q ; respectively. The model coe cient matrices ( , C ) and noise covariances
matrices (Q , Q ) are treated as tuning parameters, which can be chosen to achieve the
desired closed loop disturbance rejection characteristics. Typical choices of the articial
state variables and the corresponding coupling matrices have been discussed in Section 2.3.
This augmented model can be used for developing a Kalman predictor of the form
ea (k) = y(k)
ba (k + 1jk) =
x

Cb
xa (kjk

ba (kjk
ax

1) +

(125)

1)
ua u(k)

+ La ea (k)

(126)

where La represents the steady state Kalman gain is obtained by solving the corresponding
steady state Riccati equations. The optimal predictions of states based on the augmented

19

state space model can be generated as follows

ba (k + 1jk) =
x

ba (k + j + 1jk) =
x

ba (kjk
ax
ba (k
ax

(for j = 1::::p

1) +

+ jjk) +

ua u(kjk)
ua u(k

+ La ea (k)

+ jjk)

(128)
(129)

1)

b(k + jjk) = Ca x
ba (k + jjk)
y

(127)

(for j = 1::::p)

(130)

Using the above augmented state observer, the MPC controller is formulated as an optimization problem at every instant as discussed in Section 3.3. It is also possible to derive closed
form unconstrained control law and a QP formulation suitable for on-line implementation as
discussed in section 3.4 and 3.5, respectively.

4
4.1

Appendix: Stability
Denitions

It is assumed that the notion of stability is known from the basic texts in control theory.
Only the basic denitions are given here. Consider the discrete time, state space equation
(possibly nonlinear and time varying)
(131)

x(k + 1) = F(x(k); k)

Let x (k) and x(k) be solutions of Eq. (131) when the initial conditions are x (k0 ) and
x(k0 ), respectively. Further let k : kdenote a vector norm.
Denition A.1 (Stability): The solution x (k) of Eq. (131) if stable if for any given
> 0, there exists a ( ; k0 ) > 0 such that all solutions with k x(k0 ) x (k0 ) k< are such
that k x(k) x (k) k< and for all k = k0 :
Denition A.2 (Asymptotic stability): The solution x (k) of Eq.131 is asymptotically stable if it is stable and if k x(k) x (k) k! 0 when k ! 1 provided that
k x(k0 ) x (k0 ) k is small enough.
From the denitions, it follows that stability in general is dened for a particular solution
and not for the system.
Theorem A.1. Asymptotic stability of linear systems. A discrete-time, linear, timeinvariant system,
x (k + 1) = x (k); x (0) = a0
is asymptotically stable if and only if all eigen values of
20

(132)

are strictly inside the unit circle.

Stability with respect to disturbances in the initial value has already been dened. Other
types of stability concepts are also of interest. In control we often work with input-output
models of the form

y(k) = F [y(k

1); y(k

2); :::; y(k

ny ); u(k

1); ::::; u(k

nu )] + e(k)

(133)

In the context of such models, we are interested in knowing whether the dynamic system
will generate bounded output sequence, i.e. ky(k)k1
My < 1 for all k if the system is
subjected to a bounded input sequence, i.e. ku(k)k1 Mu < 1 for all k:
Denition A.3. BIBO stability: A linear, time-invariant system is bounded-inputbounded-output (BIBO) stable if a bounded input gives a bounded output for every initial
value.
From the denition it follows that asymptotic stability is the strongest concept. The
following theorem is a result.
Theorem A.2. Asymptotic stability implies stability and BIBO stability.

4.2

Lyapunovs Second Method

Lyapunovs second method is a useful tool for determining the stability of non-liner dynamic
systems. (Analyzing stability using eienvalues is often referred to as Lyapunovs rst method.
) Lyapunov developed the theory for dierential equation, but a corresponding theory can
also be derived for dierence equations.
Denition A.4. Lyapunov function: Let V (x) represent a Lyapunov function for the
system,
x(k + 1) = F(x(k)) ;

F(0) = 0

(134)

If:
1. V (x) is continuous in x and V (0) = 0:
2. V (x) is positive denite.
3. 4V (x) = V [x(k + 1)]

V [x(k)] = V [F(x(k))]

V [x(k)] is negative denite for all k.

A simple geometric illustration of the denition is given in Figure (4.2). The level curves
of a positive denite continuous function V [x(k)] are closed curves in the neighborhood of
the origin. Let each curve be labelled by the value of the function. Condition 3 above implies
that the dynamics of the system are such that the solution always moves toward curves with
21

lower values. All the curves encircle the origin and do not intersect any other level curve.
It thus seems that the existence of a lyapunov function ensures asymptotic stability. The
following theorem is a precise statement of this fact.
Theorem 3. Stability theorem of Lyapunov: The solution x(k) = 0 is asymptotically stable if there exists a Lyapunov function for the system given by equation (134).
Further, if there exists (k x k) such that
(k x k) < V (x)

0<

where (k x k) ! 1 as k x k! 1; then the solution is asymptotically stable for all initial

conditions.
The main obstacle to using the Lyapunov theory is nding a suitable Lyapunov function.
This is in general a di cult problem; however, for the linear system of Eq.132, it is straightforward to determine quadratic Lyapunov functions. Take V (x) = xT Px as a candidate for
Lyapunov function. The increment of V is then given by
4V (x) = V ( x)
= xT [

V (x) = xT

P x

xT Px

P ]x = xT Qx

For V to be a Lyapunov function, it is thus necessary and su cient that there exists a
positive denite matrix P that satises the equation
T

P=

22

(135)

where Q is positive denite. Eq.135 is called Lyapunovs equation. It can be shown that
there is always a solution to the Lyapunov equation when the linear system is stable. The
matrix P is positive denite if Q is positive denite.

References

Astrom, K. J. and B. Wittenmark, Computer Controlled Systems, Prentice Hall, 1990.

Franklin, G. F. and J. D. Powell, Digital Control of Dynamic Systems, Addison-Wesley,
1989.

23