Closed Loop Nonlinear Model Predictive Control

Applied On Paralyzed Muscles To Restore Lower

Limbs Functions
Samer Mohammed, Philippe Poignet and David Guiraud
DEMAR Project - Department of Robotics - LIRMM
CNRS - INRIA - University of Montpellier II
161 Rue Ada, 34392 Montpellier Cedex 5, France
Email: mohammed@lirmm.fr
AbstractThe main goal when applying Functional Electrical
Stimulation (FES) to the paralyzed lower limbs of the paraplegic
patients is to avoid hyperstimulation and to defer the muscular
fatigue as much as possible. In this paper a closed loop position
control of the knee joint actuated by the quadriceps muscle
to perform exion-extension has been presented. The feedback
control consists of a Model Predictive Control (MPC) technique
which is also known by a receding horizon control or moving
horizon control. This controller is applied to a complex physio-
mathematical muscle model that is based on a macroscopic
Hill and a microscopic Huxley concepts. An MPC constitutes
an adequate controller with nonlinear multivariable systems.
Furthermore it enables us to incorporate explicitly constraints
on inputs, outputs and system states. The controller has shown
a robustness against force perturbation and model mismatch
as well as high capability of tracking a pre-dened reference
trajectory.
I. INTRODUCTION
Closed loop control of a muscle actuating the knee joint
of a paraplegic patient constitutes a prerequisite step for a
further assisted movements such as standing up, standing
and walking. Currently most clinics still adopt an open
loop control strategy although this later induces excessive
stimulation of the main muscles and thus fast muscular fatigue
and poor return to the patient. This could be explained by the
fact that closed loop control has currently several downsides
such as overwhelming the patient by sensors feedback to
update the stimulation levels and tuning controller parameters
for every case patient. The so-called subject centered
strategies [1], [2] introduce the voluntary contribution of
the upper body of the patient as an essential part of the
control diagram. This later is not yet adopted in clinical use
because of the relative high number of parameters to be
identied. In the current study, we have used a mathematical
muscle model which has been described in previous studies
[3], representing the most dynamic phenomenon that
occurred during contraction. The number of recruited motor
units increases as a function of intensity and pulse width
stimulation patterns. This phenomenon is modeled by an
activation model representing the ratio of recruited bers
and a mechanical model (muscle contraction) expressed by a
set of nonlinear differential equations. The ability to handle
nonlinear multi-variable systems that are constrained in states
and/or control variables motivates the use of Model Predictive
Control (MPC). This approach has proved its efciency in a
large variety of industrial processes, especially on chemical
processes [4]. The MPC problem is usually stated as an
optimization one subject to physical coherent constraints,
and is solved with classical optimization algorithms. In our
particular case the nonlinearities of the muscle model, the
constraints on the input stimulation current and on the output
knee joint position lead us to adopt a controller relying
on the MPC. Few studies have used this technique in a
musculoskeletal system. Some authors have applied a MPC to
a black-box models instead of continuous time physiological
models [5]. In the next section, the system modeling is
presented, it includes model of the knee-muscles, its state
space formulation and its linearized model around an arbitrary
operating point. In the third section, the theory of MPC is
presented. The results will be detailed in the last section.
II. SYSTEM MODELING
The biomechanical model consists of two segments repre-
senting respectively the shank and thigh connected to each
other by a joint with one degree of freedom. The thigh is sup-
Fig. 1. Functional Electrical Stimulation of skeletal muscles
posed xed with respect to the patient laying supine (Fig.1).
The knee joint is modeled by a pulley. Two agonist/antagonist
1-4244-0259-X/06/$20.00 2006 IEEE
259
Proceedings of the 2006 IEEE/RSJ
International Conference on Intelligent Robots and Systems
October 9 - 15, 2006, Beijing, China
muscles act on the knee, the quadriceps acts as an extensor
muscle while the hamstring is the exor muscle. As a result
two forces F
q
and F
h
cause respectively the extension and
exion of the knee.
Fig. 2. Biomechanical model of the knee
These forces are supposed to be constant along their di-
rections on the whole corresponding muscle (Fig.2). ( = 0
corresponds to full extension of the knee. F
q
and F
h
are the
inputs of the biomechanical model while the angle is the
corresponding output. (= 90

represents the rest position)

The geometric laws allow us to evaluate quadriceps length
L
q
depending on the knee angle variable
L
q
() =
_
L
2
0
r
2
+r+
_
L
2
iq
r
2
And the hamstring length L
h
() :
L
h
() =
_
L
2
0
+L
2
ii
+2L
0
L
ii
cos()
From the above equations, we can deduce the relative elonga-
tion of the quadriceps and the hamstring.

q
() =
L
q
L
0q
L
0q
=
_
L
2
0
r
2
+r+
_
L
2
iq
r
2
L
0q
L
0q

h
() =
L
h
L
0h
L
0h
=
_
L
2
0
+L
2
ii
+2L
0
L
ii
cos() L
0h
L
0h
L
0q
and L
0h
correspond respectively to the initial quadriceps
and hamstring lengths. Moment arm of the quadriceps is
supposed to be constant and equal to the pulley radius while
the moment arm of the hamstring depends on the variable
angle .
OH =
L
0
L
ii
sin()
_
L
2
0
+L
2
ii
+2L
0
L
ii
cos()
From the above equations and the equation of motion which
is a second order nonlinear dynamical equation, we obtained
the acceleration

as a function of the inertia around the knee
joint I, gravity, linear viscous damping. This latter represents
the global viscosity of the knee biomechanics and muscles. It
was identied by means of passive pendulum tests applied on
the knee joint.

=
1
I
[ r F
q
mgcos()L
1
F
v

L
0
L
ii
sin()

L
2
0
+L
2
ii
+2L
0
L
ii
cos()
F
h
]
Parameters of the above model were taken for an aver-
age person: L
0
= 50cm, L
iq
= 4cm, L
ii
= 5cm, r = 1cm, F
v
=
0.5N.m.s, I = 0.0476N.m.s
2
, g = 9.8N.m
2
, m = 3.5Kg.
A. Muscle model
Fig. 3. The muscle model followed by the biomechanical model
In previous papers [3], [6], a physiological skeletal muscle
model has been proposed to describe the complex internal
physiological phenomenon. We have adopted this model, in
order to develop strategies for simulation, motion synthesis
and motor control during clinical restoration of movement. In
(Fig.4) we show the model with the parallel element E
p
and
two element in series E
s
and E
c
. This model is controlled
by two variables: u
ch
, a chemical control input and , the
ratio of the recruited bers. We have described this model
by two differential equations where the outputs are K
c
and

c
representing, respectively, the stiffness and the constraint
generated by the contractile element. K
m
and
m
are the
maximum values of K
c
and
c
.
Fig. 4. Muscle model and particularization of EC
_

K
c
= (s
0
K
0
s
u
K
c
+s
v
q
s
0
F
0
K
c
s
u
F
c
K
c
1+pK
c
s
v
qF
c
)u
ch

s
v
aK
c
1+pK
c
s
v
qF
c

F
c
= (
s
0
F
0
s
u
F
c
1+pK
c
s
v
qF
c
)u
ch
+(
bK
c
s
v
aF
c
1+pK
c
s
v
qF
c
)
260
TABLE I
PARAMETERS OF BOTH MUSCLES QUADRICEPS AND HAMSTRINGS
Muscle model Variable Numeric value Unit
parameters (quadriceps -hamstring)
stiffness of E
s
K
s
1.10
4
N/m
Contractile element L
c0
41.10
2
, 38.10
2
m
length E
c
Elastic element L
s0
8.10
2
, 10.10
2
m
length E
s
Maximum isometric F
max
3100, 1295 N
muscle force
s
u
= sign(u
ch
) =
_
1 i f u
ch
< 0
+1 i f u
ch
> 0
s
v
= sign(
c
) =
_
+1 i f
c
> 0
1 i f
c
< 0
s
0
=
1+s
u
2
a =
L
0
L
c0
b = L
0
p =
1
K
s
q =
1
L
c0
K
s

c
=
L
c
L
c0
L
c0

s
=
L
s
L
s0
L
s0
=
LL
0
L
0
L = L
c
+L
s
where s
u
, s
0
and s
v
are the signs of the control and the veloci-
ties of the contractile element, L
c
and L
s
represent respectively
the length of the contractile and the elastic elements. The ratio
of recruited bers is considered as a global scale factor
which gives the percentage of the maximal possible force
which can be generated by the muscle. The parameters of
the muscles as shown in (Table1) were taken from [7].
B. Muscle-Knee: State space Model
Let us consider the model of the muscles and knee joint as
a non-linear state model:

X =f(x, t, U)
Where X=[X
1
...X
6
]
T
= [K
q
K
h
F
q
F
h


]
T
is the state
vector and U = [u
q
ch

q
u
h
ch

h
]
T
the control vector. The
variable represents the joint knee angle. The state variables
K
q
, F
q
, u
q
ch
,
q
and K
h
, F
h
, u
h
ch
,
h
are respectively the state
variables of the quadriceps and hamstring. The state model
of the biomechanical model of the knee-muscle model can
be expressed as:

X
1
= (s
0q

q
K
0q
+s
vq
q
q
s
0q

q
F
0q
X
1
s
uq
X
3
X1
1+p
q
X
1
s
vq
q
q
X
3
)u
q
ch

s
vq

q
X
1
rX
6
L
0q
+p
q
X
1
s
vq
q
q
X
3
s
uq
X
1
u
q
ch

X
2
= (s
0h

h
K
0h
+s
vh
q
h
s
0h

h
F
0h
X
2
s
uh
X
4
X
2
1+p
h
X
2
s
vh
q
2
X
4
)u
h
ch

s
vh
a
h
X
2
L
0
L
ii
sin(X
5
)
L
0h

L
2
0
+L
2
ii
+2L
0
L
ii
cos(X
5
)(1+p
h
X
2
s
vh
q
h
X
4
)
s
uh
X
2
u
h
ch

X
3
=
s
0q

q
F
0q
s
uq
X
3
1+p
q
X
1
s
vq
q
q
X
3
u
q
ch

b
q
X
1
s
vq
a
q
X
3
rX
6
L
0q
(1+p
q
X
1
s
vq
q
q
X
3
)

X
4
=
b
h
X
2
s
vh
a
h
X
4
L
0
L
ii
sin(X
5
)
L
0h

L
2
0
+L
2
ii
+2L
0
L
ii
cos(X
5
)(1+p
h
X
2
s
vh
q
h
X
4
)
+
s
0h

h
F
0h
s
uh
X
4
1+p
h
X
2
s
vh
q
h
X
4
u
h
ch

X
5
= X
6

X
6
=
1
I
(X
3
r X
4
L
0
L
ii
sin(X
5
)

L
2
0
+L
2
ii
+2L
0
L
ii
cos()
F
v
X
6
mgcosX
5
L
1
)
C. Model Linearization
The system

X = f(x, t, U) is a nonlinear multivariable
system. We have made some assumptions in order to apply
a linear predictive controller in a rst step to the linearized
system to get a feasible solution before applying the controller
to the non linear plant. Some hypothesis make this particular
system easier :
We consider that the chemical control u
ch
is a positive
constant indicating a muscular ber fusion and could
be veried in our case by the fact that the knee joint
dynamic is much higher than the muscle dynamics.
Consequently we have only contractions and no
relaxations during stimulation:
s
u
= 1, s
v
= 1, s
0
= 2, |u| = u, |
c
| =
c
Only one muscle: the quadriceps has been taken into
account in the following causing though extension of the
knee. When no extension, the gravity induces the exion
to the rest position.
We suppose that the stiffness of the series element which
represents the tendon is much more greater than the
stiffness of the contractile element. This hypothesis is
true since we are performing only dynamic movements
and no isometric stimulations were considered.

c
=
L
0
L
c0

1
L
c0
K
s

F
c
We can neglect the term
1
L
c0
K
s

F
c
under the above
conditions.
By taking into account the above assumptions, the plant model

X = f(X, t, U) will have a reduced nonlinear form. Where

X=[X
1
...X
4
]
T
= [K F

]
T
is the state vector and U =
is the control input. The System plant could thus be expressed
by the following set of differential equations:
261

X
1
= aX
1
+bX
1
X
4
+c f l(X
3
) U

X
2
= dX
1
X
4
+aX
2
+bX
2
X
4
+e f l(X
3
) U

X
3
= X
4

X
4
= f X
2
+hcos(X
3
) +lX
4
Where :
a = u
ch
, b =
r
L
c0
, c = u
ch
K
m
, d = r, e = u
ch
F
m
,
f =
r
J
, h =
mgL
1
J
, l =
F
v
J
, f l = exp[(

)
2
].
A Linearization of this nonlinear system around an operating
point (X,U) has been computed using the jacobian of the
system

X = f(X, t, U) as well as its constraints. The linear
model could be formulated as follow:
_

X = A.X+B.U
Y = C.X+D.U
Where :
A =
_

_
a+b.X
4
0 c.U

f l(X
3
) b.X
1
d.X
4
a+b.X
4
e.U

f l(X
3
) d.X
1
+b.X
2
0 0 0 1
0 f h.sin(X
3
) l
_

_
B =
_

_
c.U. f l(X
3
)
e.U. f l(X
3
)
0
0
_

_
C =
_
0 0 1 0

D = [0]
III. MODEL PREDICTIVE CONTROL: MPC
The Model predictive controller has been widely used in
different application due to their interesting properties [8]
The MPC problem [4] is usually formulated as a constrained
optimization problem:
min
u
Hp
k
J(x
k
, u
H
p
k
) (1)
subject to
x
i+1|k
= f (x
i|k
, u
i|k
) x
0|k
= x
k
(2)
u
i|k
U, i [0, H
u
] (3)
x
i|k
X, i [0, H
p
] (4)
where
U := u
k
R
m
| u
min
u
k
u
max
(5)
X := x
k
R
m
| x
min
x
k
x
max
(6)
Internal controller variables predicted from time instance k are
denoted by a double index separated by a vertical line where
the second argument denotes the time instance from which
the prediction is computed. x
k
= x
0|k
is the initial state of the
system to be controlled at time instance k and
u
k
=[u
0|k
, u
1|k
, ...u
H
u
1|k
, u
H
u
1|k
...u
H
u
1|k
] of dimension H
p
an
input vector.
At each sample, a nite horizon optimal control problem is
solved over a xed interval of time, the prediction horizon.
We assume that we would like the controlled variables, y
k
, to
follow some reference trajectory. In Fig.(5) we can see the past
and the future predicted output over the prediction horizon:
y
k
and y
k
respectively. Predictive control (Fig.5) consists on
computing the vector u
k
of consecutive inputs u
i|k
over the
control horizon H
u
by optimizing the objective function J
subject to constraints (2),(3),(4). The control signal is assumed
to be xed after H
u
samples over an horizon of dimension
(H
p
H
u
). When the solution of the optimal control problem
has been obtained, the value of the rst control variable in the
optimal trajectory, u(k|k), is applied to the process. The rest
of the predicted control variable trajectory is discarded, and
at the next sampling interval the entire procedure is repeated
[9]. These computations are updated at each sampling time.
Fig. 5. Principles of Predictive control strategy design
The nonlinear equality constraint on the state (2) represents
the dynamic model of the system. Bounding constraints over
the inputs u
i|k
and the state variables x
i|k
over the prediction
horizon H
p
are dened through the sets U and X.
The objective function J is usually dened as:
J(x
k
, u
H
p
k
) =(x
H
p
|k
) +
H
u

i=0
L(x
i|k
, u
i|k
) (7)
The objective function J is usually dened as:
J(x
k
, u
H
p
k
) =(x
H
p
|k
) +
H
u

i=0
L(x
i|k
, u
i|k
) (8)
where is a constraint on the state at the end of the prediction
horizon, called state terminal constraint, and L a quadratic
262
function of the state and inputs.
The computation of the solution u
H
p
k
can be divided in two
steps : rstly, computation of a solution satisfying the con-
straints (including the state terminal constraint), and secondly
optimization. The rst step involves bounding constraints
(3,4), and nonlinear constraints expressing the dynamic model
of the system (2).
Fig. 6. Details of the Model Predictive control application
Simulations were performed in MATLAB-SIMULINK envi-
ronment using the ode45 integration algorithm with variable
stepsize. The simulation codes was adapted to the MPCtools
from [9].
IV. RESULTS AND DISCUSSION
Different simulation tests have been carried out to the linear
and the non-linear plant model. The sample period was set
to 0.01 sec and the prediction and control horizon H
p
and
H
u
were computed as a function of the time constant of the
system: H
p
=30 and H
u
=10. The constrained input u = was
the recruitment variable since the recruitment curve is a static
function and the optimal pulse width and current stimulation
could be computed by a simple inverse recruitment curve.
This variable has been constrained to be between 0 and 1
representing respectively non ber recruited and maximal ber
recruited. In the current paper we do not separate between
slow and rapid bers. The controlled variable which is also
the output was constrained to stay between = 0

(hyper-
extension) and = 90

(maximal exion). Only the knee

position was used as feedback to update the control input that
corresponds to the desired trajectory. Controller parameters
were calculated ofine. The results of the simulation is shown
in Fig.(7). Initial conditions correspond to =30

, that means
a knee extension and thus an initial muscle force and stiffness
are required. At t = 2 sec the desired trajectory was stepped
to = 80

which corresponds to exion of the knee. the

controller converge to the desired position in a nite time while
maintaining the control input between its limits. At t = 6 sec
we perform again a knee extension inducing an increase in the
muscle force and stiffness. Fig.(8) showed the optimization
time needed to perform the above simulation. In Fig.(9) we
add an external position disturbance of
dis
= 10

to the
system at time t = 7sec . The controller managed to converge
to the desired position and to compensate fully the position
disturbance without need for feedback observer and respecting
the constraints imposed to the states, input and output. It
should be noticed that the muscle parameters used in this
study relate to a healthy subject (Table I). The inaccuracy that
may occurs on these parameters when dealing with paraplegic
patients could be compensated by the robustness of the (MPC)
controller.
0 2 4 6 8 10
0
200
400
600
800
1000
M
u
s
c
l
e
S
i
t

n
e
s
s
a
n
d
F
o
r
c
e
Time [s]
0 2 4 6 8 10
20
40
60
80
100
K
n
e
e
D
e
s
i
r
e
d
t
r
a
j
e
c
t
o
r
y
Time [s]
0 2 4 6 8 10
2
1
0
1
2
3
K
n
e
e
v
e
l
o
c
i
t
y
Time [s]
0 2 4 6 8 10
0
0.2
0.4
0.6
0.8
1
I
n
p
u
t
c
o
n
t
r
o
l
Time [s]
Stiffness [N/m]
Force [N]
Current position []
Desired position []
Knee velocity [rad/sec]
Constraints
Fig. 7. Tracking knee joint trajectory and controller performances
0 2 4 6 8 10
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
O
p
t
i
m
i
z
a
t
i
o
n
t
i
m
e
[
s
]
Time [s]
Fig. 8. Predictive control optimization time
V. CONCLUSIONS AND FUTURE WORKS
A. Conclusions
The skeletal muscle has been rarely treated as an en-
tire physiological element in a closed loop control design
dedicated to restore movements for paraplegic patients. The
Model Predictive Control has shown its capability to handle a
multivariable nonlinear system which corresponds to our case.
Furthermore the MPC offers the possibility to integrate con-
straints on input, output and measured states explicitly in its
263
0 2 4 6 8 10
0
200
400
600
800
1000
S
i
t

n
e
s
s
a
n
d
F
o
r
c
e
Time [s]
0 2 4 6 8 10
0
20
40
60
80
100
K
n
e
e
D
e
s
i
r
e
d
t
r
a
j
e
c
t
o
r
y
Time [s]
0 2 4 6 8 10
0.5
0
0.5
1
1.5
2
2.5
K
n
e
e
v
e
l
o
c
i
t
y
[
r
a
d
/
s
e
c
]
Time [s]
0 2 4 6 8 10
0
0.2
0.4
0.6
0.8
1
I
n
p
u
t
c
o
n
t
r
o
l
Time [s]
Disturbance
Constraints
Fig. 9. Controller responses when applying disturbance
formulation. The MPC was applied to a new multi-scale model
developed within the DEMAR project. The muscle model is
based on internal physiological characteristics assembling two
levels: the microscopic one, involving the sliding actin-myosin
laments and the macroscopic part represented by a contractile
element and an elastic element. This model highly nonlinear
has been described by a set of differential equations. We made
some realistic assumptions to the muscle model when it has
been coupled to the knee joint and supposing that we perform
a dynamic movements. As a result we obtain a simplied
nonlinear version of the knee-muscle formulation. This later
has been linearized around an arbitrary operating point. We
were able to control the quadriceps muscle for knee exion-
extension in order to track a predened position trajectory
within a large range of movement. Satisfactory stability and
tracking error were achieved after a nite time delay. The
results show that we respect the constraints on input and
output. The controller shows also a robustness against external
disturbances and model mismatch.
B. Future Works
Further work will be carried out with paraplegic patients
to evaluate the accuracy and the robustness of the MPC
controller. We are trying to limit the computational effort
which is a common decit of the MPC design. Actually,
the optimisation time obtained (Fig.8) was around 20 ms
in Matlab environnement which is quite encouraging for
a real time implementation. An observer-based output
feedback with disturbance model should be integrated. Other
control strategies will be implemented as well to analyze
the relevance of the muscle model in much complex situations.
VI. ACKNOWLEDGMENTS
Many thanks to all DEMAR members who provided all
supports, substantive feedback and great contribution to the
present work. This work is supported by the MIMES RNTS
national grant.
VII. NOMENCLATURE
L
0
= Thigh length,
L
1
= Shank length,
O = Center of rotation (Knee),
L
iq
= Distance between O and the insertion point of the
quadriceps on the shank,
L
ii
= Distance between O and the insertion point of hamstring
on the shank,
L
q
= Length of the quadriceps,
L
h
= Initial length of the hamstring ,
r = Pulley radius,
H = Orthogonal projection of O on Li,
F
q
= Extension force generated by the quadriceps,
F
h
= Flexion force generated by the hamstring,
G = Gravity force vector,
m = Mass of the shank.
u
ch
= Constant chemical control input to the muscle model,
L
c0
= Resting muscle contractile element length,
L
s0
= Resting muscle serial element length,
= coefcient of the shank mass distribution (< 1),
F
v
= coefcient of viscous friction,
f l= Normalized Force-length relation,

f l= derivative of the normalized Force-length relation,
= Shape factor identied equal to 0.35
H
p
= Prediction Horizon,
H
u
= Control Horizon,
J = Cost function,
Q, R = Weighting matrices for the cost function.
REFERENCES
[1] Riener R., Fuhr T. : Patient-Driven Control of FES-Supported standing
Up: A simulation study , IEEE Transactions on rehabilitation engineer-
ing, Vol 6, pp.113-123, 1998.
[2] Donaldson N., Yu C. : FES standing control by handle reactions of leg
muscle stimulation (CHRELMS) , IEEE Transactions on rehabilitation
engineering, Vol 4 pp.280-284, 1996.
[3] El Makssoud H., Guiraud D., Poignet P. : Mathematical muscle
model for Electrical Stimulation control strategies , IEEE International
Conference on Robotics and Automation, pp. 1282-1287, 2004.
[4] Allgower F., Badgwell T. A., Qin S. J., Rawlings J. B., and Wright S.
J. : Nonlinear predictive control and moving horizon estimation - an
introductory overview , in Advances in Control: Highlights of ECC 99,
P. Frank, Ed. Springer-Verlag, ch. 12, pp. 391449, 1999.
[5] Schauer T. and Hunt K. J. : Nonlinear predictive control of knee-joint
angle using FES , in Proc. IFESS Conf., pp. 425-428, Aalborg, Denmark,
June 2000.
[6] El Makssoud H., Mohammed S., Fraisse P. : Control of the knee joint
under Functional Electrical Stimulation - Simulation results based on a
new Physiological muscle model , 8th Vienna International Workshop
on Functional Electrical Stimulation, pp.113-116, Vienna, Austria, 2004.
[7] Kromer V. : Analyse des forces musculaires au cours de la marche -
Approche en corps rigide et simulation en mecanismes plans exibles par
elements nis , Thesis at I.N.P.L., Lyon, France, 1993.
[8] Camacho, E., C. Bordons : Model Predictive Control in the Process
Industry , Springer, New york (1995).
[9] Johan kesson : Operator Interaction and Optimization in Control
Systems , P.hd thesis, Department of Automatic Control Lund Institute
of Technology Lund, December 2003.
264