Beruflich Dokumente
Kultur Dokumente
Abstra t
SeDuMi is an add-on for MATLAB, whi
h lets you solve optimization problems with
linear, quadrati
and semideniteness
onstraints. It is possible to have
omplex valued data
and variables in SeDuMi. Moreover, large s
ale optimization problems are solved e
iently,
by exploiting sparsity. This paper des
ribes how to work with this toolbox.
Keywords: Symmetri
one, semidenite programming, se
ond order
one programming,
self{duality, MATLAB, SeDuMi
M Master University, Hamilton, Canada. Supported by Netherlands Organization for S ienti Resear h (NWO).
whi
h are user-friendly front-ends for SP. More re
ently, the user-friendly front-end SeDuMi
Interfa
e[22 is available from
http://www.laas.fr/~peau
ell/SeDuMiInt.html
A more re
ent and faster solver for semidenite programming is SDPA, by Fujisawa, Kojima and
Nakata [11. Other solvers for semidenite programming are CSDP by Bor
hers [2, SDPHA by
Brixius and Potra [3 and SDPT3 by Toh, Todd and Tutun
u [28. See Mittelmann [19 for a
omparison of the performan
e of various solvers (in
luding SeDuMi) on semidenite programming problems of the SDPLIB test set.
For optimization over symmetri
ones, there were until re
ently only two software tools
available, viz. SDPPa
k, by Alizadeh et al. [1, and SeDuMi. (SDPT3 has been extended to this
lass of problems in 2001.) Both operate under the MATLAB environment, so that they
an
easily be used within spe
i
appli
ations. SeDuMi has some features that are not available in
SDPPa
k, namely it
allows the use of
omplex valued data,
generates Farkas-dual solutions for infeasible problems,
takes full advantage of sparsity, leading to signi
ant speed benets,
has a theoreti
ally proven O(pn log(1=)) worst-
ase iteration bound,
promotes sparsity by handling dense
olumns separately (sin
e version 1.03), using a te
hnique proposed by Goldfarb and S
heinberg [13,
an import linear programs in MPS format (either using a link to LIPSOL [32 or by using
the loadmps add-on by the author), and semidenite programs in SDPA [11 format.
It is also possible to
onvert optimization problems from SDPPa
k [1 format into SeDuMi. Noti
e
that earlier versions of the semidenite programming solver SDPT3 were also able to handle
omplex valued data [28, but this option has been removed re
ently. The issue of expoiting
sparsity in semidenite programming was studied by Fujisawa, Kojima and Nakata [10. Unlike
the approa
h of [10, SeDuMi uses always the same sparsity expoiting pro
edure to form the
normal equations; this pro
edure is e
ient regardless of the degree of sparsity. See Ross [24
for a
omparison between SDPpa
k and SeDuMi.
The remainder of this do
ument is a step-by-step tutorial for SeDuMi. The on-line help pages
serve as a referen
e to the toolbox. In addition, the Appendix to this do
ument has the details
of the
alling sequen
e for the main fun
tion, sedumi.
- SeDuMi 1.05
(OCT2001)
- Conversion to SeDuMi.
The
ommand help
onversion produ
es a list of fun
tions for importing data into SeDuMi.
This in
ludes an `umbrella' s
ript, getproblem1, whi
h works as follows:
>> pname = 'truss2'; getproblem, who
Your variables are:
At
K
MATNAME
PROBDIR
pname
This imports problem `truss2', and pla
es it in the variables At, b,
and K. To do this, SeDuMi
must be able to nd the requested problem somewhere on your disk. It
an lo
ate sparse SDPA
problems, if you have assigned a UNIX or DOS environment variable `SDPLIB' to the dire
tory
path where SDPA problems are stored. (SDPLIB and SeDuMi have a dierent
anoni
al form; if
y is a dual optimal solution
al
ulated by SeDuMi, then y is an optimal solution in the SDPLIB
anoni
al form.) If LIPSOL is installed, it uses LIPSOL's fun
tion findprob to lo
ate linear
programming problems in MPS format. Finally, if SDPPa
k is installed, and the environment
variable SDPPACK points to the SDPPa
k dire
tory, then SDPPa
k problems are sear
hed for in
the dire
tory `SDPPACK/problems'.
Typing `help SeDuMi105' produ
es a list of the fun
tions that you
an use to build and
solve optimization models over symmetri
ones. They are: sedumi, eigK, ve
, mat and eyeK.
Online help is provided by help sedumi, help eigK, and so on. The following se
tions give a
more detailed explanation of these fun
tions, with some illustrating examples.
1
This fun tion has not been updated sin e Version 1.00, and may not be ompatible with the latest software
2 linear programming
It is possible to formulate your linear programming model in either the primal standard form,
minimize
T x
su
h that
Ax = b
(1)
x 0 for i = 1; 2; : : : ; n;
or the dual standard form,
maximize
bT y
(2)
su
h that
aTy 0 for i = 1; 2; : : : ; n:
Mixed form and symmetri
form linear programming models may be formulated using the `free
variable' denition in SeDuMi, as outlined in Se
tion 2.1.
Suppose that you want to solve the following linear programming problem:
minimize
x1 x2
su
h that 10x1 7x2 5
(3)
x1 + x2 =2 3
x1 0; x2 0:
In order to formulate this LP problem in the primal standard form, we have to add sla
k
variables, say x3 and x4 . In MATLAB, we
an then enter the b and
ve
tors, and the A matrix
as follows:
i
We
an now solve problem (3) in the primal form (1) by invoking the fun
tion sedumi.
Remark that MATLAB is
ase sensitive, and it is therefore essential to write sedumi in lower
ase.
>> sedumi(A,b,
)
SeDuMi by Jos F. Sturm, 1998.
Alg = 2: xz-
orre
tor, theta = 0.250, beta = 0.500
eqs m = 2, order n = 5, dim = 5, blo
ks = 1
it :
x
gap delta rate t/maxt feas
0 :
5.00E+00 0.000
1 : 7.81E-01 9.79E-01 0.000 0.1959 0.9000 0.77
2 : -5.52E-02 9.40E-02 0.000 0.0959 0.9900 0.93
This shows that the optimal value is 0:125, as listed under
*x. The fun
tion sedumi
returns an optimal solution, whi
h in this
ase is x1 = 1:9583 and x2 = 2:0833. Noti
e that
x is indeed feasible for (1), be
ause all its
omponents are nonnegative, and Ax = b, as
an
be
he
ked by the
ommands min(x) and norm(Ax-b), respe
tively. Of
ourse, some round-o
errors may o
ur, as
an be seen from the following MATLAB output:
>> A=sparse(A);norm(A*x-b)
ans =
1.7764e-15
>> norm(A*(24*x)-24*b)
ans =
0
The rst quantity is listed as jAx bj = 1:8e 15 in the output of SeDuMi. The se
ond line
shows that the reported value of |Ax-b| does not only
ontain the residual, but also errors in
omputing the residual. The meaning of the other parts in the output of SeDuMi is explained in
Se
tion 5.
Using dual solutions, it is possible to
he
k also optimality. Namely, if we let z :=
AT y,
then if x and y are feasible to (1) and (2) respe
tively, we have
0 zTx =
T x yTAx =
Tx bTy:
Thus, if
T x bTy = 0, then
T x must be minimal, and bTy must be maximal, over all feasible
solutions. The dual solution y to (2) is assigned to the se
ond output argument of sedumi, as in
5
we see that z x = 0 for all i, proving optimality. However, due to some round-o errors,
bT y is positive in this
ase. The quantity digits = 15:1 in the output of SeDuMi, is
dened as follows:
log10 ((
T x bTy)=(jbT yj + 10 10 )) if
Tx bT y > 0
digits =
(4)
1 otherwise:
As is well known, y is a subgradient of the optimal value fun
tion in terms of
hanges in b. If
the optimal value fun
tion is lo
ally not dierentiable in b, i.e. if there are multiple dual optimal
solutions, then it is said to be primal degenerate. SeDuMi usually generates a solution y in
the relative interior of the subgradient set, be
ause it uses a Mehrotra-Ye [18 type termination
pro
edure for linear programs. For a detailed treatment of sensitivity analysis based on su
h
solutions, we refer to Monteiro and Mehrotra [20 and the book of Roos, Terlaky and Vial [23.
For large problems, it is usually not feasible to store A as a full matrix, due to memory
limitations. In this
ase, A should be stored in sparse format; type help sparfun for details.
Internally, SeDuMi always
onverts A to sparse format. The b and
ve
tors
an also be in sparse
format, if desired.
In the pre
eding, we dened b and
in MATLAB as
olumn ve
tors, but this is not essential;
SeDuMi produ
es the same output if b and/or
are dened as row ve
tors. Similarly, SeDuMi is
not pi
ky about the orientation of A: it will dete
t the
orre
t orientation based on the b and
ve
tors (ex
ept in the unrealisti
ase that A is square). In fa
t, it is good pra
ti
e to store A
in su
h a way that it has more rows than
olumns, whi
h is the transpose orientation of the A
matri
es that we have seen so far. Namely, if A is stored in sparse format, then it is stored as a
set of sparse
olumn ve
tors. Hen
e, if there are fewer
olumns, it will o
upy less spa
e.
There is a third output argument of SeDuMi,
alled info. In our example,
T x
8
<
:
puse
:
iter:
feasratio:
numerr:
pinf:
dinf:
0.1100
3
1
0
0
0
This is a
ompound output argument, or stru
ture, with a eld
puse
for the solution time,
iter for the number of iterations, a eld numerr whi
h is nonzero in
ase of numeri
al problems
(1 means premature termination: results are ina
urate, 2 means failure), a eld feasratio
for the nal value of the feasibility indi
ator, and two elds, pinf and dinf, for the dete
ted
feasibility status of the optimization problem. If pinf = 1, then the primal problem (1) is
infeasible, and y is a Farkas dual solution.
For instan
e, if we
hange the b ve
tor in the pre
eding example to b = 5; 0:4 , then
SeDuMi yields info:pinf = 1, bT y = 0:0955 > 0, max =1 2 3 4 aT y = 0:1866 0. Noti
e that
for any x with Ax = b, we have yTAx = bTy = 0:0955 > 0, whereas yTAx 0 for nonnegative
x, be
ause all
omponents of AT y are nonpositive. A Farkas dual solution thus provides a
erti
ate of infeasibility. In this example, there appears to be a Farkas dual solution for whi
h
all entries of ATy are stri
tly negative. In general though, they are merely nonpositive. For
numeri
al reasons, ATy
an then
ontain some small positive
omponents, and in this
ase we
have an approximate Farkas dual solution. Loosely speaking, su
h solutions demonstrate that
there
annot be any reasonably sized primal feasible solution; see Todd and Ye [27 for details.
Suppose now that we want to solve a problem in the dual standard form (2). In this
ase,
y with bT y > 0 and AT y 0 has the interpretation of an improving dire
tion. Namely, if
there exists a feasible solution y, i.e. if
ATy 0, then y + ty is feasible for all t 0, and
lim !1 bTy = 1. In this
ase, we say that the problem is unbounded. The other possibility is
that there does not exist any feasible solution y, i.e. the problem is infeasible. To distinguish
between an infeasible and an unbounded problem, we have to go through a se
ond stage, by
solving a feasibility problem:
h
; ; ;
In our previous example, the dual problem is feasible, and the above
ommand yields a feasible
solution y. The need for this se
ond stage feasibility problem is typi
al for interior point methods
with the self{dual embedding te
hnique of Ye, Todd and Mizuno [31.
7
0
1.1875
0.0990
0.9895
>> b*y
ans =
1.3878e-17
In this example, the primal does not have an interior solution, i.e. it is weakly feasible, and this
is demonstrated by a Gordan-Stiemke dual solution y. Namely, 0 6= ATy 0, and bTy = 0,
whi
h
learly implies that there
annot be any x > 0 su
h that A x = b.
SeDuMi treats the primal and dual in a symmetri
way, i.e. it does not favor one over the
other. From a modeling point of view however, the primal standard form and the dual standard
form are quite dierent, and it depends on the appli
ation whi
h one is more favorable. The
primal form has the advantage of expli
it equality
onstraints. In prin
iple, equality
onstraints
an be
onstru
ted in the dual form also, simply by means of two inequality
onstraints, su
h as
8
and aTy
. However, this te
hnique is not re
ommended, sin
e su
h
onstraint pairs
tend to get a pair of very large primal multipliers x , hen
e leading to numeri
al di
ulties. It
may be better to enfor
e an equality
onstraint by eliminating a y variable. However, the latter
te
hnique may destroy the sparsity stru
ture of the A-matrix, thus leading to longer solution
times.
Exa
tly the same problems arise in modeling a free (i.e. unresistri
ted in sign) variable in
the primal standard form. Splitting su
h a variable into two, its positive part and its negative
part, often results in numeri
al di
ulties. One may also try to eliminate su
h a variable by
removing an equality
onstraint, but this usually
auses an in
rease in the number of nonzeros in
the A-matrix. An alternative is to model all free variables in a quadrati
one. Quadrati
ones
are dis
ussed in Se
tion 3. To prevent numeri
al di
ulties with this te
hnique, it is desirable
to x a { possibly large { upper bound on the norm of the ve
tor of free variables, whi
h is
easily done in a quadrati
one.
Sin
e Version 1.05, the user does not need to worry about these issues, sin
e free variables
are allowed. This is the topi
of the se
tion below.
aT y
i
It is possible to formulate your linear programming model in a primal form with free variables
as follows:
minimize
T x
su
h that
Ax = b
(5)
x 2 < for i = 1; 2; : : : ; K:f;
x 0 for j = K:f + 1; K:f + 2; : : : ; n;
where K.f is the number of free variables. The asso
iated sla
k variables in the dual problem
are then restri
ted to be zero, thus allowing equality
onstraints in the dual:
maximize
bT y
T
su
h that
a y = 0 for i = 1; 2; : : : ; K:f
(6)
T
a y 0 for j = K:f + 1; K:f + 2; : : : ; n
i
positive semidenite matri
es. The standard primal form for su
h optimization problems is
minimize
T x
su
h that Ax = b
(7)
x2K
maximize
su
h that
bT y
AT y 2 K :
(8)
min
y1 + y2 y1 kq
P y3 k; y2
1 + ky3 k2 ;
(10)
where P is a given matrix, and q a given ve
tor. The above is a robust least squares problem,
see El Ghaoui and Lebret [7. The de
ision variables are the s
alars y1 and y2, and the ve
tor
y3 . This problem has two quadrati
onstraints, viz.
(y1 ; q P y3) 2 Q
one; y2; 1 2 Q
one:
(11)
"
#!
y3
Given P and q, the following MATLAB fun
tion (rls.m)
onstru
ts problem (10) in the
standard dual form (8). The A matrix will be in transposed orientation, and is hen
e denoted
as At.
1
2
3
4
5
6
7
8
% [At,b,
,K = rls(P,q)
% Creates dual standard form for robust least squares problem "Pu=q".
fun
tion [At,b,
,K = rls(P,q)
[m, n = size(P);
% ---------- minimize y_1 + y_2 ---------b = -sparse([1; 1; zeros(n,1));
% ---------- (y_1, q - P y_3) in Q
one ----------
10
9
10
11
12
13
14
15
16
17
18
Noti
e rst that the above fun
tion uses sparse data types, in order to save memory. Furthermore, a stru
ture K is dened, with a eld K.q that lists the dimensions of the quadrati
ones. (The `q' in K.q stands for `quadrati
'.) The K{stru
ture will be used to tell SeDuMi
that the
omponents of
ATy are not restri
ted to be nonnegative as they would be in linear
programming. Instead, the rst K:q(1) entries are restri
ted to a quadrati
one, and the last
K.q(2) entries are restri
ted to another quadrati
one. This is the way in whi
h we model the
symmetri
one K in (7) and (8), and hen
e
onstru
t the two quadrati
onstraints in (11).
As a numeri
al example, we solve a 4 3 robust least squares problem with dependent
olumns in P . The example is from [7.
>> P = [3 1 4;0 1 1;-2 5 3; 1 4 5; q = [0;2;1;3;
>> [At,b,
,K = rls(P,q);
>> [x,y,info = sedumi(At,b,
,K);
SeDuMi by Jos F. Sturm, 1998.
Alg = 1: v-
orre
tor, theta = 0.250, beta = 0.500
eqs m = 5, order n = 5, dim = 11, blo
ks = 3
it :
x
gap delta rate t/maxt feas
0 :
5.00E+00 0.000
1 : -1.23E+01 1.30E+00 0.000 0.2605 0.9000 -0.18
2 : -5.94E+00 3.34E-01 0.000 0.2568 0.9000 0.54
3 : -3.60E+00 6.14E-02 0.116 0.1838 0.9000 0.86
4 : -3.34E+00 1.80E-03 0.000 0.0293 0.9900 1.10
* 5 : -3.33E+00 4.00E-06 0.000 0.0022 0.9990 1.00
* 6 : -3.33E+00 9.59E-09 0.000 0.0024 0.9990 1.00
* 7 : -3.33E+00 6.06E-10 0.153 0.0632 0.9900 1.00
* 8 : -3.33E+00 1.24E-10 0.000 0.2037 0.9000 1.00
iter se
onds digits
*x
b*y
8
0.1 11.3 -3.3329085968e+00 -3.3329085968e+00
|Ax-b| = 2.8e-16, |x|= 2.0e+00, |y|= 2.5e+00
11
In the above
all to SeDuMi, we see a new input argument, viz. K. This argument makes
solve an optimization problem in the form (7){(8), where the symmetri
one K is
des
ribed by the stru
ture K. Without the fourth input argument (K), SeDuMi would solve a
linear programming problem of the form (1){(2).
To
he
k that (11) is indeed satised by the solution y, it is in prin
iple possible to verify
the inequality in denition (9) dire
tly. However, it is more
onvenient to use the fun
tion
eigK, whi
h is part of SeDuMi. This fun
tion returns the eigenvalues (or spe
tral values) of a
ve
tor with respe
t to a symmetri
one. A symmetri
one
onsists of those ve
tors whi
h
have nonnegative eigenvalues, see e.g. the book by Faraut and Koranyi [9. For a quadrati
one (9), there are merelyp two eigenvalues, viz. given a ve
tor
p (x1 ; x2 ) 2 < < 1, we have
1 (x1 ; x2 ) = (x1 kx2 k)= 2 and 2 (x1 ; x2 ) = (x1 + kx2 k)= 2.
We
an thus
he
k feasibility and optimality as follows:
SeDuMi
-0.0000
3.2307
-0.0000
1.4827
For symmetri
ones K, it holds that xTz 0 for all x 2 K and z 2 K. Therefore, x provides
an optimality
erti
ate for y just as in the
ase of linear programming. The interpretation
of Farkas dual solutions extends in the same way. See the survey paper of Luo, Sturm and
Zhang [16 for the details. However, a paradoxal phenomenon
an o
ur, viz. that x and y are
almost feasible, whereas
Tx bT y is
onsiderably negative (kxk and/or kyk must then obviously
be very large). SeDuMi will then report an innite number of digits in a
ura
y, a
ording to
formula (4). This phenomenon was explained by Luo, Sturm and Zhang [15 and Sturm [25.
It is possible that an optimization model has both nonnegativity and quadrati
one
onstraints. For instan
e, we may extend the above example with the restri
tion that y3[1 0:1,
where y3 [1 denotes the rst
omponent in the ve
tor y3. This restri
tion
an be added to the
model as follows:
>> a1 = zeros(1,length(y)); a1(3) = 1;
12
>>
>>
>>
>>
= [-0.1;
; At = [a1;At;
K.l = 1;
[x,y,info = sedumi(At,b,
,K);
eigK(
-At*y,K)'
ans =
0.0000
-0.0000
3.2307
-0.0000
1.4904
The eld K:l is the number of nonnegative variables, whi
h in this
ase is one. (The `l' in K.l
stands for `linear'.) By
onvention, the nonnegative variables are always the rst
omponents,
so that K = <+ Q
one Q
one in our
ase. As
an be seen from the output of eigK, there
are 5 eigenvalues for this
one: 1 for ea
h nonnegativity
onstraint, and 2 for ea
h quadrati
onstraint. We say that K is a symmetri
one of order 5. (SeDuMi reports `order n = 6', be
ause
of its internal self{dual reformulation.)
SeDuMi supports an alternative form of the quadrati
one, viz.
R
one := (x1 ; x2 ; x3) 2 < < < 2 x1 x2 12 kx3k2 ; x1 + x2 0 :
(12)
Geometri
ally, R
one is simply a rotation of Q
one. The spe
i
form of R
one is
onvenient for
modeling
onvex quadrati
fun
tions. Namely, by adding the linear equality
onstraint `x1 = 1'
to the model, we obtain the restri
tion
1
x2 kx3 k2 :
2
Throughout the model, we
an then use x2 as a tight upper bound on kx3 k2 =2. Fra
tions are
also
onveniently modeled by R
one
onstraints. For instan
e, we may minimize 1=x1 for x1 > 0
by solving the model
minfx2 j x1x2 1; x1 + x2 0g:
Noti
e that this problem does not have a solution: the inmum of 1=x1 is zero, for x1 ! 1.
>>
>>
>>
>>
lear K;
= [0, 1, 0; b = sqrt(2); A = [0, 0, 1; K.r = 3;
[x,y,info = sedumi(A,b,
,K);
x(2), x(1)*x(2)
ans =
1.5360e-05
13
ans =
1.0147
You may nd that x2 is not yet
lose enough to zero, and that x1 is not equal to 1 either.
However, the primal solution is feasible, the dual solution is almost feasible, and the duality
gap is even negative. This illustrates an error bound di
ulty, whi
h is usual for this type of
irregular problems. In Se
tion 5, we will see how to obtain a more a
urate solution, by setting
an optional parameter, pars.eps.
As illustrated by the above example, the eld K.r serves to list the dimensions of R
one
onstraints, analogously to the denition of Q
one
onstraints by K.q. (The `r' in K.r stands
for `rotated quadrati
one'.) Setting both K.l, K.q and K.r elds yields a symmetri
one of
the form
K = <K+l (Q
one Q
one) (R
one R
one):
For instan
e, we
an add a bound `x1 107 ' to the model as follows:
:
Some appli
ations of Q
one and R
one
onstraints are dis
ussed in Lobo et al. [14.
3.2 The positive semidenite
one
Semideniteness
onstraints are an important
lass of restri
tions that
an be modeled with
SeDuMi. As an example,
onsider the following problem:
min
(
m
X
i
=1
(m i)x
ii
m
Xk
i
=1
x +
i;i
= b for k = 0; : : : ; m 1; X is psd
k
(13)
Here, X is an m m symmetri
matrix, and x denotes the entry on row i and
olumn j . The
length m ve
tor b is given. The abbreviation `psd' stands for `positive semidenite'. The above
optimization problem yields a minimal phase spe
tral fa
torization of an auto
orrelation ve
tor
b, see Davidson, Luo and Sturm [4. Problem (13) is stated in terms of an m m symmetri
matrix of de
ision variables, whereas SeDuMi works with a ve
tor of de
ision variables, as in (7){
(8). This small issue is resolved by using the well known te
hnique of ve
torization. Ve
torization
is implemented by the fun
tions ve
and mat, whi
h are part of SeDuMi. The fun
tion ve
(X)
reates a long ve
tor, by sta
king the
olumns of the matrix X , as in:
ij
14
-3
-9
-3
-9
The inverse of ve
is mat. Thus, if x is a ve
tor of length n2, then mat(x)
onstru
ts an n n
matrix, and lls it with the entries of the ve
tor x, starting at the rst
olumn.
>> mat(x)
ans =
1
5
-3
5
2
-9
-3
-9
4
The following MATLAB fun
tion produ
es a standard primal form for problem (13).
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
The eld K:s = [m tells SeDuMi that we want the m m matrix mat(x) to be symmetri
positive semidenite. (The `s' in K.s stands for `semidenite'.) We
an now solve problem (13)
as follows:
>> b = [2; 0.2; -0.3;
>> [At,b,
,K = spe
fa
(b);
>> [x,y,info = sedumi(At,b,
,K);
15
To
he
k positive semideniteness, we
an either use the fun
tion eig that is part of MATLAB,
or the fun
tion eigK, whi
h
omes with SeDuMi.
0.0000
0.0000
2.0000
The use of eigK is more
onvenient, espe
ially if there are multiple semideniteness
onstraints, or if there are also nonnegativity or quadrati
one
onstraints. SeDuMi will always
produ
e symmetri
matrix variables, i.e. mat(x) is symmetri
. Do not add symmetry
onstraints expli
itly, as in `x x = 0'. At best, su
h
onstraints will be removed by SeDuMi
from the A matrix.
However, the dual solution
ATy need not be symmetri
, as
an be seen in the numeri
al
example that we are dealing with:
ij
ji
-0.3130
1.0727
0
0.6849
-0.3130
0.0727
16
In this
ase, the dual solution is upper triangular, be
ause mat(
) is diagonal, and mat(At(:,k))
is upper triangular for all k = 1; 2; : : : ; m. Letting Z = mat(
AT y), SeDuMi restri
ts the
symmetri
part of Z , whi
h is (Z + Z T)=2, to be positive semidenite. The fun
tion eigK yields
the eigenvalues of the symmetri
part. Thus,
>> eigK(
-At*y,K)
ans =
1.0583
2.1597
-0.0000
ii
i;i
k;i
i;i
m
Xk
i;i
k;i
The solutions x and y, as produ
ed by SeDuMi, will be exa
tly the same. However, sin
e
the
onstraints in the A matrix have been symmetrized, we nd that mat(
At y) is now
symmetri
; it is the matrix (Z + Z 0)=2.
For SeDuMi, it does not make any dieren
e whether the
onstraints in A and the obje
tive
are symmetrized or not. However, when modeling in the primal standard form, you will probably
nd it more natural to work with upper or lower triangular matri
es in A and
; your model will
also use less memory like this. On the other hand, symmetri
matri
es are more natural when
modeling in the dual form.
There
an be multiple positive semideniteness
onstraints, in whi
h
ase K.s lists the orders
of the respe
tive matri
es. This is analogous to the denition of multiple quadrati
onstraints
in K.q and/or K.r. The positive semidenite variables are always the last
omponents of x and
AT y, i.e.
K = <K+l (Q
one Q
one) (R
one R
one) (S
one S
one);
where S
one denotes the
one of positive semidenite matri
es. It is easy to remember the above
arrangement, by noting the alphabeti
al order of `l', `q', `r' and `s'.
:
17
4
omplex values
In some appli
ation areas, su
h as signal pro
essing, optimization problems may involve
omplex
valued data. An example is the Toeplitz Hermitian
ovarian
e estimation problem, whi
h is
dis
ussed in Wu, Luo and Wong [30. Other stru
tured
ovarian
e estimation problems, su
h as
dis
ussed in Deng and Hu [5,
an be treated similarly. Given a Hermitian matrix P , the goal
is to nd a Hermitian positive denite matrix Z with a Toeplitz stru
ture, su
h that kP Z k
is minimal. Thus, the optimization problem is:
minimize =1 (z p )2 + 2 = +1 jz p j2
(15)
su
h that Z is Toeplitz, i.e. z = z +1 +1 for all i; j = 1; 2; : : : ; m 1
Z is psd:
If the matrix P has
omplex entries, then we will usually also see
omplex entries in the optimal
solution Z . Noti
e that the Toeplitz property is better modeled in the dual form, than in the
primal form. In fa
t, mat(At*y) in (13) is an upper triangular real Toeplitz matrix, and in (14),
it is a symmetri
Toeplitz matrix. The MATLAB formulation of (15) therefore resembles the
MATLAB formulation of (13).
F
Pm
i
ii
i;j
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Pm
ii
ij
ij
;j
% [At,b,
,K = toepest(P)
% Creates dual standard form for Toeplitz-
ovarian
e estimation
fun
tion [At,b,
,K = toepest(P)
m = size(P,1);
% ---------- maximize y(m+1) ---------b = [sparse(m,1); 1;
% ----- Let e be all-1, and allo
ate spa
e for the A-matrix ----e = ones(m,1);
K.q = [1 + m*(m+1)/2;
K.x
omplex = 2:K.q(1);
%Norm-bound entries are
omplex valued
At = sparse([,[,[,K.q(1) + m^2,m+1,1 + 2*m^2);
% ----------
onstraints ---------% -y(m+1) >= norm( ve
(P) - sum(y_i * Ti) )
(Q
one)
% sum(y_i * Ti) is psd
(S
one)
% --------------------------------At(:,1) = [sparse(2:(m+1),1,1,K.q(1),1); -ve
(speye(m));
= [0; diag(P);
firstk = m+2;
for k = 1:(m-1)
lastk = firstk + m-k-1;
Ti = spdiags(e,k,m,m);
18
23
24
25
26
27
28
29
30
31
32
We have modeled the obje
tive fun
tion by means of an arti
ial variable, y +1, and y2 +1
is bounded from below by the original quadrati
obje
tive fun
tion, using a 1 + m(m + 1)=2dimensional quadrati
one. The Toeplitz matrix is modeled as
m
y1 I + 2
X
i
(16)
y +1 T ;
i
=1
where T is all-1 along the k-th upper diagonal, and zero everywhere else. Re
all from problem
(13) in Se
tion 3.2, that in the real
ase, SeDuMi restri
ts the symmetri
part of mat(
AT y) to
be positive semidenite. In the
omplex
ase, SeDuMi restri
ts the Hermitian part, i.e. mat(
AT y) + mat(
AT y)0 , to be positive semidenite. Letting Z denote the Hermitian part of (16),
we have
1
Z = y1 I +
(y +1T + y +1T T);
i
X
i
=1
where y +1 denotes the
omplex
onjugate of y +1. Thus, we have indeed modeled Z as a
Hermitian Toeplitz matrix, and SeDuMi further restri
ts it to be positive semidenite, be
ause
of the eld K.s. Furthermore, we tell SeDuMi to allow
omplex values for y2; y3; : : : ; y , by
setting K.y
omplex = 2:m. Remark that unlike K.l, K.q, K.r and K.s, the eld K.y
omplex is
not involved in the denition of the symmetri
one K in (7){(8).
The following lines show how to solve problem (15), for a parti
ular 3 3 Hermitian matrix
P , whi
h is neither Toeplitz, nor positive semidenite.
i
>> i = sqrt(-1);
>> P = [4, 1+2*i, 3-i; 1-2*i, 3.5, 0.8+2.3*i; 3+i, 0.8-2.3*i, 4
P =
4.0000
1.0000 - 2.0000i
1.0000 + 2.0000i
3.5000
3.0000 - 1.0000i
0.8000 + 2.3000i
19
3.0000 + 1.0000i
0.8000 - 2.3000i
4.0000
0.8079 + 1.7342i
4.2827
0.8079 - 1.7342i
2.5574 - 0.7938i
0.8079 + 1.7342i
4.2827
>> eigK(z,K)'
ans =
-0.0000
2.0517
0.0000
7.2810
5.5670
Instead of using the mat() fun
tion, one may use the
ellK() fun
tion as follows:
>> z =
ellK(
-At*y,K); Z=z.s{1}; Z=(Z+Z')/2
We have found the optimal positive semidenite Toeplitz matrix Z , whi
h has eigenvalues 0,
7:281 and 5:567. Che
king the obje
tive values reveals a new phenomenon:
>> [
'*x; b'*y
ans =
-1.4508 - 0.2428i
-1.4508
The value of
Hx, where H means
omplex
onjugate transpose, may no longer be real, and the
same is true for bHy in general. Obviously, we
annot minimize or maximize
omplex valued
fun
tions. Instead, SeDuMi minimizes Re
Hx in the primal, and maximizes Re bHy in the dual.
Here, Re stands for real part. In the sequel, we will also use the notation Im , to denote the
imaginary part.
If we make K.y
omplex = [, then all dual multipliers y are restri
ted to be real.
i
20
ans =
-4.5592 - 0.3816i
-4.5592
Clearly, by restri
ting y to be real, the dual optimal value Re bHy = y +1 gets worse. Apparently, something has
hanged in the primal problem as well, sin
e the primal optimal value
has improved from 1:4508 to 4:5592. The dieren
e is in the `Ax = b' restri
tion, as the
following lines show:
m
21
A similar
onvention holds for the rst entry in a q-se
ond order
one. The remaining entries
in a se
ond order
one that are listed in K.x
omplex are simply
omplex variables that appear
in the norm-bound restri
tion. Only entries in the f,l,q,r-
ones
an be listed in K.x
omplex;
the matrix variables are handled by the eld K.s
omplex.
On the dual side, K.x
omplex lists the equality and nonnegativity
onstraints for whi
h the
restri
tion Im
aTy = 0 must be imposed on the imaginary part. This interpretation also
works for the rst entry in a se
ond order q-
one. The remaining entries in a se
ond order
one that are listed in K.x
omplex are simply
omplex variables that appear in the norm-bound
restri
tion (this is
ompletely symmetri
to the primal).
For sensitivity analysis, it is interesting to note that Re (
)H x is a supergradient for the
optimal value fun
tion, under perturbations of the form
+ t
, whereas Re (b)H y is a subgradient of the optimal value fun
tion under perturbation of b. For a dis
ussion of sensitivity
analysis in (real symmetri
) semidenite programming, see Goldfarb and S
heinberg [12.
i
5 optional settings
By default, SeDuMi lls your terminal s
reen with some output
on
erning its iterative progress.
This
an be an annoying feature, in parti
ular if SeDuMi is merely used as a subroutine within
a larger program. To suppress the on-s
reen output of SeDuMi, it su
es to set an optional
parameter, pars.fid, to zero.
>> load truss1
>> pars.fid = 0;
>> [x,y,info = sedumi(At,b,
,K,pars);
The stru
ture pars is not only used for suppressing iterative output of SeDuMi. It
an
ontain
a number of optional elds, whi
h we will dis
uss in this se
tion.
The abbreviation `d' in pars.fid stands for `le identier': the output of SeDuMi will be
sent to the le whose le identier is pars.fid. The le identier for the null-devi
e is 0, whi
h
is useful for suppressing output, and for the terminal s
reen it is 1. Output
an also be redire
ted
to a le, e.g.
>> pars.fid = fopen('truss1.out','w');
>> [x,y,info=sedumi(At,b,
,K,pars);
>> f
lose(pars.fid); pars.fid = 1;
With the above lines, the output is redire
ted to the le `truss1.out', as
an be
he
ked with the
ommand dbtype truss1.out.
SeDuMi uses a variant of the primal{dual interior point method, whi
h is known as the
entering{predi
tor{
orre
tor method [25. There are 3 variants of the
entering{predi
tor{
orre
tor method implemented, whi
h
an be sele
ted with the eld pars.alg. With pars.alg
22
normalized vvalues
1.5
0.5
6
iterations
10
12
Reduction rates
1
reduction rate
0.8
0.6
0.4
0.2
0
5
6
iterations
10
11
you sele
t a longest-step algorithm, without any se
ond order
orre
tor. To enhan
e the
algorithm with a se
ond order
orre
tor, you
an either set pars.alg = 1 or pars.alg = 2.
With pars.alg = 1, the se
ond order
orre
tor is derived by linearization of the so-
alled vvalues, whereas pars.alg = 2 uses linearization of the squared v-values, whi
h is also known as
xz -linearization. For linear programming, xz -linearization results in the well-known Mehrotra's
orre
tor [17. In all three variants, the
entering step is determined by the
entral region parameter, pars.theta. This parameter
an take any value in (0; 1. At one extreme, pars.theta
= 1 results in path{following, whi
h typi
ally involves relatively short step lengths. Setting
pars.theta to a smaller value, su
h as 1=4, makes the algorithm work in the neighborhood of
a full dimensional
entral region, and this typi
ally allows for larger step lengths, see Sturm
and Zhang [26. The size of the neighborhood is
ontroled by the parameter pars.beta, whi
h
an be assigned any value in (0; 1). In the output of SeDuMi on the terminal s
reen, there is
a
olumn labeled `delta', whi
h lists the a
tual distan
e to the
entral region in ea
h iteration.
The step length will always be su
h that this is at most pars.beta. The ratio of the a
tual step
length and the maximal steplength to the boundary of the
one K is listed in the
olumn labeled
`t/maxt'. For some iterations, an asterisk (`*') appears in front of the output line. At these iterations, the residual ve
tor of the self{dual model has been re
omputed (to avoid a
umulation
of numeri
al errors).
For resear
h purposes, SeDuMi
an produ
e a plot of the iterative v{values. This feature is
a
tivated by setting pars:vplot = 1. For problem truss1, this results in the plots of Figure 1.
For ea
h iteration, the rst plot shows all the v{values, divided by the mean of the v-ve
tor
in that iteration. It also gives a horizontal line at value 1, representing the
entral path, and
a horizontal line at the
entral region threshold, pars.theta = 1/4. Any v-values below this
23
threshold will be
orre
ted by the
entering
omponent in the su
eeding iteration. The se
ond
plot shows the rate of linear redu
tion, whi
h is simply
duality gap in iteration k
duality gap in iteration k 1 :
The rate of linear redu
tion is also listed in the
olumn `rate' in the on-s
reen output of SeDuMi,
and the iterative duality gap is listed under `gap'. This is the duality gap in an arti
ial self{dual
model, in whi
h your original model is embedded by SeDuMi, using the te
hnique of Ye, Todd
and Mizuno [31. The self{dual model gives rise to a feasibility indi
ator, listed in the
olumn
`feas'. Ideally, the indi
ator
onverges to +1 for feasible problems, and to 1 for (primal and/or
dual) infeasible problems.
Termination
ontrol is provided by the elds maxiter, and eps in the pars stru
ture. SeDuMi
will terminate su
essfully if it nds a solution that violates feasibility and optimality requirements by no more than pars.eps. The parameter pars.maxiter allows you to set a maximum
on the number of iterations. By default, pars.eps = 1E-9 and pars.maxiter = 150. A possible experiment with these parameters is to set pars.eps = 0 in the example of minimizing
1=x1 , whi
h was dis
ussed in Se
tion 3.1.
I thank T. Terlaky for en
ouraging me to write this manual, and for pointing
out a bug in the rst publi
release of SeDuMi. P. Apkarian, M. Bengtsson, T.N. Davidson, F. Glineur,
V. Prodanovi
and A. Ross helped to improve the software, by providing bug reports and suggestions on
the rst release of the software. Numerous other users have
ontributed in the same manner sin
e.Two
anonymous referees have
ontributed in improving this do
ument.
A
knowledgments.
Referen
es
[1 F. Alizadeh, J.A. Haeberly, M.V. Nayakkankuppann, M. Overton, and S. S
hmieta. SDPPa
k user's guide. New York University, New York, USA, 1997.
[2 B. Bor
hers. CSDP, a C library for semidenite programming. Te
hni
al report, New
Mexi
o Te
h Mathemati
s Fa
ulty, USA, 1997.
[3 N. Brixius and F.A. Potra. Sdpha: A MATLAB implementation of homogeneous interiorpoint algorithms for semidenite programming. Te
hni
al report, Department of Computer
S
ien
e, University of Iowa, Iowa City, Iowa, USA, 1998.
[4 T.N. Davidson, Z.-Q. Luo, and J.F. Sturm. A (primal form of the) positive real lemma for
FIR systems. Te
hni
al report, Communi
ations Resear
h Laboratory, M
Master University, Hamilton, Canada, 1998. To appear.
24
[5 S. Deng and H. Hu. Computable error bounds for semidenite programming. Te
hni
al
report, Department of Mathemati
al S
ien
es, Northern Illinois University, DeKalb, Illinois,
USA, 1996.
[6 S. Dussy and L. El Ghaoui. Multiobje
tive robust
ontrol toolbox for LMI-based
ontrol.
Te
hni
al report, Laboratoire de Mathematiques Appliquees, ENSTA, Paris, Fran
e, 1997.
[7 L. El Ghaoui and H. Lebret. Robust solutions to least-squares problems with un
ertain
data. SIAM Journal on Matrix Analysis and Appli
ations, 18(4):1035{1064, 1997.
[8 L. El Ghaoui, R. Nikoukhah, and F. Delebe
que. LMITOOL: A front-end for LMI optimization, user's guide. Laboratoire de Mathematiques Appliquees, ENSTA, Paris, Fran
e,
1995.
[9 J. Faraut and A. Koranyi. Analysis on Symmetri
Cones. Oxford Mathemati
al Monographs. Oxford University Press, New York, 1994.
[10 K. Fujisawa, M. Kojima, and K. Nakata. Exploiting sparsity in primal-dual interior-point
methods for semidenite programming. Mathemati
al Programming, 79:235{253, 1997.
[11 K. Fujisawa, M. Kojima, and K. Nakata. SDPA (semidenite programming algorithm)
user's manual | version 4.10. Te
hni
al report, Dept. of Information S
ien
es, Tokyo
Institute of Te
hnology, 2-12-1 Oh-Okayama, Meguro-ku, Tokyo 152, Japan, 1998.
[12 D. Goldfarb and K. S
heinberg. On parametri
semidenite programming. Te
hni
al report,
Columbia University, Department of IEOR, New York, USA, 1997.
[13 D. Goldfarb and K. S
heinberg. Stability and e
ien
y of matrix fa
torizations in interior
point methods. Conferen
e presentation, HPOPT IV Workshop, June 16{18, Rotterdam,
The Netherlands.
[14 M.S. Lobo, L. Vandenberghe, S. Boyd, and H. Lebret. Appli
ations of se
ond{order
one
programming. Te
hni
al report, Information Systems Lab, Stanford University, 1997. To
appear in Linear Algebra and Appli
ations.
[15 Z.-Q. Luo, J.F. Sturm, and S. Zhang. Duality and self-duality for
oni
onvex programming. Te
hni
al Report 9620/A, E
onometri
Institute, Erasmus University Rotterdam,
Rotterdam, The Netherlands, 1996.
[16 Z.-Q. Luo, J.F. Sturm, and S. Zhang. Duality results for
oni
onvex programming. Te
hni
al Report 9719/A, E
onometri
Institute, Erasmus University Rotterdam, Rotterdam,
The Netherlands, 1997.
[17 S. Mehrotra. On the implementation of a primal{dual interior point method. SIAM Journal
on Optimization, 2:575{601, 1992.
25
[18 S. Mehrotra and Y. Ye. Finding an interior point in the optimal fa
e of linear programs.
Mathemati
al Programming, 62:497{515, 1993.
[19 H.D. Mittelmann.
Several SDP-
odes on problems from SDPLIB.
ftp://plato.la.asu.edu/pub/sdplib.txt, 1998.
[20 R.D.C. Monteiro and S. Mehrotra. A general parametri
analysis approa
h and its impli
ation to sensitivity analysis in interior point methods. Mathemati
al Programming, 72:65{82,
1996.
[21 Y. Nesterov and M.J. Todd. Self{s
aled barriers and interior{point methods for
onvex
programming. Mathemati
s of Operations Resear
h, 22(1):1{42, 1997.
[22 D. Peau
elle, D. Henrion and Y. Labit. User's guide for SeDuMi Interfa
e 1.01: Solving LMI problems with SeDuMi. LAAS - CNRS, Toulouse, Fran
e. Available from
http://www.laas.fr/ peau
ell/SeDuMiInt.html.
[23 C. Roos, T. Terlaky, and J.-Ph. Vial. Theory and algorithms for linear optimization. An
interior point approa
h. Series in dis
rete mathemati
s and optimization. John Wiley &
Sons, New York, 1997.
[24 A.M. Ross. Optimization over
ones: SDPpa
k versus SeDuMi. Te
hni
al Report 1998-1022, Department IEOR, University of California at Berkeley, 1998.
[25 J.F. Sturm. Primal{Dual Interior Point Approa
h to Semidenite Programming, volume
156 of Tinbergen Institute Resear
h Series. Thesis Publishers, Amsterdam, The Netherlands, 1997.
[26 J.F. Sturm and S. Zhang. On a wide region of
enters and primal{dual interior point
algorithms for linear programming. Mathemati
s of Operations Resear
h, 22(2):408{431,
1997.
[27 M.J. Todd and Y. Ye. Approximate Farkas lemmas and stopping rules for iterative
infeasible{point algorithms for linear programming. Mathemati
al Programming, 81:1{21,
1998.
[28 K.C. Toh, M.J. Todd, and R.H. Tutun
u. SDPT3 - a MATLAB pa
kage for semidenite
programming. Te
hni
al report, S
hool of Operations Resear
h and Industrial Engineering,
Cornell University, Itha
a, NY, USA, 1996.
[29 L. Vandenberghe and S. Boyd. SP: Software for semidenite programming. Information
Systems Laboratory, Ele
tri
al Engineering Department, Stanford University, Stanford,
USA, 1994.
26
[30 S. Wu, Z.-Q. Luo, and K. Wong. Dire
tion nding for
oherent sour
es via Toeplitz approximation. Te
hni
al report, Communi
ations Resear
h Laboratory, M
Master University,
Hamilton, Ontario, Canada, 1997.
[31 Y. Ye, M.J. Todd, and S. Mizuno. An O(pnL)-iteration homogeneous and self{dual linear
programming algorithm. Mathemati
s of Operations Resear
h, 19:53{67, 1994.
[32 Y. Zhang. Solving large{s
ale linear programs by interior{point methods under the MATLAB environment. Te
hni
al report, Department of Computational and Applied Mathemati
s, Ri
e University, Houstan, Texas, USA, 1997.
A Calling Sequen
e
The primal
anoni
al form for solving optimization problems with SeDuMi is
minf
T x j Ax = b; x 2 Kg;
and the dual
anoni
al form is
maxfbTy j
ATy 2 Kg:
The general
alling sequen
e for solving the above primal{dual pair is
[x,y,info = sedumi(A,b,
,K,pars)
Here, K is a MATLAB stru
ture to dene the symmetri
one K; it
onsists of the following
(optional) elds:
K.f The number of free primal variables, i.e. the number of dual equality
onstraints
K.l The number of nonnegativity
onstraints
K.q A list of dimensions of quadrati
one
onstraints
K.r A list of dimensions of rotated quadrati
one
onstraints
K.x
omplex A list of primal variables in the f,l,q,r blo
ks that are allowed to have a nonzero
imaginary part. The imaginary parts of the asso
iated dual
onstraints are then expli
itly
restri
ted.
K.s A list of orders of positive semideniteness
onstraints
K.s
omplex A list of matrix variables that are restri
ted to be Hermitian positive semidenite.
27
This eld is not related to K. It lists the
omponents of the y-variables that are
omplex valued. Equivalently, it lists the primal equality
onstraints (Ax) = b that have
to be satised not only in their real parts, but also in their imaginary parts.
The stru
ture K denes K to be
K = <K f <K+l (Q
one Q
one) (R
one R
one) (S
one S
one);
In total, the number of Q
one (R
one, S
one)
omponents is length(K.q) (length(K.r),
length(K.s)). The ith Q
one is
Q
one = f(x1 ; x2 ) 2 < C K q[i 1 j x1 kx2 kg;
where C denotes the spa
e of
omplex n{tuples. The j th R
one is
R
one = (x1 ; x2 ; x3) 2 < < C K r[j 2 x1 x2 12 kx3 k2 ; x1 + x2 0 ;
and the kth S
one is
S
one = x 2 C K s[k j mat(x) is Hermitian positive semidenite ;
for primal
omponents. For dual
omponents z =
ATy, we use a slightly milder denition of
S
one, viz.
S
one = z 2 C K s[k j mat(z) + mat(z)0 is positive semidenite :
If k is not listed in K.s
omplex, then the denition in the dual is even milder, namely,
S
one = z 2 C K s[k j Re mat(z) + mat(z)0 is positive semidenite :
The length of the ve
tor
should be
K.y
omplex
If the data (A; b;
) is real valued, then x and y will also be real valued.
The parameter pars is a MATLAB stru
ture,
onsisting of the following (optional) elds:
pars.fid By default, pars.fid=1, whi
h tells SeDuMi to produ
e iterative statisti
s on the
s
reen. If pars.fid=0, then SeDuMi runs quietly, i.e. no s
reen output. In general, output
is written to the le or devi
e that is identied by the le handle pars.fid. A le handle
is assigned to a le by the MATLAB fun
tion fopen, as in
pars.fid = fopen('truss1.out','w').
28
pars.alg By
2.
30