40.

510 LINEAR OPTIMIZATION Fall 2013

Problem Set 1
Deadline: 7 Oct 2013, 2pm - class
Pn
1. The L1 norm of a vector v <n , denoted kvk1 is equal to
j=1 |vj |.
Suppose that we are given a system of linear constraints Ax = b that are
overdetermined, in that there are no feasible solutions that satisfy these
constraints. One way in which we might wish to specify a nearly feasible
solution to find a vector x such that kAx bk1 is as small as possible.
(a) Show how to formulate this optimization problem as a linear program, and in fact, as a minimization problem in standard form (that
is, subject to equality constraints with non-negative variables.)
(b) Write out the linear programming dual of your answer to the previous
part.
(c) The L norm of a vector v <n , denoted kvk , is equal to maxj=1,...,n |vj |.
Use this norm to write a somewhat more compact representation of
the dual problem.
Solution:
(a) There are various ways to formulate this problem. We will introduce
nonnegative slack and surplus variables s and t so that si ti captures
the violation in ith constraint. This would give the following LP:
min

eT s + eT t

s.t.

Ax + s t = b
s 0, t 0,

where e is a vector of ones as usual. This is not in standard form

yet. We can substitute x = x+ x , with x+ , x 0, to get an LP
in the desired form:
min

eT s + eT t

s.t.

Ax Ax + s t = b
+

x+ 0, x 0, s 0, t 0.
(b) Taking the dual, we obtain:
max

bT y

s.t.

AT y 0
AT y 0
ye
y e.

item The first two inequalities in the dual LP give AT y = 0 and the
last two constraints are equal to kyk 1. Hence our final LP is
equal to:
max

bT y

s.t.

AT y = 0
kyk 1.

Alternatively, you can introduce a new variable

ri 0 for each constraint
P
with ri aTi x bi ri and minimize i ri .
2. What is the dual of the linear program with variables x 0 and an
additional single variable with constraints Ax b , where the objective
is to maximize ?
Solution: The primal LP is max{ : Ax b 0, x 0}. Taking the
dual, we get:
min

s.t.

AT y 0
bT y = 1
y 0.

Note that if the dual LP is feasible, then its objective value is zero. Additionally, note that the primal LP is always feasible as (x; ) = (0; 0)
is a feasible solution. However, if there exists a feasible solution (x; )
with > 0, then any positive scalar multiple is feasible as well, as
A(x) b = (Ax b) 0. Hence, the primal is always feasible,
but if there exists a feasible solution with strictly positive objective value,
then the primal is unbounded. (The dual would be infeasible in this case.)
3. Consider one trading period of a market that handles n assets and whose
state can be characterized as being in one of m possibilities. An investor
who invests one dollar in asset j when the market initially is in state i
receives a payoff of rij dollars at the end of the period; let R denote this
payoff matrix. Let x = (x1 , ..., xn ) denote the initial portfolio of assets,
in which each xi may be positive or negative (i.e., a negative amount
corresponds to a promise to buy back that much at the end of the period).
The
Pn value vi of a portfolio x, given the state of the market is i, is exactly
j=1 rij xj ; let v be the vector of possible values. If p is the vector of asset
prices at the start of the period, then px is the price of acquiring portfolio
x. The absence of arbitrage condition is that no investor should be able
to get a guaranteed non-negative payoff from a negative payoff: that is,
if Rx 0 then px 0. Show that this condition holds for prices p and
payoff matrix R if and only if there is a non-negative vector q such that
qR = p.
Solution: Farkas Lemma states that exactly one of the following systems
can hold:
I. x such that Ax = b, x 0

II. y such that yA 0, yb < 0 (Note that in class we have shown

AT y 0, bT y > 0, but these conditions are equivalent. Note also
that we are working with a row vector here!)
Applying this with A = RT and b = p leads: q 0 such that qR = p if
and only if (I) holds.
(I) holds if and only if (II) does not hold, i.e., @x such that Rx 0, px < 0,
which is equivalent to stating that Rx 0 implies that px 0.
4. Consider the LP problem min{cT x : Ax = b}, i.e., with equality constraints and unrestricted variables. State the dual of this problem. Show
that strong duality holds: if either problem has an optimal solution, so
does the other and their optimal values are equal. What can you say about
optimal solutions of these problems?
Solution: Let A <mn as usual. The dual of this LP is: max{bT y :
AT y = c}. To show strong duality we have to consider three cases:
Case 1. Neither problem has a feasible solution. In this case neither problem
has an optimal solution either.
Case 2. One problem has a feasible solution but the other does not. Without
loss of generality let us assume that it is the primal problem that
has a feasible solution, say x
and the dual problem is infeasible.
This means that c
/ Col(AT ) = Row(A), therefore, there must exist
d <n s.t. Ad = 0 and cT d > 0. So the objective function value is
unbounded above on the ray {x := x
+ d, 0}.
Case 3. Both problems have feasible solutions. In this case, if x and y are
feasible, then
bT y = y T (Ax) = (y T A)x = (AT y)T x = cT x,
so both solutions are optimal.
This question should persuade you that LPs without inequality constraints
are NOT interesting!
5. Let S n denote the space of real symmetric n n matrices. This is a finitedimensional vector space: indeed, by considering just the upper-triangular
entries or by taking an appropriate basis, it can be viewed as isomorphic
to <n(n+1)/2 . A matrix A S n is called positive semidefinite if xT Ax 0
for all x <. Show that the set of positive semidefinite real symmetric
n n matrices is a convex cone containing the origin. You can do this
directly, or by showing that it is an (infinite) intersection of half-spaces
corresponding to hyperplanes through the origin.
Solution: To show that the set of positive semidefinite real symmetric
n n matrices is a convex cone, we simply must show that the set of
closed under taking positive combinations. Let A and B be psd matrices
and , 0, then for any x <n , we have
xT (A + B)x = (xT A + xT B)x = xT Ax + xT Bx 0.
Furthermore if A = 0, then xT Ax = 0 0, for all x <n , so the set
contains the origin.
3

6. In the first few weeks of the class, we have spent time exploring two views
of looking at a set S <n . For many sets (we have seen some in class), we
can find a subset T S, such that every point x S can be written as a
(linear or convex or conical or positive)-combination of points in T . This
is the first view: describing the set by its generators. The second view is
describing the set S with the set of constraints that must be satisfied by
its points.
Any closed convex set can be written as the intersection of half-spaces and
also as convex combinations of some number of points.
(a) Are these two numbers (number of half-spaces and number of points)
equal? Prove or provide a counter-example.
(b) Find a closed convex set where the cardinality of the set of half-spaces
and also the cardinality of the set of generating vectors is equal to
the cardinality of the real line.
Solution: Consider a circle C. Call the interior and boundary of C
combined as D (disc). Any point in D can be expressed as a convex
combination of points in C. D can also be written as intersection of
infinitely many half-spaces, one for each point in C. In this case, the

Figure 1: A circle can be written as the interscetion of infinitely many halfspaces.

cardinality of the set of half-spaces and of the set of generating points are
equal, as both are equal to the cardinality of the real number set <.
But in general, this is not the case: Think about the unit cube in <3 , it
requires eight points to express all of its points as convex combinations,
but six half-spaces are enough to describe it, one for each face of the
cube. In fact, the number of points can be exponentially larger than the
number of half-spaces and vice versa. Consider the hypercube in <n , it
requires 2n points to generate all of its points, but 2n half-spaces describe
it. Conversely, consider the L1 -cube: {x <n : kxk1 1} requires only
2n points, but 2n half-spaces.
7. Let C be a closed convex set. Is every extreme point of C also a vertex of
C? Prove or provide a counter-example.
4

Solution: It is interesting to note that when the definitions of vertices and

extreme points are generalized for all convex sets - not just polyhedra - a
point could possibly be extreme but not be a vertex. One set of examples
are the points on an oval where the line segments of the sides meet the
curves of the ends.

8. Suppose S is an arbitrary subset of <n . There are two plausible ways to

define the convex hull conv(S) of S, by looking at convex combinations of
elements of S or by looking at convex sets containing S. Prove that the
set of all convex combinations of elements of S is equal to the intersection
of all convex sets C <n with S C (and this is defined to be the convex
hull of S).
Solution: Let C be the collection of convex sets C <n such that S C.
Let X be the set of all convex combinations of elements of S and Y =
CC .
(X Y ) Let x X, then x is a convex combination of elements of S, and
since S C for any C C, x is a convex combination of elements of C,
and x C, C C, and therefore x Y .
(Y X) X is a convex set containing S, therefore X C. Y is an
intersection of all the sets C C, therefore is a subset of all of them, and
of X in particular.
9. Show that, if x <n is a convex combination of v1 , v2 , . . . , vk , then it is
also a convex combination of at most n + 1 of these points.
Solution: This is the famous Caratheodory Theorem. (Who is this great
mathematician? Wiki: Constantin Caratheodory!)
By definition, x <n is a convex combination of vectors v1 , v2 , . . . , vk if
and only if <k such that
x=

k
X
j=1

j vj ,

k
X

j = 1, j 0, j = 1, . . . , k.

j=1

Let V := [v1 , . . . , vk ] be the matrix whose columns are vectors vj and let
vi.T be the ith row of V . Then, we can rewrite these requirements as a set
of n + 1 equality constraints and k nonnegativity constraints in terms of
the weight vector as follows:
vi.T
T

= xi ,

eTi

0,
5

i,

1,
, i.

Here we denote a vector of all ones as e and the ith standard vector in <k
as ei . The set of all satisfying this system is a nonempty polyhedron (we
know one feasible solution exists since x is a convex combination of these
points). It is pointed because 0. From the Fundamental Theorem
of LP we proved in class, we know that since there is a feasible solution,
there must be a basic feasible solution as well. Let this be 0 . Since it is a
basic feasible solution, Rank= (0 ) = k and 0 satisfies the n + 1 equality
constraints. In order to have rank k, it must satisfy at least k (n + 1)
inequality constraints as equality, i.e., at most (n+1) of its constraints can
be nonzero. This means that x can also be written as a convex combination
of at most n + 1 of the vectors v1 , . . . , vk , corresponding to the nonzero
components of .
10. (a) Prove that the recession cone of a nonempty convex set C, defined
as
rec(C) := {d <m : x C, 0, x + d C},
is a nonempty convex cone.
(b) Let Q := {y <m : ATx y cx , ATw y = cw } be nonempty, then show
that
rec(Q) = {d <m : ATx d 0, ATw d = 0}.
(c) Prove the representation theorem for pointed polyhedra:
Theorem [Representation of Pointed Polyhedra] Let Q :=
{y <m : ATx y cx , ATw y = cw } be a nonempty pointed polyhedron. Also let P be the set of all convex combinations of its vertices
and K be its recession cone. Then,
Q = P + K := {p + k : p P, d K}.
Solution:
(a) Proposition 1: The recession cone of a nonempty convex set C is
a nonempty convex cone.
Proof: Let d1 , d2 rec(C), and 1 , 2 0. We want to show that
1 d1 + 2 d2 rec(C). For any x C and any 0, let
x
:= x + (1 d1 + 2 d2 ) = [x + (1 )d1 ] + (2 )d2 .
The quantity in brackets lies in C since 1 0 and d1 rec(C).
Then the vector x
also lies in C since 2 0 and d2 rec(C). This
proves that the recession cone is closed under conic combinations,
i.e., it is a cone. Also 0 rec(C) by definition.
(b) Proposition 2: If Q is non-empty, then K = D, where D := {d
<m : ATx d 0, ATw d = 0} and K := rec(Q).
Proof:
(D K) Let d D, then ATx d 0 and ATw d = 0. Then for any
y Q and 0, we have
ATx (y + d) cx and ATw (y + d) = cw ,

which shows that y + d Q for all d 0, or equivalently d K.

(K D) Let d K and choose any y Q. Then 0,
ATx (y + d) = ATx y + ATx d cx ,
this can only happen when ATx d 0, otherwise the inequality would
fail for large enough . Similarly,
ATw (y + d) = ATw y + ATw d = cw ,
can only hold for all , when ATw d 0.
(c) Theorem [Representation of Pointed Polyhedra] Let Q :=
{y <m : ATx y cx , ATw y = cw } be a nonempty pointed polyhedron. Also let P be the set of all convex combinations of its vertices
and K be its recession cone. Then,
Q = P + K := {p + k : p P, d K}.
Proof:
(P + K Q) Every vertex of Q satisfies all linear constraints of Q
so does any p P . Any p + d P + K has
ATx (p + d) = ATx p + ATx d cx + 0 cx ;
ATw (p + d) = ATw p + ATw d cw + 0 = cw .
(P + K Q)
The proof is by induction on rank= (y). We have rank= (y) = m if
and only if y is itself a vertex of Q and d = 0 rec(C).
Suppose the hypothesis is true for all y Q s.t. rank= (y) > m k
for some k > 0 and consider y Q with rank= (y) = m k. Choose
0 6= d <m with {aTj d = 0, j I(y)} and consider y + d, <.
Since Q is pointed, there are three cases to consider:
(Case 1:) is bounded above and below, then L < 0 and U > 0
such that for any L U y + d Q but not otherwise. As in
the proof of the representation of bounded polyhedra theorem,
L
U
(y + L d) +
(y + U d),
y=
U L
U L
both (y + U d) and (y + L d) have rank= (y) > m k, so
y

=
=

U
L
(pL + dL ) +
(pU + dU )
U L
U L
L
U
L
U
pL +
pU ] + [
dL +
dU ].
[
U L
U L
U L
U L

The vector in first brackets is in P and the second is in K.

(Case 2:) is bounded below but not above. Then d K and
y = [y + L d] + ()d, with L defined as before. Again, the first
vector in brackets lies in P from the inductive hypothesis, so we have
y = (pL + dL ) + (d) = pL + (dL + ()d)
lies in P + K.
(Case 3:) is bounded above but not below. Then we can switch d
to d and U to L . and get back to Case 2.
7

11. Let Q be a nonempty pointed polyhedron in <n , and let f : <n < be
concave: for all x, y <n and all 0 1,
f ((1 )x + y) (1 )f (x) + f (y).
Suppose inf{f (x) : x Q} is attained; show that it is attained by at least
one of the vertices of Q.
Solution: First we need to extend the definition of concavity to the convex
combinations of nay number of points:
Claim 1: Let f beP
concave, x1 , . . . , xk <n , and 1 , . . . , k < such
k
that i 0, i, and i=1 i = 1. Then,
k
k
X
X
f(
i xi )
i f (xi ).
i=1

i=1

Proof: By induction on k. For k = 2, the claim is the definition of

concavity, so it is true. Assume it is true for all j < k, k 3. Then
choose
x1 , . . . , xk <n , and 1 , . . . , k < such that i 0, i, and
Pk
i=1 i = 1. And without loss of generality assume that := xk < 1.
Then since f is concave,
!
k1
k
X i
X
xi + xk
i xi ) = f (1 )
f(
1
i=1
i=1
!
k1
X i
xi + f (xk ).
(1 )f
1
i=1
1
Since 1
0, for all i = 1 . . . , k 1 and
induction hypothesis gives:

k
X
i xi )
f(

(1 )

i=1

k1
X
i=1

k
X

Pk1

i
i=1 1

1
1

= 1, the

i
f (xi ) + f (xk )
1

i f (xi ).

i=1

Claim 2: If inf{f (x) : x Q} is attained; it is attained by at least one

of the vertices of Q.
Proof: We can write Q = P + K (P and K are defined as in Question
10). Suppose x arg inf{f (x) : x Q}. We can write x = p + d for
some p P and d K. We can write x = 1/2(p) + 1/2(p + 2d), therefore
f (x ) 1/2f (p) + 1/2f (p + d).
Also since p, p + 2d Q and x is the infimum, we have f (x ) f (p)
and f (x ) f (p + 2d). These inequalities can hold altogether only if
f (x ) = f (p) = f (p + 2d). In particular, p attains the infimum and

P
P
p =
i i vi , i 0,
i i = 1 , where vi s are vertices of P . From
concavity of f and Claim 1, we have:
X
f (p)
i f (vi ).
i

We also have f (p) f (vi ), i. This can only happen if i > 0 implies that
f (p) = f (vi ). There is at least one such vertex and therefore there is one
vertex which attains the infimum.