Beruflich Dokumente
Kultur Dokumente
A quick introduction
Giuseppe Narzisi
Courant Institute of Mathematical Sciences
New York University
24 January 2008
Outline
1
Introduction
Motivations
Definition
Notion of optimum
Pareto-Optimality
Non-dominated set
Globally and Locally Pareto-optimal sets
Optimality Conditions
Motivation
Scope of Optimization
in practice
What is a MOOP?
In words
What is a MOOP?
Formal definition
min
F(x) = [f1 (x), f2 (x), . . . , fM (x)]
subject to G(x) = [g1 (x), g2 (x), . . . , gJ (x)] 0
H(x) = [h1 (x), h2 (x), . . . , hK (x)] = 0
(U)
(L)
xi xi xi , i = 1, . . . , N
(1)
(U)
Edgeworth-Pareto solution
Notion of optimum for MOOPs
Definition
A function f : Rn R is a convex function if for any two pair of
solutions x1 , x2 Rn , the following condition is true:
f (x1 + (1 )x2 ) f (x1 ) + (1 )f (x2 ),
(2)
for all 0 1
Since a MOOP has two spaces, the convexity must be
analyzed on both spaces.
Moreover, although the search space can be non-convex,
the Pareto optimal front can be convex.
I DEAL PROCEDURE:
1
Definition
The m-th component of the ideal objective vector z is the
constrained minimum of the following problem:
min
fm (x)
subject to x S
(3)
(4)
Definition
A Utopian objective vector z is a vector whose components
are slightly smaller than that of the ideal objective vector:
zi = zi i with i > 0 for all i 1, 2, . . . , M
Definition
The m-th component of the nadir objective vector znad is the
constrained maximum of the following problem:
max
fm (x)
subject to x P
(5)
fi zi
zinad zi
(6)
max
(f1 ,f2 ) w
f1
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
nad
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
* 11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
1 11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
*
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
*
2
**
Feasible space
f2
Dominance Relation
Definition
A solution x1 is said to dominate another solution x2 , and we
write x1 x2 , if both the following conditions are true:
1
(7)
Dominance Relation
Example
Dominance Relation
Properties
R EFLEXIVE
The dominance relation is not reflexive since any solution x
does not dominate itself (by definition of dominance).
S YMMETRIC
The dominance relation is not symmetric because x y
does not imply y x. In fact the opposite is true, if x y
then y x
A NTISYMMETRIC
Since the dominance relation is not symmetric it cannot be
antisymmetric as well.
T RANSITIVE
The dominance relation is transitive. If x y and y z than
x z.
Dominance Relation
Transitivity
x y iff
y z iff
fi (x) fi (y)
i 1, . . . , M
j 1, . . . , M fj (x) < fj (y)
(8)
fi (y) fi (z)
i 1, . . . , M
k 1, . . . , M fk (y) < fk (z)
(9)
xz
fi (x) fi (y) fi (z) i 1, . . . , M
l 1, . . . , M fl (x) < fl (z)
if k = j then fj (x) < fj (y) < fj (z)
if k 6= j then
if we chose j: fj (x) < fj (y) fj (z)
if we chose k : fk (x) fk (y) < fk (z)
Dominance Relation
More properties
Non-dominated set
f2
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
00000000000000000000000000000000000000000
11111111111111111111111111111111111111111
f1
x2
000000000000000000000000000000000000
111111111111111111111111111111111111
111111111111111111111111111111111111
000000000000000000000000000000000000
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
000000000000000000000000000000000000
111111111111111111111111111111111111
x1
Motivation
Approach 1
Naive and slow
Non-dominated set (V. 1)
1. i := 1
2. P 0 :=
3. while (i |P|)
4. j := 1
5. while (j |P|)
6.
if (j 6= i)
7.
if (xj xi ) continue
8.
else j := j + 1
9. end_while
10. if (j = |P|) P 0 := P 0 {i}
11. i := i + 1
12. end_while
Approach 2
Continuously updated
x2 compared with x1
x3 compared at most with x1 , x2
and so on...
This requires in the worst case:
1 + 2 + + (N 1) = N(N 1)/2
domination checks.
Although the total complexity is still
O(MN 2 ), this method requires
typically half of that required by
approach 1.
Approach 3
Kung et al.s efficient method (1975)
Front(P)
1. Sort1 (P)
2. if (|P| = 1)
3. then return(P)
4. else
5. T := Front([P1 , Pb|P|/2c ])
6. B := Front([Pb|P|/2c+1 , P|P| ])
7. i := 1, M :=
8. while (i |B|)
9.
j := 1
10.
while (j |T |)
11.
if (xj xi )
12.
then j := j + 1
13.
else continue
14.
end_while
15. if (j = |T |) M := M {i}
16. i := i + 1
17. end_while
18. return(T M)
Non-Dominated Sorting
1. for j = 1, 2, . . .
2. Pj =
4. end_for
3. j := 1
5. while (P =
6 )
6. P 0 := NDS(P)
7. Pj := P 0
8. P := P \ P 0
9. j := j + 1
10.end_while
11.return(Pi , i = 1, 2, . . . , j)
Complexity is O(MN 2 ):
O(MN 2 ) for computing ni and Si
O(N 2 ) for computing all the fronts
Pk , k = 1, 2, . . . (non-domination
levels).
Note that each solution will be visited at most N 1 times
before its domination count become zero and will never be
visited again. There are at most N 1 such solutions.
Theorem
A necessary condition for a solution x to be Pareto-optimal is
that there exist vectors > 0 and u 0 where ( RM , u RJ
and , u 6= 0) such that the following conditions are true:
PM
PJ
m=1 m 5 fm (x )
j=1 uj
uj gj (x ) = 0, j = 1, 2, . . . , j
5 gj (x ) = 0, and
(10)
Optimality conditions
special cases
5fm (x ) = 0
(12)
m=1
Theorem
Let the objective functions be convex and the constraint
functions non-convex. Let the objective and constraint functions
be differentiable at solution x . A sufficient condition for x to be
Pareto-optimal is that there exist vectors > 0 and u 0 where
( RM and u RJ ) such that the following conditions are true:
PM
PJ
m=1 m 5 fm (x )
j=1 uj
uj gj (x ) = 0, j = 1, 2, . . . , j
5 gj (x ) = 0, and
(14)
Kalyanmoy Deb
Multi-Objective Optimization Using Evolutionary
Algorithms.
John Wiley & Sons, Inc, New York, NY, USA, 2001.
Kung, H. T., Luccio, F., and Preparata, F. P. 1975.
On Finding the Maxima of a Set of Vectors.
J. ACM 22, 4 (Oct. 1975), 469-476.