DebKalyanmoy Slides1

Dagstuhl Seminar, 5-10 February,
2006
1
Functional Decomposition of NSGA-II and
Various Problem-Solving Strategies
Kalyanmoy Deb
Professor of Mechanical Engineering
Indian Institute of Technology Kanpur
Director, Kanpur Genetic Algorithms
Laboratory (KanGAL)
Email: deb@iitk.ac.in
http://www.iitk.ac.in/kangal/deb.htm
2006
2
Overview
Essentials of multi-objective
optimization
NSGA-II platform
Different multi-objective problem-
solving tasks
Omni-optimizer
Degeneracy to various single and multi-
objective tasks
Conclusions
2006
3
Multi-Objective Optimization:
Handling multiple conflicting objectives
We often face them
2006
4
Which Solutions are Optimal?
Relates to the concept
of domination
x
(1)
dominates x
(2)
, if
x
(1)
is no worse than x
(2)

in all objectives
x
(1)
is strictly better than
x
(2)
in at least one
objective
Examples:
3 dominates 2
3 does not dominate 5

2006
5
Pareto-Optimal Solutions
P=Non-dominated(P)
Solutions which are
not dominated by any
member of the set P
O(N log N)
algorithms exist
Pareto-Optimal set
= Non-dominated(S)
A number of solutions
are optimal
2006
6
Pareto-Optimal Fronts
Depends on
the type of
objectives
Definition of
domination
takes care of
possibilities
Always on the
boundary of
feasible region
2006
7
Local Versus Global Pareto-Optimal
Fronts
Local Pareto-optimal Front: Domination check is
restricted within a neighborhood (in decision
space) of P
2006
8
Some Terminologies
Ideal point (z*)
nonexistent, lower
bound on Pareto-
optimal set
Utopian point (z**)
nonexistent
Nadir point (z
nad
)
Upper bound on
Pareto-optimal set
Normalization:
2006
9
Differences with Single-Objective
Optimization
One optimum versus multiple optima
Requires search and decision-making
Two spaces of interest, instead of one
2006
10
Ideal Multi-Objective
Optimization
Step 1 :
Find a set of
Pareto-optimal
solutions

Step 2 :
Choose one from
the set
2006
11
Two Goals in Ideal Multi-Objective
Optimization
Converge to the
Pareto-optimal front

Maintain as diverse a
distribution as
possible

2006
12
Elitist Non-dominated Sorting
Genetic Algorithm (NSGA-II)
NSGA-II can
extract Pareto-
optimal frontier
Also find a well-
distributed set of
solutions
iSIGHT and
modeFrontier
adopted NSGA-II
Fast-Breaking Paper in Engineering by ISI Web of Science (Feb04)

2006
13
Functional Decomposition
Convergence:
Emphasize non-dominated
solutions
Diversity:
Prefer less-crowded
solutions
Elite-preservation
For ensuring convergence
properties
2006
14
An Iteration of NSGA-II
E
l
i
t
e
-
p
r
e
s
e
r
v
a
t
i
o
n

Convergence
Diversity-maintenance
2006
15
NSGA-II: Crowding Distance
Overall Complexity
O(N log
M-1
N)
Diversity is maintained
Improve diversity by
k-mean clustering
Euclidean distance
measure
Other techniques

2006
16
Simulation on ZDT1
2006
17
Simulation on ZDT3
2006
18
Changing Dominance Relation
Alter the meaning of Pareto-optimal
points
Constrained optimization (Fonseca and
Fleming, 1996, Deb et al., 2000)
Cone dominance (guided dominance,
Branke et al., 2000)
Distributed EMO (Deb et al., 2003)
Epsilon-MOEA (Laumanns et al., 2003;
Deb et al., 2005)
Robust and reliability-based EMO (Deb
and Gupta, 2005)

2006
19
Constraint-Domination Principle

1. i is feasible and j is not
2. i and j are both
infeasible, but i has a
smaller overall
constraint violation
3. i and j are feasible and i
dominates j
A solution i constraint-
dominates a solution j, if any
is true:
2006
20
Constrained NSGA-II Simulation
Results
2 2
) ( x x f =
Minimize
1 1
) ( x x f =
1
2
2
1
) (
x
x
x f
+
=
1 9
6 9
1 2
1 2
> +
> +
x x
x x
Where
( ) ( ) 5 . 0 5 . 0 5 . 0
0 tan 16 cos
10
1
1
2
2
2
1
2
1 1 2
2
2
1
s +
>
|
|
.
|
\
|
+

x x
x
x
x x
Minimize
1 1
) ( x x f =
Where
2006
21
Simulation on TNK
2006
22
Simulation on CTP5
2006
23
Cone Dominance
Using a DMs
preference (not a
solution but a region)

Guided domination
principle: Biased
niching approach

Weighted domination
approach
2006
24
Distributed Computing of
Pareto-Optimal Set
Guided domination concept to search different parts
of Pareto-optimal region
Distributed computing of different parts
2006
25
Distributed computing:
A Three-Objective Problem
Spatial computing, not temporal
Theory
NSGA-II Simulations
2006
26
-MOEA: Using -Dominance

EA and archive
populations evolve
One EA and one
archive member are
mated
Archive update using
-dominance
EA update using
usual dominance
2006
27
Comparative Study on
Three-Objective DTLZ Problems
2006
28
Test Problem DTLZ2
2006
29
Multi-Objective Robust
Solutions

Not all Pareto-
optimal points may
be robust
A is robust, but B is
not
Decision-makers will
be interested in
knowing robust part
of the front

2006
30
Domination Based on Aggregate
Functions
Functions averaged over a delta-
neighborhod

Alternate Strategy: (Type II Robustness)

2006
31
Effect of -neighborhood Size
Theory and NSGA-II simulation
Larger , more shift from original front
Some part is more sensitive than others
2006
32
Effect of -neighborhood Size
Theory and NSGA-II simulation
Larger , more shift from original front
Some part is no more robust
2006
33
Robust Front as
Partial Global and Partial Local
Theory:

For global front
2006
34
Simulation Using NSGA-II
Simulation:

2006
35
Reliability-Based Optimization
Deterministic
optimum often not
reliable
Due to
uncertainities in
decision
variables/problem
parameters
Find the reliable
solution for a
specified Reliability
2006
36
Constrained Domination for
Reliability Consideration
Chance
constraints:
P(g(x)0)
depends on
chosen reliability
Prefer reliable
solutions
Indicates how
P-O front moves
away with

2006
37
Goal Programming Using EMO
Target function
values are specified
Convert them to
objectives and
perform
domination
2006
38
Goal Programming Using EMO
Target function values
are specified
Convert them to
objectives and perform
domination check with
them
2006
39
Preferred Diversity
Find a subset of Pareto-optimal points
dictated by preference information
Biased EMO (Branke and Deb, 2005)
Reference-point based EMO (Deb and
Sundar, 2006)
Knee-based EMO (Branke et al., 2004)
Nadir point and EMO (Deb and
Chaudhuri, 2006)
Multi-modal EMO (Deb and Reddy, 2003)
Variable versus objective space niching
2006
40
Preference-Based EMO
EMO (NSGA-II) not efficient for many objectives
Large number of points needed
Domination-based methods are slow
2006
41
EMO for a Biased Distribution
Choose a hyper-
plane
Project points on it
Compute two
distances: d and d
Compute
D=d(d/d)^a
Point b has small D
Point a has large D
2006
42
Biased Distribution in NSGA-II
ZDT1
ZDT2: a=100
2006
43
Biased NSGA-II (cont.)
Three-objective Problems: a=0 and a=500
2006
44
Reference Point Based EMO
Wierzbicki, 1980
A P-O solution closer
to a reference point
Multiple runs
Too structured
Extend for EMO
Multiple reference
points in one run
A distribution of
solutions around each
reference point
2006
45
Reference Point Based EMO (cont.)
Ranking based on
closeness to each
reference point
Clearing within each
niche with
2006
46
More Results
Five-objective with
two reference
points (z
1-5
=0.5 &
z
1-4
=0.2, z
5
=0.8)
A engineering
design problem
with three
reference points
2006
47
Knee Based EMO
Find only the knee or
near-knee solutions
Knees are important
solutions
Not much motivation to
move out from knees
A large gain for a small
loss in any pair of
objectives
Non-convex front
No knee point
Extreme solutions are
attractors
2006
48
Finding Knee Solutions
Branke et al. (2004) for more details
2006
49
Nadir Point and EMO
Important for
knowing range and
normalization of
objectives
Difficult to find
using classical
method
Pay-off table
method does not
work

2006
50
EMO for Finding Nadir Point
Emphasize only extreme
points
M3 find complete
front, else use
extremized crowded
NSGA-II
2006
51
EMO for Finding Nadir Point (cont.)
DTLZ problems extended up to 20 objectives
2006
52
Multi-Modal EMOs
Different solutions having identical objective values
Multi-modal Pareto-optimal solutions: Design,
Bioinformatics
2006
53
Multiple Gene Subsets for
Leukemia Samples

Deb and Reddy
(BioSystems, 2003)

Multiple (26) four-
gene combinations
for 100%
classification

Discovery of some
common genes
2006
54
Parameter Versus Objective-space
Niching
Distribution depends on the space niching is
performed

2006
55
Redefining Elites
To aid in better diversity
Controlled Elitist EMO (Deb and Goel,
2001)
2006
56
Controlled Elitism
Keep solutions
from dominated
fronts in GP
2006
57
Controlled Elitism (cont.)
ZDT4 has many
local P-O fronts
g()=1 is global
Controlled elitism
can come closer to
global P-O front
2006
58
Omni-Optimizer:
Motivation from Computation
Multiple is a generic case, single is specific
Single objective as a degenerate case multi-
objective case
One algorithm for single and multi-objective
problem solving (Deb and Tiwari, 2005)
Accommodating NFL theorem, not violating it
Single-objective, uni-optimum problems
Single-objective, multi-optima problems
Multi-objective, uni-optimal front problems
Multi-objective, multi-optimal front problems

2006
59
Structure of Omni-optimizer
Very much like NSGA-II
Epsilon-dominance
Variable-space and
objective space niching
Use maximum
of both crowding
distances
2006
60
Single-Objective, Uni-Optimum
Dominance reduced to simple <
Epsilon-dominance to f
a
< f
b
-
Allows multiple solutions within to exist
Elite-preservation is similar to CHC and
(+)-ES
2006
61
Shinn et al.s 12 Problems
12 problems
2006
62
Single-Objective, Multi-Optima
Variable-space niching help find multiple
solutions
Weierstrass function
16 minima with f=0
2006
63
10
4
Sin
2
(x): 20 Minima
2006
64
Himmelblaus Function: 4 Minima
2006
65
Multi-Objective, Uni-Pareto front
Constrained and
unconstrained
test problems
2006
66
More Results
Comparable
performance
to existing
EMO
methods
2006
67
Multi-Objective, Multi-Optima
Nine regions leading to
the same Pareto-
optimal front
Multiple solutions
cause a single Pareto-
optimal point
2006
68
Nine Optimal Regions
O
m
n
i
-
o
p
t
i
m
i
z
e
r

N
S
G
A
-
I
I

2006
69
Nine Optimal Fronts
2006
70
Conclusions
Functional decomposition of NSGA-II
Non-domination for convergence
Niching for diverse set of solutions
Elite-preservation for reliable convergence
For a new problem-solving, find the
suitable place to change
Many different problem-solving tasks
achieved with NSGA-II
Omni-optimizer provides a holistic
approach for optimization

2006
71
Thank You for Your Attention
For further information:

http://www.iitk.ac.in/kangal
Email: deb@iitk.ac.in

Acknowledgement:
KanGAL students, staff and collaborators

DebKalyanmoy Slides1

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

DebKalyanmoy Slides1

Hochgeladen von

Copyright:

Verfügbare Formate

Dagstuhl Seminar, 5-10 February,

Das könnte Ihnen auch gefallen