Beruflich Dokumente
Kultur Dokumente
T accept
e
P
33
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration 1
Randomly generate a new point according to probability distribution and
current temperature
3
x
y
0.9
Temperature = 1
0.3
34
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration 1
Randomly generate a new point according to probability distribution and
current temperature, accept new point if lower value
3
x
y
0.9
Temperature = 1
0.3
35
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration 2
Lower temperature according to temperature schedule
3
x
y
0.9
Temperature = 1
0.3
36
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration 2
Lower temperature according to temperature schedule and generate new
point
3
x
y
0.9
Temperature = 0.75
0.3
-1.2
37
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration N-1
After several iterations, the search radius becomes small and we narrow
in on a local solution
3
x
y
0.9
Temperature = 0.1
0.3
-1.2
-3
38
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration N
Reset temperature and start the process again (reanneling)
3
x
y
0.9
Temperature = 1
0.3
-1.2
-3
-2
39
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration N
Reset temperature and start the process again (reannealing)
3
x
y
0.9
Temperature = 1
0.3
-1.2
-3
-2
27 . 0
1
1
/ )) 3 ( 2 (
T accept
e
P
40
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration N
Reset temperature and start the process again (reannealing)
3
x
y
0.9
Temperature = 1
0.3
-1.2
-3
-2
41
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration N+1
Reduce temperature and continue
3
x
y
0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
42
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration N+1
Reduce temperature and continue
3
x
y
0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
43
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Simulated Annealing Overview Iteration
Reduce temperature and continue
3
x
y
0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
-6.5
47
PATTERN SEARCH
(DIRECT SEARCH)
48
What is a Pattern Search?
An approach that uses a pattern of search directions around
the existing points, the mesh
Polls the mesh for a better solution and moves to that point
Expands/contracts the mesh around the current point when a
solution is not found
Does not rely on gradient information
49
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Pattern Search Overview Iteration 1
Run from specified x0
x
y
3
50
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Pattern Search Overview Iteration 1
Apply pattern vector, poll new points for improvement
x
y
3
Mesh size = 1
Pattern vectors = [1,0], [0,1], [-1,0], [-1,-1]
0
_ * _ x vector pattern size mesh Pnew
0
] 0 , 1 [ * 1 x 1.6
0.4
4.6
2.8
First poll successful
Complete Poll (not default)
51
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Pattern Search Overview Iteration 2
Increase mesh size and repeat
x
y
3
Mesh size = 2
Pattern vectors = [1,0], [0,1], [-1,0], [-1,-1]
1.6
0.4
4.6
2.8
-4
0.3 -2.8
Complete Poll
52
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Pattern Search Overview Iteration 3
Mesh expansion: increase mesh size and repeat
x
y
3
Mesh size = 4
Pattern vectors = [1,0], [0,1], [-1,0], [-1,-1]
1.6
0.4
4.6
2.8
-4
0.3 -2.8
53
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Pattern Search Overview Iteration 4
Refine mesh: decrease mesh size and repeat
x
y
3
Mesh size = 4*0.5 = 2
Pattern vectors = [1,0], [0,1], [-1,0], [-1,-1]
1.6
0.4
4.6
2.8
-4
0.3 -2.8
54
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Pattern Search Overview Iteration N
Continue expansion/contraction until convergence
x
y
3
1.6
0.4
4.6
2.8
-4
0.3 -2.8
-6.5
57
GENETIC ALGORITHM
58
What is a Genetic Algorithm?
Genetic Algorithms use concepts from evolutionary biology to
find exact or approximate solutions to optimization problems
Start with an initial generation of candidate solutions that are
tested against the objective function
Subsequent generations evolve from the 1
st
through selection, crossover and mutation
The individual that best minimizes the given
objective is returned as the ideal solution
59
How Evolution Works Binary Case
Selection
Retain the best performing bit strings from one generation to the
next. Favor these for reproduction
parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
Crossover
parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
child = [ 1 0 0 0 0 1 1 0 1 0 ]
Mutation
parent = [ 1 0 1 0 0 1 1 0 0 0 ]
child = [ 0 1 0 1 0 1 0 0 0 1 ]
60
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Genetic Algorithm Iteration 1
Evaluate initial population
x
y
61
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Genetic Algorithm Iteration 1
Select a few good solutions for reproduction
x
y
62
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Genetic Algorithm Iteration 2
Generate new population and evaluate
x
y
63
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Genetic Algorithm Iteration 2
Select a few good solutions for reproduction
x
y
64
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Genetic Algorithm Iteration 3
Generate new population and evaluate
x
y
65
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Genetic Algorithm Iteration 3
Select a few good solutions for reproduction
x
y
66
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Genetic Algorithm Iteration N
Continue process until stopping criteria are met
x
y
Solution found
69
Comparison of Solver (Default) Performance
x
y
rf([x,y])
-6 -4 -2 0 2 4 6
-6
-4
-2
0
2
4
6
FuncValue
Min
StartPt
x
y
rf([x,y])
-60 -40 -20 0 20 40 60
-60
-40
-20
0
20
40
60
FuncValue
Min
StartPt
x
y
rf([x,y])
-40 -20 0 20 40 60
-40
-20
0
20
40
60
FuncValue
Min
StartPt
70
71
Additional Resources
Upcoming Webinars
Speeding Up Optimization with Parallel Computing
(August 2010)
On-demand Webinars
Genetic Algorithm in Financial Applications
Tips & Tricks: Getting Started with Optimization
Introduction to Optimization
72
Contact Information
North America
Phone: 508-647-7000
E-mail: support@mathworks.com
Outside North America
Contact your local MathWorks office or reseller:
www.mathworks.com/contact
73
Questions?
74
MATLAB Provides the Foundation for
Optimization
The leading environment for
technical computing
Customizable
Numeric computation
Data analysis and visualization
The de facto industry-standard,
high-level programming language
for algorithm development
Toolboxes for statistics, optimization,
symbolic math, signal and image
processing, and other areas
Foundation of the
MathWorks product family
75
Optimization Toolbox
Solve standard and large-scale optimization problems
Graphical user interface and command
line functions for:
Linear and nonlinear programming
Quadratic programming
Nonlinear least squares and nonlinear
equations
Multi-objective optimization
Binary integer programming
Additional Capabilities:
Parallel computing support in selected
solvers
Customizable algorithm options
Choose between standard and large-
scale algorithms
Output diagnostics
76
Global Optimization Toolbox
Solve multiple maxima, multiple minima, and nonsmooth optimization problems
Graphical user interface and command
line functions for:
Global Search solver
Multistart solver
Genetic algorithm solver
Single objective
Multiobjective with Pareto front
Direct search solver
Simulated annealing solver
Useful for problems not easily addressed
with Optimization Toolbox:
Discontinuous
Highly nonlinear
Stochastic
Discrete or custom data types
Undefined derivatives
Multiple maxima/minima
77
Anatomy of an Optimization Problem
General Form Accepted by MATLAB Solvers
Objective Function
Subject to Constraints (i.e. such that)
u x l
x c b x A
x c b Ax
eq eq eq
0 ) ( ,
0 ) ( ,
) ( min x f
x
Typically a linear or nonlinear function
Linear constraints
inequalities
equalities
bounds
Nonlinear constraints
inequalities
equalities
Decision variables
78
MATLAB Optimization Products and Example
Applications
MATLAB Statistics Toolbox
Curve Fitting Toolbox
Optimization Toolbox Genetic Algorithm and
Direct Search Toolbox
Solving Equations
f(x) = 0
Root finding
Systems of equations
Real roots finding (1D):
Linear Systems:
F(x) = 0
Ax-b = 0 (i.e. Ax=b)
>> x = A \ b
Real roots finding (N-D):
Nonlinear Systems
F(X) = 0
Noisy, discontinuous root
finding (N-D)
Noisy, discontinuous systems:
F(X) = 0
Curve/Modeling
Fitting
curve fitting
parameter estimation
(model fitting)
Basic (linear) curve fitting Advanced (nonlinear) curve
fitting
Model Fitting (least squares)
Constrained curve fitting
Parameter Estimation
Noisy, Discontinuous
parameter estimation
Trade-Off Studies
Maximization
Minimization
Goal seeking
Multiobjective
Unconstrained nonlinear
minimization
Constrained nonlinear
minimization
Noisy, Discontinuous, ill-
defined mimization
-6 -4 -2 0 2 4 6
-100
-50
0
50
100
x
x
3
-2 x-5
0 0.5 1 1.5 2 2.5
0.5
0.6
0.7
0.8
0.9