PSO Mini Tutorial

Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Particle Swarm
optimisation: A mini
tutorial
2002-04-
24
Maurice.Clerc@Write
Me.com
The inventors (1)
Russell
Eberhart
eberhart@engr.iupui.edu
2002-04-
24
Maurice.Clerc@Write
Me.com
The inventors (2)
James
Kennedy
Kennedy_Jim@bls.gov
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 1: United we
stand
2002-04-
24
Maurice.Clerc@Write
Me.com
Cooperation example
2002-04-
24
Maurice.Clerc@Write
Me.com
Initialization.
Positions and
velocities
2002-04-
24
Maurice.Clerc@Write
Me.com
Neighbourhoods
geograp
hical
socia
l
2002-04-
24
Maurice.Clerc@Write
Me.com
The circular
neighbourhood
Virtual circle
1
5
7
6
4
3
8
2
Particle 1s
3-
neighbourh
ood
2002-04-
24
Maurice.Clerc@Write
Me.com
Psychosocial
compromise
Here I
am!
The best
perf. of my
neighbour
s
My
best
perf.
x
p
g
p
i
v
i
-
p
r
o
x
i
m
i
t
y
g
-
p
r
o
x
i
m
i
t
y
2002-04-
24
Maurice.Clerc@Write
Me.com
The historical
algorithm
( )
( )
( ) ( ) ( )
( ) ( ) ( )
'
+
+
+
t x p rand
t x p rand
t v
t v
d d g
d d i
d
d
, 2
, 1
, 0
, 0
1

for each particle

update
the
velocity
( ) ( ) 1 ) 1 ( + + + t v t x t x
then move
for each component d
At each time step t
R
a
n
d
o
m
n
e
s
s

in
s
id
e

t
h
e

lo
o
p
2002-04-
24
Maurice.Clerc@Write
Me.com
Random proximity
x
p
g
p
i
v
i
-
p
r
o
x
i
m
i
t
y
g
-
p
r
o
x
i
m
i
t
y
Hyperparallelepiped => Biased
2002-04-
24
Maurice.Clerc@Write
Me.com
Animated illustration
Globa
l
optim
um
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 2: How to choose
parameters
T
h
e
rig
h
t w
a
y
T
h
is w
a
y
O
r th
is w
ay
2002-04-
24
Maurice.Clerc@Write
Me.com
Type 1 form
2 1 2 1
' ' ) , 0 ( ) , 0 ( + + rand rand
2 1
2 1
' '
' '

+
+
g i
p p
p
'
+ + +
+ +
) ( ) 1 ( ) 1 (
))) ( ( ) ( ( ) 1 (
t x t v t x
t x p t v t v
with
'
>
+

4 for
4 2
2
2
else
Usual values:
=1
=4.1
=> =0.73
swarm size=20
hood size=3
N
o
n

d
i
v
e
r
g
e
n
c
e

c
r
i
t
e
r
i
o
n
G
lo
b
a
l
c
o
n
s
t
r
ic
t
io
n

c
o
e
f
f
ic
ie
n
t
2002-04-
24
Maurice.Clerc@Write
Me.com
5D complex space
4
2
0
-2
-4
2
1
0
-1
-2
6
5
4
3
2
1
0
}
}
Convergence
Non
diverg
ence
A 3D section
Re(y)
Re(v)
2002-04-
24
Maurice.Clerc@Write
Me.com
Move in a 2D section
(attractor)
1
0.5 0 -0.5 -1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
Im( v)
Re(v)
2002-04-
24
Maurice.Clerc@Write
Me.com
Some functions ...
Rosenbrock
Griewank
Rastrigin
2002-04-
24
Maurice.Clerc@Write
Me.com
... and some results
30D function PSO Type 1" Evolutionary
algo.(Angeline 98)
Griewank [300] 0.003944 0.4033
Rastrigin [5] 82.95618 46.4689
Rosenbrock [10] 50.193877 1610.359
Optimum=0, dimension=30
Best result after 40 000 evaluations
2002-04-
24
Maurice.Clerc@Write
Me.com
Beat the swarm!
Y
o
u
r

c
u
r
r
e
n
t

p
o
s
i
t
i
o
n
Y
o
u
r

b
e
s
t

p
e
r
f
.
B
e
s
t

p
e
r
f
.

o
f

t
h
e

s
w
a
r
m
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 3: Beyond real
numbers
0
1
2
3
4
0
1
2
3
0
1
2
3
4
1
2
3
4
5
6
0
1
2
0
1
2
3
4
8
8
Bingo!
2002-04-
24
Maurice.Clerc@Write
Me.com
Minimun
requirements
( ) ( ) ( ) ( ) ( ) ( ) ' ' , ) ' , ( x f x f x f x f H H x x <
( )
( )
( )
( ) position velocity position
velocity position position
velocity velocity velocity
velocity velocity t coefficien

,
,
,
,
Comparing
positions in the
search space H
Algebraic operators
2002-04-
24
Maurice.Clerc@Write
Me.com
velocity = pos_minus_pos(position
1
, position
2
)
velocity = linear_combin(,velocity
1
,,velocity
2
)
position = pos_plus_vel(position, velocity)
(position,velocity) = confinement(position
t+1
,position
t
)
Pseudo code form
}
algebr
aic
operat
ors
=>
2002-04-
24
Maurice.Clerc@Write
Me.com
Fifty-fifty

x
i
1...N
{ }
i j x
i
x
j
x
i
x
i
D/2+1
D
1
D /2

'

N=100, D=20. Search space: [1,N]
D
105 evaluations:
63+90+16+54+71+20+23+60+38+15
=
12+48+13+51+36+42+86+26+57+79 (=450)
g
r
a
n
u
l
a
r
i
t
y
=
1
2002-04-
24
Maurice.Clerc@Write
Me.com
Knapsack

x
i
1...N
{ }
i j x
i
x
j
x
i
S
iI , I D, I 1, N
{ }

'

N=100, D=10, S=100,
870 evaluations:
run 1 => (9, 14, 18, 1, 16, 5, 6, 2, 12,
17)
run 2 => (29, 3, 16, 4, 1, 2, 6, 8, 26, 5)
g
r
a
n
u
l
a
r
i
t
y
=
1
2002-04-
24
Maurice.Clerc@Write
Me.com
Graph Coloring
Problem
1
4 5
2
1
1
5
5
3
2
2
1
1
5
5
5
0
2
0
-
1
4
-3
-1
-1
+
=
p
o
s
-
p
l
u
s
-
v
e
l
2002-04-
24
Maurice.Clerc@Write
Me.com
The Tireless Traveller
Example of position: X=(5,3,4,1,2,6)
Example of velocity: v=((5,3),(2,5),(3,1))
2002-04-
24
Maurice.Clerc@Write
Me.com
n
1
n
3
n
2
Apple trees
( ) ( )
2
3 2
2
2 1
n n n n f +
Evaluation n1 n2 n3
0
1
2
3
Swarm size=3
Best position
7 7 6
3 11 6
6 4 10
3 0 17
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 4: Some variants
2002-04-
24
Maurice.Clerc@Write
Me.com
0
0.2
0.4
0.6
0.8
1
1.2
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Unbiased random
proximity
x
p
g
p
i
v
i
-
p
r
o
x
i
m
i
t
y
g
-
p
r
o
x
i
m
i
t
y
Hyperparallelepiped => Biased
Dimension
V
o
l
u
m
e
Hypersphere
vs hypercube
2002-04-
24
Maurice.Clerc@Write
Me.com
Clusters and queens
Each particle is
weighted by its
perf.
Dynamic clustering
Centroids =
queens =
temporary
new
particles
2002-04-
24
Maurice.Clerc@Write
Me.com
Think locally, act
locally (Adaptive
versions)
2002-04-
24
Maurice.Clerc@Write
Me.com
Adaptive swarm size
There has been enough
improvement
but there has been not enough
improvement
although I'm the worst
I'm the best
I try to kill myself
I try to generate a
new particle
2002-04-
24
Maurice.Clerc@Write
Me.com
Adaptive coefficients
The better I
am, the more I
follow my own
way
The better is my best
neighbour, the more
I tend to go towards
him
v
rand(0b)(p-x)
2002-04-
24
Maurice.Clerc@Write
Me.com
Energies: classical
process
Rosenbrock 2D. Swarm size=20, constant coefficients .
0,00E+00
1,00E+03
2,00E+03
3,00E+03
4,00E+03
5,00E+03
6,00E+03
7,00E+03
8,00E+03
9,00E+03
1,00E+04
Number of evaluations
0
5
10
15
20
25
Potential energy
Kinetic energy
Swarm size
=0.00001
after 2240 evaluations
Rosenbrock 2D. Swarm size=20, constant coefficients
2002-04-
24
Maurice.Clerc@Write
Me.com
Energies: adaptive
process
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients .
0,00E+00
1,00E+03
2,00E+03
3,00E+03
4,00E+03
5,00E+03
6,00E+03
7,00E+03
8,00E+03
9,00E+03
1,00E+04
Number of evaluations
0
5
10
15
20
25
Potential energy
Kinetic energy
Swarm size
=0.00001
after 1414 evaluations
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 5: Real
applications (hybrid)
Medical diagnosis
Industrial mixer
Electrical generator Electrical vehicle
2002-04-
24
Maurice.Clerc@Write
Me.com
Real applications
(stand alone)
Cockshott A. R., Hartman B. E., "Improving the fermentation medium for
Echinocandin B production. Part II: Particle swarm optimization", Process
biochemistry, vol. 36, 2001, p. 661-669.
He Z., Wei C., Yang L., Gao X., Yao S., Eberhart R. C., Shi Y., "Extracting
Rules from Fuzzy Neural Network by Particle Swarm Optimization", IEEE
International Conference on Evolutionary Computation, Anchorage, Alaska,
USA, 1998.
Secrest B. R., Traveling Salesman Problem for Surveillance Mission using
Particle Swarm Optimization, AFIT/GCE/ENG/01M-03, Air Force Institute of
Technology, 2001.
Yoshida H., Kawata K., Fukuyama Y., "A Particle Swarm Optimization for
Reactive Power and Voltage Control considering Voltage Security
Assessment", IEEE Trans. on Power Systems, vol. 15, 2001, p. 1232-1239.
2002-04-
24
Maurice.Clerc@Write
Me.com
To know more
Clerc M., Kennedy J., "The Particle Swarm-Explosion,
Stability, and Convergence in a Multidimensional
Somplex space", IEEE Transaction on Evolutionary
Computation, 2002,vol. 6, p. 58-73.
Clerc M., "L'optimisation par essaim particulaire.
Principes et pratique", Herms, Techniques et
Science de l'Informatique, 2002.
Particle Swarm Central, http://www.particleswarm.net
THE site:
Self advert
2002-04-
24
Maurice.Clerc@Write
Me.com
Appendix
2002-04-
24
Maurice.Clerc@Write
Me.com
Canonical form
( ) ( ) ( ) ( )
( ) ( ) ( )
'
+ + +
+ +
1 1
1
t v t x t x
t x p t v t v
( ) ( ) t x p t y
v t +1 ( ) v t ( )+y t ( )
y t +1 ( ) v t ( )+ ( )y t ( )

'

( )
( )
( )
( )
1
]
1
1
]
1
1
]
1
+
+
t y
t v
t y
t v
1 1
1
1
1
M
Eigen values e
1
and e
2
2002-04-
24
Maurice.Clerc@Write
Me.com
Constriction
( ) ( ) ( )
( ) ( ) ( )
'
+
+ + +
+
+ + + +
4 2
2 2
4 2
2 2
2
2 2
2
2
2 2
1
Constriction coefficients
2002-04-
24
Maurice.Clerc@Write
Me.com
Convergence
criterion
6 5 4 3 2 1
3.5
3
2.5
2
1.5
1
0.5
1
e
1
<1
2
e
2
<1

'

1
<1
2
e
2
<1

'

2
e
2
2002-04-
24
Maurice.Clerc@Write
Me.com
Magic Square (1)
1
1
1
]
1
D D D
j i
D
m m
m
m m
, 1 ,
,
, 1
1 , 1

( )
{ }
'
+
l k j i
j i
D i j
j i
m m
N m
x m
, ,
,
1
,
1
( )
( )
0
1
1
2
1
1 , ,
1
1
2
1
, 1 ,
,
_
,
_

+
D
j
D
i
j i j i
D
i
D
j
j i j i
m m
m m
2002-04-
24
Maurice.Clerc@Write
Me.com
Magic Square (2)
D=3x3, N=100
10 runs
13430 evaluations
10 solutions
55 30 68
42 49 62
56 74 23
30 61 53
89 32 23
25 51 68
27 96 39
73 40 49
62 26 74
22 70 58
40 75 35
88 5 57
50 43 67
58 55 47
52 62 4
43 51 78
75 33 64
54 88 30
50 65 68
69 42 72
64 76 43
18 25 59
32 53 17
52 24 26
80 3 30
22 72 19
11 38 64
65 28 64
63 55 39
29 74 54
2002-04-
24
Maurice.Clerc@Write
Me.com
Non linear system
( )
'

+
0 10 sin
0 1
2 1
2
2
2
1
x x
x x
Search space
[0,1]
2
1 run
143 evaluations
1 solution
10 runs
1430 evaluations
3 solutions
2002-04-
24
Maurice.Clerc@Write
Me.com

PSO Mini Tutorial

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

PSO Mini Tutorial

Hochgeladen von

Copyright:

Verfügbare Formate

Particle Swarm optimisation

for each particle

Das könnte Ihnen auch gefallen