Sie sind auf Seite 1von 46

Particle Swarm optimisation

2002-04-
24
Maurice.Clerc@Write
Me.com
Particle Swarm
optimisation: A mini
tutorial
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
The inventors (1)
Russell
Eberhart
eberhart@engr.iupui.edu
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
The inventors (2)
James
Kennedy
Kennedy_Jim@bls.gov
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 1: United we
stand
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Cooperation example
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Initialization.
Positions and
velocities
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Neighbourhoods
geograp
hical
socia
l
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
The circular
neighbourhood
Virtual circle
1
5
7
6
4
3
8
2
Particle 1s
3-
neighbourh
ood
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Psychosocial
compromise
Here I
am!
The best
perf. of my
neighbour
s
My
best
perf.
x
p
g
p
i
v
i
-
p
r
o
x
i
m
i
t
y
g
-
p
r
o
x
i
m
i
t
y
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
The historical
algorithm
( )
( )
( ) ( ) ( )
( ) ( ) ( )

'

+
+
+
t x p rand
t x p rand
t v
t v
d d g
d d i
d
d
, 2
, 1
, 0
, 0
1

for each particle


update
the
velocity
( ) ( ) 1 ) 1 ( + + + t v t x t x
then move
for each component d
At each time step t
R
a
n
d
o
m
n
e
s
s

in
s
id
e

t
h
e

lo
o
p
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Random proximity
x
p
g
p
i
v
i
-
p
r
o
x
i
m
i
t
y
g
-
p
r
o
x
i
m
i
t
y
Hyperparallelepiped => Biased
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Animated illustration
Globa
l
optim
um
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 2: How to choose
parameters
T
h
e
rig
h
t w
a
y
T
h
is w
a
y
O
r th
is w
ay
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Type 1 form
2 1 2 1
' ' ) , 0 ( ) , 0 ( + + rand rand
2 1
2 1
' '
' '


+
+

g i
p p
p

'

+ + +
+ +
) ( ) 1 ( ) 1 (
))) ( ( ) ( ( ) 1 (
t x t v t x
t x p t v t v
with

'

>
+

4 for
4 2
2
2

else
Usual values:
=1
=4.1
=> =0.73
swarm size=20
hood size=3
N
o
n

d
i
v
e
r
g
e
n
c
e

c
r
i
t
e
r
i
o
n
G
lo
b
a
l
c
o
n
s
t
r
ic
t
io
n

c
o
e
f
f
ic
ie
n
t
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
5D complex space
4
2
0
-2
-4
2
1
0
-1
-2
6
5
4
3
2
1
0
}
}
Convergence
Non
diverg
ence
A 3D section

Re(y)
Re(v)
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Move in a 2D section
(attractor)
1
0.5 0 -0.5 -1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
Im( v)
Re(v)
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Some functions ...
Rosenbrock
Griewank
Rastrigin
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
... and some results
30D function PSO Type 1" Evolutionary
algo.(Angeline 98)
Griewank [300] 0.003944 0.4033
Rastrigin [5] 82.95618 46.4689
Rosenbrock [10] 50.193877 1610.359
Optimum=0, dimension=30
Best result after 40 000 evaluations
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Beat the swarm!
Y
o
u
r

c
u
r
r
e
n
t

p
o
s
i
t
i
o
n
Y
o
u
r

b
e
s
t

p
e
r
f
.
B
e
s
t

p
e
r
f
.

o
f

t
h
e

s
w
a
r
m
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 3: Beyond real
numbers
0
1
2
3
4
0
1
2
3
0
1
2
3
4
1
2
3
4
5
6
0
1
2
0
1
2
3
4
8
8
Bingo!
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Minimun
requirements
( ) ( ) ( ) ( ) ( ) ( ) ' ' , ) ' , ( x f x f x f x f H H x x <
( )
( )
( )
( ) position velocity position
velocity position position
velocity velocity velocity
velocity velocity t coefficien



,
,
,
,

Comparing
positions in the
search space H
Algebraic operators
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
velocity = pos_minus_pos(position
1
, position
2
)
velocity = linear_combin(,velocity
1
,,velocity
2
)
position = pos_plus_vel(position, velocity)
(position,velocity) = confinement(position
t+1
,position
t
)
Pseudo code form
}
algebr
aic
operat
ors
=>
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Fifty-fifty

x
i
1...N
{ }
i j x
i
x
j
x
i
x
i
D/2+1
D

1
D /2


'









N=100, D=20. Search space: [1,N]
D
105 evaluations:
63+90+16+54+71+20+23+60+38+15
=
12+48+13+51+36+42+86+26+57+79 (=450)
g
r
a
n
u
l
a
r
i
t
y
=
1
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Knapsack

x
i
1...N
{ }
i j x
i
x
j
x
i
S
iI , I D, I 1, N
{ }


'









N=100, D=10, S=100,
870 evaluations:
run 1 => (9, 14, 18, 1, 16, 5, 6, 2, 12,
17)
run 2 => (29, 3, 16, 4, 1, 2, 6, 8, 26, 5)
g
r
a
n
u
l
a
r
i
t
y
=
1
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Graph Coloring
Problem
1
4 5
2
1
1
5
5
3
2
2
1
1
5
5
5
0
2
0
-
1
4
-3
-1
-1
+
=
p
o
s
-
p
l
u
s
-
v
e
l
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
The Tireless Traveller
Example of position: X=(5,3,4,1,2,6)
Example of velocity: v=((5,3),(2,5),(3,1))
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
n
1
n
3
n
2
Apple trees
( ) ( )
2
3 2
2
2 1
n n n n f +
Evaluation n1 n2 n3
0
1
2
3
Swarm size=3
Best position
7 7 6
3 11 6
6 4 10
3 0 17
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 4: Some variants
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
0
0.2
0.4
0.6
0.8
1
1.2
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Unbiased random
proximity
x
p
g
p
i
v
i
-
p
r
o
x
i
m
i
t
y
g
-
p
r
o
x
i
m
i
t
y
Hyperparallelepiped => Biased
Dimension
V
o
l
u
m
e
Hypersphere
vs hypercube
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Clusters and queens
Each particle is
weighted by its
perf.
Dynamic clustering
Centroids =
queens =
temporary
new
particles
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Think locally, act
locally (Adaptive
versions)
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Adaptive swarm size
There has been enough
improvement
but there has been not enough
improvement
although I'm the worst
I'm the best
I try to kill myself
I try to generate a
new particle
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Adaptive coefficients
The better I
am, the more I
follow my own
way
The better is my best
neighbour, the more
I tend to go towards
him
v
rand(0b)(p-x)
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Energies: classical
process
Rosenbrock 2D. Swarm size=20, constant coefficients .
0,00E+00
1,00E+03
2,00E+03
3,00E+03
4,00E+03
5,00E+03
6,00E+03
7,00E+03
8,00E+03
9,00E+03
1,00E+04
Number of evaluations
0
5
10
15
20
25
Potential energy
Kinetic energy
Swarm size
=0.00001
after 2240 evaluations
Rosenbrock 2D. Swarm size=20, constant coefficients
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Energies: adaptive
process
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients .
0,00E+00
1,00E+03
2,00E+03
3,00E+03
4,00E+03
5,00E+03
6,00E+03
7,00E+03
8,00E+03
9,00E+03
1,00E+04
Number of evaluations
0
5
10
15
20
25
Potential energy
Kinetic energy
Swarm size
=0.00001
after 1414 evaluations
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Part 5: Real
applications (hybrid)
Medical diagnosis
Industrial mixer
Electrical generator Electrical vehicle
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Real applications
(stand alone)
Cockshott A. R., Hartman B. E., "Improving the fermentation medium for
Echinocandin B production. Part II: Particle swarm optimization", Process
biochemistry, vol. 36, 2001, p. 661-669.
He Z., Wei C., Yang L., Gao X., Yao S., Eberhart R. C., Shi Y., "Extracting
Rules from Fuzzy Neural Network by Particle Swarm Optimization", IEEE
International Conference on Evolutionary Computation, Anchorage, Alaska,
USA, 1998.
Secrest B. R., Traveling Salesman Problem for Surveillance Mission using
Particle Swarm Optimization, AFIT/GCE/ENG/01M-03, Air Force Institute of
Technology, 2001.
Yoshida H., Kawata K., Fukuyama Y., "A Particle Swarm Optimization for
Reactive Power and Voltage Control considering Voltage Security
Assessment", IEEE Trans. on Power Systems, vol. 15, 2001, p. 1232-1239.
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
To know more
Clerc M., Kennedy J., "The Particle Swarm-Explosion,
Stability, and Convergence in a Multidimensional
Somplex space", IEEE Transaction on Evolutionary
Computation, 2002,vol. 6, p. 58-73.
Clerc M., "L'optimisation par essaim particulaire.
Principes et pratique", Herms, Techniques et
Science de l'Informatique, 2002.
Particle Swarm Central, http://www.particleswarm.net
THE site:
Self advert
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Appendix
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Canonical form
( ) ( ) ( ) ( )
( ) ( ) ( )

'

+ + +
+ +
1 1
1
t v t x t x
t x p t v t v
( ) ( ) t x p t y
v t +1 ( ) v t ( )+y t ( )
y t +1 ( ) v t ( )+ ( )y t ( )

'



( )
( )
( )
( )
1
]
1

1
]
1

1
]
1

+
+
t y
t v
t y
t v

1 1
1
1
1
M
Eigen values e
1
and e
2
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Constriction
( ) ( ) ( )
( ) ( ) ( )

'

+
+ + +

+
+ + + +

4 2
2 2
4 2
2 2
2
2 2
2
2
2 2
1
Constriction coefficients
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Convergence
criterion

6 5 4 3 2 1
3.5
3
2.5
2
1.5
1
0.5

1
e
1
<1

2
e
2
<1

'


1
<1

2
e
2
<1

'




2
e
2
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Magic Square (1)
1
1
1
]
1

D D D
j i
D
m m
m
m m
, 1 ,
,
, 1
1 , 1



( )
{ }

'

+
l k j i
j i
D i j
j i
m m
N m
x m
, ,
,
1
,
1
( )
( )
0
1
1
2
1
1 , ,
1
1
2
1
, 1 ,

,
_

,
_


+
D
j
D
i
j i j i
D
i
D
j
j i j i
m m
m m
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Magic Square (2)
D=3x3, N=100
10 runs
13430 evaluations
10 solutions
55 30 68
42 49 62
56 74 23
30 61 53
89 32 23
25 51 68
27 96 39
73 40 49
62 26 74
22 70 58
40 75 35
88 5 57
50 43 67
58 55 47
52 62 4
43 51 78
75 33 64
54 88 30
50 65 68
69 42 72
64 76 43
18 25 59
32 53 17
52 24 26
80 3 30
22 72 19
11 38 64
65 28 64
63 55 39
29 74 54
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com
Non linear system
( )

'


+
0 10 sin
0 1
2 1
2
2
2
1
x x
x x
Search space
[0,1]
2
1 run
143 evaluations
1 solution
10 runs
1430 evaluations
3 solutions
Particle Swarm optimisation
2002-04-
24
Maurice.Clerc@Write
Me.com