Beruflich Dokumente
Kultur Dokumente
5
Genetic Algorithms
5.1 Introduction
Genetic algorithms are computational techniques for searching the optimum
or maximum of complex objective fitness functions based on a process that
simulates Darwin’s nature evolution theory. In 1975, Holland established the
theoretical foundation that initiated the most contemporary developments
of GAs. Since then, extensive research works have been carried out on the
theoretical investigation and engineering application of GAs. Due mainly to
their applicability to problems with very complex objective functions, GAs
have been successful in a wide variety of scientific fields such as computa-
tional search algorithms, optimization, and machine learning. As effective
Maximize f (x)
(5.1)
Subject to x L ≤ x ≤ x U
5.2.1 Coding
In a plain GA program, each parameter, xi (i = 1, 2, , N ) , of a given problem
should be coded into a finite-length string according to one of the coding
methods, among which binary coding is the simplest and most popular. A
so-called chromosome is formed as a super string that combines all these
finite-length strings and represents an individual (a candidate for solution
to the given problem). After the optimal individual is found, it is then
decoded back to the physical parameter. It should be noted that binary
coding of the parameters is not absolutely necessary. As will be illustrated
in Section 5.3.2, the parameters can be directly used in the so-called real
parameter coded GA. Here, the popular binary coding is used to illustrate
the process of GAs.
The objective function is often defined using continuous variables of
parameters. GAs, however, operate on the (binary-encoded) discrete param-
eters. Therefore, these parameters in a continuous space should be first
discretized, and then are encoded in binary form. The mathematical formu -
lation for the binary encoding and decoding of the ith parameters can be
given as (Haupt and Haupt, 1998)
Encoding
xi − xiL
xi = (5.2)
xiU − xiL
m −1
gene[m] = roundxi − 2 − m −
∑ gene[m]2
k =1
−k
(5.3)
Decoding
N gene
x qn
i = ∑ gene[m]2
m =1
−m
+ 2 − ( m+1)
(5.4)
( )
xiq = xiqn xiU − xiL + xiL
where
xi : normalized i th parameter 0.0 ≤ x ≤ 1.0
xiL : smallest values of ith parameter
xiU : highest values of ith parameter
gene[m]: binary value of xi
round[]: round to nearest integer
Ngene: number of bits in the gene
xiqn : quantized value of xi
xiq : quantized value of xi
5.2.2.1 Selection
Selection is a process in which a mating pool of individual chromosomes of
the current generation is chosen in a certain way for reproduction of the
child generation according to the fitness values of the chromosomes of the
current generation. This operator is designed to improve the average quality
of the population by giving individuals of higher fitness a higher probability
to be copied to produce the new individuals of chromosomes in the child
5.2.2.2 Crossover
After the selection operation is completed and the mating pool is formed,
the so-called crossover operator may proceed. Crossover is an operation to
exchange part of the genes in the chromosomes of two parents in the mating
pool to create new individuals for the child generation; it is the most impor-
tant operator in a GA. A simple crossover proceeds in two steps. First,
members of the chromosomes in the mating pool are mated at random. Next,
each pair of the randomly selected chromosomes undergoes a crossover
using one of the following schemes to generate new chromosomes (Davis,
1991; Goldberg, 1989; Lawrence, 1987; Syswerda, 1989):
crossover (5.5)
The first part of the gene segment of the first parent is hooked up with
the second part of the gene segment of the second parent to make the first
offspring. The second offspring is built from the first part of the second
parent and the second part of the first parent:
Parent 1: 1101|010|101
crossover (5.7)
Parent 2: 0010|001|110
Offspring 1: 1101|001|101
Offspring 2: 0010|010|110 (5.8)
defined mask is used. For example, consider the following two parents that
have been selected for crossover:
Parent 1: ABCDEFGH
Parent 2: IJKLMNOP
If the probability is 0.5, approximately half of the gene bits in the offspring
will come from parent 1 and the other half will come from parent 2. With
the mask of 1 0 1 0 1 0 1 0, the possible sets of offspring after uniform
crossover are:
Offspring 1: AJCLENGP
Offspring 2: IBKDMFOH (5.10)
Offspring 1: IBKDMFOH
Offspring 2: AJCLENGP (5.11)
5.2.2.3 Mutation
The mutation operator is designed so that one or more of the chromosome’s
genes will be mutated at a small probability. The goal of the mutation oper-
ator is to prevent the genetic population from converging to a local minimum
and to introduce some new possible solutions to the generation. Without
mutation, the population would rapidly become uniform under the so-called
conjugated effect of selection and crossover operator. There are a number of
mutation methods (OpitGA: http//www.optwater.com/optiga): flip bit, ran-
dom, and min–max. For example, consider the following parent that has been
selected for mutation. The bit at a selected point is mutated from 0 to 1:
Parent: 1101010101
mutation (5.12)
Offspring: 1101011101
Now the basic operators of GAs have been briefly introduced. The con-
temporary developments of GAs have introduced many new techniques to
improve the performance of GA operators, as well as the way of performing
the coding. For additional details as well as the mathematic foundations,
readers are referred to the references listed in Section 5.11.
The next section demonstrates application of the basic operations of the
genetic algorithm via a simple example of optimization.
Minimize f ( x1 , x 2 ) = ( x1 + 1) + ( x 2 + 1)
2 2
(5.13)
Subject − 2 ≤ x1 ≤ 2, − 2 ≤ x 2 ≤ 2
Obviously, the optimum solution of the problem is (–1, –1)T with the function
value of zero.
5.2.3.1 Solution
Because GAs are often coded for the maximization problem, first transfer
the minimization problem specified by Equation 5.13 to a maximization
problem. A number of such transformations can be used. Here, the following
fitness function is employed in the GA performance according the transfor-
mation given by Deb (1998).
1.0
f ( x1 , x 2 ) = (5.14)
1.0 + f ( x1 , x 2 )
TABLE 5.1
GA Search Space for Numerical Test of Problem Defined by Equation 5.13
Parameter Search Range Possibilities # Binary Digit
x1 –2.0–2.0 4096 12
x2 –2.0–2.0 4096 12
Total population is 2 24 (1.678 × 10 7 ) .
TABLE 5.2
Initial Generation of 15 Randomly Generated Chromosomes and
the Corresponding Real Parameters and Fitness Value
No. Binary Code x1 x2 Fitness
1 100011111101010001001000 0.2476 –0.9294 0.39039
2 001101000110010010001101 –1.1814 –0.8620 0.95061
3 001110011100110010101011 –1.0974 1.1678 0.17517
4 001110110101011010010100 –1.0730 –0.3551 0.70360
5 111000101110111111010001 1.5458 1.9551 0.06168
6 111011111100000100100111 1.7470 –1.7118 0.11046
7 110101010101110010010111 1.3338 1.1482 0.09040
8 001101010100101011011110 –1.1678 0.7175 0.25139
9 001100100011101111101101 –1.2156 0.9822 0.20098
10 010100000010111101010000 –0.7477 1.8291 0.11029
11 001000110010110101110110 –1.4510 1.3661 0.14702
12 011101101110111111111011 –0.1421 1.9961 0.09335
13 111010110101110111111111 1.6777 1.4999 0.06935
14 001010100111010010101010 –1.3368 –0.8337 0.87638
15 101111100000011010000011 0.9695 –0.3717 0.18962
as seven new individuals for the next generation, and the rest are discarded
based on the rule of survival of the fittest. These seven best individuals are
also used to form the mating pool to produce the other shortfall of eight new
individuals for the next generation. Two individuals from the mating pool are
paired in a random fashion. Pairing chromosomes in a GA can be carried out
by various methods, such as pairing from top to bottom, random pairing,
ranking weighting, etc. The often used approaches are based on the probabil-
ities of individuals. The probability pi of the ith individual selected for the
pairing is proportional to its fitness value and can be computed using
fi
pi = nbi (5.15)
∑f i =1
i
where nbi is the number of the best individuals (equal to seven in this
example). A simple algorithm can be coded to pair up eight pairs of parents
using these seven individuals, based on the probability value obtained from
Equation 5.15. Using the crossover operators, these parents are then mated
to produce the shortfall of eight children for the next generation.
In the crossover operation, the crossover points must be determined first.
In this example, one crossover point is first randomly selected. The gene
segments in chromosomes of the paired parent individuals are then
exchanged. Assuming the following two pairing individuals are selected
from the mating pool:
f (C1) = 0.39039
(5.18)
f (C 2) = 0.95061
Assume that the crossover point was randomly selected after the 12th gene
bit:
f (O1) = 0.376275
(5.22)
f (O 2) = 0.963493
Mutation
Offspring 3 (O3): 100011111101 010001011000
5.2.3.5 Results
For this numerical example, the following GA operational parameters have
been used:
From the calculation result, it has been found that the best chromosome
after 100 generations is 001111111110010000000000, and corresponding values
of this chromosome are x1 = –1.0017, x2 = –0.9998; the corresponding fitness
value is 0.999993. The convergence of fitness value against the number of
generations for a GA run is plotted in Figure 5.1. It can be observed from
the convergence curve that the GA converges very fast at the beginning and
very slowly at later stages. The converging performance slows down signif-
icantly at the final stage of searching.
0.995
0.99
0.985
0.98
f
0.975
0.97
0.965
0.96
0.955
0.95
0 10 20 30 40 50 60 70 80 90 100
Generation
FIGURE 5.1
Convergence of a GA for the simple problem defined in Equation 5.14. The GA converges very
fast at the beginning and very slow at the later stage.
TABLE 5.3
Comparison between GA and Gradient-Based Optimization and Search Procedures
Items GA Gradient-Based Optimization
Search bases Groups of points Single point
Initial guess No Yes
Function information Objective function only Objective function and its
derivatives
Search rule Probabilistic in nature Deterministic laws
Convergence Fast at beginning, slow at the Relatively slow at initial stage,
later stage very fast at later stage
Applicability Global search for complex Local search for simple problem
problems with many local with single optimum
optima
Computing efficiency Computationally expensive Efficient
j = :0
Randomly
initialize P(0)
Binary encoding
Decoding
Evaluate P(0)
Elitism, with
Convergence Yes others in P(j)
criteria 1 are randomly
selected
No
j = :j+1 Binary encoding
Tournament
selection
Uniform
crossover
Decoding
Evaluate P(j)
Elitism
FIGURE 5.2
Flow chart of the µGA. (From Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With
permission.)
cess will be sequentially conducted until the global optimum is found (or
the predesignated number of generations is reached) and is schematically
depicted in Figure 5.2, where P(j) and C(j) denote the parents and child
(offspring) in the jth generation, respectively.
The key strategy of the the µGA is to divide the GA search into many
cycles, each of which will find a local optimal in the encoded space. To do
this efficiently, it uses a small population size for each “micro” generation
to achieve fast convergence to a local optimum in one cycle, and to restart
the global exploration via randomly generating a relatively large number of
individuals in the microgeneration of a new cycle. The elitism is always used
in generation to generation and cycle to cycle.
By introducing the micro technique, the µGA guarantees its robustness in
a different way: whenever the microgeneration is reborn, new chromosomes
are randomly generated, so new genetic information keeps flowing in. Krish-
nakumar’s 1989 study pointed out that a µGA can avoid premature conver-
gence and demonstrates faster convergence to the near optimal region than
does a plain GA for many multimodal problems.
j = :0
Randomly
initialize P(0)
Evaluate P(0)
Yes
Convergence Elitism
No
j = : j+1
Selection
Crossover
Evaluate P(j)
Elitism
FIGURE 5.3
Flowchart of the real µGA.
convergence occurs, this subcycle ends and the next subcycle starts. Each
subcycle typically consists of several generations. In every generation, tour-
nament selection, elitism, and crossover operators are included. A mutation
operator is not present in the process.
Although many similarities have been mentioned, some differences do
exist. Two of them are
x2 x2
x1 x1
(a) (b)
x1
x1 x2 x2
(c) (d)
Possible x location Possible x location
FIGURE 5.4
Schematic representation of different cross-over operators: (a) simple crossover; (b) uniform
arithmetical crossover; (c) uniform heuristic crossover; and (d) uniform extended arithmetical
crossover.
the real µGA, the search covers a large proportion of the entire search
space in the beginning of every subcycle. The search range covered
reduces as the search progresses, until all the candidates in a gener-
ation are crowded in a very small area and the convergence criterion
is reached. Once that happens, new randomly generated individuals
will flow in and the next subcycle starts. In the µGA, no clear phys-
ical interpretation on the convergence can be provided.
where xij stands for the jth parameter in the ith parent individual.
The four crossover operators are plotted schematically in Figure 5.4 (a–d).
They can be expressed mathematically as:
• Simple crossover
{
x = x11 x12 ... x1i x 2 , i +1 x 2 ,i + 2 ... x2n } (5.29)
xi = ai x1i + (1 − ai )x 2 i , i = 1, 2, … , n (5.30)
xi = ai ( x 2 i − x1i ) + x 2 i (5.31)
With these four crossover operators, four versions of real µGAs are con-
structed. In order to compare the performance of different algorithms mean-
ingfully, all the parameters and operations are set the same, except for the
• Tournament selection
• Uniform simple, uniform arithmetical, uniform heuristic, or uniform
extended arithmetical crossover; for uniform crossover in uniform
µGA, the probability of crossover is set to 0.5
• Elitism operator
• No mutation operation
• Population size of each generation set to 5
• The population convergence criterion for the real µGA is 2%, which
means that when all the candidates in a generation are located very
near to each other in physical space so that the maximum distance
of two candidates is less than 2% of the search range, the conver-
gence occurs. In the uniform µGA, the population convergence cri-
terion is set to be 5%, which means that when less than 5% of the
total bits of other individuals in a generation are different from the
best individual, the convergence occurs.
Test Functions
Variable
No. Objective Function Bound Global Optima Fitness Value
F1 0 < xi < 1.0 (0.0669, 0.0669) 1.0 (maximum)
−4(log 2)( x i − 0.0667 ) 2
2
p = 3.14159
F2 5 5 –10 < xi < 10 (4.8581, –7.0835) –186.7309 (minimum)
f (x1 , x 2 ) = ∑
i =1
i cos((i + 1)x 1 + i) ∑i =1
i cos((i + 1)x 2 + i) i = 1, 2 (–1.4251, –0.8003)
(–0.8003, –1.4251)
F3 –10 < xi < 10 (–1.0467, 0.0) –0.3524 (minimum)
f ( x 1 , x 2 ) = x 14 / 4 − x 12 / 2 + x 1 / 10 + x 22 / 2
i = 1, 2
F4 3 –5 < xi < 5 (1.0, 1.0, 1.0) 0.0 (minimum)
f (x1 , x 2 , x 3 ) = ∑
i =1
(( x 1 − x i2 ) 2 + ( x i − 1) 2 ) i = 1, 2, 3
d[4, 5] = (4, 4, 4, 4; 1, 1, 1, 1; 8, 8, 8, 8; 6, 6, 6, 6; 3, 7, 3, 7)
c[5] = (0.1, 0.2, 0.2, 0.4, 0.4)
Source: Xu, Y.G., et al., Appl. Artificial Intelligence, 15(7), 601–631, 2001. With permission
1523_Frame_C05.fm Page 129 Thursday, August 28, 2003 4:23 PM
X2
X1
FIGURE 5.5
Test function F1 has a number of local optima in the search space. (From Xu, Y.G. et al., Appl.
Artifi. Intelligence, 15(7), 601–631, 2001. With permission.)
X2
X1
FIGURE 5.6
Test function F2 with a number of local optima in the search space. (From Xu, Y.G. et al., Appl.
Artif. Intelligence, 15(7), 601–631, 2001. With permission.)
Generation Number Required to Achieve Best Function Values by Different GAs for Test Function F1
Real-µGA, Crossover 1 Real-µGA, Crossover 2 Real-µGA, Crossover 3 Real-µGA, Crossover 4 Binary-µGA
Best Best Best Best Best
Function Generation Function Generation Function Generation Function Generation Function Generation
Value Number Value Number Value Number Value Number Value Number
0.456 50 0.5113 170 0.5139 44 0.5655 100 0.58124 5
0.8696 191 0.8208 103 0.8398 114 0.8471 138 0.8423 242
0.95347 1101 0.9785 977 0.9785 977 0.9500 280 0.8423 (up to 2000)
0.9989 1879 0.9996 693 0.9996 1020 0.9996 450 0.8423 (up to 2000)
TABLE 5.6
Generation Number Required to Achieve Best Function Values by Different GAs for Test Function F2
Real-µGA, Crossover 1 Real-µGA, Crossover 2 Real-µGA, Crossover 3 Real-µGA, Crossover 4 Binary-µGA
Best Best Best Best Best
Function Generation Function Generation Function Generation Function Generation Function Generation
Value Number Value Number Value Number Value Number Value Number
–41.895 12 –38.82 12 –48.497 7 –33.211 13 –44.04 26
–140.68 57 –120.87 83 –147.94 183 –171.39 14 –114.4 52
–182.58 109 –182.30 140 –185.60 230 –186.51 33 –186.06 151
–186.11 1753 –186.47 157 –186.50 257 –186.70 58 –186.19 221
1523_Frame_C05.fm Page 131 Thursday, August 28, 2003 4:23 PM
© 2003 by CRC Press LLC
TABLE 5.7
Generation Number Required to Achieve Best Function Values by Different GAs for Test Function F3
Real-µGA, Crossover 1 Real-µGA, Crossover 2 Real-µGA, Crossover 3 Real-µGA, Crossover 4 Binary-µGA
Best Best Best Best Best
Function Generation Function Generation Function Generation Function Generation Function Generation
Value Number Value Number Value Number Value Number Value Number
–0.2134 16 –0.1508 53 –0.1536 50 –0.1009 46 –0.1345 28
–0.3437 128 –0.3514 146 –0.3502 77 –0.3455 47 –0.3279 86
–0.3509 1698 –0.3524 187 –0.35236 112 –0.35237 132 –0.35238 225
TABLE 5.8
Generation Number Required to Achieve Best Function Values by Different GAs for Test Function F4
Real-µGA, Crossover 1 Real-µGA, Crossover 2 Real-µGA, Crossover 3 Real-µGA, Crossover 4 Binary-µGA
Best Best Best Best Best
Function Generation Function Generation Function Generation Function Generation Function Generation
Value Number Value Number Value Number Value Number Value Number
2.043 23 0.9847 16 1.298 62 0.9928 136 0.4703 22
1.3573 (up to 2000) 0.00133 96 0.00087 122 0.000202 331 0.0092245 1180
1.3573 (up to 2000) 0.000 131 0.000 183 0.000 1464 0.0092245 2000
1523_Frame_C05.fm Page 132 Thursday, August 28, 2003 4:23 PM
TABLE 5.9
© 2003 by CRC Press LLC
Generation Number Required to Achieve Best Function Values by Different GAs for Test Function F5
Real-µGA, Crossover 1 Real-µGA, Crossover 2 Real-µGA, Crossover 3 Real-µGA, Crossover 4 Binary-µGA
Best Best Best Best Best
Function Generation Function Generation Function Generation Function Generation Function Generation
Value Number Value Number Value Number Value Number Value Number
5.8747 16 0.9927 16 0.1946 63 0.9965 136 0.47 22
1.3795 (up to 2000) 0.836e-4 130 0.9488e-4 157 0.2369e-3 352 0.999e-2 (up to 2000)
1.3795 (up to 2000) 0.153e-4 131 (up to 2000) 1.529e-5 467 1.529e-5 452 0.999e-2 (up to 2000)
TABLE 5.10
Generation Number Required to Achieve Best Function Values by Different GAs for Test Function F6
Real-µGA, Crossover 2 Real-µGA, Crossover 3 Real-µGA, Crossover 4 Binary -µGA
Best Best Best Best
Function Generation Function Generation Function Generation Function Generation
Value Number Value Number Value Number Value Number
–1.019 39 –2.119 29 –1.0557 67 –1.208 14
–5.3675 129 –5.0255 63 –5.0183 271 –2.6301 (up to 1168, failed)
–10.150 855 –10.152 156 –10.151 1302 –2.6301 (up to 1168, failed)
1523_Frame_C05.fm Page 133 Thursday, August 28, 2003 4:23 PM
0.9
0.8
0.7
Best function value 0.6
0.5
0.4
0.3
Binary coded
Real coded crossover 1
0.2
Real coded crossover 2
Real coded crossover 3
0.1 Real coded crossover 4
0
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Generation number
FIGURE 5.7
Convergence of the real µGA for test function F1.
-20
-40
-60
Best function value
-80
-100
-180
-200
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Generation number
FIGURE 5.8
Convergence of the real µGA for test function F2.
20
Binary coded
Real coded crossover 1
15 Real coded crossover 2
Real coded crossover 3
Real coded crossover 4
Best function value
10
-5
0 50 100 150 200 250 300 350 400 450 500
Generation number
FIGURE 5.9
Convergence of the real µGA for test function F3.
90
80
Binary coded
70 Real coded crossover 1
Real coded crossover 2
Real coded crossover 3
Best function value
60
Real coded crossover 4
50
40
30
20
10
0
0 50 100 150 200 250 300 350 400 450 500
Generation number
FIGURE 5.10
Convergence of the real µGA for test function F4.
90
80
Binary coded
70 Real coded crossover 1
Real coded crossover 2
50
40
30
20
10
0
0 50 100 150 200 250 300
Generation number
FIGURE 5.11
Convergence of the real µGA for test function F5.
-2
Best function value
-4
-6
Binary coded
Real coded crossover 1
Real coded crossover 2
-8 Real coded crossover 3
Real coded crossover 4
-10
-12
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Generation number
FIGURE 5.12
Convergence of the real µGA for test function F6.
TABLE 5.11
Convergence Comparison between Real-µGA with Crossover 4 and the Binary- µGA
Test Functions Real-µGA , Crossover 4 Binary -µGA
(Maximum or Convergence Generation Convergence
Minimum) Point Number Point Generation Number
F1 (1.000) 0.9996 450 0.834 Up to 2000 (failed to find
the solution)
F2 (–186.7309) –186.70 58 –186.19 221
F3 (–0.3524) –0.3524 132 –0.3524 225
F4 (0.0) 0.000 1464 0.00922 2000
F5 (0.0) 1.529e-5 452 0.96d-2 Up to 2000 (failed to find
the solution)
F6 (–10.1532) –10.151 1302 –2.6301 Up to 2000 (failed to find
the solution)
5.4.1 Modified µ GA
It is obvious that the population size and the measuring criterion for defining
the population convergence have a great influence upon the performance of
µGAs. The issue of population size was examined and the corresponding
procedure to determine the best population size was developed by Abu-
Lebdeh and Benekohal (1999). The criterion for defining the population
convergence was described by Carroll (1996a) as having less than 5% of the
genes (or bits) of the other individuals different from the best individual in
one generation; it has been successfully applied in engineering practice so
far (Xiao and Yabe, 1998; Carroll, 1996a, b).
Improvement for this criterion is still possible, however, because it takes
into account only the number of the “different genes,” but not their positions
in the compared chromosome strings. In fact, if two individuals have the
same number of the different gene from the best individual, but the different
genes in the compared chromosomes are at different positions, their Euclid-
ean distance from the best individual may be significantly different in solu-
tion space (or real-value parameter space). This can be immediately
demonstrated by the following example (see Table 5.12). Chromosomes A,
TABLE 5.12
Comparison of Euclidean Distances between Two Chromosomes in µGAs
Binary String Real Value Euclidean Distance
Chromosome A 1101|1001|0001 13 | 9 | 1 ||A–C||2 = 2
Chromosome B 0101|1001|0011 5|9|3 ||B–C||2 = 8
Chromosome C 1101|1001|0011 13 | 9 | 3
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.
B, and C are constructed by a 12-bit string coded from three real value
parameters, respectively. Chromosome A and chromosome B have one dif-
ferent gene with the chromosome C. Because the different gene in chromo-
some A is under a position different from that in chromosome B, the
Euclidean distance of chromosomes A and C is thus significantly different
from that of chromosomes B and C in the solution space.
The solution space is the real space to define whether or not the population
converges; therefore, it is insufficient for the criterion to take into account
only the number of different genes in coding space without considering the
differences of their Euclidean distances in solution space. A modified crite-
rion has been introduced to overcome this problem (Xu et al., 2001c), in
which a weight wi is introduced to take into account the position of the
different gene, i.e., the difference of Euclidean distance in solution space for
the compared individuals. The weight wi is given as:
2i
wi = np (5.33)
( N − 1)∑ n (n + 1)
j =1
j j
where i is the position of the different gene that counts from right to left in
the substring representing the ith real parameter, nj is the number of genes
(or bits) in the jth substring, np is the number of parameters to be optimized,
and N is the population size. It is obvious that in Equation 5.33, the more
leftwards the position of the different gene in two substrings of the compared
chromosomes, the larger the Euclidean distance of these two chromosomes
is in the solution space, and thus, the larger the weight wi is. This means more
influence of the different gene on the population convergence of the µGA.
As for two extreme cases where each of the genes in N – 1 compared
chromosomes is identical to (complete convergence) or different from that
in the best individual (Xu et al., 2001c):
N −1 np nj
∑∑∑
2i
np =0 (5.34)
j =1 j =1 i =1
( N − 1) ∑ n (n + 1)
j =1
j j
and
N −1 np nj
∑∑∑
2i
np =1 (5.35)
j =1 j =1 i =1
( N − 1) ∑ n (n + 1)
j =1
j j
Therefore, the criterion for defining the population convergence of the µGA
is set as:
N −1 np nj
∑∑∑
2i
np ≤γ (5.36)
j =1 j =1 i =1
( N − 1)∑ n (n + 1)
j =1
j j
(
c1 = p bj + α p bj − p bj −1 ) (5.37)
(
c2 = p bj −1 + β p bj − p bj −1 ) (5.38)
where α and β are the control parameters of the IP operator; both are
recommended to be within the range from 0.3 to 0.7. The effect of the
control parameters on the evolutionary process is addressed in the follow-
ing examples.
The two newly obtained individuals, c1 and c2, are used to replace the two
worst individuals in the present offspring. Because some kind of gradient
between the generations is used, the use of Equation 5.37 and Equation 5.38
is expected to get to a better individual. This feature is especially important
when searching has entered into the local region around the global optimum,
where the best individual is close to the global optimum.
j=0
Initiate P(j)
Evaluate P(j)
Yes
Stop criterion ? End
No
j = j+1
Niching
Selection
Intergeneration projection
(IP) operator
Crossover
Obtain C1, C2
Elitism
Generate C(j)
Evaluate C1, C2
Evaluate C(j)
Generate Ch(j)
No Population
convergence ?
Yes
Restart
FIGURE 5.13
Flow chart of the IP-GA. (From Xu, Y.G. et al., Appl. Artifi. Intelligence, 15(7), 601–631, 2001. With
permission.)
Sp1{cbj}
C1 IP operation dominates the evolution
p bj
C2
p bj−1
Sp2{cbj: f(cbj)≥f(c2)} is larger
Sp2{cbj}
Genetic operators dominate the evolution
C2
p bj−1
Fitness value
p bj C1
Individuals
FIGURE 5.14
Effect of intergeneration projection operation on the evolution process. At the early stage of
evolution, the subspace Sp{cjb: f(cjb) ≥ f(cb)} is larger, where cjb∈C(j), f(cjb) = max{f(cj1), f(cj2),…,f(cjN)},
cb ∈ (c1, c2), f(cb) = max{f(c1), f(c2)}. This means the conventional genetic operations based on the
stochastic model have a larger possibility to generate the individual cjb better than cb generated
from the IP operator. As a result, the conventional genetic operators have great dominance in
this stage. At the later stage, with the subspace Sp{cjb: f(cjb) ≥ f(cb)} growing smaller, the possibility
p(f(cjb) ≥ f(cb)) would also correspondingly become smaller and smaller. (From Xu, Y.G. et al.,
Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.)
TABLE 5.13
Comparison of Numbers of Generations to Convergence Using µGA and IP-GA
for Different α with β =0.5
nIP-GAa (β =0.5, α varies from 0.1 ~ 0.9) nµGAb nIP-GA/nµGA (%)
No. 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 (fn) Min Max
F1 189 325 213 209 44 151 96 103 293 >500 <8.8 <65
(0.9998)
F2 177 229 164 58 72 187 68 104 94 >500 <11.6 <45.8
(–185.83)
F3 261 353 236 278 202 80 198 286 326 493 16.2 71.6
(–0.3524)
F4 326 446 249 278 309 266 247 363 418 >1000 <24.7 <44.6
(–0.0090)
F5 337 458 235 274 265 188 279 389 437 >1000 <18.8 <45.8
(–0.0093)
F6 1759 959 746 532 436 596 873 682 1232 >3000 <14.5 <58.6
(–5.0556)
a nIP-GA = the number of generations to convergence using the IP-GA.
b nµGA = the number of generations to convergence using the µGA.
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.
TABLE 5.14
Comparison of Numbers of Generations to Convergence Using µGA and IP-
GA for Different α with β =0.6
nIP-GAa (α = 0.6, β varies from 0.1 ~ 0.9) nµGAb nIP-GA / nµGA (%)
No. 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 (fn) Min Max
F1 302 32 97 54 151 203 55 66 87 >500 <6.4 <60.4
(0.9998)
F2 205 71 93 110 187 105 91 40 97 >500 <8 <41
(–185.83)
F3 351 367 198 234 80 135 186 259 301 493 16.2 74.4
(–0.3524)
F4 333 482 388 198 266 231 206 589 342 >1000 <19.8 <58.9
(–0.0090)
F5 392 513 401 201 188 303 257 556 312 >1000 <18.8 <55.6
(–0.0093)
F6 770 1214 768 486 596 723 512 780 1106 >3000 <17.1 <40.5
(–5.0556)
a nIP-GA = the number of generations to convergence for the IP-GA.
b nµGA = the number of generations to convergence for the µGA.
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.
to a smaller range of 0.3 ~ 0.7, the maximal ratio nIP-GA/nµGA would decrease
from 74.4 to 56.4%.
The computation time for reproducing the same number of generations
using the IP-GA is almost equivalent to that using the conventional µGA for
each test function. For example, both take about 1 minute to complete the
evolution process of the first 500 generations for the test function F1 in the
workstation SGI/Cray. This feature clearly results from the fact that no
function evaluation is required in the added IP operator.
To reveal the evolution process, Figure 5.15 to Figure 5.20 show the con-
vergence processes of test functions F1 ~ F6, respectively, using the IP-GA
(α = 0.6, β = 0.5) against the conventional µGA.
1.2
1.0
0.8
Fitness value
µGA
0.6
IP-GA (α = 0.6 β = 0.5)
0.4
0.2
0.0
0 100 200 300 400 500
Number of generations
FIGURE 5.15
Comparison of convergence processes for test function F1. The IP-GA provide a quick conver-
gence. (From Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.)
200
160
Fitness value
120
µGA
80 IP-GA (α = 0.6 β = 0.5)
40
0
0 100 200 300 400 500
Number of generations
FIGURE 5.16
Comparison of convergence processes for test function F2. (From Xu, Y.G. et al., Appl. Artif.
Intelligence, 15(7), 601–631, 2001. With permission.)
0.5
0.3
Fitness value
0.0
µGA
IP-GA (α = 0.6 β = 0.5)
-0.3
-0.5
0 100 200 300 400 500
Number of generations
FIGURE 5.17
Comparison of convergence processes for test function F3. (From Xu, Y.G. et al., Appl. Artif.
Intelligence, 15(7), 601–631, 2001. With permission.)
1.0
0.0
Fitness value
-1.0 µGA
IP-GA (α = 0.6 β = 0.5)
-2.0
-3.0
0 200 400 600 800 1000
Number of generations
FIGURE 5.18
Comparison of convergence processes for test function F4. (From Xu, Y.G. et al., Appl. Artif.
Intelligence, 15(7), 601–631, 2001. With permission.)
1.0
0.0
Fitness value
-1.0 µGA
IP-GA (α = 0.6 β = 0.5)
-2.0
-3.0
0 200 400 600 800 1000
Number of generations
FIGURE 5.19
Comparison of convergence processes for test function F5. (From Xu, Y.G. et al., Appl. Artif.
Intelligence, 15(7), 601–631, 2001. With permission.)
12.0
10.0
µGA
8.0
Fitness value
4.0
2.0
0.0
0 500 1000 1500 2000
Number of generations
FIGURE 5.20
Comparison of convergence processes for test function F6. (From Xu, Y.G. et al., Appl. Artif.
Intelligence, 15(7), 601–631, 2001. With permission.)
1.0
0.8
α
Fitness value
0.6 = 0.1 0.6
0.2 0.7
0.4 0.3 0.8
0.4 0.9
0.2
0.5
0.0
0 50 100 150 200 250 300
Number of generations
FIGURE 5.21
Evolution processes using different α for function F1 (β = 0.5). (From Xu, Y.G. et al., Appl. Artif.
Intelligence, 15(7), 601–631, 2001. With permission.)
1.0
0.8
β
Fitness value
0.2 0.7
0.4
0.3 0.8
0.5
0.0
0 50 100 150 200 250 300
Number of generations
FIGURE 5.22
Evolution processes using different β for function F1 (α = 0.6). (From Xu, Y.G. et al., Appl. Artif.
Intelligence, 15(7), 601–631, 2001. With permission.)
TABLE 5.15
Means of the Numbers of Generations
(nIP-GAa) of IP-GA to Convergence When Using
Different Ranges of α and β
α~β F1 F2 F3 F4 F5 F6
0.3 ~ 0.7 127 113 183 264 259 627
0.1 ~ 0.9 151 120 248 338 342 845
a nIP-GA = the number of generations to conver-
gence for the IP-GA.
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7),
601–631, 2001. With permission.
TABLE 5.16
Numbers of Best Individuals (nb) Generated by the IP Operator Using Different
α for Function F1 (β = 0.5)
α 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
nb 144 278 173 168 30 121 67 73 260
nb/nIP-GA (%) 76.2 85.5 81.2 80.4 75.0 80.1 69.8 70.9 88.7
a nIP-GA = the number of generations to convergence for the IP-GA.
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.
TABLE 5.17
Numbers of Best Individuals (nb) Generated by the IP Operator Using
Different β for Function F1 (α = 0.6)
β 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
nb 249 18 75 35 121 157 29 40 67
nb/nIP-GA (%) 82.5 56.3 77.7 64.8 80.1 77.3 52.7 60.6 77.0
a nIP-GA = the number of generations to convergence for the IP-GA.
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.
TABLE 5.18
Mean Ratios of nba/nIP-GAb for All Test Functions
No. F1 F2 F3 F4 F5 F6
nb/nIP-GA (%) 69.3 75.3 80.1 73.2 75.6 78.1
a nb = number of best individuals generated by the IP operator.
b nIP-GA = the number of generations to convergence for the IP-GA.
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001.
With permission.
1.2
1.0
Fitness value
0.8
FIGURE 5.23
Effect of IP operator on evolution process of function F1. (From Xu, Y.G. et al., Appl. Artif.
Intelligence, 15(7), 601–631, 2001. With permission.)
areas containing the global optima at the early stage and to converge quickly
to the final solution at the later generations.
TABLE 5.19
Results Obtained from Hill-Climbing Method in GA-HC(1)a for Function F2
Starting Points Ending Points
Function Function
Generation Individual Variables Value Variable Value nf-hillb
1 1 –9.6728, 5.6255 22.0841 –9.7803, 5.4827 38.2959 69
2 –7.2533, 2.8965 18.8253 –7.0837, 2.7860 38.2959 63
3 0.6772, 5.3959 30.6487 0.8219, 5.4827 54.4048 57
4 –7.1868, –9.9219 19.5499 –7.0837, –9.7803 38.2959 60
5 0.0003, –9.9169 –7.1568 –0.1956, –0.0342 3.0050 91
6 –9.9774, –9.6802 –1.3402 –10.0165, –0.780 0.0946 53
7 –9.3262, 0.3177 –13.4106 –9.7803, 0.3342 10.1556 112
2 1 –9.4775, 6.1065 12.8347 –9.2865, 6.0875 30.7807 71
2 –6.9249, 2.7525 27.1076 –7.0837, 2.7860 38.2959 65
3 –8.2366, –9.8401 14.1813 –8.2904, –9.7803 16.2861 60
4 0.8219, 5.4827 54.4048 0.8219, 5.4827 54.4048 16
5 2.8336, –9.8804 –5.1788 2.7860, –10.0366 1.0463 85
6 –9.9756, –9.5239 0.4181 –9.7803, –9.2865 9.5388 49
7 –9.0576, 0.4739 –1.2259 –8.7939, 0.3342 13.8031 74
3 1 1.1429, 6.1065 9.5043 1.3199, 6.0875 24.9369 71
2 –9.5868, 8.2409 –0.9176 –9.2865, 8.0889 9.8608 79
3 –9.4903, 5.4729 –18.4041 –9.7803, 5.4827 38.2959 81
4 –9.2816, 0.9116 11.5597 –9.2865, 0.8219 13.5513 55
5 –7.1618, –7.3705 –32.5859 –7.0837, –7.7081 186.7307 92
6 0.8219, 5.4827 54.4048 0.8219, 5.4827 54.4048 16
7 –6.5575, 0.4788 16.8553 –6.4788, 0.3342 32.7709 55
aGA-HC(1) = method integrating the hill-climbing method with the µGA.
bn = the number of function evaluations taken in the hill-climbing process.
f-hill
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.
GA, where the number of possibilities for variables is 32,768. The comparison
study is done for all six test functions.
Table 5.19 shows the results obtained from the hill-climbing method in the
GA-HC (1) for the test function F2. Function F2 is selected as a representative
function for detailed discussion because function F2 has many local optima
(see Figure 5.6). In Table 5.19, the starting points are actually the offspring
in the present generation that are obtained from the present parents using
the conventional genetic operations. The ending points are the local optima,
obtained using the hill-climbing method from the corresponding starting
points; they are the parents of the next generation. Table 5.19 shows that, in
the first three generations, the conventional genetic operations in the µGA
failed to discover the global optimum. The best fitness value in the first,
second, and third generations is 30.6478 (individual 3), 54.4048 (individual
4), and 54.4048 (individual 6), respectively. However, starting from individ-
ual 5 in the offspring of generation 3, the hill-climbing method has success-
fully discovered the global optimum (fitness value = 186.7307). The total
number of function evaluations nf taken in this process is 1395 (sum of the
numbers of function evaluations taken by both the hill-climbing method and
FIGURE 5.24
Hill-climbing searching starting from offspring in generation 1 in the GA-HC(1) for function
F2. (From Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.)
FIGURE 5.25
Hill-climbing searching starting from offspring in generation 3 in the GA-HC(1) for function
F2. (From Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.)
TABLE 5.20
Best Individuals Obtained from Hill-Climbing Method in GA-
HC(2)a for Function F2
Ending Point (Best Individual) No. of
Generation (npre)b Variable Function Value Optima nf-hillc
5 0.8219, 5.4827 54.4048 — 743
10 –6.4788, 5.4827 123.5766 — 619
20 –6.4788, 5.4827 123.5766 — 801
30 –1.4249, 5.4827 186.7307 1 1087
40 –1.4249, 5.4827 186.7307 3 1060
a GA-HC(2) = GA is run at predesignated number of generations (npre), then
hill-climbing method is applied to get the final solution.
b n = predesignated number of generations from which hill-climbing meth-
pre
od starts.
c n
f-hill = number of function evaluations taken in the hill-climbing process.
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With
permission.
Table 5.20 shows the best individuals and their fitness values obtained by
the hill-climbing method in the GA-HC(2). The hill-climbing searching starts
from the offspring in the generations 5, 10, 20, 30, and 40, respectively. It can
be seen that the hill-climbing searching starting from the offspring at the
generations 5, 10, and 20 fails to obtain the global optimum (Figure 5.26 for
the first case). It is not successful until the number npre increases to 30: one
of the offspring has successfully led the hill-climbing searching to reach the
global optimum (Figure 5.27). With the further increase of npre, more offspring
Function value
FIGURE 5.26
Hill-climbing searching starting from offspring in generation 5 in the GA-HC(2) for function
F2. (From Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.)
Function value
FIGURE 5.27
Hill-climbing searching starting from offspring in generation 30 in the GA-HC(2) for function
F2. (From Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001. With permission.)
become close to the global optimum. The number of offspring that can lead
the hill-climbing searching successfully to reach the global optimum has also
correspondingly increased. For example, when npre increases to 40, there are
three offspring that have successfully led the hill-climbing searching to get
to the global optimum. However, the larger npre usually results in more
function evaluations.
Comparing the GA-HC(1) with the GA-HC(2), the total number of function
evaluations, nf, taken to reach the global optimum for test function F2 is 1395
and 1297(npre = 30), respectively. The GA-HC(2) costs less computationally
than the GA-HC(1) — also true for the other test functions. However, it is
usually difficult to designate the number npre properly in the GA-HC(2) (Yang
et al., 1995; Xiao and Yabe, 1998; Xu et al., 2001c). Improper selection of npre
usually results in the overuse of function evaluations or failing to get the
global optimum.
Table 5.21 shows the number nf for the six test functions when using the
GA-HC(1), GA-HC(2), IP-GA, and the conventional µGA, respectively. It can
be found that the IP-GA incurs the least computation cost among all three
of these algorithms. The advantage is more obvious for test functions with
more decision variables such as functions, F3, F4, and F5. In addition, the
IP-GA does not have difficulty in choosing npre in the GA-HC(2).
Nevertheless, there is likely a situation in which this IP-GA does not
perform particularly well. That is, when the individual p bj is identical to
p bj −1 at the jth generation in the evolution process, the IP operator fails to
find the new individuals different from those in the C(j). This would decrease
the population diversity and also increase unnecessary evaluations for the
same individuals in one generation.
TABLE 5.21
Comparison of Numbers of Function Evaluations (nf) Using
Different Algorithms for Test Functions
Algorithm F1 F2 F3 F4 F5 F6
GA-HC(1) a 1476 1395 6199 10004 10012 9615
GA-HC(2) b 1373 1297 3375 6736 6765 8004
IP-GA 1150 1024 1654 2383 2338 5650
µGA >3500 >3500 3451 >7000 >7000 >21,000
a GA-HC(1) = method integrating the hill-climbing method with
µGA.
b GA-HC(2) = GA is run at predesignated number of generations
(npre), then hill-climbing method is applied to get the final solution.
Source: Xu, Y.G. et al., Appl. Artif. Intelligence, 15(7), 601–631, 2001.
With permission.
{
f (c) = max f (c1 ), f (c2 ) } c ∈ {c1 , c2 }
( )
c1 = p bj + α p bj − p
c2 = p bj −1 + β( p − p)
b
j
(5.39)
p bj −1 p bj ≠ p bj −1
p= s
p j p bj = p bj −1
where α and β are recommended to be within 0.1 ~ 05 and 0.3 ~ 0.7, respec-
tively. It is clear that α and β decide how far the newly generated individual
c is from the present best individual p bj . Selection of α and β has obvious
effects on the convergence process of the IP-GA. Detailed discussion on this
point takes place in Section 5.4.4 for the IP-GA; the following examples
address it further.
This process is depicted in Figure 5.28. Basically, the improved IP-GA takes
the same strategy in incorporating the IP operator into the basic loop of the
µGA as that used in the previous IP-GA. It thus maintains the main advan-
tages of IP-GAs:
j = 0, initiate P(j)
Evaluate P(j)
Yes
Stop criterion ? End
No
j = j+1
Niching
Crossover
Elitism
Select c
Generate and evaluate C(j)
Generate Ch(j)
No Population
convergence ?
Yes
Restart
FIGURE 5.28
Flow chart of the improved IP-GA. (From Xu, Y.G. et al., AIAA J., 40(9), 1860–1866, 2002. With
permission.)
TABLE 5.22
Convergence Performance of Improved IP-GA and Comparison with IP-GA
and µGAa
Function RatioIP RationµGA
No. Global Optimum Value nIIP-GAb nIP-GAc nµGAd (%)e (%) f
F1 (0.0669, 0.0669) 1.0 158 189 984 59.7 11.5
F2 (1.4251, –0.8003) g –186.73 185 348 1136 38.0 11.6
F3 (–1.0467, 0.0) –0.352 175 437 983 28.6 12.7
F4 (1.0, 1.0, 1.0) 0.0 261 544 1561 34.3 11.9
F5 (1.0, 1.0, 1.0) 0.0 229 583 1648 28.1 9.9
F6 (4.0, 4.0, 4.0, 4.0) –10.153 652 1195 3271 39.0 14.2
a Obtained from 40 independent runs.
b nIIP-GA = number of generations to convergence for the improved IP-GA.
c nIP-GA number of generations to convergence for the IP-GA.
d nµ GA = number of generations to convergence for the µGA.
e Ratio IP (%) = (5 × nIIP-GA)/(7 × nIP-GA) × 100%.
f Ratio µGA (%) = (5 × nIIP-GA)/(7 × nµGA) × 100%.
g One of global optima.
Source: Xu, Y.G. et al., AIAA J., 40(9), 1860–1866, 2002. With permission.
0.5
0.3
Fitness value
0.0 µGA
IP-GA
-0.3
Improved IP-GA
-0.5
0 30 60 90 120 150
Number of generations
(a) Function F3
12.0
9.0
µGA
Fitness value
IP-GA
6.0
Improved IP-GA
3.0
0.0
0 200 400 600 800 1000
Number of generations
(b) Function F6
FIGURE 5.29
Convergence processes of the improved IP-GA for benchmark function F3 and F6. (From Xu,
Y.G. et al., AIAA J., 40(9), 1860–1866, 2002. With permission.)
two relative ratios are calculated and shown in Table 5.22. From this table,
it can be found that the improved IP-GA takes only 9.9 ~ 14.2% (or 28.1 ~
59.7%) of the number of function evaluations required by the µGA (or the
IP-GA) to obtain the global optimum for these benchmark functions. Figure
5.29 shows the convergence processes of functions F4 and F6 using the
improved IP-GA against the IP-GA and µGA, from which the outstanding
performance of the improved IP-GA on the convergence can be seen clearly.
TABLE 5.23
Effect of Mutation Operator on Performance of Improved IP-
GA for the Case with α = 0.2, β = 0.5, and idum = –10,000
F1 F2 F3 F4 F5 F6
nIIP-GA(a)a 97 126 104 173 170 412
nIIP-GA(b)b 107 156 116 352 417 >1000
Ratio (%) c 90.7 80.8 89.7 49.1 40.8 <41.2
a nIIP-GA(a) = number of generations to convergence for the improved
IP-GA with the mutation operator.
b nIIP-GA(b) = number of generations to convergence for the improved
IP-GA without the mutation operator.
c Ratio (%) = nIIP-GA(a) / nIIP-GA(b) × 100%.
Source: Xu, Y.G. et al., AIAA J., 40(9), 1860–1866, 2002. With permission.
TABLE 5.24
Effect of Coefficient α on Performance of Improved IP-GA for
Function F1 Where β = 0.5 and idum = –10,000
α 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
nIIP-GA(a) a 147 86 50 119 38 81 63 56
nIIP-GA(b)b 149 109 86 130 44 91 74 91
Ratio (%)c 98.7 78.9 89.3 91.5 86.4 89.0 85.1 61.5
a nIIP-GA(a) = number of generations to convergence for the improved IP-GA
with the mutation operator.
b nIIP-GA(b) = number of generations to convergence for the improved IP-GA
without the mutation operator.
c Ratio (%) = nIIP-GA(a) / nIIP-GA(b) × 100%.
Source: Xu, Y.G. et al., AIAA J., 40(9), 1860–1866, 2002. With permission.
respectively. Table 5.23 shows that the improved IP-GA with the mutation
operation finds the global optimum faster than that without the mutation
operation for all the benchmark functions. Further investigations on the
effect of the mutation operation associated with the variation of α, β, and
idum are also performed, and the results are given in Table 5.24 to Table 5.26.
TABLE 5.25
Effect of Coefficient β on Performance of Improved IP-GA
β 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
nIIP-GA(a) a 486 217 70 86 80 164 372 243
nIIP-GA(b) b 121 106 98 109 94 74 109 99
Ratio (%)c 401.6 204.7 71.4 78.9 85.1 221.6 341.3 245.5
a nIIP-GA(a) = number of generations to convergence for the improved IP-GA with
the mutation operator.
b nIIP-GA(b) = number of generations to convergence for the improved IP-GA
without the mutation operator.
c Ratio (%) = nIIP-GA(a) / nIIP-GA(b) × 100%.
Source: Xu, Y.G. et al., AIAA J., 40(9), 1860–1866, 2002. With permission.
TABLE 5.26
Effect of the Random Number Seed idum on Performance of Improved IP-GA
and µGA
i dum (× 102) 1 50 100 150 200 300 350 400 450 500
nIIP-GA(a)a 245 164 97 150 196 229 60 175 67 197
nIIP-GA(b)b 279 230 107 77 186 238 185 128 395 199
nµGAc 823 2741 1229 1512 1105 240 1043 415 526 208
a nIIP-GA(a) = number of generations to convergence for the improved IP-GA with the
mutation operator.
b nIIP-GA(b) = number of generations to convergence for the improved IP-GA without the
mutation operator.
c nµ GA = number of generations to convergence for µGA.
Source: Xu, Y.G. et al., AIAA J., 40(9), 1860–1866, 2002. With permission.
the benchmark functions. However, it has been found that any combinations
of α and β always result in the fact that the improved IP-GA converges
significantly faster than the µGA using the same genetic operators and oper-
ation parameters. It is found from this study that α and β should be within
0.1 ~ 0.5 and 0.3 ~ 0.7, respectively. The recommended choice is α = 0.l ~ 0.3
and β = 0.5, which generally ends in good results in numerical experiments.
1.0
0.8
Fitness value
0.6 α = 0.1 0.5
0.2 0.6
0.4
0.3 0.7
0.2 0.4 0.8
0.0
0 20 40 60 80 100 120
Number of generations
(a) Effect of α with β = 0.5 and idum = -10000
1.0
0.8
Fitness value
0.0
20 40 60 80 100 120 140 160 180
Number of generations
(b) Effect of β with α = 0.2 and idum = -10000
FIGURE 5.30
Effects of coefficients α and β on the convergence processes of the improved IP-GA for test
function F1. (From Xu, Y.G. et al., AIAA J., 40(9), 1860–1866, 2002. With permission.)
(
c1 = p bj + α p bj − p bj −1 ) (5.40)
(
c2 = p bj −1 + β p bj − p bj −1 ) (5.41)
(
c 3 = p bj − γ p bj − p bj −1 ) (5.42)
Get started, j = 0
Initiate P(j)
Evaluate P(j)
Yes
Stop criterion ? End
No
Generate C(j)
Evaluate C(j)
No Population
convergence ?
Yes
IP operation to generate C1, C2, C3
Generate P(j)
Evaluate C1, C2, C3
j = j+1
Restart
FIGURE 5.31
Flowchart of the IP3-GA.
For other cases (such as F3, F6), the IP3-GA performed substantially better
than the IP-GA.
TABLE 5.27
Performance of IP3-GA and Comparison with Other Methods
Test Function µGA IP-GA IP3-GA RatioIPi/
RatioIP3j
No (x1, …, xn)opta foptb nµGAc fµGAd nIP-GAe fIP-GAf nIP3-GAg fIP3-GAh (%)
F1 (0.0669, 0.0669) 1.0 984 1.0 189 1.0 221 1.0 19.2/22.5
F2 (–1.4251, –.8003) –186.73 1136 –186.73 348 –186.73 277 –186.73 30.6/24.4
F3 (–1.0467, 0.0) –0.3524 983 –0.3524 437 –0.3524 137 –0.3524 44.5/13.9
F4 (1.0, 1.0, 1.0) 0.0 1561 –2.235E–8 544 –2.235E-8 331 –2.235E-8 34.8/21.2
F5 (1.0, 1.0, 1.0) 0.0 1648 –0.917E–7 583 1.254E-5 503 1.254E-5 35.4/30.5
F6 (4.0,4.0,4.0,4.0) –10.153 3271 –10.1532 1195 –10.1532 330 –10.1532 36.5/10.1
a (x1, …, xn)opt = optimal point of the test functions.
b fopt = function values at the optimal point.
c nµ GA = number of generations to convergence for the µGA.
d fµ = function value at the convergence point of the µGA.
e nIP-GA = number of generations to convergence for the IP-GA.
f fIP-GA = function value at the convergence point of the IP-GA.
g nIP3-GA = number of generations to convergence for the IP3-GA.
h fIP3-GA = function value at the convergence point of the IP3-GA.
i RatioIP = nIP-GA / nµ GA × 100%.
j RatioIP3 = nIP3-GA / nµGA × 100%.
1523_Frame_C05.fm Page 166 Thursday, August 28, 2003 4:23 PM
(
PMAX jnew = PMAX j + α PMAX jold − PMIN jold ) (5.43)
(
PMIN jnew = PMIN j − α PMAX jold − PMIN jold ) (5.44)
where α is a predefined factor and PMAX jold , PMIN jold are the maximum
and minimum values of each parameter in the previous search domain. This
procedure is depicted in Figure 5.32.
In the SR-GA, a sufficient number of generations is first carried out to
ensure that the recorded M best individuals are covering the space that
contains the global optimum of the objective function. The parameter M
should be so chosen to avoid trapping at any local optimal point when the
objective error function is not unimodal or not continuous. The parameter
α is used to ensure the local best individual is not excluded from the new
GA search process. The combination of M and α ensures that the GAs can
find the best individual for complicated objective functions, even when the
searching space is reduced.
M=3
Fitness value
Individuals
FIGURE 5.32
Schematic drawing of the search space deduction in the SR-GA. (From Liu, G.R. et al., Comput.
Struct., 80, 23–31, 2002h. With permission.)
The SR-GA has been applied to predict engineering problems and has
proved to be very efficient compared to the plain GA (see Section 13.1.2).
The authors strongly believe that this idea of search space reduction is one
of the most effective ways to solve the convergence stagnation problem in
GAs. Therefore, much more effort should be made in this direction to further
improving GAs by developing more efficient ways to reduce the search space
while the individual is approaching the global optimum.
1. The µGA is used to determine the initial points. The main purpose is
to select a set of better solutions close to the optima. The selection
criterion is imposed to limit the function value below a required value.
Minimize F( x1 , x 2 ) = ∑ ( f (x , x ))
2
i 1 2
i =1 (5.45)
Subject to − 6 ≤ x1 , x 2 ≤ 6
where
Note that the preceding function has four minimum points that can be
obtained by solving the following equations:
The solutions to these equations are (3.0, 2.0)T, (–2.805, 3.131)T, (–3.779,
–3.283)T, and (3.584, –1.848)T. This study is concerned with designing a func-
tion, which has only one global minimum point. Add two terms to the
Himmelblau function and form the following nonlinear least square problem,
i.e.,
Minimize g( x1 , x 2 ) = ∑ ( f (x , x ))
2
i 1 2
i =1 (5.48)
Subject to − 6 ≤ x1 , x 2 ≤ 6
where
The additional terms do not alter the location of the optimum and the
function value at the global optimal point (3.0, 2.0)T. They alter locations and
function values of the other three minimum points, thereby making them
local minimum points. Therefore, the global minimum of Equation 5.48 is
still at (3.0, 2.0) T and has a function value of zero. Other three local minima
have higher function values of 3.498, 7.386, and 1.515, respectively. This
problem has been studied by Deb (1998). He has found that, on average, one
out of four simulations of the steepest descent algorithm solve the preceding
problem to the global optimum, and a successful run takes 215 function
evaluations on average to convergence. This finding is typical for many
traditional gradient-based optimization algorithms. If they do not begin with
a sufficiently good point, the algorithms may converge to a wrong solution
of a local minimum. In contrast, the GA could be the global minimum of the
function find most of the time; the average number of function evaluations
required to achieve the global minimum is 520.
As a comparison, a uniform µGA with binary parameter coding, tourna-
ment selection, uniform crossover, and elitism is adopted to solve the prob-
lem. The population size of each generation is set to be 5 and the probability
of uniform crossover is set to be 0.6. The population convergence criterion
is 5%, i.e., when less than 5% of the total bits of other individuals in a
generation are different from the best individual, the convergence occurs. A
new population, in which the best individual of the last generation is repli-
cated, will be randomly generated and the evolution process restarts.
Knuth’s subtractive method is used to generate random numbers.
The search space defined for this numerical test is listed in Table 5.28. The
two parameters are described and translated into chromosomes. In the whole
search space, a total of 214 (16,384) possible combinations of these two param-
eters exists. It has been found that the uniform µGA can find the global
minimum of the function; the average number of function evaluations
required to achieve the global minimum is 480, that is, less than using the
plain GA (520). The value of error function against the number of generation
for a GA run is also plotted in Figure 5.33. From this figure, it can be seen
that the GA can reach the “better” points fast, but its convergence to the
“best” is very slow. It has been found from this example that it can reach
TABLE 5.28
Uniform µGA Search Space for Numerical Test Defined
by Equation 5.48
Parameter Search Range Possibilities # Binary Digit
x1 –6.0, 6.0 128 7
x2 –6.0, 6.0 128 7
Note: Total population in the entire search space is 2 14 (16,384).
Source: Liu, G.R. et al., Comput. Methods Appl. Mech. Eng., 191,
1909–1921, 2002. With permission.
35
30
25
Function value
20
15
10
0
0 20 40 60 80 100 120 140 160 180 200
Number of generations
FIGURE 5.33
Convergence of a µGA for the numerical test of problem defined by Equation 5.48. (From Liu,
G.R. Comput. Methods Appl. Mech. Eng., 191, 1909–1921, 2002. With permission.)
the point (3.176, 1.906)T at 51 generations, but it does not converge to the
global minimum point (3.0, 2.0)T until 96 generations.
Now, the combined optimization method is used to solve the same prob-
lem. At the first step, the uniform µGA is used to isolate the best zones in
the parameters space. In other words, the uniform µGA is employed as a
tool to determine the initial estimations of the parameters. Four sets of the
parameters can be selected based on the results generated from the first five
generations. The selection criterion is imposed to limit the function value
below 200. The values of these selected sets and their corresponding function
values are listed in Table 5.29. When studying the features of all the parents
from the five generations of uniform µGAs, only four types of parameters
— (+, +), (–, +), (+, –), and (–, –) — are found among these parents. These
are selected as the better zones in the parameter space. At the second step,
these four sets of parameters are considered as the four initial points. The
BCLSF is applied four times, each starting from a different initial point. The
results from BCLSF are shown in Table 5.30.
All these solutions could be considered the local minima of the function;
the global minimum can be found from these solutions by comparing their
corresponding function values. The global solution is easily found to be (3.0,
2.0)T and has a function value equal to zero, as shown in bold fonts in Table
5.30. The required number of function evaluations to convergence of the
BCLSF is very few (about six function evaluations of each run of BCLSF)
because the initial points are very close to these minima.
For the present method, 25 function evaluations in GA runs and 26 function
evaluations in the BCLSF are performed. Therefore, 51 function evaluations
in total are required in the combined method, significantly less than 215
TABLE 5.29
Selected Sets of Better Solutions Close to
Optima from Uniform µGA for the
Numerical Test Defined by Equation 5.48a
Set Number Point (x1, x2) Function Value
1 (3.929, 0.635) 33.45
2 (–4.447, –4.635) 128.73
3 (–4.165, 4.259) 167.90
4 (2.329, –2.706) 78.36
a Results obtained at fifth generation.
Note: Total number of function evaluations in the
GA search stage is 5 × 5 = 25.
Source: Liu, G.R. et al., Comput. Methods Appl. Mech.
Eng., 191, 1909–1921, 2002. With permission.
TABLE 5.30
Results from Gradient-Based Method (BCLSF) for Numerical Test
Defined by Equation 5.48
Set Initial Point Corresponding Function at Function
Number Solution Evaluations Solution Point Value
1 (3.929,0.635) (3.0,2.0) 8 0.000
2 (–4.447,–4.635) (–3.763,–3.266) 6 7.367
3 (–4.165,4.259) (–2.787,3.128) 6 3.487
4 (2.329,–2.706 (3.581,–1.821) 6 1.504
Note: Total number of function evaluations in the gradient-based method
search stage is 26.
Source: Liu, G.R. et al., Comput. Methods Appl. Mech. Eng., 191, 1909–1921, 2002.
With permission.
function evaluations in the steepest descent method (even for the successful
runs), 480 function evaluations of the uniform µGA, and 520 function eval-
uations of the plain GA. This numerical test demonstrates the high efficiency
of the combined method.
Trial parameters
Search
range
(Identified parameters)
(Fitness function)
Error function
Inputs for GA
Output results
GA
Stopping
criterion
Measurement date or simulated
measurements generated by adding
the random noise to the computer-
generated results.
FIGURE 5.34
Flowchart of the computational inverse technique using the GA for the inverse analysis. (From
Han, X. et al., Inverse Probl. Eng., 10(4), 309, 2002. With permission.)
5.10 Remarks
Gen, M. and Chen, R.W., Genetic Algorithms and Engineering Design, John Wiley &
Sons, New York. 1997.
Goldberg, D.E., Genetic Algorithms in Search, Optimization, and Machine Learning, Ad-
dison-Wesley, Reading, MA, 1989.
Haupt, L.R. and Haupt, S.E., Pratical Genetic Algorithms, John Wiley & Sons, Inc., New
York, 1998.
Holland, J.H., Adaptation in Natural and Artificial Systems, University of Michigan
Press, Ann Arbor, 1975.
Krishnakumar, K., Micro-Genetic Algorithms for Stationary and Non-Stationary Function
Optimization, SPIE: Intelligent Control and Adaptive Systems, Philadelphia, PA,
1196, 1989, 289.
Lawrence, D., Genetic Algorithms and Simulated Annealing, Morgan Kaufmann Pub-
lishers, London, 1987.
Levine, D., Users Guide to the PGA Pack Parallel Genetic Algorithm Library, U.S, Argonne
National Laboratory, Argonne,. 1996.
Man, K.F., Tang, K.S. and Kwong, S., Genetic Algorithms: Concepts and Designs, Spring-
er-Verlag, London, 1999.