GENETIC ALGORITHMS

FOR ENGINEERING
OPTIMIZATION
H.D. MEHTA
ASSO. PROF (ELECT. ENGG)
L.D. COLLEGE OF ENGG.
AHMEDABAD

WHAT IS OPTIMIZATION ?
Optimization is the process of making

something better
An engineer or scientist comes up with a new
idea and optimization improves on that idea
Optimization consists in trying variations on
an initial concept and using the information
gained to improve on the idea
A computer is the perfect tool for optimization
if the data can be input in electronic format

OPTIMIZATIONCONTD
Feed the computer some data and out comes

the solution
Is this the only solution? Often times not
Is it the best solution? Thats a tough question
Optimization is the math tool that we rely on
to get these answers

OPTIMIZATIONCONTD
Our lives confront us with many opportunities for

optimization
What time do we get up in the morning so that we
maximize the amount of sleep yet still make it to
work on time?
What is the best route to work?
Which project do we tackle first?
Optimization is the process of adjusting the inputs
to or characteristics of a device, mathematical
process, or experiment to find the minimum or
maximum output or result

BASICS OF GENETIC ALGORITHMS

Genetic algorithms are computerized search and optimization

algorithms based on the mechanics of natural genetics and

natural selection
GAs are based on Darwins theory of survival of fittest.
Professor John Holland of the University of Michigan, envisaged
the concept of these algorithms.
Most of GA studies are available through few books (Davis,
1991; Goldberg, 1989; Holland, 1975; Michalewicz, 1992) and
through number of conference proceedings (Belew and Booker,
1991; Forrest, 1993; Grefenstette, 1985, 1987; Rawlings, 1991;
Schaffer, 1989, Whitely, 1993)
The basic elements of natural geneticsreproduction,
crossover, and mutationare used in the genetic search
procedure.
2/18/16

DIFFERENCE BETWEEN GA AND

TRADITIONAL OPTIMIZATION
METHODS
A population of points (trial design vectors) is used for

starting the procedure instead of a single design point. If the

number of design variables is n, usually the size of the
population is taken as 2n to 4n. Since several points are
used as candidate solutions, GAs are less likely to get
trapped at a local optimum.
GAs use only values of objective functions. The derivatives
are not used in the search procedure.
In GAs the design variables are represented as string of
binary variables that correspond to chromosomes in natural
genetics
The string length can be varied to achieve any desired
resolution.
The objective function value corresponding to a design
2/18/16
6
vector plays the role of fitness in natural genetics.

DIFFERENCE BETWEEN GA AND

TRADITIONAL OPTIMIZATION METHODS
In every new generation, a new set of strings

is produced by using randomized parents

selection and cross over from the old
generation (old set of strings). Although
randomized, GAs are not simple random
search techniques. They efficiently explore
new
combinations
with
the
available
knowledge to find a new generation with
better fitness or objective function value.

2/18/16

Key terms
Individual - Any possible solution
Population - Group of all individuals
Search Space - All possible solutions to the

problem
Chromosome - Blueprint for an individual
Trait - Possible aspect (features) of an
individual
Allele - Possible settings of trait (black, blond,
etc.)
Locus - The position of a gene on the
chromosome
Genome - Collection of all chromosomes for

Chromosome, Genes and

Genomes

Genotype and Phenotype

Genotype:

Particular set of genes in a genome

Phenotype:

Physical characteristic of the genotype

(smart, beautiful, healthy, etc.)

Genotype and Phenotype

REPRESENTATION OF DESIGN
VARIABLES
In GAs the design variables are represented as string of

binary variables, 0 and 1. For example if a design variable xi

is represented by a string of length four such as 0101, then
its decimal equivalent will be 120 + 021 + 122 + 023 = 5
If each design variable xi , i = 1,2,..n is coded in a string of
length q a design vector is represented using of string of
total length nq. The length of the string is usually
determined according to the desired solution accuracy.
fact the number of binary digits needed to represent a
continuous variable in steps (accuracy) of x, q can be
computed from the relation
2q
+ 1

2/18/16

12

REPRESENTATION OF DESIGN
VARIABLES
If a continuous variable x with bounds 1 and 5 is to be

represented with a accuracy of 0.01, we need to use a binary

representation with q digits where

2q

+1=

401 or q =9

Conversely, if a variable x (whose bounds are given by xu

and xl) is represented by a string of q binary numbers, its

decimal value can be computed as
xi = xil +

2/18/16

decoded value of string si

13

FITNESS FUNCTION
As GAs mimic the survival of the fittest principle of nature to

make a search process, they are naturally suitable for

maximization problems. Minimization problems are usually
transformed into maximization problems by some suitable
transformation.
In general, a fitness function F(x) is first derived from
objective function and is used in successive genetic
operations
For minimization problems, the fitness function is an
equivalent maximization problem chosen such that the
optimum point remains unchanged. The following fitness
function is often used
F(x) = 1/(1+ f(x))
The fitness function value of a string is known as strings
2/18/16
fitness.

14

GA OPERATION
The operation of GA begins with a population of random

strings representing design variables or decision variables

Thereafter, each string is evaluated to find the fitness value.
The population is then operated by three main operators
reproduction, crossover and mutation to create new population
of points. The new population is further evaluated and tested
for termination
If the termination criterion is not met, the population is
iteratively operated by above three operators and evaluated.
This procedure is continued until the termination criterion is
met
One cycle of these generations and the subsequent evaluation
procedure is known as generation in GAs terminology
2/18/16

15

GA OPERATORS
(1) REPRODUCTION (2) CROSSOVER (3) MUTATION
REPRODUCTION
Reproduction is the first operation applied to the population

to select good strings (designs) of the population to form a

mating pool. The reproduction operator is also called the
selection operator because it selects good strings of the
population
The reproduction operator is used to pick above-average
strings from the current population and insert their multiple
copies in the mating pool based on a probabilistic procedure
If Fi denotes the fitness of the ith string in the population of
size n, the probability for selecting the ith string for the
mating pool (pi ) is given by
pi = .. 1.1
2/18/16

16

REPRODUCTION..CONT
D
The implementation of the selection process given by

Equation 1.1 can be understood by imagining a roulette

wheel with its circumference divided into segments, one for
each string of the population, with the segment lengths
proportional to the fitness of the strings
By spinning the roulette wheel n times (n being the
population size) and selecting, each time, the string chosen
by the roulette-wheel pointer, we obtain a mating pool of
size n.

2/18/16

17

REPRODUCTION..CONT
D
In previous fig., the population size is assumed to be 6 with fitness

values of strings 1,2,3,4,5 and 6 given by 12, 4, 16, 8, 36 and 24

respectively. Since the fifth string has the highest fitness value, it is
expected to be selected most of the time (36% of the time,
probabilistically),
when the
roulette wheel
is spun nRange
times
String Fitness
Probability
Cumulativ
No

2/18/16

value

of selection

e
Probability

of Cum.
Prob

12

0.12

0.12

0.00-0.12

0.04

0.16

0.12-0.16

16

0.16

0.32

0.16-0.32

0.08

0.40

0.32-0.40

36

0.36

0.76

0.40-0.76

24

0.24

1.00

0.76-1.00

18

REPRODUCTION..CONT
D
Suppose the random numbers generated are

given in table below, then the strings which

are copied to the mating pool are given below
Random
Number
Generated

0.41

0.10

0.42

0.80

0.67

0.39

0.63

0.53

0.86

0.88

0.75

0.55

String Number
to be copied to
mating pool

2/18/16

19

SRWS
Solution, i

Fi

pi

Pi

pi N

25.0

0.25

0.25

1.25

5.0

0.05

0.30

0.25

40.0

0.40

0.70

2.00

10.0

0.10

0.80

0.50

20.0

0.20

1.00

1.00

EXAMPLE
Let us consider a simple can design problem
A cylindrical can is considered to have only

two parameters- diameter d and height h

Assume that can needs to have a volume of
at least 300 ml
Minimize
f(d,h) = c ( d2/2 + dh)
subject to
g1 (d,h) = d2h/4 300
Variable bounds dmin d dmax

hmin h hmax

EXAMPLE.contd
Suppose (d,h) = (8,10)
Chromosome = 01000 01010
Natural chromosomes are made up of many

genes, each of which can take many different

allele values (such as, the gene responsible
for the eye colour in a persons chromosomes
may be expressed as black, whereas it could
have been blue or some other colour)
To see how these 10 genes control the
phenotype (the shape) of the can, let us
investigate the leftmost bit (gene) of the

EXAMPLE.contd
A value of 0 at this bit allows the can to have

diameter values in the range [0, 15], whereas

the value 1 allows the can to have diameter
values in the range [16,31]
Clearly this bit (or gene) is responsible for

dictating the slimness of the can. If the allele

value 0 is expressed, the can is slim, while if
the value 1 is expressed the can is fat.

TOURNAMENT
SELECTION
In tournament selection, tournaments are

played between two solutions and the better

solution is placed in the mating pool.
Two other solutions are picked again and
another slot in mating pool is filled with the
better solution.
If carried out systematically, each solution can
be made to participate in exactly two
tournaments.
The best solution will win both times, thereby
making two copies of it in the mating pool

TOURNAMENT SELECTIONcontd
Using a similar argument, the worst solution

will lose in both tournaments and will be

eliminated from the population
In this way, any solution in a population will
have zero, one or two copies in the new
population.
It has been shown that tournament selection
has better or equivalent convergence as
compared to any other reproduction operator
that exists in GA literature

TOURNAMENT SELECTIONcontd

TOURNAMENT SELECTIONcontd

STOCHASTIC UNIVERSAL
SELECTION
In SUS only one random number r is chosen

for the whole selection process

Since N different solutions have to be chosen,
a set of N equi-spaced numbers is created
using the equation
R = { r, r+1/N, r+2/N, ..r+(N-1)/N}
Thereafter, a solution corresponding to each
member of R is chosen from the cumulative
probability values as before

STOCHASTIC UNIVERSAL SELECTION

CROSSOVER
After reproduction the crossover operator is implemented. The

purpose of crossover is to create new strings by exchanging

information among strings of the mating pool.
In most crossover operators, two individual strings (designs) are
picked (or selected) at random from the mating pool generated
by the reproduction operator and some portions of the strings are
exchanged between the strings.
In the commonly used process, known as a single-point crossover
operator, a crossover site is selected at random along the string
length, and the binary digits (alleles) lying on the right side of the
crossover site are swapped (exchanged) between the two strings
The two strings selected for participation in the crossover
operators are known as parent strings and the strings generated
by the crossover operator are known as child strings
2/18/16

30

CROSSOVER.CONTD

For example, if two design vectors (parents), each with a string length of 10, are given

by
(Parent 1) = {0 1 0 | 1 0 1 1 0 1 0 0 1}
(Parent 2) = {1 1 0 | 0 0 1 0 1 1 1 0 1}
The result of crossover, when the crossover site is at 3 is given by
(Child 1) = {0 1 0 | 0 0 1 0 1 1 1 0 1}
(Child 2) = {1 1 0 | 1 0 1 1 0 1 0 0 1}
Since the crossover operator combines substrings from parent strings (which have
good fitness values), the resulting child strings created are expected to have better
fitness values provided an appropriate (suitable) crossover site is selected.
The child strings generated using a random crossover site may or may not be as good

or better than their parent strings in terms of their fitness values.

2/18/16

31

CROSSOVER.CONTD
If the child strings created are worse than their parent strings, it should

not be of much concern to the success of the GAs because the bad child
strings will not survive very long as they are less likely to be selected in
the next reproduction stage (because of the survival-of-the-fittest
strategy used)
The effect of crossover may be useful or detrimental. Hence it is desirable

not to use all the strings of the mating pool in crossover but to preserve
some of the good strings of the mating pool as part of the population in
the next generation.
If pc is the crossover probability 100 pc percent of the strings in the

mating pool will be used in the crossover operator while 100 (1 pc)
percent of the strings will be retained as they are in the new generation
(of population)

2/18/16

32

n-point crossover
Choose n random crossover points
Split along those points
Glue parts, alternating between parents

Uniform crossover
Assign 'heads' to one parent, 'tails' to the

other
Flip a coin for each gene of the first child
Make an inverse copy of the gene for the
second child

MUTATION
The mutation operator is applied to the new strings with a

specific small mutation probability, p m

The mutation operator changes the binary digit (alleles value)

1 to 0 and vice versa.

In the single-point mutation, a mutation site is selected at

random along the string length and the binary digit at that site
is then changed from 1 to 0 or 0 to 1 with a probability of pm
A random number between 0 and 1 is generated/chosen. If the

random number is smaller than p m, then the binary digit is

changed. Otherwise, the binary digit is not changed
2/18/16

35

MUTATION.CONTD
As an example consider the following population of size n = 5 with a

string length 10
1000100011
1011110100
1100001101
1011010010
1110001001

Here all the five strings have a 1 in the position of the first bit.

The true optimum solution of the problem requires a 0 as the first

bit. The required 0 cannot be created by either the reproduction
or the crossover operators. However, when the mutation
operator is used, the binary number will be changed from 1 to 0
in the location of the first bit with a probability of npm.

2/18/16

36

START
1. Create initial random population
2. Set Max. Generation counter
3. Set Generation counter = 1
Evaluate fitness of each string in
population
Create mating pool and find the
number of copies of each string to
be copied to mating pool using
any selection method
Perform crossover and mutation
Generate new population
Increment gen count by one
YES

Is Generation
counter <
Max. gen.?
NO

STOP
2/18/16

37

AN EXAMPLE BASED ON
GA
The objective is to minimize the function

f(x1,x2) = (x12 + x2 11)2 + (x1 + x22 -7)2

in the interval 0 x1,x2 6

Population size =20

String length = 10
Accuracy = (6-0)/(210 1) = 0.006
Chromosome length = 20

Examplecontd

Example.contd
Average fitness of population = 0.008
Maximum fitness = 0.024
Expected count = String fitness/Average fitness
(For first string , expected count = 0.001/0.008 = 0.13)
Probability of selection = Expected count/pop size
(For first string , selection probab = 0.13/20 = 0.007 )
The inferior points have been probabilistically eliminated

from further consideration.

Notice that not all selected points are better than rejected
points
For e.g. 14th individual (fitness value 0.002) is selected but
the 16th individual (fitness value 0.005) is not selected

Example.contd
The average fitness of the new population is

calculated to be 0.015, a remarkable

improvement from that in the initial
population.
The best point in this population is found to
have a fitness equal to 0.050 which is also
better than the initial population
This continues till the maximum allowable
generation is reached or some other
termination criteria is met.

Example.contd

Pop best value v/s no. of

generations

Why do GAs work?

In this section we take an in-depth look at the

working of the standard genetic algorithm,

explaining why GAs constitute an effective search
procedure
For simplicity we discuss binary string

representation of individuals

Notation (schema)
{0,1,#} is the symbol alphabet, where # is

a special wild card symbol

A schema is a template consisting of a

string composed of these three symbols

Example: the schema [01#1#] matches the

strings: [01010], [01011], [01110] and [01111]

Notation (order)
The order of the schema S (denoted by o(S)) is the

number of fixed positions (0 or 1) presented in the

schema
Example: for H1 = [01#1#], o(H1) = 3

for H2 = [##1#1010], o(H2) = 5

The order of a schema is useful to calculate survival

probability of the schema for mutation

Notation (defining
length)
The defining length of schema S (denoted by (S)) is

the distance between the first and last fixed positions in it

Example: for S1 = [01#1#],

for S2 = [##1#1010],

(H1) = 4 1 = 3,
(H2) = 8 3 = 5

The defining length of a schema is useful to calculate

survival probability of the schema for crossovers

Growth of Schema
The growth of schema in one iteration of GA is

goven by the equation

m (H,t+1) m (H,t) ( f(H)/f avg(t) ) [1 - pc((H)/(l 1)) pmo(H)]

Example
Let us apply GA for one cycle to a numerical

maximization problem
Maximize
:sin(x)
Variable bounds
:0x
We will use 5 bit strings to represent variable
x in the range of [0, ], so that string (00000)
represents x=0 and string (11111) represents
x=
We will use pc = 1 and pm = 0 for simplicity

Example
String

DV

f(x)

fi/favg

AC

Mating
pool

01001

0.912

0.791

1.39

01001

10100

20

2.027

0.898

1.58

10100

00001

0.101

0.101

0.18

10100

11010

26

2.635

0.485

0.85

11010

Average favg = 0.569

Examplecontd
Mating
pool

CS
point

String
DV
after CS

f(x)

01001

01000

0.811

0.725

10100

10101

21

2.128

0.849

10100

10010

18

1.824

0.968

11010

11100

28

2.838

0.299

Average favg = 0.710

Examplecontd
Let us consider schema H1 = (1 0 * * * ) and

schema H2 = (0 0 * * * )
For schema H1
m (H,t+1) (1) ( 0.898/0.569 ) [1 1.0/(5 1)) 0.0(2)] = 1.184

The above calculation suggests that the

number of strings representing schema H1

must increase
For schema H2 the estimated number of
copies using the above equation is m (H,t+1)

0.133

Building blocks of GA
The schema H1 for the above example has

only two defined positions (the first two bits)

and both defined bits are tightly spaced (very
close to each other) and contain the near
possible solution
The schemata that are short, low order and
above average are known as building blocks.
While GA operators are applied on a
population of strings, a number of such
building blocks in various parts along the
string get emphasized, such as H1.

Building blocks of GA
Note that H2 is short and low order, it is not an above

average schema. Thus H2 is not a building block.

This is how GAs can emphasize different short, low
order and above average schemata in the population.
Once adequate number of building blocks are present in
a GA population, they get combined together due to
action of GA operators to form bigger and better
building blocks.
This process finally leads a GA to find the optimal
solution.
This hypothesis is known as Building Block Hypothesis