EC Theory

Questions

What laws described the macroscopic behavior

of GAs?
How do low-level operators give rise to this
behavior?
On what types of problems are GAs likely to
perform well or poorly?
What does it mean for a GA to perform well?
When will a GA outperform other search
methods?
Hollands Schema Theorem

an adaptive system must persistently

identify, test, and incorporate structural
properties hypothesized to give better
performance in some environment.
Schemas are formalizations of these
structural properties
Hollands Analysis Showed

While explicitly calculating the fitness of a

population, a GA implicitly estimates the
average fitness of a much larger number of
schemas implicit parallelism
Those schemas whose fitness estimates
remain above average receive an
exponentially increasing number of trials
Selection increasingly focuses the search on
subsets of the search space with above
average fitness
Schemata

schema
a template
the new gene alphabet {0,1,*}
allows exploration of similarities among
chromosomes
represents all matching strings
Schemata

example:
the schema (* 1 1 1 1 0 0 1 0 0) matches two
strings:
(0 1 1 1 1 0 0 1 0 0) and
(1 1 1 1 1 0 0 1 0 0)
Q. which strings does this schema match?
(0 1 1 * 1 0 1 1 * *)
Schemata

matches 2r strings
r: no. of (*) in schema
a string of length l matched by 2l schemata
for strings of length l total 3l schemata
population size n: between 2l and n*2l schemata
may be represented
Schemata

aboutO(n3) (Hollands estimate) schemata

processed successfully implicit parallelism
even though at each generation computation
proportional to size of the population (O(n)) is
performed
one of the main reasons for success of GAs
Schema Properties

order of schema S: o(S)

no. of fixed (non-*) positions
defining length of schema S: d(S)
distance between first and last fixed string positions
fitness of schema S at time t: f(S,t)
the average fitness of all strings in population
matched by S
Schema Properties

Examples:
S1: (* * * * 0 * *) o(S1)=1 d(S1)=0
S2: (* 1 * * 0 * *) o(S2)=2 d(S2)=3
S3: (0 1 * 0 0 * 1) o(S2)=5 d(S2)=6
Schema Processing
let m(S,t) denote the expected no. of individuals
matched by schema S at time t, what is m(S,t+1)?
Assume:
genotype length: l
proportionate parent selection
one-point crossover
bitwise mutation
generational
Schema Processing
(Selection)

f (S )
m( S , t 1) m( S , t ).n.
j fj

m( S , t 1) m( S , t )
f (S ) j fj
f f
n
Schema Processing
(Crossover)

If crossover site chosen uniformly at random,

schema destroyed with probability:
d (S )
pd
(l 1)
Survival probability of schema S:

d (S )
p s 1 pc
l 1
where pc is the probability of crossing over
Schema Processing
(Crossover)

example:

Consider
A=0111000 matched by
S1=*1****0
S2=***10**
Assume crossover site=2
What happens to both schemata?
 Schema Processing
(Mutation)
for a schema to survive mutation with pm
all specified positions must survive
a single position survives with probability
(1-pm)
schema survives with probability
(1-pm)o(S) which is approximately (1-pmo(S) )

Survival probability of schema S:

ps 1 pm o( S )
Schema Processing

let m(S,t) denote the expected

number of individuals belonging to
schema S at time t

f (S , t ) d (S )
m( S , t 1) m( S , t ) 1 pc pm o( S )
f (t ) l 1
where f (t ) is the average fitness of the population at time t
The Schema Theorem

building blocks receive increasing trials in

subsequent generations of a GA
building block
short (small d)
low-order (small o)
above average schemata (high f(S))
The Building Block Hypothesis

states that combining

short, low-order, above average schemata
yields
high order schemata
that also demonstrate
above average fitness
The Building Block Hypothesis

the fundamental theorem of GAs.

shows in essence how GAs explore
similarities
theoretical arguments against the schema
theorem
research continues
Modelling and Analyzing EAs

schema analysis and linkage analysis

Markov chain analysis
each population is a state in a Markov chain
transition matrices based on pc , pm and selection
Modelling and Analyzing EAs

(Markov chain analysis)

nth generation will certainly contain global optimum
if
parent selection prob. for all individuals >0
survival selection prob. for all individuals >0
elitist survival selection
prob. of creating any solution by variation operators is >0

a GA with pm>0 and elitism will always converge to

global optimum
Modelling and Analyzing EAs

dynamical systems approach

assumes infinite population
statistical
mechanics approach
reductionist approaches
isolating different parts of system to examine
seperately
neglect interaction effects but provides good
insights
Modelling and Analyzing EAs
No Free Lunch Theorems
if averaged over space of all possible problems, all non-
revisiting,black-box algorithms exhibit the same performance
debated however basically accepted
lessons:
if a new algorithm performs best for a specific class of problems,
it will perform poorly at some others
NFL can be circumvented through using problem specific
knowledge