Beruflich Dokumente
Kultur Dokumente
Application to Adaptive
Automata for Game
Theory
Rawan Ghneamat
Master's degree in Computer Science
Al-Balqa' Applied University
May 2005
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Contents
Chapter 1
Introduction...........................................................................................................................4
Chapter 2
Genetic Algorithms presentation.........................................................................................5
2.1. Evolutive systems background.......................................................................................5
2.2. History of evolutive systems...........................................................................................5
2.3. Genetic algorithms as tools for optimization problems..................................................6
2.4. Genetic Algorithms: the variation-selection principle......................................................8
2.5. Genetic Operations ......................................................................................................10
2.5.1. Reproduction/ duplication.......................................................................................................................10
2.5.2. Crossing-over..........................................................................................................................................10
2.5.3. Mutation..................................................................................................................................................10
2.5.4. Schematic vue of the operators..............................................................................................................10
2.6. A first hand made sample ............................................................................................11
2.7. General algorithm ........................................................................................................12
2.8. Variants.........................................................................................................................13
2.8.1. New genetic operators derived from the basic ones..............................................................................13
2.8.2. Ecological kennels..................................................................................................................................13
2.8.3. Preselection ...........................................................................................................................................14
2.8.4. Evolutive strategies.................................................................................................................................14
Chapter 3
Genetic Algorithm Implementation....................................................................................15
3.1. Basic overview of genetic libraries ..............................................................................15
3.2. Description of a GAlib...................................................................................................16
3.2.1. GAlib programming concepts features...................................................................................................16
3.2.2. Genetic concepts supported by GAlib....................................................................................................16
3.2.2.1. Individual based representation..........................................................................................................16
3.2.2.2. Genetic operators................................................................................................................................16
3.2.2.3. Population evolution management......................................................................................................16
3.2.3. A first basic exemple of C++ program....................................................................................................17
3.2.4. Class hierachy........................................................................................................................................17
3.3. Example of use for classical optimization problems.....................................................18
Chapter 4
Genetic operators on automata..........................................................................................19
4.1. Automata: from abstract computing devices to operational process............................19
2/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
3/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Chapter 1
Introduction
This work is based on the usage of genetic algorithms. This kind of algorithms
was proposed in the last decades (their origins are usually according to John
Holland in the end of 1960s) as metaheuristics which goals are to obtain
solutions to optimization problems. They can be adapted to hard non linear
problems where local optima exist. Based on biological inspired methods,
genetic algorithms consist of buiding a suited metaphor of the life evolution. In
such algorithms, we use in one hand individual primitive data called
chromosomes which are usually a kind of sequence and in another hand a
population representation for these chromosomes. Genetic operators processes
on individual data and selection operators processes on chromosomes
population.
The aim of this work is to show the basis of genetic algorithms and their use on
evolvable automata which can be models for game theory and economical
modeling.
4/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Chapter 2
Genetic Algorithms presentation
We present here the aim of evolutive systems and genetic algorithms. We
explain the basis of the genetic operators and the classical applications to
optimization problems.
One of the charateristic of living is that individuals have not been programmed
to answer to a specific problem but they evolve inside their environment on the
impulse - among another - of adaptation.
The aim of evolutive systems is to buid algorithms based on these concepts and
models.
● The "German" school which introduced with Rechenberg [8] the concept of
evolution strategies, a method used to optimize real-value parameters, for
example and initialy concerning airfoils. Fogel, Owens and Walsh [9]
5/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
● The "American" school led by John Holland [3] and his students from the
University of Michigan. J. Holland is usually considered as the inventor of
genetic algorithms as we define them in the following. The original goal of J.
Holland is to study the phenomenon of adaptation as it occurs in nature.
From this study concerning natural phenomenon, he wants to design artificial
systems that retain the important mechanisms of natural systems [1]. He
proposes a original life simulation called "echo" [24].
Beside the original goal of J. Holland, Genetics Algorithms (GA) are usually
considered as tools for a search and optimizing methodology [1]. In such
problems, we have to find the best parameters set characteristic of an optimal
situation.
● building a recurrent sequence which follows the slope of the function. For
exemple, the gradient method follows such process.
6/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
optimize has only one optimum as in the illustration 1, analytical methods are
able to find the good solution. If the function has more than one optimum as in
illustration 2, analytical methods may generate sequences that converge
towards a local optimum or a saddle point (for which the gradient is also zero)
and not toward the global optimum. In this illustration, if the initial approximation
is nearer the local optimum then a gradient-type method will follow the slope
toward the local optimum and are not able to found the global optimum.
More generally, many functions to optimize can be the results of previous non
uniform processes and can look like the illustration 3. In this case, such
optimization problems are not in the scope of analytical methods because they
can fall in various local optima, without finding any global one.
7/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
● the simulated annealing uses random processes to help to guide its form of
search for minimal energy states [10];
Following D.E. Goldberg [1], GAs have good robustness properties and they
differ from classical optimization methods on the 4 following aspects:
● GAs work with a coding of the parameter set, not on the parameters
themselves
● GAs search from a population of points, not a single point,
● GAs use objective function information, not partial derivatives or other
auxiliary knowledge,
● GAs uses probabilistic transition rules, not deterministic rules.
From the capabilities to solve optimization problems, GAs can find practical
applications in a great amount kind of domains and for exemple in the following
ones [6]:
● Filtering
● Computational intelligence
● Speech recognition systems
● Production planning and scheduling problems
● Communication systems
The major aspect of genetic algorithms is the adaptation property. This property
can be considered as the search of the optimum of a specific function.
8/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
● The variation step concerns the basic genetic operators on the individual
level and so this step acts on the chromosomes. These basic operators are
described in the following section. The result gives another population with a
greater number of chromosomes than the initial one.
● The selection step concerns the population level inside which a selection
function modifies the whole population constitution. This step leads to keep
only some of the chromosomes that have been generated during the
variation step and which satisfy to specific constraints.
9/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
During the variation step, we usually compute three genetic operators on the
chromosomes:
● Reproduction/duplication,
● Crossing-over,
● Mutation.
The first operator is the reproductive one. This level consists in the copy of the
string or sequence describing chromosomes and generating an identical
chromosomes. This process is made by each chromosom itself, in function of
the value of the selective function which is a kind of measure of efficience.
2.5.2. Crossing-over
2.5.3. Mutation
10/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
We want to search the maximun of the function f(x)=x² on the interval [0; 31].
The performance function for each individual correspond to the function f(x)=x².
We deduct the fitness function as the percentage repartition from these
performances:
11/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Reproduction step
We choose at random, 4 new chains from the existing, the probability is
proportional to the fitness. For exemple, we can obtain: 13, 24, 24 and 19.
Crossing-over step
We take individuals two by two. We cut the chromosoms at one position
choosen at random and then we cross the cuted parts:
Mutation step
We choose at random to modify one binary position in the chain, using a
predefined mutation probability (usually low). In this sample, we take it as 0,05.
So over the 20 binary positions of the 4 individuals, we suppose that only one
binary position is modified:
12 : 01100 is transformed using the mutation on the third binary position and
we obtain 01000 : 8
New generation
After the whole set of transformations, the new generation is the following: 8,
21, 27 and 16. If we sum the performances, we obtain 1490 for this new set and
previously, we obtained 1170.
The following abstract algorithm gives the accurate scheduling of the whole set
of operators which allows to implement a genetic algorithm for a general
problem. The first step consists in defining the elementary chromosoms which
support the evolutionary aspect of the problem.
12/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Genetic algorithm()
{
t:=0; // initial time
initPopulation(P) // initialisation of the
// chromosoms population
evaluate(P) // evaluate the fitness of
// all chromosoms
while not terminated do // evolution cycle
{
t:=t+1;
P':=generate(P) // augmented population obtained by
// reproduction step
cross_over(P') // crossing over step
mutate (P') // mutation step
evaluate (P') // P' chromosoms fitness evaluation
P:=select(P') // selection step
}
}
2.8. Variants
There are many variants on the basic algorithm which has been defined
previously. We give in the following some of them.
● crossing over using a binary mask. The mask values indicate the crossing-
over positions
● boolean function
● ...
This variant is inspired from ecological kennels and species theories. A part of
the precedent population is saved in the new one respecting these theories
13/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
2.8.3. Preselection
This variant concerns the selection step. In this one, a child (eg a chromosome
that has been created in the present iteration) replaces its parent if its
performance is better, elsewhere the parent is kept and not the child.
This variant is a major one and represents the german school of evolutive
algorithms. These approaches have been developed in concurrence whith the
american J. Holland school. In fact these two classes are quite similar and
mainly differ in the following concepts:
● The individuals are not coded with chromosomes but with real values;
● The mutation operator is a major one and mainly acts. The crossing-over
operator is less major and simply acts to assure a good population diversity
when needed.
14/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Chapter 3
Genetic Algorithm Implementation
We focus our attention in this chapter on the implementation of genetic
algorithms. The selected library used is called GAlib, initially developped by
Matthew Wall in the Massachusetts Institute of Technology. We describe here
the functionalities of this library and show its use for some classical optimization
problems.
15/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
GAlib is a C++ library for genetic algorithms programming. This library allows
the user to define the genome/chromosome class corresponding to the
individual data representation of the evolution problem. The library allows also
the user to define the genetic operators and to manage the population
evolution. There is many built-in representation or operators that make a easy
use and implementation of genetic algorithms. Using the object paradigm and
the inheritage concept, the user can also redefine or overload many of these
buil-in buidings and so adapt the basic concept to more specific ones.
● Buit-in chromosome type can be used like real number array, list, tree, 1D,
2D or 3D array, binary strings of variable length, ...
● Various built-in mutation operators and crossover operators are defined and
can be used.
16/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
step:
float Objective(GAGenome&);
main()
{
// create a genome:
GA1DBinaryStringGenome genome(length, Objective);
GASimpleGA ga(genome); // create the genetic algorithm
ga.evolve(); // do the evolution
cout << ga.statistics() << endl; // print out the results
}
float Objective(GAGenome&)
{
// your objective function goes here
}
GAGeneticAlgorithm
GASteadyStateGA (overlapping populations)
GASimpleGA (non-overlapping populations)
17/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
GAGenome
GA1DBinaryStringGenome
GABin2DecGenome
GA2DBinaryStringGenome
GA3DBinaryStringGenome
GA1DArrayGenome<>
GA1DArrayAlleleGenome<>
GAStringGenome (same as GA1DArrayAlleleGenome<char>)
GARealGenome (same as GA1DArrayAlleleGenome<float>)
GA2DArrayGenome<>
GA2DArrayAlleleGenome<>
GA3DArrayGenome<>
GA3DArrayAlleleGenome<>
GATreeGenome<>
GAListGenome<>
18/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Chapter 4
Genetic operators on automata
In this chapter, we deal with automata which are usually considered as the
basis of abstract computer devices. Inside the huge classes of automata, we
focuse our attention on the automata with outputs, and specificaly the automata
with multiplicities which allow the definition of powerful operators. This class of
finite state machines [12, 15, 20] is sufficiently versatile (by means of the
scalars and transition matrices) to cover the NFA (Nondeterministic Finite
Automata - with B as set of output), the finite state Markov chains (matrices are
stochastic), transducers, discrete events systems, etc. Using their
representation with matrix formulation, we obtain an effective descriptions on
which we define some genetic operators based on the original approaches
defined in [18].
Automata are initially considered as theorical tools. They are created in the
1950's following the works of A. Turing who previously deals with the definition
of an abstract "machine". The aim of the Turing machines is to define the
boundaries of what a computing machine could do and what it could not do.
The first class of automata, called finite automata corresponds to simpler kinds
of machines. They are studied by a great number of researchers as abstract
concepts for computable building. In that aspect, we can recall the works of
some linguist researchers, for example N. Chomsky who defined the study of
formal grammars.
Following [12], we can define a finite automaton, also called Finite State
Machine (FSM) as a model composed of states and transitions. A transition
links to states and represents the way to go from one state to another state. The
transition is characterized by a condition that would need to be fulfilled to enable
the transition. In illustration 6 from [17], we show a two-state automaton which
represents the 2 states of a door (open or close) and the 2 transitions between
these states.
19/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
● I ⊂Q is the set of initial states, that is, the states in which the automaton is
when no input has been yet processed;
20/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
In the illustration 7 from [17], we represent an automaton with one initial state,
labelled with 1 and two final states, labelled 6 and 7. This automaton allows to
recognize the word "nice".
● In deterministic automata, for each state there is atmost one transition for
each possible input and only one initial state.
21/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
● Meally machines: Output depends on input and state, i.e. the FSM uses
only input actions. The use of a Mealy FSM leads often to a reduction of
the number of states.
An automaton with multiplicities is based on the fact that the output data of the
automata with output belong to a specific algebraic structure, a semiring. In that
way, we will be able to buid effective operations on such automata, using the
power of the algebraic structures of the output data. And we are also able to
describe this automata in matrix representation with all the power of the linear
algebra.
● { }
Q= S 1 ,S 2 ,. . . ,S n is the finite set of state;
● I:Q K is a function over the set of initial states, which associates to each
initial state a value of K, called entry cost, and to non-initial state a nul value ;
● T:Q K is a function over the set of the final states, which is associated to
each final state a value of K, called final cost, and to non-final state a nul
value;
22/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Remark 2: We have not yet, on purpose, defined what a semiring is. Roughtly it
is the least structure K ,, x that allows matrix computation with units (one
can think of a ring without the "minus" operation'
In this illustration, the input states are characterized with input arrows from
nothing and the output states are characterized with output arrow to nothing. On
each transition, we indicate before the semicolumn, the input data and after the
semicolumn, the output associated for each input data.
23/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
λ= 0 . 5 ; 0 ; 0 ; 0 ; 0 . 5 ; 0
γt = 0 ; 1 ; 0 ;1 ; 0 ; 0
[ ]
0 . 2 0 . 5 0 . 3 0 0 0
0 0 0 1 0 0
0 0 1 0 0 0
μ a =
0 0 0 1 0 0
0 0 0 0 1 0
0 0 0 0 0 1
[ ]
1 0 0 0 0 0
0 0 0 . 2 0 .8 0 0
0 0 1 0 0 0
μ b =
0 0 0 0 .8 0 . 2 0
0 0 0 0 1 0
0 0 0 0 0 1
The genetic algorithm over the population of automata with multiplicities follows
a reproduction iteration broken up in three steps:
24/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Finally the whole genetic algorithm scheduling for a full process of reproduction
over all the population of automata is the evolutionary algorithm:
1. For all couple of automata, two children are created by duplication, crossover
and mutation mechanisms;
3. For all 4-uple composed of parents and children, the performless automata,
in term of fitness computed in previuous step, are suppressed. The two
automata, still living, result from the evolution of the two initial parents.
Remark: The fitness is not defined at this level of abstract formulation, but it's
defined corresponding to the context for which the automata is a model.
25/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Chapter 5
Applications of genetic automata to game theory
In this chapter, we deal with some specific domains of applications to game
theory which is one of the major class of models in the new approaches of
modelling in the economical domain. First, we will explain how adaptive
strategies are useful concerning these domains and so how genetic automata
are well-adapted for such modeling. We focus our attention on the "iterated
prisonner dilemma" and we buid an original evolutive probabilistic automaton for
strategy modeling.
Game theory has become since the five last decades a major aspect in
economical sciences modelling and in a great number of domains where
strategical aspects has to be involved.
In game theory, the Nash equilibrium is a kind of optimal strategy for games
involving two or more players, whereby the players reach an outcome to mutual
advantage. If there is a set of strategies for a game with the property that no
player can benefit by changing his strategy while the other players keep their
strategies unchanged, then that set of strategies and the corresponding payoffs
constitute a Nash equilibrium.
We can understand easily that the modelization of player behavior needs some
adaptive properties. The computable model corresponding to genetic automata
are in that way a good tool to modelize such adaptive strategy. We use them
for the game presented in the next paragraph.
26/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
The prisoner dilemma is a two-players game where each player has two
possible actions: cooperate ( C ) with its adversary or defect it ( C ). So, four
outputs are possible for the global actions of the two players. A relative payoff is
defined relatively to these possible outputs, as described in the following table
where the rows correspond to one player behaviour and the columns to the
other player one.
C C
C (3,3) (0,5)
C (5,0) (1,1)
27/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
The previous automata represent static strategies and so they are not well
adapted for the modelization of evolutive strategies. For this purpose, we
propose a model based on a probabilistic automaton described by the
illustration 11.
This automaton represents all the two-states strategies for cooperation C and
competitive C behaviour of one agent against another in prisoner dilemma.
The transitions are labeled in output by the probabilities p_i of their realization.
The state 1 is the state reached after cooperation action and the state 2 is
reached after defection.
λ= p1 ; 1 − p1
γt = 1 ;1
μ C =
[ p2
p3
1 − p 2
1 − p3 ]
μ C =
[ 1 − p 4
1 − p 5
p 4
p5 ]
28/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
We have to define the fitness in the context of the use of these automata. The
fitness here is the value of the payoff.
At the end of the set of plays, the automaton payoff is the sum of all the payoffs
of each play. This sum is the fitness of the automaton. At the end of these set of
plays, each automaton has its own fitness and so the selection process can
select the best automata. On the end of these selection process, we obtain a
new generation of automata.
29/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
The implementation of a linear algebra library has began with the BLAS libraries
which is considered as the basic functionalities of matrix operations. These
libraries have been developed since many decades and originally for Fortran.
With the use of object oriented programming, the major evolution in this
direction has been made with Lapack++ which proposes a C ++ extension for
BLAS subroutines.
A major evolution of C++ has been made with the concept of template which
allow abstract and reusable programmation. The conceptors of Lapack++ have
begin to make evolve this library toward template concept but finally stop this
evolution. The lapack++ development team finally propose to the people which
want to use the template notion to use another library which is named TNT for
Template Numerical Toolkit.
The goal is to propose reusable componants. The basic data structures are
mainly mutidirectional arrays.
The two libraries GAlib and TNT can be combinated to implement genetic
probablilistic automata which are able to modelize adaptative strategies. This
implementation is under construction. The validation phase and solution
analysis still need a important work which is presently under progress.
30/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Chapter 6
Extensions of Genetic Algorithms
Genetic classifiers are systems which are able to learn specific rules. The
genetic processus allow to optimize these rules performance. These systems
perceive informations from outside as message and the evolution is based on
the systema daptation and reaction to these messages.
● genetic algorithm.
<cond> : <result>
Example:
01#0 : 01111
● The condition means that we select all binary messages composed of 4 bits,
starting with 01 and finishing with 0;
● If the message has been accepted, we send another message which is 0111.
● The rules which are able to select it, make a proposal which is proportional
with their present credit;
31/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
● The attribution system select the rules which make the best proposals and
then make them active;
● An actived rules send its own result message associated with a reward
corresponding to its own credit. The reward is given to the rules which will
select the message.
32/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
In the early (and traditional) implementations of GP, program instructions and data
values were organized in tree-structures, thus favoring the use of declarative
languages that naturally embody such a structure (an important example
pioneered by Koza is Lisp).
33/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Chapter 7
Conclusion
In this study, we present genetic algorithm as a new methodology to explain
evolution processus in one hand and to give a new heuristic to find optima and to
solve search problems.
We presently start the implementation work which is based on the use of TNT and
GAlib in C++ environment. Because of the template ehancement proposed in each
of these libraries, we can buid generic classes to represent automata with
multiplicities in abstract way concerning the set of its output which has only to
respect their belonging to semi-ring.
Such works of modeling and experimentation has novelty aspects. This study has
been made with the coordination of a french research team lead by Prof. G.
Duchamp and C. Bertelle, and including K. Khataneh, PhD candidate who come
from BAU. In that team, beside some implementations using C++, other
implementations are also currently made using a CAS (computer Algebraic
System) called MuPAD. These different works will conduct to some publications
inside international conferences (ECCS 2005 at Paris in November and IEEE-
ICECS 2005 in Tunisia in december).
34/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Chapter 8
Bibliography
8.1. Books
8.2. Articles
35/36
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Wikipedia Encyclopedia:
16.http://en.wikipedia.org/wiki/Genetic_Algorithm
17.http://en.wikipedia.org/wiki/Automata_theory
GA archives:
19.http://aic.nrl.navy.mil/galist
20.http://gaul.sourceforge.net/intro.html
21.http://www.estec.esa.nl/outreach/gatutor/Default.htm
Game theory:
23.http://www.math.wisc.edu/~stefanss/520report.html
25.GAlib: http://lancet.mit.edu/ga/
26.TNT: http://math.nist.gov/tnt/
36/36