Importing the Computational Neuroscience Toolbox into

Neuro-Evolution—Application to Basal Ganglia

Jean-Baptiste Mouret, Stéphane Doncieux, Benoît Girard

To cite this version:

Jean-Baptiste Mouret, Stéphane Doncieux, Benoît Girard. Importing the Computational Neuroscience
Toolbox into Neuro-Evolution—Application to Basal Ganglia. GECCO’10, 2010, Portland, United
States. pp.587-594, �10.1145/1830483.1830592�. �hal-00687639v2�

HAL Id: hal-00687639

https://hal.archives-ouvertes.fr/hal-00687639v2
Submitted on 26 Apr 2019

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est

archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
 Importing the Computational Neuroscience Toolbox into
Neuro-Evolution—Application to Basal Ganglia

Jean-Baptiste Mouret Stéphane Doncieux Benoît Girard

mouret@isir.upmc.fr doncieux@isir.upmc.fr girard@isir.upmc.fr
ISIR, Universit Pierre et Marie Curie-Paris 6, CNRS UMR 7222
4 place Jussieu, F-75252, Paris Cedex 05, France

ABSTRACT
Neuro-evolution and computational neuroscience are two sci-
entific domains that produce surprisingly different artificial
neural networks. Inspired by the “toolbox” used by neu-
roscientists to create their models, this paper argues two
main points: (1) neural maps (spatially-organized identi-
cal neurons) should be the building blocks to evolve neural
networks able to perform cognitive functions and (2) well-
identified modules of the brain for which there exists com-
putational neuroscience models provide well-defined bench-
marks for neuro-evolution.
To support these claims, a method to evolve networks
of neural maps is introduced then applied to evolve neu-
ral networks with a similar functionality to basal ganglia in
animals (i.e. action selection). Results show that: (1) the
map-based encoding easily achieves this task while a direct
encoding never solves it; (2) this encoding is independent of
the size of maps and can therefore be used to evolve large
and brain-like neural networks; (3) the failure of direct en-
coding to solve the task validates the relevance of action
selection as a benchmark for neuro-evolution.

Categories and Subject Descriptors

I.2.6 [Artificial intelligence]: Learning—Connectionism
and neural nets
General Terms
Algorithms
Keywords
evolutionary algorithms; neural networks; computational neu-
roscience; basal ganglia; neuro-evolution.
1. INTRODUCTION
Evolution has been one of the prominent processes to
shape the animal brain; imitating this process by employing
evolution-inspired algorithms to design “artificial nervous
systems” therefore drew considerable attention in artificial
intelligence over the last two decades. This line of thought
drives researchers to propose many methods to exploit evo-
lutionary algorithms (EA) to design neural networks (neuro-
evolution), from efficient direct encodings [44] to grammar-
Author-generated version.
GECCO’10, July 7–11, 2010, Portland, Oregon, USA.
(c) The authors.
Figure 1: Contracting basal ganglia model for action
selection. D1/D2 Str: striatal neurons with D1 or
D2 dopamine receptors; FC: frontal cortex; FS: fast
spiking interneurons; GPe/GPi: external and inter-
nal part of the globus pallidus; S: input saliences;
SNr: substancia nigra reticulata; STN: subthalamic
nucleus; TH: thalamus; TRN: reticular thalamic nu-
cleus.
based generative systems [16, 20, 30, 31] and development-
inspired approaches [4, 43]. Resulting neural networks can
then control robots or artificial agents [28, 33, 31].
Concurrently with these researches, the field of computa-
tional neuroscience (see e.g. [6]) designs models of parts of
the nervous system based on experimental data from biol-
ogy. Among the published models, some of them are effi-
cient enough to be useful in robots [23, 22, 47, 29, 25, 35,
15] and could provide the basic blocks to build a complete
bio-inspired “artificial brain” for artificial intelligence.
Computational neuroscience and neuro-evolution there-
fore share a characteristic—creating artificial nervous sys-
tems; however, the comparison of the resulting neural net-
works is surprising: models produced in computational neu-
roscience display almost no similarities with evolved neural
networks. The latter usually involve up to a few dozens of
neurons and do not scale up to many more neurons, whereas
computational neuroscience often manipulate hundreds of
neurons in a very organized fashion. A detailed investi-
gation of published models (especially [17, 18, 15, 45, 46,
40, 39]) reveals that the chosen building blocks constitute
at least one fundamental difference; evolutionary methods
mostly use individual neurons, ideally organized in modu-
lar fashion [20, 38, 8, 30, 31], but many neuroscience mod-
els (e.g. figure 1) rely on the concepts of maps (a N × M
grid of neurons, in which N and M are free variables of the
model) connected by regular connection schemes (either one
to one connections or one to all with a regular assignation
of weights). This allows such neural networks to scale up
to larger maps (e.g. to handle higher-dimensional inputs)
while maintaining the same overall structure. This descrip-
tion of neural networks as connected maps can be seen as
the result of a developmental process in which a network of
maps is developed to form a neural network.
Besides this analysis, it appears that computational neu-
roscience can also provide an efficient approach to bench-
mark neuro-evolution methods in the context of cognitive
functions. On one hand, neuro-evolution ended up with
substantially good results to control non-linear systems [16,
44, 8], but despite Beer’s preliminary proposals [2, 41], no
clear benchmark has been widely accepted to evaluate their
potential for generating more cognitive functions. On the
other hand, experimental neuroscience isolated several mod-
ules of the nervous systems, and precisely described their
inputs and outputs such that computational neuroscience
can model them. On the road to automatically design ar-
tificial nervous systems, the minimum benchmark for any
neuro-evolution method should be to reproduce the func-
tions modeled by neuro-scientists and to equal the efficiency
of hand-designed neural networks.
Building on this inspiring parallel, this paper argues two
main points:
• Maps (and not individual neurons) should be the build-
ing blocks to evolve neural networks able to perform
cognitive functions.
• Well-identified modules of the brain for which there ex-
ists computational neuroscience models provide well-
defined benchmarks for neuro-evolution.
To support these claims, a method to evolve networks of
maps is described then applied to evolve neural networks
with a similar functionality to basal ganglia (winner-takes-
all selection by disinhibition). Results are compared with an
optimized basal ganglia model from neuroscience and net-
works evolved with a classic direct encoding.
2. RELATED WORK
2.1 Evolving Neural Networks
Most of the work on the structural evolution of neural
networks focuses on the definition of efficient encodings with
their associated genetic operators. In direct encodings, ge-
netic operators directly manipulate connections and neu-
rons. The most successful direct encoding is undoubtedly
NEAT [44], which is based on “innovation numbers” to im-
prove cross-over and structural diversity preservation. De-
spite their simplicity and their efficiency to solve some con-
trol problems, direct encodings fail to scale up to large net-
works, supposedly because they cannot capture the regu-
larity of the search space by using several times the same
sub-networks.
Noticing that biological systems widely rely on the rep-
etition and combination of hierarchically organized mod-
ules [20, 19, 30, 31], several researchers proposed to encode
neural networks indirectly by employing a compact repre-
sentation that is then developed into a neural network. One
of the straightforward way to implement this idea is to use
a list of modules as a genotype, as done in modular encod-
ings [8, 38, 31]. A more complex but potentially more pow-
erful approach is to evolve “construction programs” whose
instructions are interpreted to build neural networks. This
modus operandi was employed in cellular encoding [16] to de-
sign highly repetitive neural networks and in several other
grammar-based generative systems [20, 30, 31].
A last method is to evolve chemical gradients in Cartesian
space that can then be used to compute synaptic weights.
Provided that these gradients displays some regularities (e.g.
repetitive patterns or symmetry), the synaptic weights can
mirror them. This process is currently instantiated in Hy-
perNEAT [43], one of the rare neuro-evolution framework
able to evolve networks of several hundred of neurons. How-
ever, the published method is not designed to evolve the
topology of neural networks1 and requires the experimenter
to fix the number of neurons. Moreover, despite its inspi-
ration from biological development, it remains to be proved
that HyperNEAT can generate neural networks similar to
those observed in real animals.
2.2 EAs and Computational Neuroscience
Current neuro-evolution methods mainly draw from com-
puter science (L-systems [20, 19], formal grammars [16, 30],
graph theory, machine learning, control theory[44]) and more
rarely biological development [4, 43]. To our knowledge, only
a few attempts to explicitly link this field to computational
neuroscience have been published.
The main import from neuroscience to neuro-evolution
is undoubtedly the neuron formulation. Beer et al. [3] ana-
lyzed the evolution of networks of leaky integrators, a neuron
model initially introduced in computational neuroscience,
for “minimal cognitive behaviors”. Models at other abstrac-
tion levels, such as spiking neurons [10], nonlinear oscilla-
tors [27] and neural fields [21], have also been considered
as the building blocks of evolved neural networks. Another
important inspiration from neuroscience to neuro-evolution
is on-line learning schemes such as Hebbian-like rules [12]
and neuro-modulation [24, 42]. It should be emphasized
that these papers mostly copied a brick from neuroscience
and then evolved networks without any biological inspira-
tion about the topology. As a consequence, the resulting
neural networks are topologically different from those hand-
designed by neuro-scientists.
Following a reasoning closer to the one argued here, Ijspeert
et al. [23] imported from Ekeberg et al. [9] the overall orga-
nization and the neuron parameters—both extracted from
animal experiments—of the neural network that generates
the locomotion patterns of lampreys (the Central Pattern
Generator, CPG). In a multi-stage evolutionary process, the
authors first optimized the connectivity and the synaptic
1
HyperNEAT only evolves synaptic weights but connections
with near zero weights can be removed.

Figure 2: Three ways to connect two maps of neu-
rons. (a) one to one; (b) one to all, with the same
synaptic weight; (c) one to all with a Gaussian dis-
tribution of synaptic weights.
weights of segmental oscillators and then the connectivity
between these oscillators. The obtained network oscillates
over a wider range of frequency and amplitude than the
hand-designed neural network [23] and, although they have
only been investigated in simulation, they could be embed-
ded in snake-like robots [22]. By constraining the evolu-
tionary process with biological data, Ijspeert et al. man-
aged to automatically design a large but regular neural net-
work more efficient than a hand-designed controller. This
result concurs with the approach presented here: using data
from neuroscience (connectivity patterns, overall organiza-
tion, common parameters) could lead neuro-evolution to a
new level of functionality.
3. MAP-BASED ENCODING
3.1 Computational Neuroscience Toolbox
A careful examination of the published neuroscience mod-
els, and especially of those that could be employed in artifi-
cial intelligence, allows to extract some regularities in model
descriptions. Neuro-scientists converged to a limited set of
structures that can describe a wide range of biological neural
networks while being simple enough to be analyzed and well
understood. We list here the main features of this “compu-
tational neuroscience toolbox”.
Most neuroscience models use either spiking neurons [14]
or leaky integrators, which only simulate the overall dynamic
of a population of neurons [3]. This latter simple dynamic
neuron has also been widely used in evolutionary robotics
(see [11]) as it shows many different temporal behaviors but
remains computationally inexpensive to simulate. Further-
more, in contrast to neurons employed in neuro-evolution or
in machine learning, neurons manipulated in computational
neuroscience are either inhibitory or excitatory.
In this work, we opt for the lPDS-based (locally Projected
Dynamic System) neuron model [15], a variant of the clas-
sic leaky integrator with similar dynamics but which verify
the dynamic property of contraction [26]. Using the Euler
integration method, the output yt+dt of a lPDS neuron i at
time t + dt is computed as follows:
(i) P (j)
pt = j∈C wi,j yt (1)
(i) (i) (i) (i)
at+dt = max(0, min(1, at + τ1 (pt − at + Ti ) · dt)) (2)
( In this work, we employ straightforward mutation oper-
(i)
(i) at+dt if i is excitatory
yt+dt = (i) (3)
−at+dt otherwise
where τ is the time-constant of neuron i, Ti a threshold
(j)
value, wi,j the synaptic weight between neuron i and j, at
the activation level of neuron j and dt the integration step
(e.g. 0.001 s).
The main building blocks of the considered computational
neuroscience models are either N × M maps of neurons or
individual neurons. Maps are defined as spatially organized
grids of identical neurons (same time constant, same thresh-
old, same inhibitory status). Many models employ only
maps with the same dimension, arbitrary fixed to the di-
mension of the input. At any rate, the number of map di-
mensions in a given model is very restricted as it typically
covers only the dimension of the inputs and those of the
outputs.
Connection schemes between maps are restricted to three
cases (figure 2) in most models: (1) one to one connection
with constant weights (neuron i of map M1 is connected to
neuron j of map M2 , with a positive weight identical for
each connection), (2) one to all connections with constant
weights (neuron i of map M1 is connected to each neuron
of map M2 , with identical weights for all connections) and
(3) one to all connections with weights following a Gaussian
distribution computed as follows:
||i − j||2
di,j = , i ∈ M1 , j ∈ M2 (4)
N
−di,j
wi,j = Γ exp( 2 ), i ∈ M1 , j ∈ M2 (5)
σ
where N is the map size2 , i is the i-th neuron of map M1 , j
the j-th of M2 and wi,j the synaptic weight between i and
j. σ (standard deviation) and Γ (weight amplitude) are the
only parameters that can be changed.
This regular assignation of weights (either constant or fol-
lowing a simple distribution) contrasts with typical neuro-
evolution methods in which each synaptic weight is set-up
separately. In effect, this adds numerous regularities in neu-
ral networks that are similar to those sought by modular
and developmental encodings.
This computational neuroscience toolbox is sufficient to
describe a wide range of models, for instance models of basal
ganglia [17, 18, 15], of colliculus [45] and of visual attention
[46, 40, 39]. Additions such as Hebbian learning and neural
fields would allow to describe many other models.
3.2 Evolving a Graph of Neural Maps
Most computational neuroscience models can be described
as a graph of neural maps in which each map and each con-
nection is described by a set of parameters (time constant of
neurons, connection scheme, synaptic weights, etc.). Such
a labeled graph can be modified structurally (add/remove a
connection or a node) and parametrically (change of a la-
bel). We first describe a basic method to evolve a generic
labeled graph then we introduce the set of labels employed
to describe a graph of neural maps.
3.2.1 Evolving a Labeled Graph
The graph is represented as a classic adjacency list for
which mutation operators have been designed; cross-over is
not used.
ators that apply each of the possible changes with a user-
defined probability (see appendix to know the chosen val-
2
It is here assumed that all the maps have the same size.
This formula is easily generalized to the case of two maps
with different sizes.
Figure 3: Overview of the development process. From left to right: (1) the genotype is a labeled graph with
evolvable labels; (2) the labels are interpreted to a neuroscience-inspired description of the neural network;
(3) for a given size of maps, this neural network can be fully developed into a neural network (for instance
to evaluate its fitness).

ues). Three structural mutation operators, inspired by those Similarly, connections are labeled with four evolved real
defined in NEAT [44], have been designed: numbers {c, k, Γ, σ} ∈ [0, 1)4 that are interpreted as follows:
• addition of a node on an existent connection, with ran- 
1 to 1, if c < 0.5
dom labels; the connection is split in two and the two Scheme: (10)
1 to all, otherwise
parts keep the same labels; 
Gaussian if c > 0.5 and k < 0.5
• removal of a random node and its associated connec- Function: (11)
constant otherwise
tions; 
Γ if c > 0.5 and k < 0.5
• addition/removal of a connection between two random Parameter 1: 1 (12)
5
w (synaptic weight) otherwise.
neurons. 
σ if c > 0.5 and k < 0.5
Nodes and connections can be labeled by a list of real pa- Parameter 2: (13)
otherwise: ignored
rameters that represent weights, threshold, neuron type, ...
These parameters are mutated using polynomial mutation [7]. 3.3 Classic Direct Encoding
To initialize a random graph, a random number of nodes
The previously described evolvable labeled graph can also
(with random labels) are first created then connected by a
be employed in a more classic fashion to directly define a
random number of connections (also with random labels).
neural network. This leads to a direct encoding of neural
Connections cannot be doubled. Last, graphs are simplified
networks. In this case, each node describes a neuron (instead
by removing each sub-graph not connected to both an input
of a map). Labels used to describe neurons are the same as
and an output.
those used to describe maps (time constant, etc.). Connec-
3.2.2 Map-based Encoding tions are labeled with a single real number interpreted as
the synaptic weight. This direct encoding will be used as a
To evolve a network of maps inspired by the neuroscience
reference encoding in the next section.
toolbox, the previously defined operators are employed with
a particular labeling of nodes and connections (figure 3).
This representation is independent of the size of maps, that 4. EXPERIMENTS
is the same genotype can be developed into a neural network
with an arbitrary number of neurons. 4.1 Action Selection in Basal Ganglia
In the current implementation, nodes are labeled with four Having introduced the map-based encoding, we now focus
evolved numbers {e, v, T, a} ∈ [0, 1)4 that are scaled or bi- on evaluating its ability to generate brain-like structures; to
narized to obtain nodes’ parameters. They are interpreted that aim, we chose to reproduce the function performed by
as follows3 : the basal ganglia (BG).
 More precisely, the basal ganglia are a set of intercon-
inhibitory if e < 0.5 nected subcortical nuclei [36], that are thought to be in-
Type: (6)
excitatory otherwise. volved in action selection [37, 34], i.e. the problem, for an
Time constant (τ ): τ = vbτ 0 ×4c (7) agent, of choosing which action to perform within a reper-
−3 −3 −3
toire, given internal and external sensory information, in
where v = [5 · 10 , 10 , 20 · 10 , order to achieve its goals. Solving this generic resource al-
40 · 10−3 ] location problem seems to be a central cognitive function as
Threshold: 10 · (T − 0.5) (8) the BG circuitry is common to all vertebrates, and as it con-
 tains duplicated circuits [1] dedicated to many critical func-
isolated neuron if a > 0.75 tions (skeleton movements, eye movements, sequence learn-
Map type: (9)
standard map otherwise. ing, working memory, navigation, planning, etc.). The BG
3
bxc denotes the floor of x. performs the two main aspects of action selection: the cen-
tral process of selection of one action only among conflicting
ones, similar to a winner-takes-all (WTA), and the learn-
ing process necessary to bias the selection process towards
the most profitable option. We focus here on the WTA-like
process.
The BG circuits are subdivided in parallel channels, which
are supposed to be associated to the actions of the reper-
toire, and which receive convergent inputs from the cortex.
At rest, the BG output is tonically active and inhibits its tar-
gets. The selection of an action, caused by a strong conver-
gent input on the associated channel signaling its urgency,
causes a pause of the channel output inhibition, allowing the
activation of the targeted brain region [5]. Various models of
this WTA process have been proposed; we consider here the
CBG model proposed in [15], which derives from the new
functional interpretation of the circuitry proposed by [17],
but adds some previously unused connections (figure 1).
In the following, we test the ability of our new map-based
encoding to evolve circuits exhibiting the WTA ability ex-
pected from BG models, and compare the obtained results
with CBG models whose parameters4 are optimized by a
basic evolutionary algorithm.
4.2 Fitness
In the remaining text, the following notations are used:
• x: a developed individual (a neural network);
• k: number of inputs (also the number of outputs);
• T : the maximum simulation duration;
• γ(x, v, t)i : activation level of the output neuron i (i ∈
{1, ..., k}) at time t (t ∈ [0, T ]), given the input vector
v of saliences (v ∈ Rk );
• Tc (x, v): the duration of the simulation for the input
4.2.1 Objective Function
The main objective function (fitness) aims at checking
that the maximum salience corresponds to the minimum
activation in the corresponding output neuron (and conse-
quently to the less inhibited action). Furthermore, we are
interested in the best contrast possible, that is the selected
action should be as little inhibited as possible and the other
actions should be as inhibited as possible. To formulate this
idea, let first define min (v) (the index of the maximum value
of v) and mout (x, v) (the index of the minimum value of the
output vector, for the input v):
min (v) = arg max(vi ) (18)
i∈1,··· ,k
mout (x, v) = arg min(γc (x, v)i ) (19)
i∈1,··· ,k
We also need the index of the second minimum value of the
(2)
output vector, mout (x, v):
(2)
mout (x, v) = arg min (x, v)(γc (x, v)i ) (20)
i∈1,··· ,k,i6=mout
Let I a set of N random salience vectors in [0.1, 1], uni-
formly distributed. Given that an action will be selected if
the corresponding output channel is below γ0 (x), we define
the fitness to be maximized as the average selection success
(the correct channel was disinhibited) weighted by the differ-
ence between the minimum output (the selected action) and
the second minimum output (the first inhibited action). If
all outputs except the minimum one are above γ0 (x), γ0 (x)
is used to compute the difference. We thus foster the maxi-
mum inhibition of the selected channel but let the network
free to use any value above γ0 (x) for the other channels. The
fitness is normalized such that the maximum value is 1.
vector v (see K(x) below); 1 X 
F (x) = δmin ,mout (x, v) · γc (x, v)mout ... (21)
|I| · γ0 (x) v∈I
• γc (x, v)i : activation level of the output neuron i (i ∈
(2)


{1, ..., k}) at the end of the simulation (i.e. t = Tc (x, v)); ... − min(mout (x, v), γ0 (x))
• γ0 (x) the average activation level of the output neurons where δmin ,mout (x, v) denotes the Kronecker’s delta:
when the inputs are null, at the end of the experiment: 
1 if mout (x, v) = min (x, v)
k δmin ,mout (x, v) = (22)
1X 0 otherwise
γ0 (x) = γc (x, [0, · · · , 0], 1)i (14)
k i=1
In these experiments, N was fixed to N = 1000 and the
same vectors were employed to evaluate all individuals.
Each individual is simulated until its output converges to a
constant vector or until it reaches the maximum number of 4.2.2 Constraints
time-steps (t = T ). From a practical viewpoint, a neural
The search is restricted to networks that converge during
network is considered to have converged when 10 successive
the simulation time and that do not select anything if all the
outputs have a difference of less than ε (in these experiments,
input saliences are low. These restrictions are implemented
ε = 10−6 ). To compute Ts , we first define the “convergence
using three constraints.
function” K(x, t, v):
The first constraint C1 (x) ensures that γ0 (x) is greater
   than a minimum value γmin , fixed to 0.09 in this work (the
 0 if γ(x, v, t)i − γ(x, v, t − n)i  < ε, resting activity of the CBG model is 0.096). This means
K(x, t, v) = ∀n ∈ {1, · · · , 10}, ∀i ∈ {1, · · · , k} that all the actions should be sufficiently inhibited if all the
1 otherwise saliences are null.

(15)
Ts can now be defined as: 
0 if γ0 (x) > γmin
C1 (x) = (23)
Ts (x, v) = t such as K(x, t, v) = 1, t ∈ [0, · · · , T ](16) 1 otherwise
Tc = min(T, Ts (x, v)) (17) The second constraint C2 (x) ensures that small values are
4 filtered (this behavior is observed in animals’ basal ganglia)
the CBG is parameterized by 22 synaptic weights and 3
neuron thresholds by sampling random saliences in [0, 0.1] and checking that
 Table 1: Summary of launched experiments.
Search space Genotype Channels
1 synaptic weights of CBG 25 real numbers 6
2 topology and param. direct encoding 6
3 topology and param. map-based encoding 6
4 synaptic weights of CBG 25 real numbers 15
5 topology and param. direct encoding 15
6 topology and param. map-based encoding 15

Figure 4: Median fitness with regards to genera-

Table 2: Parameters used in the experiments. tion number (200 evaluations for each generation).
Parameter / Genotype Map-based Direct enc.
min./max. nb. of nodes (rand. gen.) 3/5 18 / 25
The map-based encoding achieve the highest fitness
min/max. nb. of links (rand. gen.) 3/5 18 / 25 while the direct encoding does not solve the task
prob. to add/remove a node 0.05 / 0.05 0.05 / 0.05 (median fitness is negative for direct encoding with
prob. to add/remove link 0.05 / 0.05 0.05 / 0.05 15 channels, therefore it does not appear on the
prob. to change each label 0.1 0.1
σ for gaussian mutation 0.05 0.05 graph).

1
0.8
the outputs are greater than γ0 (x): 0.6
 0.4
 0 if γc (x, v)i > γ0 (x),
0.2
C2 (x) = ∀v ∈ [0, 0.1]k , ∀i ∈ {1, · · · , k} (24)
0

1 otherwise CB ma dir CB ma dir
G  p e G p e
(6) -bas ct en  (15 -bas ct en
ed c.  ) ed c. 
Last, C3 (x) checks that the tested neural network con-  (6 (  (1 (15
) 6) 5) )
verges to a constant output vector before the end of the ex-
fit > 0 mean fit.
periment, for all the tests performed on the neural network
(C1 (x), C2 (x) and F (x)):
C3 (x) = max(K(x), v) (25) Figure 5: Proportion of runs that satisfy the con-
v∈I straints (fitness > 0) after 200, 000 evaluations and
These constraints are enforced with the penalty method average fitness of those runs. All differences are sta-
[7]: an arbitrary large penalty is added to the fitness each tistically significant (Student T-test, p < 0.01) except
time a constraint is violated. Instead of maximizing F (x), between map-based (6) and map-based (15).
we thus maximize Fc (x):
Fc (x) = −K(C1 (x) + C2 (x) + C1 (x)) + F (x) (26)
10
where K is an arbitrary large constant (e.g. 10 ).

4.3 Experiments
To evaluate the relevance of the proposed approach to
solve the action selection task, we ask three questions: (1)
how well does the proposed map-based encoding (section
3.2.2) solve this task compared to an optimized CBG ([15],
section 4.1)? (2) compared to a basic direct encoding (sec-
tion 3.3)? and (3) does it scale up to high numbers of neu-
rons?
To that aim, six different experiments have been launched
(table 1), each of those consisting of 10 independent evolu-
tionary runs, each of them with a budget of 200, 000 evalu-
ations. The same evolutionary algorithm, the same fitness
and the same model of neurons were employed in all experi-
ments; only the genotype/phenotype mapping was changed.
The chosen evolutionary algorithm is a single-objective im-
plementation of NSGA-2 [7], an elitist tournament-based
evolutionary algorithm. Parameters are provided in table 2
and the source code is available online5 . Experiments were
carried in the Sferesv2 framework [32].

Figure 6: Example of a neural network obtained

5. RESULTS with the map-based encoding for 6 channels (fit-
5 ness: 1.0).
http://www.isir.fr/evorob_db
 For six channels, the optimized CBGs reach a fitness of
about 0.8 (after 200, 000 evaluations, median: 0.83, mean:
0.84, s.d.: 0.035; see figures 4 and 5). This means that it
performs well, but some saliences are not well selected or
the contrast is not maximum. Surprisingly, the map-based
encoding leads to better results than this reference point by
achieving a fitness of almost 1 with a low standard devia-
tion (median: 0.951, mean: 0.950, s.d: 0.013). Overall, the
map-based encoding easily and reliably solves the task of
action selection. The discrepancy between map-based en-
coding and the reference point (optimized CBG) probably
stems from the many biological constraints on the design of
the CBG.
With the basic direct encoding, about 600 generations
(120, 000 evaluations) were required to obtain a positive me-
dian fitness (i.e. more than half of the runs obtained at least
one individual that satisfies the constraint). This contrasts
with the map-based encoding runs, in which only a few gen-
erations were necessary to satisfy the same constraints. Af-
ter 200, 000 evaluations (1000 generations), 80% of the direct
encoding runs satisfy the constraints (figure 5); however, the
mean fitness is low (0.3) and corresponds to individuals that
do not solve the task.
A typical neural network obtained with the map-based
encoding is drawn on figure 6. In addition to the input and
output maps, this network is made of only two hidden maps.
They are linked using a mix of 8 “one to one” and 2 “one
to all” connections (no Gaussian “one to all” connections are
employed; as in the CBG model, they are useless to solve
this problem). This neural network is therefore described
by 4 × 4 + 10 × 4 = 56 real numbers and graph topology.
The same network described using a direct encoding would
require (for six channels) 3+8×6+2×6 = 132 real numbers
in addition to graph topology. This illustrates the intuitive
difference in size of the search spaces using map-based and
direct encoding. The low score achieved by direct encoding
suggests that the subspace explored with the map-based en-
coding is a useful restriction to solve the targeted cognitive
function, in this case action selection.
The 15 channels experiments demonstrate how well the
map-based encoding scales up. Since the neural network
description is independent of the size of the map, results
obtained with 6 or 15 channels are statistically not different.
No runs managed to satisfy the constraints using the direct
encoding. Surprisingly, although the description of the CBG
is also based on maps, the CBG obtained lower fitness values
with 15 channels than with 6 channels. We are investigating
this issue.
6. CONCLUSIONS AND FUTURE WORK
A new encoding to evolve neural networks has been in-
troduced in this paper. It is based on the “computational
neuroscience toolbox” and especially on maps of neurons
connected with regular connections schemes. This encod-
ing is designed to explore with evolutionary algorithms a
subspace of possible neural networks similar to the one ex-
plored in computational neuroscience. The proposed encod-
ing has been compared to a basic direct encoding and to a
hand-designed model on the task of action selection, an im-
portant function of basal ganglia in animals. Results show
that:
• the map-based encoding easily and reliably solves the
action selection task whereas a basic direct encoding
never solves it;
• this description is independent of the size of maps and
can therefore be used to evolve large and brain-like
neural networks;
• the failure of direct encoding to solve the task validates
the relevance of action selection as a benchmark.
The described map-based encoding could easily be im-
proved by adding more “tools” from the computational neu-
roscience literature. Learning rules [12, 24, 42] are probably
the most useful addition. Moreover, other basic tasks of
the brain have to be investigated with the same encoding
such as auto-calibration, working memory [13] or selective
attention [41, 46, 40].
7. ACKNOWLEDGMENTS
This project was funded by the ANR EvoNeuro project,
ANR-09-EMER-005-01.
8. REFERENCES
[1] G. E. Alexander, M. R. DeLong, and P. L. Strick.
Parallel organization of functionally segregated
circuits linking basal ganglia and cortex. Annual
Review of Neuroscience, 9:357–381, 1986.
[2] R. Beer. Toward the evolution of dynamical neural
networks for minimally cognitive behavior. In From
Animals to Animats 4, pages 421–429, 1996.
[3] R. Beer and J. Gallagher. Evolving dynamical neural
networks for adaptive behavior. Adaptive behavior,
1(1):91, 1992.
[4] A. Cangelosi, D. Parisi, and S. Nolfi. Cell division and
migration in a ’genotype’ for neural networks.
Network: Computation in Neural Systems,
5(4):497–515, 1994.
[5] G. Chevalier and M. Deniau. Disinhibition as a basic
process of striatal functions. Trends in Neurosciences,
13:277–280, 1990.
[6] P. Dayan, L. Abbott, and L. Abbott. Theoretical
neuroscience: Computational and mathematical
modeling of neural systems. MIT Press, 2001.
[7] K. Deb. Multi-objectives optimization using
evolutionnary algorithms. Wiley, 2001.
[8] S. Doncieux and J.-A. Meyer. Evolving modular
neural networks to solve challenging control problems.
In Proc. of EIS 2004, 2004.
[9] O. Ekeberg. A combined neuronal and mechanical
model of fish swimming. Biological cybernetics,
69(5):363–374, 1993.
[10] D. Floreano, Y. Epars, J. Zufferey, and C. Mattiussi.
Evolution of spiking neural circuits in autonomous
mobile robots. International Journal of Intelligent
Systems, 21(9):1005–1024, 2006.
[11] D. Floreano and C. Mattiussi. Bio-Inspired Artificial
Intelligence: Theories, Methods, and Technologies.
MIT Press, 2008.
[12] D. Floreano and J. Urzelai. Evolution of plastic control
networks. Autonomous Robots, 11(3):311–317, 2001.
[13] M. J. Frank, B. Loughry, and R. C. O’Reilly.
Interactions between frontal cortex and basal ganglia
 in working memory: a computational model.
Cognitive, Affective, and Behavioral Neuroscience,
1(2):137–160, 2001.
[14] W. Gerstner and W. Kistler. Spiking Neuron Models:
An Introduction. Cambridge University Press, 2002.
[15] B. Girard, N. Tabareau, Q. C. Pham, A. Berthoz, and
J.-J. Slotine. Where neuroscience and dynamic system
theory meet autonomous robotics: a contracting basal
ganglia model for action selection. Neural Networks,
21(4):628–641, 2008.
[16] F. Gruau. Automatic definition of modular neural
networks. Adaptive Behaviour, 3(2):151–183, 1995.
[17] K. Gurney, T. J. Prescott, and P. Redgrave. A
computational model of action selection in the basal
ganglia. I. A new functional anatomy. Biological
cybernetics, 84(6):401–410, 2001.
[18] K. Gurney, T. J. Prescott, and P. Redgrave. A
computational model of action selection in the basal
ganglia. II. Analysis and simulation of behaviour.
Biological cybernetics, 84(6):411–423, 2001.
[19] G. S. Hornby. Measuring, enabling and comparing
modularity, regularity and hierarchy in evolutionary
design. In Proc. of GECCO, pages 1729–1736, 2005.
[20] G. S. Hornby and J. B. Pollack. Creating high-level
components with a generative representation for
body-brain evolution. Artificial Life, 8(3):223–246,
2002.
[21] C. Igel, W. Erlhagen, and D. Jancke. Optimization of
neural field models. Neurocomputing, 36(1-4):225–233,
2001.
[22] A. Ijspeert, A. Crespi, D. Ryczko, and J. Cabelguen.
From swimming to walking with a salamander robot
driven by a spinal cord model. Science,
315(5817):1416–1420, 2007.
[23] A. Ijspeert, J. Hallam, and D. Willshaw. Evolving
swimming controllers for a simulated lamprey with
inspiration from neurobiology. Adaptive Behavior,
7(2):151–172, 1999.
[24] T. Kondo. Evolutionary design and behavior analysis
of neuromodulatory neural networks for mobile robots
control. Applied Soft Computing Journal,
7(1):189–202, 2007.
[25] J. Krichmar and G. Edelman. Brain-based devices for
the study of nervous systems and the development of
intelligent machines. Artificial Life, 11(1-2):63–77,
2005.
[26] W. Lohmiller and J.-J. E. Slotine. Contraction
analysis for nonlinear systems. Automatica,
34(6):683–696, 1998.
[27] D. Marbach and A. J. Ijspeert. Online optimization of
modular robot locomotion. In Proc. of ICMA’2005,
pages 248–253, 2005.
[28] J.-A. Meyer, S. Doncieux, D. Filliat, and A. Guillot.
Biologically Inspired Robot Behavior Engineering,
chapter Evolutionary Approaches to Neural Control of
Rolling, Walking, Swimming and Flying Animats or
Robots, pages 1–43. Springer-Verlag, 2002.
[29] J.-A. Meyer, A. Guillot, B. Girard, M. Khamassi,
P. Pirim, and A. Berthoz. The Psikharpax project:
Towards building an artificial rat. Robotics and
Autonomous Systems, 50(4):211–223, 2005.
[30] J.-B. Mouret and S. Doncieux. MENNAG: a modular,
regular and hierarchical encoding for neural-networks
based on attribute grammars. Evolutionary
Intelligence, 1:187–207, 2008.
[31] J.-B. Mouret and S. Doncieux. Evolving modular
neural-networks through exaptation. In Proc. of
IEEE-CEC, 2009.
[32] J.-B. Mouret and S. Doncieux. Sferesv2 : evolvin’ in
the multicore world. In Proc. of IEEE-CEC, 2010.
[33] S. Nolfi and D. Floreano. Evolutionary robotics: The
biology, intelligence, and technology of self-organizing
machines. Bradford Book, 2004.
[34] T. Prescott. Action selection. Scholarpedia, 3(2):2705,
2008. doi:10.4249/scholarpedia.2705.
[35] T. J. Prescott, F. Montes-Gonzalez, K. Gurney, M. D.
Humphries, and P. Redgrave. A robot model of the
basal ganglia: Behavior and intrinsic processing.
Neural Networks, 19:31–61, 2006.
[36] P. Redgrave. Basal ganglia. Scholarpedia, 2(6):1825,
2007.
[37] P. Redgrave, T. Prescott, and K. N. Gurney. The
basal ganglia: A vertebrate solution to the selection
problem? Neuroscience, 89:1009–1023, 1999.
[38] J. Reisinger, K. O. Stanley, and R. Miikkulainen.
Evolving reusable neural modules. In Proc. of
GECCO. Springer, 2004.
[39] N. Rougier. Dynamic neural field with local inhibition.
Biological cybernetics, 94(3):169–179, 2006.
[40] N. Rougier and J. Vitay. Emergence of attention
within a neural population. Neural Networks,
19(5):573–581, 2006.
[41] A. Slocum, D. Downey, and R. Beer. Further
experiments in the evolution of minimally cognitive
behavior: From perceiving affordances to selective
attention. In From Animals to Animats 6, pages
430–439, 2000.
[42] A. Soltoggio, J. A. Bullinaria, C. Mattiussi, P. Dürr,
and D. Floreano. Evolutionary Advantages of
Neuromodulated Plasticity in Dynamic, Reward-based
Scenarios. Artificial Life, 11:569, 2008.
[43] K. Stanley, D. D’Ambrosio, and J. Gauci. A
Hypercube-Based encoding for evolving Large-Scale
neural networks. Artificial Life, 15(2):185–212, 2009.
[44] K. O. Stanley and R. Miikkulainen. Evolving neural
networks through augmenting topologies. Evolutionary
Computation, 10(2), 2002.
[45] T. Trappenberg, M. Dorris, D. Munoz, and R. Klein.
A model of saccade initiation based on the
competitive integration of exogenous and endogenous
signals in the superior colliculus. Journal of Cognitive
Neuroscience, 13(2):256–271, 2001.
[46] J. Vitay, N. Rougier, and F. Alexandre. A distributed
model of spatial visual attention. In Biomimetic
Neural Learning for Intelligent Robots, pages 54–72.
Springer, 2005.
[47] B. Webb. Robots in invertebrate neuroscience. Nature,
417(6886):359–363, 2002.