Automatic Training of Spiking Neurons

R. A. Vzquez and his students

R. A. Vazquez and B. A. Garro: Training Spiking Neurons by means of Particle Swarm Optimization. ICSI 2011. LNCS 6728, pp. 242-249. R. A. Vazquez: Izhikevich Neuron Model and its Application in Pattern Recognition. Australian Journal of Intelligent Recognition Information Processing Systems, vol. 11, pp 35-40 (2010).

Center for Computing Research IPN

GOAL

To adjust the synaptic weights of a spiking neuron to solve a given pattern classification task task.

Center for Computing Research IPN

IDEA OF SOLUTION
Training: Input pattern Input signal Spiking Neuron Stimulate during t ms. Testing: Input pattern Input signal Trained Spiking Neuron
Produce a P d fairing rate according to input pattern Compute a corresponding fairing rate

Center for Computing Research IPN

Spiking Neural Model

Dendrites
Collect signals from other neurons

Soma
Processing: Processing If total input exceeds threshold, output signal generated

Axon
i Delivers spike train to other neurons

Spikes look alike they do not carry alike, any information. Rather, it is the number and the , timing of spikes, which matter.

Center for Computing Research IPN

Spiking Neural Model

Adopted model: Izhikevich model):

a = recovery time constant. ti t t u is an amplifying if u = the recovery current. u is a resonator if C = the membrane capacitance. = the resting membrane potential potential. = the instantaneous threshold potential. c = voltage reset value. v = the membrane potential potential. I = the input signal.
Depending on the values of a and b, it can be an integrator or a resonator. p g g

Center for Computing Research IPN

Spiking Neural Model

Example of parameters for the Izhikevich neuron:

Euler method is used to solve the differential equation of the model with dt = 1.

Center for Computing Research IPN

Spiking Neural Model

The firing rate FR is computed as the number of spikes generated in an interval of duration T divided by T . Center for Computing Research IPN

Spiking Neural Model

The firing rate FR is computed as the number of spikes generated in an interval of duration T divided by T . Center for Computing Research IPN

Spiking Neural Model

A detailed description of the Izhikevich model can be found in: Izhikevich, E. M.: Simple model of spiking neurons. IEEE Trans. on Neural Networks 14(6), 15691572 (2003). Izhikevich, E.M.: Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting. The MIT press (2007). Pages 273-275 Center for Computing Research IPN

PROPOSAL

First of all, each input pattern is transformed into an input signal

I.
Next the spiking neuron is stimulated using I during T ms, , g p Then for each class, the firing rate is computed as follows:

Spiking Neuron
Stimulate the SN with I during T ms Average firing rate for each class

Center for Computing Research IPN

PROPOSAL
Spiking Neuron
Stimulate the SN with I during T ms Average firing rate for each class

We expect that patterns which belong to: - the same class generate similar firing rates (low standard deviation). - Different classes generate firing rates different enough to discriminate among classes. Center for Computing Research IPN

PROPOSAL
The class cl of an input pattern follows: is determined as

Where is the firing rate generated by the spiking neuron stimulated with the input pattern: p p

Center for Computing Research IPN

PROPOSAL
Generating Input Signals from Input Patterns:
Izhikevhic neuron I hike hic ne ron model cannot be directl stim lated with the directly stimulated ith input pattern , but with an injection current I. To get a firing rate from the neuron model greater than zero, the i input signal should be I > 55pA. i
Spiking Neuron

I is generated in terms of

can be computed as:

is the set of synaptic weights of the neuron model. is a threshold which guarantees that the neuron will fire.

Center for Computing Research IPN

PROPOSAL
The synaptic weights of the neuron, which are directly connected to the input pattern, determine the firing rate of the neuron neuron. Synaptic weights Spiking Neuron Firing rate

Learning consists in generating the desired behavior by adjusting the synaptic weights of the neuron. y j g y p g The learning phase will be done by the PSO. PSO Synaptic weights Spiking Neuron Firing rate

Center for Computing Research IPN

PROPOSAL
To adjust the synaptic weights of the neuron, the standard PSO algorithm was used:
Inertia weight Acceleration coefficients Best particle

Velocity of the particle

The particle Craziness numbers is given as:

Updating position of particle

Center for Computing Research IPN

PROPOSAL
The algorithm: Given a population of individuals:

Center for Computing Research IPN

PROPOSAL
To find the synaptic weights that maximize the accuracy of the spiking neural model during a pattern recognition task the next fitness function task, was proposed:

Center for Computing Research IPN

Experimental results
Six databases were used for experimentation: UCI database: iris plant, glass, diabetes, liver-bupa and wine datasets, and The object recognition database: R. A. Vazquez and H. Sossa, H: A new associative model with dynamical synapses. synapses Neural Processing Letters 28(3) 189207 Letters, 28(3), 189 207 (2008).

screw

sheave

eyebolt y

spike p

milano tail

Center for Computing Research IPN

Experimental results
Parameters for the PSO algorithm:

Center for Computing Research IPN

Experimental results
The set of synaptic weights found with the PSO algorithm provokes that the spiking neuron generates almost the same firing rate when it is stimulated with input patterns from the same class.

3 classes

2 classes

5 classes

Center for Computing Research IPN

Experimental results

[19] R. A. Vazquez: Izhikevich Neuron Model and its Application in Pattern Recognition. Australian Journal of Intelligent Information Processing Systems, vol. 11, pp 35-40 (2010).

Center for Computing Research IPN

OTHER REFERENCES:
R. A. Vazquez and A. Cachn (2010). Integrate and Fire Neurons and their application in Pattern Recognition. pp g 2010 7th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE 2010) Tuxtla Gutirrez , Chiapas Sept 82010). Chiapas, Sept. 10. Pp. 424-428. R. A. Vazquez R A V (2010). Pattern R (2010) P tt Recognition U i iti Using Spiking Neurons and Firing Rates. IBERAMIA 2010, LNAI 6433, pp. 423-432. , pp

Center for Computing Research IPN