0 Bewertungen0% fanden dieses Dokument nützlich (0 Abstimmungen)

1 Ansichten10 SeitenICA

Feb 12, 2018

1-s2.0-016516849190079X-main

© © All Rights Reserved

PDF, TXT oder online auf Scribd lesen

ICA

© All Rights Reserved

Als PDF, TXT **herunterladen** oder online auf Scribd lesen

0 Bewertungen0% fanden dieses Dokument nützlich (0 Abstimmungen)

1 Ansichten10 Seiten1-s2.0-016516849190079X-main

ICA

© All Rights Reserved

Als PDF, TXT **herunterladen** oder online auf Scribd lesen

Sie sind auf Seite 1von 10

Elsevier

based on neuromimetic architecture

INPG-Lab, TIRF, 46, avenue F61ix Viallet, F-38031 Grenoble Cedex, France

Revised 24 October 1990 and 21 February 1991

Abstract. The separation of independent sources from an array of sensors is a classical but difficult problem in signal

processing. Based on some biological observations, an adaptive algorithm is proposed to separate simultaneously all the

unknown independent sources. The adaptive rule, which constitutes an independence test using non-linear functions, is the

main original point of this blind identification procedure. Moreover, a new concept, that of INdependent Components

Analysis (INCA), more powerful than the classical Principal Components Analysis (in decision tasks) emerges from this work.

Zusammenfassung. Die Trennung unabh~ingiger Quellen stellt ein klassisches jedoch schwieriges Problem bei der Signalverar-

beitung dar. Aufgrund neurobiologischer Beobachtungen stellen wir in diesem Artikel einen selbstanpassenden Algorithmus

vor, der gleichzeitig alle unbekannten, unabh~ingigkeitstest unter Anwendung yon nicht linearen Funktionen darstellt, ist der

zentralste Punkt dieses blindend Identifikationsverfahrens. Ausserdem hebt sich ein neues Konzept, das der unabh~ingigen

Komponenten-Analyse (INCA), leistungsf'fihiger in den Entscheidungsvorg~ingen als die Analyse der Hauptkomponenten,

aus dieser Arbeit hervor.

R6sum6. La s6paration de sources ind6pendantes constitue un probl~me classique mais difficile de traitement du signal.

D'apr~s des observations neurobiologiques, nous proposons dans cet article un algorithme auto-adaptatif capable de s6parer

simultan6ment toutes les sources ind6pendantes inconnues. La r~gle d'adaptation, qui effectue un test d'ind6pendance gr$ce

l'utilisation de fonctions non-lin6aires, est le point le plus central de cette m6thode d'identification aveugle. De plus, un

nouveau concept, celui d'analyse en composantes ind6pendantes (INCA), plus puissant dans les op6rations de d6cision que

celui d'analyse en composantes principales, 6merge de ce travail.

Keywords. Separation of sources, high order moments, principal components, independent components, neural networks,

linear recursive adaptive filter.

this p r o b l e m is c a l l e d blind separation o f sources.

I n a l a r g e n u m b e r o f cases, t h e s i g n a l r e c e i v e d Numerous a p p r o a c h e s h a v e b e e n d e v e l o p e d to

b y a s e n s o r ( a n t e n n a , m i c r o p h o n e , etc.) is t h e s u m solve the difficult problem of extracting the

(mixture) of elementary contributions that we can different sources from signals received by the

call sources. F o r i n s t a n c e , t h e s i g n a l r e c e i v e d b y s e n s o r s . H o w e v e r , it h a s b e e n s h o w n t h a t t h e i n f o r -

a n a n t e n n a is a s u p e r i m p o s i t i o n o f s i g n a l s e m i t t e d mation contained in t h e s p e c t r a l m a t r i x is n o t

by all t h e s o u r c e s w h i c h a r e in its r e c e p t i v e field. sufficient to g i v e a s o l u t i o n [14]. M o s t o f t h e w o r k s

Generally, sources as well as mixtures are r e l a t e d to this p r o b l e m u s e t h e i n f o r m a t i o n c o n -

u n k n o w n . I n this case, w i t h o u t a n y k n o w l e d g e o n t a i n e d in t h e r e c e i v e d s i g n a l s t o g e t h e r w i t h o t h e r

C. Jutten, J. Herault / Blind separation o f sources, Part I

information. For instance, the problem becomes blind source separation have been conducted

simpler if we can assume that one source is a recently using cumulants [3, 13] or high-order

deterministic signal while all the others are moments analysis [1, 5]. For a review o f these

gaussian noises [2]. O f course, in these cases, the methods, see [4].

problem is no more blind separation of sources. Section 1 is devoted to theoretical aspects:

Such a superimposition is also observed in bio- modelling of the problem and the existence of a

logical systems. For instance, during a movement, solution with a recursive neural network. The

nervous fibres Ia and II transmit to the central second section defines the learning rule. Some

nervous system mixtures of information about joint experiments are presented in the third section. In

stretch and stretch speed [9, 16]. However, it seems the conclusion, we discuss on the extension of the

that the central nervous system is able to separate algorithm in more realistic situations: the number

speed and stretch from these mixtures. Taking into of sources is unknown and different from the num-

account the large number of muscular fibers, a ber of sensors, and outputs of the sensors are

genetically wired solution is not realistic. non-linear combinations of sources or convol-

Moreover, according to tracking abilities imposed utions of sources.

by the possible time evolution of sensors (lesion,

aging, etc.), an adaptive network constitutes the

more consistent solution for this problem of source 1. Mixture model and theoretical solution

separation. We first addressed this problem in 1985

[8] and proposed a solution based on a neural 1.1. Model of the mixture

approach. The architecture proposed in Section

First, we assume that the number of sources n

1.1 (Fig. 1) is very c o m m o n at various levels in the

is equal to the number of sensors p, and we propose

central nervous system: sensorial, cortical as well

a simple linear model for the mixture. Letting El(t)

as cerebellar level. However, taking into account

be the signal measured at the output of the ith

the linearity of 'neurons', the architecture can be

sensor, we then assume

viewed as a recursive linear adaptive filter.

Using a recursive fully interconnected neural Ei( t) = ~. aikXk( t), (1)

network with learning abilities, we propose in this k

p a p e r a blind (without a priori information) iden- where Xk(t) is the unknown signal emitted by the

tification procedure, based on the use of high order kth source and aik is a real unknown scalar.

moments. Other studies, related to the problem of In order to separate the sources Xk(t) from

signals Ei(t) simultaneously, we proposed in 1985

[8] a solution, derived from a neural approach,

El ~- -C12 -ci~,=I ~ .~ S1 the schematic architecture of which is given in

Fig. 1.

E2

-C21

~ A

The recursive network is constituted by n linear

~._ S2

'neurons'. The output Si(t) of the ith operator is

a weighted sum of input signal E~(t) and other

outputs Sk(t) ( k S i). In Fig. 1, each bold point

represents a multiplier providing a partial product

En -cn -cn~

- CikSk(t). The horizontal line and the operator

-TI Fig. I. A r c h i t e c t u r e o f t h e n e u r a l n e t w o r k .

(triangle) performs the sum of the partial products

and of the input E~(t):

k~i

l<~i<~n. (2)

signal Processing

C. Jutten, J. Herault / Blind separation of sources, Part I

Let X ( t ) , E ( t ) and S(t) be the vectors for which Fig. 2 and (1) and (5) can be written as

the components are unknown sources X~(t), the

El(t) = a l l X l ( t)+ al2X2( t),

signals Ei (t) and the network outputs Si (t), respec- (6)

tively. Let A be the mixture matrix and C the E2( t ) = az,Xl( t ) + a22X2( t ),

matrix of the network coefficients. Note that the E l ( t ) - c,~Edt)

diagonal coefficients of matrix C are equal to zero. S,(t) -

1 - c12c21

Equations (1) and (2) can be written using matrix (7)

and vector notations: Ez(t)-c2lSl(t)

Sdt) -

1- C12C21

E(t)=AX(t), (3)

By substituting El(t) and E2(t) from (6) into

S(t) = E ( t ) - CS(t). (4)

(7). we get

In (3), the matrix A and the source X ( t ) are

unknown. Moreover, there is no extra information (all - cl2a2,)Xl( t ) + (a12 - c,2a22)X2( t )

Sl(t) =

about signals or mixture. The sole hypothesis is 1 - C12C21

(8)

the statistical independence of components of the ( a21 - c21all) X l ( t ) + (a22- c21al2)X2( t )

&(t) =

vector X ( t ) . Assuming that the operators are very 1 - C12C21

fast according to variations of signals E ( t ) , and

assuming that the recursive network is stable, (4) From (8), two different pairs of coefficients C~k

becomes lead clearly toward two correct solutions:

(i) If c,2 = a,2/a22 and c21 = a21/a~l,

S(t) = ( I + C ) - I E ( t ) . (5) then S j ( t ) = a l l X l ( t ) and S2(t)= a22X2(t),

(ii) if c,2 all/aEj and c21 = a22/a12,

=

In fact, for the sake of stability, the loop gain

Pertinent values of weights Cik provide source

separation, that is to say, each output Si(t) of the recursive network must be less than one:

becomes proportional to an unknown source, say c12c21 ~ 1 , and it is easy to verify that only one of

the two previous solutions is possible.

Xk(t). Such a solution is then achieved with some

Theoretically, the architecture of the network

undetermination (especially, a permutation P

allows to separate the unknown sources. Neverthe-

defining the pairs (i, k)), which is presented in

details in Part II, Section 1. less, the theoretical values of Cik predicted above

However, we propose to examine the simple case cannot be computed directly: they are functions

n = p = 2. In this case, the network is shown in of unknown coefficients aik. Therefore, we propose

an adaptive algorithm to estimate the matrix C.

E1 ~ ~ S1S2

2.1. Error term

is to say, that the solution is defined by: Sk(t) is

proportional to Xk(t), for all k. This assumption

does not change the generality of the solution

Fig.2. The2-operatornetwork:2neuronsand2coefficients because the index of a source is arbitrary. Assume

only. that the network is very close to a solution: n - 1

Vol.24,No.1.July1991

4 C: Jutten, J. Herault / Blind separation of sources, Part I

outputs Sk(t) are already proportional to Xk(t) one. Therefore, the partial derivative of the com-

(1 ~< k ~< n - 1). According to (2), the last output ponent sin(t) is

S,(t) can be written as

Osm( t)/OC,k = --qmisk( t), (14)

S,(t) = E , ( t ) - Y~ c, gSk(t). (9)

k~n where q,,i is the term with index mi of the matrix

Using (1), we have ( I + C ) -l.

Referring to (12). it is found that

S , ( t ) = E (a,k--C,kakk)Xk(t)

k¢n

OS~( t)/OCik = --2qmis,.( t)sk( t), (15)

+a,,X,(t). (10)

and for m = i we obtain

Let s,(t) = S , ( t ) - ( S , ( t ) ) and x,(t) =

X ~ ( t ) - ( X , ( t ) ) . While signals Xk(t) are assumed 3S~( t)/OC~k = --2q.s~( t)sk( t). (16)

statistically independent, (10) leads to

By applying the gradient method, we then find

(s2,(t)) = E (a.k-- Cnkakk)2(x2(t)) the following adaptation rule:

k~n

When the nth output becomes proportional to where a is a positive adaptation gain.

X,(t), i.e., when all the coefficients ( a , k - e, kakk)

are equal to zero, the mean square term (sZ(t)) is 2.3. Evaluation of this adaptation rule

minimum. So, for the ith operator, the term s~(t)

can be considered as an error term to which we In fact, the convergence is achieved when the

can apply the gradient method. time averages (dCik/dt) are equal to zero, i.e. if

(si(t)Sk(t))=O Vi, k ~ i. These conditions mean

that the two outputs Si(t) and Sk(t) are not corre-

2.2. Gradient method

lated. R e m e m b e r that we assume the statistical

Let us compute the partial derivative of s2(t) independence of the unknown sources X~(t).

with respect to qk : Therefore, when a good solution of the problem

Os2( t ) / OC,k= 2si( t ) Os,( t ) / OCik. (12) of source separation is obtained, any pair of out-

puts S~(t) and Sk(t) must be independent and not

With zero mean vectorial signals s(t) and e(t), only non-correlated. So, (17) should be improved

(5) leads to towards an independence test of s~(t) and sk(t).

s ( t ) = ( I +C)-~e(t). Moreover, the error term we proposed in the

last section is not exact. It was computed assuming

After taking the partial derivative, we obtain

that the system was very close to a solution. I f one

OS( t)/OC,k = --(I + C ) - ' 0(I + C)/OC,k coefficient, say Cik, varies, the corresponding out-

put S~(t) is no longer proportional to one source,

x(I+C)-le(t),

but becomes a mixture of sources. Variations of

which can be written as output S~(t) then affect all the outputs of the

network, through the n - 1 connections c,,i coming

aS(t)/OC,k = --(I+ C) -10C/OC,kS(t). (13)

from output Si(t).

In the last equation, OC/OC~k is a square matrix It seems that the adaptation rule cannot be

with only one non-zero element: the one located associated with a Lyapunov function. More pre-

on row i and column k. That element is equal to cisely, we cannot find an error function, whose

global minimization provides the solution.

Signal Processing

C. Jutten, J. Herault / Blind separation of sources, Part I 5

for audiofrequency signals (speech, music, etc.),

In the adaptation rule (17), q, is the diagonal

the even probability density function is not a

term with index i of the matrix ( I + C) -~. Diagonal

restrictive condition. The adaptation rule (19) is

coefficients of matrix C being zero, so long as other

then an approximation of an independence test.

coefficients C~k ( i ~ k) remain small, first order

Finally, note that signal Si(t) is not a zero-mean

expansion leads to

signal, a priori. Therefore, the zero-mean signal

q,= l-c,= l Vi. (18) s~(t) must be estimated by the relation

As a result of (18), we have dC~k/dt= dckJdt. s,(t) = S, (t) - (S, (t)), (22)

This relation even holds in the case of n = p = 2.

where (S~(t)) is an estimation of the time average

In the following, we consider q, = 1.

of signal Si(t). In practice, a simple first-order low

We will see in the last section that the rule (17)

pass filter is a good enough estimation of the

or its simplified version with q, = 1, leads only to

average:

a zero-covariance test. To achieve an independence

test, we propose to use two non-linear and different (Si( t)) = S,( t) * e x p ( - t / T), (23)

odd functions:

where* denotes a convolution product and T is

dC~k/ dt = af(s,( t) )g(sk( t) ). (19) the time constant of the filter.

As a result of the reasoning put forward in this

O f course, these functions must be different,

section, we propose the following adaptation rule:

otherwise we always have dC~k/dt = dckJdt, which

implies symetric modifications of the matrix C. dC,k/ dt = af(s,( t) )g(Sk ( t) ), (24)

These two functions introduce high-order

which, on the average, is equivalent to

moments. In fact, let us assume that these functions

can be expanded in a Taylor series. They contain dC,k/ dt = af( S~( t) )g(sk( t ) ). (25)

only odd power terms:

As can be experimentally verified, these rules

f ( x ) =Xf2~+l x2)+1, g ( x ) = X g2m+l x2"+1. are very robust. In particular, a large class of odd

j m

functions f and g is acceptable; in most of the

Referring to these expansions, (19) can be simulations presented in the following section, we

expressed as have used f ( x ) = x 3 and g ( y ) = t a n I(y), but

g ( y ) = y or g ( y ) = s i g n ( y ) lead also to source

dCik - - a ~" x~r "

/~.(..,J2j+l,~52m+l

S2j+lZt~S2m+ltt

i \ ] k

~

~, 1"

(20)

dt j ,, separation. Limitations related to the density prob-

ability distribution of the signals are proved in

The convergence of the algorithm corresponds now Parts II and III. These limitations seem to be very

to the following condition: severe, but in many practical situations, for audio-

frequency signals for instance (speech, music, etc.),

~d¢ik~

\dt/

= a ~j ~,/,,~ J¢2 j + l ~ 2~r n + l \ /S2j+lrt~s2m+lt.t~

i ~, ] k

\

~, ]1 it can be shown that these limitations do not apply,

which has been experimentally confirmed.

=0. (21)

Relations (24) and (25) are very close to mathe-

Convergence is then achieved if all the high-order matic formalizations [6] of rules for synaptic plas-

moments (S~J+l(t)s2kr"+'(t)) are equal to zero. ticity proposed as by Hebb [7] or Rauschecker and

Statistical independence between s;(t) and sk(t) Singer [15]. These relations are local rules: the

implies (S2~+'(t)s2k"+l(t)) = (S2i+~(t))(SZk"+'(t)). If adaptation increment of synaptic weight C~k,from

the probability density of signal s~(t) is an even neuron k to neuron i, depends on the weight C~k

function, it is easy to verify that all the odd and on outputs Si(t) and Sk(t) of neurons i and

Vol. 24, No. I. July 1991

C. Jutten, J. Herault / Blind separation o f sources, Part I

the metabolism o f the central nervous system is

I a

associated with the competition o f chemical pro- I b

I I

cesses which typically induce non-linearities. I

Xl(t ) I x2(o

I

I

3. Experimental results

I C d

I

St. X1 i S2- X2

In this section, we p r o p o s e some simulation

results as illustrations o f the theory developed in ot i ,qL

ot i

the previous parts. I

I

e I f

I

t=O

t

t=0

C o n s i d e r n o w a set o f two sensors receiving an

Fig. 3. Separation of two signals with a 2-neuron network.

u n k n o w n mixture o f two u n k n o w n i n d e p e n d e n t Before t =0, coefficients of the matrix C are equal to zero,

signals. We assume n o t h i n g else: the nature o f which implies S~(t) = E~(t), t < 0. After t = 0, the learning pro-

these signals (deterministic or stochastic, with large cess becomes active, and after some hundred learning steps,

each output Si(t) extracts an unknown signal X~(t). (a), (b)

or narrow bandwidth, etc.) has no effect on the outputs of the networks, (c), (d) unknown sources to be extrac-

algorithm. ted, (e), (f) errors on the outputs.

In this experiment, there is a mixture o f a deter-

ministic signal X~(t) a n d a r a n d o m noise XE(t)

3.2. Image preprocessing

with uniform probability density. The energy o f

the noise X2(t) is about 50 times greater than that N o w consider Fig. 5(a). With a r a n d o m sampling

o f signal X t ( t ) . We use here a single 2-neuron o f points (x, y) from this image, we obtain a time

network (Fig. 2), with the initial conditions c~2 = series o f points ( x ( t ) , y ( t ) ) that we may consider

c2~ = 0. Before the beginning o f learning, from (4), as a sampled 2-dimensional signal. This signal was

we d e d u c e & ( t ) = Ei(t). In Fig. 3, we can see the processed with a 2-neuron network and the

outputs Sl(t) and S2(t): before learning (t < 0, the algorithm converged after 300 to 500 learning steps.

beginning o f the learning is indicated by an arrow),

outputs are very similar, because the signal X~(t) c2, \

is d r o w n e d by the noise X2(t). After the beginning

o f learning, the network provides an adaptive esti- 1 ~Unstable

mation o f the mixture, a n d after about 300 learning

steps, both signals X l ( t ) and X2(t) are separated

on the outputs o f the network, with a residual

crosstalk o f about - 2 5 dB.

, ,

D e p e n d i n g on the initial conditions (c~2(t = 0 )

v .5 - 1

and c21( t = 0)), the algorithm m a y converge rapidly

Fig.4. Evolution of coefficientsofmatrix C during the learning.

or slowly, or may even diverge (Fig. 4). The cross corresponds to the theoretical solution. The bold

If the mixture o f signals is time dependent, the points are the starting points for 4 different convergence trajec-

algorithm is able to track the solution if the rate tories. For the sake of stability, the recursive 2-neuron network

must satisfy the condition C~2C2~< 1. The first quadrant is thus

o f mixture evolution is less than the rate o f separated into two regions (stable or unstable) by the hyperbola

learning. Cj2C2t = 1.

Signal Processing

C. Jutten, J. Herault / Blind separation of sources, Part I 7

Input Output

, J~u

Input Output

¢ d

Fig. 5. Application to image preprocessing. A 2-neuron network can eliminate dependence between coordinates of the points of a

sloped text. After convergence, the initial images in Figs. 5(a) and 5(c) are transformed into the images in Figs. 5(b) and 5(d),

respectively. Lines of transformed images are horizontal or vertical, according to the slope of the lines in the initial images.

Then, the initial image was transformed by the this algorithm. This new concept may be compared

network into the image in Fig. 5(b), where lines to the classical concept of Principal Components

and characters, sloped in the initial image, are Analysis (PCA). In fact, emergence of independent

straightened. This result is due to the fact that a signals and not only non-correlated signals is due

sloped line introduces dependence between coor- only to non-linear functions in the learning rule

dinates x and y. The algorithm minimizes this which introduce high-order statistical moments

dependency, which is reached if lines are either and not only second-order moments as in most

horizontal or vertical. classical methods.

In fact, if the slope of the lines of the initial text To demonstrate the advantage of this new con-

is closer to a vertical line than to a horizontal line cept, we compare in the following experiment these

(Fig. 5(c)), after convergence, the lines of the initial 2 methods: INCA and PCA. We consider 3

image are vertical in the transformed image (Fig. unknown noisy digital signals, and assume they

5(d)). are transmitted along 3 wires with crosstalk. So,

at the end of the wires, we observe 3 signals (E~(t),

E2(t) and E3(t)) which are unknown mixtures of

3.3. INdependent Components Analysis (INCA) unknown digital signals (Fig. 6(a)). These signals

A new concept in data analysis, that of INdepen- may also be represented by their distribution in a

dent Components Analysis (INCA) emerges from 3-dimensional space (Figs. 6(b) and 7(a)). In these

Vol. 24, No. 1, July 1991

C. Jutten, J. Herault / Blind separation of sources, Part 1

4

t

4

0 EI E2

a b

s!t t si t

S!t .............. _t si t

c d

Fig. 6. INdependent Components Analysis (INCA) versus Principal Components Analysis (PCA). (a) Inputs are unknown mixtures

of 3 unknown noisy digital sources. In this figure, the ratio S/B of each source is equal to 10 dB. (b) Isometric representation of

input distribution in the plane (El, E2, E3). (c) Signals transformed by PCA: digital sources are not found. (d) Signals transformed

by INCA after convergence of the algorithm: each output extracts a noisy digital source. Noise can easily be eliminated by a simple

threshold element.

figures, we can see 8 clusters, associated to the 8 By using our algorithm, after convergence

possible configurations o f 3 binary variables. The (about 3000 learning steps), the network provides

diameter o f each cluster d e p e n d s on the noise a transform o f the original signals and distributions

energy. (Figs. 6(d) and 7(c)). In Fig. 6(d), it is clear that

By c o m p u t i n g the covariance matrix o f this dis- I N C A extracts 3 noisy binary signals from mixtures

tribution, and after diagonalization o f this matrix, E~(t), E 2 ( t ) and E3(t ) o f Fig. 6(a).

we obtain the K a r h u n e n - L o ~ v e Transform (or C o m p a r i n g Figs. 6(c) and 6(d), it is clear that

PCA) by choosing eigen vectors as the new base. I N C A is very a d v a n t a g e o u s for the extraction o f

Figure 6(c) shows the 3 signals observed in this i n d e p e n d e n t features. The 3 u n k n o w n digital sig-

new base, and Fig. 7(b) the transformed distribu- nals are effectively separated by this algorithm,

tion o f Fig. 7(a). and the residual noise can be easily eliminated,

Signal Processing

C. Jutten, .L Herault / Blind separation of sources, Part I 9

E3 $3 S3

Sl .'~+

~ . . ~ . . . ~ " ,~-+

a b C

Fig. 7. INdependent Components Analysis (INCA) versus Principal Components Analysis (PCA). (a) Input distribution of 3

unknown mixtures of 3 unknown noisy digital signals. In this figure, the ratio S/B of each source is equal to 23 dB. (b) Output

distribution after PCA. On the principal axis (S~), there is still a mixture of sources. (c) Output distribution after INCA. Each axis

now corresponds to an unknown source.

for instance with a simple threshold element. With corresponds to that related to the first order linear

PCA, the signal associated with the first eigen approximation of the non-linear mixture [10].

vector has the greatest energy, but it is always a In case numbers of sources and sensors are

mixture of the 3 binary sources. As a conclusion, different, we can mention the following experi-

we can say that PCA is an effective method for mental consistent results:

data compression, while I N C A is an effective - if there are more sources (n) then sensors (p),

method for the extraction of independent features. there is no exact solution. The algorithm separ-

Efficient interpretation or classification needs this ates the p most energetical signals and the

type of pre-processing. remaining signals induce noise, which decreases

• quality of the separation.

- if there are less sources (n) than sensors (p), n

4. Discussion outputs (from p) of the networks provide the n

sources, and the p - n other outputs are on the

In this paper, we have presented a new adaptive average equal to zero.

algorithm for the separation of sources. The main The main drawback of this adaptive algorithm

characteristic of this algorithm is the use of high is the unknown number of learning steps necessary

order statistical moments (with non-linear func- before convergence. Statistical explanations on

tions) to perform an independence test. Although how this algorithm works and the design of a non-

the examples given in this paper are very simple, adaptive algorithm has been studied by C o m o n

generalization to more realistic situations is easy. [3]. Statistical explanations are given in the second

Non-linear mixtures, degenerated mixtures (the part of this paper.

number of sensors n is not equal to the n u m b e r Finally, the stability of the algorithm has not

of sources p) and convolutive mixtures ('cocktail been dealt with here. This difficult problem, com-

party' problem, for instance) have already been plicated by non-linear functions, has been studied

studied with success [10, 12] and will be published by E. Sorouchyari and constitutes the third part

in a forthcoming paper. of this p a p e r on a method of sources separation.

In the case of non-linear mixtures, there is no Several hardware configurations have already

longer an exact solution with a linear network. been implemented to run this algorithm. Since

However, experimental and theoretical results 1985, a discrete analog 2-neuton network [111,

show that the solution proposed by the algorithm designed by the authors has been running in the

Vol. 24, No. 1+ July 1991

10 C. Jutten, .1. Herault / Blind separation of sources, Part I

T I R F laboratory of the INP Grenoble. Another [5] L. Fety, Mrthodes de traitement d'antenne adaptres aux

configuration, using the Digital Signal Processor radiocommunications, ThOse, ENST, Paris, 1988.

[6] S. Grossberg, "Nonlinear neural networks: Principles,

68930, with the support of Thomson Semiconduc- mechanisms, and architectures", Neural Networks, Vol. 1,

tors Inc., was implemented in 1987. In 1989, a No. 1, 1988, pp. 17-61.

CMOS analog integrated circuit of a 6-neuron [7] D.O. Hebb, The Organization of the Behaviour, Wiley,

New York, 1949.

network was designed by Arreguit and Vittoz [ 17].

[8] J. Hrrault, C. Jutten and B. Ans, "Drtection de grandeurs

This circuit can separate three mixed audio- primitives dans un message composite par une architecture

frequency signals, as speech or music, in about de calcul neuromimrtique en apprentissage non super-

5 ms. visr", Proc. X~me colloque GRETSI, Nice, France, 20-24

May 1985, pp. 1017-1022.

[9] J.C. Houk, W.Z. Rymer and P.E. Crago, "Nature of the

dynamic response and its relation to the high sensitivity

of muscles spindles to small changes in length", in: H.

Acknowledgments Taylor and A. Prochazka, eds. Muscles Receptors and

Movement, MacMillan, London, 1981, pp. 33-43.

This work was partially funded by the ESPRIT [10] C. Jutten, Calcul Neuromim&ique et Traitement du Sig-

nal. Analyse en Composante Indrpendantes, ThOse d'Etat,

II-BRA project no. 3049 entitled NERVES for INPG-USMG, Grenoble, 1987.

'NEurocomputing Research and VLSI for [11] C. Jutten and J. Hrrault, "Analog implementation of a

Enhanced Systems'. The authors would like to permanent unsupervised learning algorithm", Proc.

NA TO Advanced Research Workshop on Neurocomputing,

express their gratitude to X. Arreguit, P. Comon,

Les Arcs, France, 27 February-3 March 1989, pp. 145-152

E. Sorouchyari and E. Vittoz for very fruitful [12] C. Jutten, J. Hrrault and A. Gurrin, "IN.C.A.: An

discussions. independent components analyser based on an adaptive

neuromimetic network", in: J. Demongeot, T. Hervr, V.

Rialle and C. Roche, eds., Artificial Intelligence and Cogni-

tive Sciences, Manchester Univ. Press, Manchester, 1988,

References pp. 201-219.

[13] J.-L. Lacoume and P. Ruiz, "Sources identification: A

solution based on the cumulants", Proc. of the 4th ASSP

[1] J.-F. Cardoso, "Eigen structure of the 4th order cumulant Workshop on Spectral Estimation and Modelling,

tensor with application to the blind sources separation Minneapolis, USA, August 1988, pp. 199-203.

problem", lnternat. Conf. Acoust. Speech Signal Process., [14] H. Mermoz, "'Spatial processing beyond adaptive beam-

pp. 2655-2658. forming", J. Acoust. Soc. Amer., Vol. 70, No. 1, July 19'81,

[2] F. Christophe and C. Morisseau, "'Elimination auto-adap- pp. 74-79.

tive de brouilleurs par une mrthode de gradient appliqure [15] J.P. Rauschecker and W. Singer, "The effect of early

/l un rrseau lacunaire", Proc. X~me GRETSI, Nice, experience of the cat's visual cortex and their possible

France, 20-24 May 1985, pp. 571-576. explanation by Hebb synapses", J. Physiol., 1981, No. 310,

[3] P. Comon, "Separation of stochastic processes whose pp. 215-239.

linear mixture is observed", Proc. ONR-NSF-IEEE [16] J.-P. Roll, Contribution de la proprioception musculaire

Workshop on Higher-Order Spectral Analysis, Vail, la perception et au contrrle du mouvement chez

Colorado, 28-30 June 1989, pp. 174-179. I'homme, ThOse d'Etat, Univ. d'Aix-Marseille I, 1981.

[4] P. Duvaut, "Principes des m&hodes de srparation de [ 17] E. Vittoz and X. Arreguit, "CMOS integration of Hrrault-

sources fondres sur les moments d'ordre suprrieur", Jutten cells for separation of sources", Proc. Workshop on

Traitement du Signal, Vol. 7, No. 5, Numrro sprcial Non- "'Analog VLSI and Neural Systems", Portland, Oregon, 8

Linraire Non-Gaussien, 1990, pp. 407-418. May 1989.

Signal Processing