Beruflich Dokumente
Kultur Dokumente
Independent Components
Analysis
Professor Dr. Barrie W. Jervis
School of Engineering
Sheffield Hallam University
England
B.W.Jervis@shu.ac.uk
The Problem
Temporally independent unknown source
signals are linearly mixed in an unknown
system to produce a set of measured output
signals.
It is required to determine the source
signals.
s1
s2
x1
x2
u1
u2
sn
xn
un
g(.)
y1=g1(u1)
y2=g2(u2)
:
yn=gn(un)
Cautions I
The magnitudes and signs of the estimated source
signals are unreliable since
the magnitudes are not scaled
the signs are undefined
because magnitude and sign information is shared
between the source signal vector and the unmixing
matrix, W.
The order of the outputs is permutated compared
wiith the inputs
Cautions II
Similar overlapping source signals may not
be properly extracted.
If the number of output channels number
of source signals, those source signals of
lowest variance will not be extracted. This
is a problem when these signals are
important.
Information Theory I
H ( X ) P ( x ) log2 P ( x )
i 1
bits
Information Theory II
Note that the entropy is expressible in terms
of probability.
Given the probability density distribution
(pdf) of X we can find the associated
entropy.
This link between entropy and pdf is of the
greatest importance in ICA theory.
iff P ( x, y ) P ( x )P ( y )
Information Theory IV
The conditional entropy of Y given X
measures the average uncertainty remaining
about y when x is known, and is
H (Y | X ) P ( x, y ) log2P ( y | x )
x ,y
Differentiating:
I (Y , X )
H (Y )
H (Y | X )
H (Y )
w
w
w
w
Also H ( y ) E ln fy ( y ) fy ( y ) ln fy ( y )dy
Substituting:
y
H ( y ) E ln E ln fx ( x )
x
ln
w w x x
w x
Also assume u wx w 0
y 1 y 1 wx 1 2y
w x
from which :
1
w x 1 2y , and
w
w 0 1 2y
fx x
fy ( y )
,
J
where J is the Jacobian
y1
y1
x x
1
N
J det
y
y N
n
xN
x1
W W 1 2y xT , and
w 0 1 2y,
T 1
T 1
Thus,
H ( y)
W
1 (1 2y)uT W
W
Gaussian
Super-Gaussian
Sub-Gaussian
si
Kurtosis
mx ( 4 ) E x mx
4
E ui4
E u
2
i
W 1 K tanh u uT uuT W,
k i 1, super Gaussian,
k i 1, sub Gaussian.
Switching Decision
W 1 K tanh uu uu W,
T
k i 1, super Gaussian,
k i 1, sub Gaussian.
Alternatives
Postulate vague distributions for the source signals
which are then developed iteratively during training.
Use an alternative approach, e.g, statistically
based, JADE (Cardoso).
Summary so far
We have seen how W may be obtained by training
the network, and the extended algorithm for
switching between super- and sub-Gaussians has
been described.
Alternative approaches have been mentioned.
Next we consider how to obtain the source signals
knowing W and the measured signals, x.
si
unknown
Mixing
matrix A
xi
Unmixing
matrix W
measured
g(u)
uisi
yi
estimated
Source Signals
U
M xN M xM
Channel
number
x
M xN
u
w
x
x
MN N 1
MM M 1
MN
M1
Time, or sample number
Procedure
Record N time points from each of M sensors, where N
5M.
Pre-process the data, e.g. filtering, trend removal.
Sphere the data using Principal Components Analysis
(PCA). This is not essential but speeds up the computation
by first removing first and second order moments.
Compute the ui si. Include desphering.
Analyse the results.
Optional Procedures I
The contribution of each activation at a
sensor may be found by back-projecting
it to the sensor.
x W 1s
1
1
w11
w12
w1M1
1
1
1
x w 21 w 22 w 2M
0
0
s21 s22
0
0
0 0
s23 s2N
0 0
1
1
1
x21 w 21
.0 w 22
.s21 w 2M1 .0 w 22
.s21
1
22
x22 w .s22 ,
1
22 2 N
x2M w s
Optional Procedures II
A measured signal which is contaminated by artefacts or
noise may be extracted by back-projecting all the signal
activations to the measurement electrode, setting other
activations to zero. (An artefact and noise removal method).
1
1
w11
w12
w1M1
1
1
1
x w 21 w 22 w 2M
s11 s12
s21 s22
0
0
s13 s1N
s23 s2N
0 0
1
1
1
1
x21 w 21
.s11 w 22
.s21 w 2M1 .0 w 21
.s11 w 22
.s21
1
21
1
22
1
21
1
22
Current Developments
Overcomplete representations - more signal
sources than sensors.
Nonlinear mixing.
Nonstationary sources.
General formulation of g(u).
Conclusions
It has been shown how to extract temporally
independent unknown source signals from their
linear mixtures at the outputs of an unknown
system using Independent Components
Analysis.
Some of the limitations of the method have been
mentioned.
Current developments have been highlighted.