Sie sind auf Seite 1von 21

Adaptive Filtering – An Introduction

Jos´e C. M. Bermudez

Department of Electrical Engineering Federal University of Santa Catarina Florian´opolis – SC Brazil

ENSEEIHT - Toulouse May 2009

of Electrical Engineering Federal University of Santa Catarina Florian´opolis – SC Brazil ENSEEIHT - Toulouse May

Linear Digital Filters

Statistical Linear Filtering

Optimal Linear Filtering

Adaptive Filtering

Adaptive Filter Applications

The LMS Adaptive Algorithm

Linear Filtering Optimal Linear Filtering Adaptive Filtering Adaptive Filter Applications The LMS Adaptive Algorithm

Regular Linear Digital Filters

FIR or IIR.

Most useful for frequency selective applications.

Desired signal and interference are in different frequency bands. Example: To extract high frequency noise (> 20 kHz) from speech.

interference are in different frequency bands. Example: To extract high frequency noise ( > 20 kHz)

FIR × IIR

FIR Filters

Simple to design

Can implement linear phase

Are always stable

Require coefficients than IIR filters

IIR Filters

Harder to design than FIR filters

Introduce phase distortion

May become unstable

Require coefficients than FIR filters

FIR filters ◮ Introduce phase distortion ◮ May become unstable ◮ Require ≪ coefficients than FIR

Statistical Linear Filtering

Many applications more than just frequency band selection

Signal and interference are frequently within the same freq. band

Solution is based on the statistical properties of the signa ls

are frequently within the same freq. band ◮ Solution is based on the statistical properties of

Basic Operations in Linear Filtering

1. Filtering Extraction of information at a given time n by using data measured before and at time n .

2. Smoothing Differs from filtering in that data measured both before and after n can be used to extract information at time n .

3. Prediction Predicts how the quantity of interest will be at time n + N for some N > 0 by using data measured up to and including n . (The filter is linear if its output is a linear function of the observations applied to the input.)

and including n . (The filter is linear if its output is a linear function of

How Do We Do Linear Statistical Filtering?

We assume some statistical properties of the useful and unwanted signals are available Mean Autocorrelation Cross-correlations etc.

We define some statistical criterion for the optimization of the filter performance

Cross-correlations etc. ◮ We define some statistical criterion for the optimization of the filter performance

Optimal Linear Filtering The Wiener Filter - The FIR Case

d(n )

+ + −
+
+

e(n )

u(n )

Filter - The FIR Case d ( n ) + + − e ( n )

FIR

Linear Filter

y (n )

u(n ) and d(n ) stationary and zero-mean

d(n ): information plus some interference

y (n ): estimate of d(n ) given some observations of u(n )

ˆ

y (n ) = w k u(n k ) = d(n |U n )

e(n ): estimation error

observations of u ( n ) ˆ y ( n ) = w k u (

d(n )

+ + −
+
+

e(n )

u(n )

d ( n ) + + − e ( n ) u ( n ) FIR

FIR

Linear Filter

y (n )

Performance index: Mean-Square Error

w

y (n ) = u T (n )w

J (w) = E {e 2 (n )}

= [w 1 ,

, w N ] T

u(n ) = [u(n ),

, u(n N + 1)] T

Mean-Square Error

= E { e 2 ( n ) } = [ w 1 , , w

The Wiener Weights

e(n ) = d(n ) u T (n )w

e 2 (n ) = d 2 (n ) 2d(n )u T (n )w + w T u(n )u T (n )w

J (w) = σ d (n ) 2p T (n )w + w T R(n )w

where:

2

p(n ) = E [d(n )u(n )] : cross-corr. between d(n ) and u(n )

R(n ) = E [u(n )u T (n )]

: autocorrelation matrix of u(n )

σ d = E [d 2 (n )]

Optimum Weight Vector

w = w o such that

2p + 2Rw o = 0

2

J (w) = 0 w

Rw o = p

(Normal Equations)

w = w o such that − 2 p + 2 Rw o = 0 2
w 1 + (309 w 2 )/1000 + + 1/2 3500 3000 2500 2000 1500
w 1 + (309 w 2 )/1000 +
+
1/2
3500
3000
2500
2000
1500
1000
500
0
20
10
0
10
−10
0
−10
−20
−20
w 2
w 1

20

w o : Minimum of this surface

Solution of the Normal Equations Computationally intensive even with efficient algorithms

: Minimum of this surface ◮ Solution of the Normal Equations Computationally intensive even with efficient

What is an Adaptive Filter Anyway?

In several practical real-time applications:

The signal involved are nonstationary

Normal equations must be solved for each n !?!

The algorithms used for the stationary case become inefficien t

What about Kalman Filters?

Requires a dynamics model (state-space) for d(n)

Computationally heavy for real-time

Signal statistics may be unknown, and there may be no time to estimate them

Computational complexity between 2 input samples limited by processor speed and by cost

be no time to estimate them ◮ Computational complexity between 2 input samples limited by processor

Adaptive Filters

Change their weights as new input samples arrive

Weight updating is controlled by an adaptive algorithm

Optimal solution is approached by improving performance a little bit at each iteration

Optimal solution is approximated after several iterations (iteration complexity × convergence time)

Filter weights become random variables that converge to a region about the optimum weights

Can track signal and system nonstationarities

random variables that converge to a region about the optimum weights ◮ Can track signal and

Application – Network Echo Cancellation

2−wire 4−wire 2−wire speaker A speech from A speaker B H H echo from A
2−wire
4−wire
2−wire
speaker A
speech from A
speaker B
H
H
echo from A
speech from B
x(n ) speech from A Alg. w(n ) h yˆ(n ) echo _ y (n
x(n )
speech from A
Alg.
w(n )
h
yˆ(n )
echo
_
y (n )
+
r (n )
+
e(n )
+ d(n )
speech from B

Application – Channel Equalization

Transmitted

symbols

Application – Channel Equalization Transmitted symbols Decision Device Channel Adaptive Equalizer _ + decision

Decision

Device

– Channel Equalization Transmitted symbols Decision Device Channel Adaptive Equalizer _ + decision switch Test

Channel

Adaptive Equalizer _ + decision
Adaptive
Equalizer
_
+
decision

switch

Test signal generator
Test signal
generator
+ +
+
+

noise

error

symbols Decision Device Channel Adaptive Equalizer _ + decision switch Test signal generator + + n

Application – Feedback Cancellation in Hearing Aids

Application – Feedback Cancellation in Hearing Aids
Application – Feedback Cancellation in Hearing Aids

Application – Feedback Cancellation in Hearing Aids

 

o

 
 

w

 

y 1 (n)

e(n)

 

+

+ +
+
+
   
+ +    
 

g(n)

 

mic

 

yˆ(n)

−
 

+

+

+
+
)   mic   y ˆ( n ) −   + + + w(n) y (
w(n)
w(n)

y(n)

x(n)

input

signal

u(n)

Figure: Basic adaptive feedback cancellation in hearing aids

  + + + w(n) y ( n ) x ( n ) input signal u

Adaptive Algorithm - Least Mean Squares (LMS)

Optimization Problem

Determine w that minimizes

J (w)| w=w(n) = σ d (n ) 2p T (n )w(n ) + w(n ) T R(n )w(n )

2

Steepest Descent Algorithm

Direction contrary to the gradient J (w) w

∂J (w)

w

= 2p + 2Rw(n )

w(n + 1) = w(n ) + µ [p Rw(n )]

(µ controls de adaptation speed)

∂ w = − 2 p + 2 Rw ( n ) w ( n +

Adaptive Algorithm - Least Mean Squares (LMS)

LMS Algorithm

Steepest descent needs p and R !?

Instantaneous approximation of the gradient

∂E [e 2 (n )] w(n )

∂e 2 (n ) w(n )

= 2e(n )u(n )

LMS weight updating equation

w(n + 1) = w(n ) + µe(n )u(n )

∂ w ( n ) = − 2 e ( n ) u ( n )

LMS – Typical Convergence Behavior

Mean Weight Error Vector

E [v(n )] = E [w(n )] w o

E[V(n)] 0.15 0.13 0.11 µ = 0 , 001 0.09 µ = 0 , 002
E[V(n)]
0.15
0.13
0.11
µ = 0 , 001
0.09
µ = 0 , 002
0.07
0.05
µ = 0 , 004
0.03
0.01
−0.01
−0.03
0 2e3
4e3
6e3
8e3
10e3 12e3 14e3 16e3 18e3 20e3

iterations

0.07 0.05 µ = 0 , 004 0.03 0.01 −0.01 −0.03 0 2e3 4e3 6e3 8e3

LMS – Typical Convergence Behavior

Mean−Square Error MSE (dB) 30 10 µ = 0 , 001 −10 µ = 0
Mean−Square Error
MSE (dB)
30
10
µ = 0 , 001
−10
µ = 0 , 002
−30
µ = 0 , 004
−50
−70
−90
−110
0 2e3
4e3
6e3
8e3
10e3 12e3 14e3 16e3 18e3 20e3

iterations

0 , 002 −30 µ = 0 , 004 −50 −70 −90 −110 0 2e3 4e3