Correlated and Uncorrelated Signals

Problem: we have two signals and . How close are they to each other?
] [n x ] [n y
Example: in a radar (or sonar) we transmit a pulse and we expect a return
0 20 40 60 80 100 120 140 160 180
-2
-1.5
-1
-0.5
0
0.5
1
1.5
0 2 4 6 8 10 12 14 16 18 20
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Transmit
Receive
Example: Radar Return
Since we know what we are looking for, we keep comparing what we receive
with what we sent.
0 20 40 60 80 100 120 140 160 180
-2
-1.5
-1
-0.5
0
0.5
1
1.5
0 2 4 6 8 10 12 14 16 18 20
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Receive
0 2 4 6 8 10 12 14 16 18 20
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Similar?
NO! Think so!
Inner Product between two Signals
We need a measure of how close two signals are to each other.
This leads to the concepts of
Inner Product
Correlation Coefficient
Inner Product
Problem: we have two signals and . How close are they to each other?
] [n x ] [n y
Define: Inner Product between two signals of the same length

=
=
1
0
*
] [ ] [
N
n
xy
n y n x r
Properties:
0 ] [ ] [ ] [
1
0
2
1
0
*
> = =


=

=
N
n
N
n
xx
n x n x n x r
yy xx xy
r r r s
2
yy xx xy
r r r =
2
if and only if
] [ ] [ n x C n y = for some constant C
How we measure similarity (correlation coefficient)
yy xx
xy
xy
r r
r | |
=
Compute:
Check the value:
1 0 s s
xy

1 ~
xy

x,y strongly correlated x,y uncorrelated

0 ~
xy

Assume: zero mean

003 . 0
982
500
27 . 2
=
=
=
=
xy
yy
xx
xy
r
r
r

Back to the Example: with no return

0 100 200 300 400 500 600 700 800 900
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
0 100 200 300 400 500 600 700 800 900
-3
-2
-1
0
1
2
3
0 100 200 300 400 500 600 700 800 900 1000
-3
-2
-1
0
1
2
3
] [n x
] [n y ] [ ] [ n y n x
NO Correlation!
Back to the Example: with return
8 . 0
754
500
494
=
=
=
=

yy
xx
xy
r
r
r
0 100 200 300 400 500 600 700 800 900
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
0 100 200 300 400 500 600 700 800 900
-3
-2
-1
0
1
2
3
0 100 200 300 400 500 600 700 800 900 1000
-0.5
0
0.5
1
1.5
2
2.5
] [n x ] [n y ] [ ] [ n y n x
Good Correlation!
Inner Product in Matlab
| | ) ( ) 2 ( ) 1 ( N x x x x =
| | ) ( ) 2 ( ) 1 ( N y y y y =
Row vector
Row vector
| |
(
(
(
(
(

= =

=
) (
) 2 (
) 1 (
) ( ) 2 ( ) 1 ( ) ( ) (
*
*
*
1
*
N y
y
y
N x x x n y n x r
N
n
xy

' * y x r
xy
=
x
' y
conjugate,
transpose
Take two signals of the same length. Each one is a vector:
Define: Inner Product between two vectors
Example
Take two signals:
0 50 100 150 200 250 300
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
x
y
0 50 100 150 200 250 300
-3
-2
-1
0
1
2
3
Compute these:
Then:
0 0856 . 0
9 . 241 8 . 218
7 . 19
~ =

=
xy

x,y are not correlated

7 . 19 ' * = = y x r
xy
8 . 218 ' * = = x x r
xx
9 . 241 ' * = = y y r
yy
Example
Take two signals:
x
y
9 . 230 ' * = = y x r
xy
6 . 229 ' * = = x x r
xx
3 . 234 ' * = = y y r
yy
Compute these:
Then:
1 9955 . 0
3 . 234 6 . 229
9 . 230
~ =

=
xy

x,y are strongly correlated

0 50 100 150 200 250 300
-3
-2
-1
0
1
2
3
0 50 100 150 200 250 300
-3
-2
-1
0
1
2
3
Example
Take two signals:
Compute these:
Then:
1 9955 . 0
3 . 234 6 . 229
9 . 230
~ =

=
xy

x,y are strongly correlated

x
y
0 50 100 150 200 250 300
-3
-2
-1
0
1
2
3
0 50 100 150 200 250 300
-3
-2
-1
0
1
2
3
9 . 230 ' * = = y x r
xy
6 . 229 ' * = = x x r
xx
3 . 234 ' * = = y y r
yy
Typical Application: Radar
] [n s
n
Send a Pulse
] [n y
n
0
n
and receive it back with noise, distortion
N
Problem: estimate the time delay , ie detect when we receive it.
0
n
Use Inner Product
Slide the pulse s[n] over the received signal and see when
the inner product is maximum:
] [ s

] [ y

0
n
N
n

=
+ =
1
0
*
] [ ] [ ] [
N
ys
s n y n r

0
if , 0 ] [ n n n r
ys
= ~
Use Inner Product
] [ s

Slide the pulse x[n] over the received signal and see when
the inner product is maximum:
] [ y

N
0
n =
if
0
n n =
MAX s n y n r
N
ys
= + =

=
1
0
*
] [ ] [ ] [

Matched Filter
Take the expression
] [ ] 0 [ ] 1 [ ] 1 [ ... ] 1 [ ] 1 [
] [ ] [ ] [
* * *
1
0
*
n y s n y s N n y N s
s n y n r
N
n
ys
+ + + + + =
+ =

=

Then
] 1 [ ] 1 [ ] 1 [ ] 1 [ ... ] [ ] 0 [ ] [ + + + + = N n y N h n y h n y h n r
] [n y
] [n h
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
] 1 [ ] [ + = N n r n r
ys
Compare this, with the output of the following FIR Filter
Matched Filter
This Filter is called a Matched Filter
The output is maximum when
] [n y
] [ n r
] [n h
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
] 1 [ ] [ + = N n r n r
ys
0
1 n N n = +
1
0
+ = N n n i.e.
Example
0 2 4 6 8 10 12 14 16 18 20
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
] [n s
0 20 40 60 80 100 120 140 160 180 200
-6
-4
-2
0
2
4
6
8
10
12
0 20 40 60 80 100 120 140 160 180
-2
-1.5
-1
-0.5
0
0.5
1
1.5
] [n y
] [n y
] [ n r
] [n h
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
1 ,..., 0 ], [ = N n n s
We transmit the pulse shown below, with
length
20 = N
Received signal:
Max at n=119
100 1 20 119
0
= + = n
How do we choose a good pulse
1 ,..., 0 ], [ = N n n s
We transmit the pulse and we receive
(ignore the noise for the time being)
] [
] [ ] [ ] [
0
1
0
*
0
n n r A
s n n s A n r
ss
N
n
ys
=
+ =

=

] [ ] [
0
n n As n y =
] [ n r
] [n h
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
] 1 [ ] [ + = N n r n r
ys
where
The term
is called the autocorrelation of s[n]. This characterizes
the pulse.

=
+ =
1
0
*
] [ ] [ ] [
N
ss
s n s n r

Example: a square pulse
? ] [ ] [ ] [
1
0
*
= + =

=
N
ss
s n s n r

N s s s r
N N
ss
= = =


=

=
1
0
2
1
0
*
] [ ] [ ] [ ] 0 [


1 1 ] [ ] 1 [ ] 1 [
2
0
2
0
*
= = + =


=

=
N s s r
N N
ss


k N s k s k r
k N k N
ss
= = + =


=

=
1
0
1
0
*
1 ] [ ] [ ] [


] [n r
ss ] [n s
n 1 N
1
0
N
N N n
See a few values:
k N s k s k r
N
k
k N
ss
= = + =


=

=
1 1
0
*
1 ] [ ] [ ] [


Compute it in Matlab
] [n s
n 1 N
1
0
N=20; % data length
s=ones(1,N); % square pulse
rss=xcorr(s); % autocorr
n=-N+1:N-1; % indices for plot
stem(n,rss) % plot

-20 -15 -10 -5 0 5 10 15 20
0
2
4
6
8
10
12
14
16
18
20
Example: Sinusoid
49 ,..., 0 ], [ = n n s
-50 -40 -30 -20 -10 0 10 20 30 40 50
-20
-15
-10
-5
0
5
10
15
20
25
0 5 10 15 20 25 30 35 40 45 50
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
49 ,..., 49 ], [ = n n r
ss
Example: Chirp
49 ,..., 49 ], [ = n n r
ss
49 ,..., 0 ], [ = n n s
0 5 10 15 20 25 30 35 40 45 50
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
-50 -40 -30 -20 -10 0 10 20 30 40 50
-10
-5
0
5
10
15
20
25
30
s=chirp(0:49,0,49,0.1)
Example: Pseudo Noise
49 ,..., 49 ], [ = n n r
ss
s=randn(1,50)
49 ,..., 0 ], [ = n n s
0 5 10 15 20 25 30 35 40 45 50
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-50 -40 -30 -20 -10 0 10 20 30 40 50
-20
-10
0
10
20
30
40
50
Compare them
-50 -40 -30 -20 -10 0 10 20 30 40 50
-20
-10
0
10
20
30
40
50
-50 -40 -30 -20 -10 0 10 20 30 40 50
-10
-5
0
5
10
15
20
25
30
0 5 10 15 20 25 30 35 40 45 50
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
0 5 10 15 20 25 30 35 40 45 50
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0 5 10 15 20 25 30 35 40 45 50
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
-50 -40 -30 -20 -10 0 10 20 30 40 50
-20
-15
-10
-5
0
5
10
15
20
25
] [n s
] [n r
ss
cos
chirp pseudonoise
Two best!
Detection with Noise
Now see with added noise
] [ ] [ ] [
0
n w n n As n y + =
] [n h
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
] [ ] 1 [ ] [
0
n r N n n r n r
yw ys
+ + =
White Noise
A first approximation of a disturbance is by White Noise.
White noise is such that any two different samples are
uncorrelated with each other:
0 100 200 300 400 500 600 700 800 900 1000
-4
-3
-2
-1
0
1
2
3
] [n w
White Noise
The autocorrelation of a white noise signal tends to be a
delta function, ie it is always zero, apart from when n=0.
] [n r
ss
n
White Noise and Filters
The output of a Filter
] [n w
] [n h

=
=
1
0
] [ ] [ ] [
N
n w h n w

|
.
|

\
|
|
.
|

\
|
=
|
.
|

\
|
=
|
|
.
|

\
|
=

=
1
0
2
1
0
2
1
0
1
0
1
0
2 1 2 1
1
0
1
0
1
0
2 1 2 1
1
0
2
] [
1
] [
] [ ] [
1
] [ ] [
] [ ] [ ] [ ] [
1
] [
1
1 2
1 2
M
n
N
N N M
n
M
n
N N M
n
n w
M
h
n w n w
M
h h
n w n w h h
M
n w
M

White Noise
The output of a Filter
] [n w
] [n h

=
=
N
n w h n w
0
] [ ] [ ] [

In other words the Power of the Noise at the ouput is
related to the Power of the Noise at the input as
w
N
n
W
P n h P |
.
|

\
|
=

=
1
0
2
] [
Back to the Match Filter
At the peak:
] [ ] [ ] [
0
n w n n As n y + =
] [n h
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
] [ ] 1 [ ] [
0
n w N n n Ar n r
ss
+ + =
] 1 [ ] 0 [ ] 1 [

0 0
+ + = + N n w Ar N n r
ss
Match Filter and SNR
At the peak:
] 1 [ ] 0 [ ] 1 [

0 0
+ + = + N n r Ar N n r
sw ss
|
.
|

\
|
|
.
|

\
|
=


=

=
1
0
2
1
0
2
2
| ] [ | | ] [ | ] 0 [
N
n
N
n
ss
n s n As Ar
W
N
n
W
P n s P |
.
|

\
|
=

=
1
0
2
| ] [ |
SNR N
P n s
n s P N
SNR
W
N
n
N
n
S
peak
=
|
.
|

\
|
|
.
|

\
|

=

=
1
0
2
1
0
2
] [
] [
Example
Transmit a Chirp of length N=50 samples, with SNR=0dB
0 50 100 150 200 250 300
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
0 200 400 600 800 1000 1200
-15
-10
-5
0
5
10
15
20
25
30
Transmitted
Detected with
Matched Filter
Example
Transmit a Chirp of length N=100 samples, with SNR=0dB
0 50 100 150 200 250 300
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
0 200 400 600 800 1000 1200
-20
-10
0
10
20
30
40
50
Transmitted
Detected with
Matched Filter
Example
Transmit a Chirp of length N=300 samples, with SNR=0dB
0 50 100 150 200 250 300
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
0 200 400 600 800 1000 1200 1400
-40
-20
0
20
40
60
80
100
120
140
160
Transmitted
Detected with
Matched Filter