1/19

3.11 CRLB Examples

1. Range Estimation
sonar, radar, robotics, emitter location
2. Sinusoidal Parameter Estimation (Amp., Frequency, Phase)
sonar, radar, communication receivers (recall DSB Example), etc.
3. Bearing Estimation
sonar, radar, emitter location
4. Autoregressive Parameter Estimation
speech processing, econometrics
Well now apply the CRLB theory to several examples of
practical signal processing problems.
Well revisit these examples in Ch. 7 well derive ML
estimators that will get close to achieving the CRLB
2/19
max ,
) ; (
0 ) ( ) ( ) (
o s
t s
o
T T t t w t s t x
o

+ = + =
!" !# $
Transmit Pulse: s(t) nonzero over t[0,T
s
]
Receive Reflection: s(t
o
)
Measure Time Delay:
o
C-T Signal Model
Bandlimited
White Gaussian
t T
s
s(t)
t T
s(t
o
)
BPF
& Amp
x(t)
PSD of w(t)
f
B B
N
o
/2
Ex. 1 Range Estimation Problem
3/19
Sample Every = 1/2B sec
w[n] = w(n)
DT White
Gaussian Noise
Var
2
= BN
o
1 , , 1 , 0 ] [ ) ( ] [ = + = N n n w n s n x
o

f
ACF of w(t)

1/2B
B B
1/B 3/2B
PSD of w(t)
N
o
/2

2
= BN
o

+
+ +

=
1 ] [
1 ] [ ) (
1 0 ] [
] [
N n M n n w
M n n n n w n s
n n n w
n x
o
o o o
o

s[n;
o
]has Mnon-zero samples starting at n
o
Range Estimation D-T Signal Model
4/19
Now apply standard CRLB result for signal + WGN:

=
=
+
=
=
+
=

=
|
|
.
|

\
|

=
|
|
.
|

\
|

=
|
|
.
|

\
|

=
|
|
.
|

\
|

1
0
2
2
1
2
2
1
2
2
1
0
2
2
) (
) (
) ( ] ; [
)

var(
M
n
n t
M n
n n
n t
M n
n n
o
o
N
n
o
o
o
t
t s
t
t s
n s n s
o
o
o
o
o

Plug in and keep

non-zero terms
Exploit Calculus!!!
Use approximation:
o
= n
o
Then do change of variables!!
Range Estimation CRLB
5/19
s
T
o
s
T
o
T
o
E
dt
t
t s
N
E
dt
t
t s
N
dt
t
t s
s s s

|
.
|

\
|

=
|
.
|

\
|

=
|
.
|

\
|

0
2
0
2
0
2
2
) (
2 /
1
) (
2 /
) ( 1
)

var(

Assume sample spacing is smallapprox. sum by integral

=
s
T
s
dt t s E
0
2
) (
( )
( )

dt f S
df f S f
N
E
E
df f S f
N
E
o
s
s
T
o
s
o
s
2
2
2
0
2
2
) (
) ( 2
2 /
1
) ( 2
2 /
1
)

var(

FT Theorem
& Parseval
Parseval
( )

=
dt f S
df f S f
B
rms
2
2
2
) (
) ( 2
Define a BW measure:
B
rms
is RMS BW (Hz)
A type of SNR
Range Estimation CRLB (cont.)
6/19
Using these ideas we arrive at the CRLB on the delay:
) (sec
1
)

var(
2
2
rms
o
B SNR

To get the CRLB on the range use transf. of parms result:
) (m
4 /
)

var(
2
2
2
rms
B SNR
c
R

o
CRLB
R
CRLB
o
R

|
|
.
|

\
|

= with R = c
o
/ 2
CRLB is inversely proportional to:
SNR Measure
RMS BW Measure
CRLB is inversely proportional to:
SNR Measure
RMS BW Measure
So the CRLB tells us
Choose signal with large B
rms
Ensure that SNR is large
Better on Nearby/large targets
Which is better?
Double transmitted energy?
Double RMS bandwidth?
Range Estimation CRLB (cont.)
7/19
1 , , 1 , 0 ] [ ) cos( ] [ = + + = N n n w n A n x
o

Given DT signal samples of a sinusoid in noise.
Estimate its amplitude, frequency, and phase
DT White Gaussian Noise
Zero Mean & Variance of 2

o
is DT frequency in
cycles/sample: 0 <
o
<
Multiple parameters so parameter vector:
T
o
A ] [ =
Recall SNR of sinusoid in noise is:
2
2
2
2
2
2 /

A A
P
P
SNR
n
s
= = =
Ex. 2 Sinusoid Estimation CRLB Problem
8/19
Approach:
Find Fisher Info Matrix
Invert to get CRLB matrix
Look at diagonal elements to get bounds on parm variances
Recall: Result for FIM for general Gaussian case specialized to
signal in AWGN case:
j
N
n
i
T
j i
ij
n s n s

=
|
|
.
|

\
|

|
|
.
|

\
|

=
] ; [ ] ; [ 1
1
)] ( [
1
0
2
2

s s
I

Sinusoid Estimation CRLB Approach
9/19
Taking the partial derivatives and using approximations given in
book (valid when
o
is not near 0 or ) :
( )
( )
2
2 1
0
2 2
2
33
1
0
2
2 1
0
2 2
2
32 23
1
0
2
2
2 1
0
2
2
2 1
0
2 2 2
2
22
1
0
2
1
0
2
31 13
1
0
2
1
0
2
21 12
2
1
0
2
1
0
2
2
11
2
) ( sin
1
)] ( [
2
) ( sin
1
)] ( [ )] ( [
2
) 2 2 cos( 1
2
) ( sin ) (
1
)] ( [
0 ) 2 2 sin(
2
) sin( ) cos(
1
)] ( [ )] ( [
0 ) 2 2 sin(
2
) sin( ) cos(
1
)] ( [ )] ( [
2
) 2 2 cos( 1
2
1
) ( cos
1
)] ( [

NA
n A
n
A
n n A
n
A
n n
A
n n A
n
A
n n A
n n
A
n n An
N
n n
N
n
o
N
n
N
n
o
N
n
N
n
o
N
n
o
N
n
o
N
n
o o
N
n
o
N
n
o o
N
n
o
N
n
o
+ =
+ = =
+ = + =
+

= + +

= =
+

= + +

= =
+ + = + =

=
I
I I
I
I I
I I
I
T
o
A ] [ =
Sinusoid Estimation Fisher Info Elements
10/19

=
2
2 1
0
2
2
1
0
2
2 1
0
2
2
2
2
2 2
0
2 2
0
0 0
2
) (

NA
n
A
n
A
n
A
N
N
n
N
n
N
n
I
Fisher Info Matrix then is:
2
2
2
A
SNR =
Recall and closed form results for these sums
T
o
A ] [ =
Sinusoid Estimation Fisher Info Matrix
11/19
Inverting the FIM by hand gives the CRLB matrix and then
extracting the diagonal elements gives the three bounds:
(using co-factor &det
approach helped by 0s)
) rad (
4
) 1 (
) 1 2 ( 2
)

var(
) ) rad/sample ((
) 1 (
12
)

var(
) volts (
2
)

var(
2
2
2
2
2
N SNR N N SNR
N
N N SNR
N
A
o

Amp. Accuracy: Decreases as 1/N, Depends on Noise Variance (not SNR)

Freq. Accuracy: Decreases as 1/N
3
, Decreases as as 1/SNR
Phase Accuracy: Decreases as 1/N, Decreases as as 1/SNR
To convert to Hz
2
multiply by (F
s
/2)
2
Sinusoid Estimation CRLBs
12/19
The CRLB for Freq. Est. referred back to the CT is
) Hz (
) 1 ( ) 2 (
12
)

var(
2
2 2
2

N N SNR
F
f
s
o

Does that mean we do worse if we sample faster than Nyquist?

NO!!!!! For a fixed duration T of signal: N = TF
s
Also keep in mind that F
s
has effect on the noise structure:
Not in Book
f
ACF of w(t)

1/2B
B B
1/B 3/2B
PSD of w(t)
N
o
/2

2
= BN
o
Frequency Estimation CRLBs and Fs
13/19
Uniformly spaced linear array with Msensors:
Sensor Spacing of d meters
Bearing angle to target radians
Figure 3.8
from textbook:
Simple model
Emits or reflects
signal s(t)
) 2 cos( ) ( + = t f A t s
o t
Propagation Time to n
th
Sensor:
1 , , 1 , 0 cos
0
= = M n
c
d
n t t
n

|
|
.
|

\
|
+
|
.
|

\
|
+ =
=

cos 2 cos
) ( ) (
0
c
d
n t t f A
t t s t s
o
n n
Signal at n
th
Sensor:
Ex. 3 Bearing Estimation CRLB Problem
14/19
Now instead of sampling each sensor at lots of time instants
we just grab one snapshot of all Msensors at a single instant t
s
( )

~
cos
~
cos
2
cos
cos 2 cos ) (
0
+ =
|
|
|
|
|
|
|
|
.
|

\
|
+
|
|
|
|
|
.
|

\
|
|
.
|

\
|
=
|
|
.
|

\
|
+
|
.
|

\
|
+ =

n A n d
c
f
A
c
d
n t t f A t s
s
o
s o s n
s
s
! ! ! " ! ! ! # $
! !" ! !# $
Spatial sinusoid w/
spatial frequency
s
Spatial Frequencies:

s
is in rad/meter

s
is in rad/sensor
For sinusoidal transmitted signal Bearing
Est. reduces to Frequency Est.
And we already know its FIM & CRLB!!!
Bearing Estimation Snapshot of Sensor Signals
15/19
( ) ] [
~
cos ] [ ) ( ] [ n w n A n w t s n x
s s n
+ + = + =
Each sample in the snapshot is corrupted by a noise sample
and these Msamples make the data vector x = [x[0] x[1] x[M-1] ]:
Each w[n] is a noise sample that comes from a different sensor so
Model as uncorrelated Gaussian RVs (same as white temporal noise)
Assume each sensor has same noise variance
2
So the parameters to consider are:
T
s
A ]

[ =
which get transformed to:

|
|
.
|

\
|

=

= =

2
arccos

) (
d f
c
A A
o
s
g
Parameter of interest!
Bearing Estimation Data and Parameters
16/19
Using the FIM for the sinusoidal parameter problem together
with the transform. of parms result (see book p. 59 for details):
( )
) rad (
sin
1
1
) 2 (
12
)

var(
2
2
2
2

|
.
|

\
|

L
M
M
M SNR
Bearing Accuracy:
Decreases as 1/SNR Depends on actual bearing
Decreases as 1/M ! Best at = /2 (Broadside)
Decreases as 1/L
r
2
! Impossible at = 0! (Endfire)
L = Array physical length in meters
M = Number of array elements
= c/f
o
Wavelength in meters (per cycle)
Define: L
r
= L/
Array Length in
wavelengths
Low-frequency (i.e., long wavelength) signals need
very large physical lengths to achieve good accuracy
Bearing Estimation CRLB Result
17/19
In speech processing (and other areas) we often model the
signal as an AR random process and need to estimate the AR
parameters. An AR process has a PSD given by
2
1
2
2
] [ 1
) ; (

+
=
p
m
fm j
u
xx
e m a
f P

AR Estimation Problem: Given data x[0], x[1], , x[N-1]

estimate the AR parameter vector
| |
T
u
p a a a
2
] [ ] 2 [ ] 1 [ & =
This is a hard CRLB to find exactly but it has been published.
The difficulty comes from the fact that there is no easy direct
relationship between the parameters and the data.
It is not a signal plus noise problem
Ex. 4 AR Estimation CRLB Problem
18/19
Approach: The asymptotic result we discussed is perfect here:
An AR process is WSS is required for the Asymp. Result
Gaussian is often a reasonable assumption needed for Asymp. Result
The Asymp. Result is in terms of partial derivatives of the PSD and
that is exactly the form in which the parameters are clearly displayed!
| |
| | | |
df
f P f P N
j
xx
i
xx
ij

2
1
2
1
) ; ( ln ) ; ( ln
2
) (


I
2
1
2 2
2
1
2
2
] [ 1 ln ln
] [ 1
ln ) ; ( ln

+ =
+
=
p
m
fm j
u
p
m
fm j
u
xx
e m a
e m a
f P

Recall:
AR Estimation CRLB Asymptotic Approach
19/19
After taking these derivatives you get results that can be
simplified using properties of FT and convolution.
The final result is:
| |
N
p k
N
k a
u
u
kk
xx
u
4
2
1
2
2
)

var(
, , 2 , 1 ]) [

var(

=

R
Both Decrease
as 1/N
To get a little insight look at 1
st
order AR case (p = 1):
]) 1 [ 1 (
1
]) 1 [

var(
2
a
N
a
Complicated
dependence on
AC Matrix!!
Improves as pole
gets closer to unit
circle PSDs
with sharp
peaks are easier
to estimate
a[1]
Re(z)
Im(z)
AR Estimation CRLB Asymptotic Result