1

Alternate Form for CRLB

!
"
!
#
$
!
%
!
&
'
(
)
*
+
,
-
.
.
/
2
) ; ( ln
1
)
`
var(
0
0
0
x !
"
See Appendix 3A
Ior Derivation
Sometimes it is easier to Iind the CRLB this way.
This also gives a new viewpoint oI the CRLB:
From Gardner`s Paper (#""" %&'()* +( #(,+ %-.+&/, July 1979)
Consider the Normalized version oI this Iorm oI CRLB
Posted
on BB
!
"
!
#
$
!
%
!
&
'
(
)
*
+
,
-
.
.
/
2
2
2
) ; ( ln
1 )
`
var(
0
0
0
0
0
x !
"
We`ll 'derive
this in a way
that will re-
interpret the
CRLB
2
Consider the 'Incremental Sensitivity oI !(x;0 ) to changes in 0 0
II 0 1 0 120, then it causes !(x;0 ) 1 !(x;0 120 )
How sensitive is !(x;0 ) to that change??
(
)
*
+
,
-
(
)
*
+
,
-
2
2
3 3
(
)
*
+
,
- 2
(
)
*
+
,
- 2
3
2
) ; (
) ; (
in change
) ; ( in change
) ; (
) ; (
) (
~
0
0
0
0
0
0
0
0
0
0
0
x
x x
x
x
x
!
! !
!
!
2
!
Now let 20 1 0:
0
0
0
0
0
0
0
0
0
0
.
.
3
(
)
*
+
,
-
(
)
*
+
,
-
.
.
3 3
1 2
) ; ( ln
) ; (
) ; (
) (
~
lim ) (
0
x
x
x
x x
!
!
!
2 2
! !
Recall Irom Calculus:
3
3 ,
3 , 3
3 ,
.
.
3
.
. ) (
) (
1 ) ( ln
4 5
"
#
$
%
&
'
3
!
"
!
#
$
!
%
!
&
'
(
)
*
+
,
-
.
.
/
2
2
2
2
2
) (
1
) ; ( ln
1 )
`
var(
x
x
!
2 "
!
"
0
0
0
0
0
0
0
Interpretation
Norm. CRLB
Inverse Mean
Square
Sensitivity
3
Definition of Fisher Information
The denominator in CRLB is called the Fisher InIormation #(0 )
It is a measure oI the 'expected goodness oI the data Ior the
purpose oI making an estimate
!
"
!
#
$
!
%
!
&
'
.
.
6 3
2
2
) ; ( ln
) (
0
0
0
x !
" #
Has the needed properties Ior 'inIo (as does 'Shannon InIo):
1. #(0 ) / 0 (easy to see using the alternate Iorm oI CRLB)
2. #(0 ) is additive Ior independent observations
Iollows Irom:
4 5
7 8
3
(
(
)
*
+
+
,
-
3
( (
( 3 ! ( 3 ! ! ) |; | ( ln ) |; | ( ln ) ; ( ln 0 0 0 x
II each #
(
(0 ) is the same: #(0 ) 49#(0 )
4
3.5 CRLB for Signals in AWGN
When we have the case that our data is 'signal AWGN then
we get a simple Iorm Ior the CRLB:
Signal Model: 3|(| )|(;0 | 5|(|, ( 0, 1, 2, . , 4-1
White,
Gaussian,
Zero Mean
Q: What is the CRLB?
First write the likelihood Iunction:
: ;
: ;
!
"
!
#
$
!
%
!
&
'
6
6
3
7
6
3
1
0
2
2 2 /
2
| ; | | |
2
1
exp
2
1
) ; (
4
(
4
( ) ( 3 ! 0
<
=<
0 x
DiIIerentiate Log LF twice to get:
: ;
7
6
3
!
"
!
#
$
!
%
!
&
'
(
)
*
+
,
-
.
.
6
.
.
6 3
.
.
1
0
2
2
2
2 2
2
| ; | | ; |
| ; | | |
1
) ; ( ln
4
(
( ) ( )
( ) ( 3 !
0
0
0
0
0
<
0
0
x
Depends on
random 3|(|
so must take
E}
5
> ?
2
1
0
2
1
0
2
2
2
0
| ; |
2 2
2
| ; |
| ; | | ; |
| ; | | |
1
) ; ( ln
<
0
0
0
0
0
0
0
<
0
0
0
7
7
6
3
6
3
3
(
)
*
+
,
-
.
.
6
3
!
!
"
!
!
#
$
!
!
%
!
!
&
'
(
)
*
+
,
-
.
.
6
.
.
@
@
@
A
B
C
C
C
D
E
6 3
!
"
!
#
$
!
%
!
&
'
.
.
4
(
4
(
( )
( )
( ) ( )
( ) ( 3 " ! "
! ! ! " ! ! ! # $
" # $
x
Then using this we get the CRLB for Signal in AWGN:
7
6
3
(
)
*
+
,
-
.
.
/
1
0
2
2
| ; |
)
`
var(
4
(
( )
0
0
<
0
2
| ; |
(
)
*
+
,
-
.
.
0
0 ( )
Note: tells how
sensitive signal is to parameter
II signal is very sensitive to parameter change. then CRLB is small
. can get very accurate estimate!
6
Ex. 3.5: CRLB of Frequency of Sinusoid
Signal Model:
1 , , 2 , 1 , 0 0 | | ) 2 cos( | |
2
1
6 3 F F G G 3 4 ( , ( 5 ( , 6 ( 3
+ +
% H =
0 0.1 0.2 0.3 0.4 0.5
0
2
4
6
x 10
-4
I
o
(cycles/sample)
C
R
L
B


(
c
y
c
l
e
s
/
s
a
m
p
l
e
)
2
0 0.1 0.2 0.3 0.4 0.5
0.01
0.015
0.02
0.025
I
o
(cycles/sample)
C
R
L
B
1
/
2


(
c
y
c
l
e
s
/
s
a
m
p
l
e
)
Error in
Book
Bound on
Variance
Bound on
Std. Dev.
4 5
7
6
3
G 9
/
1
0
2
) 2 sin( 2
1
)
`
var(
4
(
+
( , ( 247 H = =
0
Signal is less
sensitive iI ,
+
near 0 or
7
3.6 Transformation of Parameters
Say there is a parameter 0 with known CRLB
0
But imagine that we instead are interested in estimating
some other parameter I that is a Iunction oI 0 :
I 8(0 )
Q: What is CRLB
I
?
0 I
0
0
I 97:;
8
97:;
2
) (
) var(
@
A
B
C
D
E
.
.
3 /
Captures the
sensitivity oI I to 0
Proved in
Appendix 3B
Large .g/. 0 1 small error in 0 gives larger error in I
1 increases CRLB (i.e., worsens accuracy)
8
Example: Speed of Vehicle From Elapsed Time
Known Distance <
start
Laser
Sensor Sensor
Laser
stop
Measure Elapsed Time %
Possible Accuracy Set by CRLB
%
%
%
% =
97:;
<
=
97:;
%
<
97:;
%
<
%
97:;
9 3
9
@
A
B
C
D
E
6 3
9
(
)
*
+
,
-
@
A
B
C
D
E
.
.
3
2
4
2
2
But. really want to measure speed = >/%
Find the CRLB
=
:
) / (
2
) ? 97:;
<
=
% =
/ <
Accuracy Bound
Less accurate at High Speeds (quadratic)
More accurate over large distances
2
9
Effect of Transformation on Efficiency
Suppose you have an eIIicient estimator oI 0 : 0
`
But. you are really interested in estimating I 8(0 )
Suppose you plan to use )
`
(
`
0 I 8 3
Q: Is this an eIIicient estimator oI I ???
A: Theorem: II 8(0 ) has Iorm 8(0 ) '0 @, then
is eIIicient.
)
`
( ` 0 I 8 3
~affine transform
Proof:
First: : ; : ; : ;
0
0 0 I 97:; ' ' @ '
2 2
`
var
`
var
`
var 3 3 G 3
' because 'eIIicient
Now, what is CRB
I
? Using transIormation result:
: ;
0 0 I
0
0
97:; ' 97:;
@ '
97:;
'
2
2
2
3
(
)
*
+
,
-
.
G .
3
3
! !" ! !# $
: ;
I
I 97:; 3
`
var
EIIicient!
10
Asymptotic Efficiency Under Transformation
II the mapping I 8(0 ) is not aIIine. this result does NOT hold
But. iI the number oI data samples used is large, then the
estimator is approximately eIIicient ('Asymptotically Efficient)
0
`
)
`
( ),
`
(
`
0 0 I ! 8 3
0
`
oI pdI
Small ! Case
PDF is widely spread
over nonlinear mapping
0
`
)
`
( ),
`
(
`
0 0 I ! 8 3
0
`
oI pdI
Large ! Case
PDF is concentrated
onto linearized section