Beruflich Dokumente
Kultur Dokumente
Professor, Department of Mathematics, Malaviya National Institute of Technology, Jaipur (Rajasthan), India
Ph.d Scholar, Department of Mathematics, Malaviya National Institute of Technology, Jaipur (Rajasthan), India
Abstract
Non-parametric measures give the amount of information supplied by the data for discriminating in favor of a probability
distribution P against another Q , or for measuring the distance or affinity between P and Q .
There are several generalized functional divergences, such as: Csiszar divergence, Renyi- like divergence, Bregman divergence,
Burbea- Rao divergence etc. all. In this paper, a non-parametric non symmetric measure of divergence which belongs to the
family of Csiszrs f-divergence is proposed. Its properties are studied and get the bounds in terms of some well known divergence
measures.
Let n P p1 , p2 , p3 ..., pn : pi 0, pi 1 , n 2
i 1
0
then we have to suppose that 0 f 0 0 f 0 .
0
pi qi
i 1
qi
P, Q
h P, Q
Ra P, Q
(1.5)
n
B P, Q pi qi
Bhattacharya
i 1
and
Hellinger
pi a
, a 1 = Renyis a
qi a 1
function
(1.4)
i 1
(1.1)
P p1 , p2 , p3 ..., pn , Q q1 , q2 , q3 ..., qn n,
where
pi qi
Square
(1.3)
convex
Chi-
discrimination [3]
and
i 1
p
C f P, Q qi f i
i 1
qi
(1.6)
n
1
(1.7)
n
p
K P, Q pi log i = Kullback- Leibler
i 1
qi
(1.2)
_______________________________________________________________________________________
Volume: 03 Issue: 05 | May-2014, Available @ http://www.ijret.org
665
Particularly
700
lim s P, Q K P, Q , lim s P, Q K Q, P
s 1
600
s 0
500
(1.8)
400
300
p qi
G P, Q i
2
i 1
n
pi qi
log 2 p
200
100
(1.9)
2
2.
NEW
INFORMATION
DIVERGENCE
MEASURE
10
C f P, Q V
pi qi
i 1
pi qi
P, Q
(2.3)
P, Q n n .
t 1
t
minimum value).
, t 0,
Since
f t
t 1
t
3t 1 ,
f t
2 t 1
t
6
2
3t
2t 1
(2.2)
Since
f t 0 t 0, f t is a convex
function.
Since
Since
f 1 0 f t is a normalized function.
f t 0 at
0,1 and
5
4
V * P, Q V * Q, P V * P, Q is non-
(2.1)
And
3
P, Q n n .
V * P, Q 0 if P Q or pi qi (Attains its
f t
distribution
f t 0 at
1, f t is monotonically decreasing in
0,1 and monotonically increasing in 1, , and
f 1 0 .
V*(P,Q)
3
2
1
0
0
0.5
Relative
Arithmetic
Geometric div.
KullbackLeibler div.
a
Figure 2 shows the behavior of V * P, Q , Relative
Arithmetic-Geometric divergence G P, Q and KullbackLeibler divergence K P, Q . We have considered
666
P, Q has a
Let
AND INEQUALITIES
2
12 2
1 2
0 V P, Q max. 3 3 2 1 , 3 3 2 1 2 P, Q
*
*
0 V P, Q V P, Q
(4.1)
*
f : I R R (I is an open interval) be
2
12
2
max. 3 3 2 1 , 3 3 2 2 1 2 P, Q
f
is
twice
differentiable
on
(4.2)
, , 0 1 with .
II.
There exist real constants m, M such that
ii.
If 1 , then
m M and m t 2s f " t M t , and s R
2
1
*
and
0 V P, Q
3 2 2 1 2 P, Q (4.3)
3
p
If P, Q n with 0 i i 1, 2,3..., n
2
1 3 2 2 1 2 P, Q
qi
0 V* P, Q V * P, Q
, then
3
m s P, Q C f P, Q M s P, Q
(4.4)
(3.1)
Proof:
Firstly, put s=2 in (1.7) and (3.4) respectively, we get
And
m s P, Q s P, Q C P, Q C f P, Q M s P, Q s P, Q
(3.2)
1 n p2
1 n p2
1 n p q 1
s P, Q i 1 i 2 pi qi i i 2 P, Q
2 i1 qi
2 i1 qi
2 i1 qi
2
2
(4.5)
Where
n
p
P2
C P, Q C f , P C f P, Q pi qi f i
i 1
Q
qi
s P, Q pi qi
i 1
n
n
pi
p2
p2
i pi i 1
qi i 1 qi
i 1 qi
(3.3)
s 1
n
p
P
1
s P, Q Cs , P Cs P, Q s 1 pi qi i , s 1
i 1
Q
qi
2
(3.4)
And C f P, Q , s P, Q are given by (1.1) and (1.7)
respectively
4.
BOUNDS
i 1
n
p q
pi 2
2 pi qi i i 2 P, Q
qi
qi
i 1
2
(4.6)
And by putting f t in (3.3), we get
n
P2
p q
V P, Q V , P V f* P, Q i i 2 3 pi qi
i 1 pi qi
Q
4
OF
NEW
INFORMATION
*
f
(4.7)
DIVERGENCE MEASURE
In this section, we derive bounds for V
P, Q in terms of
Let
g t f t
putting s=2 in t
2 s
2 t 1
f " t )
t3
3t
2t 1
(After
_______________________________________________________________________________________
Volume: 03 Issue: 05 | May-2014, Available @ http://www.ijret.org
667
6 t 4 1
g t
Then
t4
2 1
0 V P, Q V P, Q
3 2 2 1 K Q, P
24
, g t 5
t
(4.14)
If g t 0 t 1 0 t 1, 1
4
(t)
has
minimum
g 1 24 0 so
value
at
t=1,
m inf g t g 1 0
since
(4.8)
t 0,
Proof:
Firstly, put s=1 in (1.7) and (3.4) respectively, we get
n
p
lim s P, Q pi log i K P, Q
s 1
i 1
qi
(4.15)
n
p
q
lims P, Q pi log i qi log i K P, Q K Q, P
s 1
i 1
qi
pi
(4.16)
Now, we have two cases:
i.
If 0 1 , then
M sup g t max g , g
putting s=1 in t
t ,
2
2 12
2 1
2
2
max
3
1
,
3
If
1 , then
M sup g t
2 1
f " t )
t2
3t
2t 1
(After
12 t 1
4 t 1 3 2
3
t
1
,
g
t
t3
t4
4
If
2
t[1, )
2 s
2 t 1
Then
g t
(4.9)
ii.
g t t f t
Let
2 1
(4.10)
g t 0 t 1 3t 3 t 2 t 1 0 t 1, 0.63
The results (4.1), (4.2), (4.3) and (4.4) are obtained by using
(2.3), (4.5), (4.6), (4.7), (4.8), (4.9) and (4.10) in 3.1 and 3.2.
Also
(t)
has
minimum
g 1 24 0 so
value
m inf g t g 1 0
at
t=1,
since
(4.17)
t 0,
0 1 , then
2
2
we have two
cases:
2 1 Now,
2 1
*
2
2
0 V P, Q max.
3
1
,
3
1
K
,Q
0
i.
If
1 ,Pthen
2
2
M sup g t max g , g
i.
If
(4.11)
0 V P, Q V
*
t ,
P, Q
2
2 12 2
2 1
2
max.
3
1
,
3
K Q, P
2
2
(4.12)
1 , then
2
2 1
*
0 V P, Q
3 2 2 1 K P, Q
ii.
2
2 12 2
2 1
max
3 2 1 ,
3 2 2 1
2
2
(4.18)
ii.
If
1 , then
If
(4.13)
M sup g t
t [1, )
2 1
2 1
(4.19)
_______________________________________________________________________________________
Volume: 03 Issue: 05 | May-2014, Available @ http://www.ijret.org
668
Let h P, Q ,R a P, Q ,B P, Q and V * P, Q be
defined as in (1.4), (1.5), (1.6) and (2.3) respectively. Then,
we have
i.
If 0 1 , then
8 1
8 1
0 V * P, Q max 3/2 3 2 2 1 , 3/2 3 2 2 1 h P, Q
(4.20)
g t t f t
Let
Then g t
(4.21)
3/2
2 1 h P, Q
2t 1
t 1 , g t
3
2t
7
2
15t
(After
4t 3 5
(t)
has
minimum
value
at
m inf g t g 1 0
t=1,
since
(4.26)
t 0,
2
2
2 1
2 1
2
2
max
3 2 1 , 3 3 2 1
3
2
(4.27)
8 1
1
3 2 2 1 R3/2 Q, P B P, Q h P, Q
3/2
If
ii.
Proof:
Firstly, put s=1/2 in (1.7) and (3.4) respectively, we get
n
i 1
i 1
s P, Q 41 pi qi 2 2 2 pi qi
n
i 1
i 1
2 pi qi 2 pi qi 4
pi qi
2
1 , then
M sup g t
t [1, )
(4.23)
2 1
3
2
2 1
(4.28)
4h P , Q
(4.24)
i 1
If g t 0 t 1 5t t t 1 0 t 1, 0.53
0 V* P, Q V * P, Q
s P, Q 2 qi pi
3t
t ,
(4.22)
5t t
3
2
M sup g t max g , g
1 , then
8 1
5
2
2 t 1
f " t )
g 1 24 0 so
2
8 12 2
1
8 1 2
0 V * P, Q
2 s
3 t 1
t
Also
0 V* P, Q V * P, Q
If
3
2
putting s=1/2 in t
ii.
n
q 3/2
qi
2 i 1/2 pi qi 2 R3/2 Q, P B P, Q
pi i1 pi
Let K P, Q ,
P, Q and V * P, Q be
defined as
If
0 1 , then
2
2 12 2
2 1 2
0 V P, Q max
3 2 1 ,
3 2 1 K Q, P
(4.25)
(4.29)
_______________________________________________________________________________________
Volume: 03 Issue: 05 | May-2014, Available @ http://www.ijret.org
669
0 V* P, Q V * P, Q
2
2 12 2
2 1 2
max
3 2 1 ,
3 2 1 2 Q, P K Q, P
(4.30)
1 , then
2
2 1
*
0 V P, Q
3 2 2 1 K Q, P
ii.
If
(4.31)
0 V* P, Q V * P, Q
2 1
3 2 2 1 2 Q, P K Q, P
M sup g t max g , g
t ,
2
2 12
2 1
2
max
3 2 1 ,
3 2 2 1
(4.36)
ii.
If
1 , then
M sup g t
2 1
t [1, )
2 1
(4.37)
The results (4.29), (4.30), (4.31) and (4.32) are obtained by
using (2.3), (4.7), (4.33), (4.34), (4.35), (4.36), and (4.37) in
3.1 and 3.2.
(4.32)
Proof:
Firstly, put s=0 in (1.7) and (3.4) respectively, we get
q
lim s P, Q qi log i
s 0
i 1
pi
n
K Q, P
Let
2 P, Q ,R a P, Q and V * P, Q be
defined as
(4.33)
i.
n
n
n
n
p q
q2
q2
q2
s P, Q i qi i 1 i 2qi pi i i 2 Q, P
pi
i 1 pi
i 1 pi
i 1 pi
i 1
If
0 1 , then
0 V * P, Q max 1 3 2 2 1 , 1 3 2 2 1 2 Q, P
2
(4.34)
Let g t t f t
2
s=0 in t
2 s
Then
g t
f " t )
2 t 1
3t
2t 1 (After putting
0 V* P, Q V * P, Q
(4.38)
max 1 3 2 2 1 , 1 3 2 2 1 R2 Q, P R3 Q, P 2 Q, P
2 t 1 3 2
1
9t t t 1 , g t 3 36t 4 16t 3 4 ii.
2
t
t
(4.39)
If
1 , then
0 V * P, Q 1 3 2 2 1 2 Q, P
2
If
g t 0 t 1 9t 3 t 2 t 1 0 t 1, 0.43
(4.40)
0 V* P, Q V * P, Q 1 3 2 2 1R3 Q, P R2 Q, P 2 Q, P
2
(4.41)
Also
(t)
has
g 1 24 0 so
minimum
m inf g t g 1 0
t 0,
value
at
t=1,
since
Proof:
Firstly, put s=-1 in (1.7) and (3.4) respectively, we get
(4.35)
1 n q2
1 n q2
1 n p q 1
s P, Q i 1 i 2qi pi i i 2 Q, P
2 i 1 pi
2 i 1 pi
2 i 1 pi
2
2
(4.42)
_______________________________________________________________________________________
Volume: 03 Issue: 05 | May-2014, Available @ http://www.ijret.org
670
s P, Q
qi 2 1 n qi3 qi 2 1
1 n
q
i i p 2 2 i1 p 2 p 2 R3 Q, P R2 Q, P
2 i1
i
i
i
(4.43)
Let g t t 3 f t 2 t 1
s=-1 in t
3t
2t 1 (After putting
f " t )
2 s
Then g t 24 t
t 1 , g t 72 t 2 48t
If g t 0 t 0, 1
It is clear that g (t) is monotonic decreasing on (0, 1) and
monotonic increasing on [1, ).
Also
(t)
has
g 1 24 0 so
minimum
value
at
m inf g t g 1 0
t=1,
since
(4.44)
t 0,
If
0 1 , then
M sup g t max g , g
t ,
max 2 1 3 2 2 1 , 2 1 3 2 2 1
2
(4.45)
ii.
If
1 , then
M sup g t 2 1 3 2 2 1
2
t[1, )
(4.46)
The results (4.38), (4.39), (4.40) and (4.41) are obtained by
using (2.3), (4.7), (4.42), (4.43), (4.44), (4.45), and (4.46) in
3.1 and 3.2.
REFERENCES
[1]. Bhattacharyya A., On some analogues to the amount of
information and their uses in statistical estimation,
Sankhya, 8, 1-14.
[2]. Csiszar I., Information type measures of differences of
probability distribution and indirect observations, Studia
Math. Hungarica, 2(1967), 299-318.
[3]. Hellinger E., Neue begrundung der theorie der
quadratischen
formen
von
unendlichen
vielen
veranderlichen, J. Rein.Aug. Math., 136(1909), 210-271.
[4]. Kullback S. and Leibler R.A., On Information and
Sufficiency, Ann. Math. Statist., 22(1951), 79-86.
_______________________________________________________________________________________
Volume: 03 Issue: 05 | May-2014, Available @ http://www.ijret.org
671