3 views

Uploaded by Luis Alberto Fuentes

- Modal Analysis
- M16_REND6289_10_IM_C16
- Differential Equations - Solved Assignments - Semester Spring 2004
- Gupta 2010 Soil Dynamics and Earthquake Engineering
- ME-2008-Exp
- Bw Doodling Markov
- Grey Markov Theory
- Markov Chains
- Richtungsstatistik
- Class_Slides_Part_3.pdf
- Test 2 II 0809 (SET A)
- Maxima Book Chapter 7
- 08410571
- SOLUTION CHARACTERISTICS OF QUADRATIC POWER FLOW PROBLEMS
- chap8_p1_s
- Time Weight Web Rank
- -Narrowband Direction of Arrival Estimation for Antenna Arrays Synthesis Lectures on Antennas
- Math 450 Lect 04
- sol4
- Crm Modeling Markov

You are on page 1of 6

is a Markov Chain on a finite state spacce F consisting of points x F . The Markov Chain

will be assumed to have a stationary transition probability given by a stochastic matrix

= (x, y), as the probability for transition from x to y. We will assume that all the

entries of are positive, imposing thereby a strong irreducibility condition on the Markov

Chain. Under these conditions there is a unique invariant or stationary distribution p(x)

satisfying

X

p(x)(x, y) = p(y).

x

Let us supposePthat V (x) : F R is a function defined on the state space with a mean

value of m = x V (x)p(x) with respect to the invariant distribution. By the ergodic

theorem, for any starting point x,

1X

lim Px | V (Xj ) m| a = 0

n n

j

the Markov Chain initialized to start from the point x F . We expect the rates to be

exponential and the goal as it was in Cramers theorem, is to calculate

1 1 X

lim log Px V (Xj ) a

n n n

j

1

lim log Ex exp[V (X1 ) + V (X2 ) + V (Xn )] = log (V )

n n

V = V (x, y) = (x, y)eV (x)

with the largest modulus, which is positive. Such an eigenvalue exists by Frobenius theory

and is characterized by the fact that the eigenvector has positive entries. To see this we

only need to write down the formula

X

Ex exp[V (X1 ) + V (X2 ) + V (Xn )] = [V ]n (x, y)

y

that is easily proved by induction. The geometric rate of growth of any of the entries of

the n-th power of a positive matrix is of course given by the Frobenius eigenvalue . Once

we know that

1

lim log Ex exp[V (X1 ) + V (X2 ) + V (Xn )] = log (V )

n n

1

2

1

lim sup log Px V (X1 ) + V (X2 ) + V (Xn ) a log (V ) a

n n

1

lim sup log Px V (X1 ) + V (X2 ) + V (Xn ) a log (V ) a

n n

Since we can optimize over 0 we can obtain

1

lim sup log Px V (X1 ) + V (X2 ) + V (Xn ) a h(a) = sup[a log (V )]

n n 0

By Jensens inequality

1 Px

(V ) = lim log E exp [V (X1 ) + V (X2 ) + V (Xn )]

n n

1 Px

lim E [V (X1 ) + V (X2 ) + V (Xn )]

n n

= m

Therefore

h(a) = sup[a log (V )] = sup[a log (V )]

0 R

For the lower bound we can again perform the trick of Cramer to change the measure from

Px to a Qx such that under Qx the event in question has probability nearly 1. The Radon

Nikodym derivative of Px with respect to Qx will then control the large deviation lower

bound.

P Our plan then is to replace by

such that (x, y) has an invariant measure q(x)

with x V (x)q(x) = a. If Qx is the distribution of the chain with transition probability

then Z

Px [E] = Rn dQx

E

where

X (Xj , Xj+1 )

Rn = exp log

(Xj , Xj+1 )

j

This provides us a lower bound for the probbaility

1

lim inf log Px V (X1 ) + V (X2 ) + V (Xn ) a J(

)

n n

where

1 Q

) = lim

J( E [ log Rn ]

n n x

X (x, y)

= log (x, y)q(x)

x,y

(x, y)

3

The last step comes from applying the ergodic theorem for the

chain with q() as the

invariant distribution to the average

n

1 1X (Xj , Xj+1 )

log Rn = log

n n (Xj , Xj+1 )

j=1

P

Since any with x V (x)q(x) = a will do where q() is the corresponding invariant mea-

sure, the lower bound we have is quickly improved to

1

lim log Px V (X1 ) + V (X2 ) + V (Xn ) a inf )

J(

n n

V (x)q(x)=a

P

If we find the such that a log (V ) = h(a) then for such a choice of

(V )

a=

(V )

and we take (x, y)eV (y) as our nonnegative matrix. We can find for the eigenvalue a

column eigenvector f and a row eigenvector g. We define

1 f (y)

(x, y) =

(x, y)eV (y)

f (x)

One can check that is a stochastic matrix with invariant measure q(x) = g(x)f (x)

(properly normalized). An elemenatary perturbation argument involving eigenvalues gives

the relationship

X

a= q(x)V (x)

x

and

X

J= [ log + V (y) + log f (y) log f (x)]

(x, y)q(x)

x,y

= a log

= h(a)

thereby matching the upper bound. We have therefore proved the following theorem.

Theorem 1.1. For any Markov Chain P with a transition probability matrix with positive

entries, the probability distribution of n1 nj=1 V (Xj ) satisfies an LDP with a rate function

h(a) = sup[a log (V )]

u(x)

There is an interesting way of looking at (V ). If V (x) = log (u)(x) then f (x) = (u)(x)

is a column eigenfunction for V with eigenvalue = 1. Therefore

u

log (log )=0

u

4

and because log (V + c) = log (V ) + c for any constant c, log (V ) is the amount c by

which V has to be shifted so that V c is in the class

u

M = {V : V = log }

u

We now turn our attention to the number of visits

n

X

Lnx = x (Xj )

j=1

to the state x or

1 n

nx =

L

n x

the proportion of time spent in the state x. If we view {x ; x F } as a point in the

space P of probability measures on F , then we get a distribution xn for any starting point

x and time n. The ergodic theorem asserts that xn converges weakly to the degenerate

distribution at p(x), the invariant measure for the chain. In fact we have actually proved

the large deviation principle for xn .

Theorem 1.2. The distributions xn of the empirical distributions satisfy an LDP with a

rate function

X

I(q) = sup q(x)V (x)

V M x

X

E Px [exp[ V (Xi )](u)(Xn+1 )] = (u)(x)

Therefore

n

1 X

lim sup log Ex exp[ V (Xj )] 0

n n

j=1

1 X

lim sup lim sup log xn [U ] q(x)V (x)

U q n n x

where U are neighborhoods of q that shrink to U . Since the space P is compact this provides

immediately the upper bound in the LDP with the rate function I(q). The lower bound is

a matter of finding a

such that q(x) is the invariant measure for and J( ) = I(q). One

can do the variational problem of minimizing J( ) over those

that have q as an invariant

measure. The minimum is attained at a point where

f (y)

(x, y) = (x, y)

(f )(x)

5

. Since q(x) is the the invariant distribution for

an

easy calculation gives

X

J= [log f (y) log(u)(x)]

(x, y)q(x)

x,y

X

= [log f (x) log(u)(x)]q(x)

x

X

= V (x)q(x) for some V M

x

I(q)

Since we already have the upper bound with I() we are done.

Remark 1.3. It is important to note that in the special case of IID random variables

with a finite set of possible values we can take (x, y) = (y) and for any V (x) the matrix

(y)eV (y) is of rank one with the nonzero eigenvalue being equal to (V ) = x eV (x) (x)

P

and the analysis reduces to the one of Cramer. In particular

X q(x)

I(q) = q(x) log

x

(x)

is the relative entropy with respect to ().

Remark 1.4. A similar analysis can be done for the continuous time Markov chains as

well. Suppose we have a Markov chain, again on the finite state space F , with transition

probabilities (t, x, y) given by (t) = exp[tA]

P where the infinitesiaml generator or the rate

matrix A satisfies a(x, y) 0 for x 6= y and yF a(x, y) = 0. For any function V : F R

the operator (A + V ) defined by

X

(A + V )f (x) = a(x, y)f (y) + V (x)f (x)

y

has the property that the eigenvalue with the largest real part is real and if we make the

strong irreducibility asssumption as before i.e. a(x, y) > 0 for x 6= y, then this eigenvalue

is unique and is characterized by having row ( or equivalently) column eigenfunctions

that are positive. We denote this eigenvalue by (V ). If u > 0 is a positive function

on F , then Au + ( Auu )u = Au Au = 0. Therefore for V =

Au

u , the vector u is a

column eigenvector for the eigenvalue 0 and for such V , (V ) = 0. If we now denote by

[M = V : V = Au u for some u > 0} then (V ) is the unique constant such that V M.

We can now have the exact analogues of the descrete case and with the rate function

X

I(q) = sup q(x)V (x)

V M x

1 t

Z

x (t) = x (X(s))ds

t 0

6

Remark 1.5. In the special case when A is symmetric one can explicitly calculate

X

I(q) = a(x, y) q(x) q(y).

x,y

The large deviation theory gives the asymptotic relation

t X

1

Z

lim log E exp V (X(s))ds = sup q(x)V (x) I(q)

t t 0 q()P x

X X

= sup V (x)[f (x)]2 + a(x, y)f (x)f (y)

f :f 0

kf k2 =1 x x,y

X X

= sup V (x)[f (x)]2 + a(x, y)f (x)f (y)

f :kf k2 =1 x x,y

Notice that the Dirichlet form x,y a(x, y)f (x)f (y) can be rewritten as 21 x,y a(x, y)[f (x)

P P

f (y)]2 and replacing f by |f | lowers the Dirichlet form and so we have dropped the as-

sumption that f 0. This is the familiar variational formula for the largest eigenvalue of

a symmetric matrix.

- Modal AnalysisUploaded byZohaib Hasan Khan
- M16_REND6289_10_IM_C16Uploaded byJosé Manuel Orduño Villa
- Differential Equations - Solved Assignments - Semester Spring 2004Uploaded byMuhammad Umair
- Gupta 2010 Soil Dynamics and Earthquake EngineeringUploaded byquentindelvigne
- ME-2008-ExpUploaded bynkpatil
- Bw Doodling MarkovUploaded byzafor
- Grey Markov TheoryUploaded byAjeng Alfia Fadilla Teguhputri
- Markov ChainsUploaded byHaydenChadwick
- RichtungsstatistikUploaded byBauschmann
- Class_Slides_Part_3.pdfUploaded bydileep bapatla
- Test 2 II 0809 (SET A)Uploaded byuthm student
- Maxima Book Chapter 7Uploaded byprakush_prakush
- 08410571Uploaded bySalman
- SOLUTION CHARACTERISTICS OF QUADRATIC POWER FLOW PROBLEMSUploaded byjaach78
- chap8_p1_sUploaded bychitta
- Time Weight Web RankUploaded byredsox1903
- -Narrowband Direction of Arrival Estimation for Antenna Arrays Synthesis Lectures on AntennasUploaded bymaglon34
- Math 450 Lect 04Uploaded byطالب علم
- sol4Uploaded byStian Bilek
- Crm Modeling MarkovUploaded bytomekziomek
- simulation in nuclear waste managementUploaded byalber
- 200310918443436246Uploaded byPascual Onofre
- Homework 10Uploaded byAle Gomez
- Statistical Markovian Data Modeling for Natural Language ProcessingUploaded byLewis Torres
- lec6_6245_2004.pdfUploaded bykartheek2404
- eigenvectorsUploaded byHemant Kumar
- 4exUploaded byTrung Phan
- Student Lecture 31 Quadric SurfacesUploaded byuploadingperson
- QA-II General GuidelinesUploaded byparshwa shah
- konoval_volodymyr_power_90_2017 (1)Uploaded bywan ismail ibrahim

- 2Uploaded byLuis Alberto Fuentes
- 2Uploaded byLuis Alberto Fuentes
- 2Uploaded byLuis Alberto Fuentes
- 2Uploaded byLuis Alberto Fuentes
- RandUploaded byLuis Alberto Fuentes
- 4Uploaded byLuis Alberto Fuentes
- 2Uploaded byLuis Alberto Fuentes
- 2Uploaded byLuis Alberto Fuentes
- MtUploaded byLuis Alberto Fuentes
- 2Uploaded byLuis Alberto Fuentes
- 2Uploaded byLuis Alberto Fuentes
- zm2221.pdfUploaded byLuis Alberto Fuentes
- MtUploaded byLuis Alberto Fuentes
- MtUploaded byLuis Alberto Fuentes
- MtUploaded byLuis Alberto Fuentes
- MtUploaded byLuis Alberto Fuentes
- MtUploaded byLuis Alberto Fuentes
- MtUploaded byLuis Alberto Fuentes
- MatUploaded byLuis Alberto Fuentes
- 2-1Uploaded byLuis Alberto Fuentes
- 2Uploaded byLuis Alberto Fuentes
- 4tUploaded byLuis Alberto Fuentes
- 6sUploaded byLuis Alberto Fuentes
- 5fUploaded byLuis Alberto Fuentes
- 3eUploaded byLuis Alberto Fuentes
- 2DUploaded byLuis Alberto Fuentes
- InterpolationUploaded byLuis Alberto Fuentes
- 1009.3956v4Uploaded byLuis Alberto Fuentes
- 1305.6289v1.pdfUploaded byLuis Alberto Fuentes
- 1205.5252Uploaded byMaiman Lato

- C++11.pdfUploaded byEd Cornejo Mayta
- Automatic Real Time Auditorium Power Supply Control.Uploaded byKishor Mhaske
- Oracle performance tuning part 4Uploaded bytavi.fircea
- Autocad 2013 PDF Dxf Reference EnuUploaded bygsanchez57
- HPE2-T30-40Uploaded byGbenga Mally
- PHPExcel User Documentation - Reading Spreadsheet FilesUploaded byAndres Camilo Serrato
- DSettlement User ManualUploaded byBluebelgian
- Project Report AircelUploaded byasifkingkhan
- TM210TRE.40-EnG_Working With Automation Studio_V4200Uploaded byVladan Milojević
- TE040 System Test ScriptUploaded bysheilazl2108
- 21 Timers SolUploaded byMahesh Bajaj
- CCNA 2 Final V4.1 AnswersUploaded byccnadiscovery4
- Packt.postgreSQL.9.6.High.performance.1784392979Uploaded byCaio Villela
- Triptico de SpeechUploaded byStefany Briones
- OOPSUploaded byMehul K Panchal
- ecvUploaded byFrancesco Lombardi
- Discuss the Methodology of Operations ResearchUploaded byankitoye
- Pre Assess Report 2692505Uploaded byAnuj Tiwari
- Online Chess GameUploaded byFarukh Shaikh
- Manage IP Based RouteUploaded bykimlookman
- Scrum Primer 20Uploaded bysudheeral
- Lec01Uploaded byAkim Culk
- Neuron Files Download PageUploaded byscripded
- iptablesUploaded bysamuel_10
- Basics of Document Management & InstallationUploaded bysid_chh26
- 6. Initiation Template - Initiation Phase Review FormUploaded byHồ Xuân Thắng
- Manual YAMAHA M7CLUploaded byAgyness Supertramp
- Project Tool Time StudyUploaded byArvin Varghese
- Multimedia DatabaseUploaded byanilnaik287
- GovQA - Slay the Public Records DragonUploaded byAzure Ninjas