Martingale Theory

Problem set 3, with solutions

Martingales

The solutions of problems 1,2,3,4,5,6, and 11 are written down. The rest will come soon.

3.1 Let j , j = 1, 2, . . . be i.i.d. random variables with common distribution

P i = +1 = p, P i = 1 = q := 1 p,

and Fn = (j , 0 j n), n 0, their natural ltration. Denote Sn := j=1 j , n 0.

Pn

(a) Prove that Mn := (q/p)Sn is an (Fn )n0 -martingale.

(b) For > 0 determine C = C() so that

Zn := C n Sn

be an (Fn )n0 -martingale.

SOLUTION: (a)




E Mn+1  Fn = E Mn (q/p)n+1  Fn = Mn E (q/p)n+1  Fn
= Mn E (q/p)n+1 = Mn (p(q/p) + q(p/q)) = Mn .

(b)


1
C = C() = E = p + 1 q.

3.2 Gambler's Ruin, 1

A gambler wins or looses one pound in each round of betting, with equal chances and
independently of the past events. She starts betting with the rm determination that she
will stop gambling when either she won a pounds or she lost b pounds.
(a) What is the probability that she will be winning when she stops playing further.

1
(b) What is the expected number of her betting rounds before she will stop playing further.
SOLUTION: Model the experiment with simple symmetric random walk. Let j ,
j = 1, 2, . . . be i.i.d. random variables with common distribution

1

P i = +1 = = P i = 1 ,
2
and Fn = (j , 0 j n), n 0, their natural ltration. Denote
n
X
S0 = 0, Sn := j , n 1.
j=1

Dene the stopping times

TL := inf{n > 0 : Sn = b}, TR := inf{n > 0 : Sn = +a}, T := min{TL , TR }.
Note that
{the gambler wins a pounds} = {T = TR },
{the gambler looses b pounds} = {T = TL }.

(a) By the Optional Stopping Theorem

E ST = E S0 = 0.
Hence



bP T = TL + aP T = Tr = 0.
On the other hand,



P T = TL + P T = Tr = 1.
Solving the last two equations we get

a
b
P T = TL = , P T = TR = .
a+b a+b

(b) First prove that Mn := Sn2 n is yet another martingale:


2


E Mn+1  Fn = E Sn+1  Fn (n + 1)


= E Sn2 + 2Sn n+1 + 1  Fn (n + 1) = = Mn .
Now, apply the Optional Stopping Theorem
0 = E MT = E ST2 T = P T = TL b2 + P T = TR a2 E T .

Hence, using the result from (a)

E T = ab.

2
3.3 HW

Gambler's Ruin, 2

Answer the same questions as in problem 2 when the probability of winning or loosing one
pound in each round is p, respectively, q := 1 p, with p (0, 1).
Hint: Use the martingales constructed in problem 1

SOLUTION: Model the experiment with simple biased random walk. Let j , j =
1, 2, . . . be i.i.d. random variables with common distribution



P i = +1 = p, P i = 1 = q,

and Fn = (j , 0 j n), n 0, their natural ltration. Denote

n
X
S0 = 0, Sn := j , n 1.
j=1

Dene the stopping times

TL := inf{n > 0 : Sn = b}, TR := inf{n > 0 : Sn = +a}, T := min{TL , TR }.

Note that
{the gambler wins a pounds} = {T = TR },
{the gambler looses b pounds} = {T = TL }.

(a) Use the Optional Stopping Theorem for the martingale (q/p)Sn :
1 = E (q/p)Sn = (p/q)b P T = TL + (q/p)a P T = TR .

On the other hand,

P T = TL + P T = Tr = 1.

Solving the last two equations we get

1 (q/p)a
1 (p/q)b
P T = TL = , P T = TR = .
(p/q)b (q/p)a (q/p)a (p/q)b

(b) Now, apply the Optional Stopping Theorem to the martingale Sn (p q)n.
Hence
1 (p/q)b 1 (q/p)a
 
1 1



E T = (p q) E ST = (p q) a b
(q/p)a (p/q)b (p/q)b (q/p)a
a(1 (p/q)b ) + b(1 (q/p)a )
= (p q)1 .
(q/p)a (p/q)b

3
3.4 Let j , j = 1, 2, 3, . . . , be independent and identically distributed random variables and
Fn := (j , 0 j n), n 0, the natural ltration generated by them. Assume that
for some R the exponential moment m() := E ej < exists. Denote S0 := 0,

Sn := nj=1 j , n 1. Prove that the process

P

Mn := m()n exp{Sn }, n N,

is an (Fn )n0 -martingale.

SOLUTION: Very much the same as problem 1 (b).

3.5 Let (, F, (Fn )n0 , P) be a ltered probability space and Yn , n 0, a sequence of absolutely
integrable random variables adapted to the ltration (Fn )n0 . Assume that there exist real
numbers un , vn , n 0, such that


E Yn+1  Fn = un Yn + vn .

Find two real sequences an and bn , n 0, so that the sequence of random variables
Mn := an Yn + bn , n > 1, be martingale w.r.t. the same ltration.

SOLUTION: Write down the martingale condition for Mn :




E Mn+1  Fn = E an+1 Yn+1 + bn+1  Fn
= an+1 un Yn + an+1 vn + bn+1 = an Yn + bn .

We get the recursions

an+1 = an u1
n , bn+1 = bn an+1 vn .

The solution is
n1
!1
Y
a0 = 1, an = uk ,
k=0
Xn
b0 = 0, bn = ak vk1 .
k=1

3.6 HW

We place N balls in K urns (in whatever way) and perform the following discrete time
process. At each time unit we choose one of the balls uniformly at random (that is : each
ball is chosen with probability 1/N ) and place it in one of the urns also uniformly chosen
at random (that is: each urn is chosen with probability 1/K ). Denote by Xn the number

4
 of balls in the rst urn at time n and let Fn := (Xj , 1 j n), n 0, be the natural
ltration generated by the process n 7 Xn .
(a) Compute E Xn+1  Fn .


(b) Using the result from problem 5, nd real numbers an , bn , n 0, such that Zn :=
an Xn + bn be martingale with respect to the ltration (Fn )n0 .

SOLUTION: (a)
 
 Fn = (Xn + 1) N Xn 1 + (Xn 1) Xn K 1 + Xn N Xn K 1 Xn 1


E Xn+1 +
N K N K N K N K
N 1 1
= Xn + .
N K
(b) Apply the result from problem 5 with
N 1 1
un = , vn = .
N K

3.7 Let Xj , j 1, be absolutely integrable random variables and Fn := (Xj , , 1 j n),

n 0, their natural ltration. Dene the new random variables
n1
X 


Z0 := 0, Zn := Xj+1 E Xj+1  Fj .
j=0

Prove that the process n 7 Zn is an (Fn )n0 -martingale.

3.8 A biased coin shows HEAD=1 with probability (0, 1), and TAIL=0 with probability
1 . The value of the bias is not known.
For t [0, 1] and n N we dene pn,t : {0, 1}n [0, 1] by
Pn Pn
xj
pn,t (x1 , . . . , xn ) := t j=1 (1 t)n j=1 xj
.

We make two hypotheses about the possible value of : either = a, or = b, where

a, b [0, 1] and a 6= b. We toss the coin repeatedly and form the sequence of random
variables
pn,a (1 , . . . , n )
Zn := ,
pn,b (1 , . . . , n )
where j , j = 1, 2, . . . , are the results of the successive trials (HEAD=1, TAIL=0).
Prove that the process n 7 Zn is a martingale (with respect to the natural ltration
generated by the coin tosses) if and only if the true bias of the coin is = b.

SOLUTION:

5
 3.9 Bonus

Bellman's Optimality Principle

We model a sequence of gambling as follows. Let j , j = 1, 2, . . . , be independent random

variables with the following identical distribution;



P j = +1 = p, P j = 1 = 1 p := q, 1/2 < p < 1.

We denote
:= p log2 p + q log2 q + 2,

the entropy of the distribution of j .

j is the return of bet in the j th round. A gambler starts playing with initial fortune
unit

Y0 > 0 and her fortune after round n is

Yn = Yn1 + Cn n

where Cn is the amount she bets in this round. Cn may depend on the values of 1 , . . . , n1 ,
and 0 Cn Yn1 . The expected rate of winnings within n rounds is:


rn := E log2 (Yn /Y0 ) .

The gambler's goal is to maximize rn within a xed number of rounds.

(a) Prove that no matter what strategy the gambler chooses (that is: no matter how she
chooses Cn = Cn (1 , . . . , n1 ) [0, Yn1 ])

Xn := log2 Yn n

is a supermartingale and hence it follows that rn n. This means that she will not be
able to make her average winning rate, over any number of rounds, larger than .
(b) However, there exists a gambling strategy which makes Xn dened above a martingale
and thus realizes the maximal average winning rate. Find this strategy. That is: determine
the optimal choice of Cn = Cn (1 , . . . , n ).

SOLUTION:

3.10 Let n 7 n be a homogeneous Markov chain on the countable state space S := {0, 1, 2, . . . }
and Fn := (j , 0 j n), n 0, its natural ltration.
For i S denote by Q(i) the probability that the Markov chain starting from site i ever

reaches the point 0 S :



Q(i) := P n < : n = 0  0 = i .

6
 Prove that Zn := Q(n ) is an (Fn )n0 -martingale.

SOLUTION:

3.11 HW

Galton-Watson Branching Process

Let n,k , n = 1, 2, . . . , k = 1, 2, . . . be independent and identically distributed random

variables which take values from N = {0, 1, 2, . . . }. Assume that they have nite second
moment and denote := E n,k , 2 := Var n,k . Dene the Galton-Watson branching

process
Zn
X
Z0 := 1, Zn+1 := n+1,k
k=1

and let Gn := (Zj : 0 j n), n 0, be its natural ltration.

(a) Prove that
Mn := n Zn , n = 0, 1, 2, . . .

is a (Gn )n0 -martingale.

(b) Prove that
2  Gn = 2 Zn2 + 2 Zn .


E Zn+1

(c) Using the result from (b) prove that

2 n 1

Mn2 n+1 Mn if 6= 1,


Nn := 1

M 2 n 2 M if = 1

n n

is also a (Gn )n0 -martingale.

(d) Using the result from (c) prove that if > 1 then sup0n< E Mn2 < (that is: the

martingale Mn is uniformly bounded in L2 ) while if 1 then limn E Mn2 = .

SOLUTION: (a)
Zn
X Zn
X
 Fn = (n+1) E (n+1)
E n+1,k  Gn = (n+1) Zn = Mn .




E Mn+1 n+1,k Gn =

k=1 k=1

(b)
Zn X
X Zn
2


Zn+1 Gn = E n+1,k n+1,l  Gn = Zn (2 + 2 ) + (Zn2 Zn )2 = Zn2 2 + Zn 2 .
k=1 l=1

7
 (c) Consider the case 6= 1.

2 2 n+1 1 
2(n+1) 2

2 n+1 1 

E Mn+1 Mn+1
 Gn = E Z n+1
 Gn E M n+1
 Gn
n+2 1 n+2 1
2 n+1 1
= 2(n+1) Zn2 2 + Zn 2 n+2


Mn
1
2 n 1
= ... = Mn2 n+1 Mn .
1
When = 1 the computations are simpler.


E Nn = 1,
and hence
2 n 1
E Mn2 = 1 + n+1


.
1
The right hand side is bounded if > 1.
3.12 Bonus

Plya Urn, 1

At time n = 0, an urn contains B0 = 1 blue, and R0 = 1 red ball. At each time n =

1, 2, 3, . . ., a ball is chosen at random from the urn and returned to the urn, together with
a new ball of the same colour. We denote by Bn and Rn the number of blue, respectively,
red balls in the urn after the n-th turn of this procedure. (Note that Bn + Rn = n + 2.)
Denote by Fn := (Bj , 0 j n) = (Rj , 0 j n), n 0, the natural ltration of the
process. Let
Bn
Mn :=
Bn + Rn
be the proportion of blue balls in the urn just after time n.
(a) Show that n 7 Mn , is an (Fn )n0 -martingale.
(b) Show that P Bn = k = 1/(n + 2) for 0 k n + 1.

(Hint: Write down the probability of choosing k blue and n k red balls in whatever xed
order.)
(c) We will prove soon that M := lim Mn exist almost surely. What is the distribution of
M ?
(Hint: What is the limit of the distribution of Mn (identied in the previous point) as
n ?)

(d) (To be done after learning about the Optional Stopping Theorem.)
Let T be the number of balls drawn until the rst blue ball is chosen. Use the optional
stopping theorem to show that E T +2
1
= 1/4.

8
 SOLUTION:

3.13 Plya Urn, 2

Write a program code to simulate the Plya Urn. Start with B0 = 1 blue and R0 = 1 red
ball in the urn, perform 1000 steps and record the proportion of blue and red balls after
the 1000th step. Repeat this experiment 2000 times and determine the distribution of the
nal proportion of blue balls by performing elementary statistical analysis.

SOLUTION:

3.14 Plya Urn, 3

We continue the study of Plya Urn and use the notations of problem 12.
Let [0, 1] be xed and dene
(Bn + Rn 1)! Bn 1
Nn () := (1 )Rn 1 .
(Bn 1)!(Rn 1)!
Show that Nn () is is an (Fn )n0 -martingale.

SOLUTION:

3.15 Bonus

Bayes Urn

Assume we have a randomly biased coin which shows

P HEAD = , P T AIL = 1 ,

where U N I[0, 1] is a random variable which is uniformly distributed in [0, 1]. We toss
this coin many times and denote

B0 = 1, Bn := 1 + no. of HEADs in the rst n trials,

R0 = 1, Rn := 1 + no. of TAILs in the rst n trials,

and Fn := (Bj , 0 j n), n 0, the natural ltration generated by the sequence of coin
tosses. (Note the +1-s and that Bn + Rn = n + 2.)
(a) Prove that for any n N, the joint distribution of (B0 , B1 , . . . , Bn ) is the same as that
of the sequence denoted the same way in the Plya Urn, problem 12.
(b) Prove that
Bn
Mn := ,
Bn + Rn

9
(with Bn , Rn dened in this problem) is an an (Fn )n0 -martingale.

(c) Prove that

(Bn + Rn 1)! Bn 1
Nn () := (1 )Rn 1 .
(Bn 1)!(Rn 1)!
(with Bn , Rn dened in this problem) is exactly the (regular) conditional density function

of the random variable , given Fn .

10