Birth and Death Processes

Bo Friis Nielsen
1
1
DTU Informatics
02407 Stochastic Processes 6, October 8 2013
Bo Friis Nielsen Birth and Death Processes
Birth and Death Processes
Today:
Limiting behaviour of birth and death processes
Birth and death processes with absorbing states
Finite state continuous time Markov chains
Next week
Renewal phenomena
Two(three) weeks from now
Phase type distributions
Bo Friis Nielsen Birth and Death Processes
Limiting Behaviour for Birth and Death Processes
For an irreducible birth and death process we have
lim
t
P
ij
(t ) =
j
0
If
j
> 0 then
P(t ) = or A = 0
We can always solve recursively for

n
=
n+1

n+1
such that

n
=
_
n1

i =0

i +1
_

0
such that

0
=
_
1 +

n=1
_
n1

i =0

i +1
__
1
Bo Friis Nielsen Birth and Death Processes
Linear Growth with Immigration

n
= n + a,
n
= n With

k
=
k1

i =0

i +1
=
a(a +) (a + (k 1)
k!
k
=
a

(
a

+ 1) (
a

+ (k 1)
k!
_

_
k
=
_
a

+ k 1
k
__

_
k

k=0

k
=

k=0
_
a

+ k 1
k
__

_
k
=
_
1

_a

k
=
_
a

+ k 1
k
__

_
k
_
1

_a

Bo Friis Nielsen Birth and Death Processes

Logistic Model
Birth/death rate per individual
= (M X(t ))
= (X(t ) N),
such that
n
= n(M n),
n
= n(n N).

N+m
=
_

_
m N+m1

i =N
i (M i )
(i + 1)(i + 1 N)
=
N
N + m
_
M N
m
__

_
m
, 0 m M N

N+M
=
c
N + m
_
M N
m
__

_
m
Bo Friis Nielsen Birth and Death Processes
Probability of Absorption
Theorem ( Theorem 6.1 Page 319)
Consider a birth and death process with birth and death
parameters
n
,
n
, n 0 with
0
= 0 such that 0 is an
absorbing state.
The probability of absorption into state 0 from the initial state m
is
u
m
=
_

i =m

i
1+

i =1

i
if

i =1

i
<
1 if

i =1
=
the mean time to absorption is
w
m
=
_
if

i =1
1

i
=
1

i
+

m1
k=1

k

j =k+1
1

j
if

i =1
1

i
<
where
0
= 1,
i
=

i
j =1

j
.
Bo Friis Nielsen Birth and Death Processes
Derivation of Absorption Probabilities
First step analysis, with the embedded Markov chain
P =

1 0 0 0 . . .

1
+
1
0

1

1
+
1
0 . . .
0

2

2
+
2
0

2

1
+
2
. . .
.
.
.
.
.
.
.
.
.
.
.
.

Let u
i
be the probability of absorption when starting in state i
u
i
=

i

i
+
i
u
i 1
+

i

i
+
i
u
i +1
, i 1
such that

0
= 1, u
i +1
u
i
=

i

i
(u
i
u
i 1
), i 1
Bo Friis Nielsen Birth and Death Processes
Derivation of Absorption Probabilities (cont.)
With v
i
= u
i +1
u
i
we get v
i
=

i

i
v
i 1
such that
v
i
=
i
v
0
with
i
=
i

j =1

j
leading to
u
i +1
u
i
= v
i
=
i
v
o
=
i
(u
1
1)
u
m
u
1
= (u
1
1)
m

j =1

i
, m > 1
If

m
j =1

i
< we get
u
1
=

j =1

i
1 +

j =1

i
otherwise u
1
= u
i
= 1
Bo Friis Nielsen Birth and Death Processes
Derivation of Mean Time to Absorption
Let w
i
be mean time to absorption starting in i (w
0
= 0)
w
i
=
1

i
+
i
+

i

i
+
i
w
i 1
+

i

i
+
i
w
i +1
, i 1
With z
i
= w
i
w
i +1
(negative!)
z
i
=
1

i
+

i

i
z
i 1
z
m
=
m

i =1
1

i
m

j =i +1

j
+
_
_
m

j =1

j
_
_
z
0
=
m

i =1
1

i
+
m
z
0
Bo Friis Nielsen Birth and Death Processes
Derivation of Mean Time to Absorption (cont.)
Inserting z
m
= w
m
w
m1
we get
w
m
w
m1
=
m

i =1

i
+
m
(w
1
)
1

m
(w
m
w
m1
) =
m

i =1
1

i
w
1
lim
m
1

m
(w
m
w
m1
) = 0
such that
w
1
=

i =1
1

i
Bo Friis Nielsen Birth and Death Processes
Population Processes: probability of absorption

n
= n,
n
= n, X(0) = m, E(X(t )) = me
()t

i
=
_

_
i
,

i =m

i
=
_
(

)
m
1

when >
when
Bo Friis Nielsen Birth and Death Processes
Population Processes: mean time to absorption
with m = 1 and <

i =1
1

i
=
1

i =1
1
i
_

_
i
=
1

i =1
_

0
x
i 1
dx
=
1

i =1
x
i 1
dx =
1

0
1
1 x
dx
=
1

log
_
1

_
=
1

log
_


_
Bo Friis Nielsen Birth and Death Processes
Finite Contiunuous Time Markov Chains
P
ij
(t ) = P{X(t + s) = j |X(s) = i }
(a) P
ij
(t ) 0
(b)

N
j =0
P
ij
(t ) = 1
(c) P
ik
(s + t ) =

N
j =0
P
ij
(s)P
jk
(t )
(d) lim
t 0+
P
ij
(t ) =
_
1, i = j
0, i = j
(c) is the Chapman-Kolmogorov equations, in matrix form
P(t + s) = P(t )P(s)
Bo Friis Nielsen Birth and Death Processes
P

(t ) = P(t )A( forward equations)

P

(t ) = AP(t )( backward equations)

P = e
At
=

n=0
A
n
t
n
n!
= I +

n=1
A
n
t
n
n!
Bo Friis Nielsen Birth and Death Processes
Stationary Distribution
0 = A = (
0
,
1
, . . . ,
N
)

q
0
q
01
. . . q
0N
q
10
q
1
. . . q
1N
.
.
.
.
.
. . . .
.
.
.
q
N0
q
N1
. . . q
NN

elementwise

j
q
j
=

i =j

i
q
ij
with q
j
=

k=j
q
jk
Bo Friis Nielsen Birth and Death Processes
Innitesimal Description
P{X(t + h) = j |X(t ) = i } = q
ij
h + o(h) for i = j
P{X(t + h) = i |X(t ) = i } = (1 q
i
h) + o(h)
Sojourn Description
1. Embedded Markov chain of state sequences
i
has one
step transition probabilities p
ij
=
q
ij
q
i
2. Successive sojourn times S

i
are exponentially distributed
with mean
1
q

i
.
Bo Friis Nielsen Birth and Death Processes
Two State Markov Chain
A =

A
2
=

2
+
2

2

2
+

= ( +)

Thus
A
n
= [( +)]
n1
A
And
P(t ) = I
1
+

n=1
[( +)t ]
n
n!
A
= I
1
+
_
e
(+)t
1
_
A
= I +
1
+
A
1
+
Ae
(+)t
Bo Friis Nielsen Birth and Death Processes
Summary of most Important Results
P(t ) = e
At
1 for t
Under an assumption of irreducibility
Bo Friis Nielsen Birth and Death Processes
Additional Reading
Kai Lai Chung: Markov Chains with Stationary Transition
Probabilities
Bo Friis Nielsen Birth and Death Processes