Sie sind auf Seite 1von 25

# MarkovExcel Module

by Germán Riaño
Grupo COPA
griano@uniandes.edu.co

## Ver esta página en español ...

This workbook has a series of macros that can be used to analyze medium size Markov
Chain Problems. Its accuracy for large model has not been tested, and anyway Excel
cannot handle more than 256 columns
For really large models use:
jMarkov

## You have to enable macros in Excel in order to use this tool.

Go to tools/Macros/Security to enable them

DTMC

CTMC

Reference Manual
Reference

## MarkovExcel can be used free of charge for academic purposes.

No warranty is given or implied.
I am extremely grateful if you point to me any mistakes in the program. But I cannot give
personalized customer support by e-mail. In particular I canot teach you stochastics by
e-mail, nor will I help you with homework (except my own students, of course)
Discrete Time Markov Chains
Let's define P and call it "TheP", via Insert/Name/Define

## 0.2 0.7 0.1

P= 1 0 0
0.2 0.3 0.5

For steady state probabilities we mark the destination range and write
""=DTMCSteadyState(theP)" and then hit ctrl-shft-enter at the same time.
Attention: For all functions you have to hit Ctrl shift y enter at the same time.

## To obtain powers of P we use:

' =MatPower(Matrix,power)

## #VALUE! #VALUE! #VALUE!

P(4) = #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

## To find the occupancy matrix:

M(n) = I + P + P2 +...+Pn
usamos "=DTMCMatOcup(Matriz, n)"

## #VALUE! #VALUE! #VALUE!

M(1)= #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

Reducible Chains

1 2 3 4 5 6
1 0.1 0.2 0.1 0.2 0.2 0.2
2 0.2 0.3 0.1 0.3 0.1 0
3 0.1 0.3 0.2 0.2 0.1 0.1
4 0 0 0 0.2 0.8 0
5 0 0 0 0.8 0.2 0
6 0 0 0 0 0 1

## If we try DTMCSteadyState we obviously get an error:

#VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

## The former chain must be decomposed as:

T = {1,2,3} Transient states
C1 ={4,5} First closed communicating class
C2 = {6} Second communicating class

Absortion Probabilities

Find R y Q, that correspond to transitions among transient and from transient the the closed comunicating class
1 2 3
1 0.1 0.2 0.1
A= 2 0.2 0.3 0.1 R=
3 0.1 0.3 0.2

## Absortion Probabilities are computed through

DTMCAbsor(A,R)
C1 C2
1 #VALUE! #VALUE!
F= 2 #VALUE! #VALUE!
3 #VALUE! #VALUE!

## Expected time before absortion is computed through

DTMCExpected(A)
1 #VALUE!
m= 2 #VALUE!
3 #VALUE!

Limit probability exists, since there are no periodic states, and is given by

1 2 3 4 5 6
1 0 0 0 #VALUE! #VALUE! #VALUE!
2 0 0 0 #VALUE! #VALUE! #VALUE!
3 0 0 0 #VALUE! #VALUE! #VALUE!
4 0 0 0 #VALUE! #VALUE! 0
5 0 0 0 #VALUE! #VALUE! 0
6 0 0 0 0 0 1

## Lets compute a big power to check the result:

1 2 3 4 5 6
1 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
2 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
3 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
4 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
5 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
6 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

It works!
e closed comunicating classes
C1 C2
1 0.4 0.2
2 0.4 0
3 0.3 0.1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
Continuous Time Markov Chains
Define generator matrix Q y and call it "Generateor", via Insert/Name/Define

-9 4 5
Q= 10 -17 7
7 7 -14

mark destination range and write "=CTMCSteadyState(Generator)"
and hit ctrl-shift-enter at the same time.
Attention: For all functions you have to hit Ctrl shift y enter at the same time.

## To find P(t)=exp(Q t) we use:

=CTMCTransient(Matrix, time)

## #VALUE! #VALUE! #VALUE!

P(0.01) = #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

## To find the occupancy matrix

t
M (t )=∫0 P(s)ds
we use CTMCMatOccup(Generator, n)

## #VALUE! #VALUE! #VALUE!

M(0.1)= #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

Reducible Chains
Assume Q is given as
1 2 3 4 5 6
1 -9 2 1 2 2 2
2 12 -17 1 3 1 0
3 1 3 -8 2 1 1
4 0 0 0 -8 8 0
5 0 0 0 8 -8 0
6 0 0 0 0 0 0

## If we try CTMCSteadyState we get, obviously, an error:

#VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
The previous chain must be decomposed as

T = {1,2,3}
C1 ={4,5}
C2 = {6}

Absortion Probabilities:

## Lets compute R and Q

1 2 3
1 -9 2 1
A= 2 12 -17 1 R=
3 1 3 -8

Absortion Probabilities:
DTMCAbsor(A,R)
C1 C2
1 #VALUE! #VALUE!
F= 2 #VALUE! #VALUE!
3 #VALUE! #VALUE!

DTMCExpected(A)
1 #VALUE!
m= 2 #VALUE!
3 #VALUE!

## The limit matrix exists and is given by:

1 2 3 4 5 6
1 0 0 0 #VALUE! #VALUE! #VALUE!
2 0 0 0 #VALUE! #VALUE! #VALUE!
3 0 0 0 #VALUE! #VALUE! #VALUE!
4 0 0 0 #VALUE! #VALUE! 0
5 0 0 0 #VALUE! #VALUE! 0
6 0 0 0 0 0 1

## As a test, we compute P(t) for a large t

1 2 3 4 5 6
1 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
2 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
3 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
4 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
5 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
6 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

and it works!
C1 C2
1 4 2
2 4 0
3 3 1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
Reference Manual Go back to Main Menu
The following functions are defined in the Macros.
To use this functions in another worksheet save this workboos as an add-in (menu: File/Save
as, choose the last option)
If you plan to use the functions frequenlt you can install the add-in permanently (menu:
General Matrix Functions
MatPower(A, n As Integer)
Computes the n-th power of the given matrix

## Function MatSum(A, B, Optional minus = False)

Adds the given matrices. If minus is TRUE then it computes A-B

Computes alpha*A

## Function MatIden(n As Integer)

Returns an nxn identity matrix

## Function MaxRate(Q) As Double

Assuming Q is a generator matrix this computes the biggest rate, i.e., the largest entry in the
diagonal:

λ=max (−qii )
Function UniformProb(Q, Optional EnsureGenerator As Boolean = True,
Optional lda = -1#)
Assuming Q is a generator matrix, computes the associated uniformized DTMC transition
matrix. If lda is not given it is internally computed. If EnsureGenerator is true then Q is
converted to a generator rather than a rates matrix
Q
P= +I
λ
DTMC Functions
MatExpo(A, t)
Computes exp(A x t)

DTMCMatOccup(A, n As Integer)
Computes the occupancy matrix for a Discrete Markov Chain

Computes the steady state probability for DTMC with matix P. If P is not irreducible (or
unichain) this returns an error.

Function DTMCAbsor(A, R)
This computes the absorving probabilities for a markov chain with transient to transient
probabilities Q, and transient to absorving probabilities R, through the formula

( I − A )−1 R
I−
( (I−Q )−1 RA )−1 R
Function DTMCExpected(A)
Computes the expected number of steps before absorption, starting from each transient state,
through the formula
−1−1
( II−Q
− A) ) R1
1
CTMC Functions
Computes the steady state probabilities for the given generator matrix

## Function CTMCTransient(Q, t, Optional Cum = False)

Assuming that Q is a generator matrix this computes exp(Q t). If cum is true then it rathers
computes the occupancy matrix. The algorithm used is uniformization.

Function CTMCMatOccup(Q, t)
Computed teh occupancy matrix
t
M (t )=∫0 P(s)ds
Function CTMCAbsor(A, R)
Computes teh absortion probabilities where A is the generator rates matrix from transient to
transient states, and R from transient to the absorving classes

Function CTMCExpected(A)
Computes the expected value of the time to absortion starting from each of the given states.
MarkovExcel Module
por Germán Riaño
Grupo COPA
griano@uniandes.edu.co

Esta herramienta de Excel tiene una serie de macros para el análisis de Cadenas de
Markov de tamaño intermedio. Su precisión para modelos muy grandes no has sido
probada y, en todo caso, Excel no maneja más de 256 columnas.
Para nodelos grandes use:
jMarkov

CMTD

## Cadenas de markov de Tiempo Continuo

CMTC

Manual de Referencia.
Referencia

## MarkovExcel se puede usar sin costo para labores académicas.

No se da garantía alguna ni explícita ni implícita
Le agardezco que me avise si encuentra errores. Pero desafortunadamente no puedo
dar "soporte al cliente" personalizado. En particular no puedo enseñarle procesos
estocásticos por e-mail ni ayudarle en sus tareas (excepto a mis estudiantes claro).
Cadenas de Markov de Tiempo Discreto
Definimos P y le damos el nombre "laPe", via Insertar/Nombre/Definir

## 0.2 0.7 0.1

P= 1 0 0
0.2 0.3 0.5

y luego ctrl-shft-enter al tiempo.
OJO: Para todas las funciones damos Ctrl shift y enter al tiempo.

## Para potencias de P usamos

MatPower(Matriz,potencia)

## #VALUE! #VALUE! #VALUE!

P(4) = #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

## Para Hallar la matriz de ocupación:

M(n) = I + P + P^2 +...+P^n
usamos DTMCMatOcup(Matriz, n)

## #VALUE! #VALUE! #VALUE!

M(1)= #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

1 2 3 4 5 6
1 0.1 0.2 0.1 0.2 0.2 0.2
2 0.2 0.3 0.1 0.3 0.1 0
3 0.1 0.3 0.2 0.2 0.1 0.1
4 0 0 0 0.2 0.8 0
5 0 0 0 0.8 0.2 0
6 0 0 0 0 0 1

## Si intentamos DTMCSteadyState obviamente generamos un error:

#VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

## La anterior cadena se DEBE descomponer como

T = {1,2,3}
C1 ={4,5}
C2 = {6}

Calculamos A y R
1 2 3
1 0.1 0.2 0.1
A= 2 0.2 0.3 0.1 R=
3 0.1 0.3 0.2

## Las probabilidades de absroción son

DTMCAbsor(A,R)
C1 C2
1 #VALUE! #VALUE!
F= 2 #VALUE! #VALUE!
3 #VALUE! #VALUE!

DTMCExpected(A)
1 #VALUE!
m= 2 #VALUE!
3 #VALUE!

## La matriz límite existe, pues no hay estados periódicos y es:

1 2 3 4 5 6
1 0 0 0 #VALUE! #VALUE! #VALUE!
2 0 0 0 #VALUE! #VALUE! #VALUE!
3 0 0 0 #VALUE! #VALUE! #VALUE!
4 0 0 0 #VALUE! #VALUE! 0
5 0 0 0 #VALUE! #VALUE! 0
6 0 0 0 0 0 1

## A modo de revisión calculamos una potencia grande:

1 2 3 4 5 6
1 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
2 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
3 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
4 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
5 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
6 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

y si lo es!
C1 C2
1 0.4 0.2
2 0.4 0
3 0.3 0.1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
Cadenas de Markov de Tiempo Continuo
Definimos la matriz generadora Q y le damos el nombre "Generateor", via Inserter/Nombre/Definir

-9 4 5
Q= 10 -17 7
7 7 -14

y luego ctrl-shft-enter al tiempo.
OJO: Para todas las funciones damos Ctrl shift y enter al tiempo.

## Para hallar P(t)=exp(Q t) usamos

=CTMCTransient(Matriz, tiempo)

## #VALUE! #VALUE! #VALUE!

P(0.01) = #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

## Para Hallar la matriz de ocupación:

t
M (t )=∫0 P(s)ds
usamos CTMCMatOccup(Generator, tiempo)

## #VALUE! #VALUE! #VALUE!

M(0.1)= #VALUE! #VALUE! #VALUE!
#VALUE! #VALUE! #VALUE!

Suponga que Q viene dada por
1 2 3 4 5 6
1 -9 2 1 2 2 2
2 12 -17 1 3 1 0
3 1 3 -8 2 1 1
4 0 0 0 -8 8 0
5 0 0 0 8 -8 0
6 0 0 0 0 0 0

## Si intentamos SteadyState obviamente generamos un error:

#VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
La anterior cadena se DEBE descomponer como

T = {1,2,3}
C1 ={4,5}
C2 = {6}

Calculamos A y R
1 2 3
1 -9 2 1
A= 2 12 -17 1 R=
3 1 3 -8

## Las probabilidades de absroción son

=DTMCAbsor(A,R)
C1 C2
1 #VALUE! #VALUE!
F= 2 #VALUE! #VALUE!
3 #VALUE! #VALUE!

=DTMCExpected(A)
1 #VALUE!
m= 2 #VALUE!
3 #VALUE!

## La matriz límite existe y es:

1 2 3 4 5 6
1 0 0 0 #VALUE! #VALUE! #VALUE!
2 0 0 0 #VALUE! #VALUE! #VALUE!
3 0 0 0 #VALUE! #VALUE! #VALUE!
4 0 0 0 #VALUE! #VALUE! 0
5 0 0 0 #VALUE! #VALUE! 0
6 0 0 0 0 0 1

## A modo de revisión calculamos P(t) para un valor grande de t:

1 2 3 4 5 6
1 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
2 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
3 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
4 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
5 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!
6 #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! #VALUE!

y si lo es!
Nombre/Definir
C1 C2
1 4 2
2 4 0
3 3 1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
1

Check
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
#VALUE!
Manual de Referencia Regresar al Menu Principal
(Lo siento, no tengo traducción.. Quizás Ud. quiera ayudarme..)
The following functions are defined in the Macros.
To use this functions in another worksheet save this workboos as an add-in (menu: File/Save
as, choose the last option)
If you plan to use the functions frequenlt you can install the add-in permanently (menu:
General Matrix Functions
MatPower(A, n As Integer)
Computes the n-th power of the given matrix

## Function MatSum(A, B, Optional minus = False)

Adds the given matrices. If minus is TRUE then it computes A-B

Computes alpha*A

## Function MatIden(n As Integer)

Returns an nxn identity matrix

## Function MaxRate(Q) As Double

Assuming Q is a generator matrix this computes the biggest rate, i.e., the largest entry in the
diagonal:

λ=max (−qii )
Function UniformProb(Q, Optional EnsureGenerator As Boolean = True,
Optional lda = -1#)
Assuming Q is a generator matrix, computes the associated uniformized DTMC transition
matrix. If lda is not given it is internally computed. If EnsureGenerator is true then Q is
converted to a generator rather than a rates matrix
Q
P= +I
λ
DTMC Functions
MatExpo(A, t)
Computes exp(A x t)

DTMCMatOccup(A, n As Integer)
Computes the occupancy matrix for a Discrete Markov Chain

Computes the steady state probability for DTMC with matix P. If P is not irreducible (or
unichain) this returns an error.

Function DTMCAbsor(A, R)
This computes the absorving probabilities for a markov chain with transient to transient
probabilities Q, and transient to absorving probabilities R, through the formula

( I − A )−1 R
( I − A )−1 R
Function DTMCExpected(A)
Computes the expected number of steps before absorption, starting from each transient state,
through the formula
−1
( I− A ) 1
CTMC Functions
Computes the steady state probabilities for the given generator matrix

## Function CTMCTransient(Q, t, Optional Cum = False)

Assuming that Q is a generator matrix this computes exp(Q t). If cum is true then it rathers
computes the occupancy matrix. The algorithm used is uniformization.

Function CTMCMatOccup(Q, t)
Computed teh occupancy matrix
t
M (t )=∫0 P(s)ds
Function CTMCAbsor(A, R)
Computes teh absortion probabilities where A is the generator rates matrix from transient to
transient states, and R from transient to the absorving classes

Function CTMCExpected(A)
Computes the expected value of the time to absortion starting from each of the given states.
Principal