Markov Analysis

Overview
A probabilistic decision analysis
Does not provide a recommended decision
Provides probabilistic information about a
decision situation that can aid the DM
Applicable to systems that exhibit probabilistic
movement from one state to another, over time
Probability that a machine will be running one day and
broken down the next
Probability that a customer will change her
department store to the next, called brand switching

Brand Switching Example

Customers are usually royal to a particular brand or store, or

supplier
Two gas stations in a community , P and N
Study indicates customers are not royal to either one
Willing to change based on advertisement factors
If a customer bought gas from P in any given month, there was 0.6
probability that the customer would buy from P and 0.4 probability
from N the next month
If a customer traded with N in any given month, there was 0.8
probability that the customer would buy from N and 0.2 probability
from N the next month
Next Month
This month
P
N
P
0.6
0.4
N
0.8
0.2

Terminology
Gas station that a customer trades at a
given month is called state of the system
(two states of system)
Probabilities of various states are called
transition probabilities
Transition probability sum to one
Probabilities apply to all participants
Probabilities are constant over time
States are independent over time

What Information MA Provides?

Answers the probability of being in a state at some future time period
Determining the probability that a customer would trade with them in
month 3 given that the customer trades with them this month
Use the following decision tree 1
The probability of a customers purchasing gas from P in month 3 given
that the customer traded with P in month1 =0.36+0.08=0.44
The probability of a customers purchasing gas from N in month 3 given
that the customer traded with N in month1 =0.24+0.32=0.56

Use the following decision tree 1

Given that N is the starting state in month1, the probability of a
customers purchasing gas from N in month3: 0.08+0.64=0.72
Given that N is the starting state in month1, the probability of a
customers purchasing gas from P in month3: 0.12+0.16=0.28

Month 3-Result
Month 3
This month
P
N
P
0.44
0.56
N
0.28
0.72
Easy for month 3, but not for month 10 or 15
Follow the notes in class