Markov Chains

Mathematics Applications & Interpretation HL

  Understand and use Regular Markov Chains to model real-life
situations
 Represent a system involving probabilities
 Determine the powers of transition matrices
 Calculate steady-state and long-term probabilities using transition
Objectives matrices and systems of equations
 Determine the characteristic polynomial, eigenvalues and
eigenvectors of 2x2 matrices
 Diagonalize 2x2 matrices with distinct real eigenvalues
 Use eigenvalues & eigenvectors to applications to powers of 2x2
matrices
  In our experiences of modelling in this course,
we have made use of available data to generate
a function that will closely approximate the
behavior of an observed system.
Stochastic  However, it may be worth considering the role
Models of randomness into our model.
stochastic model attempts to include
 A _________________
random variation in a system by employing
probability to predict the outcomes of future
events.
 A stochastic model which describes how a
sequence of random events evolves over
time
steps
Time here is measured in distinct ______.
The system ought to have a set of
Markov Chain states
distinct ______at each time step, each
with an identifiable transition between
them.
_________________________:
Memoryless/Markov Property The
outcome of a future event depends only
on the state at the current time step.
 state

Transition
Diagrams

transition probability
 State Matrices 𝑺𝒏
 Shows the state of the system at time 𝑛
 May contain either
 the probability or proportion of the population that are in each
state, or
 The actual number from the population that are in the given state

Matrix Transition Matrix 𝑻 = 𝒕𝒊𝒋

Representation  Columns represent current state
 Rows represent next state
 𝑡𝑖𝑗 represents the probability of moving from state 𝑗 to state 𝑖
 𝑡𝑖𝑗 = 𝑃 next state is 𝑖|current state is 𝑗
 The sum of each column is 1.
The transition matrix is extremely useful when there are more than two
states in the system & you wish to predict the states of the system after
several time periods.
Trivia: Matrices play an important role in the projection of 3D images into a 2D
screen, creating the realistic seeming motions in computer-based
applications.
Ex 13D.1
Haese AIHL
 When there is very little or no
Steady State change in the values from one state
Systems
matrix to the next, the system is said
steady state
to have reached a _____________.
  A matrix 𝑇 for which there exists an 𝑛𝜖ℤ+ such
Regular that all entries in 𝑇 𝑛 are all greater than zero.
Transition  For high powers of 𝑛, a regular transition
Matrix matrix converges to a matrix in which all the
columns have the same values.
• As 𝑛 → ∞, 𝑇𝑛 → 𝑋 and 𝑆𝑛 → 𝑣.
• 𝑋 is called the __________________________.
long-term probability matrix Note that 𝑇𝑋 = 𝑋.
steady state vector i.e. 𝑇𝑣 = 𝑣 as 𝑛 → ∞ and
• 𝑣 is called the _________________,
𝑋𝑆0 = 𝑣.
 If 𝑃 is a square regular transition
Steady State matrix, there is a unique vector 𝑞
Vector
such that 𝑃𝑞 = 𝑞. 𝑞 is called the
steady state vector for 𝑃.
_______________
Ex 13D.2
Haese AIHL