Probability Notation

Mathematical language as a means for concise communication.

1
2
Probability may be dull but it has its moments!
What does probability mean?
3
Frequentists (clicker A)
Probability measures the frequency of outcomes.
Bayesians (clicker B)
Assigns a probability to a hypothesis based on the state of knowledge.
Examples
A coin toss lands heads.
Canada will win the next FIFA world cup.
Edward Thorp will beat the house at blackjack 9 times out of 10.
4
Probability
Probability is the branch of mathematics that studies the possible outcomes
of given events together with the outcomes' relative likelihoods and
distributions.
Statistics
The mathematical study of the likelihood and probability of events occurring
based on known information and inferred by taking a limited number of
samples.
Probability uses known parameters to predict likely outcomes - omniscient
Statistics deduces parameters and describes them based on a degree of
confidence.
5
What is the difference?
Contrasting Probability and Statistics (P)
Choose the scenario that describes a probability calculation:
A) A fair coin is tossed five times. What is the likelihood of it
landing (Heads, Heads, Heads, Tails, Tails)?
B) Out of a political riding of 153,000 eligible voters, 1023
people responded to a pole identifying their political party
of choice. If 453 respondents chose Green, what is the
estimated average number of Green votes in an upcoming
election?
6
Experiment: Any action or occurrence where the outcome is not certain.
Sample Space (): Set of all possible outcomes () of an experiment. (Need
not be numbers)
Event Space (): Any collection (subset) of outcomes () contained in a
sample space.
Random Variable ():
1. Any rule that associates a number () with each outcome from an
experiment. (Why is this an important step?)
2. An outcome of an experiment which is in the sample space.
7
Probability Terms I
[2] MacKay, Information Theory, Inferences & Learning Algorithms, 2004, Pg22
[1] J.L. Devore, Probability & Statistics for Engineering and the Sciences, 2000, Pg97
Probability Terms II
Discrete Probability Distribution/Mass Function (P): The probability
associated with every outcome of the discrete random variable .
Ensemble: Three values (,

) based on a random variable

where is the number associated to an outcome,

is the set of all

possible outcomes and

is probability associated with each outcome.

8
Interpretation of a coin toss
Experiment :
A coin is thrown once into the air
and the side it lands on is observed.
Sample Space Outcomes ():
Head, Tail
Random Variable ():
Head 2, Tail 1
Probability Distribution Function
((Head)=2)=0.5,((Tail)=1)=0.5
Event
(T,H,H,T,T) (1,2,2,1,1)
9
Probability Terms III
Joint Ensemble : is an ensemble in which each outcome is an
ordered pair , with

=
1
, ,

, and

=
1
, ,

.
(, )is the joint probability of and .
Marginal Probability: The marginal probability is obtained from
the joint probability (, ) by summation:
=

( =

, )
Conditional Probability:
=

, =

, if ( =

) 0
MacKay, Information Theory, Inferences & Learning Algorithms, 2004, Pg22-23
10
Alphabet Distribution
11
MacKay, Information Theory, Inferences & Learning Algorithms, 2004, Pg22
Experiment :
Select a lower case letter from an English text.
Define the Ensemble:
: Outcome of the experiment mapped to
an integer based on the Alphabet order.

: Set of lower case letters from the

English language in alphabetical order.
: Probability of an outcome of an
experiment.

Alphabet Distribution: Joint Ensemble

12
MacKay, Information Theory, Inferences & Learning Algorithms, 2004, Pg23
Experiment :
Select 2 lower case letters from an English text.
Define the Ensemble:
, : Outcome of the experiment mapped
to an integer pair based on the
Alphabet order.

: Set of lower case letters from the

English language in alphabetical order.
, : Probability of an outcome of an
experiment.
Alphabet Distribution: Conditional Probabilities
13
MacKay, Information Theory, Inferences & Learning Algorithms, 2004, Pg24
Experiment :
Given the outcome of one letter
selection, letter what is the
chance that:
1) The preceding letter is ?
2) The following letter is ?
Probability Terms IV
Product (Chain) Rule:
, , = , ,
, , = , ()
Sum Rule:
=

(, )
=

()
Independence:
, = ()
14
Learning Outcomes
1. Students can differentiate using examples two interpretations of
probability which are the Frequentist and Bayesian views.
2. Students can define and give examples of fundamental concepts in
probability: Sample Space, Event, Random Variable, Probability
Distribution Function, Bayes Theorem, Independence, Product Rule
and Sum Rule
15