LECTURE 1: Probability models and axioms

• Sample space

• Probability laws
Axioms
Properties that follow from the axioms

• Examples
Discrete
Continuous
• Discussion
Countable additivity

Mathematical subtleties

• Interpretations of probabilities
1
Sample space

• Two steps:

Describe possible outcomes

- Describe beliefs about likelihood of outcomes

2
 Sample space

-• I
• List (set) of possible outcomes, \l

• List must be:

Mutua Ily excl usive

Collectively exhaustive
At the "right" granu larity

• • • •
• • . T o.. ...J .... D
•
•
• • •

3
Sample space: discrete/ finite example

• Two rolls of a tetrahedral die sequential description

---::. 1. 1

:- 1.2

':::: 1.3
1.4

,~
Y = Second 3
roll
2 3,1
I

') I

I 2 3 4

x = Firsl roll 4.4

\ Y'ee...

4
Sample space: continuous example

• ( x,y ) such that 0 <x,y< 1

1 x

5
Probability axioms I ---:---"J

• Event: a subset of the sample space

/ / .//I
- Probability is assigned to events
o I

f(A)
• Axioms: •
Nonnegativity: peA) > 0

Normalization: perl) = 1

(Finite) additivity: (to be strengthened l ater)

If A n B = 0. then peA u B) = peA) + PCB) -..n..

, /fl./ e:
e... pJr sJ (

6
Some simple consequences of the axioms

Axioms Consequences

P(A) >0 P(A) < 1

P(r2) = 1 P(0) = 0

For disjoint events: P(A) + P(N) = 1

P(A U B) = P(A) + P(B)
P(A U B U C) = P(A) + P(B) + P(C)
and similarly for k disjoint events

P( {S1, S2,···, Sk}) = P( {S1}) + ... + p.( {Sk })

= P(S1) + ... + P(Sk)
7
 Some simple consequences of the axioms

Axioms

(0.1 peA) 2: 0

(\,1 P("') = 1

Fo r d isjoi nt eve nts:

«1 peA u B) = peA) + PCB)

c. .!

C
i :: f (Jl) }i (S2. )

i:o 1 +j(cpj ~l(r:j;)=O.

8
 Some simple consequences of the axioms

• + P(B) + P(C )
A , B , C disjoint: P(A U B U C) = P (A)

-------, I( AU!3uc.):::! ( (!luB) UC) ::- i [,4(//3) +1(c.)

=- f (/» t f(J3) +.f (c.)

l<.

~) 1 (l-i,u ••• UA):.) :: ~ E (!f<')

1 (15.~ u~SJl.S() •• ·u ~s):~)

:: f' (5.) ~ '" -+ £ (S ... )

- 9
 More consequences of the axioms

• If A C B , t he n P (A) $ P (B) B ':: A U (13 n A c)

.e (B): l? (4) t- -e. ('P/)t/~) ~ f(~)

0.0
• P (A U B) = P (A) + P (B) - 'P (A n B)1

10
More consequences of the axioms

• p eA u B u C ) = p eA) + P (A C
n B) + p eN n B C
n C) • K

l( Aue,uc ) =

11
Probability calculation: discrete/finite example

• Two rolls of a t etrahedral die • L et every possible outcome have probability 1/ 16

I I
• P (X = 1) = '-I • -­
16
!,:::;- 17....­
4 .... ~ :0­ L et Z = mi n(X, Y)

?:. ., .... ~ 2-:=2

Y = Second 3 "/ ~ )(~,..,I";o)

roll
~ ~
• P(Z=4)= '/'6
2
%- '0 ~ %;::
I
~ • P(Z = 2) = Ii. I ,
~ 1G •
I 2 3 4
x = First roll

12
Discrete uniform law

/f,." h • •
- Assume 0 consist s of n equ ally likely elements • ./-"- -.........

- Assume A co nsist s of k elements

• • A
•
• • ~.----/.
,
peA) = k •,-'11.­ 1
prob = ­
n

13
Probability calculation: continuous example

• (x,y) such that 0 <x,y< 1 • Uniform probability law: Probability = Area

y I I I
z" . -­
I
p ({(x,y) I x +y < 1( 2}) = 2· . -
2.
1 r--~::- 8

•
1 x

14
Probability calculation steps

• Specify the sam pie space

• Specify a probability law

• Iden tify an event of interest

• Ca lculate ... •

15
 Probability calculation: discrete but infinite sample space

p
• Sa m ple space: {1 ,2, ... }
112
- We are given Pen) =~,
n
n = 1, 2 , ... o••
•
<b QO
2 •••
114
••
~_I .. _~'"')~_, _I
••
•• o
c.. "'\_ L ..,,_-.
I
- ••
•••
11.
o•• 1116
",~.!! 2. .. ,0 2. 2. 1- ('/Ill • • Q •••
1 2 3 4

• P(outcomeiseven)= l ((9./1](,)"'5)
=.f Cf2~U~'1~ lJ{C.~u ' •• ) 0 1' (i) + 1 ("1)+1 (<;)1- •••

,
,
.. I I I ( I
... , -- I
• -­
I
~ .. ~ t' • ,
"

+ ... :: '1 I + '1- "+

'-j!L
• •
'-t I - -" 3
16
Countable additivity axiom

• Strengthens the finite additivity axiom

Countable Additivity AxiOm:~_ _~

If AI, A2, A3,'" is an infinite(S~quenc"3> of disjoint eve nts,

then P(AI U A2 U A3 U ... ) = P(AI) + P(A2) + P(A3) + ...
•

17
 Mathematical subtleties

Countable Additivity Axiom:

If AI, A2, A3,'" is an infinite sequence of d isj oi nt eve nts,

then P(AI U A2 U A 3 U ... ) = P ( A I) + P(A2) + P(A3 ) + ...

•
-, ?
1= i (52) =1 , U ?C'1:,YJ?. , GJ ~ f (T(?:

• Additivity holds on ly for "countable" sequences of events

• The unit square (simlarly, the real lin e, etc.) is not countable
( its elements cannot be arr anged in a sequence)

• "Area" is a leg itim ate probability law on the unit squa re,
as long as we do not try to assign probabilities/areas to "very strange" sets
18
Interpretations of probability theory

• A narrow view: a branch of math

- Axioms =? theorems "Thm:" "Frequency" of event A "is" peA)

• Are probabilities frequencies?

- P(coin toss yields heads) = 1/ 2

- P(the president of ... will be reelected) = 0.7

• Probabilities are often intepreted as:

Description of beliefs
Betting preferences

19
The role of probability theory

• A framework for analyzing phenomena with uncertain outcomes

- Rules for consistent reasoning
- Used for predictions and decisions

Predictions Proba bi lity theory

Real world
Decisions (Analysis)

Data Models

In feren ce I 5 tati sti cs

20
MIT OpenCourseWare
https://ocw.mit.edu

Resource: Introduction to Probability

John Tsitsiklis and Patrick Jaillet

The following may not correspond to a particular course on MIT OpenCourseWare, but has been provided by the author as an individual learning resource.

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.