Probability Theory: Daniel B. Rowe, PH.D

Marquette University
MSCS6010
Probability Theory
Daniel B. Rowe, Ph.D.
Professor
Department of Mathematics,
Statistics, and Computer Science
Copyright 2014 by D.B. Rowe

1
MSCS6010
Outline
Set Theory
Probability Theory
Random Variables
Distribution Functions
Density and Mass Functions
D.B. Rowe
2
MSCS6010
Set Theory
Experiment: A chance process that leads to welldefined results called outcomes.
Outcome: The result of a single trial of a probability

experiment, O.
Sample Space: The set S of all possible outcomes of
a particular experiment.
S={O1,O2,,On}
Event: An event A is any collection of possible
outcomes of an experiment, that is, any subset of S
including S itself.
D.B. Rowe
3
MSCS6010
Sets Theory
Example:
Experiment
Single Outcome, O
Event, E
Sample Space, S
Flipping a coin Head (H)
A={H}
S={H,T}
Rolling a Die
A={3,6}
S={1,2,3,4,5,6}
Six (6)
We say that the event E has occurred if the outcome

of the experiment is in E.
D.B. Rowe
4
MSCS6010
Set Theory
Union: The union of A or B, written AUB, is the set of
elements that belong to A or B or both:
A B {x : x A or x B}
S
D.B. Rowe
5
MSCS6010
Set Theory
Intersection: The intersection of A and B, written
AB, is the set of elements that belong to both A and
B:
A B {x : x A and x B}
S
D.B. Rowe
6
MSCS6010
Set Theory
Compliment: The compliment of A, written Ac, is the
set of all elements that are not in A:
Ac {x : x A}
S
D.B. Rowe
7
MSCS6010
Set Theory
Laws: For events A, B, and C from S
Commutative:
A BB A
A
Associative:
Distributive:
DeMorgans:
BB
A (B C) ( A
B) C
A (B C) ( A
B) C
A (B C) ( A
B)
( A C)
A (B C) ( A
B)
( A C)
(A
B)c Ac
Bc
(A
B)c Ac
Bc
D.B. Rowe
8
MSCS6010
Set Theory
Two events A and B are disjoint (mutually exclusive)
if A B .
The events A1, A2,
are pairwise disjoint (or mutually exclusive) if
Ai Aj for all i j .
If A1, A2, are pairwise disjoint and i1 Ai S,

then the collection A1, A2, forms a partition of S.
D.B. Rowe
9
MSCS6010
Probability Theory: Axiomatic Foundations

Given a sample space S with A an event in S,
a probability function P:
1. P ( A) 0
2. P ( S ) 1.
3. If A1 , A2 ,... are pairwise mutually exclusive
(disjoint), then P(
i 1
Ai ) P( Ai ).
i 1
D.B. Rowe
10
MSCS6010

Let S={s1,,sn} be a finite set.
Let p1,,pn be nonnegative numbers that sum to 1.
For any A define P(A) by:
P( A)
{i:si A}
pi .
(The sum over an empty set is defined to be 0.)

Then P is a probability function.
This remains true if S={s1,,sn} is a countable set.
D.B. Rowe
11
MSCS6010

If P is a probability function and A is a set, then
a. P() 0, where is the empty set;
b. P( A) 1;
c. P(Ac ) 1 P( A).
If P is a probability function and A and B are sets,

then
a. P( B Ac ) P( B) P( A B);
b. P( A B) P( A) P( B) P( A B);
c. If A B, then P( A) P( B).
D.B. Rowe
12
MSCS6010

If P is a probability function, then
a. P( A) i 1 P( A Ci ), for any partition C1, C2 ,...;
b. P(
i 1
Ai ) i 1 P( Ai ), for sets A1, A2 ,... .
(Booles Inequality)
D.B. Rowe
13
MSCS6010
Combinations and Permutations

Outcomes: six numbered balls in bucket
D.B. Rowe
14
MSCS6010

Outcomes: six numbered balls in bucket, select one
If I mix up and draw a ball at random,

how many possible outcomes?
D.B. Rowe
15
MSCS6010

Outcomes: six numbered balls in bucket, select one

1
2
1
3
4
5
6
D.B. Rowe
16
MSCS6010

Outcomes: six numbered balls in bucket, select one.

(1)
1
2
1
(2)
3
(3)
(4)
4
5
6
(5)
(6)
6 different outcomes
D.B. Rowe
17
MSCS6010

Outcomes: six numbered balls in bucket, select two with replacement
order important.
If we replace the ball, and draw a second ball,
how many ways can we get ordered pairs?
1
2
1
3
4
5
6
D.B. Rowe
18
MSCS6010

order important.
1
2
3
4
5
6
1
2
3
4
5
6
2
1
3
4
5
6
1
2
3
4
5
6
1
2
3
4
51
62
3
1 4
2 5
3 6
4
5
6
D.B. Rowe
19
MSCS6010

order important.
1
2
3
4
5
6
(1,1)
1
2
3
4
5
6
2
1
3
4
5
6
1
2
3
4
5
6
(1,2)
(1,3)
(1,4)
(1,5)
(1,6)
(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)

1
2
3
4
51
62
3
1 4
2 5
3 6
4
5
6
(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)

(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
D.B. Rowe
20
MSCS6010

Outcomes: six numbered balls in bucket, select two without replacement
order important.
If we do not replace the ball, and draw a second ball,
how many ways can we get ordered pairs?
1
2
1
3
4
5
6
D.B. Rowe
21
MSCS6010

order important.
2
3
4
5
6
(1,2)
(2,1)
3
4
5
6
(1,5)
(1,6)
1
2
3
4
5
(2,3) (2,4) (2,5) (2,6)
1
2
(3,1) (3,2)
4
51
62
(4,1) (4,2) (4,3)
3
1
2 5
3 6
4
4
5
(1,4)
2
1
(1,3)
(3,4) (3,5) (3,6)

(4,5) (4,6)
(5,1) (5,2) (5,3) (5,4)
(5,6)
(6,1) (6,2) (6,3) (6,4) (6,5)
D.B. Rowe
22
MSCS6010

order unimportant.
If we replace the ball, and draw a second ball,
how many ways can we get unordered pairs?
1
2
1
3
4
5
6
D.B. Rowe
23
MSCS6010

order unimportant.
1
2
3
4
5
6
(1,1) (1,2)
1
2
3
4
5
6
2
1
3
4
5
6
1
2
3
4
5
6
(1,3)
(1,4)
(1,5)
(1,6)
(2,2) (2,3) (2,4) (2,5) (2,6)

1
2
3
4
51
62
(3,3) (3,4) (3,5) (3,6)

(4,4) (4,5) (4,6)
(5,5) (5,6)
3
1 4
2 5
3 6
4
5
6
(6,6)
D.B. Rowe
24
MSCS6010

order unimportant.
If we do not replace the ball, draw a second ball,
how many ways can we get unordered pairs?
1
2
1
3
4
5
6
D.B. Rowe
25
MSCS6010

order unimportant.
2
3
4
5
6
(1,2) (1,3)
1
2
(3,4) (3,5) (3,6)
4
51
62
(4,5) (4,6)
(5,6)
3
1
2 5
3 6
4
(1,6)
(2,3) (2,4) (2,5) (2,6)
3
4
5
6
(1,5)
2
1
(1,4)
1
2
3
4
5
D.B. Rowe
26
MSCS6010

Permutation formula:
N!
N Pn
( N n)!
Combination formula:
N!
N Cn
n!( N n)!
D.B. Rowe
27
MSCS6010
Select two with replacement order important.
Nn
Select two without replacement order important.
N!
( N n)!
Select two with replacement order unimportant.
( N n 1)!
n!( N 1)!
Select two without replacement order unimportant.
N!
n!( N n)!
D.B. Rowe
28
MSCS6010

We discussed the number of different outcomes for sampling with and without
replacement when order is either important or unimportant.
Lets now introduce the concepts of probabilities.

Drawing balls with or without replacement when order is
important or unimportant.
1
D.B. Rowe
29
MSCS6010

order important.
Oi
1
2
3
4
5
6
(1,1)
1
2
3
4
5
6
2
1
3
4
5
6
1
2
3
4
5
6
(1,2)
(1,3)
(1,4)
(1,5)
(1,6)
(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)

1
2
3
4
51
62
3
1 4
2 5
3 6
4
5
6
(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)

(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
36 different outcomes equally likely.
P(Oi ) 1/ 36
D.B. Rowe
30
MSCS6010

order important.
Oi
2
3
4
5
6
(1,2)
(2,1)
3
4
5
6
(1,5)
(1,6)
1
2
3
4
5
(2,3) (2,4) (2,5) (2,6)
1
2
(3,1) (3,2)
4
51
62
(4,1) (4,2) (4,3)
3
1
2 5
3 6
4
4
5
(1,4)
2
1
(1,3)
(3,4) (3,5) (3,6)

(4,5) (4,6)
(5,1) (5,2) (5,3) (5,4)
(5,6)
(6,1) (6,2) (6,3) (6,4) (6,5)

30 different outcomes equally likely.
P(Oi ) 1/ 30
D.B. Rowe
31
MSCS6010

order unimportant.
Oi
1
2
3
4
5
6
(1,1) (1,2)
1
2
3
4
5
6
2
1
3
4
5
6
1
2
3
4
5
6
(1,3)
(1,4)
(1,5)
(1,6)
(2,2) (2,3) (2,4) (2,5) (2,6)

1
2
3
4
51
62
3
1 4
2 5
3 6
4
5
6
(3,3) (3,4) (3,5) (3,6)

(4,4) (4,5) (4,6)
not equally likely
(5,5) (5,6)
P(Oi ) 1 / 36
diagonals
P(Oi ) 2 / 36
off diagonals
(6,6)
D.B. Rowe
32
MSCS6010

order unimportant.
Oi
2
3
4
5
6
(1,2) (1,3)
1
2
(3,4) (3,5) (3,6)
4
51
62
(4,5) (4,6)
3
1
2 5
3 6
4
(1,6)
(2,3) (2,4) (2,5) (2,6)
3
4
5
6
(1,5)
2
1
(1,4)
1
2
3
4
5
(5,6)
P(Oi ) 2 / 30
D.B. Rowe
33
MSCS6010

Drawing balls with replacement
same as rolling a die.
D.B. Rowe
34
MSCS6010

Classical Approach to Probability: All events are equally likely.
Oi
P (Oi )
6
6
6
6
6
6
D.B. Rowe
35
MSCS6010

Classical Approach to Probability: Roll die twice, order important!
Oi
(1,1)
(1,2)
(1,3)
(1,4)
(1,5)
(1,6)
(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)

(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
P(Oi ) 1/ 36
D.B. Rowe
36
MSCS6010

Classical Approach to Probability: Roll die twice, X=sum of die.
Oi (1,1) (1,2) (1,3) (1,4) (1,5) (1,6)

(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)
(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
2 3 4 5 6 7 8 9 10 11 12
P( X )
D.B. Rowe
37
MSCS6010

Classical Approach to Probability: Roll die twice, X=sum of die.
Oi (1,1) (1,2) (1,3) (1,4) (1,5) (1,6)

(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)
(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
10
11
12
P( X ) 1 36 2 36 3 36 4 36 5 36 6 36 5 36 4 36 3 36 2 36 1 36
D.B. Rowe
38
MSCS6010

Classical Approach to Probability: Flip coin once.
Oi
P(Oi )
1/ 2
1/ 2
D.B. Rowe
39
MSCS6010

Classical Approach to Probability: Flip coin twice.
Oi
P (Oi )
HH
HT
TH
TT
D.B. Rowe
40
MSCS6010

Oi
P (Oi )
HH
1/ 4
HT
1/ 4
TH
1/ 4
TT
1/ 4
D.B. Rowe
41
MSCS6010

Oi
P (Oi )
HH
1/ 4
HT
1/ 4
TH
1/ 4
TT
1/ 4
Let X= # of Heads
P( X )
1/ 4
2/4
1/ 4
D.B. Rowe
42
MSCS6010

Classical Approach to Probability: Flip coin three times.
Oi
P (Oi )
HHH
1/ 8
HHT
1/ 8
P( X )
HTH
1/ 8
1/ 8
HTT
1/ 8
3/8
THH
1/ 8
3/8
THT
1/ 8
1/ 8
TTH
1/ 8
TTT
1/ 8
Let X= # of Heads
D.B. Rowe
43
MSCS6010

Classical Approach to Probability: Flip coin ten times.
Oi
P(Oi )
HHHHHHHHH
TTTTTTTTTT
Let X= # of Heads
9 10
P( X )
D.B. Rowe
44
MSCS6010
Conditional Probability & Independence

If A and B are events in S, and P(B)>0, then the
conditional probability of A given B written
P(A|B ) is,
P( A B)
P( A | B)
P( B)
S
A
D.B. Rowe
45
MSCS6010

Let A1, A2, be a partition of the sample space, and
let B be any set. Then for each i=1,2,,
P( Ai | B)
P( B | Ai ) P( Ai )
i 1
(Bayes Rule)
P( B | Ai ) P( Ai )
B
A1
Ai
S
D.B. Rowe
46
MSCS6010

Two events A and B, are statistically independent if
.
P( A B) P( A) P( B)
If A and B are independent events, then the following

pairs are also independent
a. A and B c ,
b. Ac and B,
c. Ac and B c .
D.B. Rowe
47
MSCS6010
Random Variables
A random variable is a function from a sample space
S into the real numbers.
Examples:
Experiment
Toss two die
Toss a coin 25 times
Apply different amounts
of fertilizer to plants
Random Variable
X=sum of numbers
X= # heads in 25 tosses
X=yield/acre
D.B. Rowe
48
MSCS6010
Random Variables
Example: Toss a coin twice
X=number of heads
Outcomes: S={HH,HT,TH, TT}
s
X(s)
HH
HT
TH
TT
x
0
P(X=x) 1/4
1
1/2
2
1/4
i.e.
P(X=1)=P(HT)+P(TH)
=2/4
D.B. Rowe
49
MSCS6010
Cumulative Distribution Functions

The cumulative distribution function or cdf of a
random variable X denoted by FX(x), is defined by
FX ( x) PX ( X x), for all x.
Example: Tossing a coin twice

x
P(X=x)
1/4
1/2
1/4
0 / 4 x 0
1 / 4 0 x 1
FX ( x)
3 / 4 1 x 2
4 / 4 2 x
1.00
0.75
0.50
0.25
0.00
0
D.B. Rowe
50
MSCS6010

The function FX(x) is a cdf if and only if the following
three conditions hold:
a. lim F ( x) 0 and lim F ( x) 1
x
b. F ( x) is a non decreasing function of x.

c. F ( x) is right continuous; that is,
for every number x0 , lim F ( x) F ( x0 )
x x0
A random variable X is continuous if FX(x) is a

continuous function of x. A random variable X is
discrete if FX(x) is a step function.
D.B. Rowe
51
MSCS6010

The random variables X and Y are identically
distributed if, for every set A,
P( X A) P(Y A).
The following two statements are equivalent:

a. The random variables X and Y are identically distributed.
b. FX ( x)=FY ( x) for every x.
D.B. Rowe
52
MSCS6010
Probability Distribution Functions

The probability mass function (pmf) of a discrete
random variable X is given by
f ( x) P( X x) for all x.
The probability density function (pdf), fX(x), of a

continuous random variable X is the function that
satisfies
x
FX ( x)
f X (t ) dt.
D.B. Rowe
53
MSCS6010
Probability Distribution Functions

A function fX(x) is a pdf (or pmf) of a random variable
X if and only if
a. f X ( x) 0 for all x.
b.
f X ( x) 1 (pmf)
or
f X ( x)dx (pdf) 1.
D.B. Rowe
54
MSCS6010
Summary
Set Theory
Probability Theory
Random Variables
Distribution Functions
Density and Mass Functions
D.B. Rowe
55
MSCS6010
Homework 2:
1) Write a Matlab simulation program to flip a coin once
where p(H)=1/2. Repeat n=1000 times.
(i.e. generate Bernoulli random variables)
2) Make a plot of flip number on horizontal axis and
running empirical p(H) on vertical axis.
i.e. p(H)=(# times H appears)/(# of coin flips)
D.B. Rowe
56
MSCS6010
Homework 2:
3) Write a Matlab simulation program to flip a coin twice
where p(H)=1/2. Repeat n=1000 times.
4) Let X= number of heads in two flips. Make a
histogram of
p(X=0)=(# times 0 H appears)/n
D.B. Rowe
57

Probability Theory: Daniel B. Rowe, PH.D

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Probability Theory: Daniel B. Rowe, PH.D

Hochgeladen von

Copyright:

Verfügbare Formate

Marquette University

Copyright 2014 by D.B. Rowe

Outcome: The result of a single trial of a probability

Flipping a coin Head (H)

We say that the event E has occurred if the outcome

If A1, A2, are pairwise disjoint and i1 Ai S,

Probability Theory: Axiomatic Foundations

Probability Theory: Axiomatic Foundations

(The sum over an empty set is defined to be 0.)

Probability Theory: Axiomatic Foundations

If P is a probability function and A and B are sets,

Probability Theory: Axiomatic Foundations

Ai ) i 1 P( Ai ), for sets A1, A2 ,... .

Combinations and Permutations

Combinations and Permutations

If I mix up and draw a ball at random,

Combinations and Permutations

If I mix up and draw a ball at random,

Combinations and Permutations

If I mix up and draw a ball at random,

Combinations and Permutations

Combinations and Permutations

Combinations and Permutations

(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)

(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)

Combinations and Permutations

Combinations and Permutations

(2,3) (2,4) (2,5) (2,6)

(4,1) (4,2) (4,3)

(3,4) (3,5) (3,6)

(5,1) (5,2) (5,3) (5,4)

(6,1) (6,2) (6,3) (6,4) (6,5)

Combinations and Permutations

Combinations and Permutations

(2,2) (2,3) (2,4) (2,5) (2,6)

(3,3) (3,4) (3,5) (3,6)

Combinations and Permutations

Combinations and Permutations

(3,4) (3,5) (3,6)

(2,3) (2,4) (2,5) (2,6)

Combinations and Permutations

Combinations and Permutations

Select two with replacement order important.

Select two without replacement order important.

Select two with replacement order unimportant.

Select two without replacement order unimportant.

Combinations and Permutations

Lets now introduce the concepts of probabilities.

Combinations and Permutations

(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)

(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)

Combinations and Permutations

(2,3) (2,4) (2,5) (2,6)

(4,1) (4,2) (4,3)

(3,4) (3,5) (3,6)

(5,1) (5,2) (5,3) (5,4)

(6,1) (6,2) (6,3) (6,4) (6,5)

Combinations and Permutations

(2,2) (2,3) (2,4) (2,5) (2,6)

(3,3) (3,4) (3,5) (3,6)

Combinations and Permutations

(3,4) (3,5) (3,6)

(2,3) (2,4) (2,5) (2,6)