Stats 241.

3
Probability Theory

Instructor:

W.H.Laverty

Office:

235 McLean Hall

Phone:

966-6096

Lectures:

T and Th 11:30am - 12:50am
GEOL 255
Lab: W 3:30 - 4:20 GEOL 155
Evaluation:

Assignments, Labs, Term tests - 40%
Final Examination - 60%
Text:
Devore and Berk, Modern Mathematical
Statistics with applications.
I will provide lecture notes (power point slides).
I will provide tables.
The assignments will not come from the textbook.
This means that the purchasing of the text is optional.
Course Outline
Introduction
Chapter 1
Probability
Counting techniques
Rules of probability
Conditional probability and independence
Multiplicative Rule
Bayes Rule, Simpsons paradox
Chapter 2
Random variables
Discrete random variables - their distributions
Continuous random variables - their
distributions
Expectation
Rules of expectation
Moments variance, standard deviation, skewness,
kurtosis
Moment generating functions
Chapters 3 and 4
Multivariate probability distributions
Discrete and continuous bivariate distributions
Marginal distributions, Conditional
distributions
Expectation for multivariate distributions
Regression and Correlation
Chapter 5
Functions of random variables
Distribution function method, moment
generating function method, transformation
method
Law of large numbers, Central Limit theorem
Chapter 5, 7
Introduction to Probability
Theory
Probability Models for random
phenomena
Phenomena
Deterministic
Non-deterministic
Deterministic Phenomena
There exists a mathematical model that allows
perfect prediction the phenomenas
outcome.
Many examples exist in Physics, Chemistry
(the exact sciences).
Non-deterministic Phenomena
No mathematical model exists that allows
perfect prediction the phenomenas
outcome.
Non-deterministic Phenomena
may be divided into two groups.
1. Random phenomena
Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.
2. Haphazard phenomena
unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.

Phenomena
Deterministic
Non-deterministic
Haphazard
Random
Haphazard phenomena
unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.
Do such phenomena exist?
Will any non-deterministic phenomena exhibit
long-run statistical regularity eventually?

Random phenomena
Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.
Examples
1. Tossing a coin outcomes S ={Head, Tail}
Unable to predict on each toss whether is Head or
Tail.
In the long run can predict that 50% of the time
heads will occur and 50% of the time tails will occur
2. Rolling a die outcomes
S ={ , , , , , }
Unable to predict outcome but in the long run can
one can determine that each outcome will occur 1/6
of the time.
Use symmetry. Each side is the same. One side
should not occur more frequently than another side
in the long run. If the die is not balanced this may
not be true.
3. Rolling a two balanced dice 36 outcomes
4. Buffoons Needle problem
A needle of length l, is tossed and allowed to land
on a plane that is ruled with horizontal lines a
distance, d, apart
d
l
A typical outcome
0
50
100
150
200
250
0 20 40 60 80 100
day
p
r
i
c
e
5. Stock market performance
A stock currently has a price of $125.50. We will
observe the price for the next 100 days
typical outcomes
Definitions
The sample Space, S
The sample space, S, for a random phenomena
is the set of all possible outcomes.
The sample space S may contain
1. A finite number of outcomes.
2. A countably infinite number of outcomes, or
3. An uncountably infinite number of outcomes.
A countably infinite number of outcomes means
that the outcomes are in a one-one
correspondence with the positive integers
{1, 2, 3, 4, 5, }
This means that the outcomes can be labeled
with the positive integers.
S = {O
1
, O
2
, O
3
, O
4
, O
5
, }
A uncountably infinite number of outcomes
means that the outcomes are can not be put in a
one-one correspondence with the positive
integers.
Example: A spinner on a circular disc is spun
and points at a value x on a circular disc whose
circumference is 1.
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
x
S = {x | 0 x <1} = [0,1)
0.0
1.0
[ )
S
Examples
1. Tossing a coin outcomes S ={Head, Tail}
2. Rolling a die outcomes
S ={ , , , , , }
={1, 2, 3, 4, 5, 6}
3. Rolling a two balanced dice 36 outcomes
S ={ (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6),
(2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (2, 6),
(3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (3, 6),
(4, 1), (4, 2), (4, 3), (4, 4), (4, 5), (4, 6),
(5, 1), (5, 2), (5, 3), (5, 4), (5, 5), (5, 6),
(6, 1), (6, 2), (6, 3), (6, 4), (6, 5), (6, 6)}
outcome (x, y),
x = value showing on die 1
y = value showing on die 2
4. Buffoons Needle problem
A needle of length l, is tossed and allowed to land
on a plane that is ruled with horizontal lines a
distance, d, apart
d
l
A typical outcome
An outcome can be identified by determining the
coordinates (x,y) of the centre of the needle and
u, the angle the needle makes with the parallel
ruled lines.
(x,y)
u
S = {(x, y, u)| - < x < , - < y < , 0 u t }
An Event , E
The event, E, is any subset of the sample space,
S. i.e. any set of outcomes (not necessarily all
outcomes) of the random phenomena
S
E
The event, E, is said to have occurred if after
the outcome has been observed the outcome lies
in E.
S
E
Examples
1. Rolling a die outcomes
S ={ , , , , , }
={1, 2, 3, 4, 5, 6}
E = the event that an even number is
rolled
= {2, 4, 6}
={ , , }
2. Rolling a two balanced dice 36 outcomes
S ={ (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6),
(2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (2, 6),
(3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (3, 6),
(4, 1), (4, 2), (4, 3), (4, 4), (4, 5), (4, 6),
(5, 1), (5, 2), (5, 3), (5, 4), (5, 5), (5, 6),
(6, 1), (6, 2), (6, 3), (6, 4), (6, 5), (6, 6)}
outcome (x, y),
x = value showing on die 1
y = value showing on die 2
E = the event that a 7 is rolled
={ (6, 1), (5, 2), (4, 3), (3, 4), (3, 5), (1, 6)}
Special Events
The Null Event, The empty event - |
| = { } = the event that contains no outcomes
The Entire Event, The Sample Space - S
S = the event that contains all outcomes
The empty event, | , never occurs.
The entire event, S, always occurs.
Set operations on Events
Union
Let A and B be two events, then the union of A
and B is the event (denoted by ) defined by:
AB = {e| e belongs to A or e belongs to B}
AB
A B
B A
B A
B A
The event AB occurs if the event A occurs or
the event and B occurs .
AB
A B
B A
B A
Intersection
Let A and B be two events, then the intersection
of A and B is the event (denoted by AB ) defined
by:
A B = {e| e belongs to A and e belongs to B}
AB
A B
B A
B A
B A
AB
A B
The event AB occurs if the event A occurs and
the event and B occurs .
B A
B A
Complement
Let A be any event, then the complement of A
(denoted by ) defined by:
= {e| e does not belongs to A}
A
A
A
A
The event occurs if the event A does not
occur
A
A
A
In problems you will recognize that you are
working with:

1. Union if you see the word or,
2. Intersection if you see the word and,
3. Complement if you see the word not.

1. A B A B =
2. A B A B =
DeMoivres laws
=

=
1. A B A B =
2. A B A B =
DeMoivres laws (in words)
The event A or B does not occur if
the event A does not occur
and
the event B does not occur
=
The event A and B does not occur if
the event A does not occur
or
the event B does not occur
( )
( )
A A B A B =
Another useful rule
=
In words
The event A occurs if
A occurs and B occurs
or
A occurs and B doesnt occur.
1. A A | =
Rules involving the empty set, |, and
the entire event, S.
2. A | | =
3. A S S =
4. A S A =
Definition: mutually exclusive
Two events A and B are called mutually
exclusive if:
A B | =
A
B
If two events A and B are are mutually
exclusive then:
A
B
1. They have no outcomes in common.
They cant occur at the same time. The outcome
of the random experiment can not belong to both
A and B.
Some other set notation
We will use the notation
e A e
to mean that e is an element of A.
We will use the notation
e A e
to mean that e is not an element of A.
Thus
{ }
or A B e e A e B = e e
{ }
and A B e e A e B = e e
{ }
A e e A = e
We will use the notation
(or ) A B B A c
to mean that A is a subset B. (B is a superset of A.)
. . if then . e A e B e e i e
B
A
Union and Intersection
more than two events
Union:
1 2 3
1

k
i k
i
E E E E E
=
=
E
1

E
2

E
3

1 2 3
1

i
i
E E E E

=
=
Intersection:
1 2 3
1

k
i k
i
E E E E E
=
=
E
1

E
2

E
3

1 2 3
1

i
i
E E E E

=
=
1.
i i
i i
E E
| |
=
|
\ .
DeMorgans laws
=
2.
i i
i i
E E
| |
=
|
\ .
Probability
Suppose we are observing a random phenomena
Let S denote the sample space for the phenomena, the
set of all possible outcomes.
An event E is a subset of S.

A probability measure P is defined on S by defining
for each event E, P[E] with the following properties
1. P[E] 0, for each E.
2. P[S] = 1.

3. | |
if for all ,
i i i i
i i
P E P E E E i j |
(
= =
(


| | | | | |
1 2 1 2
P E E P E P E = + +
| |
i i
i i
P E P E
(
=
(


P[E
1
] P[E
2
]
P[E
3
]
P[E
4
]
P[E
5
]
P[E
6
]
Example: Finite uniform probability
space
In many examples the sample space S = {o
1
, o
2
,
o
3
, o
N
} has a finite number, N, of oucomes.
Also each of the outcomes is equally likely
(because of symmetry).
Then P[{o
i
}] = 1/N and for any event E
| |
( )
( )
( ) no. of outcomes in
=
total no. of outcomes
n E n E
E
P E
n S N
= =
( )
: = no. of elements of n A A Note
Note:
with this definition of P[E], i.e.
| |
( )
( )
( ) no. of outcomes in
=
total no. of outcomes
n E n E
E
P E
n S N
= =
| |
1. 0. P E >
| |
( )
( )
2. = 1
n S
P S
n S
=
( ) ( )
1 2
3.
i
i
i
i
n E
n E n E
P E
N N
| |
|
+ +
(
\ .
= =
(

| | | |
1 2
if
i j
P E P E E E | = + + =
Thus this definition of P[E], i.e.
| |
( )
( )
( ) no. of outcomes in
=
total no. of outcomes
n E n E
E
P E
n S N
= =
satisfies the axioms of a probability measure
1. P[E] 0, for each E.
2. P[S] = 1.

3. | |
if for all ,
i i i i
i i
P E P E E E i j |
(
= =
(


| | | | | |
1 2 1 2
P E E P E P E = + +
Another Example:
We are shooting at an archery target with radius
R. The bullseye has radius R/4. There are three
other rings with width R/4. We shoot at the target
until it is hit
R
S = set of all points in the target
= {(x,y)| x
2
+ y
2
R
2
}
E, any event is a sub region (subset) of S.
E, any event is a sub region (subset) of S.
E
| |
( )
( )
( )
2
: =
Area E Area E
P E
Area S R t
= Define
| |
2
2
1
4
16
R
P Bullseye
R
t
t
| |
|
\ .
= =
| |
2 2
2
3 2
9 4 5
4 4

16 16
R R
P White ring
R
t t
t
| | | |

| |

\ . \ .
= = =
Thus this definition of P[E], i.e.
satisfies the axioms of a probability measure
1. P[E] 0, for each E.
2. P[S] = 1.

3. | |
if for all ,
i i i i
i i
P E P E E E i j |
(
= =
(


| | | | | |
1 2 1 2
P E E P E P E = + +
| |
( )
( )
( )
2
=
Area E Area E
P E
Area S R t
=
Finite uniform probability space
Many examples fall into this category
1. Finite number of outcomes
2. All outcomes are equally likely

3.
| |
( )
( )
( ) no. of outcomes in
=
total no. of outcomes
n E n E
E
P E
n S N
= =
( )
: = no. of elements of n A A Note
To handle problems in case we have to be able to
count. Count n(E) and n(S).