Basics of Probability Theory

Tieming Ji
Fall 2012
1 / 21
Denition: Sample space is the set of all possible
outcomes of an experiment. We often use S to
denote the sample space.
Example 1: Flip a coin. What are the all possible
outcomes?
S = {H, T}.
Example 2: Flip a coin twice. What are the all possible
outcomes?
S = {HH, HT, TH, TT}.
2 / 21
Denition: An event is a subset of the sample space.
Example: Flip a coin twice. Use A to denote the event that at least one
tail occurs in two ips. What are the all possible outcomes to make event
A occur?
A = {HT, TH}.
Denition: In the case where all outcomes in S are
equally likely, then the probability that an event A
happens is given by
P(A) =
#(A)
#(S)
.
If the coin is fair in the previous example, P(A) =
#{HT,TH}
#{HH,HT,TH,TT}
=
1
2
.
3 / 21
Example. Roll a fair dice twice.
What is the sample space S?
S =

(1, 1) (1, 2) (1, 6)

(2, 1) (2, 2) (2, 6)
.
.
.
.
.
.
.
.
.
.
.
.
(6, 1) (6, 2) (6, 6)

.
Let A denote the event that the sum of two rolls is 7. What is A?
A = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)}.
Compute P(A).
Since the dice is fair, P(A) =
#(A)
#(S)
=
1
6
.
4 / 21
Theorem: (Multiplication Principle) Suppose there are r
experiments. For the 1st experiment, there are n
1
outcomes.
For each outcome of the 1st experiment, there are n
2
outcomes for the 2nd experiment. For each outcome of the
1st and the 2nd experiment, there are n
3
outcomes for the 3rd
experiment. . For the all possible outcomes for the rst
(r 1) experiments, there are n
r
outcomes for the r th
experiment. Then the total number of possible outcomes for
all these r experiments is n
1
n
2
n
3
n
r
.
Multiplication principle helps to count the number of sample points (or
outcomes) in the sample space and events.
5 / 21
Example. Suppose there are totally 3 types of topping (vegetarian,
pepperoni and beef) in a pizza shop and 2 kinds of curst (thin and thick).
How many dierent pizzas can you make?
There are two steps, choose topping and then crust (or curst rst
and topping secondly). No matter which topping you choose, there
are 2 choices of curst. By the multiplication principle,
#(S) = 3 2 = 6.
Draw a tree diagram for the sample space.
6 / 21
Example. In a Mexican restaurant, a burrito can be made with 3 types of
tortilla, 2 types of rice, 2 types of beans, 3 types of meat and 3 types of
salsa. Lets assume that each burrito is going to be made and only made
by these ingredients.
How many dierent burritos can a customer order?
By the multiplication rule,
#(S) = 3(tortilla) 2(rice) 2(beans) 3(meat) 3(salsa) = 108.
How many dierent burritos can a customer order if the meat has
been chosen?
Let A denote a burrito is ordered with beef. By the multiplication
rule, when the meat is chosen,
#(A) = 3(tortilla) 2(rice) 2(beans) 3(salsa) = 36.
If each kind of burrito is equally likely to be ordered. What is the
probability that a beef burrito is ordered?
P(A) =
#(A)
#(S)
=
36
108
=
1
3
.
7 / 21
Denition: A permutation is an arrangement of
objects with a denite order.
Example. Select 2 students from 4 students, where one is the president
and the other is the vice president of the student organization. How
many ways of selection?
Solution: Label the students as student 1, student 2, student 3 and
student 4. Then, the ways to choose (President, Vice President) are
(1,2), (1,3), (1,4), (2,1), (2,3), (2,4), (3,1), (3,2), (3,4), (4,1), (4,2),
(4,3), totally 12 ways. Notice that (1,2) and (2,1) are dierent selection
since we need to consider order here (a permutation).
8 / 21
Theorem: When permute k objects chosen from n
objects without repetition. The total number of
permutations is denoted by
n
P
k
or P
n
k
, and is given
by
n
P
k
=
n!
(n k)!
.
Notice that
n
P
n
=
n!
0!
= n!, choose n objects from n objects and arrange.
n
P
0
=
n!
n!
= 1, choose 0 objects from n objects and arrange.
9 / 21
Example. Choose 3 letters out of {A, B, C, D} to make a combination
code without repetition. How many dierent combination codes are
there?
Solution: To make a combination code, we need to choose 3 letters out
of 4 letters and arrange them. This is permutation without repetition.
Thus, the total number of dierent combination codes one can make is
4
P
3
=
4!
(4 3)!
= 24.
10 / 21
Theorem: When permute k objects chosen from n
objects with repetition. The total number of
permutations is n
k
.
Example. Make a 3-letter combination code by using letters in {A, B, C,
D}. How many dierent combination codes are there?
Solution: To make a combination code, there are 4 choices for the rst
position of the 3-letter code. No matter which letter one chooses for the
rst position, there are 4 choices for the second position. Similarly for the
third position. So, by using the multiplication principle, there are totally
4 4 4 = 4
3
combination codes.
11 / 21
Denition: A combination is a selection of objects
without regard to order.
Example. Select 2 students from 4 students to serve in the student
organization. How many ways of selection?
Solution: Label the students as student 1, student 2, student 3 and
student 4. Then, the ways to choose two students are (1,2), (1,3), (1,4),
(2,3), (2,4), (3,4), totally 6 ways. Notice that (1,2) and (2,1) are the
same selections.
Note:
A permutation considers the order of selection.
A combination does not consider the order of selection.
12 / 21
Theorem: When choosing k objects from n objects.
The total number of combinations is denoted by
n
C
k
or C
n
k
or

n
k

, and is given by
n
C
k
=
n!
k! (n k)!
.
Notice that
n
C
k
=
n
P
k
k!
. This is reasonable, because for every k chosen objects,
there are k! ways to permute these k chosen objects. While a
combination will view these k! arrangements as one same
combination.
n
C
0
= 1, choose 0 objects from n objects, and there is only 1 way to
choose.
13 / 21
Example 1. There are 10 trac lights in sequence. At one moment, how
many dierent selections of 3 lights to be yellow?
Solution: If we label the 10 trac lights as light 1, light 2, ..., light 10.
Then choosing light 1, 2, 3 to be yellow is equal to choosing light 2, 3, 1
to be yellow. So this is a combination problem. The total number of
combinations is
10
C
3
.
14 / 21
Example 2. Construct a news team. Choose 1 journalist from 3
candidates, 3 photographers from 5 candidates, and 2 technicians from
10 candidates.
How many dierent teams can be constructed?
There are
3
C
1
ways to choose a journalist;
5
C
3
ways to choose
photographers; and
10
C
2
ways to choose technicians. By the
multiplication principle, there are totally
3
C
1

5
C
3

10
C
2
ways to
construct a team.
Suppose now 2 photographers out of the 5 candidates are chosen,
how many dierent teams can be constructed?
Since 2 photographers have been chosen, there are
52
C
32
=
3
C
1
ways to choose the 1 photographer vacancy left. So, there are
totally
3
C
1

3
C
1

10
C
2
ways to construct a team now.
What is the probability of getting a team with 2 specied
photographers?
Since dierent teams are equally likely to be constructed, the
probability is
3
C
1

3
C
1

10
C
2
3
C
1

5
C
3

10
C
2
.
15 / 21
Property: The number of combinations of choosing
k objects from n objects is equal to the number of
combinations of choosing (n k) objects from n
objects. That is
n
C
k
=
n
C
nk
.
This is reasonable, because choosing k objects from n objects, you have
(n k) objects left. This is equivalent to choosing (n k) objects with k
objects left.
16 / 21
One useful method to prove an equation is the story telling method,
especially when it involves combination or permutation terms. For
example, we want to prove, for positive integers k and n, where k n,
k
n
C
k
= n
n1
C
k1
.
Proof:
On the left-hand side of the equation, it equals to the number of
choosing k people out of n people to construct a team, and then choose
1 person out of the k chosen people to be the team captain.
Meanwhile, on the right-hand side of the equation, one chooses one
captain rst from n people (n ways to choose), and then choose (k 1)
non-captain members for the team out of (n 1) people left.
Both of the two sides are computing the total number of dierent ways
of constructing a k-person team with a captain from n people. Thus,
they are equal.
Challenging question:
Can you prove: k (k 1)
n
C
k
= n (n 1)
n2
C
k2
?
17 / 21
Theorem: (Permutation of Indistinguishable Objects) There
are n objects to arrange, where they are of r groups.
Specically, n
1
objects are identical in group 1, n
2
objects are
identical in group 2, ..., n
r
objects are identical in group r .
Then, the total number of permutations for these n objects is
given by
n!
n
1
!n
2
! n
r
!
, n
1
+ n
2
+ + n
r
= n.
18 / 21
Example 1. There are 10 trac lights in sequence. How many dierent
sequences of lights if 3 of them are yellow, 5 of them are green, and 2 are
red?
Solution:
10!
3!5!2!
.
Example 2. Make a 5-letter code using letters in {A, B, C, D}. How
many dierent 5-letter codes are there with 2 As, 2 Bs and 1 C?
Solution: The total number of codes with 2 As, 2 Bs and 1 C (and 0 D)
is
5!
2!2!1!0!
.
19 / 21
Set Operations
Events A and B,
A: complement of A, which contains all the sample points
not in A.
A B: union of A and B, which contains all sample points
either in A or in B or in both.
A B: intersection of A and B, which contains all sample
points that are in both A and B.
A B: A is a subset of B if and only if every sample
point in A is also in B.
A = B if and only if A B and A B.
(continued)
20 / 21
Set Operations (continued)
A and B are disjoint if and only if A B = , where
denotes an empty set.
A
1
, A
2
, ... (innite sequence of sets) are disjoint or
mutually exclusive, if no two of them have common sample
points. That is, for any i and j , where i = j , A
i
A
j
= .
Example. Flip a coin. Stop when landing a tail. The sample space is
S = {T, HT, HHT, }. Let A
k
denotes the event where there are exact
k ips. Verify that A
1
, A
2
, are mutually exclusive (or disjoint).
21 / 21