Fundamental Concepts of Probability

Probability is a measure of the likelihood that a

particular event will occur.
Certain that an event will occur, its probability is 1
or 100%.
Certainly will not occur, its probability is zero.
Say we toss a fair coin: just as likely to give heads as tails.
It is entirely possible that six tosses of the coin would give
six heads or six tails, or anything in between, so the
relative frequency of heads would vary from zero to one.
If it is just as likely that an event will occur as that it
will not occur, its true probability is 0.5 or 50%.

1
 Fundamental Concepts of Probability
The relative frequency of a particular event, or the proportion
of trials giving outcomes which meet certain requirements,
will give an estimate of the probability of that event.
The larger the number of trials, the more reliable that estimate
will be. This is the empirical or frequency approach to
probability.
Example 6.1
260 bolts are examined as they are produced. Five of them are
found to be defective. On the basis of this information,
estimate the probability that a bolt will be defective.
Answer
The probability of a defective bolt is approximately equal to
the relative frequency, which is 5/260 = 0.019.
2
 Fundamental Concepts of Probability
Another type of probability is the subjective estimate, based
on a persons experience.
Say a geological engineer examines geological information
on a particular property. He chooses the best site to drill an
oil well, and on the basis of his experience he estimates that
the probability the well will be successful is 30%.
Another geological engineer using the same information
might well come to a different estimate. This, then, is a
subjective estimate of probability.
The executives of the company can use this estimate to decide
whether to drill the well.

3
 Fundamental Concepts of Probability
A third approach includes various games, such as:
drawing a colored ball from a number of balls,
tossing an unbiased coin;
throwing an unbiased die; or
drawing a card from a well-shuffled deck of cards.

In each of these cases we can say before the trial that a

number of possible results are equally likely. This is
the classical or a priori approach.
This approach is often simple to visualize, so giving a
better understanding of probability.
4
 Example 6.2
Three nuts with metric threads have been accidentally
mixed with twelve nuts with U.S. threads. To a person
taking nuts from a bucket, all fifteen nuts seem to be
the same. One nut is chosen randomly. What is the
probability that it will be metric?

Answer
There are fifteen ways of choosing one nut, and they are
equally likely. Three of these equally likely outcomes
give a metric nut. Then the probability of choosing a
metric nut must be 3/15, or 20%.
5
 Example 6.3
Two fair coins are tossed. What is the probability of
getting one head and one tail?
Answer
For a fair or unbiased coin, for each toss of each coin:
Pr [heads] = Pr [tails] =
There are two possible results of tossing the first coin:
heads (H) and tails (T), and they are equally likely.
Whether the result of tossing the first coin is H or T,
there are two possible results of tossing the second
coin. Again, these are heads (H) and tails (T), and
they are equally likely.
6
 Example 6.3
The possible outcomes of tossing the two coins are HH,
HT, TH, and TT.
Since the results H and T for the first coin are equally
likely, and the results H and T for the second coin are
equally likely, the four outcomes of tossing the two
coins must be equally likely.
These relationships are conveniently summarized in the
following tree diagram, Figure 6.1, in which each
branch point (or node) represents a point of decision
where two or more results are possible.

7
 Figure 6.1: Simple Tree Diagram
F

8
 Fundamental Concepts of Probability
Since there are four equally likely outcomes, the
probability of each is 1/4.
Both HT and TH correspond to getting one heads and
one tails, so two of the four equally likely outcomes
give this result.
Then the probability of getting one heads and one tails
must be 2/4 = 1/2 or 0.5.
If a six-sided cube (called a die) is tossed, we define the
outcome as the number of dots on the face which is
upward when the die comes to rest.
The possible outcomes are 1, 2, 3, 4, 5, and 6. We might
call each of these outcomes a separate event. 9
 Basic Rules of Combining Probabilities
Addition Rule
This can be divided into two parts, depending upon
whether there is overlap between the events being
combined.
I) If the events are mutually exclusive, there is no
overlap: if one event occurs, other events cannot
occur.
The probability of occurrence of one or another event
is the sum of the probabilities of the separate events.
Throw a fair six-sided die: Pr [6] = 1/6, Pr [4] = 1/6,
so Pr [6 or 4] is 1/6 + 1/6 = 1/3.
10
 Basic Rules of Combining Probabilities
The Addition Rule corresponds to a logical or and
gives a sum of separate probabilities.
We can divide all possible outcomes into two groups
without overlap. If one group of outcomes is event A,
the other group is called the complement of A and is
written A or A .
Since A and A together include all possible results,
the sum of Pr [A] + Pr [A] = 1.
If Pr [A] is more easily calculated than Pr [A], the
best approach to calculating Pr [A] may be by first
calculating Pr [A].
11
 Example 6.4
A sample of four electronic components is taken from the output
of a production line. The probabilities of the various outcomes
are calculated to be: Pr [0 defectives] = 0.6561, Pr [1 defective]
= 0.2916, Pr [2 defectives] = 0.0486, Pr [3 defectives] =
0.0036, Pr [4 defectives] = 0.0001. What is the probability of at
least one defective?
Answer
It would be perfectly correct to calculate as follows:
Pr [at least one defective] = Pr [1 defective] + Pr [2 defectives]
+ Pr [3 defectives] + Pr [4 defectives]
= 0.2916 + 0.0486 + 0.0036 + 0.0001 = 0.3439; OR
Pr [at least one defective] = 1 Pr [0 defectives]
= 1 0.6561 = 0.3439.
12
 Basic Rules of Combining Probabilities
II) If the events are not mutually exclusive, there can be overlap
between them. This can be visualized using a Venn diagram.
The probability of overlap must be subtracted from the sum of
probabilities of the separate events (i.e., we must not count the
same area on the Venn Diagram twice).

Circle A represents the probability of event A, and circle B

represents the probability of event B, and the whole rectangle
represents all possible outcomes of a particular experiment and
is called the sample space of that experiment.
13
 Basic Rules of Combining Probabilities
Pr [A B) = Pr [occurrence of A or B or both], the
union of the two events A and B.
Pr [A B) = Pr [occurrence of both A and B], the
intersection of events A and B.
The intersection A B represents the overlap between
events A and B.
If the events being considered are not mutually
exclusive, and so there may be overlap between them,
the Addition Rule becomes:
Pr [A B) = Pr [A] + Pr [B] Pr [A B] . (6.1)
The next slide shows Venn diagrams representing
intersection, union, and complement.
14
 Basic Rules of Combining Probabilities
Set Relations on Venn Diagrams

The cross-hatched area on (a) represents event A.

The cross-hatched area on (b) shows the intersection of events A
and B. The union of events A and B is shown on part (c).
The cross-hatched area of part (d) represents the complement of
event A.
15
 Example 6.5
If one card is drawn from a well-shuffled bridge deck of 52
playing cards (13 of each suit), what is the probability that
the card is a queen or a heart? A queen of hearts
(queen heart) overlaps the two categories.
Pr [queen] = 4/52; Pr [heart] = 13/52; Pr [queen heart] = 1/52.

Then Pr [queen heart] = Pr [queen] + Pr [heart] Pr [queen heart]

= 4/52 + 13/52 1/52 = 16/52 16
 Example 6.6
The class registrations of 120 students are analyzed:
30 of the students do not take any of Applied Mechanics,
Chemistry, or Computers.
15 of them take only Applied Mechanics.
25 of them take Chemistry and Computers but not Applied
Mechanics.
20 of them take Applied Mechanics and Computers but
not Chemistry.
10 of them take all three of Applied Mechanics,
Chemistry, and Computers.
A total of 45 of them take Chemistry.
5 of them take only Chemistry.
17
 Example 6.6
a) How many of the students take Applied Mechanics
and Chemistry but not Computers?
b) How many of the students take only Computers?
c) What is the total number of students taking
Computers?
d) If a student is chosen at random from those who take
neither Chemistry nor Computers, what is the
probability that he or she does not take Applied
Mechanics either?
e) If one of the students who take at least two of the
three courses is chosen at random, what is the
probability that he or she takes all three courses?
18
 Example 6.6 Solution
Lets abbreviate the courses as AM, CH, and CP
The number of items in the sample space, which is the
total number of items under consideration, is 120.
Then the Venn diagram incorporating the given
information for this problem is shown below.

19
 Example 6.6 Solution
Two of the simple areas on the diagram correspond to
unknown numbers:
The first one is (AM CH CP), which is taken by
x students.
The other is (AM CH CP), so only Computers
but not the other courses, and that is taken by y
students.
In terms of quantities corresponding to simple areas
on the Venn diagram, the given information that a
total of 45 of the students take Chemistry requires
that: x + 10 + 25 + 5 = 45,
Thus, x = 5.
20
 Example 6.6 Solution
Let n(...) be the number of students who take a specified
course or combination of courses. Then from the total
number of students and the number who do not take
any of the three courses we have:
n(AM CH CP) = 120 30 = 90
But from the Venn diagram and the knowledge of the
total taking Chemistry we have:
n(AM CH CP) = n(CH) + n(AM CH CP)
+n(AM CH CP) + n(AM CH CP)
= 45 + 15 + 20 + y = 80 + y,
Then, y = 90 80 = 10.
21
 Example 6.6 Solution
Now we can answer the specific questions.
a) The # of students who take Applied Mechanics and
Chemistry but not Computers is x = 5.
b) The # of students who take only Computers is y = 10.
c) The total number of students taking Computers is
10 + 20 + 10 + 25 = 65.
d) The number of students taking neither Chemistry nor
Computers is 15 + 30 = 45. Of these, the number who
do not take Applied Mechanics is 30.
If a student is chosen randomly from those who take neither
Chemistry nor Computers, the probability that he or she
does not take Applied Mechanics either is 30/45 = 2/3. 22
 Example 6.6 Solution
e) The number of students who take at least two of the
three courses is:
n(AM CH CP) + n(AM CH CP) +
n(AM CH CP) + n(AM CH CP)
= 5 + 20 + 25 + 10 = 60.

Of these, the number who take all three courses is 10.

If a student taking at least two courses is chosen

randomly, the probability that he or she takes all three
courses is 10/60 = 1/6.
23
 Basic Rules of Combining Probabilities
Multiplication Rule
The basic idea for calculating the number of choices can
be described as follows:
Say there are n1 possible results from one operation.
For each one of these, there are n2 possible results from
a second operation.
Then there are (n1 n2) possible outcomes of the two
operations together.
In general, the numbers of possible results are given by
products of the number of choices at each step.
Probabilities can be found by taking ratios of possible
results. 24
 Basic Rules of Combining Probabilities
Example 6.7

In one case a byte is defined as a sequence of 8 bits.

Each bit can be either zero or one. How many
different bytes are possible?

Solution

We have 2 choices for each bit and a sequence of 8 bits.

Then the number of possible results is:
2x2x2x2x2x2x2x2 = (2)8 = 256.
25
 Basic Rules of Combining Probabilities
The simplest form of the Multiplication Rule for
probabilities is as follows:
If the events are independent, then the occurrence of one
event does not affect the probability of occurrence of
another event.
In that case the probability of occurrence of more than
one event together is the product of the probabilities
of the separate events.
If A and B are two separate events that are independent
of one another, the probability of occurrence of both
A and B together is given by:
Pr [A B] = Pr [A] Pr [B] .. (6.2)
26
 Example 6.8
If a player throws two fair dice, the probability of a
double one (one on the first die and one on the
second die) is (1/6)(1/6) = 1/36.

These events are independent because the result from

one die has no effect at all on the result from the
other die.

(Note that die is the singular word, and dice is

plural.)

27
 Basic Rules of Combining Probabilities
If the events are not independent, one event
affects the probability for the other event.
In this case conditional probability must be used.

The conditional probability of B given that A

occurs, or on condition that A occurs, is written
Pr [B | A].

This is read as the probability of B given A, or

the probability of B on condition that A occurs.
28
 Basic Rules of Combining Probabilities
The multiplication rule for the occurrence of both A
and B together when they are not independent is the
product of the probability of one event and the
conditional probability of the other, i.e.,:
Pr [A B] = Pr [A] Pr [B | A] = Pr [B] Pr [A | B]
This implies that conditional probability can be obtained
by using:
.. (6.4)

... (6.5)

29
 Example 6.9
Four of the light bulbs in a box of ten bulbs are defective.
If two bulbs are selected at random without
replacement and tested, (i) what is the probability that
exactly one defective bulb is found? (ii) What is the
probability that exactly two defective bulbs are found?
Solution
A tree diagram is very useful in problems involving the
multiplication rule.
Let us use the symbols D1 for a defective first bulb, D2
for a defective second bulb, G1 for a good first bulb,
and G2 for a good second bulb.

30
 Example 6.9 Solution
At the beginning the box contains four bulbs which are
defective and six which are good. Then the probability
that the first bulb will be defective is 4/10 and the
probability that it will be good is 6/10. This is shown
in the partial tree diagram below.

First Bulb
31
 Example 6.9 Solution
Probabilities for the second bulb vary, depending on
what was the result for the first bulb, and so are given
by conditional probabilities. These relations for the
second bulb are shown in the Figure below.

Second Bulb
32
 Example 6.9 Solution
If the first bulb was defective, the box will then contain three
defective bulbs and six good ones, so the conditional
probability of obtaining a defective bulb on the second
draw is 3/9, and the conditional probability of obtaining a
good bulb is 6/9.
If the first bulb was good the box will contain four defective
bulbs and five good ones, so the conditional probability of
obtaining a defective bulb on the second draw is 4/9 , and
the conditional probability of obtaining a good bulb is 5/9.
Notice that these arguments hold only when the bulbs are
selected without replacement; if the chosen bulbs had
been replaced in the box and mixed well before another
bulb was chosen, the relevant probabilities would be
different. 33
 Example 6.9 Solution
Now let us combine the separate probabilities.
The probability of getting two defective bulbs must be
(4/10)(3/9) = 12/90,
The probability of getting a defective bulb on the first
draw and a good bulb on the second draw is
(4/10)(6/9) = 24/90,
The probability of getting a good bulb on the first
draw and a defective bulb on the second draw is
(6/10)(4/9) = 24/90, and
The probability of getting two good bulbs is
(6/10)(5/9) = 30/90.
34
 Example 6.9 Solution
In symbols, we have:

Notice that both D1 G2 and G1 D2 correspond to

obtaining 1 good bulb and 1 defective bulb.

35
 Example 6.9 Solution
The complete tree diagram is shown in the figure below.

All the probabilities of events add up to one:

(12 + 24 + 24 + 30)/90 = 1 36
 Example 6.9 Solution
Now we have to answer the specific questions which were
asked:
i) Pr [exactly one defective bulb is found]
= Pr [D1 G2] + Pr [G1 D2]
= (24 + 24)/90 = 48/90 = 0.533.
The first term corresponds to getting first a defective bulb
and then a good bulb, and the second term corresponds to
getting first a good bulb and then a defective bulb.
ii) Pr [exactly two defective bulbs are found]
= Pr [D1 D2] = 12/90 = 0.133.
There is only one path which will give this result.
37
 Example 6.10
A fair six-sided die is tossed twice. What is the probability that a
five will occur at least once?
Answer: Note that this problem includes the possibility of
obtaining two fives. On any one toss, the probability of a five is
1/6 , and the probability of no fives is 5/6 . This problem will be
solved in several ways. Tree Diagram for two tosses:

38
 Example 6.10 Solution
First solution (considering all possibilities using a tree diagram):

Total of all probabilities (as a check) = 1

39
 Example 6.10 Solution
Second solution (using conditional probability):
The probability of at least one five is given by:
Pr [5 on the first toss] Pr [at least one 5 in two tosses | 5 on the
first toss] + Pr [no 5 on the first toss] Pr [at least one 5 in two
tosses | no 5 on the first toss].
But Pr [5 on the first toss] = Pr [5 on any one toss] = 1/6.
and Pr [at least one 5 in two tosses | 5 on the first toss] = 1 (a dead
certainty!)
Also Pr [no 5 on the first toss] = Pr [no 5 on any toss] = 5/6, and
Pr [at least one 5 in two tosses | no 5 on the first toss] = Pr [5 on the
second toss] = 1/6.
Then,
Pr [at least one 5 in two tosses] = (1/6)(1) + (5/6)(1/6) = 11/36.
40
 Example 6.10 Solution
Third solution (using the addition rule, eq. 6.1):
Pr [at least one 5 in two tosses]
= Pr [(5 on the first toss) (5 on the second toss)]
= Pr [5 on the first toss] + Pr [5 on the second toss]
Pr [(5 on the first toss) (5 on the second toss)]

Fourth solution: Look at the sample space (i.e., consider

all possible outcomes).
Lets use a matrix notation where each entry gives first the
result of the first toss and then the result of the second
toss, as follows: 41
 Example 6.10 Solution
In the fifth row the result of the first toss is a 5, and in the
fifth column the result of the second toss is a 5.
This row and this column have been shaded and represent
the part of the sample space which meets the
requirements of the problem.
This area contains 11 entries, whereas the whole sample
space contains 36 entries, so Pr [at least one 5 in two
tosses] = 11/36.

42
 Example 6.10 Solution
Fifth solution (and the fastest):

The probability of no fives in two tosses is:

(5/6)(5/6) = 25/36

Because the only alternative to no fives is at least one five,

Pr [at least one 5 in two tosses] = 1 (25/36) = 11/36

43