Beruflich Dokumente
Kultur Dokumente
0 OVERVIEW
Unit Structure Uncertainty plays an important role in almost all the spheres of life. In many situations
one of a number of possible outcomes may occur and the theory of probability provides
2.0 Overview methods for quantifying the chances associated with the various outcomes. In this Unit,
2.1 Learning Objectives the student is introduced to the concept to probability and to its axiomatic definition. The
2.2. The Concept of Probability concept of mutually exclusive and independent events is also covered, following by
2.2.1 Introduction Bayes Theorem. Finally, there are sections on counting techniques including
2.2.2 Sample Space and Events permutations and combinations.
2.3 Relating the Events
2.3.1 Definitions
2.3.2 Results from Set Theory 2.1 LEARNING OBJECTIVES
2.4 Axioms of Probability
2.4.1 Simple Consequences of the axioms After reading this unit, you should be able to:
A statistical experiment is any process that leads to a single outcome out of a number of
possible outcomes. We are unable to predict which outcome will actually occur.
Unit 2 1 Unit 2 2
Definition 2.1 An experiment is the process by which an observation is made. Example 2.1
Consider an experiment in which each of the three automobiles taking a particular free-
For example, consider an experiment consisting of tossing a die and observing the way exit turns left (L) or right (R) at the end of the exit ramp.
number on the upface. There are six basic possible outcomes 1, 2, 3, 4, 5, 6. If this
experiment is conducted once, we can observe only one of the six outcomes and the (a) List the 8 simple events of the experiment.
outcome cannot be predicted with certainty. (b) Give 3 examples of compound events in this experiment.
Definition 2.2 The sample space of an experiment is the set of all possible (a) The eight possible outcomes that comprise the sample space are:
outcomes of that experiment. It is denoted by S. RRR, RRL, RLR, LRR, LLR, LRL, RLL, LLL
Unit 2 3 Unit 2 4
Definition 2.4 A simple event is an event that cannot be decomposed. Each 2.3.2 Complementary Events
simple event corresponds to one and only one sample point.
Definition 2.5 The complement of an event A, denoted by A/, is the set of all
The sample space can thus also be defined as the set of all sample points. outcomes in S that are not contained in A. The null event is defined as the
complement of the sample space, i.e. / = S.
ACTIVITY 2.1
1. Suppose two balanced coins are tossed and the upper faces observed. Example 2.2
(a) List the sample points for the experiment. A fair coin is tossed twice. Define the events A and B as follows:
(b) Let A denote the event that exactly one head is observed and B the event that at A: at least one head is observed;
least head is observed. List the sample points in A and B. B: the number of heads observed is odd.
Find
2. Consider a production process that is known to produce defective parts a very small (a) A B,
rate. The process is monitored by testing randomly selected parts during the (b) A B,
production process. Suppose that as soon as a (c) A/.
defective part is found, the process is stopped and all machine settings are checked.
We are interested in studying the number of parts that are tested in order to obtain the Solution
first defective part. Let n be the outcome a part is non-defective and d be the
outcome a part is defective. Write down the sample space S for this experiment. S = {HH, HT, TH, TT},
A = {HH, HT, TH},
B = {HT, TH}.
2.3 RELATING THE EVENTS Then
(a) A B = {HH, HT, TH},
2.3.1 Unions and Intersections (b) A B = {HT, TH},
(c) A/ = {TT}.
The union of two events A and B, denoted by A B, is the event consisting of all
outcomes, that either belong to A or B or both.
Unit 2 5 Unit 2 6
2.3.6 Results From Set Theory
Definition 2.6 Events A and B are said to be mutually exclusive (or disjoint) if
A B = . The following basic laws concerning events carry over directly from set theory.
In other words, events are mutually exclusive if one event occurs, the other cannot occur. (i) Commutative laws
For example, if a battery is defective then it cannot be non-defective; thus the events A B = B A, A B = B A.
defective and non-defective are mutually exclusive.
(ii) Associative laws
2.3.4 Exhaustive Events A (B C) = (A B) C = A B C.
A (B C) = (A B) C = A B C.
Two events are said to be exhaustive if it is certain that at least one of them will occur.
(iii) Distributive laws
For instance, in the experiment of rolling a die it is certain that the outcome will be an
(iv) The Partition Principle
even number or an odd number. The two events outcome is an even number and
This is one of the most important relationships between events and plays a crucial role in
outcome is an odd number are therefore exhaustive.
many of the developments later:
Unit 2 7 Unit 2 8
The above formula can easily be verified by using the Venn Diagram below 2. Suppose a family contains two children of different ages, and we are interested in the
gender of these children. Let F denote that a child is female and M that the child is
male, and let a pair such as FM denote that the older child is female and the younger
A B/ male.
A B A B
Unit 2 9 Unit 2 10
Definition 2.9 (Axiomatic definition of probability)
Definition 2.8 (Classical definition of probability) If the sample space S Axiom 1 For any event A, Pr{A} 0 .
contains N equally likely sample points, of which nA are favourable to the event A Axiom 2 Pr{S } = 1 , where S is the sample space.
occurring, then the probability of the event A is defined by Axiom 3 If A1, A2, form a sequence of pairwise mutually exclusive
Pr {A} = A .
n
N events, then Pr{A1 A2 ...} = Pr{Ai } .
i =1
Axiom 1 places on the real axis the lower end of the scale by which we will compare the
Example 2.3
relative sizes of probabilities- they are to be non-negative. Axiom 2 states that the sample
Suppose two dice are rolled. Define A to be the event
space S is a certain event. It thus places an upper bound on the probability of any event:
A: the sum of the outcomes is 6.
no probability can exceed unity. The final axiom tells us how to add up probabilities for
disjoint events.
Calculate Pr{A} .
Note that the axiomatic definition of probability states only the properties that a
probability must satisfy: it does not tell us how to assign specific probabilities to
Solution
events. This task is taken up in Sec. 2.6.
If we let (x, y) denote the outcomes on the first and second die respectively, then
Consider again the experiment in which a single fair die is rolled. Verify that the 3
A = {(1,5), (2,4 ), (3,3), (4,2 ), (5,1)}.
n A = 5. axioms of probability are satisfied.
The student should note that other approaches to probability exist, such as the relative Since the outcomes 1 to 6 are mutually exclusive, equally likely and exhaustive events.
frequency approach and the subjective approach. However all these definitions of 1
Pr{Ei } = , i = 1,2,..,6 .
6
probability can be derived as special cases of the axiomatic definition of probability.
Unit 2 11 Unit 2 12
Note that
Finally, since
{ } Pr{E } = 1,
6
Pr{S } = Pr U 6i =1 Ei = i
i =1 Example 2.5
The distribution of blood types in the United States is roughly 41% type A, 9% type B,
Axiom 3 is also satisfied. 4% type AB, and 46% type O. An individual is brought into an emergency room and is to
be blood-typed. What is the probability that the type will be A, B, or AB?
ACTIVITY 2.3
THEOREM 2.2 1. A chemist analyses water samples for two heavy metals: lead and mercury. Past
For any A S, the probability of the complement A/ is given by experience indicates that 38% of the samples taken near the mouth of a river on which
{ }
Pr A / = 1 Pr{A}. numerous industrial plants are located contain toxic levels of lead or mercury; 32%
Unit 2 13 Unit 2 14
contain toxic levels of lead and 16% contain toxic levels of mercury. What is the
Pr{A B}
probability that a randomly selected sample contains toxic levels of lead only? Pr{A | B} = , Pr{B} > 0.
P{B}
2. A smoked detector uses two devices, A and B. If smoke is present, the probability
(2.3)
that it will be detected by device A is .95; by device B, .90; and by both devices .88. Find
Similarly, conditional probability of B given A is defined by
the probability that the smoke will be undetected.
Pr{B A}
Pr{B | A} = , Pr{A} > 0.
P{A}
3. Three radar sets, operating independently, are set to detect any aircraft flying
through a certain area. Each set has a probability of .02 of failing to detect a plane in its Viewed differently, the equation for the definition of conditional probability gives the
area. multiplication rule:
(a) If an aircraft happens into the area, what is the probability that it goes undetected?
(b) If an aircraft happens into the area, what is the probability that it is detected by all Pr{A B} = Pr{A | B}Pr{B} = Pr{B | A}Pr{A} .
three radar sets?
(2.4)
The multiplication rule can be generalised in an obvious way to more than two events: for
4. By using the axiomatic definition of probability, prove that
events A1, A2, , An, in a sample space S, if Pr{A1 A2 A3 ... An1 } > 0,
if A B then Pr{A} Pr{B}.
Pr{A1 A2 A3 ... An } = Pr{A1 }Pr{A2 | A1 }Pr{A3 | A1 A2 }... Pr{An | A1 A2 ... An 1 } (2.5)
Unit 2 15 Unit 2 16
Solution Example 2.7
th
For i = 1, 2, 3, let S i denote the event that the i person sampled is a smoker; so that Males and females are observed to react differently to a given set of circumstances. It
__ has been observed that 70% of the females react positively to these circumstances,
S i denotes the event that the ith person sampled is a non-smoker.
whereas only 40% of males react positively. A group of 20 people, fifteen female and
(a) We have
five male, was subjected to these circumstances, and the subjects were asked to
__
__ __
Pr S1 S 2 S 3 = Pr{S1 }Pr S 2 | S1 Pr S 3 | S1 S 2 describe their reactions on the question. A response is picked at random from the
twenty. What is the probability that it is negative?
10 40 9
= = .031.
50 49 48
(b) The two methods of selection are the same and there is no difference. Solution
We first define the events
M : subject is male,
To see how the multiplication rule is implemented in a useful way, we return to the P : response is positive
partition principle of Sec 2.3.6. Suppose that A and B are events in the sample space From the information given, we have
S. From the partition principle and by using Theorem 2.1, we have Pr{M } =
5 1
= ,
20 4
{ }
Pr P | M / = .70, Pr{P | M } = .40.
__
Pr{A} = Pr{A B} + Pr A B .
__ From the Law of
since the two events A B and A B are disjoint. Now by using the multiplication
{
Total Probability we Pr {P} = Pr {P | M }Pr {M } + Pr P | M / Pr M / }{ }
rule, we finally obtain the very useful result: have 1 3
= (.40 ) + (.70 )
4 4
= .625 .
__ __
Pr{A} = Pr{A | B}Pr{B} + Pr A | B Pr B
(2.6)
The above can easily be generalised: if B1 , B2 ,..., Bn is a sequence of mutually
2.5.2 Independent Events
exclusive and exhaustive events in S (i.e. B1 , B2 ,..., Bn partition S), then
n
An event A is said to be independent (or statistically independent) of event B if the
Pr{A} = Pr{A | B }Pr{B } .
i =1
i i occurrence or non-occurrence of event B does not affect the occurrence of event A.
Symbolically,
for any event A in S. This is also known as the Law of Total Probability. (2.8)
Now if A is independent of B, Eq.(2.4) gives
Unit 2 17 Unit 2 18
Pr{A}Pr{B} = Pr{B | A}Pr{A} .
3
only if Pr{A B} = Pr{A}Pr{B} . Pr{M F S } = Pr{M }Pr{F }Pr{S } = (.90) = .729.
(2.10) Thus the probability that each will operate at the time of the launch is .729.
The concept can easily be generalised to more than two events:
(b) The second probability is
3
Definition 2.11 The events A1, A2, , An are said to be pairwise independent if 1 Pr{M F S } = 1 (.90 ) = .271 .
{ } { }
Pr Ai A j = Pr{Ai }Pr A j i, j = 1,2,..., n; i j .
If we apply the Law of Total Probability (Eq. 2.7) to Pr{A} in the above we obtain
(a) What is the probability that each will be operable at the time of the launch?
Bayes Theorem:
(b) What is the probability that at least one will not be operable at the time of the
launch?
Unit 2 19 Unit 2 20
Pr{A | Bi }Pr{Bi } Pr{P | V }Pr{V }
Pr{Bi | A} = Pr{V | P} =
Pr{A | B }Pr{B }
n
, i = 1,2,..., n.
{ }{ }
Pr{P | V }Pr{V } + Pr P | V / Pr V /
,
(.99)(.01)
j j
j =1
=
(.99)(.01) + (1 .98)(1 .01)
= .333 (3 d.p.).
(2.11)
The probabilities Pr{A | Bi } are typically known prior to the conduct of the experiment
and are commonly known as transition probabilities. Note also that B1 , B2 ,..., Bn form a
partition of the sample space S.
ACTIVITY 2.4
1. If two events, A and B, are such that Pr{A} = .5, Pr{B} = .3 and Pr {AB} = .1, find
Example 2.9
the following:
(a) Pr{A | B};
Suppose that a proportion of 1% of a population is known to carry the HIV virus. Test
(b) Pr{B | A};
results over a period of time have shown that, of people carrying the virus, the test
(c) Pr{A | A B};
correctly returns a positive result 99% of the time. Of people who are not carrying the
(d) Pr{A | AB};
virus, the test correctly returns a negative result 98% of the time. Suppose an individual
(e) Pr{AB | A B}.
from this population gives a positive result when tested. What is the probability that he is
really carrying the virus?
2. A missile can be accidentally launched if two relays A and B both have failed. The
probabilities of A and B failing are known to be .01 and .03 respectively. It is also known
Solution
that B is more likely to fail (probability .06) if A has failed.
Define the events
(a) What is the probability of an accidental missile launch?
V : individual really carries the virus
(b) What is the probability that A will fail if B has failed?
P : individual tests postive
(c) Are the events A fails and B fails statistically independent?
Then, from the information given,
4. A ship can successfully arrive at its destination if its engine and its satellite
navigation system do not fail en route. If the engine and navigation system are known to
Unit 2 21 Unit 2 22
fail independently with respective probabilities .05 and .001, what is the probability of a to list them all in the form of sample space and the required probabilities can be
successful arrival? calculated by counting the number of favourable cases out of the total (Definition 2.8):
__
nA
5. In the sample space S, show that if A and B are independent then A and B are also Pr{A} = .
N
independent
However, there are experiments which result in large N and it is tedious to form a
complete list of all the outcomes. In such cases, it is possible to calculate probabilities by
6. Let A1, A2 and A3 be three events in the sample space S. Then, for example, the
exploiting some counting rules which do not require the complete listing of outcomes.
set-theoretic expression for the event that of A1, A2 and A3, only A1 occurs is A1 A2/
These rules are also useful in the problems where the outcomes are not equally likely.
A3 /.
(a) Give a set-theoretic expression for the event that at most one of the events A1, A2
2.5.1 The Multiplication Rule
and A3 occurs?
(b) Is it true that A1 A2 A3 = S? Explain briefly.
Many complicated experiments can be broken down into several stages of
experimentation. When dealing with such situations, the multiplication rule comes in
7. Two methods, A and B, are available for teaching a certain industrial skill. The
handy.
failure rate is 20% for A and 10% for B. However, B is more expensive and hence is used
only 30% of the time (A is used the other 70%). A worker was taught the skill by one of
Definition 2.12 (Multiplication Rule) In a multistage experiment with k stages,
the methods but failed to learn it correctly. What is the probability that she was taught by
if there are n1 outcomes at stage 1, n2 outcomes at stage 2, , nk outcomes at
method A?
stage k, then the whole experiment has n1 .n2 ...nk possible outcomes.
8. Assume that k1 urns contain w1 white and b1 black balls each and that k2 urns
contain w2 white and b2 black balls each. A ball drawn from a random selected urn turns
out to be white. Show that the probability that the ball drawn from an urn of the first type
is
1
k 2 w 2 (w1 + b1 )
1 + . Example 2.8
k1 w1 (w2 + b2 )
Consider an experiment that consists of recording the birthdays of twenty randomly
selected persons. If we ignore leap years and assume that there are only 365 possible
distinct birthdays,
2.6 COUNTING TECHNIQUES (a) find the number of points in the sample space S for this experiment;
(b) calculate the probability that each person in the twenty have a different birthday.
When the various outcomes of an experiment are equally likely, then the computation of
probabilities involve counting. If the total number of outcomes N is small, it is quite easy
Unit 2 23 Unit 2 24
Solution
(a) The first person can have 365 possible birthdays and this applies to each of the Suppose four brands of electric bulbs (W, X, Y, Z) are to be arranged in a line. The
remaining nineteen persons. By the multiplication rule, we therefore have a total possible arrangements are:
number of sample points
64420 447terms
4444 8 WXYZ WXZY WYXZ WYZX WZXY WZYX
20
N = 365 365 365... 365 = (365)
XWYX XWZY XYWZ XYZW XZWY
XZYW
(b) If we denote the event that each person has a different birthday by A, then we can YWXZ YWZX YXWZ YXZW YZWX YZXW
determine the number of sample points nA in A by nothing that, for all 20 persons ZWXY ZWYX ZXWY ZXYW ZYWX ZYXW
to have different birthdays, the first person can have any 365 birthdays, the
second person any 364 birthdays, the third person any 363 birthdays, and so on. Note that there are 4 candidates (W, X, Y, Z) to fill the first place. Once the first place is
By again appealing to the multiplication rule, we have filled, then we are left with 3 candidates any of whom can fill the second place. Once the
n A = 365 364 363 ... 346 . second place is also filled, then we have 2 candidates to fill the third place. The final
Hence, candidate fills the fourth place. By the multiplication rule, the total number of 4-
Note that the above result comes as a surprise, because it implies that there quite a large and is therefore usually denoted by n! (n factorial). Some examples of factorials
high probability (.4114) that at least two persons will have the same birthday. include:
0! = 1,
1! = 1,
2! = (2)(1) = 2,
3! = (3)(2)(1) = 6,
2.6.2 Permutations
.
.
Any arrangement of n distinct objects such that the order of arrangement is important is
.
called an n-permutation of the n objects. If r (< n) objects are selected to be arranged,
n! = n (n-1)!,
then the resulting arrangement is called an r-permutation of the n objects. The total
number of ordered arrangements or permutations in this case can again be calculated by
the application of the multiplication rule (see Section 2.7.1)
Unit 2 25 Unit 2 26
n
n
n! 2 n for large n .
e 2.6.3 Combinations
The last formula is known as Stirlings formula and can be used for approximating n!
when n is large. Unlike permutations, combinations consider only the possible set of objects, regardless of
the order in which the objects are chosen (without repetition, of course) or are arranged.
Thus, with the four brands of electric bulbs in Sec. 2.6.2, the 24 different permutations
THEOREM 2.5 form a single combination. When we select r objects of n distinct objects, the number of
(a) The total number of n-permutations of n distinct objects is n! permutations is n Pr . Now, if we consider r particular objects, these can be permuted in r!
n
(b) The total number of r-permutations of n distinct objects (r < n) is Pr , where
ways which are equivalent to 1 unordered arrangement or combination. Thus for the n Pr
n!
n
Pr = n(n 1)(n 2)...(3)(2 )(1) = . permutations we should have only n Pr /r! combinations.
(n r )!
THEOREM 2.6
The number of ways of selecting r distinct objects from n distinct objects (n > r) or r-
combinations of n distinct objects is
Example 2.9
Opening a combination lock requires the selection of the correct set of four different
n
Pr n(n 1)(n 2)...(3)(2)(1) n!
n
Cr = = = .
digits in sequence. The digits are set by rotating the tumbler in alternating clockwise and
r! r! (n r )!r!
anticlockwise directions. Assume that no digit is used twice. Give the total number of
combinations.
Definition 2.13
n n n!
Solution = C r = .
The total number of lock combinations is equal to the number of ways of arranging r = 4 r (n r )!r!
out of the possible n =10 digits such that the order of arrangement is important, i.e. the (2.12)
number of 4-permutations of the 10 digits is:
10 P = 10! = (10 )(9 )(8 )(7 ) = 5040 . n
so that is the number of r-combinations of n distinct objects.
4 6! r
n
is read n choose r and is called a binomial coefficient, because it is the coefficient
r
of the powers of x of the binomial expansion:
Unit 2 27 Unit 2 28
N = no. of sample points in the sample space S
(1 + x )n = x r ,
n n
n .
r
r =0
= no. of ways to select 3 engine blocks from 20
Whereas the binomial coefficients relate to the partitioning of n objects into two groups
20
(one consisting of r objects, the other consisting of n r objects), the multinomial = = 1140
3
coefficients
n n!
= . ACTIVITY 2.5
r1 , r2 ,..., rk r1!r2 !...rk !
1. An airline has six flights from New York to California and seven flights from
California to Hawaii per day. If the flights are to be made on separate days, how many
different flight arrangements can the airline offer from New York to Hawaii?
Example 2.10
2. An assembly operation in a manufacturing plant requires three steps that can be
A foundry ships a lot of 20 engine blocks of which five contain internal flaws. The
performed in any sequences. How many different ways can the assembly be performed?
purchaser will select three blocks at random and test them for hardness. The lot will be
accepted if no flaws are found. What is the probability that this lot will be accepted?
3. A brand of automobile comes in five different styles, with four types of engines,
with two types of transmissions, in eight colours.
Solution
(a) How many autos would a dealer have to stock if he included one of each style-
Let A denote the event the lot is accepted. Then,
engine-transmission combination?
Unit 2 29 Unit 2 30
(b) How many would a distribution centre have to carry if all colours of cars were
stocked for each combination in (a) above? ACTIVITY 2.4
1. (a) 1/3, (b) 1/5, (c) 5/7, (d) 1, (e) 1/7.
4. Four students are to be randomly selected to fill certain student government posts, 2. (a) 0.0006, (b) 0.02, (c) No.
from a group of three undergraduate and five graduate students. Find the probability that 3. 0.999
exactly two undergraduates will be among the four chosen? 4. 0.94905
7. 14/17
5. A manufacturer has nine distinct motors in stock, two of which came from a
particular supplier. The motors must be divided among three production lines, with three ACTIVITY 2.5
motors going to each line. If the assignment of motors to lines is random, find the 1. 42
probability that both motors from the particular supplier are assigned to the first line. 2. 6
3. (a)40, (b)320.
6. A thief has a bunch of n keys, exactly one of which fits a lock. If the thief tries to 4. 3/7
open the lock by trying the keys at random, show that the probability the he requires 5. 1/12
exactly k attempts if he rejects the keys already tried is 1/n.
2.Given that p1 = Pr{A}, p 2 = Pr{B} and p 3 = Pr{A B} , express the following in terms of
2.8 ANSWERS TO ACTIVITIES p1 , p 2 , and p3 : (i) Pr{A B }, (ii) Pr{A B}, (iii) Pr{A B}.
ACTIVITY 2.3
[Ans: (i) 1-p3, (ii) p2-p3, (iii) 1-p1+p3.]
1. 0.22
2. 0.03
3. Use the binomial expansion
3. (a) 8 10 6 , (b) .9412
Unit 2 31 Unit 2 32