Introduction to Probability and Statistics

Probability & Statistics for Engineers & Scientists, 8th Ed.

2007
Handout #1
Instructor: Kuo-Jung Lee
TA: Brian Shea

The pdf file for this class is available on the class web page.
http://www.stat.umn.edu/~kjlee/STAT3021_Summer2009.html
1

An Overview of Statistics

Whats Statistics?
Statistics is a way to get information from data

Statistics is a discipline which is concerned with:

summarizing information to aid understanding,
drawing conclusions from data,
estimating the present or predicting the future, and
designing experiments and other data collection.
In making predictions, Statistics uses the companion subject of
Probability, which models chance mathematically and enables
calculations of chance in complicated cases.
4

Today, statistics has become an important tool in the work of

many academic disciplines such as medicine, psychology, education, sociology, engineering and physics, just to name a few.
Statistics is also important in many aspects of society such as
business, industry and government. Because of the increasing
use of statistics in so many areas of our lives, it has become very
desirable to understand and practise statistical thinking. This is
important even if you do not use statistical methods directly.

Data
Data consists of information coming from observation, counts,
measurements, or responses.
Statistics
Statistics is the science of collecting, organizing, analyzing, and
interpreting data in order to make decisions.

Population
A population is the collection of all outcomes, responses, measurements, or counts that are of interest.
Sample
A sample is a subset of a population.

Parameter
A parameter is numerical description of a population characteristics.
Statistic
A statistic is numerical description of a sample characteristic.

Branches of Statistics
Descriptive statistics is the branch of statistics that involves
the organization, summarization, and display of data.
Inferential statistics is the branch of statistics that involves
using a sample to draw conclusions about a population. A basic
tool in the study of inferential statistics is probability.

Example
A large sample of men, aged 48, was studied for 18 years. For
unmarried men, 60% to 70% were alive at age 65. For married
men, 90% were alive at age 65. Which part of the study represents the descriptive branch statistics? What conclusions might
be drawn from this study using inferential statistics?

10

Solution:

Descriptive statistics: For unmarried men, 60% to 70% were

alive at age 65. For married men, 90% were alive at age 65.

A possible inference: Being married is associated with a longer

life for men.

11

Example
An instructor is teaching two separate classes, A and B each
of size is 50. After a midterm, the scores for each class are:

A: 50 47 59 49 72 41 63 79 91 65 49 59 92 42 34 43 53 89 76
93 89 51 42 46 67 48 33 47 68 51 56 53 69 53 43 36 58 85
45 64 57 32 1 60 66 60 63 86 62 55

B: 56 61 53 59 60 55 57 49 67 60 58 56 58 59 55 52 60 68 45
59 67 62 42 50 53 63 61 61 57 70 49 64 52 58 58 70 48 66
58 58 61 58 68 58 54 60 61 61 61 72
12

Histogram of Scores for Class B

10

Frequency

2
0

Frequency

10

15

12

Histogram of Scores for Class A

20

40

60
Score

80

100

40

45

50

55

60

65

70

75

Score

13

Class
A
B

1st Q
47.00
55.25

Median
56.50
58.50

3rd Q
66.75
61.00

Mean
58.34
58.56

Standard Deviation
18.24
6.29

14

Chapter 2 Probability

15

 Sample Space / Events

Counting Sample Points
Probability of an Event
Additive Rules
Conditional Probability
Multiplicative Rules
Bayes Rule
16

2.1 Sample Space

17

Experiment
Experiment is any process that generates a set of data.
Sample space
Sample space is the collection of all possible outcomes at a
probability experiment. We use the notation S for sample space.

18

Example 1
Toss a coin. The possible outcomes are heads, tails. So the
sample space is S= {Heads, Tails}.

19

Example 2
Roll a dice. The possible outcomes are the six faces, numbered
1, 2, 3, 4, 5, 6. Hence the sample space is S= {1, 2, 3, 4, 5,
6}.

20

Sample Points
Each outcome in a sample space is called an element or a member of the sample space, or simply a sample point.
Tree Diagram
In some experiments, it is helpful to list the elements of the
sample space systematical by means of a tree diagram.

21

Example 3
An experiment consists of flipping a coin and then flipping it a
second time if head occurs. If a tail occurs on the first flip, then
a die is tossed once. The sample space
S = {HH, HT, T 1, T 2, T 3, T 4, T 5, T 6}

22

Statement & Rule

Sample spaces with a large or infinite number of sample points
are best described by a statement or rule.
Example 4: Statement
The possible outcomes of an experiment are the set of cities in
the world with a population over 1 million, our sample space is
written S = {x|x is a city with a population over 1 million}.
Example 5: Rule
If S is the set of all points (x, y) on the boundary or the interior
of a circle of radius 2 with center at the origin, we write the rule:
S = {(x, y)|x2 + y 2 22}
23

2.2 Events

24

Event
Event is a sub-collection of outcomes from the sample space.
We are interested in probabilities of events.
Example 6
In the experiment of tossing a die, consider the event E that the
outcome when a die is tossed is divisible by 3.
S = {1, 2, 3, 4, 5, 6}
E = {3, 6}

25

Example 7
Given the sample space S = {t|t 0}, where t is the life in years
of a certain electronic component, then the event A that the
component fails before the end of the third year is the subset
A = {t|0 t < 3}

26

Null Set
A set contains no elements at all, and denoted by the symbol .
Example 8
Roll a die. One event that you may be interested in is E1 =
{You get an even number} = {2, 4, 6}. Another one could be
E2 = {You get a prime number} = {2, 3, 5}. Yet another one
could be E3 = {You get a multiple of 7} = .
E1 = {You get an even number} = {2, 4, 6},
E2 = {You get a prime number} = {2, 3, 5},
E3 = {You get a multiple of 7} = .
27

Complement
The complement of event A with respect to S is the subset of
all elements of S that are not in A. We denote the complement
of A by the symbol either A0 or Ac

28

Intersection
The intersection of two events A and B, denoted by the symbol
A B, is the event containing all elements that are common to
A and B.

29

Union
The union of the two events A and B, denoted by the symbol
A B, is the event containing all the elements that belong to A
or B or both.

30

Example 8 Contd
Roll a die. One event that you may be interested in is E1 =
{You get an even number} = {2, 4, 6}. Another one could be
E2 = {You get a prime number} = {2, 3, 5}. Yet another one
could be E3 = {You get a multiple of 7} = .
S = {1, 2, 3, 4, 5, 6},
E1 = {You get an even number} = {2, 4, 6},
E2 = {You get a prime number} = {2, 3, 5},
E3 = {You get a multiple of 7} = .
E10 = {1, 3, 5},
E1 E2 = {2},

E20 = {1, 4, 6},

E30 = S

E1 E2 = {2, 3, 4, 5, 6}

31

Mutually Exclusive or Disjoint

Two events A and B are mutually exclusive, or disjoint, if
A B = , that is, if A and B have no elements in common.
Example 9
Roll a die. One event that you may be interested in is E1 = You
get an even number = {2, 4, 6}. Another one could be E2 =
You get a odd number = {1, 3, 5}. Then E1 E2 = . E1 and
E2 are disjoint.

32

Venn Diagrams
The relationship between events and the corresponding sample
space can be illustrated graphically by means of Venn diagrams.

33

 A B = regions 4 and 7.
B C = regions 6 and 7.
A C = regions 1, 3, 4,
5, 6 and 7.
B 0 A = regions 1 and
5.
A B C = regions 7.
(A B) C 0 = regions 1,
2 and 4.

34

Example 10
Suppose that in a senior college class of 600 students it is found
that 230 smoke, 250 drink alcoholic beverages, 240 eat between
meals, 150 smoke and drink alcoholic beverages, 90 eat between
meals and drink alcoholic beverages, 100 smoke and eat between
meals, and 60 engage in all three of these bad health practices.

35

 A = Smoker.
B = Drinking alcoholic
beverages.
C = Eat between meals.
Smokes but does not
drink alcoholic beverage.
A B 0={1, 5}.
Eats
between
meals
and
drinks
alcoholic
beverages but does not
smoke.C B A0 =
{2, 3, 6}
Neither smokes nor eats
between meals.
(A
C)0 = A0 C 0 = {2}

Example 11
If S = {x|0 < x < 12}, A = {x|1 x < 9}, and B = {x|0 < x < 5},
find
(a)
(b)
(c)

A B ={x|0 < x < 9}

A B ={x|1 x < 5}
A0 B 0 ={x|0 < x < 1, 5 x < 12}

36

2.3 Counting Sample Points

37

GOLD
To count the the number of points in the sample space without
actually listing each element.
Multiplication Rule
If an operation can be performed in n1 ways, and if for each
of these ways a second operation can be performed in n2 ways,
then the two operations can performed together in n1 n2 ways.

38

Example 12
How many sample points are there in the sample space when a
pair of 6-sided dice is thrown once?

This first dice can land in any one of n1 = 6 ways. For each of
these 6 ways the second dice can also land in n2 ways. Therefore,
the pair of dice can land in
n1 n2 = 6 6 = 36.
39

GOLD
To count the the number of points in the sample space that
contains elements as all possible orders or arrangements of a
group of objects.
Permutation
A permutation is an arrangement of all or part of a set of
objects.

40

Example 13
Consider the three letter a, b, and c. The possible permutations
are abc, acb, bac, bca, cab, and cba. Thus we see that there are 6
distinct arrangements, because there n1 = 3 choices for the first
position, then n2 = 2 for the second, leaving only n3 = 1 choice
for the last position, giving a total of
n1 n2 n3 = 3 2 1 = 6

41

In general, n distinct objects can be arranged in

n(n 1)(n 2) (3)(2)(1) ways
We represent this product by the symbol n!, which is read n
factorial. We define 0! = 1.
Theorems
The number of permutations of n distinct objects is n!.

42

Example 14
Consider the 4 letter a, b c, and d. The number of permutation of
the 4 letters will be 4!. Now consider the number of permutations
that are possible by taking 2 letters at a time from four. The
possible permutations are ab, ac, ad, ba, bc, bd, ca, cb, cd da, db,
and dc. Thus we see that there are 12 distinct arrangements,
because there are n1 = 4 choices for the first position, then
n2 = 3 for the second,giving a total of
n1 n2 = 4 3 = 12

43

In general, n distinct objects taken r at a time can be arranged

in
n!
n(n 1)(n 2) (n r + 1) =
ways
(n r)!
Theorems
The number of permutations of n distinct objects when taken r
at a time is
n!
P
=
n r
(n r)!

44

Example 15
A president and a treasurer are to be chosen from a student club
consisting of 50 people. How many different choices of officer
are possible if

1. there are no restrictions;

2. A will serve only if he is president;

3. B and C will serve together or not at all;

4. D and E will not serve together?

45

Solution:

1. 50P2.

2. 49P1 +49 P2.

3. 2P2 +48 P2.

4. 50P2 2.

Example 16
Consider the three letter a, b, and c. The possible permutations
are abc, acb, bac, bca, cab, and cba. Thus we see that there are 6
distinct arrangements, because there n1 = 3 choices for the first
position, then n2 = 2 for the second, leaving only n3 = 1 choice
for the last position, giving a total of
n1 n2 n3 = 3 2 1 = 6
If the letters b and c are both equal, then the 6 permutation
of the letters a, b, and c become only 3 distinct permutations.
Therefore, with 3 letters, 2 begin the same, we have 3!
2! distinct
permutations.
46

Theorem
The number of distinct permutations of n things of which n1 are
of one kind, n2 of a second kind, . . . , nk of a kth kind is
n!
n1!n2! nk !

47

Example 17
How many distinct permutation can be made from the letters of

1. aabbcc;

Solution:
6!
2!2!2!

48

GOAL
To compute the number of partition of a group.
Partition
The number of ways of partitioning a set of n objects into r cells
with n1 elements in the first cell, n2 elements in the second, and
so forth, is
n
n1, n2, . . . , nr

n!
n1!n2! nr !

where n = n1 + n2 + + nr .
49

Example 18
Consider the set {A, B, C, D, E}. The possible partitions into
two cells in which the first cell contains 4 letters and the second
cell 1 element are
A, B, C, D E
A, B, C, E D
A, B, D, E C
A, C, D, E B
B, C, D, E A

50

Example 19
In how many ways can 7 graduate students be assigned to one
triple and two double hotel rooms during a conference?
Solution:
The total number of possible partitions would be
7!
=
3!2!2!

7
3, 2, 2

= 210.

51

In many problems we are interested in the number of ways of

selecting r objects from n without regard to order. The selection
are called combination. A combination is actually a partition
with two cells, the one cell containing the r objects selected and
the other cell containing the (n r) objects that are left. The
number of such combinations, denoted by
n
r, n r

, is usually shortened to

n
r

52

Combination
The number of combinations of n distinct objects taken r at a
time is

n
r

n!
=
(n r)!r!

53

Example 20
A young boy asks his mother to get five Game-BoyTM cartridges
from his collection of 10 arcade and 5 sports games. How many
ways are there that his mother will get 3 arcade and 2 sports
games, respectively?
Solution:
The number of ways selecting 3 cartridges from 10 is
10
3

10!
=
= 120
3!(10 3)!

The number of ways of selecting 2 cartridges from 5 is

5
2

5!
=
= 10
2!(5 2)!

Using the multiplication rule with n1 = 120 and n2 = 10, there

are 120 10 = 1200 ways.
54

2.4 Probability of an Event

55

Probability
Probability is a number associated to events, the number denoting the chance of that event occurring.

56

Properties of Probability
Probability is a set function P that assigns to each event A in
the sample space S a number P (A), called the probability of the
event A, such that the following properties are satisfied:
1. 0 P (A) 1.

2. P (S) = 1.

3. If Ai are mutually exclusive, then

P (A1 A2 ) =

P (Ai)

i=1
57

Theorem
If an experiment can result in any one of N different equally
likely outcomes, and if exactly n of these outcomes correspond
to event A, then the probability of event A is
n
P (A) = .
N
That is,
P (A) =

Number of outcomes favorable to A

.
Total number of outcomes for the experiment

58

Example 21
A coin is tossed twice. What is the probability that at least on
head occurs?
Solutions:
S = {HH, HT, T H, T T };
A = {HH, HT, T H}
3
P (A) =
4

59

Poker

60

Example 22
In a poker hand consisting of 5 cards, find the probability of
holding 2 aces and 3 jacks?
Solutions:
The number of ways of being dealt 2 aces from 4 is
4
2

4!
=
=6
2!(4 2)!

and the number of ways of being dealt 3 jacks from 4 is

4
3

4!
= 4.
3!(4 3)!

There are n = 6 4 = 24 hands with 2 aces and 3 jacks. The

total number of 5-card poker hands, all of which are equally
61

likely, is
N =

52
5

52!
=
= 2, 598, 960.
5!(52 5)!

Therefore, the probability of event C of getting 2 aces and 3

jacks in a 5-card poker hand is
P (C) =

24
.
2, 598, 960

2.5 Additive Rule

62

Additive Rule
If A and B are two events, then
P (A B) = P (A) + P (B) P (A B).

If they are mutually exclusive (disjoint), then

P (A B) = P (A) + P (B).
63

Example 23
What is the probability of getting a total of 7 or 11 when a pair
of fair dice are tossed?
Solutions:
Let A be the event that 7 occurs and B the event that 11 comes
up. Since events A and B are mutually exclusive, since a total
of 7 and 11 cannot both occur on the same toss. Therefore
1
2
1
= .
P (A B) = P (A) + P (B) = +
6
18
9

64

Theorem
For three events A, B, and C,
P (A B C) =P (A) + P (B) + P (C)
P (A B) P (A C) P (B C)
+P (A B C).
If A and A0 are complementary events, A A0 = and A A0 = S,
then
P (A) + P (A0) = 1.

65

2.6 Conditional Probability

66

The probability of an event B occurring when it is known that

some event A has occurred is called a conditional probability
and is denoted by P (B|A). The symbol P (B|A) is usually read
the probability that B occurs given that A occurs or simple
the probability of B, given A.
Conditional Probability
For any two events A and B with P (A) > 0, the conditional
probability of B given that A has occurred is:
P (A B)
P (B|A) =
.
P (A)

67

Example 24
Roll a dice. What is the chance that youd get a 6, given that
youve gotten an even number?
Solutions:
Let A be the event of even numbers, and B of 6.
A = {2, 4, 6},
B = {6},
A B = {6},
P (B|A) =

P (A) =

1
;
2

(1)

1
P (B) = ;
6

(2)

1
P (A B) = ;
3

(3)

P (A B)
1
=
P (A)
3
68

Example 25
In an experiment to study the relationship of hypertension and
smoking habits, the following data are collected for 200 individuals:
Nonsmokers
Hypertension
Non-hypertension

20
60

Moderate
smokers
40
30

Heavy
smokers
30
20

If one of these individuals is selected at random, find the probability that the person is

1. experiencing hypertension, given that the person is a heavy

smoker;
69

2. a nonsmoker, given that the person is experiencing no hypertension.

Solutions:

We shall concerned with the following events

90
200
110
= P (A0 )
experiences no hypertension, P (B) =
200
50
is a heavy somker, P (C) =
200
80
is a nonsmoker, P (D) =
200
experiences hypertension and is a heavy smoker

A =the person experiences hypertension, P (A) =

B =the person
C =the person
D =the person

A C =the person
30
P (A C) =
200
B D =the person experiences no hypertension and is a nonsmoker
60
P (B D) =
200

1. experiencing hypertension, given that the person is a heavy

smoker;
P (A C)
30/200
3
P (A|C) =
=
=
P (C)
50/200
5
2. a nonsmoker, given that the person is experiencing no hypertension.
6
60/200
P (D|B) =
=
110/200
11

Example 26
Tossing a 6-sided dice which is construed so that the even numbers are twice as likely to occur as the odd number.
1. Consider the event B of getting a perfect square when a dice
is tossed. P (B) =?
2. What is the probability of getting a perfect square when it is
know that the toss of the dice resulted in a number greater
than 3?
3. What is the probability of getting a perfect square when it is
know that the toss of the dice resulted in a number greater
than 4?
70

Solution:

3
1. B = {1, 4}, P (B) = 9

2. P (B|A) = 2
5 , A = {4, 5, 6}
3. P (B|A) = 0, C = {5, 6}

Conditional Probability
The conditional probability of B, given A, denoted by P (B|A) is
defined by
P (B|A) =

P (A B)
P (A)

provided

P (A) > 0.

Independent Event
Two events A and B are independent if and only if
P (B|A) = P (B)

or

P (A|B) = P (A)

provided the existences of the conditional probabilities. Otherwise, A and B are dependent.
71

Sampling With Replacement

Sampling with replacement occurs when an object is selected
and then replaced before the next object is selected.
Sampling Without Replacement
Sampling without replacement occurs when an object is not replaced after it has been selected.

72

Example 27
Consider an experiment in which 2 cards are drawn in succession
from an ordinary deck, with replacement. The events are defined
as
1. A: the first card is an ace,
2. B: the second card is a space.
Since the first card is replaced, our sample for both the first
and second draws consists of 52 cards, containing 4 aces and 13
spades. Hence
P (B|A) =

13
1
=
52
4

and

P (B) =

1
.
4
73

That is, P (B|A) = P (B). When this is true, the events A and B
are said to be independent.

2.7 Multiplicative Rules

74

Multiplicative Rule
If in an experiment the events A and B can both occur, then
P (A B) = P (A)P (B|A)

provided

P (A) > 0.

Theorem
Two events A and B are independent if and only if
P (A B) = P (A)P (B).
Therefore, to obtain the probability that two independent events
will both occur, we simply find the product of their individual
probabilities.
75

Example 28
Three cards are drawn in succession, without replacement, from
an ordinary deck of playing cards. Find the probability that the
event A1 A2 A3 occurs, where A1 is the event that the first
card is a red ace, A2 is the event that the second card is a 10
or a jack, and A3 is the event that the third card is greater tan
3 but less than 7.
Solution:

1. A1: the first card is a red ace,

2. A2: the second card is a 10 or a jack,

76

3. A3: the third card is greater than 3 but less than 7

8
2
, P (A2|A1) =
,
52
51
and hence, by Multiplicative Rule,
P (A1) =

P (A3|A1 A2) =

P (A1 A2 A3) = P (A1)P (A2|A1)P (A3|A1 A2) =

12
.
50

8
.
5525

2.8 Bayes Rule

77

Bayes Rule
The power of Bayes rule is that in many situations where we
want to compute P (A|B) it turns out that it is difficult to do
so directly, yet we might have direct information about P (B|A).
Bayes rule enables us to compute P (A|B) in terms of P (B|A).
P (A|B) =

P (A B)
P (B|A)P (A)
=
P (B)
P (B)

which is the so-called Bayes Rule.

78

Bayes Theorem
Let A and Ac constitute a partition of the sample space S such
that with P (A) > 0 and P (Ac) > 0, then for any event B in S
such that P (B) > 0,
P (A|B) =

P (B|A)P (A)
.
c
c
P (B|A)P (A) + P (B|A )P (A )

79

The denominator P (B) in the equation can be computed,

P (B) =P [(A B) (A0 B)]
=P (A B) + P (A0 B)
=P (A)P (B|A) + P (A0)P (B|A0)

(by multiplicative rule)

80

Example 29
A paint-store chain produces and sells latex and semigloss paint.
Based on long-range sales, the probability that a customer will
purchase latex paint is 0.75. Of those that purchase latex paint,
60% also purchase rollers. But only 30% of semigloss pain buyers
purchase rollers. A randomly selected buyer purchases a roller
and a can of paint. What is the probability that the paint is
latex?

81

Solution:
L ={The customer purchases latex paint.}, P (L) = 0.75
S ={The customer purchases semigloss paint.}, P (S) = 0.25
R ={The customer purchases roller.}
P (R|L) =0.6
P (R|S) =0.3
P (R) =P (R|L)P (L) + P (R|S)P (S) = 0.6 0.75 + 0.3 0.25 = 0.525
P (L|R) =

P (R|L)P (L)
0.6 0.75
P (L R)
=
=
.857
P (R)
P (R)
0.6 0.75 + 0.3 0.25

82

The denominator P (R) in the equation can be computed,

P (R) =P [(L R) (S R)]
=P (L R) + P (S R)
=P (L)P (R|L) + P (S)P (R|S)

(by multiplicative rule)

83

Example 30
Suppose that we are interested in diagnosing cancer in patients
who visit a chest clinic.

Let A represent the event Person has cancer

Let B represent the event Person is a smoker

We know the probability of the prior event P (A) = 0.1 on the

basis of past data (10% of patients entering the clinic turn out
to have cancer). We want to compute the probability of the
posterior event P (A|B). It is difficult to find this out directly.
84

However, we are likely to know P (B) by considering the percentage of patients who smoke, suppose P (B) = 0.5. We are also
likely to know P (B|A) by checking from our record the proportion
of smokers among those diagnosed. Suppose P (B|A)=0.8.
We can now use Bayes rule to compute:
P (A|B) =

P (B|A)P (A)
0.8 0.1
P (A B)
=
=
P (B)
P (B)
0.5

Bayes Theorem
Let A1, A2, . . ., Ak constitute a partition of the sample space S
such that P (Ai) > 0 for i = 1, 2, . . . , k, then for any event B in S
such that P (B) > 0,
P (B|Aj )P (Aj )

P (Aj |B) = Pk

i=1 P (B|Ai)P (Ai)

for i = 1, 2, . . . k.

85

Example 31
In a bolt factory 30%, 50%, and 20% of the production is manufactured by machines I, II, and III, respectively. If 4%, 5%,
and 3% of the outputs of these respective machines is defective, what is the probability that a randomly selected bolt that
is found to be defective is manufactured by machine III?

86

Solution:

P (B3|D) =

0.03 0.2
0.14
0.04 0.3 + 0.05 0.5 + 0.03 0.2

D is the event that a random bolt is defective.

Bi is the event that it is manufacture by i, where i = I, II, or III.