ReviewCh2-Basic Concept Probability

6/7/2011
1
Chapter Chapter 22 Basic Concepts of Basic Concepts of
241-460 I nt r oduct i on t o Queuei ng
Net wor ks : Engi neer i ng Appr oach
Chapter Chapter 2 2 Basic Concepts of Basic Concepts of
Probability Theory Probability Theory
Assoc. Prof. Thossaporn Kamolphiwong
Centre for Network Research (CNR)
Department of Computer Engineering, Faculty of Engineering
P i f S kl U i it Th il d Prince of Songkla University, Thailand
Email : kthossaporn@coe.psu.ac.th
Out l i ne
Random Experiments
Sample Space Sample Space
Type of sample space
Event
Set Theory
Chapter 2 : Basic Concepts of Probability Theory
2
6/7/2011
2
Speci f yi ng Random Exper i ment s
A random experiment is an experiment in
which the outcome varies in an which the outcome varies in an
unpredictable fashion when the
experiment is repeated under the same
condition
3
Exampl e of Random Exper i ment s
Experiment E1 : Toss a coin three times and
note the sequence of heads and tails
Experiment E2 : Toss a coin three times and
note the number of heads
Experiment E3 : Count the number of voice
packets containing only silence procedure from a
group of N speakers in a 10-ms period
E i t E4 A bl k f i f ti i Experiment E4 : A block of information is
transmitted repeatedly over a noisy channel until
an error-free block arrives at the receiver. Count
the number of transmissions required.
4
6/7/2011
3
( Cont i nue)
Experiment E5 : Measure the time between two
message arrives at a message center message arrives at a message center
Experiment E6 : Determine the value of a
voltage waveform at time t
1
and t
2
Experiment E7 : Pick a number at random
between zero and one.
Experiment E8 : Pick a number X at random Experiment E8 : Pick a number X at random
between zero and one, then pick a number Y at
random between zero and X
5
( Cont i nue)
Not e:
E pe iment 1 and E pe iment 2 a e the Experiment 1 and Experiment 2 are the
same procedure but they are different
observations di f f er ent Exper i ment s
6
6/7/2011
4
Set and Pr obabi l i t y
Out come or sampl e poi nt
any possible observation
, is an element or point in Sample Space
Sampl e Space S
The set of all possible outcomes
Set notation, tables, diagrams, intervals of the
real line regions of the plane real line, regions of the plane
Finite, Countably infinite, Uncountably infinite
7
Sampl e Space
Sampl e spaces cor r espondi ng t o t he
exper i ment exper i ment
S
1
= {HHH, HHT, HTH, THH, TTH, THT, HTT, TTT}
S
2
= {0, 1, 2, 3}
S
3
= {0, 1, 2, , N}
S
4
= {0, 1, 2, }
S
5
= {t : t > 0} = [0, )
S
6
= {(v
1
, v
2
) : - < v
1
< and - < v
2
< }
8
6/7/2011
5
( Cont i nue)
S
7
= {x : 0 < x < 1} = [0, 1]
S
S
8
= {(x,y) : 0 < y < x < 1 }
x
y
1
0 1
S
7
S
8
x
1
0
1
9
Type of Sampl e space
Type of Sample space
Discrete sample space if S is countable
Continuous sample space if S is not
countable
M lti di i l l ( Multi-dimensional sample space (one or
more observations or measurements)
10
6/7/2011
6
Event
Event
Set of outcome Set of outcome
Event c Sample Space
11
Event exampl es
Event exampl es
E
1
: The three tosses gives the same outcome
A
1
= {HHH, TTT}
E
2
: The three tosses gives # of head equals #
of tails
A
2
= {} = C
E
8
: The two numbers differ by less than 1/10
A
8
= {(x,y) : (x,y) e S
8
and |x y| < 1/10}
12
6/7/2011
7
Exampl e
Flip four coins, 0.5 baht(red), 1 baht(yellow), 5
baht(white), and 10 baht(silver). baht(white), and 10 baht(silver).
Examine the coins in order (0.5 baht, 1 baht, 5
baht, 10 baht) and observe whether each coin
shows a head (h) or a tail (t).
What is the sample space?
How many elements are in the sample space? How many elements are in the sample space?
Let B
i
= {outcomes with i heads}
13
Sol ut i on
Out come
(h h h h ) (h h h t ) (h h t h ) (h h t t ) (h t h h ) (h
r
h
y
h
w
h
s
), (h
r
h
y
h
w
t
s
), (h
r
h
y
t
w
h
s
), (h
r
h
y
t
w
t
s
), (h
r
t
y
h
w
h
s
),
(h
r
t
y
h
w
t
s
), (h
r
t
y
t
w
h
s
), (h
r
t
y
t
w
t
s
), (t
r
h
y
h
w
h
s
), (t
r
h
y
h
w
t
s
),
(t
r
h
y
t
w
h
s
), (t
r
h
y
t
w
t
s
), (t
r
t
y
h
w
h
s
), (t
r
t
y
h
w
t
s
), (t
r
t
y
t
w
h
s
),
(t
r
t
y
t
w
t
s
)
14
6/7/2011
8
( Cont i nue)
There are 16 members of the sample space
S = { (h
r
h
y
h
w
h
s
), (h
r
h
y
h
w
t
s
), (h
r
h
y
t
w
h
s
), (h
r
h
y
t
w
t
s
),
(h
r
t
y
h
w
h
s
), (h
r
t
y
h
w
t
s
), (h
r
t
y
t
w
h
s
), (h
r
t
y
t
w
t
s
),
(t
r
h
y
h
w
h
s
), (t
r
h
y
h
w
t
s
), (t
r
h
y
t
w
h
s
), (t
r
h
y
t
w
t
s
),
(t
r
t
y
h
w
h
s
), (t
r
t
y
h
w
t
s
), (t
r
t
y
t
w
h
s
), (t
r
t
y
t
w
t
s
) }
15
( Cont i nue)
Event B
0
= { (t
r
t
y
t
w
t
s
) }
Event B = { (t t t h ) (t t h t ) (t h t t ) (h t t t ) } Event B
1
= { (t
r
t
y
t
w
h
s
),(t
r
t
y
h
w
t
s
), (t
r
h
y
t
w
t
s
), (h
r
t
y
t
w
t
s
) }
Event B
2
= { (h
r
h
y
t
w
t
s
), (h
r
t
y
h
w
t
s
), (h
r
t
y
t
w
h
s
), (t
r
t
y
h
w
h
s
),
(t
r
h
y
t
w
h
s
), (t
r
h
y
t
w
h
s
)}
Event B
3
= {(h
r
h
y
h
w
t
s
),(h
r
h
y
t
w
h
s
),(h
r
t
y
h
w
h
s
), (t
r
h
y
h
w
h
s
)
}
Event B
4
= { (h
r
h
y
h
w
h
s
) }
Event B = {B
0
, B
1
, B
2
, B
3
, B
4
} is Event space
16
6/7/2011
9
Event Space
Event Space
Collectively exhaustive and Mutually exclusive Collectively exhaustive and Mutually exclusive
Col l ect i ve exhaust i ve
S = {1,2,3,4,5,6}
A = {2,4,6}
B {1 3 5} B = {1,3,5}
Mut ual l y excl usi ve
17
Appl yi ng Set Theor y t o Pr obabi l i t y
Probability is a number that describes a
set set
Set Algebra Probability
Set Event
Universal set Sample space
Element Outcome
18
6/7/2011
10
Set Theor y
The mathematical basis of probability is the theory
of sets of sets
Set is a collection of things
Things that together make up the set are
element
A {
2
| 1 2 3 4 5 } A = { x
2
|x = 1, 2, 3, 4, 5 }
= {1, 4, 9, 16, 25}
19
Set Oper at i on
Uni on
A B : the set of outcomes either in A or in B or
both
I nt er sect i on
A B : the set of outcomes in both A and B
If A B = , then A and B are mutually exclusive
Compl ement
A ' = The set of all outcomes not in A
Di f f er ence
A - B contains all elements of A that are not elements
of B
20
6/7/2011
11
A
Venn di agr am: Set Oper at i on
A B
S
S
A B
A
A
B
A
S
A
A B = C
S
A
B
A
A'
21
( Cont i nue)
B
S
A - B
S
A c B
A
B
B
A
22
6/7/2011
12
Mut ual l y excl usi ve
Mutually exclusive
A
S
A
i
A
j
= C for i = j
A B = C Disjoint
Collectively exhaustive
B
A
8
S
A
5
Collectively exhaustive
A
1
A
2
A
n
= S
A
1
A
7
A
6
A
2
A
4
A
1
A
3
23
Pr oper t i es of set oper at i on
Commutative Properties:
A B = B A and A B = B A A B B A and A B B A
Associative Properties:
A (B C) = (A B) C and
A (B C) = (A B) C
Distributive Properties:
A (B C) = (A B) (A C) and A (B C) (A B) (A C) and
A (B C) = (A B) (A C)
De Morgans law:
(A B)' = A' B' and (A B)' = A' B'
24
6/7/2011
13
( Cont i nue)
Union and intersection operation can be
repeated repeated
n
n
k
k
A A A A A
3 2 1
1
=
=
n
n
k
k
A A A A A
3 2 1
1
=
k 1 =
25
Axi oms of Pr obabi l i t y
Axiom 1 : For any event A, P[A] 0
Axiom 2 : P[S] = 1
Axiom 3 : If A B = C, then P[A B] = P[A] + P[B]
Axiom 4 : If A
1
, A
2
, is a sequence of events such that
C f ll h A
i
A
j
, = C for all i = j, then
| | | | | | ...
2 1
1 1
+ + = =
(
=
A P A P A P A P
k
k
k
k
26
6/7/2011
14
Some Consequences of The
Axi oms
The probability measure P[-] satisfies
Corollary 1 : P[A'] = 1 P[A] Corollary 1 : P[A ] 1 P[A]
Corollary 2 : P[A] < 1
Corollary 3 : P[C] = 0
Corollary 4 : If A
1
, A
2
, , A
n
are pairwise mutually
exclusive, then
27
( Cont i nue)
Corollary 5 : P[A B] = P[A] + P[B] P[A B]
A B
S
B
A
28
6/7/2011
15
( Cont i nue)
Corollary 6 :
29
( Cont i nue)
Corollary 7 : If A c B, then P[A] < P[B]
A' B
A
B
S
30
6/7/2011
16
Exampl e
A company has a model of telephone usage. It
classifies all calls as either long (L), if they last more
than three minutes or brief (B) It also observes than three minutes, or brief (B). It also observes
whether calls carry voice (V), data (D) or fax (F).
This model implies an experiment in which the
procedure is to monitor a call and the observation
consists of the type of call, V, D, or F, and the
length, L or B. The corresponding table entry is the
probability of the outcome. The table is
V D F
L 0.3 0.12 0.15
B 0.2 0.08 0.15
31
Sol ut i on
V D F
L 0.3 0.12 0.15 L 0.3 0.12 0.15
B 0.2 0.08 0.15
From the table we can read that the probability of
a brief data call is
P[BD] = 0.08.
The probability of a long call is
P[L] = P[LV] + P[LD] + P[LF] = 0.57
32
6/7/2011
17
Types of Sampl e Space
Discrete Sample Space
Continuous Sample Space Continuous Sample Space
33
Di scr et e Sampl e Space
Finite sample space : S = {a
1
, a
2
, , a
n
}
All distinct events are mutually exclusive
Event { }
' '
2
'
1
,..., ,
m
a a a A =
| | { } | |
' ' '
| | { } | |
{ } | | { } | | { } | |
' '
2
'
1
' '
2
'
1
...
,..., ,
m
m
a P a P a P
a a a P A P
+ + + =
=
34
6/7/2011
18
( Cont i nue)
Countable infinite, S = {b
1
, b
2
, }
All distinct events are mutually exclusive
Event
| | { } | | { } | |
' '
{ } ,... ,
'
2
'
1
b b B =
| | { } | | { } | | ...
'
2
'
1
+ + = b P b P B P
35
( Cont i nue)
Sample space has n elements,
S = {a
1
, a
2
, , a
n
}
Probability assignment is equal l y l i kel y
out comes
n
a P a P a P
n
1
}] [{ }] [{ }] [{
2 1
= = = =
36
6/7/2011
19
Exampl e
Selecting a ball from urn containing 10 identical
balls numbered 0,1,,9. balls numbered 0,1,,9.
A = number of ball selected is odd
B = number of ball selected is a multiple of 3
C = number of ball selected is less than 5
Find
P[A B] P[A B]
P[A B C]
37
Sol ut i on
S = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
A = {1 3 5 7 9} A = {1, 3, 5, 7, 9}
B = {3, 6, 9}
C = {0, 1, 2, 3, 4}
Assume that outcome are equally likely
10
5
}] 9 [{ }] 7 [{ }] 5 [{ }] 3 [{ }] 1 [{ ] [ = + + + + = P P P P P A P
10
3
}] 9 [{ }] 6 [{ }] 3 [{ ] [ = + + = P P P B P
10
}] [{ }] [{ }] [{ }] [{ }] [{ ] [
10
5
}] 4 [{ } 3 [{ } 2 [{ }] 1 [{ }] 0 [{ ] [ = + + + + = P P P P P C P
38
6/7/2011
20
( Cont i nue)
{ } | |
10
2
9 , 3 ] [ = = P B A P
{ } | |
10
2
3 , 1 ] [ = = P C A P
10
{ } | |
10
1
3 ] [ = = P C B P { } | |
10
1
3 ] [ = = P C B A P
10
] [ ] [ ] [ ] [ B A P B P A P B A P + =
] [ ] [ ] [
] [ ] [ ] [ ] [ ] [
C B A P C B P C A P
B A P C P B P A P C B A P
+
+ + =
39
( Cont i nue)
10
6
10
2
10
3
10
5
] [ ] [ ] [ ] [ = + = + = B A P B P A P B A P
10 10 10 10
9 1 1 2 2 5 3 5
] [ ] [ ] [
] [ ] [ ] [ ] [ ] [
+ + +
+
+ + =
C B A P C B P C A P
B A P C P B P A P C B A P
10 10 10 10 10 10 10 10
= + + + =
40
6/7/2011
21
Exampl e
A fair coin is tossed repeatedly until the first
heads shows up heads shows up
Find a probability law for this experiment
41
Sol ut i on
N
j
= number of trials in which the jth toss result
in first head in first head
n is large N
1
~ n/2
2
1
n
N ~
2
n
~
4
n
~
n
Heads
Tails
n trials
S = {1, 2, 3, }
2
1
4 2 2
1
2
n n
N ~ - ~
8
3
n
N ~
16
4
n
N ~
8
n
~
42
6/7/2011
22
( Cont i nue)
Relative frequency
f
j
= N
j
/n = (1/2)
j
j = 1 2 3 f
j
N
j
/n (1/2)
j
j 1, 2, 3,
P[j toss till first head] = (1/2)
j
j = 1, 2, 3,
Heads
Tails
n = 100 trials
N
1
~ 50
~ 50
~ 25
~ 100/8
1
N
2
~ 25
N
3
~ 100/8
N
4
~ 100/16
43
Cont i nuous Sampl e Spaces
Outcomes are numbers.
Events of interest consist of intervals of the real Events of interest consist of intervals of the real
line, rectangular regions in the plane
44
6/7/2011
23
Exampl e
Pick two number x and y at random between
zero and one. zero and one.
Find the probability of the following events:
A = {x > 0.5},
B = {y > 0.5}
C = {x > y}
45
Sol ut i on
y
1
y
1
[A] 1/2
S
x
1 0
x
1 0
x>1/2
(a) Sample space
(b) Event (x > )
P[A] = 1/2
y
1
y
1
(c) Event (y > ) (d) Event (x > y)
1
y > 1/2
x
1
0
P[B] = 1/2
x
1
0
1
x > y
P[C] = 1/2
46
6/7/2011
24
Exampl e
Suppose that the lifetime of a computer memory
chip is measured, and we find that the chip is measured, and we find that the
proportion of chips whose lifetime exceeds t
decreases exponentially at a rate o.
Find the probability of arbitrary intervals in S.
47
Sol ut i on
Probability that chips lifetime exceeds t decreases
exponentially at a rate o exponentially at a rate o
e
-ot
( ) | | , 0 for , > =

t e t P
t o
S = (0,)
1
0.5
Lifetime
[S] [(0 )] 1
t
time
P[S] = P[(0,)] = 1
48
6/7/2011
25
( Cont i nue)
Let A : event of chips lifetime in interval (r,s)
P[(r, )] = P[(r,s]] + P[(s, )]
r s
time
( ](
P[A] = P[(r,s]] = P[(r, )] - P[(s, )] = e
-or
e
-os
49
Comput i ng pr obabi l i t i es usi ng
Count i ng Met hods
The outcome of finite sample can be assumed to
be equiprobable be equiprobable
Calculation of probabilities reduces to counting
the number of outcome in an event
50
6/7/2011
26
Basi c pr i nci pl e of Count i ng r ul es
Addition rule
Multiplication rule Multiplication rule
51
Addi t i on Rul e
Addi t i on Rul e
Let E and E be mut ual l y excl usi ve events Let E
1
and E
2
be mut ual l y excl usi ve events
Let event E describe the situation where either
event E
1
or event E
2
will occur.
The number of times event E will occur can be
given by the expression:
n(E) = n(E
1
) + n(E
2
)
52
6/7/2011
27
Exampl e
Set of numbers S = {1, 2, 3, 4, 5, 6, 7, 8, 9}
Let E = choosing an odd number from S; Let E
1
= choosing an odd number from S;
E
2
= choosing an even number from S.
Find n(E) when
E = choosing an odd or an even number from S;
n(E) = n(E
1
) + n(E
2
) = 5 + 4
53
Mul t i pl i cat i on Rul e
Mul t i pl i cat i on Rul e
If one event can occur in m ways and another
event can occur in n, independently of each
other, then there are mn ways in which both
events can occur.
54
6/7/2011
28
Exampl e
Multiple-choice test has 2 questions
1
st
question the student select one of 4 possible 1 question, the student select one of 4 possible
answers and 2
nd
question the student select one
of 5 possible answers.
What is the total number of ways of answering
the entire test? the entire test?
Total number of ways = 4x5
55
( Cont i nue)
Multiple-choice test has 2 questions
For question i the student select one of n For question i the student select one of n
i
possible answers.
What is the total number of ways of answering
the entire test?
56
6/7/2011
29
Sol ut i on
x
i
: question number i
n : possible answers for question i n
i
: possible answers for question i
b
1
x
1
a
1
a
2 1
n
a
(a
1
,b
1
) (a
2
,b
1
)
(a
1
,b
2
) (a
2
,b
2
)
( )
1
,
1
b a
n
( )
2
,
1
b a
n
b
2
Total number of
ways of answering
the test = n
1
n
2
( )
2
,
1 n
b a
x
2
(
1 2
) (
2 2
)
2
n
b
( )
2 1
,
n n
b a
( )
2
,
1
n 2
1 2
57
Count i ng Met hod
The number of distinct ordered k-tuples (x
1
,,x
k
)
with component x from a set with n distinct with component x
i
from a set with n
i
distinct
element is
Number of distinct ordered k-tuples = n
1
n
2
n
k
58
6/7/2011
30
Exampl e
There are two subexperiments. The first
subexperiment is Flip a coin It has two subexperiment is Flip a coin. It has two
outcomes, H and T. The second
subexperiment is Roll a die. It has six
outcomes, 1, 2, , 6. The experiment,
Flip a coin and roll a die, has 2x6 = 12
t outcomes:
(H,1), (H,2), (H,3), (H,4), (H,5), (H,6)
(T,1), (T,2), (T,3), (T,4), (T,5), (T,6)
59
Sampl i ng
Sampling
With Replacement With Replacement
Without Replacement
With Ordering
Without Ordering
Wi t h r epl acement Wi t hout Repl acement
RR -
RB RB
BB -
BR BR
60
6/7/2011
31
Sampl i ng wi t h Repl acement and
wi t h Or der i ng
Choose k object from a set A that has n
distinct objects, with replacement and distinct objects, with replacement and
ordering
The experiment produces an ordered k-
tuple,
(x
1
,,x
k
) where x
i
e A and i = 1,, k
Number of distinct ordered k-tuples = n
k
61
Exampl e
An urn contains five ball number 1 to 5
Select two balls from urn with replacement Select two balls from urn with replacement
How many distinct ordered pairs are possible?
What is the probability that the two draws yield
the same number?
62
6/7/2011
32
Sol ut i on
Ordered pairs for sampling with replacement
(1 1) (1 2) (1 3) (1 4) (1 5) (1,1) (1,2) (1,3) (1,4) (1,5)
(2,1) (2,2) (2,3) (2,4) (2,5)
(3,1) (3,2) (3,3) (3,4) (3,5)
(4,1) (4,2) (4,3) (4,4) (4,5)
(5,1) (5,2) (5,3) (5,4) (5,5)
(a) Total distinct ordered pairs are 5
2
= 25
(b) Probability that two draws yield the same
number is 5/25 = 0.2
63
Exampl e
A laptop computer has PCMCIA expansion card
slots A and B. Each slot can be filled with either slots A and B. Each slot can be filled with either
a modem card (m), a SCSI interface (i), or a
GPS card (g). From the set {m, i, g} of possible
cards, what is the set of possible ways to fill the
two slots when we sample with replacement? In
other words, What is the probability that both
slots hold the same type of card?
64
6/7/2011
33
Sol ut i on
Let xy : the outcome that card type x is used in
slot A and card type y is used in slot B. The slot A and card type y is used in slot B. The
possible outcomes are
S = {mm, mi, mg, im, ii, ig, gm, gi, gg}
Th b f ibl t i i (3
2
) The number of possible outcomes is nine (3
2
)
The probability that both slots hold the same type
of card is 3/9
65
Sampl i ng wi t hout Repl acement and
wi t h Or der i ng
Choose k objects in succession without
replacement form A of n distinct object. k < n.
The number of possible outcomes in the first draw
is n
1
= n; the number of possible outcomes in
the second draw is n
2
= n -1, all n object except
the one selected in the first draw; and so on, up
to n = n (k 1) to n
k
= n (k-1)
Number of distinct ordered k-tuples
= n(n-1)(n k +1)
66
6/7/2011
34
Exampl e
A college club has 25 members and is electing
four officers (pres, vp, secretary, treasurer) four officers (pres, vp, secretary, treasurer)
How many ways can the officers be filled?
Order matters Bill as pres and Bob as vice-pres
is different from Bob as pres and Bill as vice-
pres
# ways = 25x24x23x22 = 303 600 # ways = 25x24x23x22 = 303,600
= 25!/21!
25 possibilities for the first officer, 24 for the
second, 23 for the third, 22 for the fourth
67
Per mut at i ons
Sampling without replacement with k < n
Order Matters Order Matters
Use the multiplication rule
Number of ways
( )
! n
P
n
k
=
is called the number of permutation of k
objects out of n
( )! k n
k
n
k
P
68
6/7/2011
35
Sampl i ng wi t hout Repl acement and
wi t hout Or der i ng
Pick k object from a set of n distinct objects
without replacement combination of size k without replacement. combination of size k
The number of ways to choose k objects out of n
distinguishable objects is
= n(n-1)(n k +1)/k! n(n )(n k )/k!
69
Exampl e
A college club has 25 members and is sending
four students to meeting four students to meeting
How many ways can the students be chosen?
Order does not matter sending Bill and Bob is
the same as sending Bob and Bill
# ways = 25x24x23x22/(4x3x2x1)
12 650 25!/(4! 21!) = 12,650 = 25!/(4!x21!)
Permutations divided by 4!, the number of ways
to mix around the four chosen
70
6/7/2011
36
Combi nat i ons
Sampling without replacement with k < n
Order does not Matters Order does not Matters
Use the multiplication rule
Number of ways
( )! !
!
k n k
n
k
n
C
n
k
=
|
|
.
|
\
|
=
is call the number of combinations of k
objects out of n distinguishable objects, and is
also called the binomial coefficient
n
k
C
71
When P and When C
Permutations
Order matters Order matters
Change the order its a different outcome
Combinations
Order does not matter
Change the order its the same outcome
72
6/7/2011
37
Per mut at i ons and Combi nat i on
A batch of 50 items contains 10 defective items.
Suppose 12 items are selected at random and Suppose 12 items are selected at random and
tested.
How many ways can the items be chosen so
that exactly 5 of the items tested are defective?
A more complicated permutation / combination
problem
73
Sol ut i on
5 defective items from the batch of 10
7 nondefective items from the batch of 40
( ) ( ) 252 1 2 3 4 5 / 6 7 8 9 10
10
5
= = C
1 2 3 4 5 6 7
34 35 36 37 38 39 40
40
7

= C
560 , 643 , 18
1 2 3 4 5 6 7
=

74
6/7/2011
38
Per mut at i ons and Combi nat i on
# of ways to selecting 5 defective and 7
nondefective items from the batch of 50 =
252x18,643,560 = 4,698,177,120
# of ways each 12 items can be ordered = 12!
# of ways to selecting 5 defective and 7 # of ways to selecting 5 defective and 7
nondefective items with ordering from the batch
of 50 = 4,698,177,120 x12!
75
Par t i t i oni ng
Combinations how to choose k out of n
Choosing divide the n items into two sets Choosing divide the n items into two sets
The ones that are chosen
The ones that are not chosen
Two results from this observation
#1 : the number of ways of partitioning n
objects into a group of k and a group of n k is objects into a group of k and a group of n -k is
#2 :
n
k
C
n
k n
n
k
C C

=
76
6/7/2011
39
Mul t i nomi al coef f i ci ent
How about partitioning n items into more than
two groups? two groups?
Partition n items into m groups containing k
1
,
k
2
, , k
m
objects (where k
1
+ k
2
+ + k
m
= n)
This is a more general version of the
combinations problem
| | !
This is called the multinomial coefficient
|
|
.
|
\
|
= =
! !... ! ! !... !
!
ways #
2 1 2 1 m m
k k k
n
k k k
n
77
Exampl e
Number of distinct
B
1
n = 9
B
3
B
2
2 2 3
5 6 7 8 9
! 2 ! 3 ! 4
! 9
2 , 3 , 4
9

=
=
|
|
.
|
\
|
n 9
k
1
= 4 k
3
= 2 k
2
= 3
m = 3
1260 =
78
6/7/2011
40
Exampl e
A six-sided die is tossed 12 times.
How many distinct sequences of faces (number How many distinct sequences of faces (number
from the set {1, 2, 3, 4, 5, 6}) have each
number appearing exactly twice?
Wh t i th b bilit f bt i i h
400 , 484 , 7
2
! 12
! 2 ! 2 ! 2 ! 2 ! 2 ! 2
! 12
6
= =
What is the probability of obtaining such a
sequence?
3
12
6
10 4 . 3
336 , 782 , 176 , 2
400 , 484 , 7
6
2 ! 12

= =
79
Exampl e
For five subexperiments with sample
space S = {0 1} how many observation space S = {0, 1}, how many observation
sequences are there in which 0 appears 2
times and 1 appears 3 times?
80
6/7/2011
41
Sol ut i on
The set of five-letter words with 0 appearing twice
and 1 appearing three times is and 1 appearing three times is
{00111, 01011, 01101, 01110, 10011, 10101, 10110,
11001, 11010, 11100}
The e a e e actl 10 s ch o ds There are exactly 10 such words
Multinomial coefficient
10
! 3 ! 2
! 5
3 , 2
5
= =
|
|
.
|
\
|
81
Sampl i ng wi t h Repl acement and
wi t hout Or der i ng
Pick k objects from set of n distinct with
replacement and record result without order replacement and record result without order
Number of different ways of picking k objects
from a set of n distinct objects with
replacement and without ordering replacement and without ordering
|
|
.
|
\
|
+
=
|
|
.
|
\
|
+
1
1 1
n
k n
k
k n
82
6/7/2011
42
Exampl e
Consider the following program segment, where
a, b, c are integer variables a, b, c are integer variables
for a = 1 to 20 do
for b = 1 to a do
for c = 1 to b do
print(a *b + c)
How many times is the print statement execute
in this program segment?
83
( Cont i nue)
The selections of a, b, and c where the print
statement is executed satisfies the condition 1 < statement is executed satisfies the condition 1 <
c < b < a < 20.
The print statement is executed
1540
3
22
3
1 3 20 1
=
|
|
.
|
\
|
=
|
|
.
|
\
|
+
=
|
|
.
|
\
|
+
k
k n
3 3
|
.
\
|
.
\
|
.
\
k
84
6/7/2011
43
Condi t i onal Pr obabi l i t y
Interested in determining whether two events,
event A and B event A and B
A and B are related in sense that knowledge B,
alters the likelihood of occurrence of A
P[A|B] : Probability of A given B P[A|B] : Probability of A given B
85
( Cont i nue)
Condi t i onal Pr obabi l i t y
The conditional probability of the event A given The conditional probability of the event A given
the occurrence of the event B is
S
B
| |
| |
| | B P
B A P
B A P

= |
,P[B] > 0
A
B
A B
86
6/7/2011
44
( Cont i nue)
Pr obabi l i t y as Rel at i ve f r equency
| |
B
B A
n
n
B A P
=
B event in time of number
B A event in time of number
|
| |
n
n
n
n
B A P
B
B A
= |
87
Cor r el at ed Event s
some definitions describing the conditional
probabilities of correlated events. probabilities of correlated events.
P[AB] or P[AB] : Joint Probability
The probability that both events A and B occur
P[A|B] : Condition Probability
The probability of event A, given that some other
event B has occurred
P[B] : Probability of event B
88
6/7/2011
45
( Cont i nue)
| |
| |
| |
B A P
B A P

= | | |
| | B P
B A P |
P[A B] = P[A|B]P[B]
= P[AB]
P[B A] = P[B|A]P[A]
= P[BA]
=
89
Exampl e
Communication systems can be modeled :
User inputs a 0 or a 1 into system, p y ,
Receiver makes a decision about what was the input
to the system, based on the signal it received.
Suppose that
User sends 0s with probability 1- p and 1s with
probability p,
Receiver makes random decision errors with Receiver makes random decision errors with
probability c.
R
0
R
1
1- c
c
1- c
c
T
0
1- p
T
1
p
90
6/7/2011
46
( Cont i nue)
R
0
1- c
c
T
0
1- p
Given P(R
0
|T
0
) = (1- c), P(R
1
|T
0
) = c,
P(R
1
|T
1
) = (1- c), P(R
0
|T
1
) = c
R
1
1- c
c
T
1
p
Find P(R
0
)
P(R
0
) = P(R
0
/T
0
)P(T
0
) + P(R
0
/T
1
)P(T
1
)
= (1- c)(1-p) + cp = 1 - c - p + 2cp
91
( Cont i nue)
For i = 0, 1, let
T be the event input was i T
i
be the event input was i,
R
j
be the event receiver decision was j.
Find probabilities P[T
i
R
j
] for i = 0,1 and
j = 0,1.
P[T R ] = ? P[T
0
R
0
] = ?
P[T
0
R
1
] = ?
P[T
1
R
0
] = ?
P[T
1
R
1
] = ?
92
6/7/2011
47
1 0
Sol ut i on
Input into binary channel
Transmitter
1 p
0 1 0
1
Output form binary channel
Receiver
1-c
1-p
p
c c 1-c
(1-p)(1-c) (1-p)c pc p(1-c)
P[T
0
R
0
] = (1 - c)(1 p)
P[T
0
R
1
] = c(1 p)
P[T
1
R
0
] = cp P[T
1
R
1
] = (1 - c)p
93
Exampl e
Ball is selected from urn containing
Two black balls numbered 1 and 2 Two black balls, numbered 1 and 2
Two white balls, numbered 3 and 4
Events
A : black ball selected
B : even-numbered ball selected
C b f b ll i t th 2 C : number of ball is greater than 2
Find P[A|B] and P[A|C]
94
6/7/2011
48
Sol ut i on
S = {(1,b), (2,b) , (3,w) , (4,w)}
A = {(1,b), (2,b)}
B = {(2,b), (4,w)}
C = {(3,w), (4,w)}
| |
| |
| |
5 . 0
5 . 0
25 . 0
= =
=
B P
B A P
B A P
| |
| |
| |
0
5 0
0
= =
=
C P
C A P
C A P
P[AB] = P[(2,b)]
P[AC] = P[C] = 0
| |
| | 5 . 0 C P
95
Exampl e
An urn contains
two black balls two black balls
three white ball
Two balls are selected at random without
replacement and the sequence of colors in
noted noted.
Find the probability that both balls are black.
96
6/7/2011
49
Sol ut i on
0
O t f fi t d
W
1
5
2
B
1
B
2
W
2
5
3
4
1
4
3
4
2
4
2
Outcome of first draw
B
2
W
2
1
2
Outcome of second draw
10
1
10
3
10
3
| |
10
1
5
2
4
1
2 1
= = B B P
10
3
| | | |
2 1 2
| B P B B P =
97
Tot al Pr obabi l i t y
Let B
1
, B
2
, , B
n
be mutually exclusive event
S B B B S = B
1
B
2
B
n
B
1
, B
2
, , B
n
form a partition of S
98
6/7/2011
50
( Cont i nue)
S
B
1
B
3
B
n-1
A = A S = A (B
1
B
2
B
n
)
= (A B
1
) (A B
2
) (A B
n
)
A
B
2
B
n
P[A] = P[A B
1
] + P[A B
2
] + + P[A B
n
]
= P[A|B
1
]P[B
1
]+P[A|B
2
]P[B
2
]+ + P[A|B
n
]P[B
n
]
99
Exampl e
An urn contains
two black balls two black balls
three white ball
Two balls are selected at random without
replacement and the sequence of colors in
noted noted.
Find the probability of the event that the
second ball is white.
100
6/7/2011
51
Sol ut i on
B
1
0
W
1
2 3
1
B
2
W
2
B
2
W
2
5
2
5
3
4
1
4
3
4
2
4
2
10
1
10
3
10
3
2
10
3
P[W
2
] = P[W
2
|B
1
]P[B
1
] + P[W
2
|W
1
]P[W
1
]
5
3
10
3
10
3
5
3
2
1
5
2
4
3
= + = + =
101
Exampl e
A manufacturing process produces a mix of
good memory chips and bad memory chips. good memory chips and bad memory chips.
The lifetime of good chips follows the
exponential law with rate of failure o. The
lifetime of bad chips also follows the exponential
law, but the rate of failure is 1000o. Suppose
that the fraction of good chips is 1 p and of
bad chips, p.
Find the probability that a randomly selected
chip is still functioning after t second.
102
6/7/2011
52
( Cont i nue)
Let
G : Chip is good 0 8
1
Lifetime
G : Chip is good
B : Chip is bad
C : Chip still
functioning after t
second
e
-1000ot
0
e
-ot
2 4 6 8 10
0.2
0.4
0.6
0.8
time
t
| | | | | | | | | | B P B C P G P G C P C P | | + =
| |( ) | |p B C P p G C P | 1 | + =
( )
t t
pe e p
o o 1000
1

+ = 103
Exampl e
Three machines B
1
, B
2
, and B
3
for making 1 kO
resistors.
B
1
produces 80% of resistors within 50 O of the
nominal value.
B
2
nominal value.
Percentage for machine B
3
is 60%.
Each hour B produces 3000 resistors B Each hour, B
1
produces 3000 resistors, B
2
produces 4000 resistors, and B
3
produces 3000
resistors.
104
6/7/2011
53
Sol ut i on
What is the probability that the company ships a
resistor that is within 50 O of the nominal resistor that is within 50 O of the nominal
values?
105
( Cont i nue)
Let A = {resistor is within 50 O of the nominal
value} value}
B
1
nominal value.
P[A|B
1
] = 0.8,
B
2
nominal value.
P[A|B
2
] = 0.9,
Percentage for machine B
3
is 60%.
P[A|B
3
] = 0.6
106
6/7/2011
54
( Cont i nue)
Each hour, B
1
produces 3000 resistors, B
2
produces
4000 resistors, and B
3
produces 3000 resistors. ,
3
p
Total product = 3000 + 4000 + 3000 = 10,000
P[B
1
] = 3000/10,000 = 0.3,
P[B
2
] = 0.4,
P[B ] 0 3 P[B
3
] = 0.3
107
( Cont i nue)
A = {resistor is within 50 O of the nominal value}
S
B
1
B
3
B
2
A
P[A] = P[A|B
1
]P[B
1
] + P[A|B
2
]P[B
2
] + P[A|B
3
]P[B
3
]
= (0.8)(0.3) + (0.9)(0.4) + (0.6)(0.3) = 0.78
108
6/7/2011
55
Bayes Rul e
Situations
Advance in formation about
P[A|B]
Need to calculate
P[B|A]
109
( Cont i nue)
Let B
1
, B
2
, , B
n
be a partition of a sample space S.
Let event A occurs; Let event A occurs;
| |
| |
| |
prob. condition , |
A P
A B P
A B P

=
| | | | B A P A B P =
| | | | | | B P B A P B A P |
| |
| | | |
| | A P
B P B A P
A B P
|
| =
| | | | | | B P B A P B A P = |
110
6/7/2011
56
Exampl e of Bayes Rul e
Communi cat i on syst em
0.92
R
0
R
1
0.92
0.08
0.95
0.05
T
0
0.45
T
1
0.55
Given : P(R
0
|T
0
)
Want to know : P(T
0
|R
0
)
111
Exampl e: Communi cat i on Syst em
Communi cat i on syst em
0 92
R
0
R
1
0.92
0.08
0.95
0.05
T
0
0.45
T
1
0.55
Given P(R
0
|T
0
) = 0.92;
P(R
1
|T
1
) = 0.95
P(T
0
) = 0.45; P(T
1
) = 0.55
112
6/7/2011
57
( Cont i nue)
G ( | ) 0 92 Given P(R
0
|T
0
) = 0.92
P(R
1
|T
1
) = 0.95
P(T
0
) = 0.45
P(T
1
) = 0.55
Find P(R
0
), P(R
1
), P(T
1
|R
1
), P(T
0
|R
0
), P(Error)
Find which input is more probable given that the
receiver has output a 1
113
Sol ut i on
P(R
0
) = P(R
0
/T
0
)P(T
0
) + P(R
0
/T
1
)P(T
1
)
= 0.92x0.45 + 0.05x0.55 = 0.4415
P(R
1
) = P(R
1
/T
1
)P(T
1
) + P(R
1
/T
0
)P(T
0
)
= 0.95x0.55 + 0.08x0.45 = 0.5585
P(R
1
) = 1 P(R
0
)
P(E ) P(R /T )P(T ) + P(R /T )P(T ) P(Error) = P(R
0
/T
1
)P(T
1
) + P(R
1
/T
0
)P(T
0
)
= 0.05x0.55 + 0.08x0.45 = 0.0635
114
6/7/2011
58
( Cont i nue)
P(T
1
/R
1
) = P(R
1
/T
1
)P(T
1
)/P(R
1
)
= 0.95x0.55/0.5585
= 0.9355
P(T
0
/R
0
) = P(R
0
/T
0
)P(T
0
)/P(R
0
)
= 0.98x0.45/0.4415
= 0.9988
115
( Cont i nue)
P[T
0
|R
1
] = P[T
0
R
1
]/P[R
1
] = [R
1
|T
0
]P[T
0
]/P[R
1
]
T
1
is more probable given that
the receiver has output a 1
= 0.08x0.45/0.5585 = 0.0645
P[T
1
|R
1
] = P[T
1
R
1
]/P[R
1
] = P[R
1
|T
1
]P[T
1
]/P[R
1
]
= 0.95x0.55x/0.5585 = 0.9355
116
6/7/2011
59
I ndependence of Event s
Two I ndependent Event s
] [
] [
] [ ] [
] [
] [
] | [ A P
B P
B P A P
B P
B A P
B A P = = =

If P[A B] = P[A]P[B] then A and B are independent.
Since
Implies that P[A | B] = P[A],
P[B | A] = P[B]
Implies that P[A] = 0 and P[B] = 0
117
I ndependent & Di sj oi nt
Two events A and B
P[A] = 0, P[B] = 0, P[A B] = 0
A and B cannot be independent.
A
S
A
B
S
Independent Disjoint
B
B
P[AB] = 0
P[AB] = P[A]P[B]
P[AB] = 0
118
6/7/2011
60
Exampl e
A short-circuit tester has a red light (r) to indicate
that there is a short circuit and a green light (g) that there is a short circuit and a green light (g)
to indicate that there is no short circuit. Consider
an experiment consisting of a sequence of three
tests. Suppose that for the three lights, each
outcome (a sequence of three lights, each either
red or green) is equally likely Are the events R red or green) is equally likely. Are the events R
2
that the second light was red and G
2
that the
second light was green independent? Are the
events R
1
and R
2
independent?
119
( Cont i nue)
A short-circuit tester
red light (r) : short circuit red light (r) : short circuit
green light (g) : no short circuit.
Consider a sequence of three tests.
The sequence of three lights is equally likely.
Are R
2
and G
2
independent?
Are R
1
and R
2
independent?
120
6/7/2011
61
Sol ut i on
Circuit 3 Circuit 2 Circuit 1
R
1
= the first circuit is red
R
2
= the second circuit is red
G
2
= the second circuit is red
Outcome rrr rrg rgr rgg grr grg ggr ggg
121
( Cont i nue)
Ar e R
2
and G
2
i ndependent ?
S = {rrr rrg rgr rgg grr grg ggr ggg} S = {rrr, rrg, rgr, rgg, grr, grg, ggr, ggg}
P[R
2
] = 4/8 = 1/2
P[G
2
] = 4/8 =1/2
s
rrr
rrg
rgr
rgg
R
2
G
2
R
2
G
2
= C
P[R
2
G
2
] = 0
rrg
grr
grg
rgg
ggr
ggg
122
6/7/2011
62
( Cont i nue)
Ar e R
2
and G
2
i ndependent ?
P[R
2
]P[G
2
] = ()() =
P[R
2
G
2
] = 0
P[R
2
G
2
] = P[R
2
]P[G
2
]
R
2
and G
2
are not independent
R
2
and G
2
are disjoint
123
( Cont i nue)
Ar e R
1
and R
2
i ndependent ?
S = {rrr rrg rgr rgg grr grg ggr ggg} S = {rrr, rrg, rgr, rgg, grr, grg, ggr, ggg}
P[R
1
] = 4/8 = 1/2
P[R
2
] = 4/8 =1/2
s
rrr
rrg
rgr
rgg
R
1
R
1
R
2
= {rrr, rrg}
P[R
1
R
2
] = 2/8 = 1/4
grr
grg
ggr
ggg
R
2
124
6/7/2011
63
Sol ut i on
Ar e R
1
and R
2
i ndependent ?
P[R
1
]P[R
2
] = ()() =
P[R
1
R
2
] =
P[R
1
R
2
] = P[R
1
]P[R
2
]
R
1
and R
2
are independent
125
Exampl e
Two number x and y are selected at random
between zero and one between zero and one
A = {x > 0.5}
B = {y > 0.5}
C = {x > y} { y}
Are A and B independent?
Are A and C independent?
126
6/7/2011
64
Sol ut i on
y
1
y
1
B
A
1
x
0
0
x
1
P[A B] =
P[A] =
P[B] =
P[A]P[B]
P[A C] = + 1/8 = 3/8
P[A] =
P[C] =
P[A]P[C ]
A C
P[A]P[B] =
P[A B] = P[A]P[B ] =
Event A and B are
independent
P[A]P[C ] =
P[A B] = P[A]P[B ]
Event A and C are not
independent
127
3 I ndependent Event s
Event A, B and C are independent if the
probability of the intersection of any pair or probability of the intersection of any pair or
triplet of events is equal to the product of the
probabilities of the individual events
P[A B C] = P[A]P[B]P[C] [ ] [ ] [ ] [ ]
128
6/7/2011
65
Exampl e
Two number x and y are selected at random
between zero and one between zero and one
A = {x > 0.5}
B = {y > 0.5}
C = {x > y} { y}
Are A and B and C independent?
129
Sol ut i on
y
1
B
A = {x > 0.5}
P[A ] =
P[B ] =
P[C]
P[A B C] = P[A]P[B]P[C]
A
1
x
0
B
C
B = {y > 0.5}
C = {x > y}
P[C] =
P[A B C] = 1/8
P[A]P[B]P[C] = ()()()
= 1/8
Event A and B and
C are independent
130
6/7/2011
66
Exampl e
Let B = {y > }
D = {x < } D = {x < }
F = {x < and y < } {x > and y > }
Are B, D and F are independent?
131
Sol ut i on
y
1
y
1
F
y
1
B
P[B]
0
x
1
D

D = {x < }
0
x
1
F

F = {x< and y<}
{x> and y>}
1
x
0

B = {y > }
P[B] =
P[D] =
P[F] =
132
6/7/2011
67
Sol ut i on
B = {y > }
y
1
B
F
P[B] =
P[D] =
P[B D F] = P[B]P[D]P[F]
Th t t i d d t
D = {x < }
F = {x< and y<}{x> and y>}
1
x
0
B

D
F
F
P[D] =
P[F] =
P[B D F] = 0
P[B]P[D]P[F] = ()()() =1/8
Three events are not independent
133
Sequent i al Exper i ment s
Sequent i al exper i ment consist of a
sequence of subexperiment sequence of subexperiment
Sequences of Independent Experiment
Sequences of Dependent Experiment
134
6/7/2011
68
Sequences of I ndependent
Exper i ment s
Experiment subexperiments subexperiments
Let Let
A
1
, A
2
,, A
n
are subexperiments
If A
1
, A
2
,, A
n
are independent
Thus
P[A
1
A
2
A
n
] = P[A
1
]P[A
2
] P[A
n
]
135
Exampl e
Suppose that 10 numbers are selected at
random from the interval [0,1]. Find the random from the interval [0,1]. Find the
probability that the first 5 numbers are less than
and the last 5 numbers are greater than .
136
6/7/2011
69
Sol ut i on
Let x
1
, x
2
, , x
10
: the sequence of 10 numbers
A
k
= {x
k
< } for k = 1,,5
A
k
= {x
k
> } for k = 6,,10
0

1
P[A
1
A
2
A
10
] = P[A
1
]P[A
2
] P[A
10
]
= ()
5
()
5
137
Ber noul l i t r i al
Ber noul l i t r i al
is an experiment whose outcome can be either of is an experiment whose outcome can be either of
two possible outcomes, "success with probability
p and "failure with probability q = 1 - p.
In practice it refers to a single experiment which
can have one of two possible outcomes.
Example Example
Toss a coin
Head (success)
Tail (fail)
138
6/7/2011
70
Bi nomi al Exper i ment
A bi nomi al experiment is one that possesses
the following properties: the following properties:
The experiment consists of n repeated trials;
Each trial results in an outcome that may be
classified as a success or a failure (hence the
name, binomial); , );
The probability of a success, denoted by p,
remains constant from trial to trial and repeated
trials are independent.
139
Bi nomi al Pr obabi l i t y Law
k : the number of successes in n independent
Bernoulli trials Bernoulli trials
The probabilities of k successes in n independent
repetitions is :
( ) ( )
k n
k
n
p p
k
n
k P

|
|
.
|
\
|
= 1
k
|
.
\
n = the number of trials
k = 0, 1, 2, ... n
p = the probability of success in a single trial
140
6/7/2011
71
( Cont i nue)
k k
n
|
|
|
P b bilit f k
k n k
n
p p
k
n
k P

|
|
.
|
\
|
= ) 1 ( ) (
)! ( !
!
k n k
n
k
n
=
|
|
.
|
\
|
Probability of k successes
in n trials
Binomial coefficient
n
k n
C
k
n
k N =
|
|
.
|
\
|
= ) (
Picking k positions out of n
for the success
141
Exampl e
Coin is tossed three time
Tosses are independent and the probability of Tosses are independent and the probability of
heads is p
Let k : the number of head in three trials
Find the probability of k success in head.
142
6/7/2011
72
Sol ut i on
Sequence of heads and tail is
P[{HHH}] = P[{H}]P[{H}]P[{H}] = p
3
P[{HHH}] P[{H}]P[{H}]P[{H}] p
P[{HHT}] = P[{H}]P[{H}]P[{T}] = p
2
(1-p)
P[{HTH}] = P[{H}]P[{T}]P[{H}] = p
2
(1-p)
P[{THH}] = P[{T}]P[{H}]P[{H}] = p
2
(1-p)
P[{TTH}] = P[{T}]P[{T}]P[{H}] = p (1-p)
2
P[{THT}] = P[{T}]P[{H}]P[{T}] = p (1-p)
2
{ }
P[{HTT}] = P[{H}]P[{T}]P[{T}] = p (1-p)
2
P[{TTT}] = P[{T}]P[{T}]P[{T}] = (1-p)
3
143
Sol ut i on
P
3
(0) = P[k = 0] = P[{TTT}] = (1 p)
3
| |
P
3
(1) = P[k = 1] = P[{TTH, THT, HTT}] = 3p(1 p)
2
( ) ( ) ( )
3 3
0
3
1 1
0
3
0 p p p P =
|
|
.
|
\
|
=
( ) ( ) ( )
2 2 1
3
|
|
|
( ) ( ) ( )
2 2 1
3
1 3 1
1
3
1 p p p p P =
|
|
.
|
\
|
=
144
6/7/2011
73
( Cont i nue)
P
3
(2) = P[k = 2] = P[{TTH, THT, HTT}] = 3p
2
(1 p)
P
3
(3) = P[k = 3] = P[{HHH}] = p
3
( ) ( ) ( ) p p p p P =
|
|
.
|
\
|
= 1 3 1
2
3
2
2 1 2
3
( ) ( )
3 0 3
3
1
3
3
3 p p p P =
|
|
.
|
\
|
=
145
Exampl e
Communication system transmits binary
information over channel that introduced information over channel that introduced
random bit errors with probability c = 10
-3
Transmitter transmits each information bit three
times
Receiver take majority vote of received bit to
decide on that the transmitted bit was
Find the probability that the receiver will make
an incorrect decision
146
6/7/2011
74
Sol ut i on
Let k : number of errors that the receiver will
make incorrect decision make incorrect decision
| | ( ) ( ) ( ) ( )
( )
6
0 3 1 2
10 3
999 . 0 001 . 0
3
3
999 . 0 001 . 0
2
3
2
=
|
|
.
|
\
|
+
|
|
.
|
\
|
= > k P
( ) 10 3 =
147
Exampl e
To communicate one bit of information reliably,
cellular phone transmit the same binary cellular phone transmit the same binary
symbol five times.
zero is transmitted as 00000
one is 11111.
The receiver detects the correct information if
three or more binary symbols are received o o b a y y bo a d
correctly.
What is the information error probability P[E], if
the binary symbol error probability is q = 0.1?
148
6/7/2011
75
Sol ut i on
k : # of errors that the receiver will make correct
decision decision
The probability of a successes is p
p = 1- q = 0.9
The information error probability P[E]
| | ( ) ( ) ( ) 2 1 0 + + = P P P E P
| | ( ) ( ) ( )
00856 . 0
2
5
1
5
0
5
2 1 0
3 2 4 5
5 5 5
=
|
|
.
|
\
|
+
|
|
.
|
\
|
+
|
|
.
|
\
|
=
+ + =
q p pq q
P P P E P
149
Mul t i nomi al Exper i ment
A mul t i nomi al exper i ment has the following
properties: properties:
The experiment consists of n repeated trials.
Each trial has a discrete number of possible
outcomes.
On any given trial, the probability that a
particular outcome will occur is constant. p
The trials are independent; that is, the outcome
on one trial does not affect the outcome on other
trials.
150
6/7/2011
76
Exampl e
Toss two dice three times, and record the outcome
on each toss. This is a multinomial experiment on each toss. This is a multinomial experiment
because:
The experiment consists of repeated trials. We toss
the dice three times.
Each trial can result in a discrete number of outcomes
- 2 through 12.
Th b bilit f t i t t it d The probability of any outcome is constant; it does
not change from one toss to the next.
The trials are independent; that is, getting a
particular outcome on one trial does not affect the
outcome on other trials.
151
Mul t i nomi al Pr obabi l i t y Law
A multinomial experiment consists of n trials, and each
trial can result in any of k possible outcomes: B
1
, B
2
, y p
1
,
2
,
. . . , B
M
. Suppose that each possible outcome can
occur with probabilities p
1
, p
2
, . . . , p
k
.
The probability that B
1
occurs k
1
times, B
2
occurs k
2
times, .
. . , and B
M
occurs k
m
times is
where n = k
1
+ k
2
+ . . . + k
M
.
( ) | |
M
k
M
k k
M
M
p p p
k k k
n
k k k P ...
! !... !
!
,..., ,
2 1
2 1
2 1
2 1
=
152
6/7/2011
77
( Cont i nue)
( ) | |
M
k k k
p p p
n
k k k P
!
2 1
= ( ) | |
M
M
M
M
p p p
k k k
k k k P ...
! !... !
,..., ,
2 1
2 1
2 1
2 1
=
Multinomial Coefficient
153
Exampl e
Pick 10 telephone numbers at random from a
telephone book and note the last digit in each of telephone book and note the last digit in each of
the numbers. What is the probability that we
obtain each of the integers from 0 to 9 only
once?
Solution
( ) ( )
4
10
10 6 . 3 1 . 0
! 1 !... 1 ! 1
! 10

=
154
6/7/2011
78
Geomet r i c Pr obabi l i t y Law
Geomet r i c Pr obabi l i t y Law
Repeat independent Bernoulli trials until the Repeat independent Bernoulli trials until the
occurrence of the first success
Let m : # of trials carried out until the occurrence of
the first success
( ) | | ( ) ,... 2 , 1 1 ...
1
'
1
'
2
'
1
= = =

m p p A A A A P m P
m
m m
Where p : probability of success for the Bernoulli trial.
A
i
: event success in i
th
trial
( ) | | ( ) , ,
1 2 1
p p
m m
155
( Cont i nue)
P(m) = (1-p)
m-1
p Geometric probability
m 1
1 = q
m-1
p q = 1 - p
( )

=
=
=
1
1
1 m
m
m
q p m P
Let i = m - 1
= x
( )

=

=
= =
n
i
i
n
i
i
m
q p q p m P
0 0 1
lim
Let i m 1
156
6/7/2011
79
( Cont i nue)
=
=
n
i
i
q x
0
x = 1 + q + q
2
++ q
n
qx = q + q
2
++ q
n
+ q
n+1
(1-q)x = 1 - q
n+1
q
q
x
n
=
+
1
1
1
q
q
q
n n
i
i
=
+
=
1
1
1
0
q 1
q
i 0
( ) 1
1
1
lim lim
1
1 1
=
= =
+

=

=

q
q
p q p m P
n
n
n
m
i
n
m
157
( Cont i nue)
K = # of Bernoulli trial
The probability that more than K trial : The probability that more than K trial :
{ } | |

+ =
+
= + + = >
1
1 1
...
K m
m K K
q p p q p q K m P


= + =
=
0 1
1
i
i K
K m
m
q q q
i = m 1 - k
158
6/7/2011
80
( Cont i nue)
q
q q
n n
i i
=
= =
+

1
1
1
1
1
lim lim
The probability that more than K trial :
{ } | |
K
q
pq K m P
= >
1
1
q q
q q
n i n i

= =

1 1
0 0
lim lim
Where p = 1 q
K
q =
159
Exampl e
Computer A sends a message to computer B
over an unreliable telephone line over an unreliable telephone line.
When B detects an error
Request A to retransmit
Probability of a message transmission error is
q = 0.1
What is the probability that a message needs to
be transmitted more than two times?
160
6/7/2011
81
Sol ut i on
Probability of m
th
success is
P(m) = q
m-1
p m = 1,2,
The probability that a message needs to be
transmitted more than two times
161
( Cont i nue)
1
st
p
2
nd
qp
The probability that a
message needs to be
2 qp
Trial more than 2 times
3
rd
q
2
p
m
th
q
m-1
p
message needs to be
transmitted more than
two times
P[m > 2] = q
2
= 10
-2
1 = p + qp + q
2
p + q
3
p +
q
2
p + q
3
p + = 1 (p + qp) = 1 p qp
= q q(1 - q) = q q + q
2
= q
2
162
6/7/2011
82
Rel i abi l i t y Pr obl em
Independent trials can be used to describe
reliability problem calculate the probability reliability problem calculate the probability
that a particular operation succeeds
Operation consist of n components
Each component succeeds with p, independent of
any other component.
Two basic types of operations Two basic types of operations
Components in series
Components in parallel
163
Type of Operation
Components in series Components in parallel
W
1 W
3
W
2
W
1
W
W
2
P[W] = P[W
1
W
2
W
n
]
= p x p xx p
= p
n
| | | |
( )
n
n
p
W W W P W P
=
= '
1
...
' '
2
'
1
W
3
164
6/7/2011
83
Sequent i al Exper i ment s
Sequent i al exper i ment consist of a
sequence of subexperiment sequence of subexperiment
Sequences of Independent Experiment
Sequences of Dependent Experiment
165
Sequences of Dependent Exper i ment
The outcome of a given experiment determines
which subexperiment is performed next. which subexperiment is performed next.
It can be represented by a tree diagram.
0 0 0 0
h
1
0
1 1
0 0
1 1 1 1
t
0 0 0
1 1 1
1 2 3 4
166
6/7/2011
84
Exampl e
Sequential experiment involves repeatedly drawing a
ball from one of two urns, noting the number on the , g
ball and replacing the ball in its urn. The urn from
which the first draw is made is selected at random
by flipping a fair coin. Urn 0 is used if the outcome
is heads and urn 1 if the outcome is tails. Thereafter
the urn used in a subexperiment corresponds to the
number on the ball selected in the previous number on the ball selected in the previous
subexperiment.
167
( Cont i nue)
0
1
0
h
0
1
0
0
1
0
0
1
0
0 0
1
1
1
1
0
1
1
0
1
1
t
1
0
1
1 2 3 4
1
0
1
1
0
1
168
6/7/2011
85
Pr obabi l i t y of Dependent
Exper i ment
Compute the probability of a particular sequence
of outcomes, say s
0
, s
1
, s
2
where of outcomes, say s
0
, s
1
, s
2,
where
s
0
is result from 1
st
outcome
s
1
is result from 2
nd
outcome
s
2
is result from 3
rd
outcome
P[{s
0
}{s
1
}{s
2
}] = ??
169
( Cont i nue)
P[{s
0
}{s
1
}{s
2
}] = ??
Let A = {s
2
}, B = {s
1
}{s
0
}
P[{s
0
}{s
1
}{s
2
}] = P[AB]
P[{s
0
}{s
1
}{s
2
}] = P[{s
2
}{s
1
}{s
0
}] = P[AB]
Since P[A B] = P[A|B]P[B]
P[{s
0
}{s
1
}{s
2
}] = P[{s
2
}|{s
1
}{s
0
}]P[{s
1
}{s
0
}]
= P[{s
2
}|{s
1
}{s
0
}]P[{s
1
}|{s
0
}]P[{s
0
}]
170
6/7/2011
86
( Cont i nue)
P[{s
0
}{s
1
}{s
2
}] = P[{s
2
}|{s
1
}{s
0
}]P[{s
1
}|{s
0
}]P[{s
0
}]
P[{s
n
}|{s
n-1
} {s
1
}{s
0
}] = ??
M k Ch i
depends on only on {s
n-1
}]
P[{s
n
}|{s
n-1
} {s
1
} {s
0
}]{s
0
}] = P[{s
n
}|{s
n-1
}]
Markov Chain
171
( Cont i nue)
P[{s
0
}{s
1
}{s
2
}] = P[{s
2
}|{s
1
}{s
0
}]P[{s
1
}{s
0
}]
= P[{s }|{s }{s }]P[{s }|{s }]P[{s }] = P[{s
2
}|{s
1
}{s
0
}]P[{s
1
}|{s
0
}]P[{s
0
}]
Therefore
P[{s
0
}{s
1
}{s
2
}] = P[{s
2
}|{s
1
}]P[{s
1
}|{s
0
}]P[{s
0
}]
{s
1
}
P[s
0
s
1
s
2
] = P[s
2
|s
1
]P[s
1
|s
0
]P[s
0
]
172
6/7/2011
87
( Cont i nue)
P[s
0
s
1
s
n
] = P[s
n
|s
n-1
]P[s
n-1
|s
n-2
]P[s
1
|s
0
]P[s
0
]
173
Exampl e
1
1
1
0
1
0
h
0
1
0
0
0
1
0
0
0
1
0
0
0 0
1
1
1
1
0
1
1
Find the probability of the sequence 0011
1
t
1
1
1 2 3 4
1
1
1
1
174
6/7/2011
88
( Cont i nue)
The probability of the sequence 0011
Meaning
P[0011] = P[{s
0
}{s
1
}{s
2
}{s
3
}]
= P[{0}{0}{1}{1}]
P[0011] = P[1|1] P[1|0] P[0|0]P[0]
= P[{s
3
}|{s
2
}] P[{s
2
}|{s
1
}]P[{s
1
}|{s
0
}]P[{s
0
}]
175
0
h
( Cont i nue)
0
1
0
0
1
0
0
1
0
0
3
1
1
t
1
0
1
1 2 3 4
0 0
3
2
0 0
3
2
3
2
1
0
1
1
0
1
0 0
1
0
1
0
1
0
1

6
1
6
5
0
1
0
1
3
1
3
1
6
1
6
1
6
5
6
5
1
1
1
0
1
1
1
176
6/7/2011
89
( Cont i nue)
P[0011] = P[1|1]P[1|0]P[0|0]P[0]
2
2 2
0 0
0
0
0
0
0
P[1|1] = 5/6
3
1

0
1
0
1
3
1
3
2
6
1
6
5
0
1
0
1
3
1
3 3
6
1
6
1
6
5
6
5
0
h
0
1
t
1
1
0
0
1
1 2 3 4
0
1
1
0
0
1
0
1
1
0
0
1
P[1|0] = 1/3
P[1|1] 5/6
P[0|0] = 2/3
| |
54
5
2
1
3
2
3
1
6
5
0011 =
|
.
|
\
|
|
.
|
\
|
|
.
|
\
|
|
.
|
\
|
= P
P[1|0] 1/3
P[0] = = P[1]
177
Exampl e
Suppose traffic engineers have coordinated the timing
of two traffic lights to encourage a run of green g g g
lights. In particular, the timing was designed so that
with probability 0.8 a driver will find the second light
to have the same color as the first. Assuming the
first light is equally likely to be red or green, what is
the probability P[G
2
] that the second light is green?
Also, that is P[W], the probability that you wait for Also, that is P[W], the probability that you wait for
at least one light? Lastly, what is P[G
1
|R
2
], the
conditional probability of a green first light given a
red second light?
178
6/7/2011
90
Sol ut i on
Given P[G
2
|G
1
] = P[R
2
|R
1
] = 0.8
P[G ] = P[R ] = 0 5 P[G
1
] = P[R
1
] = 0.5
0.5
0.8
0.2
P[G
1
G
2
] = 0.4
P[G
1
R
2
] = 0.1
G
1
R
2
G
2
0.5
0.8
0.2
P[R
1
R
2
] = 0.4
P[R
1
G
2
] = 0.1
R
1
R
2
G
2
179
( Cont i nue)
What is the probability P[G
2
] that the second
light is green? light is green?
0.5
0 5
0.8
0.2
0.2
P[G
1
G
2
] = 0.4
P[G
1
R
2
] = 0.1
P[R
1
G
2
] = 0.1
G
1
R
2
G
2
G
2
P[G
2
] = P[G
1
G
2
] + P[R
1
G
2
] = 0.4 + 0.1 = 0.5
0.5
0.8
P[R
1
R
2
] = 0.4
R
1
R
2
180
6/7/2011
91
( Cont i nue)
What is the probability P[W] that you wait for at
least one light? least one light?
0.5
0.5
0.8
0 8
0.2
0.2
P[G
1
G
2
] = 0.4
P[G
1
R
2
] = 0.1
P[R
1
G
2
] = 0.1
R
1
G
1
R
2
G
2
R
G
2
W = {G
1
R
2
R
1
G
2
R
1
R
2
}
P[W] = P[R
1
G
2
]+P[G
1
R
2
]+P[R
1
R
2
] = 0.1+0.1+0.4
= 0.6
0.8
P[R
1
R
2
] = 0.4
R
2
181
( Cont i nue)
What is P[G
1
|R
2
], the conditional probability of a
green first light given a red second light? green first light given a red second light?
0.5
0 5
0.8
0.2
0.2
P[G
1
G
2
] = 0.4
P[G
1
R
2
] = 0.1
P[R
1
G
2
] = 0.1
G
1
R
2
G
2
G
2
| |
| |
| |
2 . 0
5 . 0
1 . 0
|
2
2 1
2 1
= = =
R P
R G P
R G P
0.5
0.8
P[R
1
R
2
] = 0.4
R
1
R
2
182
6/7/2011
92
Ref er ences
1. Alberto Leon-Garcia, Probability and Random
Processes for Electrical Engineering,Addision- Processes for Electrical Engineering,Addision
Wesley Publishing, 1994
2. Roy D. Yates, David J. Goodman, Probability
and Stochastic Processes: A Friendly
Introduction for Electrical and Computer
Engineering, 2nd, John Wiley & Sons, Inc, 2005 g g, , y , ,
3. Jay L. Devore, Probability and Statistics for
Engineering and the Sciences, 3rd edition,
Brooks/Cole Publishing Company, USA, 1991.
183

ReviewCh2-Basic Concept Probability

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

ReviewCh2-Basic Concept Probability

Hochgeladen von

Copyright:

Verfügbare Formate

6/7/2011

7 nondefective items from the batch of 40

Das könnte Ihnen auch gefallen