Probability Review

Course : AAOC C312

Review of Basic Probability (Ch 12)
• Laws of probability
– Addition Law
– Conditional Probability
• Random Variables
• Probability Distribution
• Joint random variable
• Some Common Probability Distributions
– Binomial, Poisson, Exponential, Normal
 Probability
Probability provides a measure of uncertainty
associated with the occurrence of events or
outcomes of a random experiment.
• Experiments
• Sample Space
• Events
• 0 ≤ P(E) ≤ 1
• Impossible Event; P(Φ) = 0
• Certain Event; P(S) = 1
• Mutually Exclusive Events
• Pair wise Mutually Exclusive Events
• Equally likely Events
 Some definitions
• An experiment
– Any process that yields a result or an
observation
• Outcome
– A particular result of an experiment
• Sample space
– The set of all possible outcomes of an
experiment
 Some definitions

• An event
– Any subset of the sample space.
– If the event is A, then n (A) is the number
of sample points that belong to event A
– If the event is getting heads on a series of
coin flips, then n (heads) is the number of
heads in the sample of flips
 Mutually Exclusive Event
• Events defined in such a way that the
occurrence of one event precludes the
occurrence of any of the other events
• If one of them happens, the other cannot
happen
 Probability
Probability provides a measure of uncertainty
associated with the occurrence of events or
outcomes of a random experiment.
Definition:
If in a n-trial experiment an event E occurs
m times then the probability of occurrence of
event E is m
P[ E ] = lim
n →∞ n
By definition,
0 <= P[E] <=1
P[E] = 0 Impossible event
P[E] = 1 Sure event
Example: What is the probability of getting
even nos. in a rolling a die.
Example: What is the probability of getting
total of 7 on two dice?
 Addition law of Probability
For two events E and F, E + F represents
union, and EF represents intersection.

⎧ P{E} + P{F },
⎪
P{E + F } = ⎨ E and F are mutually exclusive
⎪ P{E} + P{F } − P{EF }, otherwise
⎩
 Conditional law of Probability
•The two events E and F with P[F] > 0, then the
conditional probability of E given F is
P{EF}
P{E | F} = , P{F} > 0
P{F}
•If E is a subset of F, then P[EF] =P[E].
•Two events, are independent if and only if,
P{E|F} =P {E}
In this case conditional probability become
P{EF} =P{E}P{F}
Problem: In a certain college, 25 percent of the
student failed mathematics, 15 percent failed
chemistry, and 10 percent failed both
mathematics and chemistry. A student is
selected at random.
• (a) if the student failed chemistry, what is the
probability that he failed mathematics?
• (b) if the student failed mathematics, what is the
probability that he failed chemistry?
• (c) what is the probability that he failed chemistry
or mathematics?
• (d) what is the probability that he failed neither
chemistry nor mathematics?
Problem: Two men A and B fire at a target.
Suppose P(A) = 1/3 and P(B) = 1/5 denote
their probabilities of hitting the target. ( we
assume that the events A and B are
independent). Find the probability that
• (a) A does not hit the target
• (b) Both hit the target
• (c) One of them hits the target
• (d) Neither hits the target.
 Bayes’ Theorem
•The two events A and B with P[B] > 0, then
P{B | A}P{ A}
P{ A | B} = ,
P{B}
•Let E be an event in a sample space S, and let A1,
A2,….An be mutually disjoint event whose union is
S. then

P{E} = P{E | A1}P{ A1} + P{E | A2 }P{ A2 } + ... + P{E | An }P{ An }

P{E | Ak }P{ Ak }
P{ Ak | E} = ,
P{E}
Problem: Three machines A, B and C produce,
respectively, 40%, 10% and 50% of the items in
a factory. The percentage of defective items
produced by the machines is respectively, 2%,
3% and 4%. An item from the factory is selected
at random.
(a) Find the probability that the item is defective
(b) If the item is defective, find the probability that
the item was produced by (i) machine A, (ii)
machine B, (iii) machine C.
 Random Variable
A real number x connected with the outcome of
random experiment.
or
A random variable x is a real valued function
defined on Sample space S.

A random variable may be discrete or continuous.

R.V. x is quantified by a probability density function.

 Probability Density Function (pdf)

Random variable X

Characteristics Discrete Continuous

Applicable range a, a+1, …, b a≤x≤b

b
b
Conditions for p(x)≥0, p ( x ) 1 f(x)≥0,
∑ =
∫ f ( x) = 1
pdf x=a a
 Cumulative distribution function
(CDF)
X
⎧
⎪⎪ P( X ) = ∑ p( x), x discrete
P{x ≤ X } = ⎨ x=a

⎪ F ( X ) = X f ( x)dx, x continuous
⎪⎩ ∫a
Problem: The number of units, x, needed of an
item is discrete from 1 to 5. the probability p(x)
is directly proportional to the number of units
needed. The constant of proportionality is K.
(a) find the pdf of x,
(b) Find the value of the constant k
(c) determine the CDF, and find the probability that
x is even value.
Problem: Consider the following function
k
f ( x) = 2
, 10 ≤ x ≤ 20
x
(a) find the value of the constant k that will make
f(x) a pdf
(b) determine the CDF, and find the probability that
x is (i) larger than 12, and (ii) between 13 and 15.
Expectation of Random Variable
• Given that h(x) is a real function of a
random variable x, we define the expected
value of h(x), E{h(x)}, as the weighted
average with respect to the pdf of x.

⎧b
⎪⎪∑ h( x) p( x), x discrete
E{h( x)} = ⎨ x = a
⎪ b h( x) f ( x), x continuous
⎪⎩∫a
 Moments of Random Variable
• The mth moment of a random variable x,
denoted by E(Xm), also called the
expected value of Xm, is defined

⎧b m
⎪ ∑ xi p( xi ), xi discrete
m ⎪ x=a
E{ X } = ⎨
⎪ b x m f ( x), x continuous
⎪⎩∫a
 Mean
The mean of x, E{x}, is a numeric
measure of central tendency of random
variable.
First moment of x.

⎧b
⎪⎪∑ xp ( x), x discrete
E{x} = ⎨ x = a
⎪ b xf ( x), x continuous
⎪⎩∫a
 Variance
The variance var{x}, is a measure of
dispersion of x around the mean

⎧b 2
⎪⎪∑ ( x − E{ x}) p ( x), x discrete
Var{x} = ⎨ x = a
⎪ b ( x − E{x}) 2 f ( x), x continuous
⎪⎩∫a

stdDev{x} = var{x}
 Problems
• Consider a random variable X that is equal to 1,2 or
3. If we know p(1) =1/2 and p(2) = 1/3 then p(3)=?
• Find E{x} and Var{x} where x is the outcome when
we are roll a fair die.
• Suppose the r.v. has a following distribution function

⎧0 x ≤ 0⎫
F ( x) = ⎨ 2 ⎬
⎩1 − exp( − x ) x > 0 ⎭
What is the probability that X exceeds 1?
 Problems
A construction firm has recently sent in
bids for 3 jobs worth (in profit) 10, 20 and
40 (thousand) dollars. If its probabilities of
winning the jobs are respectively 0.2, 0.8
and 0.3, what is the firm’s expected total
profit?
Some standard probability Distributions:

• Bernoulli trial :
P(A) =p and P(A’)=q=1-p
or
P(X=x) pxq1-x ; x=0,1
where X is a random variable represents
the outcome of the Bernoulli trial.
E[X] = p; Var [X] =pq
Binomial Distribution: Let X represent the no. of success in
n Bernoulli trials with probability of a success in each trial p
then X follows Binomial distribution with parameters n and
p. ⎛ n ⎞ x n− x
P( X = x) = ⎜ ⎟ p q , x = 0,1, 2,L , n.
⎝ x⎠

P( A) = q =1 − p.
P( A) = p,

E[X] = np; V[X] = npq.

Discrete-type random variables
1. Bernoulli: X takes the values (0,1), and
P( X = 0) = q, P( X = 1) = p.
2. Binomial: X ∼ B(n, p), in fig. (1)
⎛ n ⎞ k n−k
P( X = k ) = ⎜
⎜k ⎟⎟p q , k = 0,1,2, !, n.
⎝ ⎠
3. Poisson: X ∼ P(λ ) , in fig. (2)
λk
P( X = k ) = e −λ
, k = 0,1,2,!, ∞.
k!

P( X = k ) P( X = k )

k
12 n
Fig. 1 Fig. 2
• The Poisson distribution is an appropriate
model for the no. of occurrences of an event
over an interval when the event is as likely to
occur in one subinterval as in any other
subinterval of the same length and also the
occurrences of an event has no effect on
whether or not another occurs.
• λ is called the mean rate of occurrence.
• Additive property of Poisson Dist.: if X1,
X2…..Xn are independent Poisson random
variables with parameters λ1, λ2, …… λn
resp., then the random variable S = X1+
X2+…..+Xn also has Poisson dist. with
parameter λ1+ λ2+ ……+ λn .
 Problems
1. A fair die is rolled 10 times. What is the
probability that the rolled die will not show an
even no.?
2.Suppose that 5 fair coins are tossed
independently. What is the probability that
exactly one of the coins will be different from
the remaining four.
3. The probability that a patient recovers from a
rare blood disease is 0.4. if 15 people are
know to have contracted this disease, what is
the probability that
(a) At least 10 survive; (b) from 3 to 8 survive
and (c) exactly 5 survive?
4. A large chain retailer purchases a certain kind
of electronic device from a manufacturer. The
manufacturer indicates that the defective rate
of the device is 3%.The inspector of the
retailer randomly picks 20 items from a
shipment. What is the probability that there
will be at least one defective item among
these 20?
5. It is conjectured that an impurity exists in 30%
of all drinking wells in a certain rural
community. In order to gain some insight on
this problem , it is determined that some tests
should be made. It is too expensive to test all
of the many wells in the area, so 10 were
randomly selected for testing.
(a)Using the binomial distribution, what is the
probability that exactly 3 wells have the
impurity assuming that the conjecture is
6. Customers arrive at a service facility according to
a Poisson distribution at the rate of four per
minute. What is the probability that at least one
customer will arrive in any given 30 second
interval?
7. Ten is the average number of oil tankers arriving
each day at a certain port city. No. of the arrival of
the tankers has Poisson distribution. The facilities
at the port can handle at most 15 tankers per day.
What is the probability that on a given day tankers
have to be turned away?
8. Customers arrive randomly at a checkout counter
at the average rate of 20 per hour.
(a)Determine the probability that the counter is idle.
(b)What is the probability that at least two people in
line awaiting services?
Poisson Approx. to binomial : When
n is large and p is small (so that np
is moderate), then p.d.f. of binomial
distribution can be approximated by
p.d.f. of Poisson distribution with
parameter λ= np.
(This approximation is used if n ≥20,
p≤0.05. For n ≥ 100, p should be so
small that np ≤10.)
Mean and Variance :
E(X)= Var(X) = λ,
the parameter of the Poisson random
variable X.
This can be seen since
corresponding binomial dist has
mean np = λ and variance = npq
=λ, since q=1-p is close to 1.
Example : A space craft has 100,000 components (n →The
∞)
probability of any one component being defective is
2 × 10 −5 ( p → 0).

The mission will be in danger if five or more components

become defective. Find the probability of such an event.
Solution: Here n is large and p is small, and hence Poisson
approximation is valid. Thus np = λ = 100,000 × 2 × 10 −5 = 2,
and the desired probability is given by
4
λk −2
4
λk
P ( X ≥ 5) = 1 − P( X ≤ 4) = 1 − ∑ e −λ
= 1− e ∑ k!
k =0 k! k =0

⎛ 4 2⎞
= 1 − e −2 ⎜1 + 2 + 2 + + ⎟ = 0.052.
⎝ 3 3⎠
• Since Poisson distribution is used when n
is large and p is small, it is often called the
distribution of the numbers of rare events
over a large population.

36
Standard Continuous
Distributions

37
 Uniform Distribution
⎧ 1
⎪ , if a ≤ x ≤ b
f ( x) = ⎨ b − a
⎪0 , otherwise
⎩

⎧0 , x≤a
⎪x −a
⎪
F ( x) = ⎨ , if a ≤ x ≤ b
⎪b − a
⎩1
⎪ , x≥b
 Mean of Uniform
∞
Distribution
b
1
µ = ∫ xf (x ) = ∫ x dx
−∞ a
b − a

a+b
µ=
2

2 1 2
σ = (b − a )
12
 Exponential Distribution
• A continuous r.v. whose probability density
function is given, for some λ >0, by
−λx
⎪λe ,
⎧ if x ≥ 0
f ( x) = ⎨
⎩0
⎪ , if x < 0
It’s CDF is
−λx
F ( x) = 1 − e , x≥0
E[X] =1/λ, V[X] = 1/λ2,
• If the no. of arrivals at a service facility
during a specified time period follows
Poison distribution, then the distribution
of the time interval between successive
arrivals must be Exponential
distribution.
• If λ is the rate at which events occur,
then 1/ λ is the average time interval
between successive events.
41
Uniform: X ∼ U (a, b), a < b, if (Fig. 1)
⎧ 1
⎪ , a ≤ x ≤ b,
f X ( x) = ⎨ b − a
⎩ 0, otherwise.
⎪

Exponential: X ∼ E (λ ) if (Fig. 2)
⎧ λe − λx , x ≥ 0,
f X ( x) = ⎨
⎩ 0, otherwise.

f X (x) f X (x)
1
b−a
x x
a b
Fig. 1 Fig. 2
Normal Distribution:
Normal (Gaussian): X is said to be normal or Gaussian r.v,
if
1 − ( x − µ ) 2 / 2σ 2
f X ( x) = e .
2
2πσ

This is a bell shaped curve, symmetric around the

parameter µ , and its distribution function is given by
x 1 − ( y − µ ) 2 / 2σ 2 Δ ⎛x−µ⎞
FX ( x) = ∫ e dy = G⎜ ⎟,
−∞
2πσ 2 ⎝ σ ⎠
where G( x) = ∫ 1 e dy is often tabulated. Since f X (x)
x
− y2 / 2
−∞ 2π
depends on two parameters µ and σ 2 , the notation X ∼ N ( µ ,σ 2 )
will be used to represent f (x) X

x
µ
Fig.
 The Standard Normal
Distribution

•To find P(a < x < b), we need to

find the area under the appropriate
normal curve. There are several
such normal curves, but one of
them is called standard normal
curve.
 The Standard Normal
Distribution

• Definition : The normal distribution

with Mean = 0; Standard deviation = 1
is called the standard normal
distribution (standard normal variable
is denoted by Z).
Normal to Standard Normal
Normal
Distribution
σ
X −µ
Z=
σ
Standardized
Normal
µ X
Distribution

X=σZ + µ
 The Normal Approximation
to the Binomial
We can calculate binomial probabilities
using
• The binomial formula
• The cumulative binomial tables
When n is large, and p is not too close
to zero or one, areas under the
normal curve with mean np and
variance npq can be used to
approximate binomial probabilities.
 Approximating the Binomial
• While approximating a random
variable with integer values by a
continuous random variable, use
continuity correction..
• . In this, the integer value x0 of
discrete random variable is replaced
by the interval (x0 –1/2, x0 +1/2] of
the continuous random variable.
• Thus if a and b are integers, and X* is
a continuous random variable
approximating discrete random
variable X then
P(a ≤ X ≤ b) = P(a – ½ < X* ≤ a + ½)
whereas
P(a < X < b) = P(a+1 ≤X ≤ b-1)
=P(a + ½ < X* ≤ b – ½).
• But note that 3.7 ≤ X ≤ 5.4 if and only
if 4 ≤ X ≤ 5. Also 3.7 < X < 5.4 if
and only if 4 ≤ X ≤ 5. Thus both can
be approximated by interval 3.5 < X*
≤ 5.5 of the approximating
continuous random variable X*.
• Similar considerations are used when
only one of the extreme values of X is
not an integer or both are not
integers.
•Make sure that np and nq are both
greater than 15 to avoid inaccurate
approximations!
Exercise :The probability that an
electronic component will fail in less
than 1200 hours of continuous use is
0.2. Use normal approximation to find
the probability that among 250 such
components, fewer than 50 will fail in
less than 1200 hours of continuous
use.
X= no. of electronic components among 250
randomly chosen which fail in less than
1200 hours. X has binomial distribution
with n=250, p= 0.2. Can approximate X by
normal random variable X* with mean
(250)(0.2)=50 and variance =
(50)(0.8)=40.
σ=6.324.
Z=(X*-50)/6.324 has standard normal dist.
P(X<50) = P(X ≤ 49) = P(X* ≤ 49.5) =
P(Z ≤ (49.5-50)/6.324) =
F(0.079)=0.4681
 Central Limit Theorem
• Let x1, x2, …, and xn be independent and
identically distributed random variables,
each with mean µ and standard deviation
σ, and defined sn = x1+x2+….+xn.
As n become large (n→∞), the distribution
of sn becomes normal with mean nµ and
variance nσ2, regardless of the original
distribution of x1, x2, …, and xn.
. -
0.2266
From (1) and (2) we get z = -.37.
(b) P(-z < Z < z) = .9298
or F(z) – F(-z) = .9298
or F(z) – (1 - F(z)) =.9298
or 2F(z) = 1.9298
or F(z) = P(Z ≤ z)= .9649
from table, z =1.81.
73.3
 Joint random variable
Consider the two continuous r.v’s x1, a1≤x1≤ b1,
and x2, a2≤ x2 ≤ b2. Define f(x1, x2) as the joint
pdf of x1 and x2 and f1(x1) and f2(x2) as the
marginal pdfs of x1 and x2 respectively. Then
f(x1,x2) ≥ 0, a1≤x1≤ b1, a2≤ x2 ≤ b2
b1 b2
∫ dx1 ∫ f ( x1 , x2 )dx2 = 1
a1 a2
b2
f1 ( x1 ) = ∫ f ( x1 , x2 )dx2
a2
b1
f 2 ( x2 ) = ∫ f ( x1 , x2 )dx1
a1

f ( x1 , x2 ) = f1 ( x1 ) f 2 ( x2 ), if x1 and x2 are independent

• E[c1x1+c2x2]=c1E[x1]+c2E[x2]
• Var[c1x1+c2x2]=c12var[x1]+c22Var[x2]
+2c1c2 cov{x1x2}
• Cov{x1x2}=E[x1x2] – E[x1]E[x2]
Example: The joint pdf of x1 and x2, P(x1,x2), is
x2 = 3 x2 = 5 x 2 = 7
x1 = 1 0.2 0 0.2
x1 = 2 0 0.2 0
x1 = 3 0.2 0 0.2

(a)Find the marginal pdfs p1(x1) and p2(x2).

(b)Are x1 and x2 independent?
(c)Compute E{x1 + x2}
• Example: 12.3-3
A lot includes four defectives (D) items and
six good (G) ones. You select one item
randomly and test it. Then, without
replacement, you test a second item. Let
the r.v.’s x1 and x2 represents the outcomes
for the first and second item, respectively.
a) Determine the joint and marginal pdfs of x1
and x2.
b) Suppose that you get $ 5 for each good
item you select but pay $ 6 if it is defective.
Determine the mean and variance of your
revenue after two items have been selected.