PROBABILITY THEORY AND STOCHASTIC PROCESSES

Unit-I

Total Probability and Bayes Theorem

Total probability theorem:
The probability P(A) of any event A defined on a sample space S can be expressed in terms of
conditional probabilities.
Statement:
Bn , n 1, 2,...N
Suppose we are given N mutually exclusive events
shown in Fig 4.1.These events satisfy
Bm Bn
N

UB

, whose union equals S as

m n 1, 2,..., N

n 1

P ( A) P ( A / Bn ) P ( Bn )
n 1

Prove that

Which is known as total probability of event A.

Proof:
We know that

A S A

, therefore

n 1

n 1

A S A (U Bn ) U( A Bn )

A Bn
Now the events
are mutually exclusive as seen from Venn diagram. By applying axiom 3
to these events, we have
N

n 1

n 1

P ( A) P( A S ) P[U( A Bn )] P ( A Bn )

Hence proved.

PROBABILITY THEORY AND STOCHASTIC PROCESSES

Unit-I

Fig 4.1: Venn diagram of N mutually exclusive events Bn and another event A.
Bayes Theorem:
We can obtain Bayes theorem from definition of conditional probability and total probability.
The conditional probability can be written in another way as,
P( Bn / A)

P( Bn A)
P( A)

P ( A) 0

if
Where Bn is one of the events defined above in total probability.
Or, alternatively
P( A / Bn )

P( A Bn )
P( Bn )

P ( B) 0

if
Bayes theorem is obtained by equating the above two expressions,
P( Bn / A)

P ( A / Bn ) P ( Bn )
P( A)

Substituting expression for P(A) from total probability the above equation becomes,
P( Bn / A)

P( A / Bn ) P ( Bn )
P ( A / B1 ) P ( B1 ) ...... P( A / BN ) P( BN )

for n=1,2,.,N.

PROBABILITY THEORY AND STOCHASTIC PROCESSES

P( Bn )
The terms

P ( Bn / A)
are a priori and

are a posteriori probabilities.

Unit-I