Topic: Probability

KNF2053
Numerical Methods and Statistics

Ade Syaheda Wani Binti Marzuki

Faculty of Engineering
Semester 2 (2010/2011)

Probability

The ratio of the number of occurrences of an

event to the number of all possible occurrences
of equally likely and mutually exclusive events.

An idealization of the proportion of times that

an event would be expected to occur in
repeated trials.

Definition of Probability

If an experiment can result in any one of N

different equally likely outcomes, and if
exactly n of these outcomes correspond to
event A, then the probability of event A is:
n
P(A)
N

Example: Getting a head when a coin is tossed

once.
1
P(head)
2

Random Experiment

An experiment that satisfies the following

conditions
a) The outcome of experiment cannot be
predicted with certainty.
b) The experiment can be repeated under
the same condition.

A random experiment always denotes as

E.

Sample space, S

The set of all possible outcomes of a

statistical experiment.

An event is a subset of a sample space.

S S, the sample space S itself is an

event, called the certain event.

S , is called the null event.

Sample space, S
Example 1
Experiment: E :Tossing a die
Sample Space: S = {1,2,3,4,5,6}
Event:

A = {getting an even number}

P (A) = 3/6 = 1/2

Sample space, S
Simple exercise
A dice is tossed once.
a) What is the sample space, S?
b) Give the set of the following events
i.

the number shows on top is less than 3

ii.

the number shows on top is more than 4

Sample space, S
Simple exercise
A coin is tossed twice
a) What is the sample space, S?
b) Give the set of the following events:
i.

A : head appear once

ii.

B : head appear at least once

iii. C : head appear twice

Probability Rules for Sample

Points

All sample point (event ) probabilities

must lie between 0 and 1.

The probabilities of all the sample points

within a sample space must sum to 1.

E.g: A coin is tossed once

1 1
P(head) P(tail) 1
2 2

Relations between event A and

B
S

Include
A

B include A is denoted as B A.
All the elements of A is also
belonging to B.

Equal
A equal to B is denoted A = B. It
means A B and B A. A
happen if and only if B happen.

S
A=B

Relations between event A and

B
S

Union
Denoted by the symbol A B,
is the event containing all the
elements that belong to A and
B or both.

Intersection
Denoted by the symbol A B,
is the event containing all
elements that common to A and
B

Relations between event A and

B
S

Mutually exclusive or
disjoint
A and B are mutually exclusive,
or disjoint if A and B cannot
happen at the same time
(A B = )

Complement
The complement of an event A
with respect to S is the subset
of all elements of S that are not

A
B

S
A'
A

Axioms of Probability

For every A in S, 0 < P(A) < 1

The entire S has the probability P(S) = 1

P () = 0

P (A) = 1 P(A)

If A B, then P(A) < P(B)

If A and B are mutually exclusive, then:

P ( A B) P ( A) P( B )

In many cases, events will not be mutually

exclusive
P ( A B) P( A) P( B ) P ( A B)

Example 1
The probability that John passes Mathematics is 2/3,
and the probability he passes English is 3/9. If the
probability of passing both subjects is 1/4, what is
the probability that John will pass at least one of
these subjects?
Let M : the event passing Math
Let E : the event passing English

Example 1-Solution
The probability that John will pass at least one of
these subjects.
Let M : the event passing Math
Let E : the event passing English

P( M E ) P( M ) P( E ) P( M E )
2 3 1

3 9 4
3

Probability Multiple Principles

According to multiplication principle,

If the first sampling can be performed in m ways.
Second sampling can be performed in n ways

These two operations can be performed in

ways

mn

Lets look at an examples of counting sample with

replacement and without replacement

Sampling with replacement

Consider this case!
How many ways are to choose 3 balls with
replacement from a box, which contains 5 distinct
balls?
First attempt

:5 ways to choose first ball

Second attempt :5 ways to choose second ball

Third attempt :5 ways to choose third ball
Total ways to choose the ball = 5 5 5 = 125

Sampling without replacement

Consider this case!
How many ways are to choose 3 balls without
replacement from a box, which contains 5 distinct
balls?
First attempt

:5 ways to choose first ball

Second attempt :4 ways to choose second ball

Third attempt :3 ways to choose third ball
Total ways to choose the ball = 5 4 3 = 60

Permutation

Ways to arrange things

Arrangement of all or part of set of objects

Given n distinct objects, we wish to choose r of

these objects 0 < r < n and arrange the chosen r
objects.

The number of permutations of size r that can be

constructed from n objects denotes by nPr
n

n!
Pr n: no. of object
(n r )!
r: no. of ways

Permutation

Permutation without repetitions

n

n!
Pr
n: no.
n ofr!object
r: no. of ways

Permutation with repetitions

n

Pr n

Example 2

In a coded telegram the alphabet letters are

arranged in groups of five letters. What is the
arrangement of such words.
n

Pr 265
11881376

If the letters are taken no more than once,

n

n!
26!
Pr

7893600
(n r )! (26 5)!

Combination

Ways to select things

Denotes by nCr

For example, taking a combination of letter a, b, c

and two letter taken at a time.

Without repetition:
3 combinations ab, ac and bc

With repetitions:
6 combinations aa, ab , ac, bb, bc and cc

Combination

Combination without repetitions

n

Combination with repetitions

n

n
n!
Cr
r r!(n r )!

n r 1
(n r 1)!

Cr
r r!((n r 1) r )!

Every combination has r! of ways to rearrange

them (with regard to order)
n

Pr nCr r!

Example 3

Again by using the same previous example,

Consider a case of taking letters a, b, c from a

box. a) Find the combination of taking 2 letter from
a box
i) Without repetitions
ii) With repetitions:
b)How many ways to arrange the combinations in
both cases

Example 3 - Solution
a)

i) Combinations without repetitions

n

n
3!
Cr
3
r 2!(3 2)!

ii) With repetitions:

n

n r 1
4!

Cr
6
r 2!(4 2)!

Example 3 - Solution
b)

i) Permutations without repetitions

n

Pr nCr r!
3 2!
6

ii) With repetitions:

n

Pr nCr r!
6 2!
12

Conditional Probability

One occurrence event can increase the chances

of occurrence of another event

Example:
A : rain next Sunday
B : rain next Saturday

The occurrence of B increase the chance of

occurrence A.

Conditional Probability

For any two events, A and B with P(B) > 0. The

conditional probability of A given B has occurred
is defined by:

P( A B)
P( A | B)
P( B)

Example 4

Consider a case of tossing a dice.

Let event A = {1,2,3} and event B = {1,3,5}.
Find P (A|B)

Example 4 - Solution

event A = {1,2,3}

event B = {1,3,5}
P( A B)
P( B)
A B {1,3}
2 1
P( A B)
6 3
3 1
P( B)
6 2
P( A | B)

1
P( A B)
2
3
P( A | B)

1
P( B)
3
2

Multiplicative Rule

The probability of an intersection of two events

can be calculated using multiplicative rule, which
employs the conditional probabilities we defined
in the previous section.

The multiplicative rule of probability is

P ( A B ) P ( A) P ( B | A)
or equivalently

P( A B) P( B) P( A | B)

Independent Events

Two events A and B are independent if and only

if:

P( A B) P( A) P( B)

Remarks:
When P(B) > 0, A and B are independent
P(A|B) = P(A)
When P(A) = 0 or P(B) = 0, A and B are
independent

Example 5
Lets consider when a coin is tossed twice. Let A
= head appears in the first time and B = Tail
appears in the second time. Show that A and B
are independent events.

Example 5 - Solution
S = {(H,H), (H,T), (T,H), (T,T)}
A = head appears in the first time = {(H,H),
(H,T)}
B = Tail appears in the second time = {(H,T),
(T,T)}
P(B) > 0, so:
P( A | B)

P( A B)
1
4

2
P( B)
2
4

Bayes Rule

Let A and B be two events,

S
A

We may express A as
A B'

A B

A ( A B) ( A B ' )

As AB and AB are clearly mutually exclusive,

then

P ( A) P( A B) P( A B' )
P( A | B) P( B) P( A | B' ) P( B' )

Bayes Rule
Definition:

A sequence of event A1, A2,, An represents a

partition of the sample space S if
a) Ai Aj = , for every 1 < i j < n
b) Total union of Ai = S
c) P(Ai ) > 0 for all i = 1,2,, n

S
A3
A1

A2

A4

A5

Bayes Rule: Theorem of

Probability

Let A1, A2, An be a partition of the sample space

S. Then for any other event B associate with S
n

P( B) P( B Ai )
i 1
n

P( B | Ai )P( Ai )

S
B

A3

i 1

A2
A1

A4

A5

Bayes Rule

Let A1, A2, An be a partition of the sample space

S. Then for any other event B associate with S for
which P(B)>0

P( Ai B )
P( B | Ai ) P( Ai )
P( Ai | B )
n
P( B)
P( B | Ai ) P( Ai )
i 1

For i = 1, 2, , n

Example 6
In a factory, production line A,B and C are all
producing boxes with the same size. On their
production, production line A, B and C produce 1%,
2% and 3% defective boxes, respectively. Of the
total production, line A produces 25%, line B
produces 35% and line C produces 40%. A box is
selected randomly from the total production of a day
a) what is the probability that it is defective?
b) if the box is defective, what is the conditional
probability that it was produced by line C?

Example 6 - Solution
Let D = defective box
A = box from line A
B = box from line B
C = box from line C
a)

P ( D) P( D A) P ( D B) P ( D C )
P( D | A) P( A) P( D | B ) P( B ) P ( D | C ) P (C )

1 25 2 35 3 40

100 100 100 100 100 100

215
43

10000 2000

Example 6 - Solution
b) By Bayes Theorem

P(C ) P ( D | C )
P (C | D)
P( D)
40
3
100 100

43
2000
24

43