TCS3183 Random Variable and Random Processes

Chapter 1 : Review Of Probability Theory

Basic concept of probability

Probability as a general concept can be defined as the chance of an event occurring. Probability theory is used in the field of insurance, investment, weather forecasting and various other areas. The concept of probability is frequently encountered in everyday communication. Example, A physician. Most people express probabilities in terms of percentages. But, it is more convenient to express probabilities as fractions. Thus, we may measure the probability of the occurrence of some event by a number between 0 and 1.

 Experiment a situation involving chance or probability that leads to results. Outcomes - the result of a single trial of an experiment. Sample space The collection of all possible outcomes for an experiment.

Objective Probability
Classical Probability:

For example, in the rolling of the die, each of the six sides is equally likely to be observed. So, the probability that a 4 will be observed is equal to 1/6.

Equally Likely Outcomes

The set of all possible outcomes (The universal set) S

The empty set The event, E The probability of the occurrence of E

Relative Frequency Probability:

It is an approximation to the true probability of an event. However, if we were able to perform our process more and more times, the relative frequency will eventually approach the actual probability. P(E) = m/N. The example : If we tossed the two dice 100 times, 200 times, 300 times, and so on, we would observe that the proportion of 6's would eventually settle down to the true probability.

Subjective Probability:
an individual's personal judgment about whether a specific outcome is likely to occur. Subjective probabilities contain no formal calculations and only reflect the subject's opinions and past experience. The example: A Rangers supporter might say, "I believe that Rangers have probability of 0.9 of winning the Scottish Premier Division this year since they have been playing really well."

Properties of Probability
Given some process (or experiment) with n mutually exclusive events E1, E2, , En, then 1- P (Ei) 0, i = 1, 2, n 2- P (E1) + P (E2) + + P (En) = 1

Relations Between Events:

1) Union: Example:
A B means A or B.
1 2

B
6 3 9

Let S = {1,2,3,4,5,6,7,8,9,10}, 4 8 A be choosing an odd number > 2, then A = {3,5,7,9}, P(A) = 0.4 and B be choosing a number divisible by 3, then B = {3,6,9}, P(B) = 0.3. A B = {3,5,6,7,9} and P(A B) = 0.5.

5 7
A 10

2) Intersection:
A B means A and B.

Example:
In the above example, A = {3,5,7,9}, and B = {3,6,9}, then A B = {3,9} and P(A B) = 0.2.
B 1 2 4 8 6

3
9

A 5 7 10

3) Complement:
A` means the complement of A, where
A A` = S and A A` = .

Example:
In the above example, B = {3,6,9}, P(B) = 0.3, then B` = {1,2,4,5,7,8,10} and P(B`) = 0.7.

B B 1 2 4 8 6
3 9

5 7 10

Rules of Probability
1- A and B are called disjoint if A B = , and then P(A B) = 0 and P(A B) = P(A) + P(B).

For example,

if A is choosing an odd number < 11, A = {1,3,5,7,9} and B is choosing an even number < 11, B = {2,4,6,8,10}.

A
13579 2 4 6 8 10

Then P(A B) = 0 and P(A B) = P(A) + P(B) = 1.

2- If A and B are not disjoint, then P(A B) = P(A) + P(B) - P(A B)

For example,

if A is choosing a number divisible by 5 A = {5,10} and B is choosing an even number < 11, B = {2,4,6,8,10}. Then P(A B) = 0.1 and P(A B) = P(A) + P(B) - P(A B) = 0.6.

A 1 3 79 5 10 2468

Calculating The Probability of an Event Example:

Here is the data of a sample of people in a certain city:
Male (B) Diabetic (A) Normal (A`) 15 40 Female (B`) 8 62 23 102 Sum
P(A) = 23 / 125 P(A`) = 1- P(A) = 102 / 125 P(B) = 55 / 125 P(B`) = 70 / 125 P(A B) = 15 / 125 P(A B`) = 8 / 125 P(A` B) = 40 / 125 P(A` B`) = 62 / 125

Sum
B

55

70
A

125

P(A) = P(A B) + P(A B`) = 23 / 125

P(A B) = P(A) + P(B) - P(A B) =( 23 / 125) + (55 / 125) (15 / 125) = 63/125

40

15 15

62

Mutually exclusive events

Events are mutually exclusive if they cannot happen at the same time. For example, if we toss a coin, either heads or tails might turn up, but not heads and tails at the same time. Similarly, in a single throw of a die, we can only have one number shown at the top face. The numbers on the face are mutually exclusive events

 If A and B are mutually exclusive events then the probability of A happening OR the probability of B happening is P(A) + P(B). P(A or B) = P(A) + P(B)

Example
What is the probability of a die showing a 2 or a 5?

Conditional probability
Probability of an event or outcome based on the occurrence of a previous event or outcome. Conditional probability is calculated by multiplying the probability of the preceding event by the updated probability of the succeeding event.

If A and B are events with P(A) 0, then the conditional probability of B given A is defined by

Example
If a coin is tossed 3 times, what is the probability that all three tosses come up heads given that at least two of the tosses come up heads? Conditional probability of Solution:

Independence of events
Events are independent if the outcome of one event does not affect the outcome of another. For example, if you throw a die and a coin, the number on the die does not affect whether the result you get on the coin. If A and B are independent events, then the probability of A happening AND the probability of B happening is P(A) P(B). P(A and B) = P(A) P(B)

Example
If a dice is thrown twice, find the probability of getting two 5s.

Thank you
Prepared by: Nur Syahidah Ishak kavitha chandiran