3 Probability Probability is aimed to be:

A measure of uncertainty A measure of belief in uncertain circumstances

A measure of likelihood of an uncertain event

A measure with values between 0 and 1 (0% and 100%)

20

Types or concepts of probability:

Classical: Equally-likely events Frequentist: A long-run limit of the relative frequency of occurrence of an event in an experiment repeated under identical conditions. Subjective: Personal belief of the occurrence of an event (Bayesian approach) Probability theory uses mathematical set theory. A set A is dened to be a collection of elements (a 2 A, element a is a member of A). In probability theory the sets are usually called events, and the elements as objects, elementary units, elementary events, etc. In set theory the universal set (S ) is the set for which all other sets of interest are subsets (A S ). In probability theory S is usually called the sample space. Subsets of S are called events. Notations: Notation S AE A[B A\B Ac ; Set theory Universal set Sub set Union of A and B Intersection of A and B Complement of A Empty set Probability theory Sure event Event A and/or B occur A and B occur A does not occur Impossible event

21

Formally probability is dened as a set P : S !I R (real line) if it satises the following rules: (P1): P (A) 0 for all A S (P2): P (S ) = 1 (P2): For any A; B S if A \ B = ; then P (A [ B ) = P (A) + P (B ). Given these, the probability function satises e.g. the following properties: P (;) = 0 P (Ac) = 1 P (A) P (A [ B ) = P (A) + P (B ) P (A \ B ) Exercise: Prove these properties. Note: If A \ B = ; we say that the events A and B are mutually exclusive.
22

Example. Throw a die. Assuming symmetry, i.e., each outcome E = f1; 2; 3; 4; 5; 6g is equally likely, the probability is 1=6, and for P any event A, P (A) = P (fag) (Can you see why?). E.g., if a2A A = the outcome is even number, then P (A) = P (2) + P (4) + P (6) = 1=2. Example. Pick a card. Again applying symmetry, each card has the same probability of 1=52 (asses are not counted. If A is even heart then 13 P (A) = 52 What is the probability of the event "Heart or an Ace"? Let A = "Heart" and B = "Ace" then using the probability rules above P (A [ B ) = P (A) + P (B) P (A \ B ) Now A \ B ) = "Heart Ace", P (A) = 13=52, P (B ) = 4=52, and P (A \ B ) = 1=52. So P (A [ B) = 13 4 1 16 4 + = = 52 52 52 52 13

If the outcomes of the experiment cannot be considered equally likely, then the symmetry principle cannot be applied. In such circumstances there the only way to calculate the probability is to empirically run the experiment, and count the relative frequency of the events of interest and use it as an approximation of the probability.
23

Conditional probability The conditional probability of event A given the occurrence of event B is P (AjB ) = assuming P (B ) 6 = 0. Independence of events Events A and B are (statistically) independent if P (AjB ) = P (A): Then of course P (B jA) = P (B ): An equivalent denition is that A and B are independent is P (A \ B ) = P (A)P (B )
Example. Throwing a die and picking a card from a deck. What is the probability to get even number in the roll and that the card is an Ace. The events are independent, so 1 4 1 P (even \ Ace) = P (even)P (Ace) = = : 2 52 26 24

P (A \ B ) P (B )

Total Probability and Bayes' Rule Suppose that fA1; : : : ; Ang form a partition of the sample space S . Then for any B S This formula is called the Law of Total Probability. Let A and B be events, then Bayes' rule states that P (AjB )P (B ) P (B jA) = P (AjB )P (B ) + P (AjB c)P (B c)
Example. A test reveals a fatal decease in 92% of cases if the patient is infected. In the case of not infected the test falsely gives positive result (indicates infection) in 4 % cases. The occurrence of decease is in the population is rare 0.1%. Suppose a randomly selected person is tested, and the test result shows positive. What is the probability that the person is really infected? Let D = "Decease", and indicate positive result (event) simply by "+". Using the Bayes rule P (+jD )P (D ) P (Dj+) = P (+jD)P (D)+P (+jDc)P (Dc) =

P (B ) = P (A1 )P (B jA1)+ +P (An )P (B jAn ):

Note. Sometimes the Bayes' rule is called the "inverse probability".

A 1

0:920:001 0:920:001+0:040:999

= 0:0225

[ A2 [ [ An = S and Ai are mutually exclusive

25

Generalized Bayes' rule: Let B1 ; : : : ; Bn be a partition of the sample space S , then given an event A, for all j = 1; : : : ; n P (AjBj )P (Bj ) : P (Bj jA) = Pn i=1 P (AjBi)P (Bi)

P (Bi) are called prior probabilities, and P (Bj jA) posteriori probabilities, j = 1; : : : ; n.
Example. Let H; M; L denote "High", "Moderate" and "Low" economic growths with P (H ) = 0:30, P (M ) = 0:50 and P (L) = 0:20. Let A denote that dollar appreciates. Suppose we know that P (AjH ) = 0:70, P (AjM ) = 0:40, and P (AjL) = 0:20. Then e.g. P (H jA) = P (AjH )P (H ) = 0:467: P (AjH )P (H ) + P (AjL)P (L) + P (AjM )P (M )

That is, give the dollar has appreciated we can conclude that the economy is in high state growth with probability 0.467. Accordingly we can calculate the probabilities of the other two states; P (M jA) = 0:444 and P (LjA) = 0:089. Note. P (H jA) + P (M jA) + P (LjA) = 1

26