Probabilistic thinking

Nur Aini Masruroh

Events Event is a distinction about some states of the world Example:

Whether the next person entering the room is a beer drinker The date of the next general election Whether it will be raining tonight Our next head of department Etc

Clarity test
When we identify an event, we have in mind what we meant. But will other people know precisely what you mean?
Even you may not have precise definition of what you have in mind

To avoid ambiguity, every event should pass the clarity test

Clarity test: to ensure that we are absolutely clear and precise about the definition of every event we are dealing with in a decision problem

The clarity test is conducted by submitting our definition of each event to a clairvoyant A clairvoyant is a hypothetical being who is:
Competent and trustworthy Knows the outcome of any past and future event Knows the value of any physically defined quantity both in the past and future Has infinite computational (mental) power and is able to perform any reasoning and computation instantly and without any effort

Clarity test (contd) Passing the clarity test:

If and only if the clairvoyant can tell its outcome without any further judgment

Example:
The next person entering this room is a beer drinker
What is a beer drinker? What is a beer?

The next person entering this room is a graduate

What is a graduate?

Possibility tree Single event tree

Example: event the next person entering this room is a businessman Suppose B represents a businessman and B otherwise,

Possibility tree Two-event trees

Simultaneously consider several events Example: event the next person entering this room is a businessman and event the next person entering this room is a graduate can be jointly considered

Reversing the order of events in a tree

In the previous example, we have considered the distinctions in the order of businessman then graduate, i.e., B to G. The same information can be expressed with the events in the reverse order, i.e., G to B.

Multiple event trees

We can jointly consider three events businessman, graduate, and gender.

Using probability to represent uncertainty

Probability: Frequentist view
Probabilities are fundamentally dispositional properties of nondeterministic physical systems Probabilities are viewed as long-run frequencies of events This is the standard interpretation used in classical statistics

Subjective (Bayesian) view

Probabilities are representations of our subjective degree of belief Probabilities in general are not necessarily ties to any physical or process which can be repeated indefinitely

Assigning probabilities to events

To assign probabilities, it depends on our state of information about the event Example: information relevant to assessment of the likelihood that the next person entering the room is a businessman might include the followings:
There is an alumni meeting outside the room and most of them are businessman You have made arrangement to meet a friend here and she to your knowledge is not a businessman. She is going to show up any moment. Etc

After considering all relevant background information, we assign the likelihood that the next person entering the room is a businessman by assigning a probability value to each of the possibilities or outcomes

Marginal and conditional probabilities

In general, given information about the outcome of some events, we may revise our probabilities of other events We do this through the use of conditional probabilities The probability of an event X given specific outcomes of another event Y is called the conditional probability X given Y The conditional probability of event X given event Y and other background information , is denoted by p(X|Y, ) and is given by

p( X | Y , )

p( X Y | ) p(Y | )

for p(Y | ) 0

Factorization rule for joint probability

Changing the order of conditioning

Suppose in the previous tree we have

There is no reason why we should always conditioned G on B. suppose we want to draw the tree in the order G to B Need to flip the tree!

Flipping the tree

Graphical approach
Change the ordering of the underlying possibility tree Transfer the elemental (joint) probabilities from the original tree to the new tree Compute the marginal probability for the first variable in the new tree, i.e., G. We add the elemental probabilities that are related to G1 and G2 respectively. Compute conditional probabilities for B given G

Bayes theorem
Doing the above tree flipping is already applying Bayestheorem

Bayes Theorem
Given two uncertain events X and Y. Suppose the probabilities p(X|) and p(Y|X, ) are known, then

p ( X | ) p (Y | X , ) p( X | Y , ) p (Y | ) where p (Y | ) p ( X | ) p (Y | X | )
X

Probabilistic dependency or relevance

Let
A be an event with n possible outcomes ai, i=1,,n B be an event with m possible outcomes bj ,j=1,,m

Event A is said to be probabilistically dependent on event B if p(A|bj, ) p(A|bk, ) for some j k

The conditional probability of A given B is different for different outcomes or realizations of event B. we also say that B is relevant to A

Event A is said to be probabilistically independent on event B if p(A|bj, ) = p(A|bk, ) for all j = k

The conditional probability of A given B is the same for all outcomes or realizations of event B. we also say that B is irrelevant to A

In fact, if A is independent of B, then p(A|B, ) = p(A|) Intuitively, independence means knowing the outcome of one event does not provide any information on the probability of outcomes of the other event

Joint probability distribution of independent events In general, the joint probability distribution for any two uncertain events A and B is p(A, B|)=p(A|B, )p(B| ) If A and B are independent, then since p(A|B,)=p(A| ), we have p(A, B|)=p(A|) p(B|) The joint probability of A and B is simply the product of their marginal probabilities In general, the joint probability for n mutually independent events is p(X1, X2, , Xn|)=p(X1|) p(X2|) p(Xn-1|) p(Xn|)

Conditional independence or relevance

Suppose given 2 events, A and B, and they are found to be not independent Introduce event C with 2 outcomes, c1 and c2 If C=c1 is true, and we have p(A|B, c1, )=p(A|c1, ) If C=c2 is true, we have p(A|B, c2, )=p(A|c2, ) Then we say that event A is conditionally independent of event B given event C Definition (Conditional Independence): given 3 distinct events A, B, and C, if p(A|B, ck, )=p(A|ck, ) for all k, that is the conditional probability table (CPT) for A given B and C repeats for all possible realizations of C, then we say that A and B are conditional independent given C, and denote by A B | C

Conditional independence (contd)

If A B | C then p(A|B, C, )=p(A|C, ) Example: Given the following conditional probabilities: p(a1|b1, c1)= 0.9 p(a2|b1, c1)= 0.1 p(a1|b2, c1)= 0.9 p(a2|b2, c1)= 0.1 p(a1|b1, c2)= 0.8 p(a2|b1, c2)= 0.2 p(a1|b2, c2)= 0.8 p(a2|b2, c2)= 0.2 we conclude that A B | C

Note that A is not (marginally) independent of B unless we can show that p(a1|b1) = p(a1|b2) with more information

Join probability distribution of conditional probability distribution Recall, by factorization rule, the joint probability for A, B, and C is p(A, B, C|)= p(A| B, C, )p(B|C, )p(C| ) If A is independent of B given C, then since p(A| B, C, ) = p(A|C, ) we have p(A, B, C|)= p(A| C, )p(B|C, )p(C| )

Application of conditional probability

Direct conditioning: Relevance of smoking to lung cancer Suppose: S: A person is a heavy smoker which is defined as having smoked at least two packs of cigarettes per day for a period of at least 10 years during a lifetime L: A person has lung cancer according to standard medical definition A doctor not associated with lung cancer treatment assigned the following probabilities:

Relevance of smoking to lung cancer (contd)

A lung cancer specialist remarked: The probability p(L1|S1, ) = 0.1 is too low When asked to explain why, he said: Because in all these years as a lung cancer specialist, whenever I visited my lung cancer ward, it is always full of smokers. Whats wrong with the above statement? The answer can be found by flipping the tree:

Relevance of smoking to lung cancer (contd) What the specialist referred to as high is actually the probability of a person being a smoker given that he has lung cancer, i.e., p(S1|L1, ) = 0.769 is exactly what he was referring to. He has confused p(S1|L1, ) with p(L1|S1, ) Notice that p(L1|S1, ) << p(S1|L1, ) Hence even highly a trained professional can fall victim to wrong reasoning

Lets Make a Deal Game Show

Rules: Consider the TV game show where the contestant is shown o stage three boxes, one of which contains a valuable prize; the other two are empty The rules of the game are that the contestant first chooses one of the boxes. Then, the game show host who knows the location of the prize opens one of the remaining two boxes, making sure to open an empty one. the contestant then gets to decide if he wants to stick with his initial selection or switch to the remaining unopened box. If the prize is in the box that he finally chooses, he wins the prize Question: If at the start of the game the contestant chose box A and the host open box B, should the contestant keep choosing box A or swicth to box C?

Updating probabilities based on new evidence or information

Recall: p(A|) is probability of event A based on our subjective assessment of the likelihood of event using any information we have prior probability If new information E has arrived, then the probability of A is updated using Bayes Theorem:
p( A | E ) p ( A) p ( E | A) p( E )

where p(E|A) is called the likelihood function for the evidence E and

p( E ) p ( A j ) p( E | A j )
j

Example: weather forecast

Suppose, the prior probability that it will rain tonight (R1) is 0.6 and it will not rain (R2) with probability 0.4 Suppose we are using information from the weather forecast whose performance is as follows

If the weather station announces that it will rain tonight, what probability should you assign to the outcome it will indeed rain tonight?

Weather forecast (contd)

Try to use Bayes theorem!

Another example
In the city, there are only two taxicab companies, the Blue and the Green. The Blue company operates 90% of all cabs in the city and the Green company operates the rest. One dark evening, a pedestrian is killed by a hit-and-run taxicab. There is one witness to the accident. In court, the witness ability to distinguish cab colors in the dark is questioned so he is tested under conditions similar to those in which the accident occurred. If he is shown a green cab, he says it is green 80% of the time and blue 20% of the time. If he is shown a blue cab, he says it is blue 80% of the time and green 20% of the time. The judge believes that the test accurately represents the witness performance at the time of the accident. Construct the probability tree representing the judges state of information! If the witness says The cab involved in the accident was green. What probability should the judge assign to the cab involved in the accident being green?

Thank you