PROBABILITY

The tool that will allow us to bridge the gap from descriptive to inferential well start by using simple problems, in which probability can be calculated by merely COUNTING

Flipping a Coin
Say I flip a coin...
OMG Heads!!!! Do you care? Why Not?

Sample Space:
(draw on board) collection of all possible outcomes for a given phenomenon coin toss: {H,T}
mutually exclusive: either one happens, or the other

Flipping a Coin
Probability(Heads)? So.... must the next one be Tails? No!
Independent trials Random Phenomenon:
cant predict individual outcome can predict pattern in the LONG RUN

Probability: relative # times something happens in the long run

2 Coin Flips
OMG 2 Heads!
impressed yet?

Sample space
(draw on board) Prob(2 Heads): 1/4 outcome: single observation

OMG 2 of same!
Prob(2 Heads OR 2 Tails): event: subset of the sample space made of 1 or more possible outcomes

Larger Point
OMG 30 Heads in a row!
NOW maybe youre finally interested...

OMG drew 3 yellow cars!

interesting? boring? cant tell!

Descriptive Stats: measuring & summarizing outcomes Inferential Stats: to understand some outcome, must consider it in context of all possible outcomes that couldve occurred (sample space)

Counting Rules
Count up the possible outcomes
that is: define the sample space

2 Main ways to do this:

Permutations
when order matters

Combinations
when order doesnt matter

Probability: Multiple Events

Mutually Exclusive Events Independent Events Dependent Events
(Conditional Probability!)

Venn Diagrams
Sample Space: set of all possible outcomes of a random phenomenon
big rectangle (defined by Researcher [e.g., YOU!])

Event: subset of the possible outcomes

circles (or whatever) (also defined by Researcher!)

Populations...

Venn Diagrams
Probability:
# outcomes in some event/subset # outcomes in Sample Space area of some event/subset area of Sample Space

Intersection & Union

Intersection:
AB this means: BOTH/AND/ALL

Union:
AUB this means: EITHER/OR/ANY
could be both

Mutually Exclusive Events

(a.k.a. Disjoint) ex: 1 Coin toss: Heads;Tails ex: 1 Student: Freshman;Senior Intersection: p(AB)= 0 Union: p(AUB)= p(A)+p(B) A

Independent Events
(unrelated events) ex: Multiple coin tosses: Heads 1st time; Heads 2nd time ex: Draws w/ replacement: Red Car 1st time; Yellow Car 2nd time Intersection:
p(AB)= p(A)*p(B)

Union:
p(AUB)= p(A)+p(B) -p(AB)

Dependent Events
(related events) Occurrence of A affects p(B) ex: Draws w/o replacement: Red Car 1st time; Yellow Car 2nd time
Sequential

ex: Random person: Height; Weight

Partial Information

Intersection & Union?... ~~>Conditional Probability

Conditional Probability
p(B|A)
means: The probability of B, GIVEN the occurrence of A No formula needed for the following: Mutually Exclusive Events: p(B|A)= 0 Independent Events: p(B|A)= p(B)

Conditional Probability
Formula:

p(B|A) =

p(BA) p(A)

Conditional Probability
p(BA) p(B|A)= p(A) Dependent Events: Example
100 people at party Could like: Cola, Lime, Both, or Neither
p(Cola) = p(Lime) = p(ColaLime) =

You see someone drinking Cola...

p(Lime|Cola) =

Conditional Probability
p(BA) p(B|A)= p(A) Notes:
The GIVEN becomes the denominator. ~like zooming in on a subset

Back to Dependent Events:

Intersection & Union?

Intersection:
from

p(BA) p(B|A)= p(A) p(BA) = p(A)*p(B|A)

we get:

Union:
stays the same as for ind. events:

p(BUA) = p(A)+p(B)-p(BA)

Intersection & Union

Intersection:
P(A B) = P(A)*P(B) (If mutually exclusive = 0)

Union:
P(A U B) =P(A)+P(B)- P(AB)

Compliment:
p(A)=1-p(A)

Independent vs. Dependent Events

Independent Events: unrelated events that intersect at chance levels given relative probabilities of each event Dependent Events: events that are related in some way Concepts of union and intersection are the same However, P(AB) P(A)*P(B)

Do you think mutually exclusive events are dependent or independent?

Conditional Probability

p(B|A) =
Conceptually this means:

p(BA) p(A)

Bayes Theorem
p(B|A) =
p(A|B)p(B) p(A|B)p(B) + p(A|B)p(B)

Can we break this down a little to understand it better?

p(A|B)*p(B)=p(AB)

p(A|B)*p(B) + p(A|B)*p(B)
= p(AB) + p(AB) = p(A) p(B|A) = So, this is just: p(BA) p(A)

Law of Total Probabilities

_ B

p(A) = p(AB) + p(AB) p(A) = p(A|B)p(B) + p(A|B)p(B)

General Rules!
Intersection: Multiplication Rule
p(BA) = p(A)*p(B|A)

Union: Addition Rule

p(BUA) = p(A)+p(B)-p(BA)

Complement Rule:
p(A)=1-p(A)

Plus: definition of mutually exclusive events, independent events, and conditional probability

Tree Diagrams
Independent Events
all branches at a given level have same probability

Dependent Events
probability of branch at a given level is conditional on which branch of prior level

Law of Total Probabilities

_ B

p(A) = p(AB) + p(AB) p(A) = p(A|B)p(B) + p(A|B)p(B)

Bayes Theorem
(Reverse Conditional Probability)

what if we know: p(A|B) but we WANT to know: p(B|A) ? if you know p(B), you can do it! (get ready...)

Bayes Theorem
(Reverse Conditional Probability) p(B|A) =

p(A|B)p(B) p(A|B)p(B) + p(A|B)p(B)

Ex: HIV testing A=test positive B=person truly has HIV Tests Hit Rate: p(A|B)=.95 Tests Correct Rejection Rate: p(A|B)=.95 But: say someone gets a positive test result... false positive rate? say base rate of HIV: p(B)=.005 calculate false positive rate: p(B|A)=...

Venn Diagrams & Some Terms

Universal Set (W): the largest set (sample space) Mutually Exclusive Events: events that never occur together Sure Event: event that always occurs Impossible Event: event that never occurs (empty set) Exhaustive Events: If events are equal to the universal Set

Venn Diagrams and Set Theory

Now, we can count how many sample points fall in each category.
How do we talk about combinations of categories (e.g. how many people favored a team(s) in the super bowl?) Union: combination of two sets
C= A U B

Intersection: contains only objects common to A and B

D=AB

Special Lecture: Conditional Probability

Example of Conditional Probability in the real world:
This chart is from a report from the CA Dept of Forestry and Fire Prevention. It shows the probability of a structure being lost in a forest fire given its location in El Dorado county. (calculated using fuel available, land slope, trees, neighborhood etc.)

The Plan
I plan to cover material related to these topics. Specifically, well Review all the formulas well need. Go over one conceptual example in depth. Work through a number of the problems that have been giving you trouble. Address any specific questions/problems.

Formula s:
Event Probability
A
Not A A or B
(or both)

Terms/Explanation
probability of A is between 0 and 1
Compliment: Note that the probability of either getting A or not getting A sums to 1.

p(A) [0,1]
p(A) = 1 - p (A)

p(AB) = p(A) + p(B)-p(AB) Union: =p(A) + p(B) if A & B are mutually exclusive p(AB) = p(A)p(B) = p(A|B)p(B) P(A|B) = p(AB)/p(B)
Intersection: only if A and B are independent Conditional Probability: The probability of event A given that you already have event B.

A&B

A given B

Formula s: Bayes Theorem:

This is simply derived from what we already know about conditional probability. p(A|B) = p(B|A)*p(A)

p(B)
Or if we dont have p(B) we can use the more complicated variation of Bayes:

p(A|B) =

p(B|A)*p(A) p(B|A)*p(A) +p(B|A)*p(A)

The reason those two formulas are the same has to do with the

Law of Total Probabilities:

For any finite (or countably infinite) random variable,

p(A) = p(ABn) or, p (A) = p(A|Bn)p(Bn)

Formulas: All together now

Event Probability
A
p(A) [0,1] p (A) = p(ABn) = p(A|Bn)p(Bn) p(A) = 1 - p (A)

Terms/Explanation
probability of A is between 0 and 1 And is the sum of all partitions of A
Compliment: probability of either getting A or not getting A sums to 1.

Not A
A or B
(or both)

p(AB) = p(A) + p(B)-p(AB) Union: only if A & B are mutually =p(A) + p(B) exclusive p(AB) = p(A)p(B) = p(A|B)p(B) = p(B|A)p(A) P(A|B) = p(AB)/p(B) = p(B|A)p(A)/p(B) = p(B|A)p(A)
Intersection: only if A and B are independent

A&B

A given B

Conditional Probability: The probability of event A given that you already have event B.

Shapes Demo
Imagine that we have the following population of shapes: Notice that there are several dimensions that we could use to sort or group these shapes: Shape Color Size We could also calculate the frequency with which each of these groups appears and determine the probability of randomly selecting a shape with a particular dimension from the larger set of shapes. So lets do that

Shapes Demo
Imagine that we have the following population of shapes:

P(R) = 8/24 = 1/3 P(Y) = 8/24 = 1/3 P(B) = 8/24 = 1/3 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P(BIG) = 12/24 = 1/2 P(small) = 12/24 = 1/2

Now that weve figured out the probability of these events, What else can we do?

P(R) = 8/24 = 1/3 P(Y) = 8/24 = 1/3 P(B) = 8/24 = 1/3 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P(BIG) = 12/24 = 1/2 P(small) = 12/24 = 1/2

Now that weve figured out the probability of these events, What else can we do? P(R) = 8/24 = 1/3 Lots of stuff! Whats the probability of getting a blue triangle?

p( ) = p(B ) = p(B)*p( ) = 8/24 * 6/24 = 48/576 = 2/24 = 1/12

P(Y) = 8/24 = 1/3 P(B) = 8/24 = 1/3 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P(BIG) = 12/24 = 1/2 P(small) = 12/24 = 1/2

Now that weve figured out the probability of these events, What else can we do? P(R) = 8/24 = 1/3 Lots of stuff! What else? p( ) = p(B ) = 1/12

p(

or B or ) = p(B )
= p(B )+p( )- p(B ) = 8/24 +6/24 - 1/12 =12/24 =1/2

P(Y) = 8/24 = 1/3 P(B) = 8/24 = 1/3 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P(BIG) = 12/24 = 1/2 P(small) = 12/24 = 1/2

Now that weve figured out the probability of these events, What else can we do? P(R) = 8/24 = 1/3 Lots of stuff!

What else? p( ) = p(B ) = 1/12 p( or B or ) = p(B )=1/2 p( given that we have B) = p( |B) = p(B ) /p(B) = 2/24 / 8/24 = 2/8
= 1/4

P(Y) = 8/24 = 1/3 P(B) = 8/24 = 1/3 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P( ) = 6/24 = 1/4 P(BIG) = 12/24 = 1/2 P(small) = 12/24 = 1/2

Set Theory and Probability

Once we know number of total events and number of events in each category, we can calculate the probability of obtaining a result in any one category. P(A)= A/W P(B)= B/W P(A B) = A B/W P(A U B) = AUB/W

Set Theory
Set Compliment A = 1 - A Subsets
Proper Subset (all objects in subset are not only objects in set) C A Subset (all objects in subset could be only objects in set C A

Set Theory Calculations

P(A B) = P(A) * P(B)
P(A U B) = P(A) + P(B)- P(A B)

Can you tell me

A AW AU AUW AA AUA = =A =A =W = =W

Some other examples

A occurs or B does not (or both) A occurs without B or B without A A does not occur or B does not occur (or both) A occurs without B or A and B both occur B occurs but A does not B occurs without A or A and B occur

Permutations and Combinations

Permutation: Ordered Arrangement

Example used: Horse Race... MUTANT HORSE RACE!

Permutation: Ordered Arrangement

HorseFace McBusterWorthy wins 1st place!!

...in a one-horse race!

# Horses (n) # Winning Places (r) # Outcomes

1 3 3

1 1 3

Permutation: Ordered Arrangement

For n objects, when taking all of them (r=n), there are n! possible permutations. 3 horses (n) & 3 winning places (r) -->
3*2*1=6 possible outcomes

For n objects taken r at a time: n! (n-r)! 7 horses & 3 winning places?...

Combination: Unordered Arrangement

Example used: Combo Plate!

QuickTime an d a decompressor are need ed to see this picture .

Combination: Unordered Arrangement

Mexican restaurants menu:
taco, burrito, enchilada

How many different 3-item combos can you get?

# Menu Items (n) Combo Size (r) # Outcomes

Combination: Unordered Arrangement

Mexican restaurants menu:
taco, burrito, enchilada tamale, quesadilla, taquito, chimichanga

How many different 3-item combos can you get?

# Menu Items (n) Combo Size (r) # Outcomes

3 7

3 3

Combination: Unordered Arrangement

For n objects, when taking all of them (r=n), there is 1 combination For n objects taken r at a time: n! r!(n-r)!

Multiplication Principle
(a.k.a. Fundamental Counting Principle)

For 2 independent phenomenon, how many different ways are there for them to happen together?
# possible joint outcomes?

Simply multiply the # possible outcomes for the two individual phenomena Example: flip coin & roll die 2*6=12

Multiplication Principle
(a.k.a. Fundamental Counting Principle)

Can be used with Permutations &/or Combinations Ex: Lunch at the Racetrack
7 horses racing 7 items on the cafe menu I see the results of the race (1st, 2nd, 3rd) and order a 3-item combo plate. How many different ways can this happen?

Calculating Probabilities
Counting rules (Permutation, Combination, Multiplication):
Define sample space (# possible outcomes)

Probability of a specific outcome:

1 sample space

Probability of an event?
event: subset of sample space made of 1 or more possible outcomes

Calculating Probabilities
Sample Space:
7 Micro Machines (3 yellow, 4 red)

Outcome:
draw the yellow corvette Probability = 1/7

Event:
draw any yellow car there are 3 outcomes that could satisfy this event: yellow corvetter, yellow pickup, yellow taxi Probability = 3/7

Probability of Draws w/ Replacement

Replacement: resetting the sample space each time
--> independent phenomena so use multiplication principle

Ex: 3 draws with replacement

Event: drawing a red car all 3 times Probability: 4/7 * 4/7 * 4/7 = 64/343 = 0.187 =18.7%

Probability of Draws w/o Replacement

1. Use counting rules to define sample space 2. Use counting rules to figure out how many possible outcomes satisfy the event 3. divide #2 by #1.

Probability of Draws w/o Replacement

Ex: Drawing 3 cars w/o replacement
Event: drawing 2 red & 1 yellow
(dont care about order) --> use Combinations

Define Sample space: Count outcomes that satisfy event

treat red & yellow as independent use combinations, then multiplication principle

Divide

Recap
Today:
Probability is the tool well use to make inferences about a population, from a sample Counting rules: define sample space for simple phenomena Intro to calculating probability

Next time:
Probability rules, more about events, Venn diagrams

Descriptive -> Inferential?

How can we make inferences about a population if we just have data from a sample? How can we evaluate how good our estimate is? Do these sample data really reflect whats going on in the population, or are they maybe just due to chance?