Randomness and Probability

Randomness and Probability models

Probability and Randomness
Sample spaces
Probability properties
Assigning probabilities: equally likely outcomes
Assigning probabilities: finite number of outcomes
Probability rules (mutually exclusive vs. independence)
Probability
1. What is the probability that a
flipped coin comes up heads?
2. What are the odds that Microsoft
stock price will go up tomorrow?
3. What is the chance of rolling a 3
or 4 on a die?

A phenomenon is random if individual
outcomes are uncertain, but there is
nonetheless a regular distribution of
outcomes in a large number of repetitions.
Randomness and probability
The probability of any outcome of a random phenomenon can be defined as
the proportion of times the outcome would occur in a very long series of
repetitions.
Coin toss
The result of any single coin toss is random.
But the result over many tosses is
predictable, as long as the trials are
independent (i.e., the outcome of a new coin
flip is not influenced by the result of the
previous flip).
First series of tosses
Second series
The probability of
heads is 0.5 = the
proportion of
times you get
heads in many
repeated trials.
Probability models describe, mathematically, the outcome of random
processes. They consist of two parts:
1) S = Sample Space: This is a set, or list, of all possible outcomes of a
random process. An event is a subset of the sample space.
2) A probability for each possible event in the sample space S.
Probability models
Example: Probability Model for a Coin Toss:
S = {Head, Tail}
Probability of heads = 0.5
Probability of tails = 0.5
Sample Space
Every possible outcome
AKA Universe or population
S or
A
S
outcomes
Probability
P(A) =
Size of the Event A
Size of the Sample Space S
General Purpose Definition
Simple Case
If outcomes are equally likely
P(A) =
# outcomes in A
Total # outcomes
P(Heads) =
P(Draw a King) =

4/52 = 1/13
Flip a Coin Three Times
Outcomes
HHH HHT HTH HTT
THH THT TTH TTT

1. P(HHH) = 1/8 = 0.125
2. P(Two Heads) =
3. P(At least 2 Heads) =
3/8 = 0.375
4/8 = = 0.5
Roll Two Dice
Outcomes:

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)
(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)
(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
P(Sum =2) = 1/36 = 0.0278
P(Sum=9) = 4/36 = 1/9 = 0.111
P(Sum=7) = 6/36 = 1/6 = 0.167
The gambling industry relies on probability distributions to calculate the odds of
winning. The rewards are then fixed precisely so that, on average, players lose and
the house wins.

The industry is very tough on so called cheaters because their probability to win
exceeds that of the house. Remember that it is a business, and therefore it has to be
profitable.
Probability Properties
Probabilities range from 0 (no chance of the
event) to 1 (the event has to happen)
0 P(A) 1
P(A) = 0 A is impossible
Because some outcome must occur
on every trial, the sum of the probabilities
for all possible outcomes (the sample
space) must be exactly 1.
P(S) = 1
The Addition (OR) Rule
Mutually Exclusive
Events contain no common outcomes
Intersection is empty
They cant both happen
For mutually exclusive events A,B
P(A or B) = P(A) + P(B)
Mutually Exclusive
Mutually Exclusive
Roll a 2 and a 6
Draw a King and a Queen
Flip two heads and 1 head
Not Mutually Exclusive
Roll a 2 and then a 6
Draw a King and a Spade
Flip two heads and at least 1 head
OR Rule
Roll 2 dice,
P(Sum is 7 or 9) = 1/6 + 1/9 = 5/18
Flip three coins
P(1 H or 3 H) = 3/8 + 1/8 =
Draw a card
P(K or Q) = 1/13 + 1/13 = 2/13
P(Diamond or Heart) = + = 1/2
P(K or Diamond) = ????
Simple Events
Simple event contains one outcome.
{Heads} , {HHH}
{(1,1)}
{King of Spades}
Simple events are mutually exclusive
Equally likely
P(Simple Event) = 1/(total # of outcomes)

Unequal Outcomes
Assign a probability to each outcome.
All probabilities 0.
P(A) = sum P of each outcome in A
All probabilities sum to 1.
P(S) = 1
All probabilities 1.
M&M candies
Color Brown Red Yellow Green Orange Blue
Probability 0.3 0.2 0.2 0.1 0.1 ?
If you draw an M&M candy at random from a bag, the candy will have one
of six colors. The probability of drawing each color depends on the proportions
manufactured, as described here:



What is the probability that an M&M chosen at random is blue?
What is the probability that a random M&M is either red, yellow, or orange?
S = {brown, red, yellow, green, orange, blue}
P(S) = P(brown) + P(red) + P(yellow) + P(green) + P(orange) + P(blue) = 1
P(blue) = 1 [P(brown) + P(red) + P(yellow) + P(green) + P(orange)]
= 1 [0.3 + 0.2 + 0.2 + 0.1 + 0.1] = 0.1
P(red or yellow or orange) = P(red) + P(yellow) + P(orange)
= 0.2 + 0.2 + 0.1 = 0.5
Complement Rule
Complement of A
All outcomes not in A
A
c
P(A
c
) = 1 P(A)

P(Drawing a card other than an Ace) =
=1 1/13 = 12/13
General Addition (OR) Rule
For any events A, B
P(A or B) = P(A) + P(B) P(A and B)
P(King) = 4/52 = 1/13
P(Heart) = 13/52 =
P(King and Heart) = P(King of Hearts) = 1/52
P(King or Heart) =
= P(King) + P(Heart) P(King and Heart)
=4/52 + 13/52 1/52 = 16/52 = 4/13

Subjective?
There is a 30% chance of rain of tomorrow.
What does that mean?
What is the Sample Space?
What are the outcomes?
Subjective Probability
An opinion or judgment by a decision maker
about the likelihood of an event based on
their expertise

Independence
A and B independent
Are not related
Knowing A does not give information about B
A does not affect B
Two coin flips
Roll a die twice
Draw two cards
Weather on two consecutive days

Multiplication (AND) Rule
P(A and B) = P(A) P(B)
If and only if A and B are independent.
Roll a die two times
P(roll a 3 and then a 5) = (1/6)(1/6) = 1/36
P(roll an even number twice) =(1/2)(1/2) = 1/4

Independent repetition of game
Choose a number between 1 and 10
A={your number is even}
B={your number is 5}
P(A) =
P(B) =
P(A and B) =

5/10 = 1/2
5/10 = 1/2
2/10 = 0.2
A and B are NOT independent
Choose a number between 1 and 12
A={your number is even}
B={your number is 6}
P(A) =
P(B) =
P(A and B) =
6/12 = 1/2
6/12 = 1/2
3/12 = 1/4
A and B are independent
What is your favorite color?
Color Red Blue Pink Black
P
0.3 0.2 0.2 0.3
Simple Random Sample subjects are
independent.
P(two people both like black) = (0.3)(0.3) =
0.027
Sequence of Independent Events
If A
1
, A
2
, A
3
,,A
n
are all independent

P(A
1
and A
2
and and A
n
) =
P(A
1
) P(A
2
) P(A
3
) P(A
n
)

P( 10 heads in a row) = .5(.5) (.5) = .5
10
=
1/1024 = 0.000977

Ex. An automobile manufacturer buys computer
chips from a supplier. The supplier sends a
shipment containing 5% defective chips. Each
chip chosen from this shipment has
probability 0.05 of being defective, and each
automobile uses 12 chips selected
independently. Whats the probability that all
12 chips in a car will work properly?
P ( all 12 chips will work properly )
= P(1
st
properly) P(2
nd
properly ) (.) P(12
th
properly)
(multiplication rule for independent events)
=(1-P(1
st
is defective)) (1- P(2
nd
defective)) ()
(1- P(12
th
defective))
(complement rule)
= (1-0.05)(1-0.05) () (1-0.05)
= (1-0.05) to the power of 12
= (0.95) to the 12
th
power , which is around 0.5404

Q: How about the probability that 6 chips among these 12
chips will work properly ?

Independence and Mutually Exclusive
Mutually Exclusive : they cant both happen
P(A and B) = 0
They arent independent
OR rule for Independent Events
P(A or B) = P(A) + P(B) P(A) P(B)
Dependence
A says something about B
P( B|A) = conditional probability of B given A
If A and B are independent,
P(B|A) = P(B)

P(2
nd
card is K | 1
st
card is K ) = 3/51 = 1/17
P(2
nd
card is K | 1
st
card is Q ) = 4/51
Conditional Probability
S
B
A
B
P( B|A)
General Multiplication (AND) Rule
P( A and B ) = P( A ) P( B | A )
Also
P( A and B ) = P(B) P(A|B)
P(Drawing 2 Ks) = (4 /52)(3/51) = 0.0045
Sampling without Replacement
Jar has 6 black olives and 6 green olives.
You pull out 3
P(1
st
is black) = 6/12 = 0.5
P(2
nd
is black | 1
st
is black) = 5/11 = 0.45
P(3
rd
is black | 1
st
two were black) = 4/10=0.4
P(1
st
two are black) = 0.5(.45) = .227
Conditional probability
Algebra
P( A and B) = P(A) P(B|A)
P(B|A) P(A) = P(A and B)
If P(A)>0 then
P(B|A) =
P( A and B)
P(A)
What is the probability that a car has a CD
player, given that it has AC ?

i.e., we want to find P(CD | AC)
Of the cars on a used car lot, 70% have air
conditioning (AC) and 40% have a CD player
(CD), and 20% of the cars have both.
.2857 0
0.7
0.2
P(AC)
AC) and P(CD
AC) | P(CD
No CD CD Total
AC
0.2 0.5 0.7
No AC 0.2 0.1
0.3
Total 0.4 0.6 1.0
Of the cars on a used car lot, 70% have air conditioning (AC)
and 40% have a CD player (CD).
20% of the cars have both.
(continued)
No CD CD Total
AC
0.2 0.5 0.7
No AC 0.2 0.1
0.3
Total 0.4 0.6 1.0
Given AC, we only consider the top row (70% of
the cars). Of these, 20% have a CD player.
Therefore, 20% of 70% is about 28.57%.
.2857 0
0.7
0.2
P(AC)
AC) and P(CD
AC) | P(CD
(continued)
Tree diagrams
Three components
A fails 40% of the time
B fails 70% of the time
C fails 10% of the time
Choose one according to
Component A B C
P .2 .1 .7
What is the probability of failure? Answer=.22
Tree diagram
A
B
C
.2
.1
.7
fails
doesnt fail
fails
fails
doesnt fail
doesnt fail
.4
.6
.7
.3
.1
.9
=(.4)(.2)=.08
=(.1)(.3)=.03
=(.7)(.1)=.07
=(.7)(.9)=.63
=(.1)(.7)=.07
=(.2)(.6)=.12
Tree diagram
A automobile consultant records fuel type
and vehicle type for a sample of vehicles
2 Fuel types: Gasoline, Diesel
3 Vehicle types: Truck, Car, SUV

6 possible experimental outcomes:
Gasoline, Truck
Gasoline, Car
Gasoline, SUV
Diesel, Truck
Diesel, Car
Diesel, SUV
Tree Diagram
Diesel
P(E
2
) = 0.2
Gasoline
P(E
1
) = 0.8
Car: P(E
4
|E
1
) = 0.5
P(E
1
and E
3
) = 0.8 x 0.2 = 0.16
P(E
1
and E
4
) = 0.8 x 0.5 = 0.40
P(E
1
and E
5
) = 0.8 x 0.3 = 0.24
P(E
2
and E
3
) = 0.2 x 0.6 = 0.12
P(E
2
and E
4
) = 0.2 x 0.1 = 0.02
P(E
3
and E
4
) = 0.2 x 0.3 = 0.06
Car: P(E
4
|E
2
) = 0.1