Statistics 111 - Lecture 6

Probability
Introduction to Probability,
Conditional Probability and
Random Variables
June 3, 2008

Stat 111 - Lecture 6 - Probability

Administrative Note
Homework 2 due Monday, June 8th
Look at the questions now!

Prepare to have your minds blown

today

June 3, 2008

Stat 111 - Lecture 6 - Probability

Course Overview
Collecting Data

Exploring Data
Probability Intro.
Inference

Comparing Variables

Means

June 2, 2008

Proportions

Relationships between Variables

Regression

Stat 111 - Lecture 6 - Introduction

Contingency Tables

Why do we need Probability?

We have several graphical and numerical
statistics for summarizing our data
We want to make probability statements
about the significance of our statistics
Eg. In Stat111, mean(height) = 66.7 inches
What is the chance that the true height of Penn
students is between 60 and 70 inches?

Eg. r = -0.22 for draft order and birthday

What is the chance that the true correlation is
significantly different from zero?
June 3, 2008

Stat 111 - Lecture 6 - Probability

Deterministic vs. Random Processes

In deterministic processes, the outcome can be
predicted exactly in advance
Eg. Force = mass x acceleration. If we are given
values for mass and acceleration, we exactly know
the value of force

In random processes, the outcome is not

known exactly, but we can still describe the
probability distribution of possible outcomes
Eg. 10 coin tosses: we dont know exactly how
many heads we will get, but we can calculate the
probability of getting a certain number of heads
June 3, 2008

Stat 111 - Lecture 6 - Probability

Events

An event is an outcome or a set of outcomes of

a random process
Example: Tossing a coin three times
Event A = getting exactly two heads = {HTH, HHT, THH}
Example: Picking real number X between 1 and 20
Event A = chosen number is at most 8.23 = {X 8.23}
Example: Tossing a fair dice
Event A = result is an even number = {2, 4, 6}

Notation: P(A) = Probability of event A

Probability Rule 1:
0 P(A) 1 for any event A
June 3, 2008

Stat 111 - Lecture 6 - Probability

Sample Space
The sample space S of a random process is
the set of all possible outcomes
Example: one coin toss
S = {H,T}
Example: three coin tosses
S = {HHH, HTH, HHT, TTT, HTT, THT, TTH, THH}
Example: roll a six-sided dice
S = {1, 2, 3, 4, 5, 6}
Example: Pick a real number X between 1 and 20
S = all real numbers between 1 and 20

Probability Rule 2: The probability of the

whole sample space is 1
P(S) = 1
June 3, 2008

Stat 111 - Lecture 6 - Probability

Combinations of Events
The complement Ac of an event A is the event that A
does not occur
Probability Rule 3:
P(Ac) = 1 - P(A)
The union of two events A and B is the event that
either A or B or both occurs
The intersection of two events A and B is the event
that both A and B occur
Event A

June 3, 2008

Complement of A

Union of A and B

Stat 111 - Lecture 6 - Probability

Intersection of A and B

Disjoint Events
Two events are called disjoint if they can not
happen at the same time
Events A and B are disjoint means that the
intersection of A and B is zero

Example: coin is tossed twice

S = {HH,TH,HT,TT}
Events A={HH} and B={TT} are disjoint
Events A={HH,HT} and B = {HH} are not disjoint

Probability Rule 4: If A and B are disjoint

events then
P(A or B) = P(A) + P(B)
June 3, 2008

Stat 111 - Lecture 6 - Probability

Independent events
Events A and B are independent if knowing that A
occurs does not affect the probability that B occurs
Example: tossing two coins
Event A = first coin is a head
Event B = second coin is a head

Independent

Disjoint events cannot be independent!

If A and B can not occur together (disjoint), then knowing that
A occurs does change probability that B occurs

Probability Rule 5: If A and B are independent

P(A and B) = P(A) x P(B)
multiplication rule for independent events
June 3, 2008

Stat 111 - Lecture 6 - Probability

10

Equally Likely Outcomes Rule

If all possible outcomes from a random process
have the same probability, then
P(A) = (# of outcomes in A)/(# of outcomes in S)

Example: One Dice Tossed

P(even number) = |2,4,6| / |1,2,3,4,5,6|
Note: equal outcomes rule only works if the
number of outcomes is countable
Eg. of an uncountable process is sampling any fraction between
0 and 1. Impossible to count all possible fractions !
June 3, 2008

Stat 111 - Lecture 6 - Probability

11

Combining Probability Rules Together

Initial screening for HIV in the blood first uses
an enzyme immunoassay test (EIA)
Even if an individual is HIV-negative, EIA has
probability of 0.006 of giving a positive result
Suppose 100 people are tested who are all
HIV-negative. What is probability that at
least one will show positive on the test?
First, use complement rule:
P(at least one positive) = 1 - P(all negative)

June 3, 2008

Stat 111 - Lecture 6 - Probability

12

Combining Probability Rules Together

Now, we assume that each individual is
independent and use the multiplication rule for
independent events:
P(all negative) = P(test 1 negative) P(test 100 negative)

P(test negative) = 1 - P(test positive) = 0.994

P(all negative) = 0.994 0.994 = (0.994)100

So, we finally we have

P(at least one positive) =1 (0.994)100 = 0.452
June 3, 2008

Stat 111 - Lecture 6 - Probability

13

Curse of the Bambino:

Boston Red Sox traded Babe
Ruth after 1918 and did not
win a World Series again until
2004 (86 years later)
What are the chances that a team will go 86
years without winning a world series?
Simplifying assumptions:
Baseball has always had 30 teams
Each team has equal chance of winning each year
June 3, 2008

Stat 111 - Lecture 6 - Probability

14

Curse of the Bambino

With 30 teams that are equally likely to win in a year, we
have
P(no WS in a year) = 29/30 = 0.97
If we also assume that each year is independent, we can
use multiplication rule
P(no WS in 86 years)
= P(no WS in year 1) x xP(no WS in year 86)
= (0.97) x x (0.97)
= (0.97)86 = 0.05 (only 5% chance!)
June 3, 2008

Stat 111 - Lecture 6 - Probability

15

Break

June 3, 2008

Stat 111 - Lecture 6 - Probability

16

Outline
Moore, McCabe and Craig: Section
4.3,4.5
Conditional Probability
Discrete Random Variables
Continuous Random Variables
Properties of Random Variables
Means of Random Variables
Variances of Random Variables
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

17

Conditional Probabilities
The notion of conditional probability can be found
in many different types of problems
Eg. imperfect diagnostic test for a disease
Disease +

Disease -

Total

Test +

30

10

40

Test -

10

50

60

Total

40

60

100

What is probability that a person has the

disease? Answer: 40/100 = 0.4
What is the probability that a person has the
disease given that they tested positive?
More Complicated !
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

18

Definition: Conditional Probability

Let A and B be two events in sample space
The conditional probability that event B occurs
given that event A has occurred is:

P(A|B) = P(A and B) / P(B)

Eg. probability of disease given test positive
P(disease +| test +) = P(disease + and test +) / P(test +) = (30/100)/(40/100) =.75

June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

19

Independent vs. Non-independent Events

If A and B are independent, then
P(A and B) = P(A) x P(B)
which means that conditional probability is:
P(B | A) = P(A and B) / P(A) = P(A)P(B)/P(A) = P(B)
We have a more general multiplication rule for
events that are not independent:
P(A and B) = P(B | A) P(A)
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

20

Random variables
A random variable is a numerical outcome of
a random process or random event
Example: three tosses of a coin
S = {HHH,THH,HTH,HHT,HTT,THT,TTH,TTT}
Random variable X = number of observed tails
Possible values for X = {0,1, 2, 3}

Why do we need random variables?

We use them as a model for our observed data
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

21

Discrete Random Variables

A discrete random variable has a finite or
countable number of distinct values
Discrete random variables can be summarized
by listing all values along with the probabilities
Called a probability distribution

Example: number of members in US families

X

P(X)

0.413

0.236

0.211

0.090

0.032

0.018

June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

22

Another Example
Random variable X = the sum of two dice
X takes on values from 2 to 12
Use equally-likely outcomes rule to
calculate the probability distribution:
X

10

11

12

# of
Outco
mes

P(X)

1/36

2/36

3/36

4/36

5/36

6/36

5/36

4/36

3/36

2/36

1/36

If discrete r.v. takes on many values, it is

better to use a probability histogram
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

23

Probability Histograms
Probability histogram of sum of two dice:

Using the disjoint addition rule, probabilities

for discrete random variables are calculated
by adding up the bars of this histogram:
P(sum > 10) = P(sum = 11) + P(sum = 12) = 3/36
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

24

Continuous Random Variables

Continuous random variables have a noncountable number of values
Cant list the entire probability distribution, so
we use a density curve instead of a histogram
Eg. Normal density curve:

June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

25

Calculating Continuous Probabilities

Discrete case: add up bars from probability histogram
Continuous case: we have to use integration to calculate
the area under the density curve:

Although it seems more complicated, it is often easier to

integrate than add up discrete bars
If a discrete r.v. has many possible values, we often
treat that variable as continuous instead
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

26

Example: Normal Distribution

We will use the normal distribution throughout
this course for two reasons:
1.
2.

It is usually good approximation to real data

We have tables of calculated areas under the
normal curve, so we avoid doing integration!

June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

27

Mean of a Random Variable

Average of all possible values of a random
variable (often called expected value)
Notation: dont want to confuse random
variables with our collected data variables

= mean of random variable

x = mean of a data variable
For continuous r.v, we again need integration
to calculate the mean
For discrete r.v., we can calculate the mean by
hand since we can list all probabilities
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

28

Mean of Discrete random variables

Mean is the sum of all possible values, with
each value weighted by its probability:
= xi*P(xi) = x1*P(x1) + + x12*P(x12)
Example: X = sum of two dice
X

10

11

12

P(X)

1/36

2/36

3/36

4/36

5/36

6/36

5/36

4/36

3/36

2/36

1/36

= 2 (1/36) + 3 (2/36) + 4 (3/36) ++12 (1/36)

=252/36=7
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

29

Variance of a Random Variable

Spread of all possible values of a random
variable around its mean
Again, we dont want to confuse random
variables with our collected data variables:

2 = variance of random variable

s2 = variance of a data variable
For continuous r.v, again need integration to
calculate the variance
For discrete r.v., can calculate the variance by
hand since we can list all probabilities
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

30

Variance of Discrete r.v.s

Variance is the sum of the squared deviations
away from the mean of all possible values,
weighted by the values probability:
= (xi-)*P(xi) = (x1-)*P(x1) + + (x12-)*P(x12)
Example: X = sum of two dice
X

10

11

12

P(X)

1/36

2/36

3/36

4/36

5/36

6/36

5/36

4/36

3/36

2/36

1/36

2 = (2 - 7)2(1/36) + (3 7)2(2/36) ++(12 - 7)2(1/36)

= 210/36 = 5.83
June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

31

Next Class - Lecture 7

Standardization and the Normal
Distribution
Moore and McCabe: Section 4.3,1.3

June 4, 2008

Stat 111 - Lecture 6 - Random

Variables

32