Sums of Independent Random

Variables
4. Approximations
Theorem 4.1 (Chebyshev
Inequality)
Let X be random variable with mean and variance
2
.
Then for any and let > 0

( )
2
2
. P X
o
c
c
> s
Proof
( ) ( ) ( )
( ) ( )
( )
( )
2
2
2
2
x
x e
V X x f x dx
x f x dx
f x dx
P X
c

c
c c

>
>
=
>
>
> >
}
}
}
Theorem 4.2 (Law of Large
Numbers)
Let X
1
, X
2
, ... , X
n
be an independent and identically
distributed random variables with mean and variance

2
. Let S
n
= X
1
+X
2
++X
n
. Then for any real number
> 0 we have
lim 0.
n
n
S
P
n
c

| |
> =
|
\ .
Proof
| |
( )
1
2
2
2 2
1
2
2
0
Note that
1 1

1 1
.
By Chebyshev inequality
lim 0 .
n
n
n
i
n
n
n
i
n n
x
S
E E S
n n n
S
Var Var S
n n n n
S S n
P P
n n
o

o
o
o
c c
c
=
=

| |
= = =
|
\ .
| |
= = =
|
\ .
| | | |
> s > =
| |
\ . \ .

Example 4.1
Let E be an event and define independent random
variables X
i
such that X
i
is 1 if the event E occurs
and X
i
is 0 otherwise. Let p = P(E).
Then E(X
i
) = p and


This gives a mathematical justification to the
relative frequency interpretation of probability.
lim 0.
n
n
S
P p
n
c

| |
> =
|
\ .
Convergence in Distribution
Let X
1
, X
2
, ... be a sequence of random variables
and let F
i
be the distribution of X
i
for each i. Let X
be a random variable with distribution F. We say
that X
n
converges in distribution to X if
( ) ( )
lim for all .
n
n
F x F x x

=
Theorem 4.3
Let X
1
, X
2
, ... be a sequence of random variables
and let
i
be the mgf of X
i
for each i. Let X be a
random variable with mgf . Then X
n
converges in
distribution to X if


for all t in some neighborhood of 0.

( ) ( )
lim
n
n
t t | |

=
Poisson Approximation to the
Binomial
Let X
1
, X
2
, ... be independent Binomial random
variables with parameters n and p. Suppose that
lim
n
np = . Then X
n
converges in distribution to a
Poisson random variable with mean .
( )
( )
( )
( )
( )
( )
( )
1
1
1
1
1
Now, lim 1 and lim .
Hence,
lim .
t
t
n
t
n
t
n
n
t
e
n n
e
n
n
np e
t pe q
n
e
e np
n
t e

(
= + = +
(

(

(
+ = =
(

=
Proof
Example 4.2
An airline finds that 4 percent of the passengers
that make reservations on a particular flight will not
show up. Consequently, their policy is to sell 100
reserved seats on a plane that has only 98 seats.
Find the probability that every person who shows
up for the flight will find a seat available.
Solution
Let X be the number of passengers who do not
show up. Then X is binomial with parameters n =
100 and p = .04 . We want P(X 2). Using Poisson
approximation
( )
4 4
2 1 4 .908. P X e e

> = =
Theorem 4.4
Let X
1
, X
2
, ... ,X
n
be independent and identically
distributed Normal random variables with mean
and variance
2
. Define


Then has a standard Normal
distribution.
1 2
.
n
n
X X X
X
n
+ + +
=
( )
n n
Z n X o =
Proof
( )
( )
( )
2
1
2
1 1
Let . We have shown previously that
has a standard normal distribution.
1
.
n i
i i
i
n
n
n i
i
n n
t n
Z Y
i i
Y X
Y
X
Z n Y
n
t
t e
n
o

o
| |
=
= =
=

= =
| |
= =
|
\ .

[ [
Theorem 6.5 (Central Limit Theorem)
Let X
1
, X
2
, ... ,X
n
be independent and identically
distributed random variables with mean and
variance
2
. Define



Then converges in distribution to
a standard Normal distribution.
1 2
.
n
n
X X X
X
n
+ + +
=
( )
n n
Z n X o =
Proof
( )
( ) ( ) ( )
( ) ( ) ( )
2
2
Let . Then
1
0 0 0
2
with lim 0.
But 0 1, 0 0 and 0 1. Hence
1 .
2
i
i i
i
i i
Y n
n
n
Y Y
Y n
Y X
t t t
R
n n n
nR
t t
R
n
n
o
| | | |
| | |
|

=
| | | |
' ''
= = + + +
| |
\ . \ .
=
' ''
= = =
| |
= + +
|
\ .
( )
2
2
1 1
2
2
Thus
1
2
2
1
as .
n i
n n
Z Y n
i i
n
t
n
t t
t R
n
n
t nR
e
n
n
| |
= =
| |
| |
= = + +
|
|
\ .
\ .
| |
+
= +
|
\ .

[ [