1.4.

3 Chebyshevs Inequality
Let X be a random variable with mean and variance 2 . For any > 0,
2
P (|X | ) 2 .

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 46

1.4.4 The Weak Law of Large Numbers (WLLN)

Let X1 , X2 , . . . , Xn be iidrvs with mean and variance 2 . Then for all > 0,






lim P X > = 0
n

Proof
Exercise. Use Chebyshevs inequality.

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 47

1.4.5 Central Limit Theorem

Let X1 , X2 , . . . , Xn be iidrvs with mean E(Xi ) = and variance
Var(Xi ) = 2 , i = 1, 2, . . . , n. Then for large n,
X
N(0, 1)
Z=
/ n
For practical purposes n 30 is large enough. If is unknown, then
X
tn1
T =
S/ n
Note

is read as is approximately distributed as.

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 48

1.5 Some Distribution Theory

1.5.1 Uniform distribution
The random variable X has a discrete uniform distribution over for x = 1, 2, . . . , n
if it has pmf
1
pX (x) = , x = 1, 2, . . . , n
n
We write X U [1, n]. It can be shown that
n+1
n2 1
E(X) =
, Var(X) =
2
12

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 49

1.5.2 Bernoulli Distribution

A Bernoulli trial is one which has only two possible outcomes, denoted Success (S)
and Failure (F), with P (S) = p and P (F ) = 1 p. Let the random variable X
denote the number of successes in a single Bernoulli trial. Then X has a Bernoulli
distribution with parameter p. We write X Bern(p). X has pmf
(
px (1 p)ax x = 0, 1
pX (x) =
0
otherwise
that is, P (X = 1) = p, P (X = 0) = 1 p. It is easy to show that E(X) = p and
Var(X) = p(1 p).

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 50

1.5.3 The binomial distribution

Let X1 , X2 , . . . , Xn be iid Bern(p) random variables, and put
X=

n
X

Xi

i=1

Then the random variable X counts the number of successes in n independent and
identical Bern(p) trials. The random variable X has a binomial distribution with
parameters n and p. We write X Bin(n, p). The pmf of X is
!

n px (1 p)nx x = 0, 1, 2, . . . , n
pX (x) =
x

0
otherwise
It can be shown that E(X) = np and Var(X) = np(1 p).

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 51

1.5.4 Geometric Distribution

Consider a series of independent Bern(p) trials. Let the random variable X denote
the trial on which the first success occurs. Then
P (X = x) = (1 p)x1 p,

x = 1, 2, 3, . . . .

Then X Geom(p). It can be shown that

1p
1
E(X) = , Var(X) =
p
p2

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 52

1.5.5 Poisson distribution

Let the random variable X denote the number of occurrences of a phenomenon in
a fixed volume. Here volume is a generic term that could mean length, time,
area, volume, a bolt of cloth etc. . Let the mean number of occurrence be per
unit volume, where > 0. Again here unit is a generic term could mean an
interval of time, a measure of area, volume, length, etc. Then X has a Poisson
distribution with parameter . We write X Poi(). The pmf of X is
e k
, k = 0, 1, . . .
pX (k) =
k!
It can be shown that E(X) = and Var(X) = .

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 53

1.5.6 Uniform distribution (continuous)

A random variable X has a (continuous) uniform distribution over the interval
[a, b], b > a, if it has pdf
(
1
0ab
ba
fX (x) =
0
otherwise
We write X U [1, b]. It can be shown that
(b a)2
a+b
, Var(X) =
E(X) =
2
12

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 54

1.5.7 Exponential distribution

A rv X has an Exponential distribution if it has pdf given by
(
e x x 0
fX (x) =
0
otherwise
where > 0. We write X Exp(). It can be shown that
1
1
E(X) = Var(X) = 2

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 55

1.5.8 Gamma Distribution

For > 0 and > 0,

pdf

X (, )
fX (x) =

1
()

() =

(x)1 ex

x0

otherwise

where
x1 ex dx.

The properties of the Gamma distribution derive from the properties of the
function.

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 56

Properties of the function

1. For any positive integer n 2, (n) = (n 1)(n 1)

Proof

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 57

Properties of the function (ctd)

2. (1) = 1
Proof

3. For a n a positive integer, (n) = (n 1)!

Proof

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 58

Properties of the function (ctd)

4.
5.

6.

1
2

n+r1
n

(r) (s)
=
(r + s)

(n + r)
=
(n + 1) (r)

ur1 (1 u)s1 du = B(r, s),

where B(r, s) is the Beta function.

Proof Exercise.

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 59

Properties of the (, ) distribution

1. E(x) = , Var(X) =

2. If X (1, ) then X Exp()

3. If Xi (i , ), i = 1, 2, . . . , n are independent, then
Y =

n
X
i=1

Xi (

n
X

i , )

i=1

4. If Xi Exp(), i = 1, 2, . . . , n are independent, then

Y =

n
X
i=1

Xi (n, )

Proof Exercise.
STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 60

1.5.9 Chi-square distribution

2n

pdf

1
fY (y) = n/2
2

Properties
1. If Y

2n

then it has a

n 1
2, 2

n/2 y/2


y
e
, y0
n
2

distribution.

2. Let Z1 , Z2 , . . . , Zn be iid N(0, 1) rv. Put Y =

Pn

2 Then Y 2 .
Z
n
i=1 i

3. If Y1 2n and Y2 2m are independent, then Y1 + Y2 2n+m .

4. If Y 2n then

E(Y ) = n Var(Y ) = 2n

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 61

1.5.10 F-Distribution
U 2m ,

V 2n ,

where U and V are independent. Put

W =
Then

U/m
V /n

1
F (n, m)
W F (m, n) and
W

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 62

1.5.11 t-Distribution
Z N(0, 1) and V 2n
Put

Then

T =p

Z
V /n

T tn

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 63

Note
Sample variance
n

2
1 X
2
Xi X
S =
n1
i=1
S2

U = (n 1)

2n1

Then
X
X 1
=
q
S/ n
/ n S 2
2

X
1
p
=
/ n
U/(n 1)
tn1

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 64

1.6 Estimation

1.6.1 Population mean

Point estimator is sample mean X
2
,
Var(X) =
n
and this is estimated by

= ,
n

S
SE(X) =
n

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 65

1.6.2 Population proportion p

Point estimator is

X
n
where X is the number of successes in n iid Bernoulli trials, that is,
X Bin(n, p). Note that p is the sample proportion of successes.
 
np(1 p)
p(1 p)
X
Var(
p) = Var
=
=
2
n
n
n
so
r
p(1 p)
p =
n
which is estimated by
r
p(1 p)
SE(
p) =
n
p =

STAT3365: 3S5 Industrial Statistics and Total Quality Management

SMS, UWA, Semester 2, 2009 66