DISTRIBUTION OF SAMPLE MEAN AND SAMPLE VARIANCE FOR

NORMAL SAMPLES
Random samples: Let X1 , X2 , ..., Xn be a sample of observable measurements(i.e they are
random variables) of a population of size N . Further let each variable Xi ,

i = 1, 2, .., N

have probability distribution f (xi ).

If
(i) the joint distribution of the random variables is
f (x1 , x2, .., xn) = f (x1 )f (x2 )....f (xn) =

n
Y

f (xi )

i=1

(ii) f (xi ) = f(x) xi ;

then the random variables are independent and identically distributed and they constitute
a random sample from a population with a common distribution.
Sampling distribution of the sample mean:
Let X1 , X2 , ..., Xn be a random sample of size n from a population with mean and finite
variance 2. The sample mean, usually denoted by X is;
1X
X=
xi
n i=1
n

X is a random variable since it is a function of the n random variables. The mean and
variance of X are
"

#
n
n
1X
1X
E[X] = E
Xi =
E[Xi ]
n i=1
n i=1
1X
=
=
n i=1
n

and
#
n
n
X
1 X
1
V ar[X] = V ar
Xi = 2
V ar[Xi ]
n i=1
n i=1
"

n
1 X 2 2
=
n2 i=1
n

For large samples (n > 30); using Central Limit Theorem, then X is normally distributed with mean and variance

2
;
n

i.e X N (, n ) regardless of the probability

distribution of the random sample. This distribution is referred to as a sampling distribution and the standard deviation of the distribution is called the standard error of the
distribution.
Examples
(1) Let X be the sample mean of a random sample of size 60 from a distribution with
mean=42.5 and variance=135. Determine k such that P (X < k) = 0.65


; hence
The sampling distribution of X is X N 42.5, 135
60
P (X < k) = 0.65

k 42.5
X 42.5
P q
< q
= 0.65

135
60

135
60

k 42.5
= 0.65
P Z < q
135
60

From standard normal tables, P (Z < 0.39) ' 0.65 hence

k 42.5
q
= 0.39
135
60

k = 43.085
(a) Let X be the mean of a random sample of size 121 from the distribution that has
probability density function
f (x) = 4(1 x)3,

0<x<1

Use the large sampling distribution to compute the probability that

0.165 < X < 0.234

We need to first determine the mean and variance of the distribution

Z 1
E[X] =
x4(1 x)3dx
0
Z 1


4 1
(1 x)4dx
= x(1 x) 0 +
0


5 1
(1 x)
1
= 0 + x
=
5
5
0

E[X ] =

x24(1 x)3dx
0

=
=
Z

2x(1 x)4dx =
0

Z 1

 2
4 1
2x(1 x)4 dx
x (1 x) 0 +
0
Z 1
2x(1 x)4dx
0+
0
1 Z 1

(1 x)5
(1 x)5
dx
x
+
5
5
0
0

1
(1 x)6
1
0 + 2 x
=
30
15
0

Thus
V ar(X) = E[X 2] (E[X])2
1
2
1

=
=
15 25
75


2
The sampling distribution of X is X N 15 , 9075
The required probability is given by

0.165 0.2
0.234 0.2
X 0.2
P (0.165 < X < 0.234) = P q
< q
< q
2
9075

= P (2.36 < Z < 2.29)

= 0.979

2
9075

2
9075

Sampling distribution of the sample variance:

Let X1 , X2 , .., Xn be a random sample of size n from a population having normal distribution with mean and finite variance 2. Let S 2 be the sample variance,
1 X
S =
(Xi X)2
n 1 i=1
n

. S 2 is a random variable since it is a function of the random variables Xi , i = 1, 2, .., n as

well as X.
The quantity
(n 1)S 2
2(n 1)
2

Proof:
(n 1)S

n
X

i=1
n
X

n
X

(Xi X ) =

n
X

(Xi + X)2

i=1

2

(Xi ) (X )

i=1
2

(Xi ) 2

n
X

i=1

(Xi )(X ) +

i=1

n
X

(X )2

i=1

Now
n
X

(Xi )(X ) = (X1 )(X ) + (X2 )(X ) + ... + (Xn )(X )

i=1

= (X ) [(X1 ) + (X2 ) + ... + (Xn )]

= X ) [X1 + X2 + ... + Xn n]



2
= (X ) nX n = n X
and

n
X

(X )2 = n(X )2

i=1

Thus
2

(n 1)S =

n
X

(Xi ) 2n(X ) + n(X ) =

i=1

n
X
i=1

(Xi )2 n(X )2

Dividing through by 2;
n
X
(n 1)S 2
n
(Xi )2
=
2 (X )2
2
2

i=1

Finally

n
X
(Xi )2
i=1

n
X

Zi2;

Zi2

(1) hence

i=1

Therefore

(n1)S 2
2

Zi2 2(n)

i=1

and
n
(X )2 2(1);
2

n
X

since

n(X )

N (0, 1)
2

is difference between two chi-square random variables. Using proprety

of chi square distribution, this quantity is a chi square randon variable with n-1 degrees of
freedom.
Example A random sample of 81 high-school students was selected and their IQ measurements taken. Assuming the measurements are normally distributed with mean = 105
and standard deviation = 15; Find Pr(150.581 < S 2 < 299.894)
(n1)S 2
2

2(80) then


2
80

S
80

299.894
80

150.581
<
<
P r(150.581 < S 2 < 299.894) = P r
225
225
225
2
= P r(53.539 < < 106.629)

Solution Given

= P r(2 < 106.629) P r(2 < 53.539)

= 0.975 0.01 = 0.965

using chi square tables for 80d.f

FUNCTIONS OF SEVERAL RANDOM VARIABLES:

Our interest is to determine the distribution of function of several random variables;
i.e for given random variables X1 , X2 , .., Xn with joint probability distribution function
f (x1 , x2, .., xn) and set of functions say; Y1 = g1 (x1, .., xn), Y2 = g2 (x1 , .., xn),....,Ym =
gm (x1, .., xn); we want to determine the joint probability distribution of Y1 , Y2 , .., Ym using
f (x1 , x2, .., xn).
Three techniques can be used to determine the probability distribution of a function of
several random variables; namely
(i) Cumulative distribution function technique
(ii) Transformation(change of variable) technique
(iii) Moment generating function or characteristic function technique
Distribution function technique:
To obtain the joint probability distribution function of Y1 = g1 (x1 , .., xn), Y2 = g2 (x1, .., xn),....,Ym =
gm (x1, .., xn) with a given joint probability distribution f (x1 , x2, .., xn); we can first obtain
the cumulative distribution function and then the probability distribution function; i.e
F (y1, .., yk) = Pr (Y1 y1 , Y2 y2, ..., Yk yk )
= Pr (g1 (x1, .., xn) y1, g2 (x1, .., xn) y2, .., gk (x1, .., xn) yk )
Since f (x1 , x2, .., xn) is known, presumably the probability of the event
{g1 (x1 , .., xn) y1 , g2(x1 , .., xn) y2 , .., gk (x1 , .., xn) yk }
can be calculated and consequently F (y1, .., yk ) determined.
This technique is easily applicable when k=1. In this case if the n random variables are
continuous, the probability density function is obtained by differentiating the cumulative
distribution so obtained.

Example
Let X1 , X2 , .., Xn be a random sample drawn from a population having exponential distribution with parameter . Find the probability density function of Z=max(X1 , X2 , .., Xn)
using the distribution function technique.
Solution:
Let G(z) be the cumulative distribution function of Z. Now since Z is the largest of the n
random variables,
G(z) = Pr(Z z) = Pr(X1 z, X2 z, .., Xn z)
n
n Z z
Y
Y
Pr(Xi z) =
exdx
=
=

i=1
n
Y

i=1


 
n
1 ez = 1 ez

i=1

Differentiating G(z) with respect to z we get

dG(z)
= n[1 ez ]n1 ez
dz
Thus the p.d.f of Z is
f (z) = nez [1 ez ]n1 ;

z>0

Change of variable technique

Let f (x1 , x2, .., xn) be the joint probability density of the continuous random variables
X1 , X2 , .., Xn . Also let Y1 = g1 (x1 , .., xn), Y2 = g2 (x1 , .., xn),....,Yn = gn (x1 , .., xn).
If
(i) the functions y1 = g1 (x1 , .., xn), y2 = g2 (x1 , .., xn),....,yn = gn (x1 , .., xn) are partially
differentiable with respect to each variablexi, i = 1, 2, .., n
(ii) the functions y1 = g1 (x1, .., xn), y2 = g2 (x1 , .., xn),....,yn = gn (x1 , .., xn) are oneto-one transformations for which f (x1, x2 , .., xn) 6= 0, x1, x2 , .., xn in the space of
X1 , X2 , .., Xn
(iii) the inverse functions x1 = h1 (y1 , y2, .., yn), x2 = h2(y1 , y2, .., yn),..,xn = hn (y1, y2 , .., yn)
can be uniquely determined;
then joint probability density function of Y1 = g1 (x1 , .., xn), Y2 = g2 (x1 , .., xn),....,Yn =
gn (x1, .., xn) is
g(y1 , y2, .., yn) = f (g1 (x1, .., xn), g2 (x1 , .., xn), .., gn(x1 , .., xn)) | J |
where J is the Jacobian of the transformation. J is the determinant of the matrix
x x

x1
1
1
.
.
.
y
y2
yn
1

x2 x2 . . . x2
y1 y2
yn

.
.
.
.
.

.
.
.
.
.
.
.

xn
y1

xn
y2

...

xn
yn

Example:
Let the joint probability density function of X1 , X2 , X3 be
(
1,
0 < x1 < 1, 0 < x2 < 1, 0 < x3 < 1
f (x1, x2 , x3) =
0, elsewhere
Find the joint probability density function of Y1 , Y2 , Y3 where Y1 = X1 + X2 + X3 ,Y2 =
X2 + X3 and Y3 = X3 .

Solution:
Now
x3 = y3;

x2 = y2 y3 ;

x1 = y1 y2

to determine the domain of the variables y1 , y2, y3 ;

(i) since 0 < x3 < 1, and x3 = y3, then 0 < y3 < 1
(ii) since 0 < x2 < 1, and x2 = y2 y3 , then 0 < y2 y3 < 1 which implies y3 < y2 < y3 +1
(iii) since 0 < x1 < 1, and x1 = y1 y2 , then 0 < y1 y2 < 1 which implies y2 < y1 < y2 +1
the matrix of partial derivatives is

x1
y1

x1
y2

x1
y3

x2
y1

x2
y2

x2
y3

x3
y1

x3
y2

x3
y3

1 1 0

= 0 1 1
,

0 0
1

J=1;
hence
g(y1 , y2, y3) = f (y1 y2 , y2 y3, y3 ) | J |
= 1 | 1 |= 1
The joint probability density function of Y1 , Y2 , Y3 is
(
1,
0 < y3 < 1, y3 < y2 < y3 + 1, y2 < y1 < y2 + 1
g(y1 , y2, y3) =
0, elsewhere

Moment generating function technique

Let X1 , X2 , .., Xn be random variables with joint probability distribution function f (x1 , x2, .., xn)
. Also let Y1 = g1 (x1, .., xn), Y2 = g2 (x1 , .., xn),....,Ym = gm (x1, .., xn) be a set of functions
of the n random variables. Now the joint moment generating function of Y1 , Y2 , .., Yk ,if it
exists, is


MY1 ,..,Yk (t1, .., tk ) = E et1Y1 +..+tk Yk
Z
Z
=
.....
et1 g1 (x1 ,..,xn)+..+tk gk (x1 ,..,xn) f (x1 , .., xn)dx1...dxn
xn

x1

for continuous variables

or


MY1 ,..,Yk (t1, .., tk ) = E et1Y1 +..+tk Yk
X
X
.....
et1 g1 (x1 ,..,xn)+..+tk gk (x1 ,..,xn) f (x1 , .., xn)
=
xn

x1

for discrete variables. If the resulting function can be recognized as the joint moment
generating function of some known probability distribution, it follows that the joint probability distribution of Y1 , Y2 , .., Yk has that probability distribution since moment generating
function when it exists is unique. The same applies to using the characteristic function.
This technique is usually used to determine the probability distribution of functions of
independent random variables.
ExampleFind the probability distribution of the sum of n independent random variables
X1 , X2 , .., Xn that have Poisson distribution with parameters 1 , 2 , .., n respectively.
P
Solution Let Z = ni=1 Xi ; the m.g.f of Z is given by

X
tZ
MZ (t) = E[e ] =
etz f (x1 , x2, .., xn)
z=0

z=0
Y
n
X

z=0 i=1
n X

etxi

n
Y

t(x1 +x2 +..+xn )

n
Y
ei xi
i

i=1

xi !

i xi
i
txi e

i=1 xi =0

xi !
ei xi i
xi !

ei(e 1) = e(1+2 +..+n )(e 1)

i=1

this is m.g.f of a Poisson distribution with rate parameter =

Poisson distribution with rate parameter .
10

Pn

i=1

i , therefore Z has