Probability and Statistics Unit 4

Unit 4 Moment Generating Function

Structure:
4.1 Introduction
Objectives
4.2 Moment generating function- Definition
Moments from Moment generating function
Properties of Moment generating function
4.3 Cumulants
Properties of Cumulants
4.4 Characteristic function
Properties of Characteristic function
4.5 Summary
4.6 Terminal Questions
4.7 Answers

4.1 Introduction
So far we have considered in detail only the two most important
characteristics of a random variable, namely, the mean and the variance.
We have seen how these attributes enter into the fundamental limit
theorems of probability, as well as into all sorts of practical calculations. We
have seen that the mean and variance of a random variable contain
important information about the random variable, or, more precisely, about
the distribution function of that variable. Now we shall see that the mean and
variance do not contain all the available information about the density
function of a random variable. Now we will discuss about moment
generating function (mgf). These are very powerful computational tool. They
make certain computations much shorter. In this unit, we'll first learn what a
moment-generating function is, and then we'll learn how to use moment
generating functions.
to find moments and functions of moments, such as and 2
to identify which probability mass function a random variable X follows

Sikkim Manipal University Page No.: 73

Probability and Statistics Unit 4

Objectives:
At the end of this unit the student should be able :
To learn the definition of a moment-generating function.
To learn how to use a moment-generating function to find the mean and
variance of a random variable.
To learn how to use a moment-generating function to identify which
probability mass function a random variable X follows.
To be able to apply the methods learned in the lesson to new problems.

4.2 Moment generating function Definition

Let X be a discrete random variable and let be its p.m.f . Then the

moment generating function of X is the expected value of the function ,

the variable t is just a parameter and is denoted by

For continuous random variable

provided the summation or integration is finite for some interval of t around

zero. That is, it is absolutely convergent for some positive integer h such
that .

Example: The pdf for the score X of a randomly selected student is given
below
X 1 2 3 4 5
f(X) 0.15 0.20 0.40 0.15 0.10

Then what is the moment generating function of the random variable X?

Sikkim Manipal University Page No.: 74

Probability and Statistics Unit 4

Solution: By definition, we have

= +

Example : Let N denotes the number of flips of a coin until a tail appears

and let . Find its moment generating function.

Solution: By definition, we have

= + +.

= + +.

Since the above series is GM with a =

Sikkim Manipal University Page No.: 75

Probability and Statistics Unit 4

Example: Find the moment generating function of random variable X whose

probability function is .

Solution: By definition of Moment generating function, we have

SAQ 1: If X is a random variable having the pdf x = 1,2,3,,

p+q = 1. Find m.g.f. of X .

SAQ 2: If is the pdf of a r.v. X.Find the m.g.f.

of X.

SAQ 3: Obtain the MGF of the random variable X having p.d.f

f ( x) x ,0 x 1
2 x, 1 x 2
0 otherwise
2
SAQ 4: If the random variable has the MGF M X (t ) , determine the
2t
Var(X).

SAQ 5: If X represents the outcome when a fair die is tossed. Find the MGF
of X and hence find E(X) and Var(X)

Sikkim Manipal University Page No.: 76

Probability and Statistics Unit 4

4.2.1 Moments from Moment generating function

We have by definition of Moment generating function

(4.1)

Where, , X is continuous distribution

= , X is discrete distribution is the rth moment of X about origin.

Thus, the rth moment about origin is the coefficient of in .

Moment about any point X = a

The moment generating function about X = a is defined as

Sikkim Manipal University Page No.: 77

Probability and Statistics Unit 4

Where , is the rth moment about the point X = a

Note: We can also calculate rth moment about origin by differentiating (4.1)
with respect to t, r times and then putting t = 0, we get

Hence,

Example: Calculate the first three moments about origin with the help of
mgf of probability density function .

Solution: By definition of mgf, we have

= (4.2)

Now, to find the first, second and third moments about origin we will
differentiating (4.2) once, twice and thrice with respect to t at t = 0
That is,

Sikkim Manipal University Page No.: 78

Probability and Statistics Unit 4

Example: Let X be a random variable and its probability mass function is

find the m.g.f. of X and hence its mean and
variance.
Solution: By definition,

Mean = (about origin)= 1/p

Variance = = =

Example: Find the moment generating function of the random variable

whose moments are .

Solution: By definition m.g.f is defined as

Sikkim Manipal University Page No.: 79

Probability and Statistics Unit 4

SAQ 6: The probability density function of the random variable is

. Find the m.g.f. of X and also find E(X)

and V(X).
x
1
f ( x) e 2 , x 0
SAQ 7: Let the random variable X have the p.d.f 2
0 otherwise
Find moment generating function, mean and variance
4.2.2 Properties of Moment generating function
Let X be a random variable then
, where c is constant.

Proof: By definition, we have

L.H.S. =

R.H.S. =

Thus, L.H.S. = R.H.S.

2. If are independent random variables, then the moment

generating function of their sum is given by:

That is, The moment generating function of the sum of a number of

independent random variable is equal to the product of their respective
moment generating functions.

Sikkim Manipal University Page No.: 80

Probability and Statistics Unit 4

Proof:

3. Let X be a random variable. Let U be new random variable obtained

from X by changing both the origin and scale in X as follows:
, where a and h are constants.

Then the Moment generating function of U is

= , where is the m.g.f of X about origin.

4.3 Cumulants
Let X be a random variable. Then cumulant of X is denoted by and is

defined by cumulant generating function

Thus,

= (4.3)

It is called the rth cumulant and it is the coefficient of in

Sikkim Manipal University Page No.: 81

Probability and Statistics Unit 4

Relation between moments and cumulants:

From (4.3), we have

Comparing the coefficients of like powers of t on both sides, we get

=( ) - 3( )

Or

Note: can also be obtained as follows

Sikkim Manipal University Page No.: 82

Probability and Statistics Unit 4

4.3.1 Properties of Cumulants

1. Suppose X is a random variable with nth cumulant. Then for any

has an nth cumulant and

2. The rth cumulant of the sum of the independent random variables is equal
to the sum of the rth cumulants of the individual variables.
That is ,

Where, are independent random variables.

3. Let X be a random variable and let be another random variable

obtained by changing its origin and scale. Then

By definition of cumulant, we have

Where are the rth cumulants of U and X respectively.

On comparing the coefficients form both the sides, we get

Thus, except the first cumulant, all other cumulants are independent of
change of origin. But they are not invariant of change of scale as the rth
cumulant of U is times the rth cumulant of the random variable X.

Sikkim Manipal University Page No.: 83

Probability and Statistics Unit 4

4.4 Characteristic function

Let X be a random variable. Then the characteristic function of X is denoted
as and is defined as
, (for discrete probability function)
= , for continuous probability function

Note:
1. Characteristic function always exists. There may be some cases
when the m.g.f does not exists.

2. For a distribution function, characteristic function is defined as

4.4.1 Properties of Characteristic function

1. Characteristic function also generates moments.
By definition of characteristic function, we have

=
Where , is the rth moment of X about origin.
Thus,

2. For any real variable t, we have

3. If the distribution function of a random variable is symmetrical about

zero, i.e., if implies , then is real
valued and even function of t.
Proof: We have
= , by putting (x = -y)

Sikkim Manipal University Page No.: 84

Probability and Statistics Unit 4

= because
=
Hence is an even function of t.

4. is uniformly continuous function in the interval. That is for all

there exists such that , whenever
Proof: Let h = s t. Assume without any loss of generality that .
Then

but as the function converges to zero and is dominated

by constant 2. Therefore by Lebesgue dominated Convergence
theorem, so for a given we can choose
h sufficiently small, say such that
5. The characteristic function of a + bX is
Proof: By definition,
6. The characteristic function of is the complex conjugate of
.
Proof: We know that the complex conjugate of is . Thus

7. If X is a random variable with characteristic function and if

exists, then

Proof: By definition,

Sikkim Manipal University Page No.: 85

Probability and Statistics Unit 4

Differentiating under the integral r times with respect to t, we get

=
=
=

Thus,

8. c being constant
9. If are independent random variables, then

10. The characteristic function of a convolution is

Proof: Put Then by definition

= where
=

11. If being constants, then

Example: Let X be a discrete random variable having support
and the probability mass function is given by

0,
Derive the characteristic function of X.

Sikkim Manipal University Page No.: 86

Probability and Statistics Unit 4

Solution: Using the definition of characteristic function, we have

=
=
=

=
Example: Use the characteristic function from the above example to derive
the variance of X.
Solution: We know Var(X) = E(X2)- E(X)2
The expected value of X is computed by taking first derivative of

=
Now, evaluating at t = 0 and dividing it by i, we get

=
=1
The second moment of X is computed by taking the second derivative of the
characteristic function. That is

=
At t = 0 and dividing it by , we get

=
= 5/3

Sikkim Manipal University Page No.: 87

Probability and Statistics Unit 4

Var(X) = E(X2)- E(X)2

= 5/3 1 = 2/3
SAQ 8: Find the Characteristic function of X whose pdf is given by
1 x
f ( x) e , x
2
SAQ 9: Find the characteristics function of the Laplace distribution with
x
pdf f ( x) e , x . Hence find its mean and variance.
2
SAQ 10: Find the characteristics function of the Poisson distribution.

4.5 Summary
In this unit, we studied the concept of moment generating function and
technique of finding moments from moment generating function. Also we
studied about the cumulates and characteristics function of random
variables with examples.

4.6 Terminal Questions

1. Prove that for a random variable X with probability density function
has mgf . (Hint: Use the change of variable

technique to integrate with respect to instead of x).

2. Find the variance of the random variable whose m.g.f is

3. Find the MGM of the binomial distribution

P( X r ) nC r p r q nr , r 0,1,2,...n and hence find its mean and
variance.
4. If X is a random variable whose density function is
f ( x) Ae x , 0 x
0 otherwise

Sikkim Manipal University Page No.: 88

Probability and Statistics Unit 4

(i) Find the value of A (ii) Mean of X(iii) Variance of X(iv) Moment about
origin (vi) Moment about mean.
3
5. If a random variable X has the MGF , Obtain the standard deviation
3t
of X.
6. Find the characteristics of X with pdf
1
f ( x) , a x b and x
ba

4.7 Answers
Self Assessment question
1. By definition of m.g.f.,

2. By definition,

1 t
3. M X (t ) 2
(e 1) 2
t
4. Var(X)=1/4

Sikkim Manipal University Page No.: 89

Probability and Statistics Unit 4

1 t
5. MGF= (e ..... e 6t ) , E(X)=7/2 and Var(X)=35/12.
6
6. The moment generating function of X is

For

E(X) = = Coefficient of t in

1
7. M X (t ) , E(X)=2 and Var(X)=4.
1 2t
1
8. CF=
1 2

Sikkim Manipal University Page No.: 90

Probability and Statistics Unit 4

2 2
9. CF= , E ( X ) 0 and Var ( X )

2 2
2
i
10. CF= e (1e ) .
Terminal Questions:

2. Variance =

3. M (t ) ( pe q) , E(X)=np and Var(X)=npq.

t n

4. A=1, Mean =1, Variance =1 and moments about origin 1,2,6,24: and
about mean 1,1,2,9.
5. S.D.=1/3

e ib e ia
6. CF=
i (b a)

Sikkim Manipal University Page No.: 91