Beruflich Dokumente
Kultur Dokumente
Exam P
Transformations and Order Statistics
of Continuous Random Variables
©2013 Yeng M. Chang (2015 update)
From my experience in tutoring for exam P/1, it seems that these topics consistently are the most
confusing topics for students. This document is meant to try to make these topics as clear as possible
for students as they study for this exam.
I assume that the reader knows everything about expected values, probability density functions,
(cumulative) distribution functions, and independence of continuous random variables in the exam
P syllabus. Although it is not very well emphasized in later exams, these two topics are very impor-
tant in exams MLC and C. Using the concepts presented here as it pertains to the material on these
courses can help further understanding of the material and even speed up calculations (especially
on exam MLC on multi-life models).
Unless stated otherwise, assume all random variables in this document are continuous random vari-
ables.
We go by the convention that if we write an equation for fX within a set of intervals, fX is 0
elsewhere. Similarly, if we write an equation for FX in an interval [a, b], for example, then FX (x) =
0 for x < a and FX (x) = 1 for x > b.
Numbered equations are essential for understanding the topics.
Contents
1 Transformations 2
1.1 The Cumulative Distribution Function (CDF) Method . . . . . . . . . . . . . . . . 2
1.2 Method of Transformations (Univariate) . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Method of Jacobians . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Order Statistics 11
2.1 Minimum of a Set of Random Variables . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Maximum of a Set of Random Variables . . . . . . . . . . . . . . . . . . . . . . . 12
1 Transformations
Let X be a random variable (either discrete or continuous). A transformation of X is a function of
X, which is also a random variable. We will also refer to any such transformation of X as simply
a “transformation.” For example, Y = 3X, Z = 3X + 4, W = |X| are all transformations of
X and are also random variables. We are specifically interested in finding the probability density
functions (PDFs) of transformations of continuous random variables.
2. Find FX .
Suppose Y = 5X + 4.
3. We have
( ) ( )
y−4 y−4
FY (y) = P (5X + 4 ≤ y) = P X ≤ = FX = 1 − e−(y−4)/10 .
5 5
4. Therefore,
( )
d [ ] −1 1 −(y−4)/10
fY (y) = 1 − e−(y−4)/10 = −e−(y−4)/10 = e , y > 4.
dy 10 10
Example 1.1.2. Suppose X follows the exponential distribution with mean 1, with
W = X2
W ≤w
√ X
0 w
√
Looking at the graph above, if X 2 ≤ w, then 0 < X ≤ w (notice that X > 0 by definition!).
Thus,
( )
FW (w) = P (W ≤ w) = P X 2 ≤ w
( √ )
=P X≤ w
(√ )
= FX w
√
= 1 − e− w
, w > 0.
This gives
d [ √ ] √ −1 1 √
fW (w) = 1 − e− w = −e− w √ = √ e− w , w > 0.
dw 2 w 2 w
Example 1.1.3. Suppose X follows the uniform distribution in the interval (−5, 5) and W = X 2 .
Then
x+5
FX (x) = , − 5 < x < 5.
10
It can be shown that 0 ≤ W < 25. Now for a fixed value 0 ≤ w < 25, consider the set of all values
of X such that W = X 2 ≤ w.
W = X2
W ≤w
√ √ X
− w 0 w
FW (w) = P (W ≤ w)
( √ √ )
=P − w≤X≤ w
( √ ) ( √ )
=P X ≤ w −P X ≤− w
√ √
w+5 − w+5
= −
√10 10
2 w
= , 0 ≤ w < 25.
10
Therefore, ( )
2 1 1
fW (w) = √ = √ , 0 < w < 25.
10 2 w 10 w
Exercise 1.1.4. Repeat Example 1.1.3, except set W = |X|. Show that W follows a uniform
distribution in the interval [0, 5).
Exercise 1.1.6 (Discrete Random Variables). Suppose X is a discrete random variable where
Notice that there is no need to use the CDF at all in the discrete case to get the probability mass
function of W . Furthermore, verify that
∑
pW (w) = 1.
all w
Exercise 1.1.7 (Challenging). Suppose X follows the exponential distribution with mean 1, and
{
X 2, 0 < X ≤ 2
Y = √
X, X > 2.
The graph of Y is below.
Y
4 Y = X2
√
Y = X
√
2
X
0 2
Show that
√ √
− y
1 − e , 0<y≤ 2
√ √ √
FY (y) = g(y) = 1 − e− 2 + e−2 − e− y + e−2 − e−y , 2<y≤4
1/4 2
g(4) + e−16 − e−y
2
y > 4.
(Thanks to the Math StackExchange for assistance.)
Exercise 1.1.8. Suppose Z follows the standard normal distribution, so that
1
fZ (z) = √ e−z /2 ,
2
− ∞ < z < ∞.
2π
The symbol Φ denotes the CDF of Z. Consider the transformation Y = ceil(Z). Y is known as the
ceiling of Z; for every non-integer value of Z, Y is the rounded-up value to the nearest integer. If
Z is an integer, its value remains the same. For example, if {−4.7, −3, 1, 5.3} are values of Z, then
{−4, −3, 1, 6} are the corresponding values of Y .
Show that the probability mass function of Y , pY (y) ≡ Φ(y) − Φ(y − 1) for any integer y. Fur-
thermore, using Φ(2) = 0.9772, find the probability that Y = 2.
Exercise 1.1.9. Consider what is given in Exercise 1.1.8. Let W = floor(Z) be the floor of Z; for
every non-integer value of Z, W is the rounded-down value to the nearest integer. If Z is an integer,
its value remains the same. For example, if {−4.7, −3, 1, 5.3} are values of Z, then {−5, −3, 1, 5}
are the corresponding values of W .
Using Φ(1) = 0.8413, find the probability that W = 1.
Y = g(X)
y = g(x)
g(X) ≤ y
Y = g −1 (X)
X
x = g −1 (y)
Remember: our goal is to write P (g(X) ≤ y) in terms of FX . If g(X) ≤ y, what does this say
about the values of X? If you look at the graph above, if g(X) ≤ y, then X ≤ g −1 (y). This is
because all of the values of X in the shaded region above are those which are less than x = g −1 (y),
so that X ≤ x = g −1 (y). Hence
( ) ( )
FY (y) = P (g(X) ≤ y) = P X ≤ g −1 (y) = FX g −1 (y) .
Taking derivatives on both sides, we have
( ) d [ −1 ]
fY (y) = fX g −1 (y) · g (y) ,
dy
d −1
and [g (y)] > 0 since g −1 is increasing, so we don’t need the absolute value symbols.
dy
Now suppose Y = h(X) is strictly decreasing. Then, as before, FY (y) = P (g (X) ≤ y).
Y = g(X)
y = g(x)
g(
X
)≤
y
Y = g −1 (X)
X
x = g −1 (y)
( ) d [ −1 ] ( ) d [ −1 ]
fY (y) = −fX g −1 (y) · g (y) = fX g −1 (y) · − g (y) ,
dy dy
d −1 d
but notice that [g (y)] < 0 (as you can see in the graph above). So − [g −1 (y)] > 0. Thus,
dy dy
in both cases (i.e., when Y is either increasing or decreasing), we can write
( −1 ) d [ −1 ]
fY (y) = fX g (y) · g (y) .
dy
Now: an explanation on why I didn’t number the equation previously. Notice that if Y = g(X), we
can say that X = g −1 (Y ). So, in fact, you don’t need the g in the notation at all. I prefer writing
instead Y = Y (X) and X = X(Y ) (with the understanding that X = Y −1 on the functions),
and then switching to lowercase letters when referring to particular values of X and Y . By writing
Y (X) and X(Y ), it makes it clear (in my opinion) that you want Y in terms of X, and X in terms
of Y respectively. So, I prefer the equation written as
d
fY (y) = fX (x(y)) · [x(y)] .
(1.2.1)
dy
4. Find fX .
Example 1.2.1 (Example 1.1.1 with Method of Transformations). Suppose X follows the ex-
ponential distribution with mean 2, and Y = 5X + 4.
1. Then X > 0.
Y −4
2. We have Y = 5X + 4, so that X = .
5
3. Since X > 0, Y = 5X + 4 > 5(0) + 4 = 4, so Y > 4.
1
4. fX (x) = e−x/2 , x > 0.
2
y−4
5. We have x(y) = , as demonstrated in step 2. So,
5
( ) [ ] ( )
y−4 d y − 4 1 −(y−4)/10 1 1
fY (y) = fX · = e
= e−(y−4)/10 , y > 4.
5 dy 5 2 5 10
Notice how this answer matches with the answer in Example 1.1.1.
Example 1.2.2 (Example 1.1.2 with Method of Transformations). Suppose X follows the ex-
ponential distribution with mean 1, and W = X 2 .
1. Then X > 0.
√
2. We have W = X 2 , so that X = ± W . This may seem to be a problem, since we should√
only have one
√ function for X. But only one of these functions applies
√ when X > 0: W.
We ignore − W since none of the values of X are negative. So X = W for our purposes.
√
5. We have x(w) = w, so
(√ ) d [√ ] 1 √
fW (w) = fX
w · w = √ e− w , w > 0.
dw 2 w
Exercise 1.2.3. Explain why the Method of Transformations cannot be applied to Example 1.1.3.
1
fZ (z) = √ e−z /2 ,
2
− ∞ < z < ∞.
2π
Consider Y = eZ . Y is said to have a log-normal distribution. Find the equation for fY (y), y > 0.
1. Find fX,Y .
4. Calculate
∂ ∂
∂w1 [x(w1 , w2 )] ∂w2 [x(w1 , w2 )]
J =
.
∂ ∂
[y(w1 , w2 )] [y(w1 , w2 )]
∂w1 ∂w2
5. Calculate det(J).
Exercise 1.3.1. Suppose X and Y are independent exponential random variables with means 1 and
2, respectively. Let W = X + Y and Z = 2Y . Find the joint density function of W and Z.
1
fX,Y (x,y) = e−x e−y/2 , x > 0, y > 0.
2
3. We have
Z
X =W −Y =W −
2
and
Z
Y = .
2
4. We have ∂
∂ 1
[x(w, z)] [x(w, z)] 1
∂w ∂z 2
J = = .
∂ ∂ 1
[y(w, z)] [y(w, z)] 0
∂w ∂z 2
( )
1 1 1
5. det(J) = 1 − (0) = . We don’t need to worry about the absolute value, since the
2 2 2
determinant is already positive.
6. Therefore,
1 1
fW1 ,W2 (w1 , w2 ) = fX,Y (w − z/2, z/2) (1/2) = e−(w−z/2) e−(z/2)/2 = e−w ez/4 ,
4 4
where w > 0 and z > 0.
2 Order Statistics
Let X = {X1 , X2 , . . . , Xn } be a set of independent random variables. We are interested in the order
statistics X(1) = min (X) and X(n) = max (X).
It is our task to find the PDF and CDF of both X(1) and X(n) .
If all of the random variables in X are identically distributed as well with CDF F , then P (Xi > x) =
1 − F (x) for all i, and
FX(1) (x) = 1 − [1 − F (x)]n . (2.1.2)
You can take derivatives of either (2.1.1) or (2.1.2) to get the PDFs. I do not recommend mem-
orizing the equation (2.1.2), but to memorize the method to get to (2.1.1), since given random
variables will not always be identically distributed.
Example 2.1.1. Let X1 , X2 , and X3 be exponentially distributed with means 1, 2, and 3 respec-
tively. Find the PDF of X(1) = min{X1 , X2 , X3 }.
FX(1) (x) = 1−P (X1 > x) P (X2 > x) P (X3 > x) = 1−e−x e−x/2 e−x/3 = 1−e−11x/6 , x > 0.
d [ ] 11
fX(1) (x) = FX(1) (x) = e−11x/6 , x > 0.
dx 6
6
This is the PDF of an exponential distribution with mean .
11
If all of the random variables in X are identically distributed as well with CDF F , then
P (Xi ≤ x) = F (x) for all i, and
FX(n) (x) = [F (x)]n . (2.2.2)
As with min (X), I recommend memorizing the methods to get (2.2.1) rather than the formula
(2.2.2).
Example 2.2.1. Let X1 , X2 , and X3 be exponentially distributed with means 1, 2, and 3 respec-
tively. Find the CDF of X(3) = max{X1 , X2 , X3 }.
We have, as shown in (2.2.1):
( )( )( )
FX(3) (x) = P (X1 ≤ x) P (X2 ≤ x) P (X3 ≤ x) = 1 − e−x 1 − e−x/2 1 − e−x/3 , x > 0.
Order Statistics:
2. Find the CDF of the order statistic you need by using the methods
to get (2.1.1) or (2.2.1).
3. Take the derivative of this function to get the PDF of the order
statistic.