Sie sind auf Seite 1von 262

Continuous Distributions

by R.J. Reed
These notes are based on handouts for lecture modules given jointly by myself and the late Jeff Harrison.
A list of the distributions considered in this report is given on page 255.
There are many additional theoretical results in the exercises—full solutions are provided at the end.
The colour red indicates a hyperlink. The hyperlink to a page number is always to the top of the page, more
precisely to the headline of the page. Thus if a link is given as page 40(§13.1), it is better to click on the section
reference—the page number is useful if you have a printed copy.
Version 2, January 2019. Version 3, August 2019. Version 4, March 2020.

Contents

1 Foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Revision of some basic results: 3. Exercises: 5. Order statistics: 8. Stable distributions: 11.
Infinitely divisible distributions: 15. Exercises: 15.

2 Univariate Continuous Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19


The uniform distribution: 19. Exercises: 25. The exponential distribution: 27. Exercises: 31.
The Gamma and χ2 distributions: 33. Exercises: 37. The beta and arcsine distributions: 40.
Exercises: 44. The normal distribution: 46. Exercises: 49. The lognormal distribution: 52.
Exercises: 55. The power law and Pareto distributions: 56. Exercises: 60. The t, Cauchy and
F distributions: 63. Exercises: 68. 2
Non-central χ , t and F : 71. Exercises: 76. Size, shape and
related characterization theorems: 77. Exercises: 81. Laplace, Rayleigh and Weibull distributions: 81.
Exercises: 83. The logistic distribution: 86. Extreme value distributions: 89. Exercises: 96.
The Lévy and inverse Gaussian distributions: 98. Exercises: 102. Other distributions with bounded
support: 103. Exercises: 104. Other distributions with unbounded support: 107. Exercises: 110.

3 Multivariate Continuous Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117


General results: 117. Exercises: 121. The bivariate normal: 123. Exercises: 127. The multivariate
normal: 130. Exercises: 139. Quadratic forms of normal random variables: 141. Exercises: 156.
The bivariate t distribution: 159. The multivariate t distribution: 160. Exercises: 163. The Dirichlet
distribution: 163. Exercises: 165.

Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Chapter 1.
Exercises on page 5: 167. Exercises on page 15: 171.
Chapter 2.
Exercises on page 25: 175. Exercises on page 31: 181. Exercises on page 37: 185. Exercises on page 44: 189.
Exercises on page 49: 193. Exercises on page 55: 198. Exercises on page 60: 200. Exercises on page 68: 206.
Exercises on page 76: 213. Exercises on page 81: 215. Exercises on page 83: 216. Exercises on page 96: 220.
Exercises on page 102: 223. Exercises on page 104: 226. Exercises on page 110: 229.
Chapter 3.
Exercises on page 121: 235. Exercises on page 127: 236. Exercises on page 139: 241. Exercises on
page 156: 244. Exercises on page 163: 248. Exercises on page 165: 249.

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

Comments are welcome—even comments such as “not useful because it omits xxx”. Please send comments and details of errors to my
Wordpress account, bcgts.wordpress.com, where the most up-to-date version of these notes will be found.

Bayesian Time Series Analysis by R.J. Reed Mar 10, 2020(20:25) Page 1
Page 2 Mar 10, 2020(20:25) Bayesian Time Series Analysis
CHAPTER 1

Foundations
1 Revision of some basic results
1.1 Conditional variance and expectation. For any random vector (X, Y ) such that E[Y 2 ] is finite, the
conditional variance of Y given X is defined to be
2
var( Y |X ) = E[ Y 2 |X ] − E[Y |X] = E (Y − E[Y |X])2 |X
 
(1.1a)
This is a function of X. Taking expectations of the first equality and using the standard result var(Z) = E[Z 2 ] −
( E[Z] )2 with Z = E[Y |X] shows that
  
var(Y ) = E var(Y |X) + var E[Y |X] (1.1b)
Equation(1.1b) is often called the Law of Total Variance and is probably best remembered in the following form:
var(Y ) = E[conditional variance] + var(conditional mean)
This is similar to the decomposition in the analysis of variance. A generalization is given in exercise 2.13 on
page 7.
Definition(1.1a).For any random vector (X, Y, Z) such that E[XY ], E[X] and E[Y ] are all finite, the condi-
tional covariance between X and Y given Z is defined to be
cov(X, Y |Z) = E[XY |Z] − E[X|Z] E[Y |Z]
An alternative definition is
   
cov(X, Y |Z) = E X − E[X|Z] Y − E[Y |Z] Z (1.1c)
Note that cov(X, Y |Z) is a function of Z. Using the results cov(X, Y ) = E[XY ] − E[X]E[Y ] and
  
cov E[X|Z], E[Y |Z] = E E[X|Z] E[Y |Z] − E[X]E[Y ]
gives the Law of Total Covariance
  
cov(X, Y ) = E cov(X, Y |Z) + cov E[X|Z], E[Y |Z] (1.1d)
This can be remembered as
cov(X, Y ) = E[conditional covariance] + cov(conditional means)
Setting X = Y in the Law of Total Covariance gives the Law of Total Variance.

1.2 Conditional independence. Recall that X ⊥⊥ Y | Z means that X and Y are conditionally independent
given Z. By definition
X ⊥⊥ Y | Z iff P[X ≤ x, Y ≤ y|Z] = P[X ≤ x|Z] P[Y ≤ y|Z] a.e. on (x, y) ∈ R2 .
By first considering simple random variables and then non-negative random variables by taking limits, we see that
if X ⊥⊥ Y | Z and E[X], E[Y ] and E[XY ] are all finite then
E[XY |Z] = E[X|Z] E[Y |Z] (1.2a)
Example(1.2a). Conditional independence does not imply independence.
Here is a simple demonstration: suppose box 1 contains two fair coins and box 2 contains two coins which have heads on
both sides. A box is chosen at random—denote the result by Z. A coin is selected from the chosen box and tossed—denote
the result by X; then the other coin from the chosen box is tossed independently of the first coin—denote the result by Y .
Clearly X ⊥⊥ Y | Z. However
5 3
P[X = H, Y = H] = but P[X = H] = P[Y = H] =
8 4

Bayesian Time Series Analysis by R.J. Reed Mar 10, 2020(20:25) §1 Page 3
Page 4 §1 Mar 10, 2020(20:25) Bayesian Time Series Analysis

1.3 The hazard function. Suppose X is a non-negative absolutely continuous random variable with density
function fX and distribution function FX . Then the survivor function is the function SX (t) = 1 − FX (t) = P[X >
t] and the hazard function is the function
fX (t) fX (t)
hX (t) = = for t > 0. (1.3a)
1 − FX (t) SX (t)
If the random variable X represents the lifetime of some item, then the hazard function is a measure of the failure
rate given the item has already lasted for a time t.
The cumulative hazard function is the function
Z t
HX (t) = hX (u) du for t > 0. (1.3b)
0
The following relations follow:
d
hX (t) = − ln SX (t) HX (t) = −ln SX (t) SX (t) = e−HX (t) fX (t) = hX (t)e−HX (t)
dt
It follows that any one of the functions hX , HX , SX and fX determines all the others.
The function g: [0, ∞) → [0, ∞) can be used as a hazard function iff
Z ∞
g ≥ 0 and g(t) dt = ∞
0
For a justification of this result see exercise 2.19 on page 7.
Example(1.3a). Suppose X has the exponential density fX (x) = λe−λx for x > 0. Find the hazard and cumulative hazard
functions.
Solution. Now FX (t) = 1 − e−λt . Hence hX (t) = λ which is constant. Also HX (t) = λt for t > 0.
Example(1.3b). Suppose β > 0 and γ > 0 and X has the Weibull density
βxβ−1 −(x/γ)β
fX (x) = e for x > 0.
γβ
Find the hazard and cumulative hazard functions.
Solution. The distribution function of X is FX (x) = 1 − exp(−xβ /γ β ) for x > 0 and hence
βxβ−1 xβ
hX (t) = β
and HX (t) = β for t > 0.
γ γ
Note that if β < 1 then the hazard function is decreasing and if β > 1 then the hazard function is increasing. If β = 1 then
the hazard function is constant—this is just the exponential density again.

Further results about the hazard function are given in exercise 2.20 and exercise 2.21 on page 7.
1.4 Skewness. Skewness is a measure of the asymmetry of a distribution.
Definition(1.4a).Suppose X is a random variable with finite expectation µ and finite variance σ 2 . Then the
skewness of X is defined to be
"  #
X −µ 3 E[ (X − µ)3 ]
skew[X] = E = (1.4a)
σ σ3
A distribution is positively skewed iff skew(X) > 0, and then the density function has a long tail to the right of
the mean. Exercise 2.22 on page 7 shows that
E[X 3 ] − 3µσ 2 − µ3
skew[X] = (1.4b)
σ3
Example(1.4b). Suppose X is a random variable with finite mean µ and finite variance σ 2 . Suppose further that the
d d
distribution of X is symmetric about a—this means that X − a = a − X, where the notation W = Z means that the random
variables W and Z have the same distribution. Show that E[X] = a and skew[X] = 0.
Solution. The fact that E[X −a] = E[a−X] implies E[X] = a. Similarly E[(X −a)3 ] = E[(a−X)3 ] implies E[(X −a)3 ] = 0
and hence skew[X] = 0.
See exercise 2.23 on page 7 for an example of a distribution which is not symmetric but which has zero skewness.
1 Foundations Mar 10, 2020(20:25) §2 Page 5

1.5 Kurtosis. Kurtosis is a measure of the “peakedness” of a distribution or, equivalently, the fatness of the tails.
Definition(1.5a). Suppose X is a random variable with finite expectation µ and finite variance σ 2 . Then the
kurtosis of X is defined to be "  #
X −µ 4 E[ (X − µ)4 ]
κ[X] = E = (1.5a)
σ σ4
Exercise 2.25 on page 8 shows that, provided E[X 3 ] is finite,
E[X 4 ] − 4µE[X 3 ] + 6µ2 σ 2 + 3µ4
κ[X] = (1.5b)
σ4
Example(1.5b). Suppose the random variable X has the uniform distribution on (0, 1). Find the skewness and kurtosis
of X. R1 R1 R1
Solution. Now E[X] = 0 x dx = 12 , E[X 2 ] = 0 x2 dx = 13 and hence var[X] = 12 1
. Also E[X 3 ] = 0 x3 dx = 14 . Using
1
equation(1.4b) gives E[(X − µ)3 ] = E[X 3 ] − 3µσ 2 − µ3 = 41 − 3 × 24 − 18 = 0 and hence skew[X] = 0. Using equation(1.5b)
gives E[(X − µ)4 ] = E[X 4 ] − 4µE[X 3 ] + 6µ2 σ 2 + 3µ4 = 51 − 48 + 48
6 3 1 1
 1 1
+ 16 = 80 . Hence κ[X] = 80 ( 12 12 ) = 95 .
1.6 Location-scale families. Informally, a location-scale family is a family of distributions with two parameters:
one parameter determines the location and the other parameter, which must be non-negative, determines the scale.
Definition(1.6a). Suppose X and Y are real-valued random variables. Then X and Y have the same type or
belong to the same location-scale family iff there exist a ∈ R and b > 0 such that
d
Y = a + bX
Equivalently the distribution functions FX and FY havethe same type iff there exist a ∈ R and b > 0 such that
y−a
FY (y) = FX for all y ∈ R. (1.6a)
b
Two distributions have the same type if they represent the same quantity but in different physical units. The
parameter a determines the location and the parameter b determines the scale. It is easy to see that the relation of
being of the same type is an equivalence relation (reflexive, symmetric and transitive) on the class of all distribution
functions.
Definition(1.6b). The family of distributions F is said to be a location-scale family iff for all distribution
functions F ∈ F and G ∈ F there exist a ∈ R and b > 0 such that F (y) = G (y − a)/b for all y ∈ R.
2
Example(1.6c). Suppose X ∼ N (µX , σX ) and Y ∼ N (µY , σY2 ), then X and Y have the same type. We say that the family
of distributions {N (µ, σ 2 ) : µ ∈ R, σ > 0} is a location-scale family of distributions.
Similarly, a family of distributions is called a scale family of distributions if any one member can be expressed as
a positive multiple of another.
Definition(1.6d). The family of distributions F is said to be ascale family iff for all distribution functions
F ∈ F and G ∈ F there exists b > 0 such that F (y) = G y/b for all y ∈ R.
Example(1.6e). Suppose Xλ ∼ exponential (λ). In §9.1, we shall see that this implies αXλ ∼ exponential (λ/α).
Hence the family of distributions {exponential (λ) : λ > 0} is a scale family of distributions because
d µ
Xλ = Xµ
λ
• If X and Y are in the same location-scale family which has finite expectations, then there exists b > 0 such
d
that Y − µY = b(X − µX ). Hence skew(X) = skew(Y ) and κ(X) = κ(Y ).
• By differentiating equation(1.6a), if X and Y are in the same location-scale family and both have absolutely
continuous distributions with densities fX and fY respectively, then there exist a ∈ R and b > 0 such that
y−a
 
1
fY (y) = fX
b b

2 Exercises (exs-basic.tex)

Revision exercises.
2.1 The following assumptions are made about the interest rates for the next three years. Suppose the interest rate for year 1
is 4% p.a. effective. Let V2 and V3 denote the interest rates in years 2 and 3 respectively. Suppose V2 = 0.04 + U2 and
V3 = 0.04 + 2U3 where U2 and U3 are independent random variables with a uniform distribution on [−0.01, 0.01]. Hence
V2 has a uniform distribution on [0.03, 0.05] and V3 has a uniform distribution on [0.02, 0.06].
(a) Find the expectation of the accumulated amount at the end of 3 years of £1,000 invested now.
(b) Find the expectation of the present value of £1,000 in three years’ time. [Ans]
Page 6 §2 Mar 10, 2020(20:25) Bayesian Time Series Analysis

2.2 Suppose X and Y are positive random variables such that U = X/(X + Y ) and V = X + Y are independent.
(a) Prove that (X 2 + Y 2 )/XY and X + Y are independent.
(b) Prove that (X + Y )2 /XY and (X + Y )2 are independent. [Ans]
2.3 Uniform to triangular. Suppose X and Y are i.i.d random variables with the uniform distribution uniform(−a, a),
where a > 0. Find the density of W = X + Y and sketch the shape of the density. [Ans]
1
2.4 Suppose the random variable X has the density fX (x) = 2 for −1 < x < 1. Find the density of Y = X 4 . [Ans]
2.5 Suppose X is a random variable with X > 0 a.e. and such that both E[X] and E[ 1/X ] both exist. Prove that E[X] +
E[ 1/X ] ≥ 2. [Ans]
2.6 (a) Suppose X is a random variable with X ≥ 0 and density function f . Let F denote the distribution function of X.
Show that Z ∞ Z ∞
E[X] = [1 − F (x)] dx and E[X r ] = rxr−1 [1 − F (x)] dx for r = 1, 2, . . . .
0 0
(b) Now suppose X is a random variable with a ≤ X ≤ b where −∞ < a < b < ∞. Prove that
Z b Z b
E[X] = a + [1 − F (x)] dx = b − F (x) dx [Ans]
a a
2.7 Suppose X and Y are independent random variables with absolutely continuous distributions with densities fX and fY
respectively.
(a) Find the densities of W = Y − X and Z = Y + X.
(b) Find the densities of V = |Y − X| and T = (Y − X)2 . [Ans]
2.8 Suppose a > 0 and X and Y are i.i.d. random variables with the density
ex
f (x) = for −∞ < x < ln(a).
a
−w
Show that the density of W = |X − Y | is e for w > 0; this is the exponential (1) distribution. [Ans]
2.9 Suppose X1 , X2 , . . . , Xn are independent and identically distributed positive random variables and Sn = X1 + · · · + Xn .
(a) Show that  
1 1
E ≥
Sn nµ
Hint: use Jensen’s Inequality which states that if X is a random variable which takes values in the interval I and has
a finite expectation and φ : I → R is a convex function, then φ (E[X]) ≤ E [φ(X)].
(b) Show that   Z ∞
1 n
E = E[e−tX ] dt [Ans]
Sn 0
2.10 Suppose X1 , X2 , . . . , Xn are independent and identically distributed positive random variables.
(a) Suppose E[1/Xi ] is finite for all i. By using the arithmetic-geometric mean inequality, show that E[1/Sj ] is finite
for all j = 2, 3, . . . , n where Sj = X1 + · · · + Xj .
(b) Suppose E[Xi ] and E[1/Xi ] both exist and are finite for all i. Show that
 
Sj j
E = for j = 1, 2,. . . , n. [Ans]
Sn n
2.11 Suppose X and Y are positive random variables with E[Y ] > 0. Suppose further that X/Y is independent of X and X/Y
is independent of Y .
2
(a) Suppose E[X 2 ], E[Y 2 ] and E[ X /Y 2 ] are all finite. Show that E[X] = E[ X/Y ] E[Y ]. Hence deduce that there exists
b ∈ R with X/Y = b almost everywhere.
(b) Use characteristic functions to prove there exists b ∈ R with X/Y = b almost everywhere. [Ans]
Pn
2.12 Recursive calculation of the sample variance. Suppose {xn }n≥1 is a sequence in R. Let tn = i=1 xi and vn =
Pn 2
i=1 (x i − t n /n) for n ≥ 1. Hence
n
X t2
vn = x2i − n
n
i=1
(a) Show that t2n+1 = t2n + x2n+1 + 2xn+1 tn for n ≥ 1.
(b) Hence show that
 2
n tn
vn+1 = vn + xn+1 − for n ≥ 1.
n+1 n
Pn Pn
It follows that if s2n = i=1 (xi − xn )2 /(n − 1) then ns2n+1 = (n − 1)s2n + n(xn+1 − xn )2 )/(n + 1) where xn = i=1 xi /n.
[Ans]
1 Foundations Mar 10, 2020(20:25) §2 Page 7

Conditional expectation.
2.13 Suppose (X, Y ) is a random vector and g : R → R such that E[Y 2 ] < ∞ and E[g(X)2 ] < ∞. Show that
h 2 i   h 2 i
E Y − g(X) = E var[Y |X] + E E[Y |X] − g(X) [Ans]

2.14 (For this question, you need the results that if X has the uniform distribution on (0, b) which is denoted uniform(0, b),
then E[X] = b/2, E[X 2 ] = b2 /3 and var[X] = b2 /12.) Suppose X ∼ uniform(0, 1) and the distribution of Y given
X = x is uniform(0, x). By using the law of total expectation E[Y ] = E[ E[Y |X] ] and the law of total variance, which
is equation(1.1b), find E[Y ] and var[Y ]. [Ans]
2.15 The best predictor of the random variable Y . Given the random vector (X, Y ) with E[X 2 ] < ∞ and E[Y 2 ] < ∞,
find that random variable Yb = g(X) which is a function of X and provides the best predictor of Y . Precisely, show that
Yb = E[Y |X], which is a function of X, minimizes
 2 
E Y −Y b [Ans]

2.16 Suppose the random vector (X, Y ) satisfies 0 < E[X 2 ] < ∞ and 0 < E[Y 2 ] < ∞. Suppose further that E[Y |X = x] =
a + bx a.e..
(a) Show that µY = a + bµX and E[XY ] = aµX + bE[X 2 ]. Hence show that cov[X, Y ] = b var[X] and E[Y |X] =
µY + ρ σσXY
(X − µX ) a.e..
   2
(b) Show that var E(Y |X) = ρ2 σY2 and E Y − E(Y |X) = (1 − ρ2 )σY2 .
(Hence if ρ ≈ 1 then Y is near E(Y |X) with high probability; if ρ = 0 then the variation of Y about E(Y |X) is the
same as the variation about the mean µY .)
(c) Suppose E(X|Y ) = c + dY a.e. where bd < 1 and d 6= 0. Find expressions for E[X], E[Y ], ρ2 and σY2 /σX 2
in terms
of a, b, c and d. [Ans]
2.17 Best linear predictor of the random variable Y . Suppose the random vector (X, Y ) satisfies 0 < E[X 2 ] < ∞ and
0 < E[Y 2 ] < ∞.
Find a and b such that therandom variable Yb = a + bX provides the best linear predictor of Y . Precisely, find a ∈ R and
b ∈ R which minimize E ( Y − a − bX )2 .
Note. Suppose E[Y |X] = a0 + b0 X. By exercise 2.15, we know that E[Y |X] = a0 + b0 X is the best predictor of Y .
Hence a0 + b0 X is also the best linear predictor of Y . [Ans]
2.18 Suppose the random vector (X, Y ) has the density

6 2
f(X,Y ) (x, y) = 7 (x + y) for x ∈ [0, 1] and y ∈ [0, 1];
0 otherwise.
(a) Find the best predictor of Y .
(b) Find the best linear predictor of Y .
(c) Compare the plots of the answers to parts (a) and (b) as functions of x ∈ [0, 1]. [Ans]
Hazard function, skewness and kurtosis.
2.19 Suppose g is a function : [0, ∞) → [0, ∞) with
Z ∞
g ≥ 0 and g(t) dt = ∞
0
Show that g can be used as the hazard function of a distribution. [Ans]
2.20 Suppose T1 and T2 are independent non-negative absolutely continuous random variables with hazard functions h1 and
h2 respectively. Let T = min{T1 , T2 }. Show that hT (t) = h1 (t) + h2 (t) for t > 0. [Ans]
2.21 Suppose the random variable T is non-negative and absolutely continuous. Let Y = HT (T ) where HT is the cumulative
hazard function of T . Prove that Y ∼ exponential (1). [Ans]
2.22 Suppose X is a random variablewith finite expectation µ and finite variance σ 2 .
(a) Show that
E[X 3 ] − 3µσ 2 − µ3
skew[X] = (2.22a)
σ3
(b) If a ∈ R and b > 0 show that skew[a + bX] = skew[X].
(c) If a ∈ R and b < 0 show that skew[a + bX] = −skew[X]. [Ans]
2 2
2.23 Suppose X ∼ N (−2, σ = 1), Y ∼ N (1, σ = 2), and I has the Bernoulli distribution with P[I = 1] = and 1/3
P[I = 0] = 2/3. Suppose further that X, Y and I are independent and Z = IX + (1 − I)Y . Show that E[Z] = 0,
var[Z] = 11/3 and skew[Z] = 0 but the distribution of Z is not symmetric. [Ans]
Page 8 §3 Mar 10, 2020(20:25) Bayesian Time Series Analysis

2.24 Suppose the random variable B has the Bernoulli distribution with P[B = 1] = p and P[B = 0] = 1 − p where p ∈ [0, 1].
(a) Find skew[B].
(b) Find κ[B]. [Ans]
2 3
2.25 Suppose X is a random variable with finite expectation µ, finite variance σ , and finite third moment E[X ].
(a) Show that
E[X 4 ] − 4µE[X 3 ] + 6µ2 σ 2 + 3µ4
κ[X] = (2.25a)
σ4
(b) If a ∈ R and b 6= 0 show that κ[a + bX] = κ[X]. [Ans]

3 Order statistics
3.1 Basics. Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with an absolutely continuous distribution
which has distribution function F and density f . Suppose further that
X1:n , X2:n , . . . , Xn:n
denote the order statistics of X1 , X2 , . . . , Xn . This means
X1:n = min{X1 , . . . , Xn }
Xn:n = max{X1 , . . . , Xn }
and the random variables X1:n , X2:n , . . . , Xn:n consist of X1 , X2 , . . . , Xn arranged in increasing order; hence
X1:n ≤ X2:n ≤ · · · ≤ Xn:n

3.2 Finding the density of (X1:n , . . . , Xn:n ). Let g(y1 , . . . , yn ) denote the density of (X1:n , . . . , Xn:n ).
Note that (X1:n , . . . , Xn:n ) can be regarded as a transformation T of the vector (X1 , . . . , Xn ).
• Suppose n = 2. Let A1 = {(x1 , x2 ) ∈ R2 : x1 < x2 } and A2 = {(x1 , x2 ) ∈ R2 : x1 > x2 }. Then T : A1 → A1
is 1 − 1 and T : A2 → A1 is 1 − 1. Hence for all (y1 , y2 ) ∈ A1 , the density g(y1 , y2 ) of (X1:2 , X2:2 ) is
fX ,X (y1 , y2 ) fX1 ,X2 (y2 , y1 )
g(y1 , y2 ) = 1 2 +
∂(y ,y ) ∂(y1 ,y2 )
∂(x11 ,x22 ) ∂(x1 ,x2 )
fX1 ,X2 (y1 , y2 ) fX1 ,X2 (y2 , y1 )
= +
|1| |−1|
= 2f (y1 )f (y2 )
• Suppose n = 3. For this case, we need A1 , A2 , A3 , A4 , A5 and A6 where
A1 = {(x1 , x2 , x3 ) ∈ R3 : x1 < x2 < x3 }
A2 = {(x1 , x2 , x3 ) ∈ R3 : x1 < x3 < x2 }
etc. There are 3! = 6 orderings of (x1 , x2 , x3 ). So this leads to
g(y1 , y2 , y3 ) = 3!f (y1 )f (y2 )f (y3 )
• For the general case of n ≥ 2, we have
g(y1 , . . . , yn ) = n!f (y1 ) · · · f (yn ) for y1 < · · · < yn . (3.2a)

3.3 Finding the distribution of Xr:n by using distribution functions. Dealing with the maximum is easy:
n
Y
Fn:n (x) = P[Xn:n ≤ x] = P[X1 ≤ x, . . . , Xn ≤ x] = P[Xi ≤ x] = {F (x)}n
i=1
and by differentiation
fn:n (x) = nf (x) {F (x)}n−1
Provided the random variables are positive, using the result of exercise 2.6 on page 6 gives the following expression
for the expectation of the maximum:
Z ∞
1 − {F (x)}n dx
 
E[Xn:n ] =
0
1 Foundations Mar 10, 2020(20:25) §3 Page 9

Now for the distribution of the minimum, X1:n :


n
Y
P[X1:n > x] = P[X1 > x, . . . , Xn > x] = P[Xi > x] = {1 − F (x)}n
i=1
F1:n (x) = 1 − P[X1:n > x] = 1 − {1 − F (x)}n
f1:n (x) = nf (x) {1 − F (x)}n−1
Provided the random variables are positive, using the result of exercise 2.6 on page 6 gives
Z ∞
E[X1:n ] = {1 − F (x)}n dx
0
Now for the general case, Xr:n where 2 ≤ r ≤ n − 1. The event {Xr:n ≤ x} occurs iff at least r random variables
from X1 , . . . , Xn are less than or equal to x. Hence
n  
X n
P[Xr:n ≤ x] = {F (x)}j {1 − F (x)}n−j (3.3a)
j
j=r
n−1  
X n
= {F (x)}j {1 − F (x)}n−j + {F (x)}n
j
j=r
Differentiating gives
n−1  
X n
fr:n (x) = jf (x) {F (x)}j−1 {1 − F (x)}n−j −
j
j=r
n−1  
X n
(n − j)f (x) {F (x)}j {1 − F (x)}n−j−1 + nf (x) {F (x)}n−1
j
j=r
n
X n!
= f (x) {F (x)}j−1 {1 − F (x)}n−j −
(j − 1)!(n − j)!
j=r
n−1
X n!
f (x) {F (x)}j {1 − F (x)}n−j−1
j!(n − j − 1)!
j=r
n!
= f (x) {F (x)}r−1 {1 − F (x)}n−r (3.3b)
(r − 1)!(n − r)!
Note that equation(3.3b) is true for all r = 1, 2, . . . , n.
3.4 Finding the distribution of Xr:n by using the density of (X1:n , . . . , Xn:n ). Recall that the density of
(X1:n , . . . , Xn:n ) is g(y1 , . . . , yn ) = n!f (y1 ) · · · f (yn ) for y1 < · · · < yn .
Integrating out yn gives
Z ∞
 
g(y1 , . . . , yn−1 ) = n!f (y1 ) · · · f (yn−1 ) f (yn )dyn = n!f (y1 ) · · · f (yn−1 ) 1 − F (yn−1 )
yn−1
Now integrate out yn−1 :
Z ∞  
g(y1 , . . . , yn−2 ) = n!f (y1 ) · · · f (yn−2 ) f (yn−1 ) 1 − F (yn−1 ) dyn−1
yn−2
 2
1 − F (yn−2 )
= n!f (y1 ) · · · f (yn−2 )
2!
Integrating out yn−2 gives
 3
1 − F (yn−3 )
g(y1 , . . . , yn−3 ) = n!f (y1 ) · · · f (yn−3 )
3!
By induction for r = 1, 2, . . . , n − 1 we have
[1 − F (yr )]n−r
g(y1 , . . . , yr ) = n!f (y1 ) · · · f (yr ) for y1 < y2 < · · · < yr .
(n − r)!
Assuming r ≥ 3 and integrating over y1 gives
Page 10 §3 Mar 10, 2020(20:25) Bayesian Time Series Analysis
y2
[1 − F (yr )]n−r [1 − F (yr )]n−r
Z
g(y2 , . . . , yr ) = n! f (y1 ) · · · f (yr ) dy1 = n!F (y2 )f (y2 ) · · · f (yr )
y1 =−∞ (n − r)! (n − r)!
Integrating over y2 gives
[F (y3 )]2 [1 − F (yr )]n−r
g(y3 , . . . , yr ) = n! f (y3 ) · · · f (yr ) for y3 < · · · < yr .
2! (n − r)!
And so on, leading to equation(3.3b).
3.5 Joint distribution of ( Xj:n , Xr:n ) by using the density of (X1:n , . . . , Xn:n ). Suppose X1:n , . . . , Xn:n
denote the order statistics from the n random variables X1 , . . . , Xn which have density f (x) and distribution
function F (x). Suppose 1 ≤ j < r ≤ n; then the joint density of (Xj:n , Xr:n ) is
 j−1  r−1−j  n−r
f(j:n,r:n) (u, v) = cf (u)f (v) F (u) F (v) − F (u) 1 − F (v) (3.5a)
where
n!
c=
(j − 1)!(r − 1 − j)!(n − r)!
The method used to derive this result is the same as that used to derive the distribution of Xr:n in the previous
paragraph.
Example(3.5a). Suppose X1 , . . . , Xn are i.i.d. random variables with density f (x) and distribution function F (x). Find
expressions for the density and distribution function of Rn = Xn:n − X1:n , the range of X1 , . . . , Xn .
Solution. The density of (X1:n , Xn:n ) is
 n−2
f(1:n,n:n) (u, v) = n(n − 1)f (u)f (v) F (v) − F (u) for u < v.
Now use the transformation R = Xn:n − X1:n and T = X1:n . The absolute value of the Jacobian is one. Hence
 n−2
f(R,T ) (r, t) = n(n − 1)f (t)f (r + t) F (r + t) − F (t) for r > 0 and t ∈ R.
Integrating out T gives
Z ∞
 n−2
fR (r) = n(n − 1) f (t)f (r + t) F (r + t) − F (t) dt
t=−∞
The distribution function is, for v > 0,
Z v Z ∞
 n−2
FR (v) = n(n − 1) f (t)f (r + t) F (r + t) − F (t) dt dr
r=0 t=−∞
Z ∞ Z v
 n−2
= n(n − 1) f (t) f (r + t) F (r + t) − F (t) dr dt
t=−∞ r=0
Z ∞ Z ∞
h n−1 v  n−1
=n f (t) F (r + t) − F (t) dt = n f (t) F (v + t) − F (t) dt
t=−∞ r=0 t=−∞

3.6 Joint distribution of ( Xj:n , Xr:n ) by using distribution functions. Suppose u < v and then define the
counts N1 , N2 and N3 as follows:
X n Xn Xn
N1 = I(Xi ≤ u) N2 = I(u < Xi ≤ v) and N3 = n − N1 − N2 = I(Xi > v)
i=1 i=1 i=1
     
Now P X1 ≤ u = F (u); also P u < X1 ≤ v = F (v) − F (u) and P X > v = 1 − F (v). It follows  that the
vector (N1 , N2 , N3 ) has the multinomial distribution with probabilities F (u), F (v) − F (u), 1 − F (v) .

The joint distribution function of Xj:n .Xr:n is:
n X
`
    X  
P Xj:n ≤ u and Xr:n < v = P N1 ≥ j and (N1 + N2 ) ≥ r = P N1 = k and N1 + N2 = `
`=r k=j
n X
`
X n!  k  `−k  n−`
= F (u) F (v) − F (u) 1 − F (v)
k!(` − k)!(n − `)!
`=r k=j

Now the joint density of Xj:n .Xr:n is:
∂2  
f(j:n,r:n) (u, v) = P Xj:n ≤ u and Xr:n < v
∂u∂v
1 Foundations Mar 10, 2020(20:25) §4 Page 11

Using the abbreviations a = F (u), b = F (v) − F (u) and c = 1 − F (v) gives



n X `
∂ X n!
ak−1 b`−k cn−`
 
P Xj:n ≤ u and Xr:n < v = f (u)
∂u  (k − 1)!(` − k)!(n − `)!
`=r k=j

`−1
X n! 
− ak b`−k−1 cn−`
k!(` − k − 1)!(n − `)! 
k=j
n
X n!
= f (u) aj−1 b`−j cn−`
(j − 1)!(` − j)!(n − `)!
`=r
and hence
∂2  n!
aj−1 br−j−1 cn−r

P Xj:n ≤ u and Xr:n < v = f (u)f (v)
∂u∂v (j − 1)!(r − j − 1)!(n − r)!
as required—see equation(3.5a) on page 10.
3.7 Asymptotic distributions. The next proposition gives the asymptotic distribution of the median.
Proposition(3.7a). Suppose the random variable X has an absolutely
  continuous distribution with density f
which is positive and continuous at the median, µ̃. Suppose in = n/2 + 1. Then
√  D
2 nf (µ̃) Xin :n − µ̃ =⇒ N (0, 1) as n → ∞.
1
This means that Xin :n is asymptotically normal with mean µ̃ and variance 4n(f (µ̃) )2
.
Proof. See page 223 in [A RNOLD et al.(2008)].
For other results, see chapter 8 in [A RNOLD et al.(2008)].
Example(3.7b). Suppose U1 , . . . , U2n−1 are i.i.d. random variables with the uniform(0, 1) distribution. Find the asymptotic
distribution of the median of U1 , . . . , U2n−1 .
√  D
Solution. The median is Un:(2n−1) and hence by the proposition 8n − 4 Un:(2n−1) − 1/2 =⇒ N (0, 1) as n → ∞.
√ √
Of course 8n/ 8n − 4 → 1 as n → ∞. Hence by Lemma 23 on page 263 of [F RISTEDT &G RAY(1997)] we have
√  D
8n Un:(2n−1) − 1/2 =⇒ N (0, 1) as n → ∞ and
 
1 t
lim P Un:(2n−1) − < √ = Φ(t) for t ∈ R.
n→∞ 2 8n

4 Stable distributions
4.1 The basic definition. It is well known that if X1 , X2 , . . . , Xn are i.i.d. random variables with the N (µ, σ 2 )
d √ √
distribution, then X1 + · · · + Xn ∼ N (nµ, nσ 2 ); hence X1 + · · · + Xn = (n − n)µ + nX where X ∼ N (µ, σ 2 ).
This means that X1 + · · · + Xn and X are of the same type1 for all n ∈ {1, 2, . . . , }.
A distribution is stable iff its shape is preserved under addition up to shift and scale. The formal definition is
Definition(4.1a). The random variable X has a stable distribution iff it is non-constant and for every n ∈
{1, 2, . . .}, if X1 , . . . , Xn are i.i.d. random variables with the same distribution as X then the random variables
X1 + · · · + Xn and X have the same type; this means that for every n ∈ {1, 2, . . .} there exist an ∈ R and
d
bn > 0 such that, then X1 + · · · + Xn = an + bn X.
The {an } are called the centring parameters and the {bn } are called the scaling parameters. We need to insist on
non-constant in the definition—otherwise X = µ, an = (n − 1)µ and bn = 1 would be a solution.
If the centring constant is always zero, then the distribution is said to be strictly stable.
Definition(4.1b). The random variable X has a strictly stable distribution iff it is non-constant and for every
n ∈ {1, 2, . . .} there exist bn > 0 such that, if X1 , . . . , Xn are i.i.d. random variables with the same distribution
d
as X then X1 + · · · + Xn = bn X.
Similarly, the distribution function F is stable if for every n ∈ {1, 2, . . .} there exist constants an ∈ R and bn > 0
such that if X1 , . . . , Xn are independent and have distribution function F then b−1 n (X1 + · · · + Xn ) + an has

1
Same type is defined in §1.6 on page 5.
Page 12 §4 Mar 10, 2020(20:25) Bayesian Time Series Analysis

distribution function F . Again the non-constant requirement is essential—otherwise X = µ and bn = n would be


a solution.
d
Note that if a + bX = c + dX, then either a = c and b = d or X is constant. It follows that if we exclude constant
random variables, then the values of the parameters an and bn in definitions(4.1a) and (4.1b) are unique.
Example(4.1c). Suppose X has the normal N (µ, σ 2 ) distribution. Show that X is stable and find an and bn .
d √ √ √
√ X1 + · · · + Xn (n − n)µ + nX. Hence the centring parameter is an = (n − n)µ and the scaling parameter
Solution. =
is bn = n.
If X has a stable distribution with a finite variance, then by equating expectations and variances we see that
d √ √
X1 + · · · + Xn = (n − n)µ + nX.
4.2 Basic properties.
Proposition(4.2a). Suppose the random variable X is stable with centring parameter an and scaling parame-
ter bn .
(a) Suppose c ∈ R and d ∈ R. Then the random variable Y = c + dX is stable with centring parameter
dan + (n − bn )c and scaling parameter bn .
(b) The random variable −X is stable with centring parameter −an and scaling parameter bn .
Proof.
d
(a) Suppose Y1 , . . . , Yn are i.i.d. random variables with the same distribution as Y . Then Y1 +· · ·+Yn = nc+d(X1 +· · ·+Xn )
d
where X1 , . . . , Xn are i.i.d. random variables with the same distribution as X. Now X1 + · · · + Xn = an + bn X. Hence
d d
Y1 + · · · + Yn = (nc + dan ) + dbn X = dan + (n − bn )c + bn Y .
d
(b) Suppose X1 , . . . , Xn are i.i.d. random variables with the same distribution as X. Then X1 + · · · + Xn = an + bn X.
d
Hence −X1 − · · · − Xn = −an − bn X; also −X1 , . . . , −Xn are i.i.d. random variables with the same distribution as
−X. Hence result.
Proposition(4.2b). Suppose X has a stable distribution with centring parameter an and scaling parameter bn
and suppose Y has a stable distribution with centring parameter cn and the same scaling parameter bn . Suppose
further that X and Y are independent. Then the random variable Z = X + Y has a stable distribution with
centring parameter an + cn and scaling parameter bn .
d
Proof. Suppose Z1 , . . . , Zn are i.i.d. random variables with the same distribution as Z. Then Z1 + · · · + Zn = (X1 + · · · +
Xn ) + (Y1 + · · · + Yn ) where X1 , . . . , Xn are i.i.d. with the distribution of X and Y1 , . . . , Yn are i.i.d. with the distribution
d d d d
of Y . Hence X1 + · · · + Xn = an + bn X and Y1 + · · · + Yn = cn + bn Y . Hence Z1 + · · · + Zn = (an + cn ) + bn (X + Y ) =
(an + cn ) + bn Z as required.

Proposition(4.2c).Suppose X and Y are i.i.d. random variables with a stable distribution which has centring
parameters an and scaling parameters bn . Then X − Y has a strictly stable distribution with scaling parame-
ter bn .
Proof. This follows immediately from propositions(4.2a) and (4.2b).
This last proposition means that X − Y has a strictly stable distribution which is symmetric about zero.2 By using
this result, proofs about stable distributions can sometimes be reduced to proofs about strictly stable distributions
which are symmetric about 0.
4.3 More advanced results.
Proposition(4.3a). Suppose X has a stable distribution with centring parameters {an } and scaling parame-
ters {bn }. Then there exists α ∈ (0, 2] such that
bn = n1/α for every n ∈ {1, 2, . . .}.
The constant α is called the characteristic exponent of the distribution.
Proof. This proof is based on pages 170–171 of [F ELLER(1971)].
First we assume that the distribution of X is symmetric and strictly stable. Part(b) of exercise 6.18 shows that for all
d
integers m ≥ 1 and n ≥ 1 we have bm+n X = bm X1 + bn X2 where X1 and X2 are i.i.d. random variables with the same
 x ≥ 0 we have
distribution as X. Hence for   
bm+n bm bn
X1 ≥ 0, X2 ≥ x ⊆ X= X1 + X2 ≥ x
bn bm+n bm+n

2
The random variable X has a distribution which is symmetric about 0 iff X and −X has the same distribution.
1 Foundations Mar 10, 2020(20:25) §4 Page 13

Independence of X1 and X2 implies


bm+n
P[X ≥ x] ≥ P[X1 ≥ 0] P[X2 ≥ x]
bn
Because the distribution of X is symmetric, we must3 have P[X ≥ 0] ≥ 1/2. Hence
 
1 bm+n
P[X ≥ x] ≥ P X ≥ x for all x ≥ 0 and all integers m ≥ 1 and n ≥ 1.
2 bn
We now show that the set  
bn
: m ≥ 1, n ≥ 1
bm+n
is bounded above. For suppose not: then there exists a sequence (mj , nj ) such that
bnj bmj +nj
→ ∞ as j → ∞ and hence → 0 as j → ∞
bmj +nj bnj
Hence
bmj +nj
 
1 1
P[X > x] ≥ lim P X ≥ x ≥
j→∞ 2 bnj 4
1
and 1 − FX (x) = P[X > x] ≥ 4 for all x gives the required contradiction. We have shown there exists K > 0 such that
bn
≤ K for all integers m ≥ 1 and n ≥ 1.
bm+n
To clarify the following argument, we shall now treat b as a function b : {1, 2, . . .} → (0, ∞). By part(a) of exercise 6.18
we know that b(rk ) = [b(r)]k for all integers k ≥ 1 and r ≥ 1.
Fix r ∈ {1, 2, . . .} and let α = ln(r)/ ln( b(r) ); hence b(r) = r1/α . Similarly fix s ∈ {1, 2, . . .} with s 6= r and let
β = ln(s)/ ln( b(s) ); hence b(s) = s1/β . We need to show α = β.
For every j ∈ {1, 2, . . .}, we have b(rj ) = [b(r)]j = rj/α . Similarly for every k ∈ {1, 2, . . .}, we have b(sk ) = [b(s)]k =
sk/β .
Fix ` ∈ {1, 2, . . .}; then there exists j ∈ {1, 2, . . .} such that rj ≤ s` ≤ rj+1 which implies s`/β ≤ rj/β r1/β . Hence
b(s` ) ≤ [b(rj )]α/β r1/β . Hence for every ` ∈ {1, 2, . . .} there exists j ∈ {1, 2, . . .} such that
b(rj ) [b(rj )](β−α)/β
K≥ ≥
b(s` ) r1/β
j j
As ` → ∞ so j → ∞ and b(r ) = [b(r)] → ∞ because b(r) > 1. Hence β ≤ α. Interchanging r and s shows that
α ≤ β. Hence α = β and b(n) = n1/α for all n ∈ {1, 2, . . .}.
It remains to prove that α ≤ 2. Suppose α > 2; we shall obtain a contradiction.
Now
Z ∞ Z ∞ ∞ Z 2k
2 2
√ X √
E[X ] = P[X > x] dx = P[|X| > x] dx = P[|X| > x] dx (4.3a)
0 0 k=1 2k−1

Now for every t > 0 and n ∈ {1, 2, . . .} we have


P[ |X1 + · · · + Xn | > tbn ] = P[ bn |X| > tbn ] = P[ |X| > t ]
Hence there exists t > 0 such that P[ |X1 + · · · + Xn | > tbn ] ≤ 41 . Using lemma 2 on page 149 of [F ELLER(1971)] gives
1  1
1 − e−nP[|X|>tbn ] ≤ P[ |X1 + · · · + Xn | ≥ tbn ] ≤
2 4
So this implies there exists K2 such that nP[|X| > tbn ] < K2 for all n. Substituting x = tbn gives P[|X| > x] ≤
(K2 tα )x−α . Hence
Z 2k

P[|X| > x] dx ≤ K2 tα 2k(1−α/2)
2k−1
If α > 2 then equation(4.3a) implies E[X 2 ] < ∞. If a stable distribution distribution has a finite variance, then by
d √ √
equating expectations and variances we see that X1 + · · · + Xn = (n − n)µ + nX and hence α = 2. So we have a
contradiction and we must have α ∈ (0, 2].
Now suppose X is stable with centring parameters {an } and scaling parameters {bn }. Proposition(4.2c) implies the
required result.
Proposition(4.3b). Every stable distribution is continuous.
Proof. This proof is based on page 215 of [F ELLER(1971)].
First suppose X is a random variable with a strictly stable distribution with scaling factors {bn }. Part(b) of exercise 6.18
3
By definition, symmetric means that the distributions of X and −X are the same. Let α = P[X > 0] = P[X < 0]. Then
2α + P[X = 0] = 1; hence P[X ≥ 0] = α + P[X = 0] = 1 − α ≥ 1/2.
Page 14 §4 Mar 10, 2020(20:25) Bayesian Time Series Analysis

d
shows that for all integers m ≥ 1 and n ≥ 1 we have bm+n X = bm X1 +bn X2 where X1 and X2 are i.i.d. random variables
with the same distribution as X.
Suppose there exists x ∈ R with x 6= 0 such that P[X = x] = p > 0. Then
 
2 (bm + bn )x
p = P[X1 = x] P[X2 = x] = P[(X1 = x) ∩ (X2 = x)] ≤ P X =
bm+n
2
So we have infinitely many points {xj } with P[X = xj ] ≥ p ; contradiction. Hence
P[X = x] = 0 for all x 6= 0.
d
Now suppose P[X = 0] = p > 0. By proposition(4.3a) we know that b1 = 11/α = 1. Using b2 X = b1 X1 + b1 X2 = X1 + X2
shows that P[X1 + X2 = 0] = P[X = 0] = p. But we have already established that P[X = x] = 0 for all x 6= 0; hence
P[X1 + X2 = 0] = P[X1 = 0] P[X2 = 0] = p2 . Hence p = p2 and hence p = 0. This proves X has a continuous distribution.
Now suppose X has a stable distribution. Let V = X1 − X2 where X1 and X2 are i.i.d. random variables with the same
distribution as X. Hence V has a strictly stable distribution and so is continuous. Suppose P[X = x] = p > 0; then
p2 = P[X1 = x, X2 = x] ≤ P[V = 0] which gives a contradiction. Hence the distribution of X is continuous.
Now for the characteristic function:
Proposition(4.3c). Suppose X has a stable distribution with characteristic exponent α. Then there exists
β ∈ [−1, 1], c ∈ R and d > 0 such that for all t ∈ R we have
 h πα i 
E[eitX ] = exp itc − dα |t|α 1 − iβ sgn(t) tan( ) if α 6= 1.
  2  (4.3b)
2iβ
E[eitX ] = exp itc − d|t| 1 + sgn(t) ln(|t|) if α = 1.
π
where
−1 if t < 0;
(
sgn(t) = 0 if t = 0;
1 if t > 0.
Proof. See pages 204–207 of [B REIMAN(1968)].
If X has a stable distribution with a finite variance, then then by equating expectations and variances we see that
d √ √
X1 + · · · + Xn = (n − n)µ + nX. Hence α = 2 and the characteristic function is
E[eitX ] = exp itc − d2 t2 [1 − iβ sgn(t) tan(π)] = exp itc − d2 t2
 

Hence X ∼ N (c, σ 2 = 2d2 ). Hence the only stable distribution with finite variance is the normal distribution;
also, the only stable distribution with α = 2 is the normal distribution.
If X1 and X2 are independent and have stable distributions with the same characteristic exponent, then, by propo-
sition(4.3a) they have the same scaling parameter. Hence it follows by proposition(4.2b) that X1 + X2 has a stable
distribution. This leads to an alternative definition of a stable distribution.
Proposition(4.3d). The random variable X has a stable distribution iff for every γ1 > 0 and γ2 > 0 there
exist a ∈ R and b > 0 such that if X1 and X2 are i.i.d. random variables with the same distribution as X then
d
γ1 X1 + γ2 X2 = a + bX.
Proof.
⇐ Setting γ1 = γ2 = 1 shows that there exist a2 ∈ R and b2 > 0 such that if X1 and X2 are i.i.d. random variables
d
with the same distribution as X then X1 + X2 = a2 + b2 X. We now proceed by induction. Suppose X1 , . . . , Xn , Xn+1
are i.i.d. random variables with the distribution of X. By the induction assumption, there exist an ∈ R and bn > 0 such
d d
that X1 + · · · + Xn = an + bn X. By independence, X1 + · · · + Xn + Xn+1 = an + bn X1 + Xn+1 where X1 and Xn+1 are
i.i.d. with the same distribution as X. Using the given assumption shows that there exist c ∈ R and bn+1 > 0 such that
d d
bn X1 + Xn+1 = c + bn+1 X. Hence X1 + · · · + Xn+1 = (an + c) + bn+1 X. Hence the result by induction.
⇒ By exercise 6.17.
1 Foundations Mar 10, 2020(20:25) §6 Page 15

5 Infinitely divisible distributions


5.1 The definition.
Definition(5.1a). The random variable X : (Ω, F , P) → (R, B) has an infinitely divisible distribution iff for
every n ∈ {1, 2, . . .} there exists i.i.d random variables X1 , X2 , . . . , Xn such that
d
X = X1 + · · · + Xn
The key result is that stable implies infinitely divisible.
Proposition(5.1b). Suppose the random variable X has a stable distribution. Then X has an infinitely divisible
distribution.
Proof. Suppose n ∈ {1, 2, . . .} and X1 , X2 , . . . , Xn are i.i.d. random variables with the same distribution as X. By
definition (4.1a), the definition of stable distribution, we know there exist an ∈ R and bn > 0 such that
d
an + bn X = X1 + · · · + Xn
Hence
n
d X Xj − an /n
X = = Y1 + · · · + Yn
bn
j=1
where Y1 , . . . , Yn are i.i.d. Hence result.
The concept of infinite divisibility is important when deriving various limit theorems in probability theory—see
for example chapter 16 in [F RISTEDT &G RAY(1997)] and chapters 6 and 9 in [F ELLER(1971)].
5.2 Two examples. These two examples depend on properties of distributions considered later in these notes
and they can safely be omitted on a first reading. First, a distribution which is stable and infinitely divisible.
d
Example(5.2a). Suppose X ∼ N (µ, σ 2 ) and n ∈ {1, 2, . . .}. Then X = Y1 + · · · + Yn where Y1 , . . . , Yn are i.i.d. random
2
variables with the N (µ/n, σ /n) distribution.

Now for a distribution which is infinitely divisible but not stable.


d
Example(5.2b). Suppose X ∼ gamma(k, α) and n ∈ {1, 2, . . .}. Then X = Y1 + · · · + Yn where Y1 , . . . , Yn are
i.i.d. random variables with the gamma(k/n, α) distribution—see exercise 12.6 on page 38.

6 Exercises (exs-orderstable.tex)

Order statistics.
6.1 Suppose X1 and X2 are i.i.d. random variables with the uniform(0, 1) distribution. Let Y denote the point which is
closest to an endpoint—either 0 or 1.
(a) Find the distribution of Z, the distance from Y to the nearest endpoint.
(b) Find the distribution of Z, the distance from 0 to Y . [Ans]
6.2 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the uniform uniform(0, 1) distribution.
(a) Find the distribution of Xj:n . (b) Find E[Xj:n ]. [Ans]
6.3 Suppose X1 , X2 , X3 and X4 are i.i.d. random variables with the uniform(0, 1) distribution.
(a) Find the density of (X3:4 , X4:4 ).
(b) Find P[X3:4 + X4:4 ≤ 1]. [Ans]
6.4 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the uniform uniform(0, 1) distribution. Define (Y1 , Y2 , . . . , Yn )
by
X1:n X2:n X(n−1):n
Y1 = , Y2 = , . . . , Yn−1 = , Yn = Xn:n
X2:n X3:n Xn:n
Show that Y1 , . . . , Yn are independent and that V1 = Y1 , V2 = Y22 , . . . , Vn = Ynn are i.i.d. [Ans]
6.5 Suppose X1 , X2 , X3 and X4 are i.i.d. random variables with the uniform uniform(0, 1) distribution. Find the distributions
of Y = X3:4 − X1:4 and Y = X4:4 − X2:4 . [Ans]
6.6 Suppose X1 , X2 and X3 are i.i.d. random variables with the uniform uniform(0, 1) distribution. Find the conditional
density of X2:3 given (X1:3 , X3:3 ). [Ans]
Page 16 §6 Mar 10, 2020(20:25) Bayesian Time Series Analysis

6.7 Suppose X and Y are i.i.d. random variables with support in the interval [a, b] where −∞ < a < b < ∞.
(a) Let Z = max{X, Y }. Prove that
b Z b
2 2
E[Z] = z {FX (z)} − {FX (z)} dz
a a

(b) Let V = min{X, Y }. Prove that


b Z b
2 2
E[V ] = −v {1 − FX (v)} + {1 − FX (v)} dv
a a

(c) Check the value of E[Z] + E[V ]. [Ans]


6.8 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with order statistics X1:n , X2:n , . . . , Xn:n . Find an expression for
E[ X1 |X1:n , X2:n , . . . , Xn:n ]. [Ans]
6.9 Suppose X1 , . . . , Xn are i.i.d. random variables with a distribution which is symmetric about 0. Let FX denote the
distribution function of this distribution. If Y = |X1 |, then Y has distribution function FY with FY (x) = 2FX (x) − 1.
Suppose Y1 , . . . , Yn are i.i.d. random variables with distribution function FY .
(a) Prove that for r ∈ {1, . . . , n}
" r−1   n  
#
1 X n X n
E[Xr:n ] = n E[Y(r−k):(n−k) ] − E[Y(k−r+1):k ] (6.9a)
2 k k
k=0 k=r
(b) Prove that for r ∈ {1, . . . , n} and m ∈ {1, 2, . . . .}
" r−1   n  
#
m 1 X n m m
X n m
E[Xr:n ] = n E[Y(r−k):(n−k) ] + (−1) E[Y(k−r+1):k ]
2 k k
k=0 k=r
(c) Prove that for 1 ≤ j < r ≤ n
" j−1   r−1  
1 X n X n
E[Xj:n Xr:n ] = n E[Y(j−k):(n−k) Y(r−k):(n−k) ] − E[Y(k−j+1):k ]E[Y(r−k):(n−k) ]
2 k k
k=0 k=j
n  
#
X n
+ E[Y(k−r+1):k Y(k−j+1):k ]
k
k=r
(d) Prove that for 1 ≤ j < r ≤ n, m1 ∈ {1, 2, . . .} and m2 ∈ {1, 2, . . .},
" j−1   r−1  
m1 m2 1 X n m1 m2 m1
X n m1 m2
E[Xj:n Xr:n ] = n E[Y(j−k):(n−k) Y(r−k):(n−k) ] + (−1) E[Y(k−j+1):k ]E[Y(r−k):(n−k) ]
2 k k
k=0 k=j
n  
#
m1 +m2
X n m1 m2
+ (−1) E[Y(k−r+1):k Y(k−j+1):k ] [G OVINDARAJULU(1963)] [Ans]
k
k=r

6.10 Suppose k > r. It is known4 that if the random variable X has an absolutely continuous distribution with distribution
function F then the conditional distribution function P[Xk:n < y|Xr:n = x] is the same as the distribution function of
the (k − r)th order statistic in a sample of size (n − r) from the distribution function

F (y)−F (x)
F1 (y) = 1−F (y) if y > x;
0 otherwise.

Suppose X1 and X2 are i.i.d. absolutely continuous non-negative random variables with density function f (x) and
distribution function F (x). By using the above result, show that X2:2 − X1:2 is independent of X1:2 if and only if
X ∼ exponential (λ). [Ans]
6.11 Suppose X1 , X2 , . . . , Xn are i.i.d. absolutely continuous non-negative random variables with density function f (x) and
distribution function F (x). Define the vector (Y1 , Y2 , . . . , Yn ) by
X2:n Xn:n
Y1 = X1:n , Y2 = , . . . , Yn =
X1:n X1:n
(a) Find an expression for the density of the vector (Y1 , Y2 , . . . , Yn ) in terms of f and F .
(b) Hence derive expressions for the density of the vector (Y1 , Y2 ) = (X1:n , X2:n/X1:n ) and the density of the random
variable Y1 = X1:n . [Ans]

4
For example, page 38 of [G ALAMBOS &KOTZ(1978)].
1 Foundations Mar 10, 2020(20:25) §6 Page 17

6.12 Record values. Suppose X0 , X1 , X2 , . . . are i.i.d. random variables with an absolutely continuous distribution. Let T
denote the index value of the first variable which is greater than X0 . Hence
{ N = 1 } = { X1 > X0 }
{ N = 2 } = { X1 < X0 , X2 > X0 }
{ N = 3 } = { X1 < X0 , X2 < X0 , X3 > X0 } etc.
Find the distribution of N and E[N ]. [Ans]
Stable distributions.
6.13 Suppose the random variable X has a stable distribution with characteristic exponent α 6= 1. Prove there exists β ∈ R
such that X − β has a strictly stable distribution. Hint: Express Snm as the sum of m independent random variables
each with the same distribution as X1 + · · · + Xn . [Ans]
6.14 Suppose the random variable X has a stable distribution with characteristic exponent α = 1. Prove that anm = man +
d
nam and X1 + · · · + Xn = n ln(n) + nX, and hence deduce E[X] does not exist. [Ans]
6.15 Suppose X has a strictly stable distribution with characteristic exponent α ∈ (0, 2]. Prove that for integers m ≥ 1 and
d
n ≥ 1 we have m1/α X1 + n1/α X2 = (n + m)1/α X. [Ans]
6.16 This question uses the same notation as in equation(4.3b) for the characteristic function of a stable distribution. Suppose
X1 has a stable distribution with characteristic exponent α and parameters {c1 , d1 , β1 } and X2 has a stable distribution
with the same characteristic exponent α and parameters {c2 , d2 , β2 }. Suppose further that X1 and X2 are independent.
Show that X1 + X2 has a stable distribution with parameters {c = c1 + c2 , d = (dα α 1/α
1 + d2 ) , β} where
α α
β1 d1 + β2 d2
β= [Ans]
dα α
1 + d2
6.17 (a) Suppose the random variable X has a stable distribution with characteristic exponent α 6= 1 and characteristic
function with parameters {c, d, β}. Suppose γ > 0. Show that γX has a stable distribution with characteristic
exponent α 6= 1 and characteristic function with parameters {γc, γd, β}.
(b) Suppose X1 and X2 are i.i.d. random variables with a stable distribution with characteristic exponent α 6= 1 and
characteristic function with parameters {c, d, β}. Suppose γ1 > 0 and γ2 > 0. Prove there exist a ∈ R and b > 0
d
such that γ1 X1 + γ2 X2 = a + bX.
(c) Suppose X1 and X2 are i.i.d. random variables with a stable distribution with characteristic exponent α = 1 and
characteristic function with parameters {c, d, β}. Suppose γ1 > 0 and γ2 > 0. Prove there exist a ∈ R and b > 0
d
such that γ1 X1 + γ2 X2 = a + bX. [Ans]
6.18 Suppose the random variable X has a strictly stable distribution with scaling factors {bn }.
(a) Prove that bmn = bm bn for all integers m ≥ 1 and n ≥ 1. Hence prove that for integers k ≥ 1 and r ≥ 1, if n = rk
then bn = bkr . Prove also that b1 = 1 and br > 1 for r ∈ {2, 3, . . .}.
Hint: Express Snm as the sum of m independent random variables each with the same distribution as X1 + · · · + Xn .
d
(b) Prove that for all integers m ≥ 1 and n ≥ 1 we have bm+n X = bm X1 + bn X2 where X1 and X2 are i.i.d. random
variables with the same distribution as X. [Ans]
Page 18 §6 Mar 10, 2020(20:25) Bayesian Time Series Analysis
CHAPTER 2

Univariate Continuous Distributions

7 The uniform distribution


7.1 Definition of the uniform distribution.
Definition(7.1a). Suppose a ∈ R, b ∈ R and a < b. Then the random variable X has the uniform distribution
uniform(a, b) iff X has density

 1
for x ∈ (a, b).
f (x) = b − a
0 otherwise.
The distribution function is
0 if x < a;

x−a
F (x) = if x ∈ (a, b);
 b−a
1 if x > b.
The uniform distribution is also called the rectangular distribution.
If X ∼ uniform(0, 1) and Y = a + (b − a)X then Y ∼ uniform(a, b). It follows that the family of distributions
{uniform(a, b) : a ∈ R, b ∈ R, a < b} is a location-scale family—see definition(1.6b) on page 5.
Moments. The moments E[X n ] are finite for n 6= 1:
Z b Z b
a+b a2 + ab + b2 (b − a)2
E[X] = xf (x) dx = E[X 2 ] = x2 f (x) dx = var[X] =
a 2 a 3 12
Z b n+1 n+1
b −a
E[X n ] = xn f (x) dx = for n 6= −1, n ∈ R.
a (n + 1)(b − a)
The moment generating function and characteristic function.
(b−a)t
 tb
e − eta

Z b tx
e  2 sin( 2 )ei(a+b)t/2
for t 6
= 0; for t 6= 0;

E[etX ] = dx = t(b − a) and E[eitX ] = t(b − a)
a b−a for t = 0.
1 for t = 0.
 
1
fX (x) ................ FX (x) ...........
.... .....
... ..
... .
...........
...
...
...........................................................................................
.... ....
1 .... ..... .
.....................................
......
... ... ... .... ..... ..
... ... ... .. ..
....... .
.. .
... ... ... ... .....
... ... ... ... ..... .
... ... ... ... ...
...... .
.
... 1 .
. ... ... .
...... .
.. .... .
... . ... ... .
..... .
...b−a ..
.
..
... ... ....
... .
.
... . ... ... ......
. .
... ..
. ... ... .... .
... ..
. ... ... ..
..... .
... ..
. ... ... ......
. .
.. ... .
... . ... ... .....
.. .
. .. . . . .
. . .
......................................................................................................................................................................................................
. . .......................................................................................................................................................................................................
. ..
. .
... ...
0 a b x 0 a b x
Figure(7.1a). Left: plot of density of uniform(a, b). Right: plot of distribution function of uniform(a, b).
(PICTEX)

7.2 Sum of two i.i.d. uniform random variables.


Example(7.2a). Suppose X ∼ uniform(0, a), Y ∼ uniform(0, a) and X and Y are independent. Find the distribution of
Z =X +Y.
Solution. Clearly Z ∈ (0, 2a). The usual convolution integral gives
1 a
Z Z
fZ (z) = fX (x)fY (z − x) dx = fY (z − x) dx
x a 0
where fY (z − x) = 1/a when 0 < z − x < a; i.e. when z − a < x < z. Hence

Bayesian Time Series Analysis by R.J. Reed Mar 10, 2020(20:25) §7 Page 19
Page 20 §7 Mar 10, 2020(20:25) Bayesian Time Series Analysis

min{a,z}  z

if 0 < z < a;
min{a, z} − max{0, z − a}  a2
Z
1
fZ (z) = dx = =
a2 max{0,z−a} a2 2a − z
if a < z < 2a.


a2

and the distribution function is


 2
z
if 0 < z < a;


 2
2a
FZ (z) = 2 2
2z z (2a − z)
− 2 −1=1− if a < z < 2a.



a 2a 2a2
A graph of the density and distribution function is shown in figure(7.2a). For obvious reasons, this is called the triangular
distribution.
fX (x) ................. FX (x) ............
.... ....
... ...
.. ..
.......... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...........................................................
...
...
.........
..... . .....
..... . ....
1 .... ........
....... .
.
... .... .. ......... ... ....... .
... .
.. ... ......
...
.
.... . .....
... ......
.
.
.
.... ..... .
.....
... ..... .
. ..... ... ....
... .
.
... ..... . ..... ... ..
. .
... ..
...... . .....
..... ... ...
.
.... .
... .
....
.
...
. .
. .....
.....
1/2 .. . . . . . . . . . . . . . . . . . .......... .
.
... ..
. . .... ..
...
. .
... .
....
.
...
. 1/a .
.
.....
..... ... .
.....
.... .
.
.
.
... ..
. . ..... ... .
...
.
.. . .
... .
....
. . .....
..... ... ..
..
. . .
... ......... . ..... ... ..
..
..... . .
.... ...... . ..... ... ...
......
. . .
........
.
.
. .....
. ..
. .
...
...
...... .
.
.
. .
........................................................................................................................................................................................................ ...............................................................................................................................................................................................................
. .
.... ....
0 a 2a x 0 a 2a x
Figure(7.2a). Plot of density (left) and distribution function (right) of triangular distribution.
(PICTEX)

7.3 Sum of n i.i.d. uniform random variables—the Irwin-Hall distribution. Now for the general result on
the sum of n independent and identically distributed uniforms.
Proposition(7.3a). Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the uniform(0, 1) distribution.
Let Sn = X1 + · · · + Xn . Then the density and distribution function of Sn are given by
n  
1 X k n
n
(t − k)+

Fn (t) = (−1) for all t ∈ R and all n = 1, 2, . . . . (7.3a)
n! k
k=0
n  
1 k n
X n−1
(t − k)+

fn (t) = (−1) for all t ∈ R and all n = 2, 3, . . . . (7.3b)
(n − 1)! k
k=0
Proof. We prove the result for Fn (t) by induction on n.
Now if n = 1, then the right hand side of equation(7.3a) gives [t+ − (t − 1)+ ] which equals
0 if t < 0;
(
t if 0 < t < 1;
1 if t > 1.
as required. Also, for t ∈ R and n = 2, 3, . . . , we have
Z 1 Z 1 Z t
 
fn (t) = fn−1 (t − x)f1 (x) dx = fn−1 (t − x) dx = fn−1 (y) dy = Fn−1 (t) − Fn−1 (t − 1)
0 0 y=t−1
Assume that equation(7.3a) is true for n; to prove it true for n + 1:
fn+1 (t) = [Fn (t) − Fn (t − 1)]
" n   n   #
1 X n n X n n
(−1)k (t − k)+ − (−1)k (t − k − 1)+
   
=
n! k k
k=0 k=0
" n   n+1   #
1 k n + n n + n
X X
`
   
= (−1) (t − k) + (−1) (t − `)
n! k `−1
k=0 `=1
n+1  
1 X n+1  n
= (−1)k (t − k)+
n! k
k=0

by using the combinatorial identity nk + k−1 n


= n+1
  
k . Integrating fn+1 (t) gives
Z t n+1  
1 k n+1
X n+1
(t − k)+

Fn+1 (t) = fn+1 (x) dx = (−1)
0 (n + 1)! k
k=0
This establishes the proposition.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §7 Page 21

Corollary(7.3b). Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the uniform(0, 1) distribution. Let
Sn = X1 + · · · + Xn . Then the density of Sn is given by
[t]  
1 X n
fn (t) = (−1)k (t − k)n−1 for all t ∈ R and all n = 2, 3, . . . . (7.3c)
(n − 1)! k
k=0
Proof. Note that [t] is the floor function which is defined by
[t] = j if j ∈ {1, 2, 3, . . .} and j ≤ t < j + 1.
Equation(7.3c) follows immediately from equation(7.3b).
The random variable X is said to have the Irwin-Hall distribution with order n iff X has the density given in
equation(7.3b).
The proposition easily extends to other uniforms. For example. Suppose X1 , X2 , . . . , Xn are i.i.d. random
variables with the uniform(0, a) distribution and Sn = X1 + · · · + Xn . Then the proposition can be applied to the
sum Sn0 = Y1 + · · · + Yn , where Yj = Xj/a. Hence
Fn (t) = P[X1 + · · · + Xn ≤ t] = P[Sn0 ≤ t/a]
n  
1 X k n
n
(t − ka)+

= n (−1) for all t ∈ R and all n = 1, 2, . . . .
a n! k
k=0
n  
1 X n  n−1
fn (t) = n (−1)k (t − ka)+ for all t ∈ R and all n = 2, 3, . . . .
a (n − 1)! k
k=0
Similarly if the random variables come from the uniform(a, b) distribution.
An alternative proof of proposition(7.3a) is based on taking the weak limit of the equivalent result for discrete
uniforms; the proof below is essentially that given on pages 284–285 in [F ELLER(1968)].
Proof. (An alternative proof of proposition(7.3a).) For m = 1, 2, . . . and n = 2, 3, . . . , suppose Xm1 , Xm2 , . . . , Xmn
are i.i.d. random variables with the discrete uniform distribution on the points 0, 1/m, 2/m, . . . , m/m = 1.
0 0 0 0
Let Xmj = mXmj + 1. Then Xm1 , Xm2 , . . . , Xmn are i.i.d. random variables with the discrete uniform distribution on
the points 1, 2, . . . , m + 1.
Now the discrete uniform distribution on the points 1, 2, . . . , m + 1 has probability generating function
s(1 − sm+1 )
for |s| < 1.
(m + 1)(1 − s)
0 0
Hence the probability generating function of Xm1 + · · · + Xmn is
0 0 sn (1 − sm+1 )n
E[sXm1 +···+Xmn ] = for |s| < 1.
(m + 1)n (1 − s)n
0 0
Hence the probability generating function of the sequence P[Xm1 + · · · + Xmn ≤ j] is
n m+1 n
s (1 − s )
for |s| < 1. (7.3d)
(m + 1)n (1 − s)n+1
Also, for j ∈ {0, 1/m, 2/m, . . . , mn−1/m, mn/m = n} we have
0 0
P[Xm1 + · · · + Xmn ≤ j] = P[Xm1 + · · · + Xmn ≤ mj + n]
mj+n
and this is the coefficient of s in the expansion of equation(7.3d), which in turn is
n  
( )
m+1 n m`+`
1 (1 − s ) X n ` s
× coefficient of smj in the expansion of = (−1)
(m + 1)n (1 − s)n+1 ` (1 − s)n+1
`=0
which is
n  
( )
1 0
X n sm`−mj+`
`
× coefficient of s in the expansion of (−1)
(m + 1)n ` (1 − s)n+1
`=0
This is clearly 0 if l > j. Otherwise it is
n   n
` mj − m` − ` + n (mj − m` − ` + n)!
   
1 X n 1 X ` n
(−1) = (−1) (7.3e)
(m + 1)n ` n n! ` (m + 1)n (mj − m` − `)!
`=0 `=0
P∞
by using the binomial series 1/(1−z)n+1 = `=0 `+n
 `
n z for |z| < 1. Taking the limit as m → ∞ of the expression in (7.3e)
gives
n  
1 X ` n
n
(j − `)+

(−1)
n! `
`=0
This proves the result when j is an integer. If j is any rational, take a series m tending to ∞ with mj an integer. Hence
the result in equation(7.3a) for any t by right continuity.
Page 22 §7 Mar 10, 2020(20:25) Bayesian Time Series Analysis

A note on the combinatorialP  by equation(7.3a) on page 20. Now Fn (t) = P[Sn ≤ t] = 1 for t ≥ n. By
identity implied
n
equation(7.3a), this implies k=0 (−1)k nk (t − k)n = n! for t ≥ n. How do we prove this identity without probability?
For all t ∈ R we have the identity
n  
X n
(−1)k etk = (1 − et )n
k
k=0
Pn n
Setting t = 0 gives k=0 k (−1)k = 0. Differentiating the identity once and setting t = 0 gives
n  
X n
(−1)k k = 0
k
k=0
Similarly, differentiating r times and setting t = 0 gives
n   
X n 0 if r = 0, 1, 2. . . . , n − 1;
(−1)k k r = (7.3f )
k (−1)n n! if r = n.
k=0
and hence
n  
X n n
0 if r = 1, 2. . . . , n − 1;
(−1)n−k k r =
k n! if r = n.
k=0
For all t ∈ R
n   n X n    
X n X n n j
(−1)k (t − k)n = (−1)k t (−1)n−j k n−j
k k j
k=0 k=0 j=0
n   n  
X n j j
X n
= (−1) t (−1)n−k k n−j = n!
j k
j=0 k=0
This generalizes the combinatorial result implied by equation(7.3a). See also question 16 on page 65 in [F ELLER(1968)].
7.4 Representing the uniform distribution as the sum of independent Bernoulli random variables. Now
every y ∈ [0, 1) can be represented as a ‘binary decimal’. This means we can write
y = 0.x1 x2 x3 . . . where each xj is either 0 or 1.
This representation motivates the following result:
Proposition(7.4a). Suppose X1 , X2 , . . . are i.i.d. random variables with the Bernoulli distribution

1 with probability 1/2;
0 with probability 1/2.
Then the random variable

X Xk
V =
2k
k=1
has the uniform uniform(0, 1) distribution.
P n Xk Xk
Proof. Let Sn = k=1 2k for n = 2, 3, . . . . Now the moment generating function of 2k
is
k 1 1 k
E[etXk /2 ] = + et/2
2 2
Hence
n
1 Y h t/2k i
E[etSn ] = e + 1
2n
k=1
  
t/2n+1 t/2n+1 t/2n
Using the identity e −1 e +1 =e − 1, and induction, it is possible to show
n
1 Y t/2k
h i 1 et − 1
E[etSn ] = n e + 1 = n t/2n (7.4a)
2
k=1
2 e −1
and hence
1 et − 1 et − 1
E[etSn ] =
n → as n → ∞.
2n et/2 − 1 t
t
Because (e − 1)/t is the moment generating function of uniform(0, 1), we see that V ∼ uniform(0, 1) as required.
Equation(7.4a) proves the following representation. Suppose Vn ∼ uniform(0, 1/2n ), and X1 , X2 , . . . , Xn , Vn are
all independent; then
n
X Xk
Vn + ∼ uniform(0, 1) for all n ∈ {1, 2, . . .}.
2k
k=1
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §7 Page 23

7.5 Order statistics for the uniform distribution. Suppose X1 , . . . , Xn are i.i.d. with the uniform(0, 1) dis-
tribution.
Maximum and minimum. It is straightforward to check the following results:
n
P[Xn:n ≤ x] = xn for 0 ≤ x ≤ 1, and E[Xn:n ] =
n+1
1
P[X1:n ≤ x] = 1 − (1 − x)n for 0 ≤ x ≤ 1, and E[X1:n ] =
n+1
Note that E[X1:n ] = 1 − E[Xn:n ] → 0 as n → ∞.
x n
→ e−x as n → ∞. This implies

For 0 ≤ x < 1 we have P[nX1:n > x] = P[X1:n > x/n] = 1 − n
D D
nX1:n =⇒ exponential (1) as n → ∞ where =⇒ denotes weak convergence (also called convergence in
distribution). Now for Xk:n .
The distribution of Xk:n . By equations(3.3a) and (3.3b) we have
n  
X n j
P[Xk:n ≤ t] = t (1 − t)n−k
j
j=k
n − 1 k−1
 
fXk:n (t) = n t (1 − t)n−k for 0 ≤ t ≤ 1. (7.5a)
k−1
This is the Beta density beta(k, n − k + 1) which is considered later—see §13.1 on page 40.
The limiting distribution of Xk:n as n → ∞.
P[nXk:n > t] = 1 − P[Xk:n ≤ 1/n]
n    j 
t n−j

X n t
=1− 1−
j n n
j=k

t n t n−1
  k−1 
t n−k
       
n t n t
= 1− + 1− + ··· + 1−
n 1 n n k−1 n n
k−1
t e −t
→ e−t + te−t + · · · +
k!
which is equal to P[Y > t] where Y ∼ gamma(k, 1) distribution—this will be shown in §11.2 on page 34.
D
We have shown that nXk:n =⇒ gamma(k, 1) as n → ∞ for any fixed k ∈ {1, 2, . . . , n}.
n
Note also that for x < 0, we have P[n(Xn:n − 1) < x] = P[Xn:n < 1 + x/n] = 1 + nx → ex as n → ∞ and
P[X1:n < x/n] = 1 − (1 − x/n)n → 1 − e−x as n → ∞.
7.6 The probability integral transform. Every probability distribution function F is monotonic increasing;
if F is a strictly increasing probability distribution function on the whole of R, then we know from elementary
analysis that F has a unique inverse G = F −1 . If U ∼ uniform(0, 1) and Y = G(U ), then clearly Y has distribution
function F . Hence we can simulate variates from the distribution F by simulating variates x1 , x2 , . . . xn from the
uniform(0, 1) distribution and calculating G(x1 ), G(x2 ), . . . , G(xn ).
Now for the general case. Suppose F : R → [0, 1] is a distribution function (not necessarily continuous). We first
need to define the “inverse” of F .
Proposition(7.6a). Suppose F : R → [0, 1] is a distribution function and we let G(u) = min{x: F (x) ≥ u}
for u ∈ (0, 1). Then
{x: G(u) ≤ x} = {x: F (x) ≥ u}.
Proof. Fix u ∈ (0, 1); then
x0 ∈ R.H.S. ⇒ F (x0 ) ≥ u
⇒ G(u) ≤ x0 by definition of G
⇒ x0 ∈ L.H.S.
Conversely
x0 ∈ L.H.S. ⇒ G(u) ≤ x0
⇒ min{x: F (x) ≥ u} ≤ x0
Let x = min{x: F (x) ≥ u}. Hence x ≤ x0 . Choose a sequence {xn }n≥1 with xn ↓↓ x∗ as n → ∞ (this means
∗ ∗

that the sequence {xn }n≥1 strictly decreases with limit x∗ ). Hence F (xn ) ≥ u for all n = 1, 2, . . . . Now F is a
Page 24 §7 Mar 10, 2020(20:25) Bayesian Time Series Analysis

distribution function; hence F is right continuous; hence F (x∗ ) ≥ u. Also x0 ≥ x∗ and F is monotonic increasing; hence
F (x0 ) ≥ F (x∗ ) ≥ u. Hence x0 ∈ R.H.S.
Suppose the distribution function F is continuous at α ∈ R. Then G(β) = α implies F (α) ≥ β. Also, for every
x < α we have F (x) < β. Hence F (α) = β. We have shown that G(β) = α implies F (α) = β and hence
F G(β) = β. If the random variable X has the distribution function F and F is continuous, then P[F (X) ≥ u] =
P[G(u) ≤ X] = 1 − F G(u) = 1 − u.
We have shown the following two important results.
• If the random variable X has the distribution function F and F is continuous, then the random variable F (X)
has the uniform(0, 1) distribution.
• Suppose F is a distribution function and G is defined in terms of F as explained above. If U has a uniform
distribution on (0, 1) and X = G(U ) then
P[X ≤ x] = P [G(U ) ≤ x] = P [F (x) ≥ U ] = F (x).
Hence
If U ∼ uniform(0, 1), then the distribution function of G(U ) is F .

As explained before the proposition, if the distribution function F is strictly increasing on the whole of R then
F −1 , the inverse of F , exists and G = F −1 . If F is the distribution function of a discrete distribution, then F is
constant except for countably many jumps and the inverse of F does not exist. However, G(u) is still defined by
the proposition and this method of simulating from the distribution F still works.
7.7 Using the probability integral transformation to prove results about order statistics.
Suppose X1 , . . . , Xn are i.i.d. with the uniform(0, 1) distribution. By equation(7.5a) on page 23 we have
n!
fXk:n (x) = xk−1 (1 − x)n−k for 0 ≤ x ≤ 1.
(k − 1)!(n − k)!
Suppose Y1 , . . . , Yn are i.i.d. with an absolutely continuous distribution with distribution function FY and density
function fY and we wish to find the distribution of Yk:n .
Then X1 = FY (Y1 ), . . . , Xn = FY (Yn ) are i.i.d. with the uniform(0, 1) distribution and hence
n!
fXk:n (x) = xk−1 (1 − x)n−k
(k − 1)!(n − k)!
Now FY is monotonic increasing and continuous and the transformation (X1 , . . . , Xn ) 7→ (Y1 , . . . , Yn ) is order
preserving; hence
Z FY (y)
P[Yk:n ≤ y] = P[FY (Yk:n ) ≤ FY (y)] = P[Xk:n ≤ FY (y)] = fXk:n (x) dx
−∞
and hence
n!
fYk:n (y) = {FY (y)}k−1 {1 − FY (y)}n−k fY (y)
(k − 1)!(n − k)!
This approach provides a general method for proving results about the order statistics of a sample from a contin-
uous distribution function.
7.8 Random partitions of an interval. Suppose U1 , . . . , Un are i.i.d. random variables with the uniform
uniform(0, 1) distribution and let U1:n , . . . , Un:n denote the order statistics. These variables partition the interval
[0, 1] into n + 1 disjoint intervals with the following lengths:
D1 = U1:n , D2 = U2:n − U1:n , . . . , Dn = Un:n − U(n−1):n , Dn+1 = 1 − Un:n
Clearly D1 + · · · + Dn+1 = 1. The absolute value of the Jacobian of the transformation (U1:n , . . . , Un:n ) 7→
(D1 , . . . , Dn ) is
∂(d1 , . . . , dn )
∂(u1:n , . . . , un:n ) = 1

The density of (U1:n , . . . , Un:n ) is given by (3.2a) on page 8. Hence the density of (D1 , . . . , Dn ) is
Pn
f(D1 ,...,Dn ) (d1 , . . . , dn ) = n! for d1 ≥ 0,. . . , dn ≥ 0, `=1 d` ≤ 1. (7.8a)
There are many results on random partitions of an interval—see [F ELLER(1971)], [DAVID &BARTON(1962)] and
[W HITWORTH(1901)]. One result is given in exercise 14.9 on page 44.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §8 Page 25

7.9
Summary. The uniform distribution.
• Density. Suppose a < b. Then X has the uniform uniform(a, b) density iff
1
fX (x) = for x ∈ (a, b).
b−a
• The distribution function.
x−a
F (x) = for x ∈ (a, b).
b−a
• Moments.
a+b (b − a)2 bn+1 − an+1
E[X] = var[X] = E[X n ] = for n 6= −1, n ∈ R.
2 12 (n + 1)(b − a)
• M.g.f. and c.f.
(b−a)t
 tb
 e − eta  2 sin( 2 )ei(a+b)t/2

tX for t 6= 0; itX for t 6= 0;
MX (t) = E[e ] = t(b − a) and φX (t) = E[e ] = t(b − a)
1 for t = 0. for t = 0.
 
1
• Properties.
The sum of two independent uniforms on (0, a) is the triangular distribution on (0, 2a).
If X has the distribution function F and F is continuous, then F (X) ∼ uniform(0, 1).

8 Exercises (exs-uniform.tex)

8.1 Suppose X ∼ uniform(a, b). Show that


n
(b − a) n

1 − (−1) n+1  (b − a)
 
n if n is even;
E[(X − µ) ] = = (n + 1)2n [Ans]
(n + 1)2n+1 
0 if n is odd.
8.2 Suppose X and Y are independent random variables such that X has density fX (x) = 6x(1 − x) for 0 < x < 1 and Y
has density fY (y) = 2y for 0 < y < 1. Find the density of Z = X 2 Y . [Ans]
8.3 Transforming uniform to exponential. Suppose X ∼ uniform(0, 1). Find the distribution of Y = − ln X. [Ans]
8.4 Product of independent uniforms.
(a) Suppose X and Y are i.i.d. with the uniform(0, 1) distribution. Let Z = XY . Find the density and distribution
function of Z. (An alternative expression of this problem is given in exercise 8.5.)
(b) (Note: this part makes use of the fact that the sum of independent exponentials is a gamma distribution—see propo-
sition(11.6a) on page 35.) Suppose X1 , . . . , Xn are i.i.d. with the uniform(0, 1) distribution and let Pn = X1 · · · Xn .
Find the density of − ln Pn and hence find the density of Pn . [Ans]
8.5 Suppose X1 ∼ uniform(0, 1), X2 ∼ uniform(0, X1 ), X3 ∼ uniform(0, X2 ), and in general, Xn ∼ uniform(0, Xn−1 )
for n ∈ {2, 3, . . .}.
(a) Prove by induction that the density of Xn is
n−1
ln 1/xn
f (x) = for 0 < xn < 1.
(n − 1)!
(b) By using the result of part (b) exercise 8.4, find the density of Xn . [Ans]
8.6 Suppose X is a random variable with an absolutely continuous distribution with density f . Then entropy of X is defined
to be Z
H(X) = − f (x) ln f (x) dx
Suppose X ∼ uniform(a, b). Find the entropy of X.
(It can be shown that the continuous distribution on the interval (a, b) with the largest entropy is the uniform.) [Ans]
8.7 Sum and difference of two independent uniforms. Suppose X ∼ uniform(0, a) and Y ∼ uniform(0, b) and X and Y are
independent.
(a) Find the density of V = X + Y and sketch its shape.
(b) Find the density of W = Y − X and sketch its shape. [Ans]
8.8 Suppose X ∼ uniform(0, a) and Y ∼ uniform(0, b) and X and Y are independent. Find the distribution of V =
min{X, Y } and find P[V = X]. [Ans]
Page 26 §8 Mar 10, 2020(20:25) Bayesian Time Series Analysis

8.9 A waiting time problem. Suppose you arrive at a bus stop at time t = 0. The stop is served by two bus routes. From past
observations, you assess that the time X1 to wait for a bus on route 1 has the uniform(0, a) distribution and the time X2
to wait for a bus on route 2 has the uniform(0, b) distribution. Also X1 and X2 are independent. (Clearly this assumption
will not hold in practice!!) A bus on route 1 takes the time α to reach your destination whilst a bus on route 2 takes the
time α + β.
Suppose the first bus arrive at the stop at time t0 and is on route 2. Should you catch it if you wish to minimize your
expected arrival time? [Ans]
8.10 Suppose U ∼ uniform(0, 1) and the random variable V has an absolutely continuous distribution with finite expectation
and density f . Also U and V are independent. Let W denote the fractional part of U + V ; this means that W =
U + V − bU + V c. Show that W ∼ uniform(0, 1).
(See also Poincaré’s roulette problem; pages 62–63 in [F ELLER(1971)] for example.) [Ans]
8.11 Suppose X and Y are i.i.d. random variables with the uniform(0, 1) distribution. Let V = min{X, Y } and W =
max{X, Y }.
(a) Find the distribution functions, densities and expectations of V and W .
(b) Find P[V ≤ v, W ≤ w] and hence derive f(V,W ) (v, w), the joint density of (V, W ).
(c) Find the density of (W |V ≤ v) and hence derive E[W |V ≤ v]. [Ans]
8.12 Suppose two points are chosen independently and at random on a circle with a circumference which has unit length.
(a) Find the distribution of the lengths of the intervals (X1 , X2 ) and (X2 , X1 ).
(b) Find the distribution of the length of the interval L which contains the fixed point Q.
(See page 23 in [F ELLER(1971)].) [Ans]
8.13 Suppose n points are distributed independently and uniformly on a disc with radius r. Let D denote the distance from
the centre of the disc to the nearest point. Find the density and expectation of D. [Ans]
8.14 Suppose the random vector (X1 , X2 ) has a distribution which is uniform over the disc {(x, y) ∈ R2 : x2 + y 2 ≤ a2 }.
Find the density of X1 . (See also the semicircle distribution defined in equation (34.1b) on page 103.) [Ans]
8.15 Suppose V1 , V2 and V3 are i.i.d. random variables with the uniform(0, 1) distribution.
(a) Let R = max{V1 , V2 } and Θ = 2πV3 . Find the density of (R, Θ).
(b) Let X = R cos Θ and Y = R sin Θ. Show that the random vector (X, Y ) is uniformly distributed on the unit disc
{(x, y) ∈ R2 : 0 < x2 + y 2 < 1}. [Ans]
8.16 Suppose X1 , . . . , Xn are i.i.d. with the uniform(0, 1) distribution.
(a) Find E[Xk:n ] and var[Xk:n ] for k = 1, 2, . . . , n.
(b) Find the joint density of (Xj:n , Xk:n ).
(c) Find E[Xj:n Xk:n ], cov[Xj:n , Xk:n ] and corr[Xj:n , Xk:n ]. [Ans]
8.17 Suppose Y1 , . . . , Yn are i.i.d. with an absolutely continuous distribution with distribution function FY and density
function fY . By transforming to the uniform distribution, find the density of (Yi:n , Yj:n ) where 1 ≤ i < j ≤ n. [Ans]
8.18 Suppose X ∼ uniform(a, b).
(a) Find skew[X], the skewness of X. (b) Find κ[X], the kurtosis of X.
Note: skew[X] = E[(X − µ) ]/σ and κ[X] = E[(X − µ)4 ]/σ 4 where σ 2 = E[(X − µ)2 ]. See §1.4 and §1.5.
3 3
[Ans]
8.19 Tukey’s lambda distribution. Suppose a > 0 and X ∼ uniform(0, 1). Define the random variable Y by
aX λ − (1 − X)λ Xa
 
Y = if λ 6= 0 and Y = ln if λ = 0.
λ 1−X
Then Y has the Tukey lambda (a, λ) distribution. Note that if λ > 0, then − 1/λ ≤ Y ≤ a/λ.
Suppose λ > 0; find E[Y n ] for n ∈ {1, 2, 3, . . .}.
(Note there is no simple closed form for the density and distribution functions.) [Ans]
8.20 The triangular density. Suppose a, b and c are real numbers with a < c < b. Define the function f by
 2(x−a)
 (b−a)(c−a) if a ≤ x ≤ c
f (x) = 2(b−x)
 (b−a)(b−c) if c ≤ x ≤ b
0 otherwise
(a) Check that f is a density function and plot its shape.
(b) Suppose the random variable X has the density f . Find E[X] and var[X].

[Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §9 Page 27

The Irwin-Hall distribution.


8.21 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the uniform(0, 1) distribution. Let Sn = X1 + · · · + Xn .
(a) Show that the density of Sn is given by
n  
1 k n
X
gn (t) = (−1) sgn(t − k)(t − k)n−1 for all t ∈ R and all n = 2, 3, . . . . (8.21a)
2(n − 1)! k
k=0
where
−1 if k > t;
(
sgn(t − k) = 0 if k = t;
1 if k < t.
(b) Show that the distribution function of Sn is given by
[t]  
1 X k n
Fn (t) = (−1) (t − k)n for all t ∈ R and all n = 1, 2, . . . . (8.21b)
n! k
k=0
and
n
"   #
1 1 X n
Fn (t) = 1+ (−1)k sgn(t − k)(t − k)n for all t ∈ R and all n = 1, 2, . . . . (8.21c)
2 n! k
k=0
[Ans]
8.22 Suppose Xn has the Irwin-Hall distribution with order n.
(a) Show that E[Xn ] = n/2 and var[Xn ] = n/12 and the m.g.f. of Xn is
 t n
tXn e −1
E[e ] = for all t ∈ R with t 6= 0.
t
(b) Show that
Xn − n/2 D
p =⇒ N (0, 1) as n → ∞. [Ans]
n/12
8.23 Suppose X has the Irwin-Hall distribution with order n.
(a) Show that E[X 2 ] = (3n2 + n)/12, E[X 3 ] = (n3 − n2 + 2n)/8 and E[X 4 ] = (n/240)[15n3 + 30n2 + 5n − 2].
(b) Show that the skewness of X is skew[X] = 0.
(c) Show that the kurtosis of X is κ[X] = 3 − 6/5n. [Ans]
8.24 Suppose a < b and X1 , X2 , . . . , Xn are i.i.d. random variables with the uniform(a, b) distribution. Let Sn = X1 + · · · +
Xn . Find expressions for the distribution function and density function of Sn . [Ans]

9 The exponential distribution


9.1 The basics
Definition(9.1a). Suppose λ > 0. Then X has the exponential distribution, exponential (λ), iff X has an
absolutely continuous distribution with density

λe−λx if x > 0;
f (x) =
0 if x < 0.
Clearly the density f of the exponential distribution is decreasing on [0, ∞) and is convex on [0, ∞) because
f 00 > 0. Also f (x) → 0 as x → ∞.
The distribution function. 
F (x) = 1 − e−λx if x > 0;
0 if x < 0.
The quantile function and median. The quantile function is
ln(1 − p)
F −1 (p) = − for p ∈ [0, 1).
λ
The median is F −1 ( 1/2) = ln 2/λ.
Moments. These can be obtained by integrating by parts.
1 2 n!
E[X] = E[X 2 ] = 2 E[X n ] = n for n = 1, 2, . . . . (9.1a)
λ λ λ
Page 28 §9 Mar 10, 2020(20:25) Bayesian Time Series Analysis

More generally, E[X n ] = Γ(n + 1)/λn for all n ∈ (0, ∞). Hence
n
1 n! X (−1)k
var[X] = 2 and E[ (X − 1/λ)n ] = n for n = 1, 2, . . . .
λ λ k!
k=0

The moment generating function and characteristic function.


λ λ
E[etX ] = and φ(t) = E[eitX ] =
λ−t λ − it
Multiple of an exponential distribution. Suppose X ∼ exponential (λ) and Y = αX where α > 0. Then
P[Y > t] = P[X > t/α] = e−λt/α and hence Y ∼ exponential ( λ/α).
It follows that the family of exponential distributions {exponential (λ) : λ > 0} is a scale family of distributions—
see definition(1.6d) on page 5 for the meaning of this term.
Sum of i.i.d. exponentials. The sum of i.i.d. random variables with an exponential distribution has a gamma
distribution—this is explained in proposition(11.6a) on page 35.
9.2 The exponential as the limit of geometric distributions.
Suppose events can only occur at times δ, 2δ, 3δ, . . . , and events at different times are independent. Let
P[event occurs at time kδ] = p
Let T denote the time to the first event. Then
P[T > kδ] = (1 − p)k
Hence P[T = kδ] = (1 − p)k−1 p for k ∈ {1, 2, . . .} and E[T ] = δ/p.
Now suppose δn → 0 as n → ∞ in such a way that E[Tn ] = δn/pn = 1/α is constant. Then
lim P[Tn > t] = lim (1 − pn )t/δn = lim (1 − αδn )t/δn = e−αt (9.2a)
n→∞ n→∞ n→∞
D
and hence Tn =⇒ T as n → ∞ where T ∼ exponential (α).
This result can be rigorously stated as follows.
Proposition(9.2a). Suppose that for every p ∈ (0, 1), the random variable Xp has the geometric distribution
P[Xp = k] = (1 − p)k−1 p for k = 1, 2, . . .
Then
D
pXp =⇒ exponential (1) as p → 0.
Proof. The m.g.f. of Xp is

X pet
E[etXp ] = etk (1 − p)k−1 p = for t < − ln(1 − p).
1 − (1 − p)et
k=1
It follows that the m.g.f. of pXp is

X petp
E[etpXp ] = etpk (1 − p)k−1 p = for t < − ln(1 − p)/p.
1 − (1 − p)etp
k=1
Setting x = pt in the standard inequality x < ex − 1 < x/(1 − x) for x < 1 shows that
etp − 1 t etp − 1
t< < for p < 1/t, and hence lim =t
p 1 − tp p→0 p
Hence for every t ∈ (0, ∞) we have
1 − (1 − p)etp 1 etp − 1
= 1 − −→ 1 − t as p → 0.
petp etp p
We have shown that
1
lim E[etpXp ] =
p→0 1−t
and the limit on the right hand side is the m.g.f. of the exponential (1) distribution. Hence, by the result1 on page 390 of
D
[B ILLINGSLEY(1995)], we have pXp =⇒ exponential (1) as p → 0.

1
Suppose s0 > 0 and {Mn } is a sequence of moment generating functions which exist for t ∈ (−s0 , s0 ) and Mn (t) → M (t)
as n → ∞ for all t ∈ (−s0 , s0 ) where M is also a moment generating function. Then this implies convergence in distribution
of the corresponding distributions.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §9 Page 29

Corollary(9.2b). Suppose λ ∈ (0, ∞) and {pn }n≥1 is a sequence in (0, 1) such that npn → λ as n → ∞.
Suppose further that for every n ∈ {1, 2, . . .} the random variable Xn has the geometric distribution P[Xn =
k] = (1 − pn )k pn for k = 1, 2, . . . . Then
Xn D
=⇒ exponential (λ) as n → ∞.
n
Proof. The assumption npn → λ as n → ∞ implies pn → 0 as n → ∞. Proceeding as in the proposition gives
D
pn Xn =⇒ exponential (1) as n → ∞. Let λn = npn ; hence λn → λ as n → ∞. The Convergence of Types theorem2
then implies
pn Xn D
=⇒ exponential (λ) as n → ∞.
λn
Here is another approach to defining points distributed randomly on [0, ∞). Suppose X1 , . . . , Xn are i.i.d. random
variables with the uniform(0, `n ) distribution. Then P[X(1) > t] = (1 − t/`n )n for t ∈ (0, `n ). Now suppose
n → ∞ in such a way that n/`n = λ is fixed. Then limn→∞ P[X(1) > t] = e−λt , which means that
D
X(1) =⇒ T as n → ∞
where T ∼ exponential (α). Informally, this result says that if points are distributed randomly on the line such that
the mean density of points is λ, then the distance to the first point has the exponential (λ) distribution.
9.3 The lack of memory or Markov property of the exponential distribution. Suppose the random variable T
models the lifetime of some component. Then the random variable T is said to have the lack of memory property
iff the remaining lifetime of an item which has already lasted for a length of time x has the same distribution as T .
This means
P[T > x + t|T > x] = P[T > t]
and hence
P[T > x + t] = P[T > t] P[T > x] for all t > 0 and x > 0.
Similarly, the distribution with distribution function F has the lack of memory property iff [1 − F (x + t)] =
[1 − F (x)][1 − F (t)] for all x > 0 and all t > 0.
If X ∼ exponential (λ) then 1 − F (x) = e−λx and hence X has the lack of memory property. Conversely
Proposition(9.3a). Suppose X is an absolutely continuous random variable on [0, ∞) with the lack of memory
property. Then there exists λ > 0 such that X ∼ exponential (λ).
Proof. Let G(x) = P[X > x]. Then
G(x + y) = G(x)G(y) for all x ≥ 0 and all y ≥ 0.
 m  n  mn
Suppose x = m/n is rational. Then G(m/n) = G(1/n) . Raising to the nth power gives G(m/n) = G(1/n) =
[G(1)]m . Hence G(m/n) = [G(1)]m/n .
Now suppose x is any real number in [0, ∞). Choose sequences qn and rn of rationals such that qn ≤ x ≤ rn and
qn → x as n → ∞ and rn → x as n → ∞. Hence G(1)qn = G(qn ) ≥ G(x) ≥ G(rn ) = G(1)rn . Letting n → ∞ gives
G(x) = G(1)x .
Now let λ = − ln [G(1)]. Then G(x) = G(1)x = e−λx . See also [F ELLER(1968)], page 459.
The proof of proposition(9.3a) depends on finding a solution of the functional equation f (x + y) = f (x)f (y).
Taking logs of this equation gives the Cauchy functional equation f (x + y) = f (x) + f (y). Both of these equations
have been studied very extensively—see [S AATY(1981)], [ACZ ÉL(1966)], [K UCZMA(2009)], etc.
9.4 Distribution of the minimum and maximum. Suppose Xj ∼ exponential (λj ) for j = 1, 2, . . . , n. Suppose
further that X1 , X2 , . . . , Xn are independent. Let X1:n = min{X1 , . . . , Xn }. Then
P[X1:n > t] = P[X1 > t] · · · P[Xn > t] = e−(λ1 +···+λn )t for t > 0.
Hence X1:n ∼ exponential (λ1 + · · · + λn ).
In particular, we have shown that if X1 , . . . , Xn are i.i.d. random variables with the exponential (λ) distribution,
then X1:n ∼ exponential (nλ). Hence
X1
X1:n has the same distribution as
n
This property characterizes the exponential distribution—see page 39 of [G ALAMBOS &KOTZ(1978)].
The maximum is Xn:n and clearly P[Xn:n ≤ t] = nk=1 [1 − e−λk t ] for t ∈ [0, ∞). See also exercise 10.15.
Q

2
See Lemma 1 on page 193 of [B ILLINGSLEY(1995)]
Page 30 §9 Mar 10, 2020(20:25) Bayesian Time Series Analysis

9.5 The order statistics of the exponential distribution. Suppose we think of X1 , . . . , Xn as the times when
n events occur. Then we have shown that the time to the first event has the exponential (nλ) distribution. Using
the lack of memory property suggests that the extra time to the second event, X2:n − X1:n , should have the
exponential ( (n − 1)λ ) distribution. And so on. This result is established in the following proposition.
Proposition(9.5a). Suppose X1 , . . . , Xn are i.i.d. random variables with the exponential (λ) distribution. De-
fine Z1 , . . . , Zn by
Z1 = nX1:n , Z2 = (n − 1)(X2:n − X1:n ), . . . , Zn = Xn:n − X(n−1):n
Then Z1 , . . . ,Zn are i.i.d. random variables with the exponential (λ) distribution.
P n
−λ xj
Proof. We know that the density of (X1:n , . . . , Xn:n ) is g(x, . . . , xn ) = n!λn e j=1 for 0 < x1 < · · · < xn .
Also
Z1 Z1 Z2 Z1 Z2 Zn−1
X1:n = X2:n = + . . . , Xn:n = + + ··· + + Zn
n n n−1 n n−1 2
and hence the Jacobian of the transformation is
∂(x1:n , . . . , xn:n ) 1
=
∂(z1 , . . . , zn ) n!
Hence the density of (Z1 , . . . , Zn ) is
1
f(Z1 ,...,Zn ) (z1 , . . . , zn ) = n!λn e−λ(z1 +···+zn ) for z1 > 0, . . . , zn > 0.
n!
This establishes the proposition.
9.6 Link with the the order statistics from a uniform distribution.
Proposition(9.6a). Suppose Y1 , . . . , Yn+1 are i.i.d. random variables with the exponential (λ) distribution, and
let
S` = Y1 + · · · + Y` for ` = 1, . . . , n + 1.
Then  
Y1 Yn
,..., is independent of Sn+1 . (9.6a)
Sn+1 Sn+1
Suppose U1 , . . . , Un are i.i.d. random variables with the uniform(0, 1) distribution and denote the vector of
order statistics by (U1:n , . . . , Un:n ). Let
D1 = U1:n , D2 = U2:n − U1:n , . . . , Dn = Un:n − U(n−1):n , Dn+1 = 1 − Un:n
Then  
Y1 Yn+1
,..., has the same distribution as (D1 , . . . , Dn+1 ) (9.6b)
Sn+1 Sn+1
Proof. By equation(3.2a), the density of the vector (U1:n , . . . , Un:n ) is
n! if 0 < x1 < · · · < xn < 1
n
g(x1 , . . . , xn ) =
0 otherwise
Also
f(Y1 ,...,Yn+1 ) (y1 , . . . , yn+1 ) = λn+1 e−λ(y1 +···+yn+1 ) for y1 > 0,. . . , yn+1 > 0.
Consider the transformation:
Y1 Y2 Yn
X1 = , X2 = , . . . , Xn = , Xn+1 = Y1 + · · · + Yn+1
Sn+1 Sn+1 Sn+1
Or
Y1 = X1 Xn+1 , Y2 = X2 Xn+1 , . . . , Yn = Xn Xn+1 , Yn+1 = (1 − X1 − X2 − · · · − Xn )Xn+1
The absolute value of the Jacobian of the transformation is:
∂(y1 , . . . , yn+1 ) n
∂(x1 , . . . , xn+1 ) = xn+1

The determinant can be easily evaluated by replacing the last row by the sum of all the rows—this gives an upper triangular
determinant.
Hence the density of (X1 , . . . , Xn+1 ) is
 Pn+1
n+1 −λxn+1 n
f(X1 ,...,Xn+1 ) (x1 , . . . , xn+1 ) = λ e xn+1 for x1 ≥ 0, . . . , xn+1 ≥ 0, `=1 x` = 1
0 otherwise
Now Xn+1 = Y1 + · · · + Yn+1 is the sum of (n + 1) i.i.d. exponentials; hence by proposition(11.6a) on page 35, Xn+1 ∼
gamma(n + 1, λ) and has density
λn+1 xnn+1 e−λxn+1
fXn+1 (xn+1 ) =
n!
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §10 Page 31

It follows that (X1 , . . . , Xn ) is independent of Xn+1 and (X1 , . . . , Xn ) has density


Pn
f(X1 ,...,Xn ) (x1 , . . . , xn ) = n! for x1 ≥ 0, . . . , xn ≥ 0, `=1 x` ≤ 1.
This establishes (9.6a).
Using equation(7.8a) on page 24 shows that the density of (D1 , . . . , Dn ) is the same as the density of (X1 , . . . , Xn ). Also
D1 + · · · + Dn + Dn+1 = 1 and X1 + · · · + Xn + Yn+1/Sn+1 = 1. Hence (9.6b).
Corollary(9.6b). With the same notation as the proposition,
 
S1 Sn
,..., has the same distribution as (U1:n , . . . , Un:n ) (9.6c)
Sn+1 Sn+1
Also  
S1 Sn
,..., is independent of Sn+1 . (9.6d)
Sn+1 Sn+1
Proof. The proposition shows that (X1 , . . . , Xn ) has the same distribution (D1 , . . . , Dn ). Taking partial sums of both
sides gives (9.6c). Result (9.6d) follows directly from (9.6a).
9.7 Decomposition of an exponential. In exercise 12.6 on page 38 it is shown that if X ∼ gamma( 1/2, α),
d
Y ∼ gamma( 1/2, α) and X and Y are independent, then X +Y has the gamma(1, α) = exponential (α) distribution.
Thus an exponential can be expressed as the sum of two independent gammas. A further decomposition is given
in exercise 10.4.

9.8
Summary. The exponential distribution.
• Density. X ∼ exponential (λ) iff fX (x) = λe−λx for x > 0.
• The distribution function. FX (x) = 1 − e−λx for x > 0.
• Moments.
1 2 1
E[X] = E[X 2 ] = 2 var[X] = 2
λ λ λ
• M.g.f. and c.f.
λ λ
MX (t) = E[etX ] = for t < λ. φX (t) = E[eitX ] =
λ−t λ − it
• Properties.
Suppose X ∼ exponential (λ) then αX ∼ exponential (λ/α).
The lack of memory property: P[X > x + t|X > x] = P[X > t].
If X1 , . . . , Xn are i.i.d. exponential (λ), then X1:n ∼ exponential (nλ).

10 Exercises (exs-exponential.tex)

10.1 Suppose X ∼ exponential (λ).


(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
10.2 The failure rate function of the exponential distribution.
(a) Suppose λ ∈ (0, ∞) and X ∼ exponential (λ). Show that the hazard function hX is constant on (0, ∞). The
exponential is said to have a constant failure rate.
(b) Suppose X is a random variable with an absolutely continuous distribution on (0, ∞) and X has a constant failure
rate. Show that X has an exponential distribution. [Ans]
10.3 Most elementary analysis texts contain a proof of the result:
 n
1
lim 1 − =e
n→∞ n
By using this result, show that if {δn } is a real sequence in (0, ∞) such that δn → 0 as n → ∞ and α ∈ (0, ∞), then
lim (1 − αδn )t/δn = e−αt
n→∞
This is used in equation(9.2a) on page 28. [Ans]
10.4 Suppose X ∼ exponential (λ).
(a) Let Y be the integer part of X; hence Y = bXc. Let Z be the fractional part of X; hence Z = X − bXc.
Find the distributions of Y and Z and show that Y and Z are independent.
(b) Now let W = dXe, the ceiling of X; hence {W = n} = {n − 1 < X ≤ n}. Find the distribution of W . [Ans]
Page 32 §10 Mar 10, 2020(20:25) Bayesian Time Series Analysis

10.5 Suppose X ∼ uniform(0, 1) and λ > 0. Prove that


ln(1 − X)
Y =− ∼ exponential (λ) [Ans]
λ
10.6 Suppose X and Y are i.i.d. random variables with the exponential (λ) distribution.
(a) Find the distribution of V = X/(X + Y ). This is a special case of part (a) of exercise 12.7 on page 38.
(b) Find the distribution of Z = (X − Y )/(X + Y ). [Ans]
10.7 Suppose X and Y are i.i.d. random variables with the exponential (λ) distribution. Find the distribution of Z = X/(Y +1).
[Ans]
10.8 (a) Suppose X ∼ exponential (λ), Y ∼ exponential (µ) and X and Y are independent. Show that
λ
P[X < Y ] =
λ+µ
(b) Suppose X1 , . . . , Xn are independent random variables with Xj ∼ exponential (λj ) for j = 1, . . . , n. Show that for
k ∈ {1, 2, . . . , n} we have
λk
P[ Xk < min{X` : ` 6= k} ] = Pn
j=1 λj
(c) Suppose X1 , . . . , Xn are independent random variables with Xj ∼ exponential (λj ) for j = 1, . . . , n. Show that
     Yn
λ1 λ2 λn−1 λ
P[X1 < X2 < · · · < Xn ] = ··· = Pn k
λ1 + · · · + λn λ2 + · · · + λn λn−1 + λn j=k λj
k=1
(d) Deduce that the event A = { X1 < min(X2 , . . . , Xn ) } is independent of the event B = {X2 < · · · < Xn }. [Ans]
10.9 Suppose X ∼ exponential (µ) and Y ∼ exponential (δ) where 0 < δ ≤ µ. Suppose further that f : (0, ∞) → (0, ∞)
with f differentiable and f 0 (x) > 0 for all x > 0. Prove that E[f (X)] ≤ E[f (Y )]. [Ans]
10.10 Suppose X and Y are i.i.d. random variables with the exponential (λ) distribution. Find the conditional density of X
given X + Y = z. What is E[X|X + Y ]? [Ans]
10.11 Suppose X1 and X2 are i.i.d. random variables with the exponential (λ) distribution. Let Y1 = X1 −X2 and Y2 = X1 +X2 .
(a) Find the densities of Y1 and Y2 . Note that Y1 has the Laplace(0, λ) distribution—see §27.1 on page 81.
(b) What is the density of R = |X1 − X2 |? [Ans]
10.12 Suppose the random variables X and Y are i.i.d. with the exponential (1) distribution. Let U = min{X, Y } and
V = max{X, Y }. Prove that U ∼ exponential (2) and V ∼ X + 12 Y . [Ans]
10.13 A characterization of the exponential distribution. Suppose X1 and X2 are i.i.d. random variables which are non-negative
and absolutely continuous. Let Y = min{X1 , X2 } and R = |X1 − X2 |. Then Y and R are independent iff X1 and X2
have the exponential distribution. [Ans]
10.14 Suppose X1 ∼ exponential (λ1 ), X2 ∼ exponential (λ2 ) and X1 and X2 are independent.
(a) Find P[min{X1 , X2 } = X1 ].
(b) Show that {min{X1 , X2 } > t} and {min{X1 , X2 } = X1 } are independent.
(c) Let R = max{X1 , X2 } − min{X1 , X2 }. Find P[R > t].
(d) Show that min{X1 , X2 } and R are independent. [Ans]
10.15 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the exponential (λ) distribution and let Xn:n denote the max-
imum. Suppose further thatPY1 , Y2 , . . . , Yn are independent random variables such that Yk ∼ exponential (kλ) for
n
k = 1, 2, . . . , n and let Y = k=1 Yk .
d
Show that Xn:n = Y . [Ans]
10.16 Suppose X1 , . . . , Xn are i.i.d. random variables with the exponential (λ) distribution. Find
(a) E[Xk:n ] (b) var[Xk:n ] (c) cov[Xj:n , Xk:n ]. [Ans]
10.17 Suppose X1 , . . . , Xn are i.i.d. random variables with the exponential (λ) distribution.
Let Z = nX1:n + (n − 1)X2:n + · · · + 2X(n−1):n + Xn:n . Find E[Z] and var[Z]. [Ans]
10.18 Suppose X1 , . . . , Xn are i.i.d. random variables with the exponential (λ) distribution. Prove that
Xn
X1:n is independent of (X` − X1:n ) [Ans]
`=1

10.19 Suppose X1 , X2 , X3 , . . . is a sequence of i.i.d. random variables with the exponential (1) distribution. For n =
1, 2, 3, . . . , let Zn = max{X1 , . . . , Xn }. Show that
n
e−x

−x
P[Zn − ln(n) < x] = 1 − and lim P[Zn − ln(n) < x] = e−e [Ans]
n n→∞
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §11 Page 33

10.20 (a) Sum of two independent exponentials. Suppose X ∼ exponential (λ), Y ∼ exponential (µ) and X and Y are
independent. Find the density of Z = X + Y .
(b) Ratio of two independent exponentials. Suppose X ∼ exponential (λ), Y ∼ exponential (µ) and X and Y are
independent. Find the distribution of Z = X/Y .
(c) Product of two independent exponentials. Suppose X ∼ exponential (λ), Y ∼ exponential (µ) and X and Y are
independent.
(i) Find the distribution function of Z = XY . Express your answer in terms of the modified Bessel function of the
second kind, order 1, which is:
Z ∞
K1 (y) = cosh(x)e−y cosh(x) dx for <(x) > 0.
x=0

(ii) Find the density of Z = XY . Express your answer in terms of the modified Bessel function of the second kind,
order 0, which is:
Z ∞
K0 (y) = e−y cosh(x) dx for <(x) > 0.
x=0

(iii) Write down what these answers become when λ = µ. [Ans]


10.21 Suppose X1 , . . . , Xn are i.i.d. random variables with the exponential (λ) distribution. Define Y1 , . . . , Yn as follows:
Y1 = X1 , Y2 = X1 + X2 , . . . , Yn = X1 + · · · + Xn .
Find the density of the vector (Y1 , . . . , Yn ). [Ans]
10.22 Sum of a random number of independent exponentials. Suppose X1 , X2 , X3 , . . . are i.i.d. random variables with
the exponential (λ) distribution. Suppose further that N is an integer valued random variable which is independent of
X = (X1 , X2 , . . .) and such that P[N = k] = q k−1 p for k ∈ {1, 2, . . .} where q ∈ (0, 1) and p = 1 − q. Let
Z = X1 + · · · + XN
Show that Z ∼ exponential (pλ). [Ans]
10.23 Suppose λ ∈ (0, ∞) and X ∼ exponential (λ). Suppose further that Y is a random variable which is independent of X
and takes values in (0, ∞).
(a) Prove that the conditional distribution of X − Y given X > Y is exponential (λ).
(b) By using part(a) or otherwise, prove that Y and X − Y are conditionally independent given X > Y . [Ans]
10.24 As usual, let N denote the positive integers {1, 2, . . .}. Suppose
P∞ {Xk : k ∈ N} is a countable collection of random
variables such that Xk ∼ exponential (λk ) for every k ∈ N and k=1 λk < ∞.
P∞
(a) Let M = inf{Xk : k ∈ N}. Show that M ∼ exponential ( k=1 λk ).
(b) Suppose i ∈ {1, 2, . . .}. Show that
λi
P[Xi < Xj for all j ∈ N − {i} ] = P∞
k=1 λk
P∞ P∞
(c) Let Y = k=1 Xk and µ = k=1 1/λk . Because all terms arePnon-negative, we can interchange the integral and

summation and obtain E[Y ] = µ. Show that P[Y < ∞] = 1 iff k=1 1/λk < ∞. [Ans]

11 The gamma and chi-squared distributions


11.1 Definition of the Gamma distribution.
Definition(11.1a). Suppose n > 0 and α > 0. Then the random variable X has the gamma(n, α) distribution
iff X has density
αn xn−1 e−αx
f (x) = for x > 0. (11.1a)
Γ(n)
R∞
By definition, Γ(n) = xn−1 e−x dx for all n ∈ (0, ∞)3 . It follows that
0
Z ∞
Γ(n)
xn−1 e−αx dx = n provided α > 0 and n > 0. (11.1b)
0 α
We shall see that n is a shape parameter and 1/α is a scale parameter; sometimes α is called the rate parameter.
The gamma distribution when n is an integer is called the Erlang distribution.

R∞ √
3
The integral 0
xn−1 e−x dx diverges for n ≤ 0. Also Γ( 1/2) = π, Γ(1) = 1 and Γ(n) = (n − 1)! for n ∈ {2, 3, . . .}.
Page 34 §11 Mar 10, 2020(20:25) Bayesian Time Series Analysis

11.2 The distribution function. There is a simple expression for the distribution function only when n is a
positive integer: if X ∼ gamma(n, α) where n is a positive integer then
αx (αx)2 (αx)n−1
 
−αx
P[X ≤ x] = 1 − e 1+ + + ··· + (11.2a)
1! 2! (n − 1)!
This is easy to check—differentiating the right hand side of equation(11.2a) gives the density in equation(11.1a).
Equation(11.2a) also implies
αxe−αx (αx)2 e−αx (αx)n−1 e−αx
P[X > x] = e−αx + + + ··· +
1! 2! (n − 1)!
Example(11.2a). Suppose X ∼ Γ(4, 1/100). Find P[X > 300].
Now αx = 3. Hence P[X > 300] = e−3 1 + 3 + 9/2 + 27/6 = 13e−3 .
 
Solution.

11.3 Multiple of a gamma distribution. Suppose n > 0 and α > 0 and X ∼ gamma(n, α) with density fX (x).
Suppose further that Y = βX where β > 0. Then the density of Y is given by:
fX (x) αn ( y/β )n−1 exp(−αy/β)
fY (y) = =
dy
dx βΓ(n)

Hence Y = βX ∼ gamma(n, α/β ). Thus the parameter 1/α is a scale parameter of the gamma distribution—if
X has a gamma distribution with scale parameter b and Y = βX, then Y has a gamma distribution with scale
parameter βb. Also, for any fixed n > 0, the family of distributions {gamma(n, α) : α > 0} is a scale family—see
definition(1.6d) on page 5.
R
11.4 Moments and shape of the gamma distribution. Using the result that f (x) dx = 1 easily gives
Γ(n + k)
E[X k ] = for n + k > 0. (11.4a)
αk Γ(n)
and so 
n n 1 α
E[X] = and var(X) = 2 and E = for n > 1. (11.4b)
α α X n−1

0.8 n = 21
n=1
0.6 n=2

0.4

0.2

0.0

1 2 3 4
Figure(11.4a). Plot of gamma density function for n = 21 , n = 1 and n = 2 (all with α = 1).
(wmf/gammadensity-fig001,121mm,73mm)

Figure (11.4a) shows that n is a shape parameter of the gamma distribution. By §11.3, we know that if X ∼
gamma(n, α) then Y = X/α ∼ gamma(n, 1). So without loss of generality, we now consider the shape of the
density of gamma(n, 1) distribution.
Let g(x) = xn−1 e−x . If n ≤ 1, then g(x) = e−x /x1−n is monotonic decreasing and hence the density of the
gamma(n, 1) distribution is monotonic decreasing.
If n > 1, then g 0 (x) = e−x xn−2 [n − 1 − x]. Clearly, if x < n − 1 then g 0 (x) > 0; if x = n − 1 then g 0 (x) = 0 and
if x > n − 1 then g 0 (x) < 0. Hence the density first increases to the maximum at x = n − 1 and then decreases.
By using §11.3, it follows that the maximum of the density of a gamma(n, α) density occurs at x = (n − 1)/α.
See also exercises 12.3 and 12.4 on page 38.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §11 Page 35

11.5 The moment generating function of a gamma distribution. Suppose X ∼ gamma(n, α). Then
Z ∞ n n−1 e−αx Z ∞
tx α x αn
tX
MX (t) = E[e ] = e dx = xn−1 e−(α−t)x dx
0 Γ(n) Γ(n) 0
αn Γ(n) 1
= = for t < α. (11.5a)
Γ(n) (α − t)n (1 − t/α)n
Hence the characteristic function is 1/(1 − it/α)n ; in particular, if n = 1, the characteristic function of the
exponential(α) distribution is α/(α − it).
Equation(11.5a) shows that for integral n, the Gamma distribution is the sum of n independent exponentials. The
next paragraph gives the long proof of this.
11.6 Representing the gamma distribution as a sum of independent exponentials. The following propo-
sition shows that the distribution of the waiting time for the nth event in a Poisson process with rate α has the
gamma(n, α) distribution.
Proposition(11.6a).Suppose X1 , X2 . . . . , Xn are i.i.d. random variables with the exponential density αe−αx
for x ≥ 0. Then Sn = X1 + · · · + Xn ∼ gamma(n, α).
Proof. By induction: let gn denote the density of Sn . Then for all t > 0 we have
Z t Z t n
α (t − x)n−1 e−α(t−x) −αx
gn+1 (t) = gn (t − x)αe−αx dx = αe dx
0 0 Γ(n)
x=t
αn+1 e−αt t αn+1 e−αt (t − x)n αn+1 tn e−αt
Z 
= (t − x)n−1 dx = − = as required.
Γ(n) 0 Γ(n) n x=0 Γ(n + 1)
The result that the sum of n independent exponentials has the Gamma distribution is the continuous analogue of
the result that the sum of n independent geometrics has a negative binomial distribution.
Link with the Poisson distribution and Poisson process. Equation(11.2a) on page 34 implies P[X ≤ x] = P[Y ≥
n] where Y has a Poisson distribution with expectation αx. In terms of the Poisson process with rate α, the
relation P[X ≤ x] = P[Y ≥ n] means that the nth event occurs before time x iff there are at least n events in
[0, x].
Suppose Sn = X1 + · · · + Xn as in the proposition. Let Nt denote the number of indices k ≥ 1 with Sk ≤ t. Then
e−αt (αt)n
P[Nt = n] = P[Sn ≤ t and Sn+1 > t] = Gn (t) − Gn+1 (t) =
n!
by using equation(11.2a) on page 34. An alternative statement of this result: suppose X1 , . . . , Xn+1 are i.i.d. ran-
dom variables with the exponential (λ) distribution and x0 > 0. Then
e−αx0 (αt)n
P[X1 + · · · + Xn ≤ x0 < X1 + · · · + Xn+1 ] =
n!
11.7 Normal limit and approximation. Suppose Gn ∼ gamma(n, α). Using the representation in proposi-
tion(11.6a) and the fact that each Xj has expectation 1/α and variance 1/α2 , the central limit theorem implies
Gn − E[Gn ] Gn − n/α D
√ = p =⇒ N (0, 1) as n → ∞.
var[Gn ] n/α2
and hence for large n
αx − n
 
P[Gn ≤ x] ≈ Φ √
n
The local central limit theorem4 showsthat
√ √ 
n n+z n 1 1 2
lim fGn = n(z) where n(x) = √ e− 2 x (11.7a)
n→∞ α α 2π
See exercise 12.18 on page 39 below.

4
The local central limit theorem. Suppose Y1 , Y2 , . . . are i.i.d. random variables with mean 0 and variance 1 and characteristic
k
√ |φY | is integrable for some positive k and sup{|φY (t)| : |t| ≥ δ} < 1 for all δ > 0. Let
function φY . Suppose further that
Sn = Y1 + · · · + Yn ; then Sn / n has a bounded continuous density fn for all n ≥ k and supx∈R |fn (x) − n(x)| → 0 as
n → ∞.
This formulation is due to Michael Wichura: galton.uchicago.edu/~wichura/Stat304/Handouts/L16.limits.pdf.
See also page 516 in [F ELLER(1971)].
Page 36 §11 Mar 10, 2020(20:25) Bayesian Time Series Analysis

11.8 Lukacs’ characterization of the gamma distribution.


Proposition(11.8a). Suppose X and Y are both positive, non-degenerate 5 and independent random variables.
Then X/(X +Y ) is independent of X +Y iff there exist k1 > 0, k2 > 0 and α > 0 such that X ∼ gamma(k1 , α)
and Y ∼ gamma(k2 , α).
Proof.
⇐ This is exercise 12.7 on page 38.
⇒ This is proved in [L UKACS(1955)] and [M ARSAGLIA(1989)].
We can easily extend this result to n variables:
Proposition(11.8b). Suppose X1 , X2 , . . . , Xn are positive, non-degenerate and independent random variables.
Then Xj /(X1 + · · · + Xn ) is independent of X1 + · · · + Xn for j = 1, 2, . . . , n iff there exist α > 0,
k1 > 0, . . . , kn > 0 such that Xj ∼ gamma(kj , α) for j = 1, 2, . . . , n.
Proof.
⇐ Let W = X2 + · · · + Xn . By equation(11.5a), W ∼ gamma(k2 + · · · + kn , α). Also X1 ∼ gamma(k1 , α) and
W and X1 are independent positive random variables. Hence X1 /(X1 + · · · + Xn ) is independent of X1 + · · · + Xn by
proposition(11.8a). Similarly Xj /(X1 + · · · + Xn ) is independent of X1 + · · · + Xn for j = 2, . . . , n.
⇒ Let Wj = X1 + · · · + Xn − Xj . Then Wj and Xj are independent positive random variables. Also Xj /(Wj + Xj ) is
independent of Wj + Xj . By proposition(11.8a), there exist kj > 0, kj∗ > 0 and αj > 0 such that Xj ∼ gamma(kj , αj )
and Wj ∼ gamma(kj∗ , αj ). Hence X1 + · · · + Xn = Wj + Xj ∼ gamma(kj + kj∗ , αj ). The same argument works for
j = 1, 2, . . . , n; this implies α1 = · · · = αn . The result follows.

11.9 The χ2 distribution. For n ∈ (0, ∞) the gamma( n/2, 1/2) distribution has density:
xn/2−1 e−x/2
f (x) = for x > 0. (11.9a)
2n/2 Γ( n/2)
This is the density of the χ2n distribution. If n is a positive integer, then n is called the degrees of freedom.
Definition(11.9a). Suppose n > 0. Then the random variable has a χ2n distribution iff X ∼ gamma( n/2, 1/2).
Moments of the χ2 distribution. If Y ∼ χ2n = gamma( n/2, 1/2), then equation(11.4a) shows that the k th moment of
Y is given by
( n
k Γ(k+ /2) if n > −2k;
E[Y ] = 2 Γ( n/2)
k
(11.9b)
∞ if n ≤ −2k.
In particular E[Y ] = n, E[Y 2 ] = n(n + 2), var[Y ] = 2n,


 
2Γ( (n+1)/2) 1 1
E[ Y ] = and E = provided n > 2. (11.9c)
Γ( /2)
n Y (n − 2)

Convolutions of independent χ2 distributions. By equation(11.5a), the c.f. of the χ2n distribution is 1/(1 − 2it)n/2 .
It immediately follows that if X ∼ χ2m , Y ∼ χ2n and X and Y are independent, then X + Y ∼ χ2m+n .
If X ∼ χ2k and n ∈ {1, 2, . . .}, then X = Y1 + · · · + Yn where Y1 , . . . ,Yn are i.i.d. random variables with the
χ2k/n distribution. Hence by definition(5.1a) on page 15, the distribution χ2k is infinitely divisible.
Relations with other distributions.
• If n ∈ (0, ∞) and X ∼ gamma(n, α) then 2αX ∼ gamma(n, 1/2) = χ22n .
• It immediately follows from the expressions for the densities that χ22 = exponential ( 1/2).
• If X ∼ N (0, 1), then X 2 ∼ χ21 ; also if X1 , . . . , Xn are i.i.d. random variables with the N (0, 1) distribution
then X12 + · · · + Xn2 ∼ χ2n . See §15.5 on page 48 for the proofs of these two results.
• Suppose Vn ∼ χ2n ; then we can represent Vn as the sum of n i.i.d. random variables each with expectation 1
and variance 2. Hence, by the central limit theorem,
Vn − E[Vn ] Vn − n D
√ = √ =⇒ N (0, 1) as n → ∞.
var[Vn ] 2n

5
To exclude the trivial case that both X and Y are constant. In fact if one of X and Y is constant and X/(X +Y ) is independent
of X + Y , then the other must be constant also.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §12 Page 37

11.10 The generalized gamma distribution.


Definition(11.10a). Suppose n > 0, λ > 0 and b > 0. Then the random variable X has the generalized
gamma distribution ggamma(n, λ, b) iff X has density
bλn bn−1 −λxb
f (x) = x e for x > 0. (11.10a)
Γ(n)
Note that
• if n = b = 1, then the generalized gamma is the exponential distribution;
• if b = 1, the generalized gamma is the gamma distribution;
• if n = 1, the generalized gamma is the Weibull distribution—introduced below in §27.2 on page 82;
• if n = 1, b = 2 and λ = 1/2σ2 , the generalized gamma is the Rayleigh distribution—introduced below in §27.4
on page 83;
and finally
• if n = 1/2, b = 2 and λ = 1/2σ2 then the generalized gamma is the half-normal distribution—introduced in
exercise 16.25 on page 52.
It is left to an exercise (see exercise 12.22 on page 39) to check:
• The function f in equation(11.10a) integrates to 1 and so is a density.
• If X ∼ ggamma(n, λ, b) then Y = X b ∼ gamma(n, λ).
• The central moments are given by the expression:
Γ( k/b + n)
E[X k ] = b/k
λ Γ(n)
The generalized gamma distribution is used in survival analysis and reliability theory to model lifetimes.

11.11
Summary. The gamma distribution.
• Density. X has the gamma(n, α) density for n > 0 and α > 0 iff
αn xn−1 e−αx
fX (x) = for x > 0.
Γ(n)
• Moments. E[X] = n/α; var[X] = n/α2 and E[X k ] = Γ(n+k)/αk Γ(n) for n + k > 0.
• M.g.f. and c.f.
1 1
MX (t) = E[etX ] = for t < α. φX (t) = E[eitX ] =
(1 − t/α)n (1 − it/α)n
• Properties.
gamma(1, α) is the exponential (α) distribution.
If X ∼ gamma(n, α) and β > 0 then βX ∼ gamma(n, α/β ).
The gamma(n, α) distribution is the sum of n independent exponential (α) distributions.
If X ∼ gamma(m, α), Y ∼ gamma(n, α) and X and Y are independent, then X + Y ∼ gamma(m + n, α).
The χ2n distribution.
• This is the gamma( n/2, 1/2) distribution.
• If X ∼ χ2n , then E[X] = n, var[X] = 2n and the c.f. is φ(t) = 1/(1 − 2it)n/2 .
• If X ∼ χ2m , Y ∼ χ2n and X and Y are independent, then X + Y ∼ χ2m+n .
• The χ22 distribution is the exponential ( 1/2) distribution.

12 Exercises (exs-gamma.tex)
R∞ x−1 −u
12.1 The Gamma function. This is defined to be Γ(x) = 0 u e du for x > 0. Show that
(a) Γ(x + 1) = x Γ(x) for all x > 0; (b) Γ(1) = 1

(c) Γ(n) = (n − 1)! for all integral n ≥ 2; (d) Γ( 1/2) = π
 1.3.5 . . . (2n − 1) √ (2n)! √
(e) Γ n + 1/2 = π = 2n π for integral n ≥ 1 [Ans]
2n 2 n!
12.2 Suppose X ∼ Γ(n, α).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X.
q
In particular, if X ∼ χ2n , then skew[X] = n8 and κ[X] = 3 + 12 n. [Ans]
Page 38 §12 Mar 10, 2020(20:25) Bayesian Time Series Analysis

12.3 By §11.4 on page 34, we know that if n > 1, the maximum of the gamma(n, 1) density occurs at x = n − 1. Show that
the maximum value of the density when n > 1 is approximately
1

2π(n − 1)

Hint: Stirling’s formula is n! ∼ nn e−n 2πn as n → ∞. [Ans]
12.4 Suppose f is the density of the Γ(n, α) distribution.
(a) Show that if 0 < n ≤ 1 then the function f is convex (f 00 ≥ 0).

(b) Show that if 1 < n ≤ 2 then f is concave and then convex with a point of inflection at x = (n − 1 + n − 1)/α.

(c) Show that if n√> 2 then f is convex, then concave, and then convex with points of inflection at (n − 1 − n − 1)/α
and (n − 1 + n − 1)/α. [Ans]
12.5 Suppose X ∼ gamma(m, α) and Y ∼ gamma(n, α) and X and Y are independent. Find E[ Y /X ]. [Ans]
12.6 (a) Gamma densities are closed under convolution. Suppose X ∼ gamma(n1 , α), Y ∼ gamma(n2 , α) and X and Y are
independent. Prove that X + Y has the gamma(n1 + n2 , α) distribution.
(b) Hence show that the distribution gamma(n, α) is infinitely divisible. [Ans]
12.7 Suppose X ∼ gamma(m, α) and Y ∼ gamma(n, α) and X and Y are independent.
(a) Show that U = X + Y and V = X/(X + Y ) are independent..
(b) Show that U = X + Y and V = Y /X are independent. fs In both cases, find the densities of U and V . [Ans]
12.8 Suppose X1 ∼ gamma(k1 , λ), X2 ∼ gamma(k2 , λ) and X3 ∼ gamma(k3 , λ). Suppose further that X1 , X2 and X3 are
independent. Let
X1 X1 + X2
Y1 = Y2 = Y3 = X1 + X2 + X3
X1 + X2 X1 + X2 + X3
Show that Y1 , Y2 and Y3 are independent and find their distributions. [Ans]
12.9 Suppose X ∼ gamma(n, α). Show that
   n
2n 2
P X≥ ≤ [Ans]
α e
12.10 Suppose the random variable X has the following central moments:
(a) E[X k ] = 2k−1 (k + 2)! for k = 1, 2, 3, . . . . (b) E[X k ] = 2k (k + 1)! for k = 1, 2, 3, . . . .
In both cases, find the distribution of X. [Ans]
12.11 Suppose X ∼ gamma(m, α) and Y ∼ gamma(n, α) and X and Y are independent. Show that
n mv
if v > 0;
E[X|X + Y = v] = m+n [Ans]
0 otherwise.
12.12 Suppose the random variable X ∼ exponential (Y ) where Y ∼ gamma(n, α).
(a) Find the density of X and the value of E[X].
(b) Find the conditional density of Y given X = x. [Ans]
12.13 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the exponential (λ) distribution. Suppose further that Z ∼
exponential (µ) and is independent of X1 , X2 , . . . , Xn . Find
P[X1 + · · · + Xn > Z] [Ans]
12.14 Suppose X ∼ gamma(α, λ). Show that
α
[ψ(α + 1) − ln λ]
E[ X ln X ] =
λ
6
where ψ denotes the digamma function which satisfies
Z ∞
d Γ0 (z)
ψ(z) = ln( Γ(z) ) = and hence Γ(z)ψ(z) = Γ0 (z) = xz−1 e−x ln x dx [Ans]
dz Γ(z) x=0

12.15 Suppose X ∼ exponential (λ), and given X = x, the n random variables Y1 , . . . , Yn are i.i.d. exponential (x).7 Find the
distribution of (X|Y1 , . . . , Yn ) and E[X|Y1 , . . . , Yn ]. [Ans]
12.16 Suppose X ∼ gamma(n, α). By expressing the density of X in the form
1
fX (x) = αn exp [(n − 1) ln x − αx] for x ∈ (0, ∞).
Γ(n)
show that gamma(n, α) belongs to the exponential family of distributions with natural parameters n − 1 and −α and
natural statistics ln X and X. [Ans]
6
See page 258 in [A BRAMOWITZ &S TEGUN(1965)]
7
This means that f(Y1 ,...,Yn )|X (y1 , . . . , yn |x) = Πni=1 fYi |X (yi |x) = xn e−x(y1 +···+yn ) .
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §12 Page 39

12.17 The Poisson-Gamma mixture. Suppose X ∼ gamma(n, α); suppose further that given the random variable X then Y
has a Poisson distribution with expectation X. Compute P[Y = j] for j = 0, 1, 2, . . . . [Ans]
12.18 Suppose Gn ∼ gamma(n,
Pn α) and Sn = α(Gn − n/α) where n > 1 is an integer. Hence Sn = α(Gn − n/α) =
Pn
i=1 α(Xi − 1/α) = i=1 Yi where each Xi ∼ exponential (α) and each Yi has mean 0 and variance 1.
Check that the conditions of the local central limit theorem (§11.7 on page 35) are satisfied and hence verify the limiting
result(11.7a) on page 35. [Ans]
12.19 Length biased sampling in the Poisson process. Suppose {Xj }j≥1 is a sequence of i.i.d. random variables with the
exponential (α) distribution. For n ≥ 1, let Sn = X1 + · · · + Xn and suppose t ∈ (0, ∞).
Define the random variable K to be the unique integer with SK−1 < t ≤ SK ; equivalently K = min{j : Sj ≥ t}.
(a) Find the density of XK . Find E[XK ] and compare with 1/α, the expectation of an exponential (α) distribution.
Note that a longer interval has a higher chance of containing t!
(b) Let Wt denote the waiting time to the next event after time t; hence Wt = SK − t.
Find the distribution of Wt . [Ans]
12.20 Suppose X ∼ χ2n .
By expressing the density of X in the form
e−x/2  
fX (x) = n/2 exp ( n/2 − 1) ln x for x ∈ (0, ∞).
2 Γ(n/2)
2
show that χn belongs to the exponential family of distributions with natural parameter n/2 − 1 and natural statistic ln X.
[Ans]
12.21 Suppose U ∼ uniform(0, 1). Find the distribution of Y = −2 ln(U ). [Ans]
12.22 The generalized gamma distribution.
(a) Show that the function f defined in equation(11.10a) is a density.
(b) Suppose X ∼ ggamma(n, λ, b). Show that Y = X b ∼ gamma(n, λ).
(c) Suppose X ∼ ggamma(n, λ, b). Find the central moments E[X k ] for k = 1, 2, . . . . [Ans]
8
12.23 (a) Suppose that for every p ∈ (0, 1), the random variable Xp has the negative binomial distribution with parameters
k and p; this means that
k−1 j
 
P[Xp = k] = p (1 − p)k−j for k ∈ {j, j + 1, j + 2, . . .}.
k−j
Prove that
D
pXp =⇒ gamma(j, 1) as p → 0.
(b) Suppose j ∈ {1, 2, . . .} and λ ∈ (0, ∞); suppose further that {pn }n≥1 is a sequence in (0, 1) such that npn → λ
as n → ∞. Finally, Suppose that for every n ∈ {1, 2, . . .} the random variable Xn has the negative binomial
k−1 j
distribution with parameters j and pn ∈ (0, 1); this means P[Xn = k] = k−j pn (1 − pn )k−j for k ∈ {j, j + 1, j +


2, . . .}. Prove that


Xn D
=⇒ gamma(j, λ) as n → ∞ [Ans]
n

8
The negative binomial distribution is the distribution of the serial number of the j th success in a sequence of Bernoulli trials.
Page 40 §13 Mar 10, 2020(20:25) Bayesian Time Series Analysis

13 The beta and arcsine distributions


The beta distribution.
The distribution of order statistics from the uniform distribution uniform(0, 1) leads to the beta distribution—this
is considered in §7.5 on page 23.
13.1 The density and distribution function.
Definition(13.1a). Suppose α > 0 and β > 0. Then the random variable X has the beta distribution,
beta(α, β), iff X has density
Γ(α + β) α−1
f (x; α, β) = x (1 − x)β−1 for 0 < x < 1. (13.1a)
Γ(α)Γ(β)
R∞
• Checking equation(13.1a) is a density function. Now Γ(α) = 0 uα−1 e−u du by definition. Hence
Z ∞Z ∞
Γ(α)Γ(β) = uα−1 v β−1 e−u−v du dv
0 0
Now use the transformation x = u/(u + v) and y = u + v; hence u = xy and v = y(1 − x). Clearly 0 < x < 1 and

0 < y < ∞; also ∂(u,v)
∂(x,y) = y. Hence

Z 1 Z ∞
Γ(α)Γ(β) = y α+β−1 xα−1 (1 − x)β−1 e−y dx dy
x=0 y=0
Z 1
= Γ(α + β) xα−1 (1 − x)β−1 dx
x=0

• The beta function is defined by


Z 1
Γ(α)Γ(β)
B(α, β) = tα−1 (1 − t)β−1 dt = for all α > 0 and β > 0.
0 Γ(α + β)
Properties of the beta and gamma functions can √ be found in most advanced calculus books. Recall that Γ(n) =
(n − 1)! if n is a positive integer and Γ( 1/2) = π.
Equation (13.1a) now becomes
xα−1 (1 − x)β−1
f (x; α, β) = for 0 < x < 1.
B(α, β)

• The distribution function of the beta distribution, beta(α, β) is


Z x Z x
1 Ix (α, β)
F (x; α, β) = f (t; α, β) dt = tα−1 (1 − t)β−1 dt = for x ∈ (0, 1).
0 B(α, β) 0 B(α, β)
The integral, Ix (α, β), is called the incomplete beta function.
13.2 Shape of the density. The beta density can take many different shapes—see figure(13.2a).
Finding the mode. By differentiation, f 0 (x; α, β) = 0 implies x(α + β − 2) = α − 1. This has a solution for x
with x ≥ 0 if either (a) α + β > 2 and α ≥ 1 or (b) α + β < 2 and α ≤ 1. Hence we have a solution for x with
x ∈ [0, 1] if either (a) α ≥ 1 and β ≥ 1 with α + β 6= 2 or (b) α ≤ 1 and β ≤ 1 with α + β 6= 2.9
By checking the second derivative, we see
α−1
mode[X] = if α ≥ 1 and β ≥ 1 with α + β 6= 2.
α+β−2
If α = β, then the density is symmetric about x = 1/2 and hence the expectation is 1/2.
For further results about the shape of the beta density, see exercise 14.1 on page 44.
R1
13.3 Moments. Using the fact that 0 xα−1 (1 − x)β−1 dx = B(α, β), it is easy to check that
α (α + 1)α αβ
E[X] = E[X 2 ] = and hence var[X] = 2
(13.3a)
α+β (α + β + 1)(α + β) (α + β) (α + β + 1)

9
The exceptional case α + β = 2 implies α = β = 1 and the density is uniform on [0, 1].
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §13 Page 41

3.0 α = 1/2 , β = 1/2 3.0 α = 2, β = 2


α = 5, β = 1 α = 2, β = 5
2.5 α = 1, β = 3 2.5 α = 5, β = 2

2.0 2.0

1.5 1.5

1.0 1.0

0.5 0.5

0.0 0.0

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

Figure(13.2a). Shape of the beta density for various values of the parameters.
(wmf/betadensity1,wmf/betadensity2,72mm,54mm)

13.4 Some distribution properties.


• Suppose X ∼ beta(α, β), then 1 − X ∼ beta(β, α).
• The beta(1, 1) distribution is the same as the uniform distribution on (0, 1). The beta(α, 1) distribution is the
same as the powerlaw(α, 0, 1) distribution.
• Suppose X ∼ gamma(n1 , α) and Y ∼ gamma(n2 , α). Suppose further that X and Y are independent. Then
X/(X + Y ) ∼ beta(n1 , n2 ). See exercise 12.7 on page 38. In particular, if X ∼ χ22k = gamma(k, 1/2),
Y ∼ χ22m = gamma(m, 1/2) and X and Y are independent, then X/(X + Y ) ∼ beta(k, m).
• If X ∼ beta(α, 1) then − ln(X) ∼ exponential (α). See exercise 14.4 on page 44.
• For the link between the beta and F distributions, see exercises 22.29 and 22.30 on page 71.
13.5 Linear transformation of a beta distribution. Suppose a ∈ R, h > 0 and X ∼ beta(α, β). Then the
density of the linear transformation Y = a + hX is
Γ(α + β) (y − a)α−1 (h + a − y)β−1
fY (y) = for a < y < a + h.
Γ(α)Γ(β) hα+β−1
This is called the beta(α, β) distribution with location parameter a and scale parameter h.
Clearly for fixed α > 0 and fixed β > 0, this family of beta distributions for a ∈ R and h ∈ (0, ∞) forms a
location-scale family—see definition(1.6b) on page 5.
The beta prime distribution.
13.6 The beta prime distribution—definition. A random variable X with the beta distribution takes values in
[0, 1] and so is often used to model a probability. The corresponding odds ratio, which is X/(1 − X), has a beta
prime distribution.
Definition(13.6a). Suppose α > 0 and β > 0. Then the random variable X is said to have the beta prime
distribution, beta0 (α, β), iff it has density
xα−1 (1 + x)−α−β
f (x) = for x > 0. (13.6a)
B(α, β)
The beta prime distribution is sometimes called the beta distribution of the second kind.
13.7 The beta prime distribution—basic properties.
Shape of the density function. See exercise 14.10.
Moments of the beta prime distribution. See exercise 14.11 on page 45.
X
Link between the beta and beta prime distributions. The key result is that if X ∼ beta(α, β), then 1−X ∼
0
beta (α, β). See exercise 14.13 on page 45.
If X ∼ beta(α, β), then Y = 1 − X ∼ beta(β, α); hence the previous result implies X1 − 1 ∼ beta0 (β, α).
Conversely, if X ∼ beta0 (α, β) then X/(1 + X) ∼ beta(α, β).
Page 42 §13 Mar 10, 2020(20:25) Bayesian Time Series Analysis

Distribution function. Suppose Y ∼ beta0 (α, β); then Y has distribution function
 
y
FY (y) = FX for y > 0.
1+y
where FX denotes the distribution function of X ∼ beta(α, β).
Links with other distributions. We shall see later (see §21.8 on page 66) that the beta prime distribution is just a
multiple of the F -distribution. Also, the standard beta prime distribution, beta0 (1, 1) is the same as the standard
log-logistic distribution, loglogistic(1, 1)—see equation(36.4d) on page 109.

The arc sine distribution.



13.8 The arcsine distribution on (0, 1). The arcsine distribution is the beta 1/2, 1/2 distribution.
Definition(13.8a). The random variable has the arcsine distribution on (0, 1) iff X has density
1
fX (x) = √ for x ∈ (0, 1).
π x(1 − x)
A plot of the density is given in figure(13.8a).
3.0

2.5

2.0

1.5

1.0

0.5

0.0 0.2 0.4 0.6 0.8 1.0


Figure(13.8a). Plot of the arcsine(0, 1) density
(wmf/arcsineDensity,72mm,54mm)

The distribution function. Suppose X has the arcsine distribution; then


Z x
du 2 √ arcsin(2x − 1) 1
FX (x) = P[X ≤ x] = √ = arcsin( x) = + for x ∈ [0, 1]. (13.8a)
u=0 π u(1 − u) π π 2
Linear transformation of the arcsine(0, 1) distribution. Suppose X ∼ arcsine(0, 1) and Y = a + bX where a ∈ R
and b ∈ (0, ∞). Then Y has density
1
fY (y) = √ for x ∈ (a, a + b).
π (y − a)(a + b − y)
This leads to the definition of the arcsine(a, b) distribution.
13.9 The arcsine distribution on (a, a + b).
Definition(13.9a). Suppose a ∈ R and b ∈ (0, ∞). Then the random variable X has the arcsine distribution
on (a, a + b), denoted arcsin(a, b), iff X has density
1
fX (x) = √ for x ∈ (a, a + b).
π (x − a)(a + b − x))
This means that the distribution defined in definition(13.8a) can also be described as the arcsin(0, 1) distribution.
Linear transformation of an arcsine(a, b) distribution. Suppose X ∼ arcsine(a, b) and c ∈ R and d ∈ (0, ∞).
Then c + dX ∼ arcsine(c + ad, bd).
In particular,
if X ∼ arcsine(0, 1) then a + bX ∼ arcsine(a, b);
if X ∼ arcsine(a, b) then (X − a)/b ∼ arcsine(0, 1).
It follows that the family of distributions {arcsin(a, b) : a ∈ R, b > 0} is a location-scale family—see defini-
tion(1.6b) on page 5.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §13 Page 43

The distribution function of the arcsine(a, b) distribution.


If X ∼ arcsine(a, b) then (X − a)/b ∼ arcsine(0, 1); hence
r !
X −a x−a x−a
 
2
FX (x) = P[X ≤ x] = P ≤ = arcsin for a ≤ x ≤ a + b. (13.9a)
b b π b
Quantile function and median. The quantile function of the arcsine(a, b) distribution is
2
FX−1 (p) = a + b sin(πp/2)

for p ∈ (0, 1). (13.9b)

Hence the median of the distribution is FX−1 (1/2) = a + b(1/ 2)2 = a + b/2.
Distribution of the square of an arcsine distribution.
By part(b) of exercise 14.20 on page 46, if X ∼ arcsine(−1, 2), then X 2 ∼ arcsine(0, 1). It follows that if
X ∼ arcsine(−1, 2), then
1+X
X 2 has the same distribution as
2
2
If also follows that if X ∼ arcsine(0, 1), then X and 4 X − 1/2 have the same distribution. This property
effectively characterizes the arcsine distribution—see theorem 1 in [A RNOLD &G ROENEVELD(1980)].
Further properties of the arcsine distribution are in exercises 14.15–14.18 on page 45.
13.10 The generalized arcsine distribution. This is the beta(1 − α, α) distribution where α ∈ (0, 1). Now
Euler’s reflection formula states that
π
Γ(x)Γ(1 − x) = for x ∈ (0, 1) (13.10a)
sin(πx)
This implies the density of the generalized arcsine distribution is
1 sin(πα) −α
f (x) = x−α (1 − x)α−1 = x (1 − x)α−1 for x ∈ (0, 1).
Γ(α)Γ(1 − α) π
The expectation is 1 − α and the variance is (1 − α)α/2. Setting α = 1/2 in this distribution gives the standard
arcsine(0, 1) distribution.

13.11
Summary.
The beta distribution. Suppose α > 0 and β > 0; thenX ∼ beta(α, β) iff X has density
Γ(α + β) α−1
f (x; α, β) = x (1 − x)β−1 for 0 < x < 1.
Γ(α)Γ(β)
• Moments:
α αβ
E[X] = var[X] =
α+β (α + β)2 (α + β + 1)
• Suppose X ∼ beta(α, β), then 1 − X ∼ beta(β, α).
• The beta(1, 1) distribution is the same as the uniform distribution on (0, 1).
• Suppose X ∼ gamma(n1 , α) and Y ∼ gamma(n2 , α). Suppose further that X and Y are independent. Then
X/(X + Y ) ∼ beta(n1 , n2 ).
nX
• If X ∼ beta(α, 1) then − ln(X) ∼ Exponential(α). If X ∼ beta( m/2, n/2) then m(1−X) ∼ Fm,n .
The beta prime distribution. Suppose α > 0 and β > 0; then X ∼ beta 0 (α, β) iff the density is
xα−1 (1 + x)−α−β
f (x) = for x > 0.
B(α, β)
X 0
• If X ∼ beta(α, β), then 1−X ∼ beta (α, β).
d
The arcsine distribution. If X ∼ arcsine(0, 1) = beta( 1/2, 1/2) then X has density
1
fX (x) = √ for x ∈ (0, 1).
π x(1 − x)
• Moments: E[X] = 1/2 and var[X] = 1/8.
Page 44 §14 Mar 10, 2020(20:25) Bayesian Time Series Analysis

14 Exercises (exs-betaarcsine.tex)

The beta distribution.

14.1 Shape of the beta density. Let


1−α
x0 =
2−α−β
(a) Suppose α ∈ (0, 1) and β ∈ (0, 1). Show that fX first decreases and then increases with the minimum value at
x = x0 . Also fX (x) → ∞ as x ↓ 0 and fX (x) → ∞ as x ↑ 1.
(b) If α = β = 1, then fX (x) = 1, which is the density of the uniform uniform(0, 1) distribution.
(c) If α ∈ (0, 1) and β ≥ 1 then fX (x) is decreasing with fX (x) → ∞ as x ↓ 0. If α ≥ 1 and β ∈ (0, 1) then fX (x) is
increasing with fX (x) → ∞ as x ↑ 1.
(d) If α = 1 and β > 1 then fX (x) is decreasing with mode at x = 0; if α > 1 and β = 1 then fX (x) is increasing with
mode at x = 10.
(e) If α > 1 and β > 1 then fX increases and then decreases with mode at x = x0 . [Ans]
14.2 Moments of the beta distribution. Suppose X ∼ beta(α, β).
(a) Find an expression for E[X m ] for m = 1, 2, . . . .
(b) Find E[X] and var[X].
(c) Find E[1/X] if α > 1. [Ans]
14.3 Suppose α > 0 and β > 0 and X ∼ beta(α, β).
(a) Find skew[X], the skewness of X. (Note: the distribution is positively skewed if α < β and negatively skewed if
α > β. If α = β then the distribution is symmetric about 1/2.)
(b) Find κ[X], the kurtosis of X.
(c) In particular, find the values of the skewness and kurtosis when X1 ∼ beta(2, 2), X2 ∼ beta(3, 2), X3 ∼ beta(2, 3)
d
and X4 ∼ beta( 1/2, 1/2) = arcsine(0, 1). [Ans]
14.4 Suppose X ∼ beta(α, 1). Show that − ln(X) ∼ exponential (α). [Ans]
14.5 Link between the beta and Pareto distributions.
(a) Suppose X ∼ beta(α, β). Find the density of Y = 1/X.
(b) In particular, if X ∼ beta(α, 1), show that Y = 1/X has the Pareto(α, 1, 0) distribution.
Conversely, if Y ∼ Pareto(α, 1, 0) then X = 1/Y ∼ beta(α, 1). [Ans]
14.6 Link between the beta distribution function and the binomial distribution function. Suppose X ∼ beta(k, n − k + 1)
and Y ∼ binomial (n, p). Prove that P[X > p] = P[Y ≤ k − 1], or equivalently P[X ≤ p] = P[Y ≥ k]. We assume
p ∈ [0, 1], k ∈ {1, 2, . . .} and n ∈ {k, k + 1, . . .}.
Note. This has already been proved—see equation(7.5a) on page 23 where it is shown that if X1 , . . . , Xn are i.i.d. with
the uniform(0, 1) distribution then the density of Xk:n is the beta(k, n − k + 1) distribution. Clearly {Xk:n ≤ p} if there
at least k of the n random variables X1 , . . . , Xn in the interval [0, p]. [Ans]
14.7 Suppose α > 0 and β > 0. Suppose further that Y ∼ beta(α, β) and X is a random variable such that the distribution
of X given Y = y is binomial (n, y). Show that the distribution of Y given X = k is beta(α + k, β + n − k). [Ans]
14.8 Suppose Y ∼ beta(α, β) where α > 0 and β > 0, and X is a random variable such that the distribution of X given
Y = y is negativebinomial (y, k)10 . Show that the distribution of Y given X = k is beta(α + k, β + n − k). [Ans]
14.9 Random partitions of an interval—see §7.8 on page 24. Suppose U1 , . . . , Un are i.i.d. random variables with the uniform
uniform(0, 1) distribution and let U1:n , . . . , Un:n denote the order statistics. These variables partition the interval [0, 1]
into n + 1 disjoint intervals with the following lengths:
D1 = U1:n , D2 = U2:n − U1:n , . . . , Dn = Un:n − U(n−1):n , Dn+1 = 1 − Un:n
Clearly D1 + · · · + Dn+1 = 1. Find E[Dk ] for k = 1, 2, . . . , n + 1. [Ans]

10
The random variable X ∼ negativebinomial (y, k) is X is the serial number of the k th success in a sequence of Bernoulli
trials with probability of success equal to p. Hence for n ∈ {k, k + 1, . . .} we have P[X = n] = n−1
 k n−k
k−1 p (1 − p) .
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §14 Page 45

The beta prime distribution


14.10 Shape of the density. Suppose α > 0 and β > 0. Suppose further that X ∼ beta 0 (α, β) has density function f .
(a) Find f 0 (x), the derivative of the density.
(b) Suppose α ∈ (0, 1); show that f is decreasing with f (x) → ∞ as x ↓ 0.
(c) Suppose α = 1; show that f is decreasing with mode at x = 0.
(d) Suppose α > 1; show that f first increases and then decreases with mode at (α − 1)/(β + 1).
(e) Find f 00 (x), the second derivative of the density.
(f) Let
√ √
(α − 1)(β + 2) − (α − 1)(β + 2)(α + β) (α − 1)(β + 2) + (α − 1)(β + 2)(α + β)
x1 = and x2 =
(β + 1)(β + 2) (β + 1)(β + 2)
Suppose α ∈ (0, 1]; show that f is convex. Suppose α ∈ (1, 2]; show that f is initially concave and then convex
with inflection point at x = x2 . Suppose α > 2; show that f is initially convex, then concave and then convex again
with inflection points at x = x1 and x = x2 . [Ans]
14.11 Moments.Suppose X has the beta prime distribution, beta 0 (α, β).
(a) Show that E[X] = α/(β − 1) provided β > 1.
(b) Show that var[X] = α(α + β − 1)/(β − 2)(β − 1)2 provided β > 2.
(c) Suppose −α < k < β; show that
B(α + k, β − k)
E[X k ] = [Ans]
B(α, β)
14.12 Suppose X ∼ beta 0 (α, β).
(a) Find skew[X], the skewness of X. (b) Find κ[X], the kurtosis of X. [Ans]
X 0
14.13 Suppose X ∼ beta(α, β). Show that 1−X ∼ beta (α, β). [Ans]
14.14 Suppose X has the beta prime distribution, beta 0 (α, β).
(a) 1/X ∼ beta 0 (β, α).
(d) Suppose X ∼ gamma(n1 , 1) and Y ∼ gamma(n2 , 1). Suppose further that X and Y are independent. Show that
X/Y ∼ beta 0 (n1 , n2 ).
(e) Suppose X ∼ χ2n1 , Y ∼ χ2n2 and X and Y are independent. Show that X/Y ∼ beta 0 (n1 /2, n2 /2). [Ans]

The arcsine distribution


14.15 Prove equation(13.9a) on page 43. [Ans]
14.16 Shape of the arcsine density. Suppose X ∼ arcsine(a, b) has density function, f .
(a) Show that f is symmetric about a + b/2.
(b) Show that f first decreases and then increases with minimum value at x = a + b/2.
(c) Show that f is convex. [Ans]
14.17 Moments of the arcsine distribution. Suppose X ∼ arcsine(0, 1), the standard arcsine distribution.
(a) Show that  
1 2n
E[X n ] = n for n = 1, 2, . . . .
4 n
In particular, E[X] = 1/2, E[X 2 ] = 3/8 and hence var[X] = 1/8.
(b) Let µ = E[X] = 1/2. Show that E[ (X − µ)n ] = 0 if n is odd and
 
n 1 n
E[ (X − µ) ] = n n if n is even.
4 /2
In particular var[X] = 1/8.
(c) Show that the characteristic function of X is
∞  
itX
X 1 2k
φX (t) = E[e ] = (it)k
4k k! k
k=0

(d) Now suppose X ∼ arcsine(a, b). Find E[X] and var[X]. [Ans]
14.18 Suppose X ∼ arcsine(a, b).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
Page 46 §15 Mar 10, 2020(20:25) Bayesian Time Series Analysis

14.19 (a) Suppose U ∼ uniform(0, 1), a ∈ R and b ∈ (0, ∞). Show that
π 
X = a + b sin2 U ∼ arcsine(a, b)
2
(b) Suppose X ∼ arcsine(a, b). Show that
r !
2 X −a
U = arcsin ∼ uniform(0, 1) [Ans]
π b

14.20 (a) Suppose X ∼ arcsine(−1, 2). Prove that X 2 ∼ arcsine(0, 1).


(b) Suppose X ∼ uniform(−π, 2π). Prove that sin(X), sin(2X) and −cos(2X) all have the arcsine(−1, 2) distribution.
(c) Suppose X ∼ uniform(0, π). Show that Y = cos(X) ∼ arcsine(−1, 2).
(d) Suppose X ∼ uniform(0, π/2). Show that Y = sin2 (X) ∼ arcsine(0, 1). [Ans]
14.21 Suppose X ∼ gamma( 21 , α), Y ∼ gamma( 12 , α) and X and Y are independent. Show that X/(X + Y ) ∼ arcsine(0, 1).
[Ans]
14.22 Suppose X ∼ uniform(−π, 2π), Y ∼ uniform(−π, 2π) and X and Y are independent.
(a) Prove that sin(X + Y ) ∼ arcsine(−1, 2). (b) Prove that sin(X − Y ) ∼ arcsine(−1, 2). [Ans]

15 The normal distribution


15.1 The density function.
Definition(15.1a). Suppose µ ∈ (−∞, ∞) and σ ∈ (0, ∞). Then the random variable X has the normal
distribution N (µ, σ 2 ) if it has density
(x − µ)2
 
1
fX (x) = √ exp − for x ∈ R. (15.1a)
σ 2π 2σ 2
The normal density has the familiar “bell” shape. There are points of inflection at x = µ − σ and x = µ + σ—this
means the f 00 (x) = 0 at these points and the curve changes from convex, when x < µ − σ, to concave and then to
convex again when x > µ + σ. Clearly the mode is at x = µ.

A B

µ−σ µ µ+σ

Figure(15.1a). The graph of the normal density. Points A and B are points of inflection.
(wmf/normaldensity,72mm,54mm)

To check that the function fX defined in equation(15.1a) is a density function:


Clearly fX (x) ≥ 0 for all x ∈ R. Using the substitution t = (x − µ)/σ gives
Z ∞ Z ∞
(x − µ)2
 
1
I= fX (x) dx = √ exp − dx
−∞ −∞ σ 2π 2σ 2
Z ∞ Z ∞ r
1  2  2  2  2
=√ exp − t /2 dt = √ exp − t /2 dt = J
2π −∞ 2π 0 π
where
Z ∞Z ∞ Z π/2 Z ∞
2 1 2 2 π
J = exp[− 2 (x + y )]dy dx = r exp[− 21 r2 ]dr dθ =
0 0 0 0 2
and hence r
π
J=
2
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §15 Page 47

This shows that fX integrates to 1 and hence is a density function.


One reason for the importance of the normal distribution is the central limit theorem—this asserts that the
normalized sum of independent random variables tends to a normal distribution. This theorem is explained in
most probability books; see for example page 258 in [F ELLER(1971)].
15.2 The distribution function, mean and variance. The standard normal distribution is the normal distribu-
tion N (0, 1); its distribution function is
Z x
1
Φ(x) = √ exp[− 12 t2 ] dt
−∞ 2π
This function is widely tabulated. Note that:
• Φ(−x) = 1 − Φ(x). See exercise 16.1 on page 49.
• If X has the N (µ, σ 2 ) distribution, then for −∞ < a < b < ∞ we have
Z b Z (b−µ)/σ
(x − µ)2
 
1 1
exp − t2/2 dt
 
P[a < X ≤ b] = √ exp − 2
dx = √
a σ 2π 2σ 2π (a−µ)/σ
b−µ a−µ
   
=Φ −Φ
σ σ
2
The mean of the N (µ, σ ) distribution:
Z ∞
(x − µ)2
 
1
E[X] = [(x − µ) + µ] √ exp − dx = 0 + µ = µ
−∞ σ 2π 2σ 2
because the function x 7−→ x exp[− x2/2] is odd.
The variance of the N (µ, σ 2 ) distribution: use integration by parts as follows
Z ∞ Z ∞
(x − µ)2 σ2
 
2 1
var[X] = (x − µ) √ exp − 2
dx = √ t2 exp[ −t2/2] dt
−∞ σ 2π 2σ 2π −∞
2 Z ∞ 2 Z ∞
2σ 2σ
=√ t t exp[ −t2/2] dt = √ exp[− t2/2] dt = σ 2
2π 0 2π 0
15.3 The moment generating function and characteristic function. Suppose X ∼ N (µ, σ 2 ) and X = µ+σY .
Then Y ∼ N (0, 1) by using the usual change of variable method. It has already been pointed out in example (1.6c)
on page 5 that the family of distributions {N (µ, σ 2 ) : µ ∈ R, σ > 0} is a location-scale family.
For s ∈ R, the moment generating function of X is given by
Z ∞
1 1 2
sX sµ
MX (s) = E[e ] = e E[e ] = e sσY sµ
esσt √ e− 2 t dt
−∞ 2π
sµ Z ∞  2 sµ Z ∞
t − 2σst (t − σs)2 σ 2 s2
  
e e
=√ exp − dt = √ exp − + dt
2π −∞ 2 2π −∞ 2 2
= exp sµ + 12 σ 2 s2
 
1 2 2
Similarly the characteristic function of X is E[eitX ] = eiµt− 2 σ t .
15.4 Moments of the normal distribution. Moments of a distribution can be obtained by expanding the mo-
ment generating function as a power series: E[X r ] is the coefficient of sr /r! in the expansion of the moment
generating function. It is easy to find the moments
 1 about  the mean of a normal distribution in this way: if
2 sY 2 2
X ∼ N (µ, σ ) and Y = X − µ then E[e ] = exp 2 σ s which can be expanded in a power series of powers
of s. Hence
E (X − µ)2n+1 = E Y 2n+1 = 0 for n = 0, 1, . . .
   

and
 (2n)!σ 2n
E (X − µ)2n = E Y 2n =
  
for n = 0, 1, . . . (15.4a)
2n n!
For example, E[(X − µ)2 ] = σ 2 and E[(X − µ)4 ] = 3σ 4 .
Similarly we can show that (see exercise 16.22 on page 51):
 2n/2 σ n
 
 n n+1
E |X − µ| = √ Γ for n = 0, 1, . . . .
π 2
There are available complicated expressions for E[X n ] and E[|X|n ]; for example, see [W INKELBAUER(2014)].
Page 48 §15 Mar 10, 2020(20:25) Bayesian Time Series Analysis

15.5 Sum of squares of independent N (0, 1) variables.


Proposition(15.5a). Suppose X1 ,. . . , Xn are i.i.d. random variables with the N (0, 1) distribution.
Let Z = X12 + · · · + Xn2 . Then Z ∼ χ2n .
Proof. Consider n = 1. Now X1 has density
1 2
fX1 (x) = √ e−x /2 for x ∈ R.

Then Z = X12 has density
√ dx z −1/2 e−z/2

1 1
fZ (z) = 2fX1 ( z) = 2 √ e−z/2 √ = 1/2 1 for z > 0.
dz 2π 2 z 2 Γ( /2)
Thus Z ∼ χ21 . By §11.9 on page 36, we know that if X ∼ χ2m , Y ∼ χ2n and X and Y are independent, then X+Y ∼ χ2n+m .
Hence Z ∼ χ2n in the general case.

15.6 Linear combination of independent normals.


2
Proposition(15.6a). Pn X1 , X2 , . . . , Xn are independent random variables with Xi ∼ N (µi , σi ) for
Suppose
i = 1, 2, . . . , n. Let T = i=1 bi Xi where bi ∈ R for i = 1, 2, . . . , n. Then
n n
!
X X
2 2
T ∼N bi µi , bi σi
i=1 i=1
Proof. Using moment generating functions gives
n n n  
Y Y Y 1
MT (s) = E[esT ] = E[esbi Xi ] = MXi (sbi ) = exp sbi µi + s2 b2i σi2
2
i=1 i=1 i=1
n n
!
X 1 X 2 2
= exp s bi µi , + s2 bi σ i
2
i=1 i=1
Pn Pn 2 2 
which is the mgf of N i=1 bi µi , i=1 bi σi .
Pn
Corollary(15.6b). If X1 , . . . , Xn are i.i.d. N (µ, σ 2 ), then X = i=1 Xi /n ∼ N (µ, σ 2 /n).
This result implies the normal distribution is stable—see definition(4.1a) on page 11.
d
Note also that if X ∼ N (µ, σ 2 ) and n ∈ {1, 2, . . .}, then X = Y1 + · · · + Yn where Y1 , . . . , Yn are i.i.d. random
variables with the N (µ/n, σ 2 /n) distribution. Hence by definition(5.1a) on page 15, the distribution N (µ, σ 2 ) is
infinitely divisible.

15.7 Independence of two linear combinations of independent normals.


2
Proposition(15.7a). Suppose n are independent random variables with Xi ∼ N (µi , σi ) for
Pn X1 , X2 , . . . , XP n
i = 1, 2, . . . , n. Let V = i=1 bi Xi and W = i=1 ci Xi where bi ∈ R and ci ∈ R for i = 1, 2, . . . , n. Then V
and W are independent iff
Xn
bi ci σi2 = 0 (15.7a)
i=1
Proof. Using moment generating functions gives
n n n  
Y Y Y 1
M(V,W ) (s, t) = E[esV +tW ] = E[e(sbi +tci )Xi ] = MXi (sbi + tci ) = exp (sbi + tci )µi + (sbi + tci )2 σi2
2
i=1 i=1 i=1
n  
Y 1 1
= exp (sbi + tci )µi + s2 b2i σi2 + stbi ci σi2 + s2 c2i σi2
2 2
i=1
n
!
X
= MV (s)MV (t) exp st bi ci σi2
i=1
which proves the proposition.
A particular case. Suppose X and Y are i.i.d. random variables with the normal N (µ, σ 2 ) distribution. Let
V = X + Y and W = X − Y . Using the notation of the proposition we have b1 = 1, b2 = 1, c1 = 1 and c2 = −1.
Hence V and W are independent.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §16 Page 49

15.8 Characterizations of the normal distribution. There are many characterizations of the normal distribu-
tion11 —here are two of the most useful and interesting.
Proposition(15.8a). Cramér’s theorem. Suppose X and Y are independent random variables such that Z =
X + Y has a normal distribution. Then both X and Y have normal distributions—although one may have a
degenerate distribution.
Proof. See, for example, page 298 in [M ORAN(2003)].
Proposition(15.8b). The Skitovich-Darmois theorem. Suppose n ≥ 2 and X1 , . . . , Xn are independent
non-degenerate random variables. Suppose a1 , . . . , an , b1 , . . . , bn are all in R and
L1 = a1 X1 + · · · + an Xn L2 = b1 X1 + · · · + bn Xn
If L1 and L2 are independent, then all random variables Xj with aj bj 6= 0 are normal.
Proof. See, for example, page 89 in [K AGAN et al.(1973)].
A particular case of the Skitovich-Darmois theorem. Suppose X and Y are independent random variables and
V = X + Y and W = X − Y . If V and W are independent then X and Y have normal distributions.
Example(15.8c). Suppose X1 , X2 , . . . , Xn are independent non-degenerate random variables such that V = X and
W = X1 − X are independent. Show that X1 , X2 , . . . , Xn all have normal distributions.
Solution. Now V = a1 X1 + · · · + an Xn with a1 = · · · = an = 1/n and W = b1 X1 + · · · + bn Xn with b1 = 1 − 1/n,
b2 = · · · = bn = −1/n. The result follows by the Skitovich-Darmois theorem.

15.9
Summary. The normal distribution.
• Density. X has the N (µ, σ 2 ) distribution iff it has the density
(x − µ)2
 
1
fX (x) = √ exp − for x ∈ R.
σ 2π 2σ 2
• Moments: E[X] = µ and var[X] = σ 2
• The distribution function: P[X ≤ x] = Φ(x) which is tabulated.
• The moment generating function: MX (t) = E[etX ] = exp[tµ + 21 t2 σ 2 ]
• The characteristic function: φX (t) = E[eitX ] = exp[iµt − 21 σ 2 t2 ]
• A linear combination of independent normals has a normal distribution.
• The sum of squares of n independent N (0, 1) variables has the χ2n distribution.

16 Exercises (exs-normal.tex)

−1 −1
16.1 Show that Φ(−x) = 1 − Φ(x). Hence deduce that Φ (p) + Φ (1 − p) = 0 for p ∈ [0, 1]. [Ans]
2 2
16.2 Suppose X ∼ N (µ, σ ). Suppose further that P[X ≤ 140] = 0.3 and P[X ≤ 200] = 0.6. Find µ and σ . [Ans]
16.3 Suppose X ∼ N (µ, σ 2 ).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
16.4 (a) Show that
Z ∞ Z ∞
1
Φ(−x)ϕ(x) dx = Φ(x)ϕ(x) dx =
−∞ −∞ 2
(b) Suppose a ∈ R and σ > 0. Show that
Z ∞  Z ∞ 
a−x 1 x−a x−a 1 x−a
     
1
Φ ϕ dx = Φ ϕ dx = [Ans]
−∞ σ σ σ −∞ σ σ σ 2
16.5 Suppose a ∈ R, b ∈ R and d ∈ R. Suppose further that c > 0. Prove that
Z ∞
cd − bd − ac
 
1
Φ(d − a − bx)ϕ(cx − d) dx = Φ √
x=−∞ c c 2 + b2
Now suppose c < 0. Prove that
Z ∞
cd − bd − ac ac + bd − cd
   
1 1 1
Φ(d − a − bx)ϕ(cx − d) dx = Φ √ − =− Φ √
x=−∞ c c 2 + b2 c c c2 + b2
If c = 0, the integral diverges. [Ans]
11
For example, see [M ATHAI &P EDERZOLI(1977)] and [PATEL &R EAD(1996)]
Page 50 §16 Mar 10, 2020(20:25) Bayesian Time Series Analysis

16.6 Suppose ϕ denotes the density function of the N (0, 1) distribution.


(a) Prove that, for w > 0 we have
∞  
ϕ(w)
Z
1
1+ 2 ϕ(x) dx =
w x w
2 2 2
(b) Suppose a ≤ b and m = max{a , b }. Prove that
Z b
(b − a)ϕ(m) ≤ ϕ(x) dx ≤ (b − a)ϕ(0) [Ans]
a

16.7 Equality of distributions of linear combinations of independent normals. Suppose X1 , X2 , . . . , Xn are independent
normal random variables with distributions N (µ1 , σ12 ), N (µ2 , σ22 ) . . . , N (µn , σn2 ) respectively. Suppose further that
a1 , . . . , an , b1 , . . . , bn are all in R.
Pn Pn Pn Pn Pn 2 2
(a) Show
Pn 2 2 that k=1 ak X k and k=1 b k Xk have the same distribution iff k=1 ak µ k = k=1 b k µ k and k=1 ak σk =
b
k=1 k k σ .
√ √
(b) Show that (X1 + · · · + Xj )/ j and (X1 + · · · + Xm )/ m have the same distributions. [Ans]
16.8 Tail probabilities of the normal distribution. Suppose ϕ denotes the density function of the N (0, 1) distribution.
(a) Prove that

1 1 xϕ(x) ϕ(x)
− ϕ(x) ≤ ≤ 1 − Φ(x) ≤ for all x > 0.
x x2 1 + x2 x
(b) Prove that 1 − Φ(x) ∼ ϕ(x)/x as x → ∞.
(c) Prove that for any x < y we have
x2
 
ϕ(x) ϕ(y) ϕ(x) ϕ(y)
− ≤ Φ(y) − Φ(x) ≤ −
1 + x2 x y x y
(d) Prove that for any α > 0 we have
ϕ(x) ϕ(x + α)
Φ(x + α) − Φ(x) ∼ − as x → ∞.
x x+α
(e) Prove that for all x > 0 and all j ∈ {1, 2, 3, . . .} we have
1.3.5 . . . (4j − 3)
 
1 1 1.3 1.3.5
ϕ(x) − + − 7 + ··· − ≤ 1 − Φ(x)
x x3 x5 x x4j−1
1.3 . . . (4j − 1)
 
1 1 1.3 1.3.5
≤ ϕ(x) − + − 7 + ··· +
x x3 x5 x x4j+1
(f) Prove that for all k ∈ {1, 2, 3, . . .} we have
k 1.3.5 . . . (2k − 1)
 
1 1 1.3 1.3.5
1 − Φ(x) ∼ ϕ(x) − + − 7 + · · · + (−1) as x → ∞. [Ans]
x x3 x5 x x2k+1
16.9 Suppose Y has the distribution function FY (y) with
1
Φ(y) if y < 0;
FY (y) = 21 1
2 + 2 Φ(y) if y ≥ 0.
n
Find E[Y ] for n = 0, 1, . . . . [Ans]
−Q(x) 2
16.10 Suppose X is a random variable with density fX (x) = ce for all x ∈ R where Q(x) = ax − bx and a 6= 0.
(a) Find any relations that must exist between a, b and c and show that X must have a normal density.
(b) Find the mean and variance of X in terms of a and b. [Ans]
16.11 The entropy of a normal random variable. For the definition of entropy, see exercise 8.6 on page 25. Suppose
X ∼ N (µ, σ 2 ). Find the entropy of X.
(It can be shown that the continuous distribution with mean µ and variance σ 2 with the largest entropy is N (µ, σ 2 ).)
[Ans]
16.12 (a) Suppose X and Y are i.i.d. random variables with the N (0, σ 2 ) distribution. Find the density of Z = X 2 + Y 2 .
(b) Suppose X1 ,. . . , Xn are i.i.d. random variables with the N (0, σ 2 ) distribution. Let Z = X12 + · · · + Xn2 . Find the
distribution of Z. [Ans]
16.13 Suppose X and Y are i.i.d. random variables. Let V = X 2 + Y 2 and W = X/Y . Are V and W independent? [Ans]
16.14 Suppose X ∼ N (µ, σ 2 ). Suppose further that, given X = x, the n random variables Y1 , . . . , Yn are i.i.d. N (x, σ12 ).12
Find the distribution of (X|Y1 , . . . , Yn ). [Ans]

12
This means that f(Y1 ,...,Yn )|X (y1 , . . . , yn |x) = Πni=1 fYi |X (yi |x).
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §16 Page 51

16.15 Suppose X and Y are i.i.d. N (0, 1).


(a) Let Z1 = X + Y and Z2 = X − Y . Show that Z1 and Z2 are independent. Hence deduce that the distribution of
X−Y /X+Y is the same as the distribution of X/Y .
2 2
(b) By using the relation XY = X+Y 2 − X−Y 2 , find the characteristic function of Z = XY .
itXY
R∞ ityX
(c) By using the relation E[e ] = −∞ E[e ]fY (y) dy, find the characteristic function of Z = XY .
(d) Now suppose X and Y are i.i.d. N (0, σ 2 ). Find the c.f. of Z = XY .
(e) Now suppose X and Y are i.i.d. N (µ, σ 2 ). Find the c.f. of Z = XY . [Ans]
16.16 Suppose X1 , X2 , X3 and X4 are i.i.d. N (0, 1). Find the c.f. of X1 X2 + X3 X4 and the c.f. of X1 X2 − X3 X4 . See also
exercise 28.12 on page 84. [Ans]
16.17 (a) Suppose b ∈ (0, ∞). Show that

b2
   r
π −b
Z
1 2
exp − u + 2 du = e (16.17a)
0 2 u 2
(b) Suppose a ∈ R with a 6= 0 and b ∈ R. Show that
Z ∞
b2
    π 1/2
1
exp − a2 u2 + 2 du = 2
e−|ab| (16.17b)
0 2 u 2a
This result is used in exercise 28.28 on page 86. [Ans]
16.18 (a) Suppose X ∼ N (0, 1), Y ∼ N (0, 1) and X and Y are independent. Find the density of X|X + Y = v.
(b) Suppose X ∼ N (µ1 , σ12 ), Y ∼ N (µ2 , σ22 ) and X and Y are independent. Find the density of X|X + Y = v [Ans]
16.19 Suppose X1 , X2 , . . . , Xn are independent random variables with distributions N (µ1 , σ12 ), N (µ2 , σ22 ), . . . , N (µn , σn2 )
respectively. Suppose further that X1 − X is independent of X. Prove that
σ 2 + · · · + σn2
σ12 = 2 [Ans]
n−1
16.20 Suppose X1 , X2 , . . . , Xn are independent random variables with distributions N (µ1 , σ 2 ), N (µ2 , σ 2 ), . . . , N (µn , σ 2 )
respectively.
(a) By using characteristic functions, prove that (X1 − X, X2 − X, . . . , Xn − X) is independent of X.
Pn
(b) Deduce that X is independent of the sample variance j=1 (Xj − X)2 /(n − 1).
(c) Prove that the range Xn:n − X1:n is independent of X. [Ans]
2
16.21 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the normal N (0, σ ) distribution.
(a) Show that the random variables
Xk2
Wk = 2 and X12 + · · · + Xn2
X1 + · · · + Xn2

are independent for any k = 1, 2, . . . , n and show that Wk ∼ beta 1/2, (n−1)/2 .
In particular
X12 
2 2
∼ beta 1/2, 1/2 = arcsine(0, 1)
X1 + X2
(b) Show that idexDDnormal distributionuniform distributionnormal uniform
X12 + X22
∼ uniform(0, 1) [Ans]
X1 + X22 + X32 + X42
2

The folded normal and half-normal distributions. Suppose X ∼ N (µ, σ 2 ). Then |X| has the folded normal
distribution, folded (µ, σ 2 ). The half-normal is the folded (0, σ 2 ) distribution.
16.22 Suppose X ∼ N (µ, σ 2 ). Show that
 2n/2 σ n
 
n+1
E |X − µ|n = √ Γ

for n = 0, 1, . . . .
π 2
This also gives E[|X|n ] for the half-normal distribution. [Ans]
2
16.23 The folded normal distribution. Suppose Y ∼ folded (µ, σ ).
(a) Find the density of Y . (b) Find the distribution function of Y .
(c) Find E[Y ] and var[Y ]. (d) Find the c.f. of Y . [Ans]
d
16.24 (a) Show that folded (µ, σ 2 ) = folded (−µ, σ 2 ).
(b) Suppose X ∼ folded (µ, σ 2 ) and b ∈ (0, ∞). Show that bX ∼ folded (bµ, b2 σ 2 ). [Ans]
Page 52 §17 Mar 10, 2020(20:25) Bayesian Time Series Analysis

16.25 The half-normal distribution, folded (0, σ 2 ). Suppose X ∼ N (0, σ 2 ).


(a) Find fX , the density of X. (b) Show that fX is decreasing and hence has mode at x = 0.
(c) Show that fX is initially concave and then convex with inflection point at x = σ.
(d) Find the distribution function of X. (e) Find E[X] and var[X].
(f) Find the c.f. of X. [Ans]
16.26 Suppose X ∼ folded (0, σ 2 ).
(a) Show that for n = 0, 1, . . .
1
σ 2n (2n)! σ 2n+1 2n+ 2 n!
E[X 2n ] = n
and E[X 2n+1 ] = √
n!2 π
(b) Show that the skewness and kurtosis are

2(4 − π) 3π 2 − 4π − 12
skew[X] = and κ[X] = [Ans]
(π − 2)3/2 (π − 2)2
16.27 (a) Suppose X ∼ folded (0, 1). Show that X ∼ χ1 where the chi distribution is explained on page 110.
(b) Suppose X ∼ folded (0, σ 2 ) and b ∈ (0, ∞). Show that bX ∼ folded (0, b2 σ 2 ). Hence the family of distributions
{ folded ( 0, σ 2 ) : σ ∈ (0, ∞) } is a scale family of distributions—see definition(1.6d) on page 5. [Ans]

17 The lognormal distribution


17.1 The definition.
Definition(17.1a). Suppose µ ∈ R and σ ∈ R; then the random variable X has the lognormal distribution,
logN (µ, σ 2 ), iff ln(X) ∼ N (µ, σ 2 ).
Hence:
• if X ∼ logN (µ, σ 2 ) then ln(X) ∼ N (µ, σ 2 );
• if Z ∼ N (µ, σ 2 ) then eZ ∼ logN (µ, σ 2 ).
d d
It follows that if X ∼ logN (µ, σ 2 ) then ln(X) = µ + σY where Y ∼ N (0, 1). Hence X = eµ (eY )σ . We have
shown the following.
d
If X ∼ logN (µ, σ 2 ) then X = eµ W σ where W ∼ logN (0, 1).

17.2 The density and distribution function. Suppose X ∼ logN (µ, σ 2 ) and let Z = ln(X). Then
ln(x) − µ
 
FX (x) = P[X ≤ x] = P[Z ≤ ln(x)] = Φ
σ
hence the distribution function of the logN (µ, σ 2 ) distribution is
ln x − µ
 
FX (x) = Φ for x > 0.
σ
Differentiating the distribution function gives the density:
ln x − µ (ln x − µ)2
   
1 1
fX (x) = φ =√ exp − for x > 0.
σx σ 2πσx 2σ 2
Z
The density can also be obtained by transforming the normal density
2 dx z dz
 as follows. Now X = e where2 Z ∼
N (µ, σ ). Hence | dz | = e = x; hence fX (x) = fZ (z)| dx | = fZ (ln x) x where fZ is the density of N (µ, σ ).
The shape of the density function fX is considered in exercise 18.1 on page 55.

17.3 Moments. Suppose X ∼ logN (µ, σ 2 ). Then E[X n ] = E[enZ ] = exp nµ + 12 n2 σ 2 for any n ∈ R. In
 
particular
 
1 2
E[X] = exp µ + σ (17.3a)
2
2
 2 
var[X] = E[X 2 ] − {E[X]}2 = e2µ+σ eσ − 1
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §17 Page 53

µ = 0, σ = 0.25
1.5
µ = 0, σ = 0.5
µ = 0, σ = 1.0

1.0

0.5

0.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0

Figure(17.2a). The graph of the lognormal density for (µ = 0, σ = 0.25), (µ = 0, σ = 0.5) and (µ = 0, σ = 1).
In all 3 cases, we have median = 1, mode < 1 and mean > 1—see exercise 18.4 on page 55.
(wmf/lognormaldensity,72mm,54mm)

17.4 The moment generating function. Suppose X ∼ logN (µ, σ 2 ) and consider E[etX ].
Case 1. Suppose t < 0. Now X ≥ 0 and hence tX ≤ 0 and etX ≤ 1. Hence E[etX ] ≤ 1.
Case 2. Suppose t > 0. Without loss of generality, consider the case when µ = 0 and σ = 1. Hence X = eZ where
Z ∼ N (0, 1)and Z ∞
z2
 
tX 1 z
E[e ] = √ exp te − dz
2π z=−∞ 2
2 3 2 2
For z > 0 we have ez > 1 + z + z2 + z6 and hence tez − z2 > t + tz + z6 [3t − 3 + tz] > t + tz for z sufficiently
R∞
large, say z > K. But K exp[t + tz] dz = ∞. Hence for all t > 0 we have E[etX ] = ∞.
We have shown that the m.g.f. only exists when t < 0. There is no simple closed form for its value.
17.5 Other properties.
• Suppose X1 , . . . , Xn are independent random variables with Xi ∼ logN (µi , σi2 ) for i = 1, . . . , n. Then
n n n
!
Y X X
Xi = X1 · · · Xn ∼ logN µi , σi2
i=1 i=1 i=1
• Suppose X1 , . . . , Xn are i.i.d. with the logN (µ, σ 2 ) distribution. Then
σ2
 
1/n
(X1 · · · Xn ) ∼ logN µ,
n
• If X ∼ logN (µ, σ 2 ) , b ∈ R and c > 0 then
cX b ∼ logN ln(c) + bµ, b2 σ 2

(17.5a)
See exercises 18.11 and 18.9 below for the derivations of these results.
17.6 The multiplicative central limit theorem.
Proposition(17.6a). Suppose X1 , . . . , Xn are i.i.d. positive random variables such that
E[ ln(X) ] = µ and var[ ln(X) ] = σ 2
both exist and are finite. Then
 √
X1 · · · Xn 1/ n D

−→ logN (0, σ 2 ) as n → ∞.
enµ
Proof. Let Yi = ln(Xi ) for i = 1, 2, . . . , n. Then
" 1/√n # Pn
X1 · · · Xn (Yi − µ) D
ln = i=1√ −→ N (0, σ 2 ) as n → ∞. 13
enµ n

13
The classical central limit theorem asserts that if X1 , X2 , . . . is a sequence of i.i.d. random variables with finite expectation µ
and finite variance σ 2 and Sn = (X1 + · · · + Xn )/n, then
√ D
n (Sn − µ) −→ N (0, σ 2 ) as n → ∞.
See page 357 in [B ILLINGSLEY(1995)].
Page 54 §17 Mar 10, 2020(20:25) Bayesian Time Series Analysis
D D
Now if Xn −→X as n → ∞ then g(Xn ) −→g(X) as n → ∞ for any continuous function, g. Taking g(x) = ex proves
the proposition.

Using equation(17.5a) shows that if X ∼ logN (0, σ 2 ) then X 1/σ ∼ logN (0, 1). It follows that
"  √1 #
X1 · · · Xn σ n
lim P ≤ x = Φ(x) for all x > 0.
n→∞ enµ
Also, if we let
 √
X1 · · · Xn 1/ n √ √ √

1/ n µ n 1/n µ 1/ n
W = then (X 1 · · · Xn ) = e W and (X 1 · · · Xn ) = e W
enµ
and hence by equation(17.5a), (X1 · · · Xn )1/n is asymptotically logN (µ, σ 2 /n).
We can generalise proposition(17.6a) as follows:
Proposition(17.6b). Suppose X1 , X2 , . . . is a sequence of independent positive random variables such that for
all i = 1, 2, . . .
E[ ln(Xi ) ] = µi , var[ ln(Xi ) ] = σi2 and E |ln(Xi ) − µi |3 = ωi3
all exist and are finite. For n = 1, 2, . . . , let
n
X Xn n
X
2 2 3
µ(n) = µi s(n) = σi ω(n) = ωi3
i=1 i=1 i=1
Suppose further that ω(n) /s(n) → 0 as n → ∞. Then
X1 · · · Xn 1/s(n) D
 
−→ logN (0, 1) as n → ∞.
eµ(n)
Proof. Let Yi = ln(Xi ) for i = 1, 2, . . . , n. Then
1/s(n) Pn
X1 · · · Xn (Yi − µi ) D

ln µ
= i=1 −→N (0, 1) as n → ∞. 14
e (n) s(n)
Using the transformation g(x) = ex proves the proposition.
Also, if we let
1/s(n)
X1 · · · Xn

W = then X1 · · · Xn = eµ(n) W s(n)
eµ(n)
 
and hence by equation(17.5a), the random variable (X1 · · · Xn ) is asymptotically logN µ(n) , s2(n) .
17.7 Usage. The multiplicative central limit theorem suggests the following applications of the lognormal which
can be verified by checking available data.
• Grinding, where a whole is divided into a multiplicity of particles and the particle size is measured by volume,
mass, surface area or length.
• Distribution of farm size (which corresponds to a division of land)—where a 3-parameter lognormal can be
used. The third parameter would be the smallest size entertained.
• The size of many natural phenomena is due to the accumulation of many small percentage changes—leading to
a lognormal distribution.

17.8
Summary. The lognormal distribution.
• X ∼ logN (µ, σ 2 ) iff ln(X) ∼ N (µ, σ 2 ).
2 2
• Moments: if X ∼ logN (µ, σ 2 ) then E[X] = exp µ + 12 σ 2 and var[X] = e2µ+σ ( eσ − 1 )
 
• The product of independent lognormals is lognormal.
• If X ∼ logN (µ, σ 2 ) , b ∈ R and c > 0 then cX b ∼ logN ln(c) + bµ, b2 σ 2

• The multiplicative central limit theorem.
14
Lyapunov central limit theorem with δ = 1. Suppose X1 , X2 , . . . is a sequence of independent random variables such that
E[Xi ] = µi and var[Xi ] = σi2 are both finite. Let sn = σ12 + · · · + σn2 and suppose
n Pn
1 X 3 i=1 (Xi − µi ) D
lim E |Xi − µi | = 0, then −→N (0, 1) as n → ∞.
n→∞ s3n sn
i=1
See page 362 in [B ILLINGSLEY(1995)].
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §18 Page 55

18 Exercises (exs-logN.tex)

2
18.1 Shape of the density function of the lognormal distribution. Suppose X ∼ logN (µ, σ ) with density function fX .
2
(a) Show that fX first increases and then decreases with mode at x = eµ−σ . Show also that fX (x) → 0 as x ↓ 0 and as
x → ∞.
(b) Show that fX is initiallyconvex, then concave andthen convex againwith points of inflectionat
3 1 √ 3 1 √
x1 = exp µ − σ 2 − σ σ 2 + 4 and x2 = exp µ − σ 2 + σ σ 2 + 4 [Ans]
2 2 2 2
18.2 Suppose X ∼ logN (µ, σ 2 ). Let E[X] = α and var[X] = β. Express µ and σ 2 in terms of α and β. [Ans]
18.3 An investor forecasts that the returns on an investment over the next 4 years will be as follows: for each of the first
2 years he estimates that £1 will grow to £(1 + I) where I is a random variable with E[I] = 0.08 and var[I] = 0.001;
for each of the last 2 years he estimates that £1 will grow to £(1 + I) where I is a random variable with E[I] = 0.06 and
var[I] = 0.002.
Suppose he further assumes that the return Ij in year j is independent of the returns in all other years and that 1 + Ij has
a lognormal distribution, for j = 1, 2, . . . , 4.
Calculate the amount of money which must be invested at time t = 0 in order to ensure that there is a 95% chance that
the accumulated value at time t = 4 is at least £5,000. [Ans]
18.4 Suppose X ∼ logN (µ, σ 2 ).
(a) Find the median and mode and show that: mode < median < mean.
(b) Find expressions for the lower and upper quartiles of X in terms of µ and σ.
(c) Suppose αp denotes the p-quartile of X; this means that P[X ≤ αp ] = p. Prove that αp = eµ+σβp where βp is the
p-quartile of the N (0, 1) distribution. [Ans]
18.5 Suppose X ∼ logN (µ, σ 2 ).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
18.6 The geometric mean and geometric variance of a distribution. Suppose each xi P in the data set {x1 ., . . . , P
xn } satisfies
n
xi > 0. Then the geometric mean of the data set is g = (x1 · · · xn )1/n or ln(g) = n1 i=1 ln(xi ) or ln(g) = n1 j fj ln(xj )
where fj is the frequency of the observation xj . This definition motivates the following.
Suppose
R∞ X is a random variable with X > 0. Then GMX , the geometric mean of X is defined by ln(GMX ) =
0
ln(x)f X (x) dx = E[ln(X)].
Similarly, we define the geometric variance, GVX , by
ln(GVX ) = E (ln X − ln GMX )2 = var[ln(X)]
 

and the geometric standard deviation by GSDX = GVX .
Suppose X ∼ logN (µ, σ 2 ). Find GMX and GSDX . [Ans]
18.7 Suppose X ∼ logN (µ, σ 2 ) and k > 0. Show that  
ln(k)−µ−σ 2
1 2
Φ σ
E[X|X < k] = eµ+ 2 σ  
ln(k)−µ
Φ σ
and  
µ+σ 2 −ln(k)
Φ σ
µ+ 12 σ 2
E[X|X ≥ k] = e   [Ans]
ln(k)−µ
1−Φ σ

18.8 Suppose X ∼ logN (µ, σ 2 ). Then the j th moment distribution function of X is defined to be the function G : [0, ∞) →
[0, 1] with Z x
1
G(x) = uj fX (u) du
E[X j ] 0
(a) Show that G is the distribution function of the logN (µ + jσ 2 , σ 2 ) distribution.
(b) Suppose γX denotes the Gini coefficient of X Z (also called the coefficient of mean difference of X). By definition
∞Z ∞
1
γX = |u − v|fX (u)fX (v) dudv
2E[X] 0 0
Hence
E|X − Y |
γX =
2E[X]
where X and Y are independent with the same distribution. Prove that
γX = 2Φ( σ/√2) − 1 [Ans]
Page 56 §19 Mar 10, 2020(20:25) Bayesian Time Series Analysis

18.9 Suppose X ∼ logN (µ, σ 2 ).


(a) Find the distribution of 1/X.
(b) Suppose b ∈ R − {0} and c > 0. Find the distribution of cX b .
In particular, if X ∼ logN (µ, σ 2 ) and c > 0 then cX ∼ logN (µ + ln c, σ 2 ). Hence if σ > 0 and Xµ ∼ logN (µ, σ 2 ),
d
then eλ−µ Xµ = Xλ . Hence the family of distributions {Xµ : µ ∈ R} is a scale family of distributions—see
definition(1.6d) on page 5. [Ans]
18.10 Suppose X1 and X2 are independent random variables with Xi ∼ logN (µi , σi2 ) for i = 1 and i = 2. Find the distribution
of X1 /X2 . [Ans]
18.11 (a) Suppose X1 , .Q. . , Xn are independent random variables with Xi ∼ logN (µi , σi2 ) for i = 1, . . . , n. Find the
n
distribution of i=1 Xi = X1 · · · Xn .
(b) Suppose X1 , . . . , Xn are i.i.d. with the logN (µ, σ 2 ) distribution. Find the distribution of (X1 · · · Xn )1/n .
(c) Suppose X1 , . . . , Xn be independent random variables with Xi ∼ logN (µi , σi2 ) for i = 1, . . . , n. Suppose further
that a1 , . . . , an are real constants. Show that
Yn
Xiai ∼ logN (mn , s2n )
i=1
for some mn and sn and find explicit expressions for mn and sn . [Ans]

19 The power law and Pareto distributions


The power law distribution.
19.1 The power law distribution.
Definition(19.1a). Suppose a0 ∈ R, h ∈ (0, ∞) and α ∈ (0, ∞). Then the random variable X has the power law
distribution, powerlaw(α, h, a0 ), iff X has density
α(x − a0 )α−1
f (x) = for a0 < x < a0 + h. (19.1a)

It is easy to see that if α < 1 then f is monotonic decreasing and if α > 1 then f is monotonic increasing. Also,
if α < 1 or α > 2 then f is convex, and if 1 < α < 2 then f is concave. The density is shown in figure (19.1a)
for three values of α.
4
α = 1/2
α=2
3
α=4

0
0.0 0.2 0.4 0.6 0.8 1.0

Figure(19.1a). The density of powerlaw(α, 1, 0) for α = 1/2, α = 2 and α = 4.


(wmf/powerdensity,72mm,54mm)

The distribution function of powerlaw(α, h, a0 ) is


(x − a0 )α
F (x) = for a0 < x < a0 + h.

The standard power law distribution is powerlaw(α, 1, 0); this has density f (x) = αxα−1 for 0 < x < 1 and
distribution function F (x) = xα for 0 < x < 1. Clearly
d
powerlaw(α, 1, 0) = beta(α, 1)
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §19 Page 57

19.2 Moments and elementary transformations. If X ∼ powerlaw(α, 1, 0) = beta(α, 1), then E[X] = α/(α +
1), E[X 2 ] = α/(α + 2) and var[X] = α/(α + 1)2 (α + 2); Further results are given in exercise 20.1 on page 60.
Clearly,
X ∼ powerlaw(α, h, a0 ) iff (X − a0 )/h ∼ powerlaw(α, 1, 0)
and
if X ∼ powerlaw(α, h, 0) and β ∈ (0, ∞), then βX ∼ powerlaw(α, βh, 0)
It follows that for fixed α ∈ (0, ∞), the family of distributions {powerlaw(α, b, 0) : b ∈ (0, ∞)} is a scale family
of distributions——see definition(1.6d) on page 5.
19.3 A characterization of the power law distribution. Suppose X ∼ powerlaw(α, h, 0); then
αxα−1 xα
f (x) = and F (x) = for x ∈ (0, h).
hα hα
Also, for all c ∈ (0, h) we have
Z c
αxα−1 α c
E[X|X ≤ c] = x α dx = c = E[X]
0 c α+1 h
The next proposition shows this result characterizes the power law distribution (see [DALLAS(1976)]).
Proposition(19.3a). Suppose X is a non-negative absolutely continuous random variable such that there exists
h > 0 with P[X ≤ h] = 1. Suppose further that for all c ∈ (0, h) we have
c
E[X|X ≤ c] = E[X] (19.3a)
h
Then there exists α > 0 such that X ∼ powerlaw(α, h, 0).
Proof. Let f denote the density and F denote the distribution function of X. Then equation(19.3a) becomes
Z c
xf (x) c h
Z
dx = xf (x) dx
0 F (c) h 0
Rh
Let δ = h1 0 xf (x) dx. Then δ ∈ (0, 1) and
Z c
xf (x) dx = cF (c) δ for all c ∈ (0, h). (19.3b)
0
Differentiating with respect to c gives
cf (c) = [F (c) + cf (c)] δ
and hence
F 0 (c) α δ
= where α = > 0.
F (c) c 1−δ
Integrating gives ln F (c) = A + α ln(c) or F (c) = kcα . Using F (h) = 1 gives F (c) = cα /hα for c ∈ (0, h), as required.
The above result leads on to another characterization of the power law distribution:
Proposition(19.3b). Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with a non-negative absolutely con-
tinuous distribution function F and such that there exists h > 0 with F (h) = 1. Then
 
Sn
E X(n) = x = c with c independent of x for all x ∈ (0, h) (19.3c)
X(n)
iff there exists α > 0 such that F (x) = xα /hα for x ∈ (0, h).
Proof. ⇒ Writing Sn = X(1) + · · · + X(n) in equation(19.3c) gives
 
(c − 1)x = E X(1) + · · · + X(n−1) |X(n) = x for all x ∈ (0, h).
It is easy to see that given X(n) = x, then the vector (X(1) , . . . , X(n−1) ) has the same distribution as the vector of n − 1
order statistics (Y(1) , . . . , Y(n−1) ) from the density f (y)/F (x) for 0 < y < x. Hence Y(1) + . . . + Y(n−1) = Y1 + · · · + Yn−1
and
(c − 1)x = (n − 1)E[Y ] where Y has density f (y)/F (x) for y ∈ (0, x). (19.3d)
Hence Z x
c−1
yf (y) dy = xF (x)
0 n−1
Because X(j) < X(n) for all j = 1, 2, . . . , n, equation(19.3c) implies c < n; also equation(19.3d) implies c > 1. Hence
c−1
δ = n−1 ∈ (0, 1). This applies for all x ∈ (0, h) and c is independent of x. So we have equation(19.3b) again and we
must have F (x) = xα /hα for x ∈ (0, h).
⇐ See part (a) of exercise 20.7 on page 61.
Page 58 §19 Mar 10, 2020(20:25) Bayesian Time Series Analysis

The next result is an easy consequence of the last one—it was originally announced in [S RIVASTAVA(1965)] but
the proof here is due to [DALLAS(1976)].

Proposition(19.3c). Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with a non-negative absolutely con-
tinuous distribution function F and such that there exists h > 0 with F (h) = 1. Then
Sn
is independent of max{X1 , . . . , Xn } (19.3e)
max{X1 , . . . , Xn }
iff there exists α > 0 such that F (x) = xα /hα for x ∈ (0, h).
Proof. ⇒ Clearly equation(19.3e) implies equation(19.3c). ⇐ See part (b) of exercise 20.7 on page 61.

The Pareto distribution.

19.4 The Pareto distribution. Suppose α > 0 and x0 > 0. Then the random variable X is said to have a Pareto
distribution iff X has the distribution function
 x α
0
FX (x) = 1 − for x ∈ [x0 , ∞).
x
It follows that X has density
αxα0
fX (x) = for x ∈ [x0 , ∞).
xα+1
Shifting the x-axis by the distance a leads to the general definition:

Definition(19.4a). Suppose α ∈ (0, ∞), x0 ∈ (0, ∞) and a ∈ [0, ∞). Then the random variable X has the
Pareto distribution, Pareto(α, x0 , a), iff X has density
αxα0
fX (x) = for x ∈ [a + x0 , ∞) (19.4a)
(x − a)α+1
It follows that X has distribution function
xα0
FX (x) = 1 − for x ∈ [a + x0 , ∞). (19.4b)
(x − a)α
1
The standard Pareto distribution is Pareto(1, 1, 0). This has density f (x) = x2
for x ∈ [1, ∞) and distribution
function F (x) = 1 − x1 for x ∈ [1, ∞).
By differentiation, we see that the density is monotonic decreasing and convex on (a + x0 , ∞). The shape of the
Pareto density is shown in figure(19.4a).
The Pareto distribution has been used to model the distribution of incomes, the distribution of wealth, the sizes of
human settlements, etc.
3.0
x0 = 1, α = 1
x0 = 1, α = 2
2.5
x0 = 1, α = 3
2.0

1.5

1.0

0.5

0.0
0 1 2 3 4 5

Figure(19.4a). The density of Pareto(α, x0 , 0) for α = 1, α = 2 and α = 3 (all with x0 = 1).


(wmf/Paretodensity,72mm,54mm)
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §19 Page 59

19.5 Elementary transformations of the Pareto distribution. Clearly


X ∼ Pareto(α, x0 , a) iff X − a ∼ Pareto(α, x0 , 0)
and
X ∼ Pareto(α, x0 , a) iff (X − a)/x0 ∼ Pareto(α, 1, 0)
Also
suppose X ∼ Pareto(α, x0 , 0) and b ∈ (0, ∞), then bX ∼ Pareto(α, bx0 , 0) (19.5a)

It follows that for fixed α ∈ (0, ∞), the family of distributions {Pareto(α, b, 0) : b ∈ (0, ∞)} is a scale family
of distributions——see definition(1.6d) on page 5. The parameter α is called the shape parameter and the
parameter x0 is called the scale parameter of the Pareto distribution.

Link between the Pareto and power law distributions. It is important to note that
if X ∼ Pareto(α, h, 0) then 1/X ∼ powerlaw(α, 1/h, 0),
and
if X ∼ powerlaw(α, h, 0) then 1/X ∼ Pareto(α, 1/h, 0).
So a result about one distribution can often be transformed into an equivalent result about the other.

Link between the Pareto and beta distributions. The previous results specialize to
if X ∼ Pareto(α, 1, 0) then 1/X ∼ powerlaw(α, 1, 0) = beta(α, 1),
and
if X ∼ beta(α, 1) = powerlaw(α, 1, 0) then 1/X ∼ Pareto(α, 1, 0).

19.6 Quantile function, median and moments. Using equation(19.4b) gives the quantile function
x0
FX−1 (p) = a + for p ∈ [0, 1).
(1 − p)1/α
Hence the median is FX ( 1/2) = a + 21/α x0 .
Clearly the mode of the distribution Pareto(α, x0 , a) is at a + x0 .

Moments of the Pareto(α, x0 , 0) distribution. Suppose X ∼ Pareto(α, x0 , 0). Then


Z ∞ 
n α n−α−1 ∞ if n ≥ α;
E[X ] = αx0 x dx = n /(α − n) if n ∈ (0, α).
x=x0
αx0

In particular, if X ∼ Pareto(α, x0 , 0), then


αx0 αx20
E[X] = if α > 1, and var[X] = if α > 2.
α−1 (α − 1)2 (α − 2)

19.7 A characterization of the Pareto distribution. Suppose X ∼ Pareto(α, x0 , 0). Suppose further that
α > 1 so that the expectation is finite. We have
αxα0 xα0
f (x) = and F (x) = 1 − for x > x0 .
xα+1 xα
Because the expectation is finite, we have for all c > x0
Z ∞ Z ∞
αxα0 1 αc c
E[X|X > c] = x α+1 dx = αcα α
dx = = E[X]
c x [1 − F (c)] c x α − 1 x 0
The next proposition shows this result characterizes the Pareto distribution (see [DALLAS(1976)]).

Proposition(19.7a). Suppose X is a non-negative absolutely continuous random variable with a finite expec-
tation and such that these exists x0 > 0 with P[X > x0 ] = 1. Suppose further that for all c > x0 we have
c
E[X|X > c] = E[X] (19.7a)
x0
Then there exists α > 1 such that X ∼ Pareto(α, x0 , 0).
Page 60 §20 Mar 10, 2020(20:25) Bayesian Time Series Analysis

Proof. Let f denote the density and F denote the distribution function of X. Then equation(19.7a) becomes
Z ∞ Z ∞
xf (x) c
dx = xf (x) dx (19.7b)
c 1 − F (c) x0 x0
R∞
Let δ = x10 x0 xf (x) dx. We are assuming E[X] is finite; hence δ ∈ (1, ∞). Equation(19.7b) leads to
Z ∞
xf (x) dx = c[1 − F (c)] δ for all c > x0 . (19.7c)
c
Differentiating equation(19.7c) with respect to c gives
−cf (c) = [1 − F (c) − cf (c)] δ
and hence
cf (c)[δ − 1] = [1 − F (c)]δ
F 0 (c) α δ
= where α = > 1.
1 − F (c) c δ−1
Integrating gives − ln[1 − F (c)] = A + ln(cα ) or 1 − F (c) = k/cα . Using F (x0 ) = 0 gives 1 − F (c) = xα α
0 /c for c > x0 ,
as required.
The above result leads on to another characterization of the Pareto distribution:
Proposition(19.7b). Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with a non-negative absolutely con-
tinuous distribution function F with a finite expectation and such that there exists x0 > 0 with P[X > x0 ] = 1.
Then  
Sn
E X(1) = x = c with c independent of x for all x > x0 (19.7d)
X(1)
iff there exists α > 1 such that X ∼ Pareto(α, x0 , 0).
Proof. ⇒ Writing Sn = X(1) + · · · + X(n) in equation(19.7d) gives
 
(c − 1)x = E X(2) + · · · + X(n) |X(1) = x
It is easy to see that given X(1) = x, then the vector (X(2) , . . . , X(n) ) has the same distribution as the vector of n − 1 order
statistics (Y(1) , . . . , Y(n−1) ) from the density f (y)/[1 − F (x)] for y > x. Hence Y(1) + . . . + Y(n−1) = Y1 + · · · + Yn−1 and
(c − 1)x = (n − 1)E[Y ] where Y has density f (y)/[1 − F (x)] for y > x. (19.7e)
Hence Z ∞
c−1
yf (y) dy = x[1 − F (x)]
x n −1
Because X(j) > X(1) for all j = 2, 3, . . . , n, equation(19.7d) implies c > nx c−1
x = n. Hence δ = n−1 ∈ (1, ∞). Recall c is
α α
independent of x; hence we have equation(19.7c) again and we must have F (x) = 1 − x0 /x for x ∈ (x0 , ∞).
⇐ See part (b) of exercise 20.22 on page 62.
The next result is an easy consequence of the last one—it was originally announced in [S RIVASTAVA(1965)] but
the proof here is due to [DALLAS(1976)].
Proposition(19.7c). Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with a non-negative absolutely con-
tinuous distribution function F with finite expectation and such that there exists x0 > 0 with P[X > x0 ] = 1.
Then
Sn
is independent of min{X1 , . . . , Xn } (19.7f )
min{X1 , . . . , Xn }
iff there exists α > 1 such that X ∼ Pareto(α, x0 , 0).
Proof. ⇒ Clearly equation(19.7f ) implies equation(19.7d). ⇐ See part (c) of exercise 20.22 on page 62.

20 Exercises (exs-powerPareto.tex)

The power law distribution.


20.1 Suppose X has the powerlaw(α, h, a) distribution. Find E[X], E[X 2 ] and var[X]. [Ans]
20.2 Suppose X ∼ powerlaw(α, h, a0 ).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
20.3 Transforming the power law distribution to the exponential. Suppose X ∼ powerlaw(α, h, 0). Let Y = − ln(X);
equivalently Y = ln( 1/X ). Show that Y − ln( 1/h) ∼ exponential (α). [Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §20 Page 61

20.4 Suppose U1 , U2 , . . . , Un are i.i.d. random variables with the uniform(0, 1) distribution.
(a) Find the distribution of Mn = max(U1 , . . . , Un ).
1/n
(b) Find the distribution of Y = U1 .
(c) Suppose X ∼ powerlaw(α, h, a). Show that X ∼ a + hU 1/α where U ∼ uniform(0, 1). Hence show that
n  
n
X α n j n−j
E[X ] = h a for n = 1, 2, . . . . [Ans]
α+j j
j=0

20.5 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the powerlaw(α, h, a) distribution. Find the distribution of
Mn = max(X1 , . . . , Xn ). [Ans]
20.6 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the power law distribution powerlaw(α, 1, 0). By using the
2
density of Xk:n , find E[Xk:n ] and E[Xk:n ]. [Ans]
20.7 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the powerlaw(α, h, 0) distribution.
 
Sn
(a) Show that E X(n) = x = c where c is independent of x.
X(n)
Sn
(b) Show that is independent of max{X1 , . . . , Xn }. [Ans]
max{X1 , . . . , Xn }
20.8 Suppose r > 0 and X1 , X2 , . . . , Xn are i.i.d. random variables with a non-negative absolutely continuous distribution
function F such that there exists h > 0 with F (h) = 1.
(a) Show that for some i = 1, 2, . . . , n − 1
" #
r
X(i)
E X(i+1) = x = c with c independent of x for x ∈ (0, h)

r
X(i+1)
iff there exists α > 0 such that F (x) = xα /hα for x ∈ (0, h).
(b) Assuming the expectation is finite, show that for some i = 1, 2, . . . , n − 1
" #
r
X(i+1)
E r X(i+1) = x = c with c independent of x for x ∈ (0, h)

X(i)
iff there exists α > 0 such that F (x) = xα /hα for x ∈ (0, h). [DALLAS(1976)] [Ans]
20.9 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the power law distribution powerlaw(α, 1, 0), which has
distribution function F (x) = xα for 0 < x < 1 where α > 0.
(a) Let
X1:n X2:n X(n−1):n
W1 = , W2 = , . . . , Wn−1 = , Wn = Xn:n
X2:n X3:n Xn:n
Prove that W1 , W2 , . . . , Wn are independent and find the distribution of Wk for k = 1, 2, . . . , n.
2
(b) Hence find E[Xk:n ] and E[Xk:n ]. [Ans]

The Pareto distribution.


20.10 Relationship with the power law distribution. Relationship between the Pareto and uniform distributions.
Recall that if α > 0, then U ∼ uniform(0, 1) iff Y = U 1/α ∼ powerlaw(α, 1, 0) = beta(α, 1).
(a) Suppose α > 0. Show that U ∼ uniform(0, 1) iff Y = U −1/α ∼ Pareto(α, 1, 0).
In particular, if U ∼ uniform(0, 1) then x0 U −1/α ∼ Pareto(α, x0 , 0).
(b) Suppose α > 0 and x0 > 0. Show that Y ∼ Pareto(α, x0 , a) iff Y = a + x0 U −1/α where U ∼ uniform(0, 1).
In particular, if Y ∼ Pareto(α, x0 , a) then xα α
0 /(Y − a) ∼ uniform(0, 1).
(c) Show that X ∼ powerlaw(α, 1, 0) iff 1/X ∼ Pareto(α, 1, 0). [Ans]
20.11 Suppose X ∼ Pareto(α, x0 , a).
(a) Find E[X n ] for n = 1, 2, . . . .
(b) Let µX = E[X]. Find E[(X − µX )n ] for n = 1, 2, . . . . In particular, find an expression for var[X].
(c) Find MX (t) = E[etX ], the moment generating function of X and φX (t) = E[eitX ], the characteristic function of X.
[Ans]
20.12 Suppose X ∼ Pareto(α, x0 , a).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
Page 62 §20 Mar 10, 2020(20:25) Bayesian Time Series Analysis

20.13 Suppose X ∼ Pareto(α, x0 , 0) and d ∈ [x0 , ∞). Show that the conditional distribution of X given X ≥ d is
Pareto(α, d, 0). [Ans]
20.14 Show that the Pareto( 1/2, 1, 0) distribution provides an example of a distribution with E[1/X] finite but E[X] infinite.
[Ans]
20.15 Positive powers of a Pareto distribution. Suppose α ∈ (0, ∞) and x0 ∈ (0, ∞) and X ∼ Pareto(α, x0 , 0). Show that
X n ∼ Pareto(α/n, xn0 , 0) for n ∈ (0, ∞).
It follows that if X ∼ Pareto(1, 1, 0), then X 1/α ∼ Pareto(α, 1, 0) and, by (19.5a), x0 X 1/α ∼ Pareto(α, x0 , 0). [Ans]
20.16 Link between the Pareto and exponential distributions.
(a) Suppose X ∼ Pareto(α, x0 , 0). Let Y = ln(X). Show that Y has a shifted exponential distribution: Y − ln(x0 ) ∼
exponential (α).
(b) Suppose X ∼ exponential (λ). Show that Y = eX ∼ Pareto(λ, 1, 0). [Ans]
20.17 Suppose X ∼ Pareto(α, x0 , 0). Find the geometric mean of X and the Gini coefficient of X. The geometric mean of a
distribution is defined in exercise 18.6 on page 55 and the Gini coefficient is defined in exercise 18.8 on page 55. [Ans]
20.18 Suppose X1 , X2 , . . . , Xn are independent random variables with Xj ∼ Pareto(αj , x0 , a) for j = 1, 2, . . . , n. Find the
distribution of Mn = min(X1 , X2 , . . . , Xn ). [Ans]
20.19 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the Pareto distribution Pareto(α, 1, 0). By using the density of
2
Xk:n , find E[Xk:n ] and E[Xk:n ]. [Ans]
20.20 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the Pareto(α, 1, 0) distribution.
(a) Let
X2:n X(n−1):n Xn:n
W1 = X1:n W2 = , . . . , Wn−1 = , Wn =
X1:n X(n−2):n X(n−1):n
Prove that W1 , W2 , . . . , Wn are independent and find the distribution of Wk for k = 1, 2, . . . , n.
2
(b) Hence find E[Xk:n ] and E[Xk:n ]. See exercise 20.19 for an alternative derivation. [Ans]
20.21 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the powerlaw(α, 1, 0) distribution. Suppose also Y1 , Y2 , . . . , Yn
are i.i.d. random variables with the Pareto(α, 1, 0) distribution. Show that for k = 1, 2, . . . , n
1
Xk:n and have the same distribution. [Ans]
Y(n−k+1):n
20.22 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables with the Pareto(α, x0 , 0) distribution.
(a) Prove that the random variable X1:n is independent of the random vector ( X2:n/X1:n , . . . , Xn:n/X1:n ).
 
Sn
(b) Show that E X(1) = x = c where c is independent of x.
X(1)
(c) Prove that X1:n is independent of Sn/X1:n = (X1 +···+Xn )/X1:n . [Ans]
20.23 Suppose r > 0 and X1 , X2 , . . . , Xn are i.i.d. random variables with a non-negative absolutely continuous distribution
function F with finite expectation and such that there exists x0 > 0 with P[X > x0 ] = 1.
(a) Show that for some i = 1, "2, . . . , n − 1 #
r
X(i+1)
E r X(i) = x = c with c independent of x for x > x0

X(i)
iff there exists α > r/(n − i) such that F (x) = 1 − xα α
0 /x for x > x0 .
(b) Show that for some i = 1, "2, . . . , n − 1 #
r
X(i)
E X(i) = x = c with c independent of x for x > x0

r
X(i+1)
iff there exists α > r/(n − i) such that F (x) = 1 − xα α
0 /x for x > x0 . [DALLAS(1976)] [Ans]
20.24 Suppose X and Y are i.i.d. random variables with the Pareto(α, x0 , 0) distribution. Find the distribution function and
density of Y /X . [Ans]
20.25 Suppose X and Y are i.i.d. random variables with the Pareto(α, x0 , 0) distribution. Let M = min(X, Y ). Prove that M
and Y /X are independent. [Ans]
20.26 A characterization of the Pareto distribution. It is known that if X and Y are i.i.d. random variables with an absolutely
continuous distribution and min(X, Y ) is independent of X − Y , then X and Y have an exponential distribution—see
[C RAWFORD(1966)].
Now suppose X and Y are i.i.d. positive random variables with an absolutely continuous distribution and min(X, Y ) is
independent of Y /X . Prove that X and Y have a Pareto distribution.
Combining this result with exercise 20.25 gives the following characterization theorem: suppose X and Y are i.i.d. pos-
itive random variables with an absolutely continuous distribution; then min(X, Y ) is independent of Y /X if and only if
X and Y have the Pareto distribution. [Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §21 Page 63

20.27 Another characterization of the Pareto distribution. Suppose X1 , X2 , . . . , Xn are i.i.d. absolutely continuous non-
negative random variables with density function f (x) and distribution function F (x). Suppose further that F (1) = 0 and
f (x) > 0 for all x > 1 and 1 ≤ i < j ≤ n. Show that Xj:n/Xi:n is independent of Xi:n if and only if there exists β > 0
such that each Xi has the Pareto(β, 1, 0) distribution.
Using the fact that X ∼ Pareto(α, x0 , 0) iff X/x0 ∼ Pareto(α, 1, 0), it follows that if F (x0 ) = 0 and f (x) > 0 for all
x > x0 where x0 > 0 then Xj:n/Xi:n is independent of Xi:n if and only if there exists β > 0 such that each Xi has the
Pareto(β, x0 , 0) distribution. [Ans]

21 The t, Cauchy and F distributions


The tn distribution

21.1 Definition of the tn distribution.


Definition(21.1a). Suppose n ∈ (0, ∞). Then the random variable T has a t-distribution with n degrees of
freedom iff
X
T =p (21.1a)
Y /n
where X ∼ N(0, 1), Y ∼ χ2n , and X and Y are independent.
It follows that the conditional distribution of T given Y = y is N (0, n/y).

Density: Finding the density is a routine calculation and is left to exercise 22.1 on page 68 where it is shown that
the density of the tn distribution is
−(n+1)/2   −(n+1)/2
t2 Γ (n+1)/2 t2

1
fT (t) = √ 1+ =  √ 1+ for t ∈ R. (21.1b)
B( 1/2, n/2) n n Γ n/2 πn n
We can check that the function fT defined in equation(21.1b) is a density for any n ∈ (0, ∞) as follows. Clearly
fT (t) > 0; also, by using the transformation θ = 1/(1 + t2 /n), it follows that
Z ∞ −(n+1)/2 Z ∞ −(n+1)/2 Z 1
t2 t2 √
1+ dt = 2 1+ dt = n θ(n−2)/2 (1 − θ)−1/2 dθ
−∞ n 0 n 0

= n B(1/2, n/2)
Hence fT is a density.
Now Y in equation(21.1a) can be replaced by Z12 + · · · + Zn2 where Z1 , Z2 , . . . , Zn are i.i.d. with the N (0, 1) distri-
bution. Hence Y /n has variance 1 when n = 1, but its distribution becomes more clustered about the constant 1 as
n becomes larger. Hence T has a larger variance then the normal when n = 1, but tends to the normal as n → ∞.
See exercise 22.7 on page 68 for a mathematical proof of this limiting result.
Figure(21.1a) graphically demonstrates the density of the t-distribution is similar to the shape of the normal den-
sity but has heavier tails.
0.4 t2
t10
normal
0.3

0.2

0.1

0.0
−4 −2 0 2 4

Figure(21.1a). Plot of the t2 , t10 and standard normal densities.


(wmf/tdensity,72mm,54mm)
Page 64 §21 Mar 10, 2020(20:25) Bayesian Time Series Analysis
R∞
21.2 Moments of the tn distribution. Suppose T ∼ tn . Now it is well-known that the integral 1 x1j dx
converges if j > 1 and diverges if j ≤ 1. It follows that
Z ∞
tr
dt converges if r < n.
1 (n + t2 )(n+1)/2
Hence the function tr fT (t) is integrable iff r < n.
√ √
Provided n > 1, E[T ] exists and equals nE[X] E[1/ Y ] = 0.
Provided n > 2, var(T ) = E[T 2 ] = nE[X 2 ] E[1/Y ] = n/(n − 2) by using equation(11.9b) on page 36 which
gives E[1/Y ] = 1/(n − 2).
21.3 Linear transformation of the tn distribution. Suppose m ∈ R, s > 0 and V = m + sT . Then
2 !−(n+1)/2


dt 1 1 v m
fV (v) = fT (t) = √ 1+ (21.3a)
dv B( 1/2, n/2)s n n s
Also E[V ] = m for n > 1 and
n
var(V ) = s2 for n > 2 (21.3b)
n−2
This is called a tn (m, s2 ) distribution.
The Cauchy distribution
21.4 The Cauchy distribution. The Cauchy distribution is basically the t1 distribution.
Definition(21.4a). Suppose a ∈ R and s ∈ (0, ∞). Then the random variable X has the Cauchy distribution,
Cauchy(a, s), iff X has the t1 (a, s2 ) distribution; hence X has the following density
s
γs (x) =  2  for x ∈ R.
π s + (x − a)2
The standard Cauchy distribution, denoted Cauchy(0, 1), is the same as the t1 distribution and has density
1
γ1 (x) = for x ∈ R.
π(1 + x2 )
Clearly if a ∈ R, s ∈ (0, ∞) and X ∼ Cauchy(0, 1) = t1 , then a + sX ∼ Cauchy(a, s) = t1 (a, s2 ).
Shape of the density of the Cauchy(a, s) distribution. See exercise 22.9 and figure(21.4a).
0.4 normal
γ1 = t1
γ2 = t1 (0, 4)
0.3

0.2

0.1

0.0
−4 −2 0 2 4

Figure(21.4a). Plot of the normal, standard Cauchy and the Cauchy(0, 2) = t1 (0, 4) densities.
(wmf/cauchydensity,72mm,54mm)

21.5 Elementary properties of the Cauchy distribution.


• Moments. The expectation, variance and higher moments of the Cauchy distribution are not defined.
• The distribution function of the Cauchy(a, s) distribution. This is
−1 x − a
 
1
where tan−1 (x−a)/s ∈ (0, π).

Fs (x) = tan
π s
This is probably better written as
−1 x − a
 
1 1
where now tan−1 (x−a)/s ∈ (− π/2, π/2).

Fs (x) = + tan (21.5a)
2 π s
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §21 Page 65

• The characteristic function. Suppose the random variable T has the standard Cauchy distribution Cauchy(0, 1).
Then
φT (t) = E[eitT ] = e−|t| (21.5b)
and hence if W ∼ Cauchy(a, s), then W = a + sT and E[e ] = eitW iat−s|t| .
Note. The characteristic function can be derived by using the calculus of residues, or by the following trick. Using integration
by parts gives
Z ∞ Z ∞ Z ∞ Z ∞
−y −y −y
e cos(ty) dy = 1 − t e sin(ty) dy and e sin(ty) dy = t e−y cos(ty) dy
0 0 0 0
and hence Z ∞
1
e−y cos(ty) dy =
0 1 + t2
Now the characteristic function of the bilateral exponential15 density f (x) = 21 e−|x| for x ∈ R is
1 ∞
Z Z ∞
−|y| 1
φ(t) = (cos(ty) + i sin(ty))e dy = e−y cos(ty) dy =
2 −∞ 0 1 + t2
Because this function is absolutely integrable, we can use the inversion theorem to get
Z ∞ −ity Z ∞ ity
1 −|t| 1 e 1 e
e = 2
dy = dy as required.
2 2π −∞ 1 + y 2π −∞ 1 + y 2

The Cauchy distribution is infinitely divisible. If X ∼ Cauchy(a, s) then E[eitX ] = eiat−s|t| ]. Hence if n ∈
{1, 2, . . .}, then X = Y1 + · · · + Yn where Y1 , . . . ,Yn are i.i.d. random variables with the Cauchy(a/n, s/n) dis-
tribution. Hence by definition(5.1a) on page 15, the distribution Cauchy(a, s) is infinitely divisible. Note that
infinite divisibility is a consequence of stability which is proved in exercise 22.14.
Further properties of the Cauchy distribution can be found in exercises 22.9–22.22 starting on page 69.
The F distribution
21.6 Definition of the F distribution.
Definition(21.6a). Suppose m ∈ (0, ∞) and n ∈ (0, ∞). Suppose further that X ∼ χ2m , Y ∼ χ2n and X and Y
are independent. Then
X/m
F = has the Fm,n distribution.
Y /n
Finding the density of the Fm,n distribution is a routine calculation and is left to exercise 22.23 on page 70 where
it is shown that the density of the Fm,n distribution is
Γ( m+n
2 ) m
m/2 nn/2 xm/2−1
fF (x) = for x ∈ (0, ∞). (21.6a)
Γ( m n
2 )Γ( 2 ) [mx + n]
(m+n)/2

Shape of the density function. If m ∈ (0, 2] then fF is decreasing, whilst if m ∈ (2, ∞) then fF first increases and
then decreases with mode at x = (m − 2)n/m(n + 2). See also exercise 22.24 and figure(21.6a).
F10,4 density
0.8 F10,50 density

0.6

0.4

0.2

0.0
0 1 2 3 4 5

Figure(21.6a). Plot of the F10,4 and F10,50 densities.


(wmf/fdensity,72mm,54mm)

15
The bilateral exponential or Laplace distribution is considered in §27.1 on page 81.
Page 66 §21 Mar 10, 2020(20:25) Bayesian Time Series Analysis

21.7 The connection between the t and F distributions. Recall the definition of a tn distribution:
X
Tn = p
Y /n
where X and Y are independent, X ∼ N (0, 1) and Y ∼ χ2n .
Now X 2 ∼ χ21 ; hence
X2
Tn2 = 2 ∼ F1,n (21.7a)
Y /n
It follows that if X ∼ Cauchy(0, 1) = t1 , then X 2 ∼ F1,1 .
Example(21.7a). Using knowledge of the F density and equation(21.7a), find the density of Tn .
Solution. Let W = Tn2 ; hence W ∼ F1,n . Then
1  √ √ 
fW (w) = √ fTn (− w) + fTn ( w)
2 w
But equation(21.7a) clearly implies the distribution of Tn is symmetric about 0; hence for w > 0
−(n+1)/2
√ Γ( n+1 nn/2 w−1 Γ( n+1 w2

1 2 2 ) 2 )
fW (w) = √ fTn ( w) and fTn (w) = wfW (w ) = w 1 =√ 1+
w Γ( 2 )Γ( n2 ) [w2 + n](n+1)/2 nπΓ( n2 ) n
Finally, by symmetry, fTn (−w) = fTn (w).
21.8 Properties of the F distribution. The following properties of the F -distribution are considered in exercises
22.25–22.33 on pages 70–71.
• If X ∼ Fm,n then 1/X ∼ Fn,m .
• If X ∼ Fm,n then E[X] = n/(n − 2) for n > 2 and var[X] = 2n2 (m + n − 2)/[m(n − 2)2 (n − 4)] for n > 4.
See exercise 22.25 on page 70.
• If X1 ∼ gamma(n1 , α1 ), X2 ∼ gamma(n2 , α2 ) and X1 and X2 are independent then
n2 α1 X1
∼ F2n1 ,2n2 (21.8a)
n1 α2 X2
In particular, if X and Y are i.i.d. with the exponential (λ) distribution, then X/Y ∼ F2,2 .
nX
• If X ∼ beta( m/2, n/2) then m(1−X) ∼ Fm,n . See exercise 22.29 on page 71. (21.8b)
mX n
• If X ∼ Fm,n then n+mX ∼ beta( m/2, n/2) and n+mX ∼ beta( n/2, m/2). See exercise 22.30 on page 71. (21.8c)
• If X ∼ Fm,n then mX/n ∼ beta 0 ( m/2, n/2). See exercise 22.31 on page 71. (21.8d)
D
• Suppose X ∼ Fm,n . Then mX −→χ2m as n → ∞. See exercise 22.32 on page 71.
21.9 Fisher’s z distribution.
Definition(21.9a). If X ∼ Fm,n , then
ln(X)
∼ FisherZ (m, n)
2
It follows that if X ∼ FisherZ (n, m) then e2X ∼ Fn,m .
The skew t distribution
21.10 The skew t-distribution. The idea is to split the factor in the tn density
 n     
t t
into 1 − √ × 1 + √
n + t2 n + t2 n + t2
and apply different powers to each factor. This leads to the following definition.
Definition(21.10a). Suppose a > 0 and b > 0. Then the random variable X has the skew t-distribution iff X
has density
 a+1/2  b+1/2
Γ(a + b) x x
fX (x) = √ 1+ √ 1− √ for x ∈ R. (21.10a)
2a+b−1 Γ(a)Γ(b) a + b a + b + x2 a + b + x2
Note:
• This distribution is denoted skewt(a, b).
• Using the standard algebraic result (1 + y)(1 − y) = 1 − y 2 shows that an alternative expression is
 a  b
Γ(a + b) x x
fX (x) = √ 1+ √ 1− √ for x ∈ R.
2a+b−1 Γ(a)Γ(b) a + b + x2 a + b + x2 a + b + x2
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §21 Page 67

• To prove fX (x) in equation(21.10a) integrates to 1 we use the transformation


x dv a+b
v=√ which implies =
a + b + x2 dx (a + b + x2 )3/2
dv
Note that dx > 0 for all x and hence the transformation is a 1-1 map: (−∞, ∞) −→ (−1, 1). Hence
Z ∞  a  b
1 x x
√ 1+ √ 1− √ dx
−∞ a + b + x2 a + b + x2 a + b + x2
Z ∞  a−1  b−1
a+b x x
= 2 3/2
1+ √ 1− √ dx
−∞ (a + b + x ) a + b + x2 a + b + x2
Z 1 Z 1
a−1 b−1 Γ(a)Γ(b)
= (1 + v) (1 − v) dv = 2a+b−1 wa−1 (1 − w)b−1 dw = 2a+b−1
−1 0 Γ(a + b)
where w = (1 + v)/2. This proves that fX (x) in equation(21.10a) is a density.
21.11 Properties of the skew t-distribution.
• Suppose X ∼ skewt(a, b) and Y ∼ skewt(b, a). Then fY (x) = fX (−x) for all x ∈ R.
• Suppose a ∈ (0, ∞). The skewt(a, a) distribution is the same as the t2a distribution. (Exercise 22.34.)
The shape of the density is displayed in figure (21.11a).
0.4 0.4

a = 3, b = 3 a = 3, b = 3
0.3 0.3
a = 4, b = 2
a = 2, b = 4
0.2 0.2

a = 1, b = 5 a = 5, b = 1
0.1 0.1

0.0 0.0
−15 −10 −5 0 5 10 15 −15 −10 −5 0 5 10 15
Figure(21.11a). Plot of the skew t-density for various a and b with a + b = 6.
Note that a = b = 3 is the t6 density.
(wmf/skewtDensity-1,wmf/skewtDensity-2,80mm,60mm)

21.12
Summary.
• The tn distribution. The random variable T ∼ tn iff
X
T =p
Y /n
where X ∼ N (0, 1), Y ∼ χ2n and X and Y are independent.
• Moments:  
n 1 1
E[T ] = 0 var[T ] = for n > 2. E =
n−2 T n−2
• Suppose T ∼ tn , m ∈ R and s > 0. Then V = m + sT ∼ tn (m, s2 ).
The Cauchy distribution. This has density
1
γ1 (t) = for t ∈ R.
π(1 + t2 )
It is the t1 distribution. The Cauchy(a, s) distribution is the same as the t1 (a, s2 ) distribution.
The F distribution. Suppose m > 0 and n > 0. Suppose further that X ∼ χ2m , Y ∼ χ2n and X and Y are
independent. Then
X/m
F = has an Fm,n distribution.
Y /n
• If X ∼ tn then X 2 ∼ F1,n .
Page 68 §22 Mar 10, 2020(20:25) Bayesian Time Series Analysis

22 Exercises (exs-tCauchyF.tex)

The t distribution.
22.1 (a) Using the definition of the tn distribution given in definition(21.1a) on page 63, show that the density of the tn dis-
tribution is given by equation(21.1b).
(b) Using the fact that the conditional distribution of T given Y = y is N (0, n/y), show that the density of the tn distri-
bution is given by equation(21.1b). [Ans]
22.2 Shape of the tn density function. Suppose X ∼ tn has the density fX .
(a) Show that fX (x) is symmetric about x = 0.
(b) Show that fX is initially increasing and then decreasing with mode at x = 0. Also fX (x) → 0 as x → ±∞.
p
√ fX is initially convex, then concave and then convex again with points of inflection at x = − n/(n + 2)
(c) Show that
and x = n(n + 2). [Ans]
22.3 Moments of the tn distribution. Suppose T ∼ tn and k ∈ {1, 2, 3, . . . , n − 1}. Clearly E[T k ] = 0 when k is odd. If k is
even, and hence k = 2r where r ∈ {1, 2, 3, . . .} and r ≤ (n − 1)/2, then prove that
k+1 n−k
 
k k/2 Γ 2 Γ (k − 1)(k − 3) · · · 3.1
E[T ] = n 1 n
2
= nk/2
Γ 2 Γ 2 (n − 2)(n − 4) · · · (n − k + 2)(n − k)
or, in terms of r:
1 n
 
rΓ r+ 2  Γ 2 − r (2r − 1)(2r − 3) · · · 3.1
2r
E[T ] = n 1 n
= nr
Γ 2 Γ 2
(n − 2)(n − 4) · · · (n − 2r + 2)(n − 2r)
If k ≥ n then 
∞ if k is even, k ≥ n;
E[T k ] = [Ans]
undefined if k is odd, k ≥ n.
22.4 Suppose X ∼ tn .
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
2
22.5 Suppose X, Y1 , . . . , Yn are i.i.d. random variables with the N (0, σ ) distribution. Find the distribution of
X
Z=q [Ans]
(Y12 + · · · + Yn2 )/n
22.6 Suppose n > 0, s > 0 and α ∈ R. Show that
Z ∞ n/2
1 n−1
 
1
dt = sB , [Ans]
−∞ 1 + (t − α)2 /s2 2 2
22.7 (a) Prove that the limit as n → ∞ of the tn density given in equation(21.1b) is the standard normal density.
a.e.
(b) Suppose Tn ∼ tn for n ∈ {1, 2, . . .}. Show that Tn −→Z as n → ∞. [Ans]
22.8 The t2 distribution.
(a) Show that the density of the t2 distribution is
1
f (x) = for x ∈ R.
(2 + x2 )3/2
(b) Show that the distribution function of the t2 distribution
 is 
1 x
F (x) = 1+ √ for x ∈ R.
2 2 + x2
(c) Show that the quantile function Q(u) = F −1 (u) is
2u − 1
Q(u) = √ for u ∈ (0, 1).
2u(1 − u)
(d) Moments. E[X] exists and equals 0; E[X n ] does not exist for n > 1.

(e) Measures p of variability. Suppose T ∼ t2 . Show that the mean absolute deviation E|T | = 2 and the interquartile
range is 2 2/3 = 1.633.
(f) Suppose X ∼ N (0, 1) and Y ∼ exponential (1) and X and Y are independent. Show that
X
√ ∼ t2
Y
(g) Suppose X and Y are i.i.d. random variables with the exponential (1) distribution. Show that
X X −Y
∼ F2,2 and hence √ ∼ t2 [Ans]
Y 2XY
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §22 Page 69

The Cauchy distribution.


22.9 Shape of the Cauchy density. Suppose γs denotes the density of the distribution Cauchy(a, s).
(a) Show that γs (x) is symmetric about x = a.
(b) Show that γs (x) first increases and then decreases with mode at x = a.

γs (x) is initially convex, then concave and then convex with points of inflection at x = a − s/ 3 and
(c) Show that √
x = a + s/ 3. [Ans]
22.10 The quantile function and median of the Cauchy distribution. Suppose a ∈ R, s ∈ (0, ∞) and X ∼ Cauchy(a, s). Show
that the quantile function of X is
Fs−1 (p) = a + s tan[π(p − 1/2)] for p ∈ (0, 1).
and the median of X is a. [Ans]
22.11 Linear transformation of a Cauchy distribution. Suppose X ∼ Cauchy(a, s), c ∈ R and d ∈ (0, ∞). Show that
Y = c + dX ∼ Cauchy(c + da, ds).
It follows that the family of Cauchy distributions {Cauchy(a, s) : a ∈ R, s > 0} form a location-scale family—see
definition(1.6b) on page 5. [Ans]
22.12 Sums of independent Cauchy distributions.
(a) Suppose X1 ∼ Cauchy(a1 , s1 ) and X2 ∼ Cauchy(a2 , s2 ). Suppose further that X1 and X2 are independent. Show
that X1 + X2 ∼ Cauchy(a1 + a2 , s1 + s2 ).
(b) Suppose X1 , . . . , Xn are i.i.d. random variables with the Cauchy(a, s) distribution. Show that X1 + · · · + Xn ∼
Cauchy(na, ns). [Ans]
22.13 Suppose X1 , . . . , Xn are i.i.d. with density γs .
X1 +···+Xn
(a) Show that Y = n also has the Cauchy(0, s) distribution.
(b) Let Mn = median(X1 , . . . , Xn ). Show that Mn is asymptotically normal with mean 0 and a variance which tends
to 0. [Ans]
22.14 The Cauchy distribution is stable. Suppose X ∼ Cauchy(a, s) where s > 0.
(a) Prove that X has a strictly stable distribution with characteristic exponent α = 1.
(b) Using the notation in equation(4.3b), show that X is stable with characteristic function {c = a, d = s, β = 0}. [Ans]
22.15 (a) Suppose X ∼ Cauchy(0, 1). Show that Y = 1/X ∼ Cauchy(0, 1).
(b) Suppose X ∼ Cauchy(0, s). Show that Y = 1/X ∼ Cauchy(0, 1/s).
(c) Suppose X has a non-central Cauchy(m, s) distribution with median m. Hence
s
fX (x) = for x ∈ R.
π[s2 + (x − m)2 ]
Find the density of Y = 1/X . [Ans]
22.16 Suppose X and Y are i.i.d. with the N (0, σ 2 ) distribution. Find the distribution of:
(a) W = X/Y ; (b) W = X/|Y |; (c) W = |X|/|Y |. [Ans]
22.17 (a) Suppose U has the uniform distribution uniform(− π/2, π/2). Show that tan(U ) ∼ Cauchy(0, 1).
(b) Suppose U has the uniform distribution uniform(−π, π). Show that tan(U ) ∼ Cauchy(0, 1).
(c) Suppose a ∈ R and s ∈ (0, ∞). Suppose further that U ∼ uniform(0, 1). Show that a + s tan[π(U − 1/2)] ∼
Cauchy(a, s). Conversely, if X ∼ Cauchy(a, s) then 12 + π1 tan−1 ( X−a
s ) ∼ uniform(0, 1). [Ans]
22.18 (a) Suppose X ∼ Cauchy(0, s). Find the density of 2X. (This shows that 2X has the same distribution as X1 + X2
where X1 and X2 are i.i.d. with the same distribution as X.)
(b) Supppose U and V are i.i.d. with the Cauchy(0, s) distribution. Let X = aU + bV and Y = cU + dV . Find the
distribution of X + Y . [Ans]
22.19 Suppose X and Y are i.i.d. with the N (0, 1) distribution. Define R and Θ by R2 = X 2 + Y 2 and tan(Θ) = Y /X where
R > 0 and Θ ∈ (−π, π). Show that R2 has the χ22 distribution, tan(Θ) has the Cauchy(0, 1) distribution, and R and Θ
2
are independent. Show also that the density of R is re−r /2 for r > 0. [Ans]
22.20 Suppose X has the Cauchy(0, 1) distribution.
2X
 
(a) Find the density of 1−X 2
. Hint: tan(2θ) = 2 tan(θ) 1 − tan2 (θ) .
1
X − X1 .

(b) Find the density of V = 2 [Ans]
Page 70 §22 Mar 10, 2020(20:25) Bayesian Time Series Analysis

22.21 From a point O, radioactive particles are directed at an absorbing line which is at a distance b from O. Suppose OP
denotes the perpendicular from the point O to the absorbing line—and hence the length of OP is b. The direction of
emission is measured by the angle Θ from the straight line OP . Suppose Θ is equally likely to be any direction in
(− π/2, π/2). Formally, Θ ∼ uniform(− π/2, π/2).
(a) Determine the density of X, the distance from P where the particle hits the absorbing line.
(b) What is the density of 1/X ? [Ans]
2 2
22.22 The symmetric Cauchy distribution in R . Define the function f : R → (0, ∞) by
1
f (x, y) =
2π(1 + x2 + y 2 )3/2
(a) Show that f is a density function.
(b) Find the marginal densities.
(c) Suppose (X, Y ) has the density f and we transform to polar coordinates: X = R cos Θ and Y = R sin Θ. Show that
R and Θ are independent and find the distributions of R and Θ.
The last question can be generalized to produce this density—in this case, the direction must be uniform over the surface
of a hemisphere. [Ans]
The F distribution.
22.23 (a) By using a bivariate transformation and definition(21.6a) on page 65, show that the density of the Fm,n distribution
is given by equation(21.6a) on page 65.
(b) Using definition(21.6a) on page 65, we see that the distribution of F given Y = y is
 
n
X where X ∼ χ2m = gamma( m/2, 1/2).
my
By §11.3 on page 34, we see that the distribution of F given Y = y is gamma( m/2, my/2n). Hence derive the density
of the Fm,n distribution given in equation(21.6a) on page 65. [Ans]
22.24 Shape of the F density. Suppose m ∈ (0, ∞), n ∈ (0, ∞), and X ∼ Fm,n with density f .
(a) Suppose m ∈ (0, 2). Show that f is decreasing on (0, ∞) with f (x) → ∞ as x → 0.
(b) Suppose m = 2. Show that f is decreasing with mode at x = 0 with f (0) = 1 − 2/n.
(c) Suppose m ∈ (2, ∞). Show that f first increases and then decreases with mode at x = (m − 2)n/m(n + 2).
(d) Suppose m ∈ (0, 2]; show that f is convex. Suppose m ∈ (2, 4]; show that f is initially concave and then convex
with point of inflection at

n(m − 2) n 2(m − 2)(n + 4)(m + n)
β= +
m(n + 2) m (n + 2)(n + 4)
Suppose m ∈ (4, ∞); show that f is initially convex, then concave and then convex again with points of inflection
at √
n(m − 2) n 2(m − 2)(n + 4)(m + n)
α= − and at β.
m(n + 2) m (n + 2)(n + 4)
Note that α > 0 when m > 4. [Ans]
22.25 Moments of the F distribution.Suppose F has the Fm,n distribution.
(a) Show E[F ] = ∞ for n ∈ (0, 2] and
n
E[F ] = for n > 2.
n−2
(b) Show that var[F ] is undefined for n ∈ (0, 2]; var[F ] = ∞ for n ∈ (2, 4] and
2n2 (m + n − 2)
var[F ] = for n > 4.
m(n − 2)2 (n − 4)
(c) Suppose r ∈ {1, 2, . . .}; show that
 n r m(m + 2) · · · (m + 2r − 2)
E[F r ] = for n > 2r. [Ans]
m (n − 2)(n − 4) · · · (n − 2r)
22.26 Suppose X ∼ F (m, n).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
2
22.27 Suppose X and Y are i.i.d. N (0, σ ). Find the density of Z where

2 2
Z = Y /X if X 6= 0; [Ans]
0 if X = 0.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §23 Page 71

22.28 (a) Suppose X ∼ Fm,n . Show that 1/X ∼ Fn,m .


(b) Suppose X1 ∼ gamma(n1 , α1 ), X2 ∼ gamma(n2 , α2 ) and X1 and X2 are independent. Show that
n2 α1 X1
∼ F2n1 ,2n2
n1 α2 X2
In particular, if X1 ∼ exponential (λ), X2 ∼ exponential (λ) and X1 and X2 are independent, then
X1
∼ F2,2 [Ans]
X2
nX
22.29 Suppose X ∼ beta( m/2, n/2). Show that m(1−X) ∼ Fm,n . [Ans]
mX n
22.30 (a) Suppose X ∼ Fm,n . Show that n+mX ∼ beta( m/2, n/2) and n+mX ∼ beta( n/2, m/2).
(b) Suppose X ∼ F2α,2β where α > 0 and β > 0. Show that αX/β ∼ beta 0 (α, β). [Ans]
22.31 Suppose W ∼ Fm,n . Show that mW/n ∼ beta 0 ( m/2, n/2). 0
Conversely, if Y ∼ beta (a, b) then bY /a ∼ F2a,2b . [Ans]
D 2
22.32 Suppose W ∼ Fm,n . Show that mW −→χm as n → ∞. [Ans]
22.33 (a) Suppose X ∼ Fn,n . Show that
√ 
n √

1
X−√ ∼ tn
2 X
(b) Suppose X ∼ χ2n , Y ∼ χ2n and X and Y are independent. Let

nX −Y
T = √
2 XY
Prove that T ∼ tn . [Ans]
The skew t distribution.
22.34 Suppose a ∈ (0, ∞). Show that the skewt(a, a) distribution is the same as the t2a distribution. [Ans]
22.35 (a) Suppose X ∼ F2m,2n and W = mX/n. Show that

m+n √
 
1
W− √ ∼ skewt(m, n)
2 W
(b) Suppose Y ∼ χ22m , Z ∼ χ22n and Y and Z are independent. Show that

m + n (Y − Z)
√ ∼ skewt(m, n)
2 YZ
(c) Suppose B ∼ beta(m, n). Show that

m + n (2B − 1)
√ ∼ skewt(m, n) [Ans]
2 B(1 − B)
22.36 Suppose X ∼ skewt(m, n). Show that
(m + n)1/2 (m − n) Γ(m − 1/2) Γ(n − 1/2) (m + n) (m − n)2 + m + n − 2
 
E[X] = and E[X 2 ] =
2 Γ(m) Γ(n) 4 (m − 1)(n − 1)
where the result for E[X] holds for m > 1/2 and n > 1/2 and the result for E[X 2 ] holds for m > 1 and n > 1. [Ans]
22.37 Suppose X ∼ skewt(m, n). Show that the density of X is unimodal with mode at

(m − n) m + n
√ √ [Ans]
2m + 1 2n + 1

23 Non-central chi-squared, t and F


23.1 The non-central χ2 -distribution with 1 degree of freedom. We know that if Z ∼ N (0, 1), then Z 2 ∼ χ21 .
Now suppose
W = (Z + a)2 where Z ∼ N (0, 1) and a ∈ R.
Then W is said to have a non-central χ21 distribution with non-centrality parameter a2 .
We can also write W ∼ Y 2 where Y ∼ N (a, 1).
The moment generating function of W .
Z ∞
t(Z+a)2 1 2 1 2
tW
E[e ] = E[e ]= √ et(z+a) e− 2 z dz
2π −∞
But
Page 72 §23 Mar 10, 2020(20:25) Bayesian Time Series Analysis

t(z + a)2 − 1/2 z 2 = z 2 t + 2azt + a2 t − 1/2 z 2 = z 2 (t − 1/2) + 2azt + a2 t


" 2 #
2t − 2t 2 t2
 
2 4azt 2a 1 2t 2at 2a 4a
= (t − 1/2) z − − =− z− − −
1 − 2t 1 − 2t 2 1 − 2t 1 − 2t (1 − 2t)2
" #
1 − 2t 2at 2 2a2 t

=− z− −
2 1 − 2t (1 − 2t)2
and hence, if α = 2at/(1 − 2t) and t < 1/2,
 2  Z ∞
(1 − 2t)(z − α)2
 
tW t(Z+a)2 a t 1
E[e ] = E[e ] = exp √ exp − dz
1 − 2t 2π −∞ 2
 2 
−1/2 a t
= (1 − 2t) exp (23.1a)
1 − 2t
The density of W . Using the usual transformation formula for densities shows that for w > 0 we have
√ √ √ √
φ( w − a) + φ(− w − a) φ( w − a) + φ( w + a)
fW (w) = √ = √
2 w 2 w
1 √ √ 
exp − 1/2(w + a2 ) exp(a w) + exp(−a w)

= √ (23.1b)
2 2πw
1 √
exp − 1/2(w + a2 ) cosh(a w) because cosh(x) = (ex + e−x )/2 for all x ∈ R.

=√
2πw

The standard expansion for cosh is cosh(x) = ∞ 2j n
P
j=0 x /(2j)! for all x ∈ R; also Γ(n + /2) = (2n)! π/(4 n!) for
1

all n = 0, 1, 2, . . . ; hence
∞ ∞
1 2
X (a2 w)j 1 2
X (a2 w/4)j
fW (w) = √ exp − /2(w + a )
1 √ =√ exp − /2(w + a )
1
2w π(2j)! 2w j!Γ(j + 1/2)
j=0 j=0
 √ 1/2
1 √ a w
exp − 1/2(w + a2 ) I−1/2 (a w)

=√
2w 2
 2 1/4
1  a √
= exp − 1/2(w + a2 ) I−1/2 (a w) (23.1c)
2 w
where, for all x > 0,
∞  x 2j− 1/2
X 1
I−1/2 (x) =
j!Γ(j + 1/2) 2
j=0
is a modified Bessel function of the first kind.
Note. The general definition of a modified Bessel function of the first kind is

 x ν X x2j
Iν (x) = for all ν ∈ R and x ∈ C. (23.1d)
2 4j j!Γ(ν + j + 1)
j=0

23.2 The non-central χ2 -distribution with n degrees of freedom where n ∈ {1, 2, .P . .}.
Suppose X1 ∼ N (µ1 , 1), X2 ∼ N (µ2 , 1), . . . , Xn ∼ N (µn , 1) are independent. Then nj=1 (Xj − µj )2 ∼ χ2n but
Pn 2 2
j=1 Xj does not have a χ distribution. We say
Xn
Xj2 has a non-central χ2n distribution with non-centrality parameter λ = nj=1 µ2j .
P
W = (23.2a)
j=1
This can be written as: suppose X ∼ N (µ, In ) then XT X ∼ χ2n,µT µ . In particular, if X ∼ N (µ, σ 2 ) then
X 2 /σ 2 ∼ χ21,µ2 /σ2 , the non-central χ21 distribution with non-centrality parameter µ2 /σ 2 .
Note that some authors define the non-centrality parameter to be λ/2.
Moments. See exercise 24.1 on page 76 for the following moments:
E[W ] = n + λ and var[W ] = 2n + 4λ
2 2
Pn 2 χn distribution with non-centrality parameter λ and W2 ∼ χn , then
So we see that if W1 has a non-central
E[W1 ] ≥ E[W2 ] because λ = j=1 µj ≥ 0.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §23 Page 73

The moment generating function of W . By equation(23.1a) the moment generating function of X12 is
 2 
tX12 1 µ1 t
E[e ] = exp
(1 − 2t)1/2 1 − 2t
Hence  
1 λt
E[etW ] = exp for t < 1/2. (23.2b)
(1 − 2t)n/2 1 − 2t
Distribution of S 2 . Suppose X1 ∼ N (µ1 , σ 2 ), X2 ∼ N (µ2 , σ 2 ), . . . , Xn ∼ N (µn , σ 2 ) are independent. Then
Pn 2
Pn 2
(n − 1)S 2 k=1 (Xk − X) 2 k=1 (µk − µ)
= ∼ χ n−1,λ where λ =
σ2 σ2 σ2
For the proof see example(42.13b) on page 137.
23.3 The non-central χ2 -distribution with n degrees of freedom—the basic decomposition theorem. The
easiest proof is by using moment generating functions and equation(23.2b). Here is a proof from first principles:
Proposition(23.3a). Suppose n ∈ {1, 2, . . .} and W has a non-central χ2n distribution with non-centrality
parameter λ > 0. Then W has the same distribution as U + V where:
U has a non-central χ21 distribution with non-centrality parameter λ;
V has a χ2n−1 distribution;
U and V arepindependent.
n
Proof. Let µj = λ/n for j = 1, . . . , n; hence j=1 µ2j = λ.
P
Pn
We are given that W has a non-central χ2n distribution with non-centrality parameter λ > 0. Hence W ∼ j=1 Xj2 where
X1 , . . . , Xn are independent with Xj ∼ N (µj , 1) for j = 1, . . . , n.
√ √
Let e1 = (1, 0, . . . , 0), . . . , en = (0, . . . , 0, 1) denote the standard basis of Rn . Set b1 = (µ1 / λ, . . . , µn / λ). Then
{b1 , e2 , . . . , en } form a basis of Rn . Use the Gram-Schmidt orthogonalization procedure to create the basis {b1 , . . . , bn }.
Define B to be the n × n matrix with rows {b1 , . . . , bn }; then B is orthogonal.

Suppose X = (X1 , . . . , Xn ) and set Y = BX. Then Y ∼ N (Bµ, BIBT = I) where µ = (µ1 , . . . , µn ) = λb1 . Hence
n×1 n×1

Y1 ∼ N (bT1 µ = λ, 1) and Yj ∼ N (bTj µ = 0, 1) for j = 2, . . . , n and Y1 , . . . , Yn are independent. Also YT Y = XT X.
Pn
Finally, let U = Y12 and V = j=2 Yj2 .

23.4 The non-central χ2 -distribution with n degrees of freedom—the density function. We use proposi-
tion(23.3a). Now U has a non-central χ21 distribution with non-centrality parameter λ. Using equation(23.1b)
gives
1 1
h √ √ i
fU (u) = 3/2 1 √ e− 2 (u+λ) e λu + e− λu for u > 0.
2 Γ( /2) u
Also, V ∼ χ2n−1 has density
e−v/2 v (n−3)/2
fV (v) = for v > 0.
2(n−1)/2 Γ( (n−1)/2)
Using independence of U and V gives
u−1/2 v (n−3)/2 e−(u+v)/2 e−λ/2 h √λu √ i
− λu
f(U,V ) (u, v) = e + e
2(n+2)/2 Γ( 1/2)Γ( (n−1)/2)
Now use the transformation X = U + V and Y = V . The Jacobian equals 1. Hence for y > 0 and x > y
1/2 " √λ(x−y) √ #
e−x/2 e−λ/2 x(n−4)/2  y (n−3)/2 − e− λ(x−y)

x e
f(X,Y ) (x, y) = n/2 1
2 Γ( /2)Γ( (n−1)/2) x x−y 2
Now
1/2 " √λ(x−y) √ #  ∞
1/2 X
− e− λ(x−y) λj (x − y)j

x e x
=
x−y 2 x−y (2j)!
j=0

X (λx)j  y j−1/2
= 1−
(2j)! x
j=0
and so we have
Page 74 §23 Mar 10, 2020(20:25) Bayesian Time Series Analysis


e−x/2 e−λ/2 x(n−4)/2 X (λx)j  y (n−3)/2  y j−1/2
f(X,Y ) (x, y) = n/2 1 1 − for y > 0 and x > y.
2 Γ( /2)Γ( (n−1)/2) j=0 (2j)! x x

We need to integrate out y. By setting w = y/x we get


Z x  (n−3)/2  Z 1
y y j−1/2
1− dy = x w(n−3)/2 (1 − w)j−1/2 dw
y=0 x x w=0
= xB( (n−1)/2, j + 1/2))
Γ( (n−1)/2)Γ(j + 1/2))
=x
Γ( n/2 + j)
and hence for x > 0

e−x/2 e−λ/2 x(n−2)/2 X (λx)j Γ(j + 1/2)) Γ( n/2)
fX (x) =
2n/2 Γ( n/2) j=0
(2j)! Γ( 1/2) Γ( n/2 + j)

e−x/2 e−λ/2 x(n−2)/2 X (λx)j
= (23.4a)
2n/2 4j j!Γ( n/2 + j)
j=0

The expression for the modified Bessel function of the first kind in equation(23.1d) on page 72 gives
√ ∞
(λx)(n−2)/4 X (λx)j
I n2 −1 ( λx) =
2n/2−1 4j j!Γ( n/2 + j)
j=0

Hence an alternative expression for the density is


1  x (n−2)/4 √
fX (x) = e−x/2 e−λ/2 I n2 −1 ( λx) (23.4b)
2 λ
This is the same as equation(23.1c) if we set n = 1 and λ = a2 .
A plot of the density of the χ28 distribution and the density of the non-central χ28 distribution with non-centrality
parameter µ equal to 5 and 10 is given in figure(23.4a).
0.15
n = 8, µ = 0
n = 8, µ = 5
n = 8, µ = 10
0.10

0.05

0.00
0 5 10 15 20
Figure(23.4a). Plot of the non-central χ28 density for various values of the non-centrality parameter µ.
(wmf/noncentralchisquared,79mm,56mm)

23.5 The general definition of the non-central χ2 for any n ∈ (0, ∞). Now exercise 24.2 shows that if f is
the density of the non-central χ2n distribution with non-centrality parameter λ, then
∞ −λ/2 λ j
X e ( /2)
f (x) = fn+2j (x) (23.5a)
j!
j=0

where fn+2j (x) is the density of the χ2n+2j distribution. This representation permits the following generalization:
Definition(23.5a). Suppose λ ∈ [0, ∞) and n ∈ (0, ∞). Then the random variable X has the non-central χ2
distribution with n degrees of freedom and non-centrality parameter λ iff X has the density function given
in equation (23.5a).
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §23 Page 75

Suppose X ∼ χ2n,λ where λ ∈ [0, ∞) and n ∈ (0, ∞).


Distribution function. By integrating equation(23.5a) we get the distribution function of X:
∞ −λ/2 λ j
X e ( /2)
FX (x) = Fn+2j (x) for x ∈ (0, ∞).
j!
j=0

where Fn+2k is the distribution function of the χ2n+2j distribution.


Moment generating function. For t ∈ (−∞, 1/2) we have
∞ −λ/2 λ j Z ∞
tX
X e ( /2)
MX (t) = E[e ] = etx fn+2j (x) dx
j! 0
j=0
∞ ∞ −λ/2 λ j
X e−λ/2 ( λ/2)j 1 1 X e ( /2)
= =
j! (1 − 2t)n/2+j (1 − 2t) n/2 j!(1 − 2t)j
j=0 j=0
   
1 λ λ 1 λt
= exp − + = exp
(1 − 2t)n/2 2 2(1 − 2t) (1 − 2t)n/2 1 − 2t
Representation in terms of the Poisson distribution. Suppose V has a Poisson distribution with mean λ/2 and the
distribution of W given V = j is the χ2n+2j distribution. Then W has the moment generating function
∞ ∞
tW
X
tW e−λ/2 ( λ/2)j X 1 e−λ/2 ( λ/2)j
E[e ]= E[e |V = j] = n/2+j
= E[etX ]
j! (1 − 2t) j!
j=0 j=0

Hence the distribution of W is the non-central χ2n,λ distribution.


Moments. Using the fact that X ∼ W we have
∞ ∞
X e−λ/2 ( λ/2)j X e−λ/2 ( λ/2)j
E[X] = E[W ] = E[W |V = j] = (n + 2j) = n + 2E[V ] = n + λ (23.5b)
j! j!
j=0 j=0
Similarly

2
X e−λ/2 ( λ/2)j
E[X ] = (n + 2j)(n + 2j + 2) = n2 + 2n + 4(n + 1)E[V ] + 4E[V 2 ]
j!
j=0

= n2 + 2n + 2(n + 1)λ + 4( λ/2 + λ2/4) (23.5c)


and hence
var[X] = 2(n + 2λ)
This could also be obtained by using the law of total variance which is equation(1.1b) on page 3.
Asymptotic normality Suppose X ∼ χ2n,λ . Then
X − (n + λ) D
√ =⇒ N (0, 1) as n → ∞. (23.5d)
2(n + 2λ)
which implies
X − (n + λ) D
√ =⇒ N (0, 1) as λ → ∞.
2(n + 2λ)
23.6 The non-central t distribution.
Definition(23.6a). Suppose n ∈ (0, ∞) and µ ∈ R. Then the random variable T has a non-central t-
distribution with n degrees of freedom and non-centrality parameter µ iff
X +µ
T =p (23.6a)
Y /n
where X ∼ N(0, 1), Y ∼ χ2n , and X and Y are independent.
See exercise 24.6 on page 76 for the following moments:
"  #2
n Γ n−1 n−1

n(1 + µ2 ) µ2 n Γ
r
2 2
E[T ] = µ for n > 1 and var[T ] = − for n > 2.
2 Γ n2 n−2 2 Γ n
2
Page 76 §24 Mar 10, 2020(20:25) Bayesian Time Series Analysis

If X ∼ N (µ, σ 2 ) and Y ∼ χ2n and X and Y are independent, then


X/σ
T =p
Y /n
has the non-central tn distribution with non-centrality parameter µ/σ.
23.7 The non-central F distribution.
Definition(23.7a). Suppose m > 0 and n > 0. Suppose further that X has a non-central χ2m distribution with
non-centrality parameter λ, Y ∼ χ2n and X and Y are independent. Then
X/m
F = has a non-central Fm,n distribution with non-centrality parameter λ.
Y /n
See exercise 24.8 for the following moments:
n(m + λ) (m + λ)2 + (m + 2λ)(n − 2)  n 2
E[F ] = for n > 2 and var[F ] = 2 for n > 4.
m(n − 2) (n − 2)2 (n − 4) m
If F1 has the non-central Fm,n distribution with non-centrality parameter λ and F2 has the Fm,n distribution, then
E[F1 ] ≥ E[F2 ]. This follows from the corresponding property of the non-central χ2 distribution.
The F -statistic used to test a hypothesis will usually have a central F distribution if the hypothesis is true and a
non-central F distribution if the hypothesis is false. The power of a test is the probability of rejecting the null
hypothesis when it is false. Hence calculating the power of a test will often involve calculating probabilities from
a non-central F distribution.
Similarly we have
Definition(23.7b). Suppose m > 0 and n > 0. Suppose further that X has a non-central χ2m distribution with
non-centrality parameter λ1 , Y has a non-central χ2n distribution with non-centrality parameter λ2 , and X and
Y are independent. Then
X/m
F = has a doubly non-central Fm,n distribution with non-centrality parameters (λ1 , λ2 ).
Y /n

24 Exercises (exs-noncentral.tex)

24.1 Suppose n ∈ {1, 2, . . .} and W has a non-central χ2n distribution with non-centrality parameter λ. By using the repre-
sentation in equation(23.2a), find E[W ] and var[W ]. [Ans]
24.2 Suppose λ ∈ (0, ∞) and the random variable V has the Poisson distribution with mean λ/2. Suppose further that the
distribution of W given V = j is the χ2n+2j distribution where n ∈ {1, 2, . . .}. Show that the distribution of W is the
non-central χ2n with non-centrality parameter λ. [Ans]
24.3 Suppose n ∈ {2, 3, . . .} and X1 , . . . , Xn are independent random variables. Suppose further that for j = 1, . . . , n we
have Xj ∼ χ2kj ,λj where kj ∈ (0, ∞) and λj ∈ [0, ∞). Find the distribution of Z = X1 + · · · + Xn . [Ans]

24.4 Suppose X ∼ χ2n,λ where n ∈ (0, ∞) and λ ∈ [0, ∞).


(a) Find the skewness, skew[X].
(b) Find the kurtosis, κ[X]. [Ans]
24.5 Suppose X ∼ χ2n,λwhere λ ∈ [0, ∞) and n ∈ (0, ∞). By using moment generating functions, prove the limiting result
in equation(23.5d).
24.6 Suppose T has the non-central t distribution with n degrees of freedom and non-centrality parameter µ. Show that
"  #2
n Γ n−1 n(1 + µ2 ) µ2 n Γ n−1
r 
2 2
E[T ] = µ for n > 1 and var[T ] = − for n > 2. [Ans]
2 Γ n2 n−2 2 Γ n2

24.7 Suppose T has the non-central t distribution with n degrees of freedom and non-centrality parameter µ. Show that T 2
has the non-central F1,n distribution with non-centrality parameter µ2 . [Ans]
24.8 Suppose F has the non-central Fm,n distribution with non-centrality parameter λ. Show that
n(m + λ) (m + λ)2 + (m + 2λ)(n − 2)  n 2
E[F ] = for n > 2 and var[F ] = 2 for n > 4. [Ans]
m(n − 2) (n − 2)2 (n − 4) m
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §25 Page 77

24.9 Suppose λ ∈ (0, ∞) and the random variable N has the Poisson distribution with mean λ/2. Suppose further that the
distribution of X given N = j is the Fm+2j,n distribution where m ∈ (0, ∞) and n ∈ (0, ∞).
Show that the distribution of X is the non-central Fm,n with non-centrality parameter λ.
(Hint: use exercise 24.2.) [Ans]

25 Size, shape and related characterization theorems


25.1 Size and shape: the definitions. The results in this section on size and shape are from [M OSSIMAN(1970)]
and [JAMES(1979)].
Definition(25.1a). The function g : (0, ∞)n → (0, ∞) is an n-dimensional size variable iff
g(ax) = ag(x) for all a > 0 and all x ∈ (0, ∞)n .
Definition(25.1b). Suppose g : (0, ∞)n → (0, ∞) is an n-dimensional size variable. Then the function z :
(0, ∞)n → (0, ∞)n is the shape function associated with g iff
x
z(x) = for all x ∈ (0, ∞)n .
g(x)
25.2 Size and shape: standard examples.
• The standard size function. This is g(x1 , . . . , xn ) = x1 + · · · + xn . The associated shape function is the function
z : (0, ∞)n → (0, ∞)n with !
x1 xn
z(x1 , . . . , xn ) = Pn , . . . , Pn
j=1 xj j=1 xj
• Dimension 1 size. This is g(x1 , . . . , xn ) = x1 . The associated shape function is
 
x2 xn
z(x1 , . . . , xn ) = 1, , . . . ,
x1 x1
• Dimension 2 size. This is g(x1 , . . . , xn ) = x2 . The associated shape function is
 
x1 xn
z(x1 , . . . , xn ) = , 1, . . . ,
x2 x2
• Volume. This is g(x1 , . . . , xn ) = (x21 + · · · + x2n )1/2 . The associated shape function is
 
x1 xn
z(x1 , . . . , xn ) = ,..., 2
(x21 + · · · + x2n )1/2 (x1 + · · · + x2n )1/2
• The maximum. This is g(x1 , . . . , xn ) = max{x1 , . . . , xn }. The associated shape function is
 
x1 xn
z(x1 , . . . , xn ) = ,...,
max{x1 , . . . , xn } max{x1 , . . . , xn }
• Root n size. This is g(x1 , . . . , xn ) = (x1 x2 . . . xn )1/n . The associated shape function is
 
x1 xn
z(x1 , . . . , xn ) = ,...,
(x1 x2 . . . xn )1/n (x1 x2 . . . xn )1/n
25.3 Size and shape: the fundamental result. We shall show that:
• if any one shape function z(X) is independent of the size variable g(X), then every shape function is independent
of g(X);
• if two size variables g(X) and g ∗ (X) are both independent of the same shape function z(X), then g(X)/g ∗ (X) is
almost surely constant.
First a specific example16 of this second result:
Example(25.3a). Suppose X = (X1 , X2 , X3 ) ∼ logN (µ, Σ) distribution. By definition, this means that if Y1 = ln(X1 ),
Y2 = ln(X2 ) and Y3 = ln(X3 ), then (Y1 , Y2 , Y3 ) ∼ N (µ, Σ).
Define the three size functions:

g1 (x) = x1 g2 (x) = x2 x3 g3 (x) = (x1 x2 x3 )1/3
and let z1 , z2 and z3 denote the corresponding shape functions. Suppose g1 (X) is independent of z1 (X).
(a) Show that var[Y1 ] = cov[Y1 , Y2 ] = cov[Y1 , Y3 ].
(b) Show that g1 (X) is independent of z2 (X). (c) Show that g1 (X) is independent of g2 (X)/g1 (X).
16
Understanding this example is not necessary for the rest of the section. The example makes use of the definition of the
multivariate normal and the fact that normals are independent if the covariance is zero. See Chapter3:§40.6 on page 126.
Page 78 §25 Mar 10, 2020(20:25) Bayesian Time Series Analysis

Now suppose g3 (X) is also independent of z1 (X).


(d) Show that cov[Y1 , S] = cov[Y2 , S] = cov[Y3 , S] where S = Y1 + Y2 + Y3 .
(e) Show that var[Y2 ] + cov[Y2 , Y3 ] = var[Y3 ] + cov[Y2 , Y3 ] = 2var[Y1 ].
(f) Show that var[2Y1 − Y2 − Y3 ] = 0 and hence g1 (X)/g3 (X) is constant almost everywhere.

Solution. We are given X1 is independent of 1, X2 /X1 , X3 /X1 . Taking logs shows that Y1 is independent of (Y2 −
Y1 , Y3 − Y1 ) and these are normal. Hence cov[Y1 , Y2 − Y1 ] = cov[Y1 , Y3 − Y1 ] = 0 and hence (a).
(b) follows because Y1 is independent of (Y1 − 21 Y2 − 12 Y3 , 21 Y2 − 12 Y3 , 12 Y3 − 21 Y2 ).
(c) Now cov[Y1 , 12 (Y2 + Y3 ) − Y1 ) = 21 cov[Y1 , Y2 ] + 21 cov[Y1 , Y3 ] − var[Y1 ] = 0. By normality, ln (g1 (X)) = Y1 is independent
of log (g2 (X)) − ln (g1 (X)). Because the exponential function is one-one, we have (c).
(d) The assumption g3 (X) is independent of z1 (X) implies, by taking logs, that S is independent of (Y2 − Y1 , Y3 − Y1 ) and
these are normal. Hence (d).
(e) Expanding cov[Y1 , S] and using part (a) shows that cov[Y1 , S] = 3var[Y1 ]. Similarly, expanding cov[Y2 , S] shows that
var[Y2 ] + cov[Y2 , Y3 ] + cov[Y1 , Y2 ] = cov[Y2 , S] = cov[Y1 , S] = 3var[Y1 ]. Hence (e).
(f) Now var[2Y1 − Y2 − Y3 ] = 4var[Y1 ] − 4cov[Y 1 , Y2 ] − 4cov[Y1 , Y3 ] + var[Y2 ] + var[Y3 ] + 2cov[Y2 , Y3 ] = 0. Hence
var[Y1 − 31 S] = 0; hence var[ln g1 (X)/g3 (X) ] = 0. Hence (f).


Now for the general result:


Proposition(25.3b). Suppose g : (0, ∞)n → (0, ∞) is an n-dimensional size variable and z ∗ : (0, ∞)n →
(0, ∞)n is any shape function. Suppose further that X is a random vector such that z ∗ (X) is non-degenerate
and independent of g(X). Then
(a) for any other shape function z1 : (0, ∞)n → (0, ∞)n , z1 (X) is independent of g(X);
(b) if g2 : (0, ∞)n → (0, ∞) is another size variable such that z ∗ (X) is independent of both g2 (X) and g(X),
then
g2 (X)
is constant almost everywhere.
g(X)
Proof. Let g ∗ and g1 denote the size variables which lead to the shape functions z ∗ and z1 . Hence
x x
z ∗ (x) = ∗ and z1 (x) = for all x ∈ (0, ∞)n .
g (x) g1 (x)
For all x ∈ (0, ∞)n we have
 

 x g1 (x)
g1 z (x) = g1 ∗
= ∗ by using g1 (ax) = ag1 (x).
g (x) g (x)
Hence for all x ∈ (0, ∞)n
z ∗ (x) x g ∗ (x)
z1 z ∗ (x) =

= × = z1 (x) (25.3a)
g1 ( z ∗ (x) ) g ∗ (x) g1 (x)
∗ ∗
Equation(25.3a) shows that z1 (X) is a function of z (X); also, we are given that z (X) is independent of g(X). Hence
z1 (X) is independent of g(X). This proves (a).
(b) Because of part (a), we can assume
X X
z2 (X) = is independent of g(X) and z(X) = is independent of g2 (X)
g2 (X) g(X)
Applying g to the first and g2 to the second gives
g(X) g2 (X)
g ( z2 (X) ) = is independent of g(X) and g2 ( z(X) ) = is independent of g2 (X)
g2 (X) g(X)
and hence
g2 (X)
is independent of both g2 (X) and g(X).
g(X)
Hence result by part (b) of exercise 2.11 on page 6.

25.4 A characterization of the gamma distribution.


Proposition(25.4a). Suppose X1 , X2 , . . . , Xn are independent positive non-degenerate random variables.
Suppose g ∗ (0, ∞)n → (0, ∞) denotes the size variable
n
X
g ∗ (x) = xj
j=1
Then there exists a shape vector z(X) which is independent of g ∗ (X) iff there exist α > 0, k1 > 0, . . . , kn > 0
such that Xj ∼ gamma(kj , α) for j = 1, 2, . . . , n.
Proof.
⇐ Now g ∗ (X) = nj=1 Xj ∼ gamma(k1 + · · · + kn , α). Proposition(11.8b) implies Xj /(X1 + · · · + Xn ) is independent
P
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §25 Page 79

of g ∗ (X) = X1 + · · · + Xn for j = 1, 2, . . . , n. Hence the standard shape vector


 
X1 X2 Xn
z(X) = , ,...,
X1 + · · · + Xn X1 + · · · + Xn X1 + · · · + Xn
∗ ∗
is independent of g (X). Hence all shape vectors are independent of g (X).
⇒ By proposition(25.3b), if there exists one shape vector which is independent of g ∗ (X), then all shape vectors are
independent of g ∗ (X). Hence Xj /(X1 + · · · + Xn ) is independent of g ∗ (X) = X1 + · · · + Xn for j = 1, 2, . . . , n. Hence
by proposition(11.8b), there exists α > 0 and kj > 0 such that Xj ∼ gamma(kj , α) for j = 1, 2, . . . , n.
This result implies many others. For example, suppose X1 , X2 , . . . , Xn are independent random variables with
Xj ∼ gamma(kj , α). Then every shape vector is independent of X1 + X2 + · · · + Xn ; in particular,
Xj
X1 + X2 + · · · + Xn is independent of
max{X1 , X2 , . . . , Xn }
and
X1 + X2 + · · · + Xn
X1 + X2 + · · · + Xn is independent of
max{X1 , X2 , . . . , Xn }
25.5 A characterization of the Pareto distribution.
Proposition(25.5a). Suppose X1 , X2 , . . . , Xn are independent positive non-degenerate random variables.
Suppose g ∗ (0, ∞)n → (0, ∞) denotes the size variable
g ∗ (x) = min{x1 , . . . , xn }
Then there exists a shape vector z(X) which is independent of g ∗ (X) iff there exists x0 > 0 and αj > 0 for
j = 1, 2, . . . , n such that Xj ∼ Pareto(αj , x0 , 0) for j = 1, 2, . . . , n.
Proof.
⇐ Let Y1 = ln(X1 ) and Y2 = ln(X2 ). Then Y1 −ln(x0 ) ∼ exponential (α1 ) and Y2 −ln(x0 ) ∼ exponential (α2 ) and Y1 and
Y2 are independent. By exercise 26.5 on page 81, we know that if Y1 −a ∼ exponential (λ1 ) and Y2 −a ∼ exponential (λ2 )
and Y1 and Y2 are independent, then min{Y1 , Y2 } is independent of Y2 − Y1 .
This establishes U = min{Y1 , Y2 } is independent of V = Y2 − Y1 . But (Y3 , . . . , Yn ) is independent of U and V . Hence
min{Y1 , . . . , Yn } is independent of Y2 −Y1 . Similarly min{Y1 , . . . , Yn } is independent of Yj −Y1 for j = 2, . . . , n. Hence
min{Y1 , . . . , Yn } is independent of the vector (Y2 − Y1 , Y3 − Y1 , . . . , Yn − Y1 ). And hence g ∗ (X) = min{X1 , . . . , Xn ) is
independent of the shape vector (1, X2/X1 , . . . , Xn/X1 ) as required.
⇒ Suppose n = 2. Using the shape vector (1, x2/x1 ) implies that we are given min{X1 , X2 } is independent of X2 /X1 .
Taking logs shows that min{Y1 , Y2 } is independent of Y2 − Y1 where Y1 = ln(X1 ) and Y2 = ln(X2 ).
It is known (see [C RAWFORD(1966)]) that if Y1 and Y2 are independent random variables with an absolutely continuous
distribution and min(Y1 , Y2 ) is independent of Y2 − Y1 , then there exist a ∈ R, α1 > 0 and α2 > 0 such that Y1 − a ∼
exponential (α1 ) and Y2 − a ∼ exponential (α2 ). Hence fY1 (y1 ) = α1 e−α1 (y1 −a) for y1 > a and fY2 (y2 ) = α2 e−α2 (y2 −a) for
y2 > a.
Hence X1 = eY1 ∼ Pareto(α1 , x0 = ea , 0) and X2 = eY2 ∼ Pareto(α2 , x0 = ea , 0) where x0 > 0.
For n > 2 we are given that
Xj
is independent of min{X1 , . . . , Xn } for j = 1, 2, . . . , n.
min{X1 , . . . , Xn }
But
min{X1 , . . . , Xn } = min{Xj , Zj } where Zj = min{Xi : i 6= j}
Hence for some x0j > 0, λj > 0 and λ∗j > 0, Xj ∼ Pareto(λj , x0j , 0) and Zj ∼ Pareto(λ∗j , x0j , 0). Because Zj =
min{Xi : i 6= j} we must have x0j ≤ x0i for j 6= i. It follows that all x0j are equal. Hence result.

25.6 A characterization of the power law distribution. Because the inverse of a Pareto random variable
has the power law distribution, the previous proposition can be transformed into a result about the power law
distribution.
Proposition(25.6a). Suppose X1 , X2 , . . . , Xn are independent positive non-degenerate random variables.
Suppose g ∗ (0, ∞)n → (0, ∞) denotes the size variable
g ∗ (x) = max{x1 , . . . , xn }
Then there exists a shape vector z(X) which is independent of g ∗ (X) iff there exists x0 > 0 and αj > 0 for
j = 1, 2, . . . , n such that Xj ∼ powerlaw(αj , x0 , 0) for j = 1, 2, . . . , n.
Proof.
⇐ Let Y1 = ln( 1/X1 ) and Y2 = ln( 1/X2 ). By exercise 20.3 on page 60, Y1 −ln( 1/x0 ) ∼ exponential (α1 ) and Y2 −ln( 1/x0 ) ∼
exponential (α2 ) and Y1 and Y2 are independent. By exercise 26.5 on page 81, we know that if Y1 − a ∼ exponential (λ1 )
and Y2 − a ∼ exponential (λ2 ) and Y1 and Y2 are independent, then min{Y1 , Y2 } is independent of Y1 − Y2 .
Page 80 §25 Mar 10, 2020(20:25) Bayesian Time Series Analysis

This establishes U = min{Y1 , Y2 } is independent of V = Y1 − Y2 . But (Y3 , . . . , Yn ) is independent of U and V . Hence


min{Y1 , . . . , Yn } is independent of Y1 −Y2 . Similarly min{Y1 , . . . , Yn } is independent of Y1 −Yj for j = 2, . . . , n. Hence
min{Y1 , . . . , Yn } is independent of the vector (Y1 − Y2 , Y1 − Y3 , . . . , Y1 − Yn ). And hence g ∗ (X) = max{X1 , . . . , Xn ) is
independent of the shape vector (1, X2/X1 , . . . , Xn/X1 ) as required.
⇒ Suppose n = 2. Using the shape vector (1, x2/x1 ) implies that we are given max{X1 , X2 } is independent of X2 /X1 .
Set Y1 = ln( 1/X1 ) and Y2 = ln( 1/X2 ). Hence min{Y1 , Y2 } is independent of Y2 − Y1 .
It is known (see [C RAWFORD(1966)]) that if Y1 and Y2 are independent random variables with an absolutely continuous
distribution and min(Y1 , Y2 ) is independent of Y2 − Y1 , then there exist a ∈ R, α1 > 0 and α2 > 0 such that Y1 − a ∼
exponential (α1 ) and Y2 − a ∼ exponential (α2 ). Hence fY1 (y1 ) = α1 e−α1 (y1 −a) for y1 > a and fY2 (y2 ) = α2 e−α2 (y2 −a) for
y2 > a.
Hence X1 = e−Y1 ∼ powerlaw(α1 , h = e−a , 0) and X2 = e−Y2 ∼ powerlaw(α2 , h = e−a , 0) where h > 0.
For n > 2 we are given that
Xj
is independent of max{X1 , . . . , Xn } for j = 1, 2, . . . , n.
max{X1 , . . . , Xn }
But
max{X1 , . . . , Xn } = max{Xj , Zj } where Zj = max{Xi : i 6= j}
Hence for some hj > 0, λj > 0 and λ∗j > 0, Xj ∼ powerlaw(λj , hj , 0) and Zj ∼ powerlaw(λ∗j , hj , 0). Because
Zj = max{Xi : i 6= j} we must have hj ≥ hi for j 6= i. It follows that all hj are equal. Hence result.

25.7 Independence of size and shape for the multivariate lognormal. This result requires a basic knowledge
of the multivariate normal—see Chapter3:§42 on page 130.
We say that the random vector X = (X1 , . . . , Xn ) ∼ logN (µ, Σ) iff ln(X) = ( ln(X1 ), . . . , ln(Xn ) ) ∼ N (µ, Σ).
Proposition(25.7a). Suppose X = (X1 , . . . , Xn ) ∼ logN (µ, Σ). Suppose further that g1 : (0, ∞)n → (0, ∞)
denotes the size variable
g1 (x) = (x1 · · · xn )1/n
Then g1 (X) is independent of every shape vector z(X) iff there exists c ∈ R such that cov[Yj , Y1 + · · · + Yn ] = c
for all j = 1, 2, . . . , n, where Y = (Y1 , . . . , Yn ) = (ln(X1 ), . . . , ln(Xn ) ).
Proof. By proposition(25.3b) on page 78, we need only prove g1 (X) is independent of one shape function. Consider the
shape function z ∗ (x) = 1, x2/x1 , . . . , xn/x1 .
Now g1 (X) is independent of z ∗ (X) iff (X1 · · · Xn )1/n is independent of 1, X2/X1 , . . . , Xn/X1 . This occurs iff Y1 +· · ·+Yn


is independent
Pof (Y2 − Y1 , . . . , Yn − Y1 ). But the Y ’s are normal; hence by
Pproposition(42.8b) Pon page 133, this occurs iff
n n n
cov[Yi − Y1 , j=1 Yj ] = 0 for i = 2, 3, . . . , n; and this occurs iff cov[Yi , j=1 Yj ] = cov[Y1 , j=1 Yj ] for i = 2, 3, . . . , n.

This result implies many others. For example, suppose X1 , X2 , . . . , Xn are independent random variables with
Xj ∼ logN (µj , σ 2 ) for j = 1, 2, . . . , n. Then
Xj
(X1 X2 · · · Xn )1/n is independent of
max{X1 , X2 , . . . , Xn }
and
X1 + X2 + · · · + Xn
(X1 X2 · · · Xn )1/n is independent of etc.
max{X1 , X2 , . . . , Xn }

Proposition(25.7a) leads to the following characterization of the lognormal distribution.


Proposition(25.7b). Suppose X1 , X2 , . . . , Xn are independent positive non-degenerate random variables.
Suppose g1 (0, ∞)n → (0, ∞) denotes the size variable
g1 (x) = (x1 · · · xn )1/n
Then there exists a shape vector z(X) which is independent of g1 (X) iff there exists σ > 0 such that every
Xj ∼ logN (µj , σ 2 ).
Proof.
⇐ Let Yj = ln(Xj ). Then Yj ∼ N (µj , σ 2 ); also Y1 , . . . , Yn are independent. Hence cov[Yj , Y1 + · · · + Yn ] = σ 2 for
j = 1, 2, . . . , n. Hence result by previous proposition.
⇒ By proposition(25.3b), if there exists one shape  vector which is independent of g1 (X), then all shape vectors are inde-
pendent of g1 (X). Hence 1, X2/X1 , . . . , Xn/X1 is independent of g1 (X) = (X1 · · · Xn )1/n . Hence Yk − Y1 is independent
of Y1 + · · · + Yn for k = 2, . . . , n. Hence, by the Skitovich-Darmois theorem—see proposition(15.8b), every Yk is normal.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §27 Page 81

26 Exercises (exs-sizeshape.tex)

26.1 Suppose X = (X1 , X2 ) is a 2-dimensional random vector with X1 = aX2 where a ∈ R. Show that if z : (0, ∞)2 →
(0, ∞)2 is any shape function, then z(X) is constant almost everywhere. [Ans]
26.2 Suppose X = (X1 , X2 ) is a 2-dimensional random vector with the distribution given in the following table:
X2
1 2 3 6
1 0 0 1/4 0
2 0 0 0 1/4
X1 1/4
3 0 0 0
6 0 1/4 0 0

Define the size variables g1 (x) = x1 x2 and g2 (x) = x1 + x2 .
(a) Suppose z is any shape function: (0, ∞)2 → (0, ∞)2 . Show that z(X) cannot be almost surely constant. Also, show
that z(X) is independent of both g1 (X) and g2 (X).
(b) Find the distribution of g1 (X)/g2 (X). [Ans]
26.3 A characterization of the generalized gamma distribution. Prove the following result.
Suppose X1 , X2 , . . . , Xn are independent positive non-degenerate random variables. Suppose g ∗ (0, ∞)n → (0, ∞)
denotes the size variable
 1/b
n
X
g ∗ (x) =  xbj  where b > 0.
j=1
Then there exists a shape vector z(X) which is independent of g ∗ (X) iff there exist α > 0, k1 > 0, . . . , kn > 0 such that
Xj ∼ ggamma(kj , α, b) for j = 1, 2, . . . , n.
Hint: use the result that X ∼ ggamma(n, λ, b) iff X b ∼ Γ(n, λ) and proposition(25.4a). [Ans]
26.4 Suppose X1 ∼ exponential (λ1 ), Y ∼ exponential (λ2 ) and X and Y are independent.
(a) Find P[X1 < X2 ].
(b) By using the lack of memory property of the exponential distribution, find the distribution of X1 − X2 .
(c) By using the usual convolution formula for densities, find the density of X1 − X2 . [Ans]
26.5 Suppose Y1 − a ∼ exponential (λ1 ) and Y2 − a ∼ exponential (λ2 ) and Y1 and Y2 are independent. Show that U =
min{Y1 , Y2 } is independent of V = Y2 − Y1 . [Ans]
26.6 A generalization of proposition(25.5a) on page 79. Suppose X1 , X2 , . . . , Xn are independent positive non-degenerate
random variables. and θ1 , θ2 , . . . , θn are positive constants. Suppose g ∗ (0, ∞)n → (0, ∞) denotes the size variable
 
∗ x1 xn
g (x) = min , ...,
θ1 θn
Prove there exists a shape vector z(X) which is independent of g ∗ (X) iff there exists x0 > 0 and αj > 0 for j =
1, 2, . . . , n such that Xj ∼ Pareto(αj , θj x0 , 0) for j = 1, 2, . . . , n. [JAMES(1979)] [Ans]
26.7 A generalization of proposition(25.6a) on page 79. Suppose X1 , X2 , . . . , Xn are independent positive non-degenerate
random variables and θ1 , θ2 , . . . , θn are positive constants. Suppose g ∗ (0, ∞)n → (0, ∞) denotes the size variable
 
∗ x1 xn
g (x) = max , ...,
θ1 θn
Prove there exists a shape vector z(X) which is independent of g ∗ (X) iff there exists x0 > 0 and αj > 0 for j =
1, 2, . . . , n such that Xj ∼ powerlaw(αj , θj x0 , 0) for j = 1, 2, . . . , n. [JAMES(1979)] [Ans]

27 Laplace, Rayleigh and Weibull distributions


27.1 The Laplace or bilateral exponential distribution. Suppose µ ∈ R and α > 0. Then the random
variable X is said to have the Laplace(µ, α) distribution iff X has the density
α
fX (x) = e−α|x−µ| for x ∈ R.
2
Clearly if X ∼ Laplace(µ, α), then X − µ ∼ Laplace(0, α) and α(X − µ) ∼ Laplace(0, 1); see also exercise 28.2.
As figure(27.1a) shows, the density consists of two equal exponential densities spliced back to back.
Decomposition. If V ∼ Laplace(0, 1), then V = X − Y where X and Y are i.i.d. random variables with the
exponential (1) distribution. See part(b) of exercise 28.1 on page 83.
Page 82 §27 Mar 10, 2020(20:25) Bayesian Time Series Analysis

3.0

2.5

2.0

1.5

1.0

0.5

0.0

−2 −1 0 1 2
Figure(27.1a). The bilateral exponential density for µ = 0 and α = 6.
(wmf/bilateralExponential,72mm,54mm)

Characteristic function. Suppose V ∼ Laplace(0, 1); then by the decomposition just described,
1 1 1
E[eitV ] = E[eitX ] E[e−itY ] = =
1 − it 1 + it 1 + t2
It follows that if X ∼ Laplace(µ, α), then
α2
E[eitX ] = eitµ 2 2
α +t
The usual inversion formula for characteristic functionsZ shows that
1 −|x| 1 1
e = e−itx dt
2 2π R 1 + t2
So this gives a way of finding the characteristic function of the Cauchy distribution—see equation(21.5b) on
page 65.
Moments. Suppose X ∼ Laplace(0, α). Then
(2n)!
E[X 2n−1 ] = 0 and E[X 2n ] = for n = 1, 2, . . . . (27.1a)
α2n
It follows that var[X] = E[X 2 ] = 2/α2 .
Entropy. For the definition of entropy, see exercise 8.6 on page 25. For the entropy of the Laplace distribution, see
exercise 28.7 on page 84. Now consider the class C of continuous distributions with E[ |X| ] finite and non-zero
density on R. It can be shown that the distribution in C with maximum entropy is the Laplace distribution—see
pages 62–65 in [KOTZ et al.(2001)].
Convolutions of independent Laplace distributions. See exercise 28.14 on page 84.
27.2 The Weibull distribution. Suppose β > 0 and γ > 0. Then the random variable X is said to have the
Weibull (β, γ) distribution iff X has the density
βxβ−1 −(x/γ)β
fX (x) = e for x > 0. (27.2a)
γβ
The distribution function is F (x) = 1 − exp(−xβ /γ β ) for x > 0. The Weibull distribution is frequently used to
model failure times.
The density can take several shapes as figure(27.3a) illustrates.
27.3 Elementary properties of the Weibull distribution.
Multiple of a Weibull distribution. Suppose X ∼ Weibull (β, γ) and d ∈ (0, ∞). Let Y = dX. Then Y has the
density
dx 1 βy β−1 −(x/(dγ) )β
fY (y) = fX (x)| | = fX (y/d) = e
dy d (dγ)β
which is the density of the Weibull (β, dγ) distribution.
It follows that for fixed β ∈ (0, ∞), the family of distributions {Weibull (β, γ) : γ ∈ (0, ∞)} is a scale family of
distributions—see definition(1.6d) on page 5.
Link with the exponential distribution. By setting β = 1 in equation(27.2a), we see that
d
Weibull (1, γ) = exponential (1/γ)
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §28 Page 83

This result can be generalised to

if Y ∼ Weibull (β, γ) then X = (Y /γ)β ∼ exponential (1)


Conversely,
if X ∼ exponential (1) then Y = γX 1/β ∼ Weibull (β, γ) (27.3a)
2.5
β = 1/2, γ = 1
2.0

1.5 β = 5, γ = 1

1.0 β = 1.5, γ = 1

0.5 β = 1, γ = 1

0.0
0.0 0.5 1.0 1.5 2.0 2.5

Figure(27.3a). The Weibull density for shape parameter β = 1/2, β = 1, β = 1.5 and β = 5;
all with scale parameter γ = 1.
(wmf/weibulldensity,72mm,54mm)


27.4 The Rayleigh distribution. Suppose σ > 0; then the Weibull (2, σ 2) is called the Rayleigh (σ) distribu-
tion. Hence, the random variable X has the Rayleigh (σ) distribution if X has the density
r 2 2
fR (r) = 2 e−r /2σ for r > 0.
σ
Clearly if R ∼ Rayleigh (σ) and b > 0 then bR ∼ Rayleigh (bσ). It follows that the family of distributions
{ Rayleigh (σ) : σ ∈ (0, ∞) } is a scale family of distributions—see definition(1.6d) on page 5.
The Rayleigh distribution is used to model the lifetime of various items and the magnitude of vectors—see exer-
cise 28.26 on page 86. There are plots of the density in figure(27.4a).
1.2 σ = 0.5
σ = 1.5
1.0 σ=4

0.8

0.6

0.4

0.2

0.0
0 2 4 6 8 10

Figure(27.4a). The Rayleigh distribution density for σ = 0.5, σ = 1.5 and σ = 4.


(wmf/Rayleighdensity,72mm,54mm)

28 Exercises (exs-LaplaceRayleighWeibull.tex)

The Laplace or bilateral exponential distribution.


28.1 (a) Suppose α > 0. Suppose further that X has the exponential density αe−αx for x > 0 and Y has the exponential
density αeαx for x < 0 and X and Y are independent. Show that X + Y ∼ Laplace(0, α).
(b) Suppose α > 0 and the random variables X and Y have the exponential density αe−αx for x > 0. Suppose further
that X and Y are independent. Show that X − Y ∼ Laplace(0, α). [Ans]
28.2 Linear transformation of a Laplace distribution.
(a) Suppose X ∼ Laplace(µ, α); suppose further that k > 0 and b ∈ R. Show that kX + b ∼ Laplace(kµ + b, α/k).It
follows that the family of distributions { Laplace(µ, σ) : µ ∈ R, σ ∈ (0, ∞) } is a location-scale family—see
definition(1.6b) on page 5.
(b) Suppose X ∼ Laplace(µ, α). Show that α(X − µ) ∼ Laplace(0, 1). [Ans]
Page 84 §28 Mar 10, 2020(20:25) Bayesian Time Series Analysis

28.3 Shape of the Laplace density. Suppose µ ∈ R and α ∈ (0, ∞). Suppose further that X ∼ Laplace(µ, α) has density fX .
(a) Show that fX is symmetric about x = µ.
(b) Show that fX increases on (−∞, µ) and decreases on (µ, ∞) and hence the mode is at x = µ.
(c) Show that fX is convex on (−∞, µ) and on (µ, ∞). [Ans]
28.4 Suppose X has the Laplace(µ, α) distribution.
(a) Show that the distribution function of X is
−α(µ−x)
1
2e if x ≤ µ;
FX (x) =
1 − 12 e−α(x−µ) if x ≥ µ.
(b) Show that the quartile function is

−1 µ + ln(2p)/α if p ∈ [0, 1/2];
FX (p) =
µ − ln(2 − 2p)/α if p ∈ [ 1/2, 1];
and hence the median is µ.
(c) Show that E[X] = µ and var[X] = 2/α2 .
(d) Show that the moment generating function of X
α2 eµt
E[etX ] = for |t| < α. [Ans]
α 2 − t2
28.5 Suppose Y ∼ Laplace(0, α). Show that
(2n)!
E[Y 2n−1 ] = 0 and E[Y 2n ] = for n = 1, 2, . . . .
α2n
d
In particular, if X ∼ Laplace(µ, α), then X = Y + µ and E[X] = µ and var[X] = E[Y 2 ] = 2/α2 . [Ans]
28.6 Suppose X ∼ Laplace(µ, α).
(a) Show that the skewness of X, skew[X] = 0.
(b) Show that the kurtosis of X, κ[X] = 6. [Ans]
28.7 Entropy of the Laplace distribution.For the definition of entropy, see exercise 8.6 on page 25.
Suppose X ∼ Laplace(µ, α). Find the entropy of X. [Ans]
28.8 (a) Suppose X has the Laplace(0, α) distribution. Show that |X| ∼ exponential (α).
(b) Suppose X ∼ exponential (λ), Y ∼ exponential (µ) and X and Y are independent. Find the density of Z = X − Y .
[Ans]
28.9 Suppose X and Y are independent random variables with X ∼ exponential (α) and Y ∼ Bernoulli ( 1/2).
Show that X(2Y − 1) ∼ Laplace(0, α). [Ans]
Pn 2
28.10 (a) Suppose X1 , . . . , Xn are i.i.d. with the Laplace(µ, α) distribution. Show that 2α i=1 |Xi − µ| ∼ χ2n .
(b) Suppose X and Y are i.i.d. Laplace(µ, α). Show that
|X − µ|
∼ F2,2 [Ans]
|Y − µ|
28.11 Suppose X and Y are i.i.d. uniform uniform(0, 1). Show that ln( X/Y ) ∼ Laplace(0, 1). [Ans]
28.12 Suppose X1 , X2 , X3 and X4 are i.i.d. N (0, 1).
(a) Show that X1 X2 − X3 X4 ∼ Laplace(0, 1).
(b) Show that X1 X2 + X3 X4 ∼ Laplace(0, 1). (See also exercise 16.16 in §2.16 on page 51.) [Ans]
28.13 Exponential scale mixture of normals.
√ Suppose X and Y are independent
√ random variables with X ∼ exponential (1)
and Y ∼ N (0, 1). Show that Y 2X ∼ Laplace(0, 1) and µ + Y 2X/α ∼ Laplace(µ, α).
Note. We can use either characteristic functions or densities. [Ans]
28.14 Convolutions of independent Laplace distributions.
(a) Suppose X and Y are i.i.d. random variables with the Laplace(0, α) distribution. Find the densities of V = X + Y
and W = X − Y .
(b) Suppose X ∼ Laplace(0, α1 ), Y ∼ Laplace(0, α2 ) and X and Y are independent. Find the densities of V = X + Y
and W = X − Y . (Hint: use characteristic functions.) [Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §28 Page 85

The Weibull distribution.


28.15 Shape of the Weibull density. Suppose the random variable X ∼ Weibull (β, γ) has density function fX .
0 00
(a) Find the derivatives fX (x) and fX (x).
(b) Suppose β ∈ (0, 1). Show that fX is decreasing and convex with fX (x) → ∞ as x ↓ 0.
(c) Suppose β = 1. Show that fX is decreasing and convex with mode at x = 0.
(d) Suppose β ∈ (1, 2]. Show that fX (x) is concave and then convex with point of inflection at
√ 1/β
3(β − 1) + (5β − 1)(β − 1)

x2 = γ

(e) Suppose β ∈ (2, ∞). Show that fX (x) is initially convex, then concave and then convex again with points of
inflection at
√ 1/β √ 1/β
3(β − 1) − (5β − 1)(β − 1) 3(β − 1) + (5β − 1)(β − 1)
 
x1 = γ and x2 = γ [Ans]
2β 2β
β
28.16 Suppose X has the Weibull (β, γ) distribution; hence fX (x) = βxβ−1 e−(x/γ) /γ β for x > 0.
(a) Suppose α > 0; find the distribution of Y = αX. (b) Find an expression for E[X n ] for n = 1, 2, . . . .
(c) Find the mean, variance, median and mode of X. (d) Find E[et ln(X) ], the m.g.f. of ln(X).
(e) Show that E[X] → γ and var[X] → 0 as β → ∞. [Ans]
28.17 Suppose X ∼ Weibull (β, γ).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
28.18 Suppose X has the Weibull (β, γ) distribution.
(a) Find hX (x) = f (x)/[1 − F (x)], the hazard function of X.
(b) Check that if β < 1 then hX decreases as x increases; if β = 1 then hX is constant; and if β > 1 then hX increases
as x increases. [Ans]
28.19 (a) Suppose U ∼ uniform(0, 1) distribution. Show that X = γ(− ln U )1/β ∼ Weibull (β, γ).
(b) Suppose X ∼ Weibull (β, γ). Show that U = exp −(X/γ)β ] ∼ uniform(0, 1).
 
[Ans]
28.20 Suppose X1 , X2 , . . . , Xn are i.i.d. random variables each with the Weibull (β, γ) distribution. As usual, let X1:n =
min{X1 , . . . , Xn }. Show that X1:n ∼ Weibull (β, γ/n1/β ). [Ans]
D
28.21 Suppose Xβ ∼ Weibull (β, γ). Show that Xβ =⇒ γ as β → ∞. [Ans]

The Rayleigh distribution.


28.22 Suppose R has the Rayleigh (σ) distribution:
r −r2 /2σ2
fR (r) = e for r > 0.
σ2
(a) Show that fR increases and then decreases with mode at r = σ.

(b) Show that fR is initially concave, then convex with inflection point at r = 3σ.
(c) Find the distribution function of R. (d) Find an expression for E[Rn ] for n = 1, 2, . . . .
(e) Find E[R] and var[R]. (f) Find the quartile function and median of R. [Ans]
(g) Find the hazard function of R.
28.23 Suppose X ∼ Rayleigh (σ).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
2
28.24 (a) Suppose X ∼ N (µ, σ ). Show that
Z ∞
(x − µ)2 µ2
   
1 σ µ
x√ exp − dx = √ exp − + µΦ
0 2πσ 2 2σ 2 2π 2σ 2 σ
(b) Suppose X ∼ Rayleigh (σ) where σ ∈ (0, ∞). Find the moment generating function of X. [Ans]

28.25 (a) Suppose U ∼ uniform(0, 1) and X = σ −2 ln U . Show that X ∼ Rayleigh (σ).
Conversely, if X ∼ Rayleigh (σ), then exp −X 2 /(2σ 2 ) ∼ uniform(0, 1).

(b) Suppose X has the exponential (λ) = gamma(1,
√ λ) distribution. Find the distribution of Y = X. In particular,
suppose X ∼ exponential (1); show that R = 2X has the Rayleigh (1) distribution. [Ans]
Page 86 §29 Mar 10, 2020(20:25) Bayesian Time Series Analysis

28.26 (a) Suppose X and Y are i.i.d. with the N (0, σ 2 ) distribution. Define R and Θ by R = X 2 + Y 2 , X = R cos Θ and
Y = R sin Θ with Θ ∈ [0, 2π). Prove that R and Θ are independent and find the density of R and Θ.
(b) Suppose R ∼ Rayleigh (σ), Θ ∼ uniform(−π, π) and R and Θ are independent. Show that X = R cos Θ and
Y = R sin Θ are i.i.d. N (0, σ 2 ).
(c) The Box-Muller √transformation.√ Suppose X ∼ exponential (1), Y ∼ uniform(0, 1) and X and Y are independent.
Let (V, W ) = ( 2X sin(2πY ), 2X cos(2πY )). Show that V and W are i.i.d. N (0, 1) random variables.
(d) Suppose U1 and√U2 are i.i.d. random variables
√ with the uniform(0, 1) distribution.
Let (V, W ) = ( −2 ln U1 sin(2πU2 ), −2 ln U1 cos(2πU2 )). Show that V and W are i.i.d. N (0, 1) random vari-
ables.
(e) Suppose X and Y are i.i.d. random variables with the N (0, σ 2 ) distribution. Let
2XY X2 − Y 2
W =√ and Z = √
X2 + Y 2 X2 + Y 2
Prove that W and Z are i.i.d. random variables with the N (0, 1) distribution. [Ans]
28.27 (a) Suppose R has the Rayleigh (σ) distribution. Find the distribution of R2 .
(b) Suppose
√ R has the Rayleigh (1) distribution. Show that the distribution of R2 is χ22 . Conversely, if X ∼ χ22 then
X ∼ Rayleigh (1).
Pn
(c) Suppose R1 , . . . , Rn are i.i.d. with the Rayleigh (σ) distribution. Show that Y = i=1 Ri2 has the gamma(n, 1/2σ 2 )
distribution. [Ans]
28.28 Suppose X ∼ Rayleigh (s) where s > 0, and Y |X ∼ N (µ, σ = X). Show that Y ∼ Laplace(µ, 1/s). [Ans]

29 The logistic distribution


29.1 The logistic distribution.
Recall the logistic function is the sigmoid curve
M
f (x) = for x ∈ R.
1+ e−k(x−x0 )
The logistic distribution has the logistic curve as its distribution function. The basic version of the density is

π e−πx/ 3
fX (x) = √ √ (29.1a)
3 (1 + e−πx/ 3 )2
and this has distribution function
1
FX (x) = √ for x ∈ R. (29.1b)
1 + e−πx/ 3
A linear transformation gives the following general form of the logistic distribution:
Definition(29.1a). Suppose µ ∈ R and σ > 0. Then the random variable X has the logistic distribution,
logistic(µ, σ 2 ), iff X has density function

π e−π(x−µ)/(σ 3)
fX (x) = √ h i for x ∈ R. (29.1c)
σ 3 1 + e−π(x−µ)/(σ√3) 2

The distribution function is


1
FX (x) = √ for x ∈ R. (29.1d)
1+e −π(x−µ)/(σ 3)

Another parametrization has s = σ 3/π; this gives
e−(x−µ)/s 1
fX (x) = 2 and FX (x) = −(x−µ)/s
(29.1e)
1 + e

s 1+e −(x−µ)/s

Alternative expressions for equations(29.1c), (29.1d) and (29.1e) are given in exercise 31.2.

Shape of the logistic density. It is easy to see that the density is symmetric about µ and the mode is µ–see also
exercise 31.1 and plots of the density in figure(29.1a).
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §29 Page 87

0.5
µ = 0, σ = 1
1.0
µ = 0, σ = 1
0.4 µ = 0, σ = 2
0.8

0.3
0.6 µ = 0, σ = 4

0.2 µ = 0, σ = 2
0.4

0.1 µ = 0, σ = 4 0.2

0.0 0.0
−15 −10 −5 0 5 10 15 −5 0 5 10 15 20
Figure(29.1a). Plots of the density and distribution function of the logistic distribution.
(wmf/logisticDensity,wmf/logisticDf,80mm,60mm)

29.2 Basic properties of the logistic distribution.


• If X ∼ logistic(µ, σ 2 ), and Y = a + bX, then Y ∼ logistic(a + bµ, b2 σ 2 ). See exercise 31.5 on page 96. It
follows that the family of distributions {logistic(µ, σ 2 ) : µ ∈ R, σ > 0} form a location-scale family—see
definition(1.6b) on page 5.
• If X ∼ logistic(0, 1) then X has the density in equation(29.1a) and distribution function in equation(29.1b).
• If X ∼ logistic(0, π 2 /3) then
e−x 1
fX (x) = −x
and FX (x) = for x ∈ R. (29.2a)
(1 + e ) 2 1 + e−x
This is the simplest form of the logistic density and sometimes called the standard logistic distribution. Multi-
plying the numerator and denominator of the density by e2x shows that fX (x) = ex /(1 + ex )2 .
• If X ∼ logistic(µ, σ 2 ), then
π
fX (x) = √ FX (x)[1 − FX (x)] for x ∈ R.
σ 3

29.3 Moments of the logistic distribution. Recall that if Y ∼ logistic(0, π 2 /3) then
e−y
fY (y) = for y ∈ R.
(1 + e−y )2
Straightforward algebra show that fY (−y) = fY (y) and hence E[Y n ] = 0 when n is odd. By using the expansion
∞  ∞
−2 n X

1 X
= x = (−1)n (n + 1)xn
(1 + x)2 n
n=0 n=0
we get

y 2 e−y
Z
E[Y 2 ] = 2 dy
0 (1 + e−y )2
Z ∞ X∞
=2 y 2 e−y (−1)n (n + 1)e−ny dy
0 n=0

X Z ∞
=2 n
(−1) (n + 1) y 2 e−(n+1)y dy
n=0 0

X Γ(3)
=2 (−1)n (n + 1) by using equation(11.1b) on page 33.
(n + 1)3
n=0

X (−1)n
=4
(n + 1)2
n=0
Because the series is absolutely convergent, we can rearrange and get
Page 88 §29 Mar 10, 2020(20:25) Bayesian Time Series Analysis
   
∞ ∞ ∞ ∞
X 1 X 1  X 1 X 1 
E[Y 2 ] = 4  −  = 4 −2

(n + 1)2 (n + 1) 2 (n + 1) 2 (n + 1)2

n=0 n=0 n=0 n=0
n even n odd n odd
"∞ ∞
# ∞
X 1 X 1 X 1 π2 π2
=4 − 2 = 2 = 2 =
k2 (2k)2 k2 6 3
k=1 k=1 k=1
by using equation 23.2.24 on page 807 in [A BRAMOWITZ &S TEGUN(1965)].
shown that if Y ∼ logistic(0, π 2 /3) then E[Y ] = 0 and var[Y ] = π 2 /3. If X ∼ logistic(µ, σ 2 ), then
We have √
X = µ + 3σY /π where Y ∼ logistic(0, π 2 /3). Hence
E[X] = µ and var[X] = σ 2

General result for moments of the logistic(0, π 2 /3) distribution. Suppose X ∼ logistic(0, π 2 /3). Because the
distribution is symmetric, we have
E[X 2r+1 ] = 0 for r = 0, 1, 2, . . . .
It remains to find the even moments: for r = 1, 2, . . . , we have
Z ∞ Z ∞
2r 2r e−x 2r e−x
E[X ] = x dx = 2 x dx (29.3a)
−∞ (1 + e−x )2 0 (1 + e−x )2
Consider the standard expansion
∞  
1 X `+n `
= z for |z| < 1
(1 − z)n+1 n
`=0
Setting n = 1 and replacing z by −z gives
∞   ∞
1 X `+1 ` `
X
= (−1) z = (` + 1)(−1)` z ` for |z| < 1.
(1 + z)2 1
`=0 `=0
Applying this result to equation(29.3a) gives
Z ∞ ∞
X ∞
X Z ∞
` −(`+1)x
2r
E[X ] = 2 x 2r
(` + 1)(−1) e dx = 2 (` + 1)(−1)`
x2r e−(`+1)x dx
0 `=0 `=0 0
∞ ∞
X Γ(2r + 1) X (−1)j−1
=2 (` + 1)(−1)` = 2Γ(2r + 1)
(` + 1)2r+1 j 2r
`=0 j=1
Because r ≥ 1, the series is absolutely convergent, and we can rearrange to get
   
∞ ∞ ∞ ∞
X 1 X 1  X 1 1 X 1 
= 2Γ(2r + 1)  2r
−2 2r
= 2Γ(2r + 1)  2r
− 2r−1
j (2`) j 2 `2r
j=1 `=1 j=1 `=1
 
1
= 2Γ(2r + 1) 1 − 2r−1 ζ(2r)
2
where ζ denotes the Riemann zeta function—see page 807 in [A BRAMOWITZ &S TEGUN(1965)]. By equation
23.2.16 in that reference, we have
(2π)2r
ζ(2r) = |B2r |
2(2r)!
where Bn are the Bernoulli numbers—see page 804 in [A BRAMOWITZ &S TEGUN(1965)]. Hence
E[X 2r ] = (22r − 2)π 2r |B2r |
Equation 23.1.3 in the same reference gives
1 1 1
B0 = 1, B1 = − , B2 = , B3 = 0, B4 = −
2 6 30
and hence
π2 7π 4
E[X 2 ] = 2π 2 |B2 | = and E[X 4 ] = 14π 4 |B4 | = (29.3b)
3 15
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §30 Page 89

29.4 Characteristic function of the logistic distribution. Suppose Y ∼ logistic(0, π 2 /3). Then
Z ∞ Z ∞
itY eity−y eity ey
φY (t) = E[e ] = −y 2
dy = y 2
dy
−∞ (1 + e ) −∞ (1 + e )

Consider the transformation u/(1 − u) = ey or u = ey /(1 + ey ). which maps y ∈ (−∞, ∞) 7→ u ∈ (0, 1). Note
that du y y 2
dy = e /(1 + e ) . Hence

u it
Z 1  Z 1
φY (t) = du = uit (1 − u)−it du = B(1 + it, 1 − it) = Γ(1 + it) Γ(1 − it) (29.4a)
0 1 − u 0

Euler’s reflection formula (equation 6.1.17 on page 256 in [A BRAMOWITZ &S TEGUN(1965)]) states that
π
Γ(z) Γ(1 − z) = provided z is not an integer.
sin(πz)
Using this result and the facts that Γ(z + 1) = zΓ(z) and sinh x = −i sin(ix) gives
itπ πt
φY (t) = itΓ(it) Γ(1 − it) = = for all t ∈ R.
sin(πit) sinh(πt)

If X ∼ logistic(µ, σ 2 ), then X = µ + 3σY /π where Y ∼ logistic(0, π 2 /3). Hence
√ ! √ !
iµt
√ iµt 3σ 3σ
φX (t) = e φY ( 3σt/π) = e Γ 1 + it Γ 1 − it
π π

iµt 3σt
=e √ for all t ∈ R.
sinh( 3σt)

30 Extreme value distributions


30.1 Extreme value distributions. The term extreme value distribution is used to denote a distribution which
is the limit as n → ∞ of the distribution of max{X1 , . . . , Xn } or min{X1 , . . . , Xn } where X1 , . . . , Xn are
i.i.d. random variables from an unbounded distribution. Further information about extreme value distributions can
be found in [KOTZ &NADARAJAH(2000)].

30.2 The standard Gumbel distribution.

Definition(30.2a). The random variable X has the standard Gumbel distribution iff X has an absolutely
continuous distribution with density function
fX (x) = exp −(e−x + x) for x ∈ R.
 
(30.2a)

Elementary properties.
• X has distribution function
FX (x) = exp −e−x for x ∈ R.
 

and this shows that the function defined by equation(30.2a) is a valid probability density function.
h √ i h √ i
• The mode is at x = 0, with points of inflection at x = ln (3 − 5)/2 and x = ln (3 + 5)/2 . See exer-
cise 31.12 on page 97.
• Median is ln( − ln 2 ). See exercise 31.13 on page 97.
• The moment generating function of X is
MX (t) = E[etX ] = Γ(1 − t) for t < 1.
See exercise 31.14 on page 97.
Page 90 §30 Mar 10, 2020(20:25) Bayesian Time Series Analysis

30.3 First and second moments of the standard Gumbel distribution. The derivative of the moment gener-
ating function is
Z ∞ Z ∞ Z ∞
0 d −t d −t ln(v)
MX (t) = v exp(−v) dv = e exp(−v) dv = − ln(v)v −t exp(−v) dv
0 dt 0 dt 0
R∞
Hence E[X] = γ where γ = − 0 e−x ln(x) dx is Euler’s constant.
Using equations 6.3.1 and 6.3.16 on pages 258–259 in [A BRAMOWITZ &S TEGUN(1965)] shows that the digamma
function ψ(z) satisfies Γ0 (z) = ψ(z)Γ(z) and
∞ 
Γ0 (z) X

d 1 1
ψ(z) = ln( Γ(z) ) = = − − γ for z > 0.
dz Γ(z) k+1 z+k
k=0
The derivative of the digamma function is the trigamma function. Using equation 6.4.10 on page 260 and equation
23.2.24 on page 807 in [A BRAMOWITZ &S TEGUN(1965)] shows that

X 1 π2
ψ 0 (1) = =
k2 6
k=1
Differentiating MX (t) = E[etX ] = Γ(1 − t) gives E[X] = −Γ0 (1) = −ψ(1)Γ(1) = γ and
π2
E[X 2 ] = Γ00 (1) = ψ 0 (1)Γ(1) + ψ(1)Γ0 (1) = ψ 0 (1) + [ψ(1)]2 = + γ2
6
Hence
var[X] = π 2 /6
Differentiating again and using equation 6.4.2 on page 260 in [A BRAMOWITZ &S TEGUN(1965)] gives
2π 2 γ π2 3π 2 γ
Γ000 (1) = ψ 00 (1)Γ(1) + 2ψ 0 (1)Γ0 (1) + ψ(1)Γ00 (1) = ψ 00 (1) − − γ( + γ 2 ) = ψ 00 (1) − − γ3
6 6 6
3π 2 γ
= −2ζ(3) − − γ3
6
and hence
E[X 3 ] − 3µσ 2 − µ3 −Γ000 (1) − 3µσ 2 − µ3 12ζ(3)
skew[X] = = =
σ3 σ3 π2
Finally
E[X 4 ] = Γ0000 (1) = ψ 000 (1)Γ(1) + 3ψ 00 (1)Γ0 (1) + 3ψ 0 (1)Γ00 (1) + ψ(1)Γ000 (1)
π2 π2 3π 2 γ
= 6ζ(4) + 6γζ(3) + 3 ( + γ 2 ) + γ(2ζ(3) + + γ3)
6 6 6
π4
= 6ζ(4) + 8γζ(3) + + π2γ 2 + γ 4
12
and hence E[X 4 ] − 4µE[X 3 ] + 6µ2 σ 2 + 3µ4 = 6ζ(4) + π 4 /12 which implies
E[X 4 ] − 4µE[X 3 ] + 6µ2 σ 2 + 3µ4 27
κ[X] = =
σ4 5
30.4 Relationships with other distributions. The following relationships are left to exercise 31.15.
• If X has the standard Gumbel distribution then FX (X) = exp(e−X ) ∼ uniform(0, 1).
• If U ∼ uniform(0, 1) then FX−1 (U ) = −ln( − ln U ) has the standard Gumbel distribution.
• If Z ∼ exponential (1) then − ln Z has the standard Gumbel distribution.
• If X has the standard Gumbel distribution then Z = e−X ∼ exponential (1).
• Suppose X and Y are i.i.d. random variables with the standard Gumbel distribution. Then Z = X − Y ∼
logistic(0, π 2 /3). This can be proved by using the convolution integral or by using characteristic functions—
see exercise 31.17 on page 97.
The following proposition shows why the Gumbel distribution is an extreme value distribution:
Proposition(30.4a). Suppose X1 , X2 , . . . is a sequence of i.i.d. random variables with the exponential (1)
distribution. Let
Vn = max{X1 , . . . , Xn } − ln n for n = 1, 2, . . .
Then
D
Vn =⇒ Z as n → ∞.
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §30 Page 91

where Z has the standard Gumbel distribution.


Proof. Now
P[Xn:n ≤ x] = [ FX (x) ]n = (1 − e−x )n for x > 0.
Hence for x > 0 n
e−x

n −x
P[Vn ≤ x] = P[Xn:n ≤ x + ln n] = 1 − e−(x+ln n) = → ee

1− as n → ∞.
n
D
Similarly, if Wn = min{−X1 , . . . , −Xn } + ln n = − max{X1 , . . . , Xn } + ln n, then Wn =⇒ −Z as n → ∞.
The distribution of −Z is called the reverse standard Gumbel distribution, or the standard Gumbel distribution for
minima.
Definition(30.4b). The random variable X has the reverse standard Gumbel distribution iff X has an abso-
lutely continuous distribution with density function
fX (x) = ex exp −ex for x ∈ R.
 
(30.4a)
30.5 The general Gumbel or type I extreme value distribution—definition.
Definition(30.5a). Suppose a ∈ R and b 6= 0. Then the random variable X has the Gumbel distribution,
Gumbel (a, b) iff X has density
x−a x−a
    
1
fX (x) = exp − exp − exp − for x ∈ R.
|b| b b
Clearly the Gumbel (0, 1) distribution is the standard Gumbel distribution defined in definition (30.4b) above.
0.4 0.4

a = 0, b = 1 a = 0, b = −1
0.3 0.3

a = 0, b = 2 a = 0, b = −2
0.2 0.2

0.1 0.1 a = 0, b = −6
a = 0, b = 6

0.0 0.0
−10 −5 0 5 10 15 20 −20 −15 −10 −5 0 5 10

Figure(30.5a). Plots of the density function of the Gumbel distribution.


(wmf/GumbelDensity1,wmf/GumbelDensity2,80mm,60mm)

If a = 0 and b ∈ (0, ∞), then the Gumbel (0, b) distribution has density
1  
fX (x) = e−x/b exp −e−x/b for x ∈ R.
b
Suppose b ∈ (0, ∞). Then the random variable X has the reverse Gumbel (0, b) distribution or the Gumbel (0, b)
distribution for minima iff X has an absolutely continuous distribution with density function
h i
fX (x) = ex/b exp −ex/b for x ∈ R. (30.5a)

30.6 The general Gumbel distribution—properties.


Linear transformation of a Gumbel distribution.
dy
Suppose X ∼ Gumbel (a, b), c ∈ R and d 6= 0. Let Y = c + dX. Then | dx | = |d| and

y − c − ad y − c − ad
    
dx 1 1
fY (y) = fX (x) = fX (x) = exp − exp − exp −
dy |d| |bd| bd bd
which is the density of the Gumbel (c + ad, bd) distribution.
Relationship to the standard Gumbel distribution. The last result implies that if X ∼ Gumbel (a, b) then X =
a + bY where Y ∼ Gumbel (0, 1). Hence the family of distributions {Gumbel (a, b) : a ∈ R, b > 0} is a location-
scale family, and the family of distributions {Gumbel (a, b) : a ∈ R, b < 0} is a location-scale family—see
definition(1.6b) on page 5.
Page 92 §30 Mar 10, 2020(20:25) Bayesian Time Series Analysis

The distribution function of X ∼ Gumbel (a, b). For x ∈ R we have


FX (x) = P[X ≤ x] = P[X − a ≤ x − a]
 
 P X−a ≤ x−a

b b if b > 0;
=
 P X−a ≥ x−a = 1 − P X−a ≤ x−a
   
b b b b if b < 0;

 FV x−a = exp −e−(x−a)/b
 
b if b > 0;
=
 1 − FV x−a = 1 − exp −e−(x−a)/b
 
b if b < 0.
where V ∼ Gumbel (0, 1), the standard Gumbel distribution.

The quantile function, moments and the moment generating function. See exercise 31.16.

30.7 Links with other distributions.


• Suppose X ∼ Gumbel (a, b) where a ∈ R and b > 0. Let Y = e−X . Then Y ∼ Weibull ( 1b , e−a ).
For the proof see exercise 31.18.
• Suppose Y ∼ Weibull (β, γ) where β > 0 and γ > 0. Then X = −ln Y ∼ Gumbel ( − ln γ, β1 ).
For the proof see exercise 31.19.
• Suppose X ∼ Gumbel (aX , b) and Y ∼ Gumbel (aY , b). Suppose further that X and Y are independent. Then
X − Y ∼ logistic(aX − aY , π 2 b2 /3). For the proof see exercise 31.17.

30.8 The Fréchet or type II extreme value distribution.


Definition(30.8a). Suppose α > 0. Then the random variable X has the standard Fréchet or type II extreme
value distribution iff X has distribution function

FX (x) =
0  if x < 0; (30.8a)
exp −x−α if x > 0.

Elementary properties.
• X has density
fX (x) = αx−1−α exp −x−α

for x > 0. (30.8b)

• Suppose Y ∼ exponential (1) and X = Y −1/α . Then for x > 0 we have


P[X ≤ x] = P[Y −1/α ≤ x] = P[Y 1/α ≥ 1/x] = P[Y ≥ 1/xα ] = exp(−1/xα )
and this is the distribution function of the Fréchet distribution.

• Moments. Now
Z ∞  
−1/α −1/α −y 1
E[X] = E[Y ]= y e dy = Γ 1 − for α > 1 by equation(11.1b) on page 33.
y=0 α
Similarly
Z ∞  
2 −2/α −2/α −y 2
E[X ] = E[Y ]= y e dy = Γ 1 − for α > 2.
y=0 α

1
• Median. By setting FX (x) = 2 we see that the median is at
1

α
ln 2

• Shape of the density. By differentiating the density we see that the mode is at
 α 1/α
xM =
1+α
and fX increases for x < xM and decreases for x > xM .
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §30 Page 93

30.9 The general Fréchet or type II extreme value distribution. This is a linear transformation of the standard
Fréchet distribution defined above.
Definition(30.9a). Suppose a ∈ R, b > 0 and α > 0. Then the random variable X has the Fréchet distribution,
Fréchet(α, a, b) iff X has density
 !
α x − a −1−α x − a −α
  
fX (x) = exp − for x > a. (30.9a)
b b b
1.2
0.5
a = 0, b = 1, α = 1 1.0 a = 0, b = 1, α = 3
0.4
0.8
0.3 a = 0, b = 2, α = 1 a = 0, b = 2, α = 3
0.6

0.2 0.4
a = 0, b = 6, α = 1 a = 0, b = 6, α = 3
0.1 0.2

0.0 0.0
0 2 4 6 8 10 12 14 0 2 4 6 8 10 12 14

Figure(30.9a). Plots of the density function of the Fréchet distribution.


Left: α = 1; right: α = 3. Note the difference in scales on the y-axis.
(wmf/Frechet-1,wmf/Frechet-3,80mm,60mm)

Linear transformation of the Fréchet distribution. If X ∼ Fréchet(α, a, b), a1 ∈ R and b1 > 0 then a1 + b1 X ∼
Fréchet(α, a1 + b1 a, bb1 )—see exercise 31.21 on page 98.
It follows that Fréchet(α, 0, 1) is the standard Fréchet distribution defined in equations(30.8b) and (30.8a). Also,
if X ∼ Fréchet(α, a, b), then X = a + bV where V ∼ Fréchet(α, 0, 1).
Basic properties.
• The distribution function of the Fréchet(α, a, b) distribution is
 !
x − a −α

F (x) = exp − for x > a. (30.9b)
b

• Median. The median is at


b
a+ √
α
ln 2
• Moments. 
E[X] = a + bΓ(1 − α1 ) if α > 1;
∞ otherwise

2 2 2 1 2
 
var[X] = b Γ 1 − α − b Γ 1 − α if α > 2;
∞ otherwise.
Links with other distributions. The following results are left to exercise 31.21 on page 98.
• If U ∼ uniform(0, 1), α > 0, a ∈ R, and b > 0 then a + b (− ln U )−1/α ∼ Fréchet(α, a, b).
• If X1 , . . . , Xn are i.i.d. random variables with the Fréchet(α, a, b) distribution and Y = max{X1 , . . . , Xn }
then
Y ∼ Fréchet(α, a, n1/α b)

• If X ∼ Fréchet(α, a, b) then
1
∼ Weibull (α, b−1 )
X
Page 94 §30 Mar 10, 2020(20:25) Bayesian Time Series Analysis

30.10 The reverse Weibull or type III extreme value distribution.


Definition(30.10a). Suppose α > 0. Then the random variable X has the standard type III extreme value
distribution iff X has distribution function 
exp (−(−x)α ) if x < 0;
FX (x) =
1 if x > 0.
Elementary properties.
• X has density
fX (x) = α(−x)α−1 exp (−(−x)α ) for x < 0. (30.10a)
• Suppose Y ∼ exponential (1) and X = −Y 1/α . Then for x < 0 we have
P[X ≤ x] = P[−Y 1/α ≤ x] = P[Y 1/α ≥ −x] = P[Y ≥ (−x)α ] = exp (−(−x)α )
and this is the distribution function of the type III extreme value distribution.
• Moments. Now for any α > 0 we have
Z ∞  
1
E[X] = −E[Y 1/α ] = − y 1/α e−y dy = −Γ 1 +
y=0 α
Similarly Z ∞  
2 2/α 2/α −y 2
E[X ] = E[Y ]= y e dy = Γ 1 +
y=0 α
1
• Median. By setting FX (x) = 2 we see that the median is at
√α
− ln 2

30.11 Link with the Weibull distribution. Suppose α > 0 and Y ∼ Weibull (α, 1). By equation(27.2a) on
page 82, Y has density
α
fY (y) = αy α−1 e−y for y > 0.
Let X = −Y . Then X has density
fX (x) = α(−x)α−1 exp (−(−x)α ) for x < 0.
and this is the same as equation(30.10a).
Hence we see that if X has the standard type III extreme value distribution, then −X ∼ Weibull (α, 1) and hence
properties of the type III extreme value distribution can be derived from properties of the Weibull distribution.
30.12 The general type III extreme value distribution. This is a linear transformation of the standard type III
distribution.
Definition(30.12a). Suppose a ∈ R, b > 0 and α > 0. Then the random variable X has the type III extreme
value distribution, extremeIII (α, a, b) iff X has density
x − a α−1 x−a α
     
α
fX (x) = − exp − − for x < a. (30.12a)
b b b
1.0 2.5

a = 0, b = 1, α = 1 a = 0, b = 1, α = 6
0.8 2.0

0.6 a = 0, b = 2, α = 1 1.5 a = 0, b = 2, α = 6

0.4 1.0
a = 0, b = 6, α = 1 a = 0, b = 6, α = 6
0.2 0.5

0.0 0.0
−8 −6 −4 −2 0 −8 −6 −4 −2 0

Figure(30.12a). Plots of the density function of the type III extreme value distribution. Left: α = 1; right: α = 6.
Note the difference in scales on the y-axis. When a = 0, b = 6 and α = 1 then f (x) = 61 exp( x6 ) for x < 0.
(wmf/extremeIII-alpha1,wmf/extremeIII-alpha6,80mm,60mm)
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §30 Page 95

Basic properties.
• The distribution function of the extremeIII (α, a, b) distribution. This is
x−a α
   
F (x) = exp − − for x < a. (30.12b)
b
with F (x) = 1 for x ≥ a.
• Linear transformation of the type III extreme value distribution. Suppose X ∼ extremeIII (α, a, b), a1 ∈ R and
b1 > 0. Then a1 + b1 X ∼ extremeIII (α, a1 + b1 a, b1 b).
In particular, if X ∼ extremeIII (α, a, b), then X = a + bV where V ∼ extremeIII (α, 0, 1).
• Median. This occurs at √
α
a − b ln 2

• Moments.  
1
E[X] = a − bΓ 1 +
α
1 2
   
2 2 2
var[X] = b Γ 1 + −b Γ 1+
α α

30.13 Unified formulation—the generalized extreme value distribution. The Fréchet(α, 0, 1) distribution
has density
fX (x) = αx−1−α exp −x−α for x > 0.


Consider the transformation y = α(x − 1); then y ∈ (−α, ∞) and


  
dx fX (x)  y −1−α y −α
fY (y) = fX (x) =
= 1+ exp − 1 + for y > −α. (30.13a)
dy α α α
The extremeIII (α, 0, 1) distribution has density
fX (x) = α(−x)α−1 exp (−(−x)α ) for x < 0.

dy
Consider the transformation y = α(1 + x); then y ∈ (−∞, α), dx = α and

dx y α−1   y α 
fY (y) = fX (x) = 1 − exp − 1 − for y < α.
dy α α
Using the parameter β = −α where now β < 0 gives
 !
y −1−β y −β
  
fY (y) = 1 + exp − 1 + for y < −β. (30.13b)
β β
Comparing equations(30.13a) and (30.13b) leads to the following unified formulation of an extreme value distri-
bution:
Definition(30.13a).The standardized extreme value density is defined to be the density
( −1−α  −α 
1 + αx exp − 1 + αx if α 6= 0;
f (x; α) = (30.13c)
−x
 
exp −(e + x) if α = 0, any x ∈ R.
If α > 0 then x > −α, and if α < 0 then x < −α.
Note that the expression for α = 0 in equation(30.13c) is the limit as α → 0 of the expression for α 6= 0. The
distribution function is 
( −α 
exp − 1 + αx if α 6= 0;
F (x; α) = (30.13d)
−x
 
exp −(e ) if α = 0, any x ∈ R.
Just as for the density, the expression for the distribution function for α = 0 is the limit as α → 0 of the expression
for α 6= 0.
Equation(30.13d) is valid for x > −α when α > 0 and is valid for x < −α when α < 0. In detail
(
0   if x ≤ −α;
when α > 0, then F (x; α) = exp − 1 + x −α if x > −α.
α
Page 96 §31 Mar 10, 2020(20:25) Bayesian Time Series Analysis

and (  −α 
when α < 0, then F (x; α) = exp − 1 + αx if x < −α;
1 if x ≥ −α;
A linear transformation leads to the following density.
Definition(30.13b). Suppose α ∈ R, µ ∈ R and σ > 0. Then the extreme value density GEV (α, µ, σ) is defined
to be the density
1+1/α −τ (x)
(1
σ [τ (x)] e if α 6= 0;
f (x; α, µ, σ) = 1 −τ (x)
σ τ (x)e if α = 0, and x ∈ R.
for x > µ − ασ if α > 0 and for x < µ − σα if α < 0, where
−α
1 + x−µ
(
ασ if α 6= 0;
τ (x) = x−µ 
exp − σ if α = 0.
The distribution function of GEV (α, µ, σ) is F (x; α, µ, σ) = e−τ (x) .
Clearly GEV (0, µ, σ) = Gumbel (µ, σ).
30.14 Properties of the generalized extreme value distribution. For proofs of the following, see exer-
cise 31.22.
• If X ∼ exponential (1), then µ − σ ln X ∼ GEV (0, µ, σ) = Gumbel (µ, σ).
• If X ∼ GEV (α, µ, σ), a > 0 and b ∈ R, then aX + b ∼ GEV (α, aµ + b, aσ).
• If X ∼ Weibull (β, γ) then β(1 − γ ln(X/γ)) ∼ GEV  (0, β,γ) = Gumbel (beta, γ).
• If X ∼ GEV (0, µ, σ) = Gumbel (µ, σ) then σ exp − X−µ µσ ∼ Weibull (µ, σ).

31 Exercises (exs-extreme.tex)

The logistic distribution.


31.1 Shape of the logistic density. Suppose X ∼ logistic(µ, σ 2 ) has density function f .
(a) Show that f is symmetric about x = µ.
(b) Show that f increases for x < µ and decreases for x > µ.
√ √
(c) Show that f is√initially convex
√ then concave and then convex with points of inflection at x = µ − σ 3 ln(2 + 3)/π
and x = µ + σ 3 ln(2 + 3)/π. [Ans]
31.2 Alternative expressions for the density and distribution function. √
(a) Show that the density of the logistic(µ, σ 2 ) distribution is, where s = σ 3/π
π(x − µ) x−µ
   
π 1
f (x) = √ sech2 √ = sech2 for x ∈ R.
4σ 3 2σ 3 4b 2b
This result explains why the logistic distribution is sometimes called the sech-squared distribution.

(b) Show that the distribution function of the logistic(µ, σ 2 ) distribution is,where s = σ 3/π
π(x − µ) x−µ
   
1 1 1 1
FX (x) = + tanh √ = + tanh for x ∈ R. [Ans]
2 2 2σ 3 2 2 2b
31.3 Quantile function and median of the logistic distribution. Show that the quantile function of the logistic(µ, σ 2 ) distribu-
tion is √  
−1 σ 3 p
FX (p) = µ + ln for p ∈ (0, 1).
π 1−p
−1 1
and hence the median of the logistic distribution is FX ( /2) = µ. [Ans]
31.4 Suppose Y ∼ logistic(µ, σ 2 ). Find the coefficients of skewness and kurtosis of X. [Ans]
2
31.5 Linear transformation of the logistic distribution.Suppose X ∼ logistic(µ, σ ) and Y = a + bX where a ∈ R and b ∈ R.
Prove that Y ∼ logistic(a + bµ, b2 σ 2 ). [Ans]
31.6 Suppose X ∼ logistic(µ, σ 2 ). Find the hazard function of X and show that it is proportional to the distribution function
of X. [Ans]
31.7 Suppose U ∼ uniform(0, 1) and  
U
Y = ln
1−U
Prove that Y ∼ logistic(0, π 2 /3).
Conversely, if Y ∼ logistic(0, π 2 /3) then U = eY /(1 + eY ) ∼ uniform(0, 1). [Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §31 Page 97

31.8 Suppose Y ∼ logistic(0, π 2 /3) and X = |Y |.


(a) Find the density and distribution function of X. (This is called the half-logistic distribution.)
(b) Find E[X] and var[X]. [Ans]
31.9 Suppose Y1 , Y2 , . . . , Yn are i.i.d. random variables with the logistic(0, π 2 /3) distribution.
(a) Find the density of Yr:n , the rth order statistic.
(b) Find the characteristic function of Yr:n .
(c) Find E[Yr:n ] and var[Yr:n ]. [Ans]
31.10 (a) Suppose µ ∈ R, σ > 0 and X ∼ exponential (1). Prove that

σ 3
Y =µ+ ln(eX − 1) ∼ logistic(µ, σ 2 )
π
In particular, if X ∼ exponential (1) then Y = ln(eX − 1) ∼ logistic(0, π 2 /3), the standard logistic distribution.
Conversely, if Y ∼ logistic(0, π 2 /3), the standard logistic distribution, then X = ln(eY + 1) ∼ exponential (1).
(b) Suppose X and Y are i.i.d. random variables with the exponential (1) distribution. Prove that
√  
σ 3 X
Y =µ+ ln ∼ logistic(µ, σ 2 ) [Ans]
π Y

31.11 (a) Suppose X ∼ logistic(0, π 2 /3), the standard logistic distribution. Prove that Y = eX + 1 ∼ Pareto(1, 1, 0) which
has density fY (y) = 1/y2 for y > 1.
(b) Suppose Y ∼ Pareto(1, 1, 0) which has density fY (y) = 1/y2 for y > 1. Prove that X = ln(Y −1) ∼ logistic(0, π 2 /3),
the standard logistic distribution. [Ans]
Extreme value distributions.
31.12 Suppose f is the density function of the standard Gumbel distribution.
(a) Find the derivative of f and show that f increases for x < 0, decreases for x > 0 and has a unique mode at x = 0.
00
(b) Find the second derivative of f and showh that√f is convex(f >h0) , then concave (f 00 < 0), then convex again
00
i √ i
(f > 0) with inflection points at x = ln (3 − 5)/2 and x = ln (3 + 5)/2 . [Ans]

31.13 Suppose X has the standard Gumbel distribution. Find the quantile function (the inverse of the distribution function)
and the median of the distribution. [Ans]
31.14 Suppose X has the standard Gumbel distribution. Find the moment generating function MX (t) = E[etX ]. [Ans]
31.15 Show that
(a) if U ∼ uniform(0, 1) then − ln( − ln U ) has the standard Gumbel distribution;
(b) if X has the standard Gumbel distribution then exp(e−X ) ∼ uniform(0, 1);
(c) if Z ∼ exponential (1) then − ln Z has the standard Gumbel distribution;
(d) if X has the standard Gumbel distribution then Z = e−X ∼ exponential (1). [Ans]
31.16 Suppose X ∼ Gumbel (a, b).
−1
(a) Find the quantile function, FX of X.
(b) Find the moment generating function of X.
(c) Find E[X] and var[X]. [Ans]
31.17 Suppose X and Y are i.i.d. random variables with the Gumbel (0, 1) distribution.
(a) By using the convolution integral, prove that Z = X − Y ∼ logistic(0, π 2 /3).
(b) By using characteristic functions, prove that Z = X − Y ∼ logistic(0, π 2 /3).
(c) Suppose X ∼ Gumbel (aX , b), Y ∼ Gumbel (aY , b) and X and Y are independent. Prove that Z = X − Y ∼
logistic(aX − aY , π 2 b2 /3). [Ans]

31.18 Suppose X ∼ Gumbel (a, b) where a ∈ R and b > 0. Let Y = e−X . Prove that Y ∼ Weibull ( b1 , e−a ). [Ans]

31.19 Suppose Y ∼ Weibull (β, γ) where β > 0 and γ > 0. Prove that X = −ln Y ∼ Gumbel ( − ln γ, β1 ). [Ans]

31.20 Suppose X ∼ Gumbel (0, 1) and Y = (−X|X < 0). Hence Y ≥ 0. Find the density of Y . [Ans]
Page 98 §32 Mar 10, 2020(20:25) Bayesian Time Series Analysis

31.21 (a) Suppose U ∼ uniform(0, 1), α > 0, a ∈ R, and b > 0. Prove that a + b (− ln U )−1/α ∼ Fréchet(α, a, b).
(b) Suppose X ∼ Fréchet(α, a, b), a1 ∈ R and b1 > 0. Prove that a1 + b1 X ∼ Fréchet(α, a1 + b1 a, bb1 ).
(c) Suppose X1 , . . . , Xn are i.i.d. random variables with the Fréchet(α, a, b) distribution and Y = max{X1 , . . . , Xn }.
Prove that
Y ∼ Fréchet(α, a, bn1/α )
(d) Suppose X ∼ Fréchet(α, 0, b). Prove that
1
∼ Weibull (α, b−1 ) [Ans]
X
31.22 (a) Suppose X ∼ exponential (1). Prove that µ − σ ln X ∼ GEV (0, µ, σ).
(b) Suppose X ∼ Weibull (β, γ) where β > 0 and γ > 0. Prove that β(1 − γ ln(X/γ)) ∼ GEV (0, β, γ).
 
(c) Suppose X ∼ GEV (0, µ, σ) where µ ∈ R and σ > 0. Prove that σ exp − X−µµσ ∼ Weibull (µ, σ).
(d) Suppose X ∼ GEV (α, µ, σ), a > 0 and b ∈ R.. Prove that Y = aX + b ∼ GEV (α, aµ + b, aσ). [Ans]

32 The Levy and inverse Gaussian distributions


The Lévy distribution
32.1 The standard Lévy distribution
Definition(32.1a). Suppose Z ∼ N (0, 1) and U = 1/Z 2 . Then U has the standard Lévy distribution,
Lévy(0, 1).

Derivation of the following properties of the standard Lévy distribution is left to exercise 33.1 on page 102:
• Density function of U .  
1 1
fU (u) = √ exp − for u ∈ (0, ∞).
2πu3 2u
• Distribution function of U .
  
1
FU (u) = 2 1 − Φ √ for u ∈ (0, ∞).
u
• Shape of the density function of U . The function fU√first increases and√ then decreases with mode at /3. The
1
density function fU has points of inflection at /3 −
1 10/15 and /3 +
1 10/15; it is initially convex, then concave
and finally convex.
• Moments of U .
E[U ] = ∞ and E[U k ] = ∞ for k = 1, 2, . . . .
It follows that the variance, skewness and kurtosis of U do not exist. Also, the moment generating function
MU (t) = E[etU ] = ∞ for all t > 0.
• Quantile function of U .
1
FU−1 (u) =  2 for p ∈ [0, 1).
−1
Φ (1 − p/2)
32.2 Characteristic function of the standard Lévy distribution. This derivation is a little tricky—here are the
details
Proposition(32.2a). Suppose U ∼ Lévy(0, 1). Then U has characteristic function
−1 if t < 0;
(
 
φU (t) = exp −|t|1/2 1 − i sgn(t)
 
for t ∈ R, where sgn(t) = 0 if t = 0; (32.2a)

1 if t > 0.
Proof. Suppose t > 0. Using the transformation w = (1 − i) t implies w2 = −2it and hence
Z ∞ Z ∞
2itu2 − 1
   
itu 1 1 1
φU (t) = e √ exp − du = √ exp du
0 2πu3 2u 0 2πu3 2u
Z ∞
e−w ∞ 1
   Z  2 
1 1 2 1 1  −1/2 1/2
= √ exp − w u+ du = √ √ exp − u − wu du (32.2b)
0 2πu3 2 u 2π 0 u3 2

Now let y = 1/(w2 u); hence y > 0, du 2 2
dy = w u , and u
−1/2
− wu1/2 = wy 1/2 − y −1/2 . Hence

2 Univariate Continuous Distributions Mar 10, 2020(20:25) §32 Page 99

e−w ∞ −1/2
Z  2 
1  1/2
φU (t) = √ wy exp − wy − y −1/2 dy (32.2c)
2π 0 2
Taking the average of equations(32.2b) and (32.2c) gives
e−w ∞ wy −1/2 y −3/2
Z    2 
1  1/2 −1/2
φU (t) = √ + exp − wy − y dy
2π 0 2 2 2
Now let z = wy 1/2 − y −1/2 ; hence
e−w ∞ −z2 /2 √
Z 
φU (t) = √ e dz = exp (−w) = exp −(1 − i) t
2π −∞
√
Finally, using complex conjugates gives, for t > 0, φU (−t) = φU (t) = exp −(1 + i) t .

32.3 The general Lévy distribution.


Definition(32.3a). Suppose a ∈ R and b > 0. Then V has the Lévy distribution, Lévy(a, b), iff V = a + bU
where U ∼ Lévy(0, 1).
Most properties of the Lévy(a, b) follow immediately from those for the standard Lévy distribution, Lévy(0, 1).
• Clearly V takes values in (a, ∞).
• Suppose X ∼ Lévy(a, b), c ∈ R and d > 0. Then c + dX ∼ Lévy(c + ad, bd).
It follows that the family of Lévy distributions {Lévy(a, b) : a ∈ R, b > 0} form a location-scale family—see
definition(1.6b) on page 5.
• Density function of V .
r  
b 1 b
fV (v) = exp − for v ∈ (a, ∞).
2π (v − a)3/2 2(v − a)
• Distribution function of V . " r !#
b
FV (v) = 2 1 − Φ
v−a
• Shape of the density function of V . The function fV √first increases and then √ decreases with mode at a + /3.
b
The function fV has points of inflection at a + ( /3 −
1 10/15)b and a + ( /3 +
1 10/15)b; it is initially convex, then
concave and finally convex.
• Moments of V .
E[V ] = ∞ and E[V k ] = ∞ for k = 1, 2, . . . .
It follows that the variance, skewness and kurtosis of V do not exist.
• Quantile function of V .
b
FV−1 (v) = a +  2 for p ∈ (0, 1).
−1
Φ (1 − p/2)
• Characteristic function of V .
 
φV (t) = exp ita − b1/2 |t|1/2 1 − i sgn(t)

for t ∈ R

1.0

0.8 b = 1/2
b=1
0.6 b=2

0.4

0.2

0.0
0 1 2 3 4
Figure(32.3a). Plot of the density of the Lévy distribution for a = 0 and various values of b.
The height of the mode decreases as b increases.
(wmf/LevyDistribution,80mm,60mm)
Page 100 §32 Mar 10, 2020(20:25) Bayesian Time Series Analysis

The inverse Gaussian distribution

32.4 The inverse Gaussian or Wald distribution. The name “inverse Gaussian” is misleading because this
distribution is not the inverse of the normal distribution. In fact the name is due to the fact that its cumulant
generating function (logarithm of the characteristic function) is the inverse of the cumulant generating function of
a Gaussian random variable.
Definition(32.4a). Suppose µ > 0 and λ > 0. Then the inverse Gaussian distribution, inverseGaussian(µ, λ),
is defined to be the distribution with density
r  
λ λ 2
f (x) = exp − 2 (x − µ) for x > 0. (32.4a)
2πx3 2µ x
There are two parameters: it is explained below that λ is the shape parameter and µ is the mean. The distribution
inverseGaussian(1, 1) is called the standard inverse Gaussian distribution.
It is easy to check that if X ∼ inverseGaussian(µ, λ) and α > 0, then αX ∼ inverseGaussian(αµ, αλ). This
implies that if X ∼ inverseGaussian(1, λ/µ) and Y = µX then Y ∼ inverseGaussian(µ, λ).
It follows that the family of distributions {inverseGaussian(µ, λ) : µλ = α}, where α > 0, is a scale family—see
definition (1.6d) on page 5.

32.5 The inverse Gaussian distribution: the distribution function. Consider the function
r √ 
λ x−µ λ 1/2 µ 
f (x) = = x − 1/2 for x > 0.
x µ µ x
Let Y = f (X) where X ∼ inverseGaussian(µ, λ). Then
√  
df λ 1 µ
= + > 0 for all x > 0.
dx 2µ x1/2 x3/2
Also f (x) → −∞ as x → 0 and f (x) → ∞ as x → ∞.
Hence this is a 1-1 transformation from x ∈ (0, ∞) 7→ y ∈ (−∞, ∞). The density of Y is
2µx3/2
 2
dx 2µ 1 y
fY (y) = fX (x) = fX (x) √
= √ exp − (32.5a)
dy λ(x + µ) (x + µ) 2π 2
It is left to an exercise to show that ! 2
y e−y /2
fY (y) = 1− p √ for y ∈ R. (32.5b)
4λ/µ + y 2 2π
Hence the distribution function of Y is
s !
2λ/µ 4λ
FY (y) = Φ(y) + e Φ − y2 + for y ∈ (−∞, ∞). (32.5c)
µ
Because the transformation is 1-1, we have
√ ! r ! r !
λ(x − µ) λ x−µ λ x + µ
FX (x) = FY √ =Φ + e2λ/µ Φ − for x > 0. (32.5d)
xµ x µ x µ

32.6 The inverse Gaussian distribution: the characteristic function. Let α = 2itµ2 /λ. Then
Z ∞ r Z ∞  
itX itx λ λ 2
φX (t) = E[e ] = e f (x; µ, λ) dx = exp itx − 2 (x − µ) dx
0 2πx3 0 2µ x
r Z ∞  
λ λ
= 3
exp itx − 2 (x2 − 2µx + µ2 ) dx
2πx 0 2µ x
 r Z ∞  
λ λ λx λ
= exp exp itx − 2 − dx
µ 2πx3 0 2µ 2x
 r Z ∞
xλ(1 − α) 2itµ2
 
λ λ λ
= exp exp − − dx where α = .
µ 2πx3 0 2µ2 2x λ
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §32 Page 101

Hence " #
r Z ∞ 1/2
− −

λ λ λ λ(1 α) xλ(1 α) λ
φX (t) = exp − (1 − α)1/2 exp − − dx
µ µ 2πx3 0 µ 2µ2 2x
r Z ∞ " 2 #

 
λ λ λ λ(1 α) µ
= exp − (1 − α)1/2 exp − x− dx
µ µ 2πx3 0 2xµ2 (1 − α)1/2
 
λ λ 1/2
= exp − (1 − α)
µ µ
because the integral of the density of the inverseGaussian µ(1 − α)−1/2 , λ is equal to 1. We have shown that

" r #
itX λ λ 2itµ2
φX (t) = E[e ] = exp − 1− (32.6a)
µ µ λ

32.7 The inverse Gaussian distribution: other properties.


Shape of the density. See exercise 33.9. The density f first increases and then decreases with mode at
"r #
9µ2 3µ
µ 1+ 2 −
4λ 2λ
The density is first convex, then concave and finally convex.
3.0 3.0

2.5 µ = 1, λ = 0.2 2.5 µ = 3, λ = 0.2

2.0 2.0
µ = 1, λ = 1 µ = 3, λ = 1
1.5 1.5

1.0 µ = 1, λ = 3 1.0 µ = 3, λ = 3

0.5 0.5

0.0 0.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5

Figure(32.7a). Plots of the density function of the inverse Gaussian distribution. Left: µ = 1; right: µ = 3.
(wmf/inverseGaussian-mu1,wmf/inverseGaussian-mu3,80mm,60mm)
• Moments—see exercise 33.7. Suppose X ∼ inverseGaussian(µ, λ). Then
µ3
E[X] = µ and var[X] =
λ
• Convolutions—see exercise 33.10. The following results are proved by using characteristic functions.
(a) Suppose X1 , . . . , Xn are independent random variables with Xk ∼ inverseGaussian(µk , λk ) for k = 1, . . . , n.
Then
n n
X λk Xk 2
X λk
2
∼ inverseGaussian(µ, µ ) where µ =
µk µk
k=1 k=1
(b) Suppose X1 , . . . , Xn are i.i.d. random variables with the inverse Gaussian inverseGaussian(µ, λ) distribution.
X ∼ inverseGaussian(µ, nλ)
(c) Suppose α > 0, X ∼ inverseGaussian(µ1 , αµ21 ), Y ∼ inverseGaussian(µ2 , αµ22 ) and X and Y are indepen-
dent, then X + Y ∼ inverseGaussian( µ, αµ2 ) where µ = µ1 + µ2 .
• Limiting distributions. See exercise 33.12.
D
inverseGaussian(µ, λ) =⇒ Lévy(0, λ) as µ → ∞
and
D
inverseGaussian(µ, λ) =⇒ µ as λ → ∞
Page 102 §33 Mar 10, 2020(20:25) Bayesian Time Series Analysis

33 Exercises (exs-LevyInverseGaussian.tex.tex)

The Lévy distribution.


33.1 The standard Lévy distribution. Suppose U has the standard Lévy distribution, Lévy(0, 1).
(a) Show that the density function of U is
 
1 1
fU (u) = √ exp − for u ∈ (0, ∞).
2πu3 2u
(b) Show that the distribution function of U is   
1
FU (u) = 2 1 − Φ √ for u ∈ (0, ∞).
u
(c) Show that E[U k ] = ∞ for k = 1, 2, . . . .
(d) Show that the quantile function of U is
1
FU−1 (u) =  2 for p ∈ [0, 1).
Φ−1 (1 − p/2)
(e) Show that fU first increases and then decreases with mode at 1/3.
√ √
(f) Show that the function fU has points of inflection at 1/3 − 10/15 and 1/3 + 10/15; show further that fU is initially
convex, then concave and finally convex. [Ans]

33.2 Suppose U ∼ Lévy(0, 1). Show that X = 1/ U has the standard half-normal distribution. [Ans]
33.3 Suppose X andY are independent random
 variables such that X ∼ Lévy(a1 , b1 ) and Y ∼ Lévy(a2 , b2 ). Show that
1/2 1/2 2
X + Y ∼ Lévy a1 + a2 , (b1 + b2 ) . [Ans]
33.4 Suppose X ∼ Lévy(a, b) where a ∈ R and b > 0.
(a) Prove that X has a stable distribution with characteristic exponent α = 1/2.
(b) Using the notation in equation(4.3b), show that X is stable with characteristic function {c = a, d = b, β = 1}. [Ans]

The inverse Gaussian distribution.


33.5 Suppose we denote the density of the inverse Gaussian distribution in equation(32.4a) by f (x; µ, λ). Prove that
1 x λ 1 x µ
f (x; µ, λ) = f ( ; 1, ) and f (x; µ, λ) = f ( ; , 1)
µ µ µ λ λ λ
The first equality says if X ∼ inverseGaussian(1, µλ ) then µX ∼ inverseGaussian(µ, λ), and the second equality says if
X ∼ inverseGaussian( µλ , 1) then λX ∼ inverseGaussian(µ, λ). [Ans]
33.6 (a) Derive equation(32.5b) from equation(32.5a).
(b) Derive equation(32.5c) from equation(32.5b). [Ans]
33.7 Suppose X ∼ inverseGaussian(µ, λ).
(a) Show that
µ3
E[X] = µ and var[X] =
λ
(b) Find expressions for E[X 3 ] and E[X 4 ]. [Ans]
33.8 Suppose X ∼ inverseGaussian(µ, λ).
(a) Find skew[X], the skewness of X.
(b) Find κ[X], the kurtosis of X. [Ans]
33.9 Suppose X ∼ inverseGaussian(µ, λ) with density function f .
(a) Show that f increases and then decreases with"mode at #
r
9µ2 3µ
µ 1+ 2 −
4λ 2λ
(b) Show that f 00 is given by
r
λ(x − µ)2
 2 4
6λx3 2λ2
 
λ λx
f 00 (x) = exp − + + x 2
(15 − ) − 10λx + λ 2
32πx11 2µ2 x µ4 µ2 µ2
By investigating the quadratic in x inside the last set of square brackets, it is possible to show that f 00 is initially
positive, then negative and then positive again. Hence f is initially convex, then concave and then convex again. The
positions of the two points of inflection are the positive roots of this quartic—very complicated! (Note that we need
only consider the special case of the quartic when µ = 1 by the first equality in exercise 33.5.) [Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §34 Page 103

33.10 (a) Suppose X1 , . . . , Xn are independent random variables with Xk ∼ inverseGaussian(µk , λk ) for k = 1, . . . , n. Show
that
n n
X λk Xk 2
X λk
2
∼ inverseGaussian(µ, µ ) where µ =
µk µk
k=1 k=1
(b) Suppose X1 , . . . , Xn are i.i.d. random variables with the inverse Gaussian inverseGaussian(µ, λ) distribution. Show
that
X ∼ inverseGaussian(µ, nλ) [Ans]
33.11 Suppose X ∼ inverseGaussian(µ, λ). Show that X is infinitely divisible. [Ans]
D
33.12 (a) Show that inverseGaussian(µ, λ) =⇒ Lévy(0, λ) as µ → ∞.
D
(b) Show that inverseGaussian(µ, λ) =⇒ µ as λ → ∞. [Ans]
33.13 Suppose X ∼ inverseGaussian(µ, λ). By expressing the density of X in the form
r    
λ λ λ λ 1
fX (x) = exp exp − 2 x − for x ∈ (0, ∞).
2πx3 µ 2µ 2 x
show that inverseGaussian(µ, λ) belongs to the exponential family of distributions with natural parameters −λ/(2µ2 )
and −λ/2 and natural statistics X and 1/X. [Ans]

34 Other distributions with bounded support


34.1 The Wigner or semicircle distribution. The random variable X has the standard semicircle distribution
iff X has density
2p
fX (x) = 1 − x2 for x ∈ [−1, 1]. (34.1a)
π
Suppose µ ∈ R and r > 0. Then the random variable X has the semicircle distribution with mean µ and radius r
iff X has density
2 p
fX (x) = 2 r2 − (x − µ)2 for x ∈ [µ − r, µ + r]. (34.1b)
πr

34.2 The triangular distribution. Suppose a ∈ R, b ∈ (0, ∞) and p ∈ [0, 1]; then the random variable X has
the triangular distribution, triangular(a, b, p) iff X has the following density.
2
If p = 0: fX (x) = b2
(a + b − x) for x ∈ [a, a + b].
( 2
pb2
(x − a) if x ∈ [a, a + pb];
If 0 < p < 1: fX (x) = 2
b2 (1−p)
(a + b − x) if x ∈ [a + pb, a + b].
2
If p = 1: fX (x) = b2
(x − a) for x ∈ [a, a + b].

fX (x) .............. fX (x) .............. fX (x) ..............


.. .. ..
... ... ...
.. ............ .. .......... .. ...
.........
... ... ......... ... .... ..... ... ...... ....
... ... ...... ... .. . ......... ... ...... ...
... ...
......
...... ... .
... . ..... ... .
.......... ...
. . ..... .
... ... ...... ... ... . ..... ... ...... ...
... ...
......
...... ... ... . ..... ... .
.......... ...
... ... ...... ... .. . ..... ... ....... ...
... 2/b ...
......
...... ... .
.
.. 2/b
.
.
.....
..... ... .
...........
. . 2/b ...
...
... ... ...... ... ... . ..... ... ..
...
... ... ......
...... ... ..
. . .....
..... ... ......... ...
...
... ... ...... ... ..
. .
. ..... ... ...
.....
. ...
... .
.
......
...... ... .... . ..... ... ..
..
.... ...
. ..... ....
... ..
. ...... ... ... ..... ... ..
..
. ...
... ... ...... . ... ..
.
.
. .... . ... .
..
.....
. .. .
.....................................................................................................................................................................................
. ...................................................................................................................................................................................... .....................................................................................................................................................................................
. . .
.... .... ....
0 a a+b x 0 a a + pb a+b x 0 a a+b x
Figure(34.2a). Plot of triangular density for p = 0(left), 0 < p < 1(centre) and p = 1(right).
(PICTEX)

34.3 The sine distribution. Suppose a ∈ R and b ∈ (0, ∞); then the random variable X has the sine distribu-
tion, which we shall denote as sinD (a, b), iff X has density
x−a
 
π
fX (x) = sin π for x ∈ [a, a + b]. (34.3a)
2b b
From standard properties of the sine function, it is clear that the density first increases and then decreases with
mode at x = a + b/2; also f 00 < 0 and hence the density is concave.
Page 104 §35 Mar 10, 2020(20:25) Bayesian Time Series Analysis

34.4 The U-power distribution. Suppose a ∈ R, b ∈ (0, ∞) and k ∈ {1, 2, 3, . . .}; then the random variable X
has the U-power distribution, which we shall denote as Upower(a, b, k), iff X has density
2k + 1 x − a 2k
 
fX (x) = for x ∈ [a − b, a + b]. (34.4a)
2b b
The density of the Upower(0, 1, k) distribution is f (x) = (k + 1/2)x2k for x ∈ [−1, 1].
Shape of the Upower(a, b, k) density. Differentiating equation(34.4a) gives, for x ∈ [a − b, a + b]
(2k + 1)(2k) x − a 2k−1 (2k + 1)(2k)(2k − 1) x − a 2k−2
   
0 00
fX (x) = and fX (x) =
2b2 b 2b3 b
Hence fX is convex and fX first decreases and then increases with minimum at x = a. Also the modes are at a − b
and a + b and fX is symmetric about x = a.
Illustrative plots of the density are given in figure (34.4a).
3.0

2.5

2.0

1.5

1.0

0.5

0.0
−1.0 −0.5 0.0 0.5 1.0

Figure(34.4a). Plots of the U-power density: Upower(0, 1, 1) (black) and Upower(0, 1, 3) (red).
(wmf/upower,72mm,54mm)

35 Exercises (exs-otherbounded.tex)

The Wigner or semicircle distribution


35.1 The density and distribution function of the standard semicircle distribution.
(a) Check that equation(34.1a) integrates to 1 and so is a density.
(b) Show that the density increases for x ≤ 0, decreases for x ≥ 0, has a mode at 0 and is concave for all x ∈ [−1, 1].
(c) Show that the distribution function is
1 1 p 1
FX (x) = + x 1 − x2 + arcsin x for x ∈ [−1, 1]. [Ans]
2 π π
35.2 Moments of the standard semicircle distribution. Suppose X has the standard semicircle distribution with the density in
equation(34.1a).
(a) Show that the moments of X are given by
 
2n−1 2n1 1 2n
E[X ] = 0 and E[X ] = 2n for n = 1, 2, . . . .
2 n+1 n
In particular E[X] = 0 and var[X] = E[X 2 ] = 1/4.
(b) Show that the skewness is skew[X] = 0 and the kurtosis is κ[X] = 2. [Ans]
2 2 2
35.3 Suppose the random vector (X, Y ) has the uniform distribution on the unit disc {(x, y) ∈ R : 0 < x + y < 1}. Prove
that both X and Y have the standard semicircle distribution. [Ans]
35.4 Let semicircle(µ, r) denote the semicircle distribution with mean µ and radius r, where µ ∈ R and r > 0.
Suppose the random variable X ∼ semicircle(µ, r).
(a) Prove that X = µ + rY where Y has the standard semicircle distribution, semicircle(0, 1).
(b) Show that the distribution function of X is
1 x − µp 2 x−µ
 
1
FX (x) = + r − (µ − x) + arcsin
2 for x ∈ (µ − r, µ + r).
2 πr2 π r
Note that part(a) implies E[X] = µ, var[Y ] = r2 /4, skew[X] = 0 and κ[X] = 2.
It also implies that the family of distributions {semicircle(µ, r) : µ ∈ R, r > 0} is a location-scale family—see
definition (1.6b) on page 5. [Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §35 Page 105

35.5 Show that beta( 3/2, 3/2) = semicircle( 1/2, 1/2). [Ans]

The triangular distribution


35.6 The triangular(0, 1, p) has the following density.
If p = 0: fX (x) = 2(1 − x) for x ∈ [0, 1].
(
2x
p if x ∈ [0, p];
If 0 < p < 1: fX (x) = 2(1−x)
(35.6a)
1−p if x ∈ [p, 1].
If p = 1: fX (x) = 2x for x ∈ [0, 1].
Suppose X ∼ triangular(a, b, p). Show that X = a + bY where Y ∼ triangular(0, 1, p).
It follows that the family of distributions {triangular(a, b, p) : a ∈ R, b ∈ (0, ∞) } is a location-scale family—see
definition(1.6b) on page 5. [Ans]
35.7 (a) Suppose X ∼ triangular(0, 1, 1). Show that E[X n ] = 2/(n + 2) for n ∈ {1, 2, . . .}.
(b) Suppose X ∼ triangular(0, 1, p) where p ∈ [0, 1) Show that
2 1 − pn+1
E[X n ] = for n ∈ {1, 2, . . .}.
(n + 1)(n + 2) 1 − p
(c) Suppose X ∼ triangular(a, b, p) where a ∈ R, b ∈ (0, ∞) and p ∈ [0, 1]. Show that
b b2
E[X] = a + (1 + p) and var[X] = (1 − p + p2 ) [Ans]
3 18
35.8 Skewness and kurtosis. Suppose X ∼ triangular(a, b, p) where a ∈ R, b ∈ (0, ∞) and p ∈ [0, 1].
(a) Show that the skewness is
21/2 (1 − 2p)(1 + p)(2 − p)
skew[X] =
5(1 − p + p2 )3/2
If Y ∼ triangular(a, b, 1 − p), then skew[Y ] = −skew[X] and if X ∼ triangular(a, b, 21 ), then skew[X] = 0.
(b) Show that the kurtosis is κ[X] = 12/5. [Ans]
35.9 The distribution function. Suppose X ∼ triangular(a, b, p) where a ∈ R, b ∈ (0, ∞) and p ∈ [0, 1].
(a) Show that the distribution function of X is as follows.
If p = 0: FX (x) = 1 − b12 (a + b − x)2 for x ∈ [a, a + b].
( 1
pb2
(x − a)2 if x ∈ [a, a + pb];
If 0 < p < 1: FX (x) =
1
1 − b2 (1−p) (a + b − x)2 if x ∈ [a + pb, a + b].
1
If p = 1: FX (x) = b2
(x − a)2 for x ∈ [a, a + b].
(b) Show that the quantile function of X is, for all p ∈ [0, 1]:
 √
−1 a + b pu√ if 0 ≤ u ≤ p;
FX (u) =
a + b − b (1 − u)(1 − p) if p ≤ u ≤ 1.
−1
Hence if U ∼ uniform(0, 1), then FX (U ) ∼ triangular(a, b, p). [Ans]
35.10 Suppose X ∼ triangular(0, 1, p). Show that 1 − X ∼ triangular(0, 1, 1 − p). [Ans]
35.11 Suppose X ∼ triangular(a, b, p) where a ∈ R, b ∈ (0, ∞) and p ∈ [0, 1]. Suppose further that c ∈ R and d ∈ (0, ∞).
(a) Show that c + dX ∼ triangular(c + da, db, p).
(b) Show that c − dX ∼ triangular(c − da − db, db, 1 − p).
In particular, if X ∼ triangular(a, b, p), then −X ∼ triangular(−a − b, b, 1 − p). [Ans]
35.12 Suppose U1 and U2 are i.i.d. random variables with the uniform(a, a + b) distribution where a ∈ R and b ∈ (0, ∞).
(a) Show that min{U1 , U2 } ∼ triangular(a, b, 0) and max{U1 , U2 } ∼ triangular(a, b, 1).
(b) Show that |U2 − U1 | ∼ triangular(0, b, 0).
(c) Show that U1 + U2 ∼ triangular(2a, 2b, 1/2) and U2 − U1 ∼ triangular(−b, 2b, 1/2).
In particular, if U1 and U2 are i.i.d. random variables with the uniform(0, 1) distribution, then we have U1 + U2 ∼
triangular(0, 2, 1/2). This is the Irwin-Hall distribution with n = 2. [Ans]
35.13 Link with the beta distribution. Show that triangular(0, 1, 0) = beta(1, 2) and triangular(0, 1, 1) = beta(2, 1). [Ans]
Page 106 §35 Mar 10, 2020(20:25) Bayesian Time Series Analysis

The sine distribution


35.14 Suppose X ∼ sinD (a, b) where a ∈ R and b ∈ (0, ∞). Show that X = a + bY where Y ∼ sinD (0, 1).
Hence show that the family of distributions {sinD (a, b) : a ∈ R and b ∈ (0, ∞) } is a location-scale family of
distributions—see definition(1.6b) on page 5. [Ans]
35.15 Distribution function, quantile function and m.g.f. of the sine distribution. Suppose X ∼ sinD (a, b) where a ∈ R and
b ∈ (0, ∞).
(a) Show that the distribution function of X is given by
x−a
  
1
FX (x) = 1 − cos π for x ∈ [a, a + b].
2 b
(b) Show that the quantile function of X is
−1 b
FX (p) = a + arccos(1 − 2p) for p ∈ (0, 1).
π
and the median is a + b/2.
(c) Show that the m.g.f. of X is given by
π 2 (eat + e(a+b)t )
MX (t) = for t ∈ R. [Ans]
2(b2 t2 + π 2 )
35.16 Moments of the sine distribution.
(a) Suppose X ∼ sinD (0, 1). Show that E[X] = 1/2, E[X 2 ] = 1/2 − 2/π2 , var[X] = 1/4 − 2/π2 , E[X 3 ] = 1/2 − 3/π2 and
E[X 4 ] = 1/2 + 24/π4 − 6/π2 .
(b) Suppose X ∼ sinD (a, b) where a ∈ R and b ∈ (0, ∞). Show that E[X] = a + b/2, E[X 2 ] = a2 + b2 ( 1/2 − 2/π2 ) + ab,
and var[X] = b2 ( 1/4 − 2/π2 ). [Ans]
35.17 Skewness and kurtosis of the sine distribution. Suppose X ∼ sinD (a, b) where a ∈ R and b ∈ (0, ∞).
(a) Show that the skewness of X is skew[X] = 0.
(b) Show that the kurtosis of X is κ[X] = (384 − 48π 2 + π 4 )/(π 2 − 8)2 . [Ans]

The U-power distribution


35.18 Suppose X ∼ Upower(a, b, k) where a ∈ R, b ∈ (0, ∞) and k ∈ {1, 2, . . .}. Show that X = a + bY where Y ∼
Upower(0, 1, k).
Hence show that for every k ∈ {1, 2, . . .}, the family of distributions {Upower(a, b, k) : a ∈ R and b ∈ (0, ∞) } is a
location-scale family of distributions—see definition(1.6b) on page 5. [Ans]
35.19 Distribution function and quantile function of the Upower distribution. Suppose X ∼ Upower(a, b, k) where a ∈ R,
b ∈ (0, ∞) and k ∈ {1, 2, . . .}.
(a) Show that the distribution function of X is given by
" 2k+1 #
x−a

1
FX (x) = 1+ for x ∈ [a − b, a + b]. (35.19a)
2 b

(b) Show that the quantile function of X is


−1
FX (p) = a + b(2p − 1)1/(2k+1) for p ∈ (0, 1).
and the median is a.
It hfollows that if Ui ∼ uniform(0, 1) then a + b(2U − 1)1/(2k+1) ∼ Upower(a, b, k) and if X ∼ Upower(a, b, k) then
1 x−a 2k+1

2 1+ b ∼ uniform(0, 1). [Ans]

35.20 Moments of the U-power distribution.


(a) Suppose X ∼ Upower(0, 1, k) where k ∈ {1, 2, . . .}. Show that

n 0 if n is odd;
E[X ] = 2k+1
n+2k+1 if n is even.
(b) Deduce that E[X] = 0 and var[X] = (2k + 1)/(2k + 3).
(c) Suppose X ∼ Upower(a, b, k) where a ∈ R, b ∈ (0, ∞) and k ∈ {1, 2, . . .}. Show that

0 if n is odd;
E[(X − a)n ] =
bn n+2k+1
2k+1
if n is even.
Deduce that E[X] = a and var[X] = b2 (2k + 1)/(2k + 3). [Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §36 Page 107

35.21 Skewness and kurtosis of the U-power distribution. Suppose X ∼ Upower(a, b, k) where a ∈ R, b ∈ (0, ∞) and
k ∈ {1, 2, . . .}.
(a) Show that the skewness of X is skew[X] = 0.
(b) Show that the kurtosis of X is
(2k + 3)2
κ[X] = [Ans]
(2k + 1)(2k + 5)
35.22 (a) Suppose Xk ∼ Upower(0, 1, k) for k ∈ {1, 2, . . .}. Show that
D
Xk =⇒ Y as k → ∞ where Y is a discrete random variable with P[Y = −1] = P[Y = 1] = 1/2.
(b) Suppose a ∈ R, b ∈ (0, ∞) and Xk ∼ Upower(a, b, k) for k ∈ {1, 2, . . .}.
D
Xk =⇒ Y as k → ∞ where Y is a discrete random variable with P[Y = a − b] = P[Y = a + b] = 1/2. [Ans]

36 Other distributions with unbounded support



36.1 The chi distribution. Suppose X ∼ χ2n where n ∈ (0, ∞). Then Y = X has the chi distribution with n
degrees of freedom. This distribution is denoted the χn distribution.
0.8
n=1
n=2
0.6 n=3
n=6

0.4

0.2

0.0
0 1 2 3 4 5
Figure(36.1a). Density of the chi distribution for n = 1, n = 2, n = 3 and n = 6.
(wmf/chiDistribution,72mm,54mm)

36.2 The Maxwell distribution. q Suppose β > 0 Then the random variable X has the Maxwell (β) distribution
iff X has the same distribution as N12 + N22 + N32 where N1 , N2 and N3 are i.i.d. random variables with the
N (0, β) distribution.

Relation to other distributions. Clearly if X ∼ Maxwell (1) then X 2 ∼ χ23 . Conversely if Y ∼ χ23 then Y ∼
Maxwell (1). Finally, the Maxwell (1) distribution is the same as the chi distribution with 3 degrees of freedom.
Density of the Maxwell distribution. Now
s
N12 N22 N32 √
X=β + + = β Y where Y ∼ χ23 .
β2 β2 β2
Hence  2 r
2x x 2 x2 −x2 /(2β 2 )
fX (x) = 2 fY 2
= e for x ∈ (0, ∞).
β β π β3
For the shape of the density see exercise 37.8 on page 112.
Distribution function of the Maxwell distribution. For x ∈ (0, ∞) we have
Z x r Z
1 2 x u −u2 /(2β 2 )
Fx (x) = fX (u) du = u 2e du
0 β π 0 β
r " Z x/β −v2 /2 #

r  x Z x 
1 2 2 2 2 2 1 2 2 2 e
= −ue−u /(2β ) + e−u /(2β ) du = −xe−x /(2β ) + β 2π √ dv

β π 0 0 β π 0 2π

r     
1 2 −x2 /(2β 2 ) x 1
= −xe + β 2π Φ −
β π β 2
  r
x 1 2 −x2 /(2β 2 )
= 2Φ −1− xe
β β π
Page 108 §36 Mar 10, 2020(20:25) Bayesian Time Series Analysis

Multiple of a Maxwell distribution. Suppose X ∼ Maxwell (β) and α ∈ (0, ∞). Let Y = αX. Then Y has density

dx 1  y  r 2 y2 2 2 2
fY (y) = fX (x) = fX = 3 3
e−y /(2α β )
dy α α πα β
Hence Y ∼ Maxwell (αβ). It follows that the family of distributions { Maxwell (β) : β ∈ (0, ∞) } is a scale
family of distributions—see definition(1.6d) on page 5.
Moment generating function of the Maxwell distribution. First, consider the m.g.f. of the Maxwell (1) distribution.
For t ∈ R we haver r
2 ∞ 2 tx−x2 /2 2 t2 /2 ∞ 2 −(x−t)2 /2
Z Z
tX
E[e ] = x e dx = e x e dx
π x=0 π x=0
r
2 t2 /2 ∞
Z
2
= e (v + t)2 e−v /2 dv
π v=−t
r Z ∞ Z ∞ Z ∞ 
2 t2 /2 −v 2 /2 −v 2 /2 2 −v 2 /2
= e v ve dv + 2t ve dv + t e dv
π v=−t v=−t v=−t
√ √
r r
2 t2 /2 h 2 2
i 2 2
= e −te−t /2 + 2πΦ(t) + 2te−t /2 + 2π t2 Φ(t) = t + 2(1 + t2 )et /2 Φ(t)
π π
It follows that if X ∼ Maxwell (β) then X has m.g.f.
r
tX 2 2 2
E[e ] = βt + 2(1 + β 2 t2 )eβ t /2 Φ(βt)
π
Moments of the Maxwell distribution. See exercise 37.9 and 37.10 on page 112.
36.3 The exponential logarithmic distribution. Suppose p ∈ (0, 1) and b ∈ (0, ∞); then the random variable
X has the exponential logarithmic distribution, denoted expLog(b, p), iff X has density
(1 − p)e−x/b
fX (x) = − for x ∈ (0, ∞). (36.3a)
b ln(p)[1 − (1 − p)e−x/b ]
Illustrative plots of the density are given in figure(36.3a).
2.0

1.5

1.0

0.5

0.0
0 1 2 3 4
Figure(36.3a). Plot of the expLog(1, 0.25) density in black and the expLog(1, 0.75) density in green.
The exponential density 2e−2x is the dotted line in red.
(wmf/expLog,72mm,54mm)

36.4 The log-logistic distribution. Suppose k ∈ (0, ∞) and b ∈ (0, ∞); then the random variable X has the
log-logistic distribution, denoted loglogistic(k, b), iff X has density
bk kxk−1
f (x) = k for x ∈ (0, ∞). (36.4a)
(b + xk )2
An alternative expression is
k x k−1

b b
f (x) = 2 for x ∈ (0, ∞).
1 + ( xb )k
The distribution function of the loglogistic(k, b) distribution. Integrating equation(36.4a) gives
xk
F (x) = k for x ∈ [0, ∞). (36.4b)
b + xk
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §36 Page 109

The quantile function is


 1/k
−1 p
F (p) = b for p ∈ [0, 1). (36.4c)
1−p
and the median is F −1 ( 1/2) = b.
Shape of the density function. The derivative of f is
bk (k − 1) − (k + 1)xk
f 0 (x) = kbk xk−2
(bk + xk )3
Hence if k ∈ (0, 1) then f is decreasing with f (x) → ∞ as x → 0. If k = 1, then f is decreasing with mode at
x = 0. If k > 1, then f first increases and then decreases with mode at
k − 1 1/k
 
x=b
k+1
Further results about the shape of the density can be found in exercise 37.17.
Distribution of a multiple of a log-logistic distribution. Suppose X ∼ loglogistic(k, b) where b ∈ (0, ∞) and k ∈
(0, ∞). Suppose further that α > 0 and Y = αX. Applying the usual transformation formula to equation(36.4a)
shows that Y ∼ loglogistic(k, αb).
It follows that if k ∈ (0, ∞) is fixed, then the family of distributions {loglogistic(k, b) : b ∈ (0, ∞)} is a scale
family of distributions—see definition(1.6d) on page 5.
The standard log-logistic distribution. The standard log-logistic distribution is loglogistic(1, 1) and this has den-
sity
1
f (x) = for x ∈ (0, ∞). (36.4d)
(1 + x2 )2
This is the same as the beta0 (1, 1) distribution.
36.5 The hyperbolic secant distribution. Suppose µ ∈ R and σ ∈ (0, ∞); then the random variable X has the
hyperbolic secant distribution, which we shall denote as Hsecant(µ, σ), iff X has density
π x−µ
  
1
fX (x) = sech for x ∈ R. (36.5a)
2σ 2 σ
The density of the Hsecant(0, 1) distribution is
1  πx  1
fX (x) = sech = πx/2 for x ∈ R. (36.5b)
2 2 e + e−πx/2

0.5

0.4

0.3

0.2

0.1

0.0
−3 −2 −1 0 1 2 3

Figure(36.5a). Plot of the Hsecant(0, 1) density in black and the N (0, 1) density in red.
The Hsecant(0, 1) distribution has the higher peak.
(wmf/Hsecant,72mm,54mm)

Shape of the Hsecant(0, 1) density. Differentiating equation(36.5b) gives, for x ∈ R


0 π eπx/2 − e−πx/2
fX (x) = −
2 [eπx/2 + e−πx/2 ]2
00 π 2 [eπx/2 + e−πx/2 ]3 − 2[eπx/2 − e−πx/2 ]2 [eπx/2 + e−πx/2 ]
fX (x) = −
4 [eπx/2 + e−πx/2 ]4
π 2 [eπx/2 + e−πx/2 ]2 − 2[eπx/2 − e−πx/2 ]2
=−
4 [eπx/2 + e−πx/2 ]3
Page 110 §37 Mar 10, 2020(20:25) Bayesian Time Series Analysis

π 2 [eπx/2 − e−πx/2 ]2 − 4
 
=
4 [eπx/2 + e−πx/2 ]3
So fX is initially increasing for x < 0 and then decreasing for x > 0; hence the mode is at x = 0. Also fX
is initially convex, √then concave
√ and then convex
√ again with points of inflection
√ when [eπx/2 − e−πx/2 ]2 = 4,
i.e. when e πx/2 = 2 + 1 or 2 − 1 = 1/( 2 + 1); hence πx/2 = ± ln( 2 + 1). Clearly the density fX is
symmetric about 0 because the sec function is symmetric about 0.
36.6 The Gompertz distribution. The Gompertz distribution has an exponentially increasing failure rate; for
this reason it is often used to model human lifetimes.
The definition. Suppose Y has the reverse Gumbel distribution defined in equation(30.4a) on page 91. Hence Y
has density fY (y) = exp [−ey + y] and distribution function FY (y) = 1 − exp [−ey ] for y ∈ R.
Define the random variable X to be the same as Y given Y > 0, then X has density
fY (x) ex exp[−ex ]
= ex exp −(ex − 1) for x ∈ (0, ∞).
 
f (x) = = −1
P[Y > 0] e
This is defined to be the Gompertz(1, 1) distribution and is generalised as follows.
Definition(36.6a). Suppose a ∈ (0, ∞) and b ∈ (0, ∞). Then the random variable X has the Gompertz
distribution, Gompertz(a, b), iff X has the density
a h i
f (x) = ex/b exp −a(ex/b − 1) for x ∈ [0, ∞). (36.6a)
b
Distribution function. The distribution function of i b) distribution is
h the Gompertz(a,
F (x) = 1 − exp −a(ex/b − 1) for x ∈ [0, ∞). (36.6b)
Multiple of a Gompertz distribution. Suppose X ∼ Gompertz(a, b) and c ∈ (0, ∞). Then cX ∼ Gompertz(a, bc).
It follows that if a ∈ (0, ∞) is fixed, then the family of distributions { Gompertz(a, b) : b ∈ (0, ∞) } is a scale
family of distributions—see definition(1.6d) on page 5.
Mean of the Gompertz distribution.Z This is
aea ∞ x/b
E[X] = xe exp[−aex/b ] dx
b 0
Z ∞
a
= abe ln y exp[−ay] dy by using the transformation y = ex/b .
1
Z ∞
a 1 −ay
= be e dy by integrating by parts
1 y
Z ∞
1 −z
= bea e dz
a z
and this is as far as we can get—the integral is a version of the exponential integral which is considered in
[A BRAMOWITZ &S TEGUN(1965)].

37 Exercises (exs-other.tex)

The chi (χ) distribution


37.1 The density and distribution function of the chi distribution.
(a) Suppose Y ∼ χn . Show that the density of Y is
2
y n−1 e−y /2
fY (y) = (n−2)/2 for y > 0. (37.1a)
2 Γ(n/2)
(b) Show that the distribution function of Y is
2 Z x
Γ( n/2, y /2)
FY (y) = where Γ(n, x) is the incomplete gamma function: Γ(n, x) = y n−1 e−y dy [Ans]
Γ( n/2) 0
37.2 Links between the chi distribution and other distributions.
d
(a) Suppose X ∼ χ1 . Show that X = |Y | where Y ∼ N (0, 1). More generally, show that if X ∼ χ1 then σX ∼
folded (0, σ 2 ), which is the half-normal distribution. See exercise 16.25 and exercise 16.23 on page 51.
(b) Suppose X ∼ χ2 . Show that X ∼ Rayleigh (1).
(c) Suppose X ∼ χ3 . Show that X has the standard Maxwell density
r
2 2 −y2 /2
fY (y) = y e for y > 0. [Ans]
π
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §37 Page 111

37.3 Moments of the chi distribution. Suppose n ∈ (0, ∞) and X ∼ χn . Show that

(n+k)/2
k k/2 Γ
E[X ] = 2 for k ∈ (0, ∞). (37.3a)
Γ( n/2)
In particular
  !2
(n+1)/2 (n+1)/2
1/2 Γ Γ
E[X] = 2 E[X 2 ] = n var[X] = n − µ2 = n − 2 [Ans]
Γ( n/2) Γ( n/2)
37.4 Skewness and kurtosis of the chi distribution. Suppose X has the chi distribution with n degrees of freedom. Show that
µ(1 − 2σ 2 ) 2 − 2µσ skew[X] + σ 2
skew[X] = and κ[X] = [Ans]
σ3 σ2
37.5 Shape of the density of the chi distribution. Denote the density of χn by f .
p
(a) Suppose n = 1. Show that f is decreasing with mode at 0 equal to f (0) = 2/π . Show that f is concave for
0 < x < 1 and convex for x > 1 with point of inflection at x = 1.
(b) Suppose 0 < n < 1. Show that f is decreasing with f (x) → ∞ as x → 0. √
Suppose n > 1. Show that f increases and then decreases with mode at x = n − 1.
(c) If n ≤ 7/8 show that f (x) is convex for all x.
If 7/8 < n < 1, show that f is initially convex, then concave and then convex again with two points of inflection at
q
1

2 [2n − 1 ± 8n − 7].
q √
If 1 ≤ n ≤ 2 show that f is initially concave and then convex with one point of inflection at 12 [2n − 1 + 8n − 7].
If n > 2, show that f is initially convex, then concave and then convex again with two points of inflection at
q
1

2 [2n − 1 ± 8n − 7] [Ans]

37.6 For x > − 1/2, let


 2
Γ(x + 1)
θ(x) = −x
Γ(x + 1/2)
Then θ(x) → 1/4 as n → ∞. This result is proved in [WATSON(1959)].
For n ∈ {1, 2, . . .}, suppose Xn has the χn distribution and let µn = E[Xn ].
(a) By using Watson’s result, prove that
E[Xn ]
→ 1 as n → ∞, and E[Xn ] − n1/2 → 0 as n → ∞.
n1/2
(b) By using Watson’s result, prove that
1
var[Xn ] → as n → ∞. [Ans]
2
37.7 Asymptotic normality of the chi distribution. Suppose {Vn } is a sequence of random variables such that Vn ∼ χ2n for
every n ∈ {1, 2, . . .}. By the central limit theorem we know that
Vn − n D
√ =⇒ N (0, 1) as n → ∞.
2n
(a) By using the delta17 method, prove that if {Wn } is a sequence of random variables such that Wn ∼ χn , then

Wn − n D
√ =⇒ N (0, 1) as n → ∞.
1/ 2
(b) Prove that if {Wn } is a sequence of random variables such that Wn ∼ χn , then
W n − µn D
=⇒ N (0, 1) as n → ∞, where µn = E[Wn ] and σn2 = var[Wn ]. [Ans]
σn

17
The delta method is the following theorem: suppose {Xn } is a sequence of random variables such that
Xn − µ D
=⇒ N (0, 1) as n → ∞.
σn
where µ ∈ R and {σn } is a sequence in (0, ∞) such that σn → 0 as n → ∞. Suppose further that g is a real-valued function
which is differentiable at µ and g 0 (µ) 6= 0. Then
g(Xn ) − g(µ) D
=⇒ N (0, 1) as n → ∞.
g 0 (µ)σn
Page 112 §37 Mar 10, 2020(20:25) Bayesian Time Series Analysis

The Maxwell distribution


37.8 Shape of the Maxwell density. Suppose X ∼ Maxwell (β) with density function fX .

(a) Show that fX initially increases and then decreases with mode at x = β 2.
(b) Show that fX is initially convex, then concave and then convex again with points of inflection at
s √ s √
5 − 17 5 + 17
x1 = β and x2 = β [Ans]
2 2
37.9 Moments of the Maxwell distribution.Suppose X ∼ Maxwell (β).
p
(a) Show that E[X] = 2β 2/π and var[X] = β 2 (3 − 8/π).
(b) Show that
2n/2+1 β n
 
n+3
E[X n ] = √ Γ for n ∈ {1, 2, 3, . . .}. [Ans]
π 2
37.10 Skewness and kurtosis of the Maxwell distribution. Suppose X ∼ Maxwell (β). Show that

2 2(16 − 5π) 15π 2 + 16π − 192
skew[X] = and κ[X] = [Ans]
(3π − 8)3/2 (3π − 8)2

The exponential logarithmic distribution


37.11 (a) Check that equation(36.3a) defines a density function.
(b) Show that the density of expLog(b, p) is decreasing and convex on (0, ∞) and hence has a unique mode at 0.
(c) Find the distribution function of the expLog(b, p) distribution.
(d) Find the quartile function of the expLog(b, p) distribution and find an expression for the median. [Ans]
37.12 (a) Suppose X ∼ expLog(b, p). Show that X = bY where Y ∼ expLog(1, p).
Definition(1.6d) on page 5 implies that for fixed p ∈ (0, 1), the family of distributions {expLog(b, p) : b > 0} is a
scale family of distributions.
(b) Find hX (x), the hazard function of the expLog(b, p) distribution and show hX (x) = b1 hY ( xb ) where X = bY .
(c) Show that the hazard function of the expLog(b, p) distribution is decreasing on [0, ∞). Thus the exponential-
logarithmic distribution can be used to model objects which improve with age. [Ans]
37.13 Moments of the expLog(b, p) distribution. For n ∈ {1, 2, . . .} show that

bn n! X (1 − p)k
E[X n ] = −
ln(p) k n+1
k=1
Hence show that E[X n ] → 0 as p → 0 and E[X n ] → bn n! as p → 1. [Ans]
37.14 (a) Suppose U ∼ uniform(0, 1), p ∈ (0, 1) and b ∈ (0, ∞). Show that
1−p
 
X = b ln ∼ expLog(b, p)
1 − pU
(b) Suppose X ∼ expLog(b, p) where p ∈ (0, 1) and b ∈ (0, ∞). Show that
ln[1 − (1 − p)e−X/b ]
∼ uniform(0, 1) [Ans]
ln(p)
37.15 Suppose Y1 , Y2 , . . . are i.i.d. random variables with the exponential (λ) distribution. Suppose further that N is indepen-
dent of {Y1 , Y2 , . . .} and has the logarithmic distribution:
(1 − p)n
P[N = n] = − for n ∈ {1, 2, 3, . . .}.
n ln(p)
Show that
X = min{Y1 , Y2 , . . . , YN } ∼ expLog( 1/λ, p) [Ans]
37.16 Suppose b ∈ (0, ∞) and Xn ∼ expLog(b, pn ).
D
(a) Show that Xn =⇒ 1 as pn → 0.
D
(b) Show that Xn =⇒ exponential (1/b) as pn → 1. [Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §37 Page 113

The log-logistic distribution


37.17 Shape of the density function. Suppose X ∼ loglogistic(k, b) has density function f .
(a) Show that the second derivative of f is
b2k (k − 1)(k − 2) − 4(k 2 − 1)xk bk + (k + 1)(k + 2)x2k
f 00 (x) = kbk xk−3
(bk + xk )4
(b) Suppose k ∈ (0, 1]. Show that f is convex.
(c) Suppose k ∈ (1, 2]. Show that f is initially concave and then convex with point of inflection at
" p #1/k
2(k 2 − 1) + k 3(k 2 − 1)
x2 = b
(k + 1)(k + 2)

(d) Suppose k > 2. Show that f is initially convex, then concave and then convex again with points of inflection at
" p #1/k
2(k 2 − 1) − k 3(k 2 − 1)
x1 = b
(k + 1)(k + 2)
and at x2 . [Ans]
37.18 The hazard function of the log-logistic distribution. Suppose X ∼ loglogistic(k, b).
(a) Show that the hazard function of X is
kxk−1
h(x) = for x ∈ (0, ∞).
bk + xk
(b) Show that if k ∈ (0, 1] then h is decreasing. Show that if k > 1 then h first decreases and then increases with
minimum at x = b(k − 1)1/k . [Ans]
37.19 Moments of the log-logistic distribution. Suppose X ∼ loglogistic(k, b) where k ∈ (0, ∞) and b ∈ (0, ∞).
(a) By using Euler’s reflection formula, (13.10a), show that
(
∞ if n ≥ k;
n
E[X ] = bn B 1 − n , 1 + n  = bn πn/k
if n ∈ [0, k).
k k sin(πn/k)

(b) In particular, show that if k > 1 then


 
1 1 (π/k)
E[X] = b B 1 − , 1 + =b
k k sin(π/k)
and if k > 2 then  
2 22 2 (2π/k)
E[X ] = b B 1 − , 1 + = b2 [Ans]
k k sin(2π/k)
37.20 Power transformation of a log-logistic distribution. Suppose X ∼ loglogistic(k, b) and Y = X n where n ∈ (0, ∞).
Show that Y ∼ loglogistic(k/n, bn ).
In particular, if X ∼ loglogistic(1, 1) then Y = bX 1/k ∼ loglogistic(k, b). [Ans]
37.21 (a) Suppose U ∼ uniform(0, 1), b ∈ (0, ∞) and k ∈ (0, ∞). Show that X = b[U/(1 − U )]1/k ∼ loglogistic(k, b).
(b) Suppose X ∼ loglogistic(k, b). Show that Y = X k /(bk + X k ) ∼ uniform(0, 1).
(c) Suppose X ∼ loglogistic(k, b). Show that Y = ln X ∼ logistic( ln b, σ 2 = π 2 /(3k 2 ) ).
(d) Suppose X ∼ logistic(a, σ 2 = π 2 /(3k 2 ) ). Show that Y = eX ∼ loglogistic(k, ea ). [Ans]
37.22 Limiting distribution of the log-logistic distribution.
D
(a) Suppose Xk ∼ loglogistic(k, 1). Show that Xk =⇒ 1 as k → ∞.
D
(b) Suppose Xk ∼ loglogistic(k, b). Show that Xk =⇒ b as k → ∞.

The hyperbolic secant distribution


37.23 (a) Check that equation(36.5b) defines a density function.
(b) Suppose µ ∈ R and σ ∈ (0, ∞). Show that if X ∼ Hsecant(µ, σ), then X = µ + σY where Y ∼ Hsecant(0, 1).
Hence deduce that {Hsecant(µ, σ) : µ ∈ R, σ ∈ (0, ∞) } forms a location-scale family of distributions—see
definition(1.6b) on page 5.
Note that parts(a) and (b) show that equation(36.5a) defines a density function.
(c) Shape of the Hsecant(µ, σ) density. Suppose fX is the density of the Hsecant(µ, σ) distribution. Show that fX is
symmetric about µ and fX increases for x < µ and decreases for x > µ with mode at x = µ. Show√also that fX is
initially convex, then concave and then convex again with points of inflection when x = µ ± π2 σ ln( 2 + 1). [Ans]
Page 114 §37 Mar 10, 2020(20:25) Bayesian Time Series Analysis

37.24 The distribution function and quantile function of the hyperbolic secant distribution.
(a) Suppose X ∼ Hsecant(0, 1). Show that the distribution function of X is
2 h  πx i
FX (x) = arctan exp for x ∈ R.
π 2
(b) Suppose X ∼ Hsecant(µ, σ). Show that the distribution function of X is
π(x − µ)
  
2
FX (x) = arctan exp for x ∈ R.
π 2σ
(c) Suppose X ∼ Hsecant(µ, σ). Show that the quantile function of X is
−1 2σ h  πp i
FX (p) = µ + ln tan for p ∈ (0, 1).
π 2
Hence deduce that the median of X is µ. [Ans]
37.25 Moment generating function and characteristic function.
(a) Suppose X ∼ Hsecant(0, 1). Show that the m.g.f. of X is MX (t) = sec(t) for t ∈ (−π/2, π/2) and the characteristic
function of X is φX (t) = E[eitX ] = sech(t) for t ∈ R.
(b) Suppose X ∼ Hsecant(µ, σ). Show that
MX (t) = eµt sec(σt) for t ∈ − 2σ
π π
and φX (t) = E[eitX ] = eµt sech(σt) for t ∈ R.

, 2σ
Hint for part(a). First show that

e−cαw
Z
B(α, β) = c dw
w=−∞ (1 + e−cw )α+β
and then deduce that for any q > p > 0 we have
Z ∞
e−pw
q −qw
dw = B(p/q, 1 − p/q)
w=−∞ 1 + e
Then use the fact that B(x, 1 − x) = π/ sin(πx) for any x ∈ (0, 1). [Ans]
37.26 Moments of the hyperbolic secant distribution.
(a) Suppose X ∼ Hsecant(0, 1). Clearly all odd moments are 0 because the density is symmetric about 0. Show that
E[X] = 0 E[X 2 ] = 1 E[X 3 ] = 0 E[X 4 ] = 5
Hence deduce var[X] = 1, skew[X] = 0 and κ[X] = 5.
(b) Suppose X ∼ Hsecant(µ, σ). For this case, all odd moments about µ are zero. Show that
E[X] = µ E[X 2 ] = σ 2 + µ2
Hence deduce var[X] = σ 2 , skew[X] = 0 and κ[X] = 5. [Ans]

The Gompertz distribution


37.27 Suppose X ∼ Gompertz(a, b).
(a) Find the quantile function of X and show that the median is b ln(1 + ln 2/a).
(b) Show that X has hazard function
a x/b
h(x) = e for x ∈ [0, ∞).
b
and hence X has an exponentially increasing hazard rate. [Ans]
37.28 Shape of the Gompertz density. Suppose X ∼ Gompertz(a, b) and f denotes the density of X.
(a) Suppose a < 1. Show that f first increases and then decreases with mode at x = −b ln a.
(b) Suppose a ≥ 1. Show that f is decreasing with mode at x = 0.
(c) Find the second derivative of f .

(d) Suppose a < (3 − 5)/2. √Show that f is initially convex√ and then concave and then convex again with points of
inflection at x1 = b ln[(3 − 5)/2a] and x2 = b ln[(3 + 5)/2a].
√ √
(e) Suppose (3 − √5)/2 ≤ a < (3 + 5)/2. Show that f is initially concave and then convex with point of inflection at
x2 = b ln[(3 + 5)/2a].

(f) Suppose a ≥ (3 + 5)/2. Show that f is convex. [Ans]
37.29 Suppose b ∈ (0, ∞) and the random variable X has the reverse Gumbel (0, b) distribution. The density of this distribution
is given in equation(30.5a) on page 91. Show that the conditional distribution of X given X ≥ 0 is the Gompertz(1, b)
distribution. [Ans]
2 Univariate Continuous Distributions Mar 10, 2020(20:25) §37 Page 115

37.30 Link between the Gompertz and uniform distributions.


(a) Suppose X ∼ uniform(0, 1), a ∈ (0, ∞) and b ∈ (0, ∞). Show that
 
1
Y = b ln 1 − ln X ∼ Gompertz(a, b)
a
(b) Suppose X ∼ Gompertz(a, b). Show that
h i
Y = exp −a(eX/b − 1) ∼ uniform(0, 1) [Ans]

37.31 Link between the Gompertz and exponential distributions.


(a) Suppose X ∼ Gompertz(a, b). Show that Y = eX/b − 1 ∼ exponential (a).
X

(b) Suppose X ∼ exponential (1), a ∈ (0, ∞) and b ∈ (0, ∞). Show that Y = b ln a + 1 ∼ Gompertz(a, b). [Ans]

The Linnik distribution


37.32 (a) Suppose X and Y are independent random variables such that X ∼ exponential (1) and Y ∼ Cauchy(1). Show that
the characteristic function of the random variable Z = XY is
1
φZ (t) = E[eitZ ] =
1 + |t|
(b) A generalization of part (a). Suppose X and Y are independent random variables such that X ∼ exponential (1) and
α
Y has the symmetric stable distribution with characteristic function φY (t) = E[eitY ] = e−|t| where 0 < α < 2.
Show that the characteristic function of the random variable Z = X 1/α Y is
1
φZ (t) = E[eitZ ] =
1 + |t|α
This distribution is called the Linnik distribution. [Ans]
Page 116 §37 Mar 10, 2020(20:25) Bayesian Time Series Analysis
CHAPTER 3

Multivariate Continuous Distributions

38 General results
38.1 The mean and variance matrices. If
X1

.
X =  .. 
n×1
Xn
is a random vector, then, provided the univariate expectations E[X1 ], . . . , E[Xn ] exist, we define
E[X1 ]
 
.
E[X] =  .. 
n×1
E[Xn ]
Provided the second moments E[X12 ], . . . , E[Xn2 ] are finite, the variance matrix or covariance matrix of X is the
n × n matrix
var[X] = E[(X − µ)(X − µ)T ] where µ = E[X].
n×n

The (i, j) entry in the variance matrix is cov[Xi , Xj ]. In particular, the ith diagonal entry is var[Xi ].
Clearly:
• the variance matrix is symmetric;
• if X1 , . . . , Xn are i.i.d. with variance σ 2 , then var[X] = σ 2 I;
• var[X] = E[XXT ] − µµT ; (38.1a)
• we shall denote the variance matrix by Σ or ΣX .
We shall often omit stating “when the second moments are finite” when it is obviously needed. Random vectors
will be nearly always column vectors, but may be written in text as row vectors in order to save space.

38.2 Linear transformations. If Y = X + a then var[Y] = var[X].


More generally, if Y = A + BX where A is m × 1 and B is m × n, then µY = A + BµX and
   
var[Y] = E (Y − µY )(Y − µY )T = E B(X − µX )(X − µX )T )BT = B var[X] BT

In particular, if a = (a1 , . . . , an ) is a 1 × n vector, then aX = ni=1 ai Xi is a random variable and


P

n X
X n
var[aX] = a var[X] aT = ai aj cov[Xi , Xj ]
i=1 j=1

Example(38.2a). Suppose the random vector X = (X1 , X2 , X3 ) has variance matrix


6 2 3
" #
var[X] = 2 4 0
3 0 2
Let Y1 = X1 + X2 and Y2 = X1 + X2 − X3 . Find the variance matrix of Y = (Y1 , Y2 ).
Solution. Now Y = AX where  
1 1 0
A=
1 1 −1
Hence  
14 11
var[Y] = var[AX] = Avar[X]AT =
11 10

Bayesian Time Series Analysis by R.J. Reed Mar 10, 2020(20:25) §38 Page 117
Page 118 §38 Mar 10, 2020(20:25) Bayesian Time Series Analysis

38.3 Positive definiteness of the variance matrix. Suppose X is an n × 1 random vector. Then for any n × 1
vector c we have
cT var[X]c = var[cT X] ≥ 0 (38.3a)
Hence var[X] is positive semi-definite (also called non-negative definite).
Proposition(38.3a). Suppose X is a random vector with finite second moments and such that no element of X
is a linear combination of the other elements. Then var[X] is a symmetric positive definite matrix.
Proof. No element of X is a linear combination of the other elements; this means that if a is an n × 1 vector with aT X
constant then we must have a = 0.
Now suppose var[cT X] = 0; this implies cT X is constant and hence c = 0. Hence cT var[X]c = 0 implies var[cT X] = 0
which implies c = 0. This result, together with equation(38.3a) shows that var[X] must be positive definite.

Example(38.3b). Consider the random vector Z = (X, Y, X + Y )T where µX = E[X], µY = E[Y ] and ρ = corr[X, Y ].
Show that var[Z] is not positive definite.
Solution. Let a = (1, 1, −1). Then a var[Z] aT = var[aZ] = var[0] = 0.

38.4 The square root of a variance matrix; the transformation to independent random variables Suppose
C is a real symmetric positive definite n × n matrix. Because C is real and symmetric, we can write C = LDLT
where L is orthogonal1 and D = diag(d1 , . . . , dn ) is diagonal and d1 , . . . , dn are the eigenvalues of C. Because
C is also positive definite, we have d1 > 0, . . . , dn > 0. Hence we can write C = (LD1/2 LT )(LD1/2 LT ) = QQ
where Q is symmetric and nonsingular.
If we only know C is real, symmetric and non-negative definite, then we only have d1 ≥ 0, . . . , dn ≥ 0. We can
still write C = (LD1/2 LT )(LD1/2 LT ) = QQ where Q is symmetric but Q is now possibly singular.

Now suppose X is a random vector with finite second moments and such that no element of X is a linear combi-
nation of the other elements; then var[X] is a real symmetric positive definite matrix. Hence var[X] = QQ and
if Y = Q−1 X then var(Y) = Q−1 var[X] (Q−1 )T = I. This means that if X is a random vector with finite second
moments and such that no element of X is a linear combination of the other elements, then there exists a linear
transformation of X to independent variables.

38.5 The covariance between two random vectors.


Definition(38.5a). If X is an m × 1 random vector with finite second moments and Y is an n × 1 random vector
with finite second moments, then cov[X, Y] is the m × n matrix with (i, j) entry equal to cov[Xi , Yj ].
Because the (i, j) entry of cov[X, Y] equals cov[Xi , Yj ], it follows that
cov[X, Y] = E[(X − µX )(Y − µy )T ] = E[XYT ] − µX µTY
Also:
• because cov[Xi , Yj ] = cov[Yj , Xi ], it follows that cov[X, Y] = cov[Y, X]T ;
• if n = m, the covariance matrix cov[X, Y] is symmetric;
• cov[X, X] = var[X];
• we shall often denote the covariance matrix by Σ or ΣX,Y .

38.6 The correlation matrix. Suppose the √ n × 1 random vector X has the variance matrix Σ. Let D be the
n × n diagonal matrix with diagonal equal to diag(Σ). Then the correlation matrix of X is given by
corr[X] = D−1 ΣD−1
Clearly, the (i, j) element of corr[X] is corr(Xpi , Xj ). Also, corr[X] is the variance matrix of the random vector
Z = (Z1 , . . . , Zn ) where Zj = (Xj − E[Xj ])/ var(Xj ).
Conversely, given corr[X] we need the vector of standard deviations in order to determine the variance  matrix. In
fact, var[X] = D corr[X] D where D is the diagonal matrix with entries stdev(X1 ), . . . , stdev(Xn ) .

1
Orthogonal means that LT L = I and hence L−1 = LT . Because C = LDLT , we have LT CL = D and hence LT (C − λI)L =
D − λI; hence |C − λI| = |D − λI| and the eigenvalues of C equal the eigenvalues of D—see page 39 of [R AO(1973)].
3 Multivariate Continuous Distributions Mar 10, 2020(20:25) §38 Page 119

38.7 Quadratic forms. Results about quadratic forms are important in regression and the analysis of variance.
A quadratic form in (x, y) is an expression of the type ax2 +by 2 +cxy; a quadratic form in (x, y, z) is an expression
of the form ax2 + by 2 + cz 2 + dxy + exz + f yz. Thus, for example, 2x2 + 4x + 3y 2 is not a quadratic form in (x, y).
Definition(38.7a). Suppose A is a real n × n symmetric matrix. Then the quadratic form of A is the function
qA : Rn → Rn with
Xn X n
qA (x) = ajk xj xk = xT Ax
j=1 k=1
Suppose we have x Ax where the matrix A is not symmetric. Because xT Ax is a scalar, we have xT Ax = xT AT x
T

and hence xT Ax = xT Bx where B = 21 (A + AT ). In this way, we can work round the requirement that A is
symmetric.
Pn
Example(38.7b). Suppose A = I, the identity matrix. Then qA (X) = XT AX = k=1 Xk2 .
Pn 2
Example(38.7c). Suppose A = 1, the n × n matrix with every entry equal to 1. Then qA (X) = XT AX = k=1 Xk .
If A and B are both real n×n symmetric matrices and a, b ∈ R, then aA+bB can be used to create a new quadratic
form:
qaA+bB (X) = XT (aA + bB)X = aqA (X) + bqB (X)
Pn Pn 2
Example(38.7d). Suppose A = I and B = 1. Then qaI+b1 (X) = a k=1 Xk2 + b k=1 Xk .
Pn Pn 2 Pn Pn
In particular qI−1/n (X) = k=1 Xk2 − n1 k=1 Xk = k=1 (Xk − X)2 and hence the sample variance S 2 = k=1 (Xk −
X)2 /(n − 1) is a quadratic form in X = (X1 , . . . , Xn ).
Example(38.7e). Suppose
0 1 0 ··· 0 0 0 1 0 ··· 0 0
   
0 0 1 ··· 0 0 1 0 1 ··· 0 0
0 0 0 ··· 0 ···
   
0 0 1 0 0 0
A1 =  ... ... ... . . . ... ..  and A2 =   ... ... ... .. .. .. 
 .  . . . 
0 0 0 ··· 0 1 0 0 0 ··· 0 1
0 0 0 ··· 0 0 0 0 0 ··· 1 0
Then XT A1 X = X1 X2 + · · · + Xn−1 Xn . Note that the matrix A1 is not symmetric. Now A2 = A1 + AT1 is symmetric and
Pn−1
qA2 (X) = XT A2 X = 2XT A1 X = 2 k=1 Xk Xk+1 .
38.8 Mean of a quadratic form.
Proposition(38.8a). Suppose X is an n × 1 random vector with E[X] = µ and var[X] = Σ. Suppose A is a
real n × n symmetric matrix. Then qA (X), the quadratic form of A has expectation
E[XT AX] = trace(AΣ) + µT Aµ (38.8a)
Proof. Now XT AX = (X − µ)T A(X − µ) + µT AX + XT Aµ − µT Aµ and hence
E[XT AX] = E (X − µ)T A(X − µ) + µT Aµ
 

Because (X − µ)T A(X − µ) is a scalar, we have


E[ (X − µ)T A(X − µ) ] = E trace (X − µ)T A(X − µ)
 

= E trace A(X − µ)(X − µ)T


 
because trace(AB) = trace(BA).
T
 
= trace E A(X − µ)(X − µ) because E[trace(V)] = trace(E[V]).
Hence result.
The second term in equation(38.8a) is xT Ax evaluated at x = µ; this simplifies some derivations. We now apply
this result to some of the examples above.
Pn
Example(38.8b). Suppose A = I, the identity matrix. Then qA (X) = XT AX = j=1 Xj2 . and equation(38.8a) gives
 
Xn n
X n
X
E Xj2  = σj2 + µ2j
j=1 j=1 j=1
P 2
n
Example(38.8c). Suppose A = 1, the n × n matrix with every entry equal to 1. Then qA (X) = XT AX = j=1 Xj and
equation(38.8a) gives
 2   2
n
X n X
X n Xn
E  Xj   = σjk +  µj 
 
j=1 j=1 k=1 j=1
Page 120 §38 Mar 10, 2020(20:25) Bayesian Time Series Analysis

P Suppose X1 , . . . , Xn are i.i.d. random variables with mean µ and


Example(38.8d). Continuation of example(38.7d).
n 1
variance σ 2 . Consider the quadratic form qA (X) = 2 T
k=1 (Xk − X) . Then qA (X) = X AX where A = I − n 1. Now
2
var[X] = σ I; hence equation(38.8a) gives
n
1 X
E[qA (X)] = σ 2 trace(I − 1) + xT Ax = (n − 1)σ 2 + (xk − x)2 = (n − 1)σ 2

n x=µ x=µ
k=1
Pn
Hence if S 2 = k=1 (Xk − X)2 /(n − 1), then E[S 2 ] = σ 2 .
Example(38.8e). Suppose X1 , . . . , Xn are i.i.d. random variables with mean µ and variance σ 2 . First we shall find the
Pn−1
matrix A with qA (X) = k=1 (Xk − Xk+1 )2 = (X1 − X2 )2 + (X2 − X3 )2 + · · · + (Xn−1 − Xn )2 . Now qA (X) = X12 + 2X22 + · · · +
2 n−1
+Xn2 −2 k=1 Xk Xk+1 . Using the matrix A2 in example(38.7e) gives qA (X) = X12 +2X22 +· · ·+2Xn−1 2
+Xn2 −XT A2 X.
P
2Xn−1
T T
Hence qA (X) = X A1 X − X A2 X where A1 = diag [ 1 2 2 · · · 2 1 ].
Hence qA (X) = XT AX where
1 −1 0 · · · 0 0
 
 −1 2 −1 · · · 0 0 
 0 −1 2 · · · 0 0 
 
A = A1 − A2 =   ... .
.. .
.. .. .
.. . 
.. 
 . 
 0 0 0 · · · 2 −1 
0 0 0 ··· −1 1
2
Equation(38.8a) gives E[qA (X)] = σ trace(A) + qA (X) = σ (2n − 2) + 0 = σ 2 (2n − 2).
2

X=(µ,...,µ)

38.9 Variance of a quadratic form. This result is complicated!


Proposition(38.9a). Suppose X1 , . . . , Xn are independent random variables with E[Xj ] = 0 for j = 1, . . . , n.
Suppose all the random variables have the same finite second and fourth moments; we shall use the following
notation:
E[Xj2 ] = σ 2 and E[Xj4 ] = µ4
Suppose A is an n × n symmetric matrix with entries aij and d is the n × 1 column vector with entries
(a11 , . . . , ann ) = diag(A). Then
var[XT AX] = (µ4 − 3σ 4 )dT d + 2σ 4 trace(A2 )
  2
Proof. Now var[XT AX] = E (XT AX)2 − E[XT AX] . Pn
Because E[X] = 0, using equation(38.8a) gives E[XT AX] = trace(AΣ) = σ 2 trace(A) = σ 2 j=1 ajj . Let c = XT A and
Z = XXT ; then c is a 1×n row vector and Z is an n×n matrix and (XT AX)2 = XT AXXT AX = cZcT = j k cj ck Zjk =
P P
P P th T
P n th T
j k cj ck Xj Xk . The j entry in the row vector c = X A is i=1 Xi aij and the k entry in the row vector c = X A
P n
is `=1 X` a`k . Hence
XXXX
(XT AX)2 = aij a`k Xi Xj Xk X`
i j k `
Using independence of the X’s gives
µ4
if i = j = k = `;
(
E[Xi Xj Xk X` ] = σ4
if i = j and k = `, or i = k and j = `, or i = ` and j = k.
otherwise. 0
Hence  
X X X X
E[ (XT AX)2 = µ4 a2ii + σ 4  a2ij 
 
aii akk + aij aji +
 
i i,k i,j i,j
i6 =k i6 =j i6 =j

Now
X n
X n
X n
X
aii akk = aii akk = aii (trace(A) − aii ) = [trace(A)]2 − dT d
i,k i=1 k=1 i=1
i6 =k k6 =i

X n X
X n n X
X n
a2ij = a2ij = a2ij − dT d = trace(A2 ) − dT d
i,j i=1 j=1 i=1 j=1
i6 =j j6 =i
X X
aij aji = a2ij = trace(A2 ) − dT d
i,j i,j
i6 =j i6 =j

and hence
E[ (XT AX)2 = (µ4 − 3σ 4 )dT d + σ 4 trace(A)2 + 2 trace(A2 )
   

and hence the result.


3 Multivariate Continuous Distributions Mar 10, 2020(20:25) §39 Page 121

Example(38.9b). Suppose X1 , . . . , Xn are i.i.d. random variables with the N (0, σ 2 ) distribution and A is a symmetric
n × n matrix. By §15.4 on page 47, we know that E[Xj4 ] = 3σ 4 . Hence
var[XT AX] = 2σ 4 trace(A2 )

We can generalize proposition(38.9a) to non-zero means as follows:


Proposition(38.9c). Suppose X1 , . . . , Xn are independent random variables with E[Xj ] = µj for j = 1, . . . , n.
Suppose all the random variables have the same finite second, third and fourth moments about the mean; we
shall use the following notation: E[X] = µ and
E[(Xj − µj )2 ] = σ 2
E[(Xj − µj )3 ] = µ3
E[(Xj − µj )4 ] = µ4
Suppose A is an n × n symmetric matrix with entries ai,j and d is the n × 1 column vector with entries
(a11 , . . . , ann ) = diag(A). Then
var[XT AX] = (µ4 − 3σ 4 )dT d + 2σ 4 trace(A2 ) + 4σ 2 µT A2 µ + 4µ3 µT Ad (38.9a)
Proof. See exercise 39.11 on page 122.

38.10
Summary.
The variance matrix.
• var[X] = E[ (X − µ)(X − µ)T ] = E[XXT ] − µµT .
• var[A + BX] = B var[X] BT
• var[X] is symmetric positive semi-definite
• if no element of X is a linear combination of the others, then var[X] is symmetric positive definite
• if var[X] is positive definite, there exists symmetric non-singular Q with var[X] = QQ
• if X has finite second moments and no element is a linear combination of the other elements, then there exists a linear
transformation of X to independent variables
The covariance matrix.
• cov[X, Y] = E[ (X − µX )(Y − µY )T ] = E[XYT ] − µX µTY
• cov[X, Y] = cov[Y, X]T
• cov[X, X] = var[X]
• if the dimensions of X and Y are equal, then cov[X, Y] is symmetric
Quadratic forms.
• qA (x) = xT Ax where A is a real symmetric matrix
• E[qA (X)] = trace(AΣ) + µT Aµ

39 Exercises (exs-multiv.tex)

39.1 Suppose X is an m × 1 random vector and Y is an n × 1 random vector. Suppose further that all second moments are
finite of X and Y and suppose a is an m × 1 vector and b is an n × 1 vector. Show that
Xm X n
cov[aT X, bT Y] = aT cov[X, Y]b = ai bj cov[Xi , Yj ] [Ans]
i=1 j=1

39.2 Further properties of the covariance matrix. Suppose X is an m × 1 random vector and Y is an n × 1 random vector.
Suppose further that all second moments are finite.
(a) Show that for any m × 1 vector b and any n × 1 vector d we have
cov[X + b, Y + d] = cov[X, Y]
(b) Show that for any ` × m matrix A and any p × n matrix B we have
cov[AX, BY] = Acov[X, Y]BT
(c) Suppose a, b, c and d ∈ R; suppose further that V is an m × 1 random vector and W is an n × 1 random vector with
finite second moments.
cov[aX + bV, cY + dW] = ac cov[X, Y] + ad cov[X, W] + bc cov[V, Y] + bd cov[V, W]
Both sides are m × n matrices.
(d) Suppose a and b ∈ R and both X and V are m × 1 random vectors. Show that
var[aX + bV] = a2 var[X] + ab cov[X, V] + ab cov[V, X] + b2 var[V] [Ans]
Page 122 §39 Mar 10, 2020(20:25) Bayesian Time Series Analysis

39.3 Suppose X is an m-dimensional random vector with finite second order moments and such that such that no element of
X is a linear combination of the other elements.
Show that for any n-dimensional random vector Y, there exists an n × m matrix A such that
cov[Y − AX, X] = 0 [Ans]
39.4 Suppose X is an n × 1 random vector with E[X] = µ and var[X] = Σ. Prove the following results:
(a) E[(AX + a)(BX + b)T ] = AΣBT + (Aµ + a)(Bµ + b)T where A is m × n, a is m × 1, B is r × n and b is r × 1.
(b) E[(X + a)(X + a)T ] = Σ + (µ + a)(µ + a)T where a is n × 1.
(c) E[XaT X] = (Σ + µµT )a where a is n × 1. [Ans]
39.5 Suppose Y1 , Y2 , . . . , Yn are independent random variables each with variance 1. Let X1 = Y1 , X2 = Y1 + Y2 , . . . , Xn =
Y1 + · · · + Yn . Find the n × n matrix var[X]. [Ans]
39.6 Suppose X is an n × 1 random vector with finite second moments. Show that for any n × 1 vector α ∈ Rn we have
E[(X − α)(X − α)T ] = var[X] + (µX − α)(µx − α)T [Ans]
39.7 Suppose X is an n × 1 random vector with E[X] = µ and var[X] = Σ. Prove the following results:
(a) E[(AX + a)T (BX + b)] = trace(AΣBT ) + (Aµ + a)T (Bµ + b) where A and B are m × n, and a and b are m × 1.
(b) E[XT X] = trace(Σ) + µT µ
(c) E[(AX)T (AX)] = trace(AΣAT ) + (Aµ)T (Aµ) where A is n × n.
(d) E[(X + a)T (X + a)] = trace(Σ) + (µ + a)T (µ + a) where a is n × 1. [Ans]
39.8 Suppose X is an m-dimensional random vector, Y is an n-dimensional random vector and A is an m × n real matrix.
Prove that
E[ XT AY ] = trace(AΣY,X ) + µTX AµY
where ΣY,X is the n × m matrix cov[Y, X]. [Ans]
39.9 Quadratic forms. Suppose X is an n × 1 random vector with E[X] = µ and var[X] = Σ. Suppose A is a real n × n
symmetric matrix and b is an n × 1 real vector.
Show that
E[(X − b)T A(X − b)] = trace(AΣ) + (µ − b)T A(µ − b)
In particular
• E[ (X − µ)T A(X − µ) ] = trace(AΣ). √
• If kak denotes the length of the vector a, then kak = aT a and EkX − bk2 = trace(Σ) + kµ − bk2 . [Ans]
39.10 Suppose X1 , . . . , Xn are random variables with E[Xj ] = µj and var[Xj ] = σj2 for j = 1, . . . , n; also cov[Xj , Xk ] = 0
for k > j + 1. If
Xn
Q= (Xk − X)2
k=1
show that
(n − 1)α − 2β
E[Q] = +γ
n
Pn 1
Pn 2
where α = σ12 + · · · + σn2 , β = cov[X1 , X2 ] + cov[X2 , X3 ] + · · · + cov[Xn−1 , Xn ] and γ = k=1 µ2k − n k=1 µk .
Note that if all variables have the same mean, then γ = 0. [Ans]
39.11 Variance of a quadratic form—proof of proposition(38.9c) on page 121.
(a) Show that XT AX = W1 + W2 + c where W1 = (X − µ)T A(X − µ), W2 = 2µT A(X − µ) and c = µT Aµ.
(b) Show that var[W2 ] = 4σ 2 µT A2 µ.
(c) Show that cov[W1 , W2 ] = E[W1 W2 ] = 2µT A E[YYT AY] = 2µ3 µT Ad where Y = X − µ.
(d) Hence show var[XT AX] = (µ4 − 3σ 4 )dT d + 2σ 4 trace(A2 ) + 4σ 2 µT A2 µ + 4µ3 µT Ad [Ans]
2
39.12 Suppose X1 , . . . , Xn are random variables
Pnwith common expectation µ and common variance σ . Suppose further that
cov[Xj , Xk ] = ρσ 2 for j 6= k. Show that k=1 (Xk − X)2 has expectation σ 2 (1 − ρ)(n − 1) and hence
Pn 2
k=1 (Xk − X)
(1 − ρ)(n − 1)
2
is an unbiased estimator of σ . [Ans]
39.13 Suppose X1 , . . . , Xn are i.i.d. random variables with the N (µ, σ 2 ) distribution. Let
Pn Pn−1
(Xk − X)2 (Xk+1 − Xk )2
S 2 = k=1 and Q = k=1
n−1 2(n − 1)
(a) Show that var[S 2 ] = 2σ 4 /(n − 1). (See also exercise 43.8 on page 139.)
(b) Show that E[Q] = σ 2 and var[Q] = 2σ 4 (6n − 8)/4(n − 1)2 . [Ans]
3 Multivariate Continuous Distributions Mar 10, 2020(20:25) §40 Page 123

39.14 (a) Expectation of XT AY. Suppose X is an n-dimensional random vector with expectation µX , Y is an m-dimensional
random vector with expectation µY and A is an n × m real matrix. Let Z = XT AY.
Prove that E[Z] = trace( Acov[Y, X] ) + µTX AµY .
(b) Suppose (X1 , Y1 ), . . . , (Xn , Yn ) are i.i.d. random vectors with a distribution with expectation (µX , µY ) and variance
 2 
σX σXY
where σXY = cov[X, Y ].
σXY σY2
Suppose
Pn
j=1 (Xj − X)(Yj − Y)
SXY =
n−1
Show that E[SXY ] = σXY . [Ans]

40 The bivariate normal


40.1 The density. Here is the first of several equivalent formulations of the density.

The random vector (X1 , X2 ) has a bivariate normal distribution iff it has density
Definition(40.1a).
|P|1/2 (x − µ)T P (x − µ)
 
fX1 X2 (x1 , x2 ) = exp − (40.1a)
2π 2
   
x1 µ1
where x = , µ = ∈ R2 and P is a real symmetric positive definite matrix.
2×1 x 2 2×1 µ 2 2×2

Suppose the entries in the 2 × 2 real symmetric matrix P are denoted as follows2 :
 
a1 a2
P=
a2 a3
It follows that equation(40.1a) is equivalent to
q
a1 a3 − a22 
a1 (x1 − µ1 )2 + 2a2 (x1 − µ1 )(x2 − µ2 ) + a3 (x2 − µ2 )2

f (x1 , x2 ) = exp − (40.1b)
2π 2
A more common form of the density is given in equation(40.3a) on page 124.

To show that equation(40.1b) defines a density. Clearly f ≥ 0. It remains to check that f integrates to 1. Let
y1 = x1 − µ1 and y2 = x2 − µ2 . Then
Z Z
fX1 X2 (x1 , x2 ) dx1 dx2
x1 x2
|P|1/2 a1 y12 + 2a2 y1 y2 + a3 y22
Z Z  
= exp − dy1 dy2 (40.1c)
2π y1 y2 2
( 2 )  2
|P|1/2 a22
 
a1 a2 y2
Z Z
= exp − y1 + y2 exp − a3 − dy1 dy2 (40.1d)
2π y1 y2 2 a1 2 a1

√  
a1 a3 −a22
Now use the transformation z1 = a1 y1 + aa12 y2 and z2 = y2 √ a1 . This transformation has Jacobian
q
a1 a3 − a22 = |P|1/2 and is a 1 − 1 map R2 → R2 ; it gives
 2
z1 + z22
Z Z Z Z 
1
fX1 X2 (x1 , x2 ) dx1 dx2 = exp − dz1 dz2 = 1
x1 x2 2π z1 z2 2
by using the integral of the standard normal density equals one.

2
It is easy to check that the real symmetric matrix P is positive definite iff a1 > 0 and a1 a3 − a22 > 0.
Page 124 §40 Mar 10, 2020(20:25) Bayesian Time Series Analysis

40.2 The marginal distributions of X1 and X2 . For the marginal density of X2 we need to find the following
integral:
Z
fX2 (x2 ) = fX1 X2 (x1 , x2 ) dx1
x1
First let Y1 = X1 − µ1 and Y2 = X2 − µ2 and find the density of Y2 :
|P|1/2 a1 y12 + 2a2 y1 y2 + a3 y22
Z Z  
fY2 (y2 ) = fY1 ,Y2 (y1 , y2 ) dy1 = exp − dy1
y1 2π y1 2
Using the decomposition in equation(40.1d) gives
 2 ( 2 )
|P|1/2 a22
 Z 
y2 a1 a2
fY2 (y2 ) = exp − a3 − exp − y1 + y2 dy1
2π 2 a1 y1 2 a1
 2  r
|P|1/2 y2 a1 a3 − a22 y2
 
2π 1
= exp − =q exp − 22
2π 2 a1 a1 2πσ22 2σ2

where σ22 = a1 /(a1 a3 − a22 ) = a1 /|P|. It follows that the density of X2 = Y2 + µ2 is


(x2 − µ2 )2
 
1 a1 a1
fX2 (x2 ) = exp − 2
where σ22 = 2
=
|P|
q
2πσ22 2σ2 (a1 a3 − a2 )

We have shown that the marginal distributions are normal:


X2 has the N (µ2 , σ22 ) distribution.
Similarly,
X1 has the N (µ1 , σ12 ) distribution.
where
a3 a3 a1 a1
σ12 = 2
= and σ22 = 2
=
a1 a3 − a2 |P| a1 a3 − a2 |P|
If X1 and X2 are both normal, it does not follow that (X1 , X2 ) is normal—see example (42.8a) on page 133.
40.3 The covariance and correlation between X1 and X2 . Of course, cov[X1 , X2 ] = cov[Y1 , Y2 ] where
Y1 = X1 − µ1 and Y2 = X2 − µ2 . So it suffices to find cov[Y1 , Y2 ] = E[Y1 Y2 ]. The density of (Y1 , Y2 ) is:
|P|1/2 a1 y12 + 2a2 y1 y2 + a3 y22
 
fY1 Y2 (y1 , y2 ) = exp −
2π 2
It follows that
a1 y12 + 2a2 y1 y2 + a3 y22
Z Z  

exp − dy1 dy2 = q
y1 y2 2 a a − a2
1 3 2
Differentiating with respect to a2 gives
a1 y12 + 2a2 y1 y2 + a3 y22
Z Z  

(−y1 y2 ) exp − dy1 dy2 = a2
y1 y2 2 (a1 a3 − a22 )3/2
and hence
a2
cov[X1 , X2 ] = E[Y1 Y2 ] = −
a1 a3 − a22
The correlation between X1 and X2 is
cov[X1 , X2 ] a2
ρ= = −√
σ1 σ2 a1 a3
These results lead to an alternative expression for the density of a bivariate normal:
(x1 − µ1 )2 (x1 − µ1 )(x2 − µ2 ) (x2 − µ2 )2
  
1 1
fX1 X2 (x1 , x2 ) = exp − − 2ρ +
2(1 − ρ2 ) σ12 σ22
p
2πσ1 σ2 1 − ρ2 σ 1 σ2
(40.3a)
We have also shown that  
σ12 cov[X1 , X2 ]
var[X] = = P−1
cov[X1 , X2 ] σ22
P is sometimes called the precision matrix—it is the inverse of the variance matrix var[X].
3 Multivariate Continuous Distributions Mar 10, 2020(20:25) §40 Page 125

Summarizing some of these results:


   
a1 a2 −1 1 a3 −a2
P= |P| = a1 a3 − a22 P =
a2 a3 a1 a3 − a22 −a2 a1
   
−1 σ12 ρσ1 σ2 −1 1 σ22 −ρσ1 σ2
Σ=P = |Σ| = (1 − ρ2 )σ12 σ22 Σ =P=
ρσ1 σ2 σ22 (1 − ρ2 )σ12 σ22 −ρσ1 σ2 σ12
(40.3b)
Example(40.3a). Suppose (X, Y ) has a bivariate normal distribution with density
 
1 1 2 2
f (x, y) = exp − (x + 2y − xy − 3x − 2y + 4)
k 2
Find the mean vector and the variance matrix of (X, Y ). What is the value of k?
Solution. Let Q(x, y) = a1 (x − µ1 )2 + 2a2 (x − µ1 )(y − µ2 ) + a3 (y − µ2 )2 . So we want Q(x, y) = x2 + 2y 2 − xy − 3x − 2y + 4.
Equating coefficients of x2 , xy and y 2 gives a1 = 1, a2 = − 12 and a3 = 2. Hence
1 − 12 4 2 12
   
−1
P= and Σ = P =
− 12 2 7 21 1

Also |P| = 74 and hence k = 2π/|P|1/2 = 4π/ 7.
Now ∂Q(x,y)
∂x = 2a1 (x − µ1 ) + 2a2 (y − µ2 ) and ∂Q(x,y)
∂y = 2a2 (x − µ1 ) + 2a3 (y − µ2 ). If ∂Q(x,y)
∂x = 0 and ∂Q(x,y)
∂y = 0 then we
2
must have x = µ1 and y = µ2 because |P| = a1 a3 − a2 6= 0.
Applying this to Q(x, y) = x2 + 2y 2 − xy − 3x − 2y + 4 gives the equations 2µ1 − µ2 − 3 = 0 and 4µ2 − µ1 − 2 = 0. Hence
(µ1 , µ2 ) = (2, 1).

40.4 The characteristic function. Suppose XT = (X1 , X2 ) has the bivariate density defined in equation(40.1a).
Then for all t ∈ R2 , the characteristic function of X is
2×1

|P|1/2 (x − µ)T P(x − µ)


h i Z Z  
itT X itT x
φ(t) = E e = e exp − dx dy
2π x1 x2 2
|P|1/2 itT µ
 T
2it y − yT Py
Z Z 
= e exp dy1 dy2 by setting y = x − µ.
2π y1 y2 2
But y Py − 2it y = (y − iΣt) P(y − iΣt) + tT Σt where Σ = P−1 = var[X]. Hence
T T T

|P|1/2 itT µ− 1 tT Σt (y − iΣt)T P(y − iΣt)


Z Z  
φ(t) = e 2 exp − dy1 dy2
2π y1 y2 2
T
µ− 21 tT Σt
= eit
by using the integral of equation(40.1a) is 1.
Example(40.4a). Suppose X = (X1 , X2 ) ∼ N (µ, Σ). Find the distribution of X1 + X2 .
Solution. The c.f. of X is φX (t1 , t2 ) = exp iµ1 t1 + iµ2 t2 − 21 (t21 σ12 + 2t1 t2 σ12 + t22 σ22 ) . Setting t1 = t2 = t gives the


c.f. of X1 + X2 to be φX1 +X2 (t) = exp i(µ1 + iµ2 )t − 21 t2 (σ12 + 2σ12 + σ22 ) . Hence X1 + X2 ∼ N (µ1 + µ2 , σ 2 ) where


σ 2 = σ12 + 2σ12 + σ22 = σ12 + 2ρσ1 σ2 + σ22 .

40.5 The conditional distributions. We first find the conditional density of Y1 given Y2 where Y1 = X1 − µ1
and Y2 = X2 − µ2 . Now
fY Y (y1 , y2 )
fY1 |Y2 (y1 |y2 ) = 1 2
fY2 (y2 )
We use the following forms:
 n o
2 2ρσ1 σ12 2
1 y1 − σ2 y1 y2 + σ2 y2
fY1 Y2 (y1 , y2 ) = exp − 2
 
2
p 2
2σ1 (1 − ρ )
2

2πσ1 σ2 1 − ρ

y22
 
1
fY2 (y2 ) = q exp − 2
2πσ22 2σ2
Page 126 §40 Mar 10, 2020(20:25) Bayesian Time Series Analysis

Hence  n o
2ρσ1 ρ2 σ12 2
1 y12 − σ2 y1 y2 + σ22 2
y
f (y1 |y2 ) = √ q exp −
 
2σ12 (1 − ρ2 )

2π σ12 (1 − ρ2 )
 n  o2 
ρσ1
1 y1 − y
σ2 2
=√ q exp −
 
2σ 2 (1 − ρ 2) 
2
2π σ (1 − ρ2 ) 1
1
 
ρσ1 2
which is the density of the N σ2 y2 , σ1 (1 − ρ2 )
distribution.
 
It follows that the density of X1 given X2 is the N µ1 + ρσ σ2
1
(x 2 − µ 2 ), σ1
2 (1 − ρ2 ) distribution, and hence

ρσ1
E[X1 |X2 ] = µ1 + σ2 (X2 − µ2 ) and var[X1 |X2 ] = σ12 (1 − ρ2 ).
of the original notation, σ12 (1 − ρ2 ) = 1/a1 and ρσ1 /σ2 = −a2 /a1 and hence the distribution of X1 given
In terms 
a2
X2 is N µ1 − a1 (x2 − µ2 ), a11 .
Example(40.5a). Suppose the 2-dimensional random vector X has the bivariate normal N (µX , ΣX ) distribution where
   
2 4 2
µX = and ΣX =
1 2 3
Find the distribution of X1 + X2 given X1 = X2 .
Solution. Let Y1 = X1 + X2 and Y2 = X1 − X2 . Then   
Y1 1 1
Y= = BX where B =
Y2 1 −1
Hence Y ∼ N (µY , ΣY ) where
   
3 T 11 1
µY = and ΣY = BΣX B =
1 1 3
2 2

 the random vector Y we haveσ1 = 11, σ2 = 3 and ρ = 1/ 33. We now want the distribution of Y1 given Y2 = 0.
Note that for
This is N µ1 + ρσ 2 2
σ2 (y2 − µ2 ), σ1 (1 − ρ ) = N ( /3, /3).
1 5 32

We have also shown that if the random vector (X1 , X2 ) is bivariate normal, then E[X1 |X2 ] is a linear function
of X2 and hence the best predictor and best linear predictor are the same—see exercises 2.15 and 2.18 on page 7.
40.6 Independence of X1 and X2 .
Proposition(40.6a). Suppose (X1 , X2 ) ∼ N (µ, Σ). Then X1 and X2 are independent iff ρ = 0.
Proof. If ρ = 0 then fX1 X2 (x1 , x2 ) = fX1 (x1 )fX2 (x2 ). Conversely, if X1 and X2 are independent then cov[X1 , X2 ] = 0
and hence ρ = 0.
In terms of entries in the precision matrix: X1 and X2 are independent iff a2 = 0.
40.7 Linear transformation of a bivariate normal.
Proposition(40.7a). Suppose X has the bivariate normal distribution N (µ, Σ) and C is a 2 × 2 non-singular
matrix. Then the random vector Y = a + CX has the bivariate normal distribution N (a + Cµ, CΣCT ).
Proof. The easiest way is to find the characteristic function of Y. For t ∈ R2 we have
2×1
itT Y itT a itT CX itT a itT Cµ− 21 tT CΣCT t
φY (t) = E[e ] = e E[e ]=e e
which is the characteristic function of the bivariate normal N (a + Cµ, CΣCT ).
We need C to be non-singular in order to ensure the variance matrix of the result is non-singular.

We can transform a bivariate normal to independent normals as follows.


Proposition(40.7b). Suppose X has the bivariate normal distribution N (µ, Σ) where
   
µ1 σ12 ρσ1 σ2
µ= and Σ =
µ2 ρσ1 σ2 σ22
Define Y to be the random vector with components Y1 and Y2 where:
X1 = µ1 + σ1 Y1
p
X2 = µ2 + ρσ2 Y1 + σ2 1 − ρ2 Y2
Then Y ∼ N (0, I).
3 Multivariate Continuous Distributions Mar 10, 2020(20:25) §41 Page 127

−1
Proof. Note that X = µ + BY  and Y = B (X −µ) where  
σ1 0 −1
1/σ 0
B= p and B = √ 1

ρσ2 σ2 1 − ρ2 − ρ/σ1 1−ρ2 1/σ2 1−ρ2
It is straightforward to check that B−1 Σ(B−1 )T = I. Hence Y ∼ N (0, I).
Example(40.7c). Suppose X1 and X2 are i.i.d. random variables with the N (0, 1) distribution. Suppose further that µ1 ∈ R,
µ2 ∈ R, σ1 ∈ (0, ∞), σ2 ∈ (0, ∞) and ρ ∈ (−1, 1).
(a) Let p
Y1 = µ1 + σ1 X1 and Y2 = µ2 + ρσ2 X1 + 1 − ρ2 σ2 X2
Find the distribution of (Y1 , Y2 ).
(b) Let p
Y10 = µ1 + ρσ1 X1 + 1 − ρ2 σ1 X2 and Y20 = µ2 + σ2 X1
Find the distribution of (Y10 , Y20 ).
Solution. (a) Now Y = a + CX where X ∼N (0, I) and  
µ1 σ1 p 0
a= and C =
µ2 ρσ2 1 − ρ2 σ2
Hence Y ∼ N (a, CCT ) and C is non-singular with
 
σ12 ρσ1 σ2
CCT =
ρσ1 σ2 σ22
(b) The same distribution as Y = (Y1 , Y2 ).
For the transformation of a bivariate normal distribution by polar coordinates and the Box-Muller transformation
see exercise 28.26 on page 86.
40.8
Summary. The bivariate normal distribution.
Suppose X = (X1 , X2 ) ∼ N (µ, Σ) where µ = E[X] and Σ = var[X].
• Density.
|P|1/2 (x − µ)T P(x − µ)
 
fX (x) = exp − where P = Σ−1 is the precision matrix.
2π 2
(x1 − µ1 )2 (x1 − µ1 )(x2 − µ2 ) (x2 − µ2 )2
  
1 1
= exp − − 2ρ +
2(1 − ρ2 ) σ12 σ22
p
2πσ1 σ2 1 − ρ2 σ1 σ2
   
a1 a2 1 a3 −a2
• If P = then Σ = P−1 = .
a2 a3 a1 a3 − a2 2 −a 2 a1
 
1 σ22 −ρσ1 σ2
• P = Σ−1 = 2 2 and |Σ| = (1 − ρ2 )σ12 σ22 .
σ1 σ2 (1 − ρ2 ) −ρσ1 σ2 σ12
• The marginal distributions. X1 ∼ N (µ1 , σ12 ) and X2 ∼ N (µ2 , σ22 ).
T 1 T T 1 T
• The characteristic function: φ(t) = eit µ− 2 t Σt ; the m.g.f. is E[etX ] = et µ+ 2 t Σt
.
• The conditional distributions.  
The distribution of X1 given X2 = x2 is N µ1 + ρσ σ2
1
(x2 − µ 2 ), σ1
2 (1 − ρ2 ) .
 
ρσ2
The distribution of X2 given X1 = x1 is N µ2 + σ1 (x1 − µ1 ), σ22 (1 − ρ2 ) .
• X1 and X2 are independent iff ρ = 0.
• Linear transformation of a bivariate normal. If C is non-singular, then Y = a + CX has a bivariate normal
distribution with mean a + Cµ and variance matrix CΣCT .

41 Exercises (exs-bivnormal.tex)

41.1 Sum of two independent bivariate normals. Suppose X = (X1 , X2 ) ∼ N (µX , ΣX ) and Y = (Y1 , Y2 ) ∼ N (µY , ΣY ).
Prove that X + Y ∼ N (µX + µY , ΣX + ΣY ). [Ans]
41.2 Suppose (X, Y ) has the density
2 2
fXY (x, y) = ce−(x −xy+y )/3

(a) Find c. (b) Are X and Y independent? [Ans]


Page 128 §41 Mar 10, 2020(20:25) Bayesian Time Series Analysis

41.3 Suppose (X, Y ) has a bivariate normal distribution with density


f (x, y) = k exp −(x2 + 2xy + 4y 2 )
 

Find the mean vector and the variance matrix of (X, Y ). What is the value of k? [Ans]
41.4 Suppose (X, Y ) has a bivariate normal distribution
 with density 
1 1 2 2
f (x, y) = exp − (2x + y + 2xy − 22x − 14y + 65)
k 2
Find the mean vector and the variance matrix of (X, Y ). What is the value of k? [Ans]
41.5 Suppose the random vector Y = (Y1 , Y2 ) has the density 
1 1 2 2
f (y1 , y2 ) = exp − (y1 + 2y2 − y1 y2 − 3y1 − 2y2 + 4) for y = (y1 , y2 ) ∈ R2 .
k 2
Find E[Y] and var[Y]. [Ans]
41.6 Evaluate the integral Z ∞
exp −(y12 + 2y1 y2 + 4y22 ) dy1 dy2
 
[Ans]
−∞
41.7 Suppose the random vector Y = (Y1, Y2 ) has the density 
1 2 2
f (y1 , y2 ) = k exp − (y1 + 2y1 (y2 − 1) + 4(y2 − 1) for y = (y1 , y2 ) ∈ R2 .
12
Show that Y ∼ N (µ, Σ) and find the values of µ and Σ. [Ans]
2
41.8 Suppose (X, Y ) has the bivariate normal distribution N (µ, Σ). Let σX
= var[X], σY2
= var[Y ] and ρ = corr(X, Y ).
(a) Show that X and Y − ρσY X/σX are independent.
(b) Suppose θ satisfies tan(θ) = σX/σY , show that X cos θ + Y sin θ and X cos θ − Y sin θ are independent. [Ans]
41.9 Suppose X = (X1 , X2 ) has a bivariate normal distribution with E[X1 ] = E[X2 ] = 0 and variance matrix Σ. Prove that
X2
XT PX − 21 ∼ χ21
σ1
where P is the precision matrix of X. [Ans]
41.10 (a) Suppose E[X1 ] = µ1 , E[X2 ] = µ2 and there exists α such that Y = X1 + αX2 is independent of X2 . Prove that
E[X1 |X2 ] = µ1 + αµ2 − αX2 .
(b) Use part (a) to derive E[X1 |X2 ] for the bivariate normal. [Ans]
41.11 Suppose X = (X, Y ) has the bivariate normal distribution N (µ, Σ). Because the matrix Σ is positive definite, the
Cholesky decomposition asserts that there is a unique lower triangular matrix L with Σ = LLT . Define the random
vector Z by X = µ + LZ. Show that Z ∼ N (0, I) and hence the two components of Z are i.i.d. random variables with
the N (0, 1) distribution. [Ans]
41.12 An alternative method
p for constructing the bivariate normal. Suppose X and Y are i.i.d. N (0, 1). Suppose ρ ∈ (−1, 1)
and Z = ρX + 1 − ρ2 Y .
(a) Find the density of Z.
(b) Find the density of (X, Z).
(c) Suppose µ1 ∈ R, µ2 ∈ R, σ1 > 0 and σ2 > 0. Find the density of (U, V ) where U = µ1 + σ1 X and V = µ2 + σ2 Z.
[Ans]
41.13 Suppose X1 ∼ N (0, σ12 ), X2 ∼ N (0, σ22 ) and X1 and X2 are independent. Let Z = X1 + X2 .
(a) Find the distribution of the random vector (X1 , Z).
(b) Find E[X1 |Z] and E[eitX1 |Z]. [Ans]
41.14 Suppose X1 , . . . , Xn are i.i.d. random variables with the N (0, σ12 ) distribution and Y1 , . . . , Yn are i.i.d. random variables
with the N (0, σ22 ) distribution. Suppose further that all the random variables X1 , . . . , Xn , Y1 , . . . , Yn are independent.
Let Zj = Xj + Yj for j = 1, . . . , n; hence Zj ∼ N (0, σ12 + σ22 ).
Define α : Rn → {1, 2, . . . , n} by 
α(x1 , . . . , xn ) = min j ∈ {1, 2, . . . , n} : xj = max{x1 , . . . , xn }
Let W = Xα(Z1 ,...,Zn ) . Find expressions for E[W ] and E[eitW ]. (We can think of this as a problem where the Zj are
observed but we are really interested in the Xj .) [Ans]
41.15 Suppose (X1 , X2 ) has the bivariate normal distribution with density given by equation(40.3a). Define Q by:
e−Q(x1 ,x2 )
fX1 X2 (x1 , x2 ) = p
2πσ1 σ2 1 − ρ2
Hence
(x1 − µ1 )2 (x1 − µ1 )(x2 − µ2 ) (x2 − µ2 )2
 
1
Q(x1 , x2 ) = − 2ρ +
2(1 − ρ2 ) σ12 σ1 σ2 σ22
Define the random variable Y by Y = Q(X1 , X2 ). Show that Y has the exponential density. [Ans]
3 Multivariate Continuous Distributions Mar 10, 2020(20:25) §41 Page 129

41.16 (a) Suppose (X1 , X2 ) has a bivariate normal distribution with E[X1 ] = E[X2 ] = 0. Hence it has characteristic function
   
1 T 1 2 2 2 2

φX (t) = exp − t Σt = exp − σ1 t1 + 2σ12 t1 t2 + σ2 t2
2 2
Explore the situations when Σ is singular.
(b) Now suppose (X1 , X2 ) has a bivariate normal distribution without the restriction of zero means. Explore the situa-
tions when the variance matrix Σ is singular. [Ans]
41.17 Suppose T1 and T2 are i.i.d. N (0, 1). Set X = a1 T1 + a2 T2 and Y = b1 T1 + b2 T2 where a21 + a22 > 0 and a1 b2 6= a2 b1 .
(a) Show that E[Y |X] = X(a1 b1 + a2 b2 )/(a21 + a22 ).
 2
(b) Show that E Y − E(Y |X) = (a1 b2 − a2 b1 )2 /(a21 + a22 ). [Ans]
41.18 (a) Suppose (X, Y ) has a bivariate normal distribution with var[X] = var[Y ]. Show that X + Y and X − Y are
independent random variables.
(b) Suppose (X, Y ) has a bivariate normal distribution with E[X] = E[Y ] = 0, var[X] = var[Y ] = 1 and cov[X, Y ] =
ρ. Show that X 2 and Y 2 are independent iff ρ = 0.
2
(Note. If var[X] = σX and var[Y ] = σY2 then just set X1 = X/σX and Y1 = Y /σY .) [Ans]
41.19 (a) Suppose (X, Y ) has a bivariate normal distribution N (µ, Σ) with
   2 
0 σ ρσ 2
µ= and Σ=
0 ρσ 2 σ 2
Let (R, Θ) denote the polar coordinates of (X, Y ). Find the distribution of (R, Θ) and the marginal distribution of
Θ.
If ρ = 0, equivalently if X and Y are independent, show that R and Θ are independent.
(b) Suppose X and Y are i.i.d. random variables with the N (0, σ 2 ) distribution. Let
X2 − Y 2 2XY
T1 = √ and T2 = √
2
X +Y 2 X2 + Y 2
Show that T1 and T2 are i.i.d. random variables with the N (0, 1) distribution. [Ans]
41.20 Suppose (X1 , X2 ) has a bivariate normal distribution with E[X1 ] = E[X2 ] = 0. Let Z = X1 /X2 .
(a) Show that
p
σ1 σ2 1 − ρ2
fZ (z) =
π(σ22 z 2 − 2ρσ1 σ2 z + σ12 )
(b) Suppose X1 and X2 are i.i.d. random variables with the N (0, σ 2 ) distribution.
(i) What is the distribution of Z?
(ii) What is the distribution of W = (X − Y )/(X + Y )? [Ans]
41.21 Suppose (X1 , X2 ) has a bivariate normal distribution with E[X1 ] = E[X2 ] = 0 and var[X1 ] = var[X2 ] = 1. Let
ρ = corr[X1 , X2 ] = cov[X1 , X2 ] = E[X1 X2 ]. Show that
X12 − 2ρX1 X2 + X22
∼ χ22 [Ans]
1 − ρ2

41.22 Suppose (X, Y ) has a bivariate normal distribution with E[X] = E[Y ] = 0. Show that
1 1
P[X ≥ 0, Y ≥ 0] = P[X ≤ 0, Y ≤ 0] = + sin−1 ρ
4 2π
1 1
P[X ≤ 0, Y ≥ 0] = P[X ≥ 0, Y ≤ 0] = − sin−1 ρ [Ans]
4 2π
41.23 Normality of conditional distributions does not imply normality. Suppose the random vector (X, Y ) has the density
f(X,Y ) (x, y) = C exp −(1 + x2 )(1 + y 2 )
 
for x ∈ R and y ∈ R.
Find the marginal distributions of X and Y and show that the conditional distributions (X|Y = y) and (Y |X = x) are
both normal. [Ans]
Page 130 §42 Mar 10, 2020(20:25) Bayesian Time Series Analysis

42 The multivariate normal


42.1 The multivariate normal distribution. The n × 1 random vector X has a non-singular multivariate
normal distribution iff X has density
 
1 T
fX (x) = C exp − (x − µ) P(x − µ) for x ∈ Rn (42.1a)
2
where
• C is a constant so that the density integrates to 1;
• µ is a vector in Rn ;
• P is a real symmetric positive definite n × n matrix called the precision matrix.
42.2 Integrating the density. Because P is a real symmetric positive definite matrix, there exists an orthogonal
matrix L with
P = LT DL
where L is orthogonal and D is diagonal with entries d1 > 0, . . . , dn > 0. This result is explained in §38.4 on
page 118; the values d1 , . . . , dn are the eigenvalues of P.
Consider the transformation Y = L(X − µ); this is a 1 − 1 transformation: Rn → Rn which has a Jacobian with
absolute value:
∂(y1 , . . . , yn )
∂(x1 , . . . , xn ) = |(det(L)| = 1

Note that X − µ = LT Y. The density of Y is


    n  
1 T T 1 T Y 1 2
fY (y) = C exp − y LPL y = C exp − y Dy = C exp − dj yj
2 2 2
j=1
It follows that Y1 , . . . , Yn are independent with distributions N (0, 1 ), . . . , N (0, 1/dn ) respectively, and
1/d
√ √ √
d1 · · · dn det(D) det(P)
C= n/2
= n/2
=
(2π) (2π) (2π)n/2
So equation(42.1a) becomes
√  
det(P) 1
fX (x) = exp − (x − µ) P(x − µ)T
for x ∈ Rn (42.2a)
(2π)n/2 2
Note that the random vector Y satisfies E[Y] = 0 and var[Y] = D−1 . Using X = µ + LT Y gives
E[X] = µ
var[X] = var[L Y] = LT var[Y]L = LT D−1 L = P−1
T

and hence P is the precision matrix—the inverse of the variance matrix. So equation(42.1a) can be written as
 
1 1 T −1
fX (x) = √ exp − (x − µ) Σ (x − µ) for x ∈ Rn (42.2b)
(2π)n/2 det(Σ) 2
where µ = E[X] and Σ = P−1 = var[X]. This is defined to be the density of the N (µ, Σ) distribution.
Notes.
• A real matrix is the variance matrix of a non-singular normal distribution iff it is symmetric and positive
definite.
• The random vector X is said to have a spherical normal distribution iff X ∼ N (µ, σ 2 I). Hence X1 , . . . , Xn are
independent and have the same variance.
42.3 The characteristic function. Suppose the n-dimensional random vector X has the N (µ, Σ) distribution.
We know that using the transformation Y = L(X − µ) leads to Y ∼ N (0, D−1 ). Because L is orthogonal we have
X = µ + LT Y. Hence the characteristic function of X is:
h T i h T T T
i T
h T T i
φX (t) = E eit X = E eit µ+it L Y = eit µ E eit L Y for all n × 1 vectors t ∈ Rn .
But Y1 , . . . , Yn are independent with distributions N (0, 1/d1 ), . . . , N (0, 1/dn ), respectively. Hence
h T i 2 2 1 T −1
E eit Y = E ei(t1 Y1 +···+tn Yn ) = e−t1 /2d1 · · · e−tn /2dn = e− 2 t D t for all n × 1 vectors t ∈ Rn .
 

Applying this result to the n × 1 vector Lt gives


h T T i 1 T T −1 1 T
E eit L Y = e− 2 t L D Lt = e− 2 t Σt
3 Multivariate Continuous Distributions Mar 10, 2020(20:25) §42 Page 131

We have shown that if X ∼ N (µ, Σ) then


h T i T 1 T
φX (t) = E eit X = eit µ− 2 t Σt for all t ∈ Rn .
T T
µ+ 21 tT Σt
The moment generating function of X is E[et X] = et .

42.4 The singular multivariate normal distribution. In the last section we saw that if µ ∈ Rn and Σ is an
n × n real symmetric positive definite matrix, then the function φ : Rn → Cn with
T
µ− 12 tT Σt
φ(t) = eit for t ∈ Rn .
is a characteristic function. The condition on Σ can be relaxed to non-negative definite as follows:
Proposition(42.4a). Suppose µ ∈ Rn and V is an n × n real symmetric non-negative definite matrix. Then
the function φ : Rn → Cn with
T
µ− 21 tT Vt
φ(t) = eit for t ∈ Rn (42.4a)
is a characteristic function.
Proof. For n = 1, 2, . . . , set Vn = V + n1 I where I is the n × n identity matrix. Then Vn is symmetric and positive
definite and so
T 1 T
φn (t) = eit µ− 2 t Vn t
is a characteristic function.
Also φn (t) → φ(t) as n → ∞ for all t ∈ Rn . Finally, φ is continuous at t = 0. It follows that φ is a characteristic function
by the multidimensional form of Lévy’s convergence theorem3 .

If V is symmetric and positive definite, then we know that φ in equation(42.4a) is the characteristic function of
the N (µ, V) distribution.
If V is only symmetric and non-negative definite and not positive definite, then by §38.3 on page 118, we know
that some linear combination of the components is zero and the density does not exist. In this case, we say that
the distribution with characteristic function φ is a singular multivariate normal distribution.

42.5 Linear combinations of the components of a multivariate normal.


Proposition(42.5a). Suppose the n-dimensional random vector X has the possibly singular N (µ, Σ) distribu-
tion. Then for any n × 1 vector ` ∈ Rn the random variable Z = `T X has a normal distribution.
Proof. Use characteristic functions. For t ∈ R we have
T T 1 2 T
φZ (t) = E[eitZ ] = E[eit` X ] = φX (t`) = eit` µ− 2 t ` Σ`

and hence Z ∼ N `T µ, `T Σ` .
Conversely:
Proposition(42.5b). Suppose X is an n-dimensional random vector such that for every n × 1 vector ` ∈ Rn
the random variable `T X is univariate normal. Then X has the multivariate normal distribution.
T
Proof. The characteristic function of X is φX (t) = E[eit X ]. Now Z = tT X is univariate normal. Also E[tT X] = tT µ
and var[tT X] = tT Σt where µ = E[X] and Σ = var[X]. Hence Z ∼ N (tT µ, tT Σt). Hence the characteristic function
of Z is, for all u ∈ R:
T 1 2 T
φZ (u) = eiut µ− 2 u t Σt
Take u = 1; hence
T 1 T
φZ (1) = eit µ− 2 t Σt
T
But φZ (1) = E[eiZ ] = E[eit X ]. So we have shown that
T T
µ− 21 tT Σt
E[eit X
] = eit
and so X ∼ N (µ, Σ).
Combining these two previous propositions gives a characterization of the multivariate normal distribution:
the n-dimensional random vector X has a multivariate normal distribution iff every linear combination of
the components of X has a univariate normal distribution.

3
Also called the “Continuity Theorem.” See, for example, page 361 in [F RISTEDT &G RAY(1997)].
Page 132 §42 Mar 10, 2020(20:25) Bayesian Time Series Analysis

42.6 Linear transformation of a multivariate normal.


Proposition(42.6a). Suppose the n-dimensional random vector X has the non-singular N (µ, Σ) distribution.
Suppose further that B is an m × n matrix with m ≤ n and rank(B) = m; hence B has full rank.
Let Z = BX. Then Z has the non-singular N (Bµ, BΣBT ) distribution.
Proof. We first establish that BΣBT is positive definite. Suppose x ∈ Rm with xT BΣBT x = 0. Then yT Σy = 0 where
y = BT x. Because Σ is positive definite, we must have y = 0. Hence BT x = 0; hence x1 αT1 + · · · + xm αTm = 0 where
α1 , . . . , αm are the m rows of B. But rank(B) = m; hence x = 0 and hence BΣBT is positive definite.
The characteristic function of Z is, for all t ∈ Rm :
T T T 1 T T
φZ (t) = E[eit Z ] = E[eit BX ] = φX (BT t) = eit Bµ− 2 t BΣB T (42.6a)
Hence Z ∼ N (Bµ, BΣBT ).
What if B is not full rank? Suppose now that X has the possibly singular N (µ, Σ) distribution and B is any m × n
matrix where now m > n is allowed. Equation(42.6a) for φZ (t) still holds. Also if v is any vector in Rm , then
vT BΣBT v = zT Σz where z is the n × 1 vector BT v. Because Σ is non-negative definite, it follows that BΣBT is
non-negative definite. Hence Y = BX has the possibly singular N (Bµ, BΣBT ) distribution. We have shown the
following result:
Corollary(42.6b). Suppose that X has the possibly singular N (µ, Σ) distribution and B is any m × n matrix
where m > n is allowed. Then Y = BX has the possibly singular N (Bµ, BΣBT ) distribution.
Here is an example where AX and BX have the same distribution but A 6= B.

Example(42.6c). Suppose
√ the 2-dimensional random vector X ∼ N (0, I). Let Y1 = X1 + X2 , Y2 = 2X1 + X2 , Z1 = X1 2
and Z2 = (3X1 + X2 )/ 2. Then
 √ 
1 1 2 0
Y = AX where A = and Z = BX where B = 3/√2 1/√2
2 1
Let Σ = AAT = BBT . Then Y ∼ N (0, Σ) and Z ∼ N (0, Σ).

42.7 Transforming a multivariate normal into independent normals. The following proposition shows that
we can always transform the components of a non-singular multivariate normal into i.i.d. random variables with
the N (0, 1) distribution; see also §38.4 on page 118. We shall show below in §42.14 on page 138 how to convert
a singular multivariate normal into a non-singular multivariate normal.
Proposition(42.7a). Suppose the random vector X is has the non-singular N (µ, Σ) distribution.
Then there exists a non-singular matrix Q such that the QT Q = P, the precision matrix, and
Q(X − µ) ∼ N (0, I)
Proof. From §42.2 on page 130 we know that the precision matrix P = Σ−1 = LT DL where √ L is orthogonal
√ and
D = diag[d1 , . . . , dn ] with d1 > 0, . . . , dn > 0. Hence P = LT D1/2 D1/2 L where D1/2 = diag[ d1 , . . . , dn ]. Hence
P = QT Q where Q = D1/2 L. Because L is non-singular, it follows that Q is also non-singular.
If Z = Q(X − µ), then E[Z] = 0 and var[Z] = QΣQT = QP−1 QT = I. Hence Z ∼ N (0, I) and Z1 , . . . , Zn are
i.i.d. random variables with the N (0, 1) distribution.

It also follows that if Y = LX where L is the orthogonal matrix which satisfies LT DL = Σ−1 , then Y ∼
N (Lµ, D−1 ). Hence Y1 , Y2 , . . . , Yn are independent with var[Yk ] = 1/dk where 1/d1 , 1/d2 , . . . , 1/dn are the
eigenvalues of Σ.
We now have another characterization of this result: the random vector X has a non-singular normal distri-
bution iff there exists an orthogonal transformation L such that the random vector LX has independent
normal components.
An orthogonal transformation of a spherical normal is a spherical normal: if the random vector has the spherical
normal distribution N (µ, σ 2 I) and L is orthogonal, then Y ∼ N (Lµ, σ 2 I) and hence Y also has a spherical normal
distribution.
42.8 The marginal distributions. Suppose the n-dimensional random vector X has the N (µ, Σ) distribution.
Then the characteristic function of X is
T 1 T
φX (t) = E[ei(t1 X1 +···+tn Xn ) ] = eit µ− 2 t Σt for t ∈ Rn .
and hence the characteristic function of X1 is
1 2
φX1 (t1 ) = φX (t1 , 0, . . . , 0) = eiµ1 t1 − 2 t1 Σ11
and so X1 ∼ N (µ1 , Σ11 ). Similarly, Xj ∼ N (µj , Σjj ) where Σjj is the (j, j) entry in the matrix Σ.
3 Multivariate Continuous Distributions Mar 10, 2020(20:25) §42 Page 133

Similarly, the random vector (Xi , Xj ) has the bivariate normal distribution with mean vector (µi , µj ) and variance
matrix  
Σii Σij
Σij Σjj
In general, we see that every marginal distribution of a multivariate normal is normal.
An alternative method for deriving marginal distributions is to use proposition (42.6a) on page 132: for example,
X1 = aX where a = (1, 0, . . . , 0).
The converse is false!!
Example(42.8a). Suppose X and Y are independent two dimensional random vectors with distributions N (µ, ΣX ) and
N (µ, ΣY ) respectively where
     
µ1 σ12 ρ1 σ1 σ2 σ12 ρ2 σ1 σ2
µ= ΣX = and ΣY =
µ2 ρ1 σ1 σ2 σ22 ρ2 σ1 σ2 σ22
and ρ1 6= ρ2 . Let

X with probability 1/2;
Z=
Y with probability 1/2.
Show that Z has normal marginals but is not bivariate normal. See also exercise 43.15 on page 140.
Solution. Let Z1 and Z2 denote the components of Z; hence Z = (Z1 , Z2 ). Then Z1 ∼ N (µ1 , σ12 ) and Z2 ∼ N (µ2 , σ22 ).
Hence every marginal distribution of Z is normal.
Now E[Z] = µ and var[Z] = E[(Z − µ)(Z − µ)T ] = 1/2(ΣX + ΣY ). The density of Z is fZ (z) = 21 fX (z) + 12 fY (z) and this is
not the density of N (µ, 1/2(ΣX + ΣY ))—we can see that by comparing the values of these two densities at z = µ.
A special case is when ρ1 = −ρ2 ; then cov[Z1 , Z2 ] = 0, Z1 and Z2 are normal but Z = (Z1 , Z2 ) is not normal.
Now for the general case of a subvector of X.
Proposition(42.8b). Suppose X is an n-dimensional random vector with the possibly singular N (µ, Σ) distri-
bution, and X is partitioned into two sub-vectors:
X