ENCS 6161

Probability and Stochastic Processes

Concordia University
Course Material 4

Cumulative distribution function for

one random variable

Let a random experiment have a sample space and an event class .

A random variable is a function from the sample space to ( the
set of real numbers) with the property that the set
= : is in for every in .
The reason for this definition is that all events of interest in the real
line can be expressed in terms of sets of the form : .
The cumulative distribution function (cdf) of random variable is
defined as the probability of the event :
=
< <
It is the probability that the random variable takes on a value in the set (, ], or
in terms of the underlying sample space, it is the probability of the event
: .

Since the event and its probability vary as is varied, ()

is a function of the variable .
2

Example 1: Three coin tosses

X takes on only the values 0, 1, 2, and 3 with
probabilities 1/8, 3/8, 3/8, and 1/8, respectively.
Find .

 of is the sum of the probabilities of the

outcomes from 0,1,2,3 that are less than or
equal to .
The resulting cdf is a nondecreasing staircase
function that grows from 0 to 1. It has jumps at
the points 0, 1, 2, 3 of magnitudes 1/8, 3/8, 3/8,
and 1/8, respectively.

 If is a small positive number:

1
1 = 1 = 0 =
8
1 3 4 1
1 = 1 = 0 1 = + = =
8 8 8 2

Thus, the limit of the cdf as approaches 1 from the left is 1/8; but its
value at = 1 is 4/8.

1 + = 1 + = 0 1

4
=
8

Thus, the limit of the cdf as approaches 1 from the right is 4/8.

The cdf is continuous from the right and equal to at the point = 1.
Dots are used to indelicate the value of the cdf at the points of
discontinuity.

The cdf can be written compactly in terms of the unit step functions:
0,
<0
=
1,
0
1
3
3
1
= + 1 + 2 + ( 3)
8
8
8
8
5

Uniform random variable in the interval ,

The random variable takes on values uniformly in the
interval [, ]
= = = 0

= =

= = = 1

>

() is a nondecreasing continuous function that grows from 0 to 1 as

varies from its minimum value to its maximum value.

Example 2
The waiting time of a customer at a taxi stand
is zero if the customer finds a taxi parked at the
stand, and a uniformly distributed random length
of time in the interval 0, 1 (in hours) if no taxi
is found upon arrival. The probability that a taxi
is at the stand when the customer arrives is .
Find the cdf of .

By applying the theorem on total probability:

=
= | + | (1 )

0
| =
1
0
| =
1
=

0 + (0)(1 )
1 + 1
1 + (1)(1 )

<0
0
<0
01
1
<0
01
1
8

0
= + 1
1

<0
01
1

The cdf has a discontinuity at = 0 and has a

continuity over the interval (0, )

1. The cdf of discrete random variables is a right

continuous staircase function of with jumps
at a countable set of point 0 , 1 ,
=

( ) =

( )( )

where the pmf:

= [ = ]
gives the magnitude of the jumps in the cdf.

10

2. The cdf of a continuous random variables is

continuous everywhere and, in addition, is
sufficiently smooth so that it can be written as
an integral of some nonnegative function
():

11

3. The cdf of a random variables of mixed type

is a random variable that has jumps on a
countable set of points 0 , 1 , , but that also
increases continuously over at least one
interval of values of . The cdf for these
random variables has the form:
= 1 + 1 2 ()
where 0 < < 1, and 1 () is the cdf of a discrete
random variable and 2 () is the cdf of a continuous
random variable.
12

Properties of cdf
i.
0 1
It follows from the fact that cdf is a probability.

ii.
lim () = 1

iii.

lim () = 0

iv. () is a nondecreasing function of , i.e., if < ,

then (). It follows from the fact that the
event { } is a subset of the event { } and so
it must have a smaller or equal probability.
13

v. is continuous from the right, i.e., for

> 0,
= lim ( + ) = (+ )
0

vi.
< = ()
This property readily follows from the fact that the event
{ } can be expressed as the union of mutually
exclusive events:

= <

vii.

= = ( )

This property states that the probability that = is

given by the magnitude of the jump of the cdf at the point
. This implies that if the cdf is continuous at a point ,
then = = 0
14

 Properties (vi) and (vii) can be combined to

compute the probabilities of other types of
intervals. For example,
= = <
Then,

= = + <
=
+
=
()

viii.
> = 1 ()
It follows from the fact: > =

15

Example 3
Let be the number of heads in three tosses of a
fair coin. Use its cdf to find the probability of the
following events:
(a) = 1 < 2
(b) = 0.5 < 2.5
(c) = 1 < 2

16

(a) = 1 < 2
By property (vi):

7 4 3
1 < 2 = 2 1 = =
8 8 8
(b) = 0.5 < 2.5
Since cdf is continuous at = 0.5 and = 2.5:
7 1 6
0.5 < 2.5 = 2.5 0.5 = =
8 8 8
(c) = 1 < 2
Since,
1<2 =2 = 12
Therefore,
1 <2 + =2 = 1 2
Since,
and
we have

= 2 = 2 2

(From property vii )

1 2 = 2 1

(From Eq. A)

1 < 2 + 2 2 = 2 1
4 1 3

1 < 2 = 2 1 = =
8 8 8
17

Example 4
Let X be a uniform random variable with = 0
and = 1. Use the cdf of to find the
probability of the following events:
(a) = 0.5 < 0.25
(b) = 0.3 < < 0.65
(c) = 0.4 > 0.2

18

(a) = 0.5 < < 0.25

Since the cdf of is continuous at every point,
0.5 < < 0.25 = 0.25 0.5
= 0.25 0 = 0.25
(b) = 0.3 < < 0.65
0.3 < < 0.65 = 0.65 0.3
= 0.65 0.3 = 0.35
(c) = 0.4 > 0.2
0.4 > 0.2 = 0.4 < 0.2 0.4 > 0.2
= < 0.2 > 0.6

= < 0.2 + > 0.6

= 0.2 + 1 0.6
= 0.2 + (1 0.6) = 0.6
19

The probability density function

The probability density function of , if it exists, is defined as the
derivative of its cdf ()
()
=

The pdf is an alternative and more useful way of specifying the

information contained in the cdf.
The pdf represents the density of probability at point x in the
following sense:
< + = +
+
=

Thus, if the cdf has derivative, then as becomes very small,

< + =
Thus, represents the density of the probability at point in
the sense that the probability that is in a small interval in the
vicinity of is approximately equal to .
20

Properties of pdfs
Since the cdf is a nondecreasing function of ,
0.
ii. Since the probability of the event of the form: falls
in a small interval of width about the point is
, the probabilities of events involving can
then be found in terms of the pdf by adding the
probabilities of intervals of width . As the width of
the intervals approaches zero:
i.

The probability of an interval is the area under () in

that interval.
21

iii. The cdf of can be obtained by integrating the pdf:

Thus, the pdf completely specifies the behavior of a continuous

random variable.
iv. By letting x tend to infinity in the above equation, we obtain a
normalization condition for pdfs:

= 1
22

 A valid pdf can be formed from any nonnegative,

piecewise continuous function () that has a
finite integral:

= <

Then, letting

=
,

we obtain a function that satisfies, the

normalization condition.

Note that a pdf () must be defined for all real

values of . If does not take on values from
some region of the real line, we set = 0 in
that region.
23

Uniform random variable

0

1
1
=
0

<

>

< , >

24

Exponential random variable

1
0
> =

>0
= = 1 >
0
<0
=
1
0
()
0
=
=

<0
0

25

Example 5: Laplacian Random

Variable
The pdf of the samples of the amplitude of
speech waveforms is found to decay
exponentially at a rate as follows:
=
< <
Find: (a) the constant , and (b) the probability
of <

26

(a)

1=

1=

= 2

= 2

(b)

2
=

=
2

||
< =

= 2

2
0

=
= 1
0
27

pdf of a discrete random variable

()
=

The derivative of the cdf does not exist at points where the cdf is not
continuous. Therefore, the concept of pdf as defined above does not
apply to discrete random variables.

()
=
=

(1)

28

Example 6
Let X be the number of heads in three coin
tosses.
(a) Find the pdf of X.
(b) Find [1 < 2] and [2 < 3] by
integrating the pdf obtained in (a).

29

(a)

1
3
3
1
= + 1 + 2 + 3
8
8
8
8
By using (1)
1
3
3
1
= + 1 + 2 + 3
8
8
8
8

(b)

2+

3
8
1+
3
3
2<3 =
=
8

2
1<2 =

30

Conditional cdfs and pdfs

The conditional cdf of , given , is defined as

=
,
> 0
[]
The conditional pdf of , given , is then given
by
(|)
| =

31

Example 7
The lifetime of a machine has a continuous cdf
. Find the conditional cdf and pdf given the
event = > , that is, the machine is still
working at time .

32

 > = >
>
=
>

< > =
<
>
0
> = ()
1 ()

> =
>

0

=
1 ()

<

<

33

 Let 1 , 2 , , form the partition of the sample

space , then the cdf and pdf of can be obtained
in terms of conditional cdfs and pdfs by using
the theorem on total probability:
=

|
=1

|
=1

=
=

|
=1
34

Expected value of a random variable

The expected value or mean of a random
variable is defined as:

provided that this integral converges

absolutely, that is,

|| =

|| <

35

Mean of a uniform random variable

=

< , >

1
=
=

1
=
2

+
=
2
36

Mean of an exponential random

variable
0
=

<0
0

Integration by part: =
= ,
=
= ,
=
=

1 1
=0+ =

0
37

 The mean of a random variable whose pdf is symmetric

about = , i.e.,
= +
is =

Consider the integral

=
(1)
Now, since in the integrand of and ( ) has odd
symmetry about = , () has even symmetry about the
same point, the integrand has an odd symmetry about the
point = . Hence, the = 0.
Thus, from (1):

=
38

Mean of a Gaussian random variable

=

2 /2 2

< <

2
Since the pdf of a Gaussian random
variable is symmetric about the point
= .
=

39

Expected value of a function of a

random variable, = ()
[] can be found directly in terms of the pdf of .
Divide the -axis into intervals of length , index the
intervals with the index and let be the value in the
center of the kth interval.

40

 Suppose that () is strictly increasing, then the

kth interval of width in the y-axis has a unique
corresponding equivalent event of width in the
x-axis.
Let be the value of the kth interval of the xaxis such that = . Since
=

By letting 0, we have

This equation is valid even if () is not strictly

increasing
41

TO BE CONTINUED

42