ECE270: Handout 5

Introduction to Random Variables (RVs)

Outline: 1. informal denition of a RV, 2. three types of a RV: a discrete RV, a continuous RV, and a mixed RV, 3. a general rule to nd probability of events concerning a RV, 4. cumulative distribution function (CDF) of a RV, 5. formal denition of a RV using CDF, 6. discrete RV: probability mass function (pmf ) and CDF, 7. continuous RV: probability density function (pdf ) and CDF, 8. basic properties of the CDF.

The outcome of a random experiment need not be a numbers. Examples are: coding the incoming patients in a hospital according to their insurance and health status, randomly selecting a committee from a group of people, randomly selecting balls from an urn, randomly selecting cards from a deck. Usually we are interested in some measurement or numerical attribute of the outcome. Examples are: counting the number of heads when tossing a coin 10 times, the number of re-transmission needed until the receiver receives the data packet correctly, the number of errors in erroneous received data packets, the lifetime of a memory chip, the number of packets arriving in t sec at a server, the number of queries arriving in t sec at a call center, the number of particles emitted by a radioactive mass during a xed time period, the random thermal noise being added to the signal at the receiver of a communication system at a specic time. In these examples, we assign a real number to the outcome of the random experiment through measurement. Since the outcomes are random, the results of the measurements will be random too. Hence it makes sense to talk about the probabilities of the resulting numerical values. Informal Denition of a RV A RV X is a function that assigns a real valued number x = X ( ) to each outcome S (Recall: a function is a rule for assigning a numerical value to each element of a set). Sample space S is the domain of the RV X and the set of all real numbers taken on by X , Sx , Sx R, is the range of the RV. Three Types of a RV Three types of RVs: i) discrete, ii) continuous, iii) mixed.

i) The range of a discrete RV X is a countable set (either nite or innite) Sx = {x1 , x2 , ...} or Sx = {x1 , x2 , ..., xn }. ii) The range of a continuous RV X is an uncountable set, e.g., Sx = [0, ), Sx = (, ), Sx = [0, 1], Sx = [a, b] for < a < b < . iii) The range of a mixed RV X is the union of an uncountable and a countable sets. Notation: capital letters X , Y , Z , V , U , ... denote RVs, lowercase letters x, y, z, v, u, ... denote possible values of RVs. Consider the RV X , the function or rule that assigns a real number x = X ( ) to the outcome S is xed and deterministic, e.g., the rule of count the number of heads when we toss a coin 3 times. The randomness in the experiment is complete as soon as we toss the coin 3 times. The process of counting is deterministic. In some random experiments the outcome is already the numerical value we are interested in, e.g., measure the lifetime of a chip under certain conditions, X ( ) = (identity function). The distribution of the values of a RV X is determined by the probabilities of the basic events of the underlying random experiment, i.e., we should be able to compute the probability of the observed value of X in terms of the probability of the underlying event. The specication of the measurements on the outcomes of a random experiment denes a function on S and hence a RV. A function of a RV is another RV. EXAMPLE 1 A fair coin is tossed 3 times and the sequence of heads and tails is noted. The sample space is S = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T } and the outcomes are equally probable. a) Let X be the number of heads in the 3 tosses. Sx = {0, 1, 2, 3}. b) A player pays 1.5$ to play the following game: the player receives 1$ if X = 2 and 8$ if X = 3, but nothing otherwise. Let Y be the reward to the player. Sy = {0, 1, 8}. c) Let Z be a function of X such that Z = 0 if X {0, 1, 2} and Z = 1 if X = 3. Sz = {0, 1}. X ( ) Y ( ) Z ( ) HHH 3 8 1 HHT 2 1 0 HTH 2 1 0 THH 2 1 0 HTT 1 0 0 THT 1 0 0 TTH 1 0 0 TTT 0 0 0

P (X = 0) = P ({T T T }) = 1/8 P (X = 1) = P ({T T H, T HT, HT T }) = 3/8 P (X = 2) = P ({T HH, HHT, HT H }) = 3/8

Page 2

P (X = 3) = P ({HHH }) = 1/8 P (Y = 0) = P ({T T T, T T H, T HT, HT T }) = 4/8 = 1/2 P (Y = 1) = P ({T HH, HT H, HHT }) = 3/8 P (Y = 8) = P ({HHH }) = 1/8 P (Z = 0) = P (X {0, 1, 2}) = P (X = 0) + P (X = 1) + P (X = 2) = 1/8 + 3/8 + 3/8 = 7/8 P (Z = 1) = P (X = 3) = 1/8 or alternatively: P (Z = 0) = P ({T T T, T T H, T HT, HT T, HHT, HT H, T HH }) = 7/8 P (Z = 1) = P ({HHH }) = 1/8

A General Rule to Find Probabilities of Events Concerning a RV X The example shows a general technique to nd the probability of events involving RVs. To nd the probability of X B (where B R) we need to nd the set of outcomes A, A S that are mapped to B , i.e., the set A = { : X ( ) B }. If the experiment outcome A then event A occurs. Hence X ( ) B event B occurs. If event B occurs then X ( ) B implies A event A occurs. So: P (X B ) = P (A) = P ({ : X ( ) B }). We refer to A and B as equivalent events. In some random experiments the outcome is already the numerical value we are interested in. In such cases we simply let X ( ) = , i.e., the identity function is used to obtain a random variable. Example: measure the received signal at a receive antenna. CDF of a RV X Regardless of X being discrete, continuous, or mixed, the cumulative distribution function (CDF) of a RV X is dened as the probability of the event B = {X x}: FX (x) = P (B ) = P ({X x}) P (X x) for <x<

The event B = {X x} and its probability vary as x is varied. Hence, FX (x) is a function of the variable x. In terms of the underlying random experiment FX (x) = P ({ : X ( ) x}). an event Formal Denition of a RV Using CDF We are ready for a formal denition of a RV: Consider a random experiment with sample space S . The RV X is a function from S to real line R with the property that the set A = { : X ( ) b} is an event for every b R. This denition simply requires that every set A has a well dened probability in the underlying random experiment.

Page 3

Discrete RV: pmf and CDF A discrete RV X has the CDF FX (x) that is right continuous, non-decreasing staircase function of x, with jumps at points x1 , x2 , ... where xi Sx , and it grows from 0 to 1: FX (x) =
i=1

P (X = xi ) u(x xi )
pmf

pX (x) = P (X = x) = P ({ : X ( ) = x}), for x a real number, is called the probability mass function (pmf) of X , and is the magnitude of the jump in CDF at the point x. pX (x) is a function of x over the real line and is nonzero only at the points x1 , x2 , ... Sx . pmf pX (x) for all x Sx provides all information required to calculate probabilities of any event involving a discrete RV X . We can obtain CDF from pmf and vice versa. We can forget about the underlying random experiment, its sample space S and its associated probability law and just work with Sx and the pmf of X , or equivalently the CDF of X . The pmf of X is the probability of all elementary events from Sx . The pmf pX (x) satises three properties: 1) 0 pX (x) 1, 2) xSx pX (x) = 1, 3) P (X B ) = xB pX (x) where B R. EXAMPLE 2 (CDF of a discrete RV, back to example 1) Find and plot FX (x) in example 1.

Continuous RV: pdf and CDF A continuous RV X has the CDF FX (x) that is continuous everywhere and is a non-decreasing function of x that grows from 0 to 1 as x ranges from its minimum to its maximum values. Also, FX (x) is smooth enough that it can be written as an integral of some nonnegative function fX (x):
x

FX (x) =

fX (x) dx
pdf

Considering FX (x) we calculate probabilities as integral of probability density function (pdf) over an interval of the real line (, x]. We can obtain CDF from pdf and vice versa. For a continuous RV X we have P (X = x) = 0. EXAMPLE 3 (CDF of a continuous RV) We spin an arrow attached to the center of a circular board. Let be the angle of the arrow where 0 < 2 . The probability that falls in a subinterval of (0, 2 ] is equal to the length of the subinterval divided by 2 . We dene the RV X () = /2 . What is FX (x)?

Page 4

EXAMPLE 4 (CDF of a mixed RV) The waiting time X of a customer at a taxi stand is zero if the customer nds a taxi parked at the stand, and a uniformly distributed random length of time in the interval [0, 1] (in hour) if no taxi is found upon arrival. The probability that a taxi is at the stand when the customer arrives is p. What is the CDF of X ? SOLUTION: using the total probability theorem we have FX (x) = P (X x) = P (X x|nd taxi)p + P (X x|no taxi)(1 p) 1 x>1 1 x0 x 0x1 P (X x|nd taxi) = P (X x|no taxi) = 0 x<0 0 x<0 x>1 1 p + 1 (1 p) = 1 1 p + x(1 p) = p + x(1 p) 0 x 1 FX (x) = P (X x) = 0 x<0

Basic Properties of the CDF The three axioms of probability and the corresponding properties imply that the CDF has the following properties: (i) 0 FX (x) 1 (ii) limx FX (x) = 1 (iii) limx FX (x) = 0 (iv) FX (x) is a nondecreasing function of x, that is, if a < b then FX (a) FX (b) (v) FX (x) is continuous from the right, i.e., for h > 0, FX (b) = limh0 FX (b + h) = FX (b+ ). This property implies that at points of discontinuity, the CDF is equal to the limit from the right. These ve properties conrm that, in general, CDF is a nondecreasing function of x that grows from 0 to 1 as x increases from to . More properties follow: (vi) P (a < X b) = FX (b) FX (a) (vii) P (X = b) = FX (b) FX (b ) = FX (b+ ) FX (b ). This property says that the probability that X = b is equal to the magnitude of the jump of the CDF at the point b. So if the CDF is continuous at a point b, then P (X = b) = 0. (viii) P (X > x) = 1 FX (x)

Page 5

Based on above properties we can compute P (a X b), P (a X < b), P (a < X < b). Since {a X b} = {a} {a < X b}, we have: P (a X b) = P (X = a) + P (a < X b) = FX (a) FX (a ) + FX (b) FX (a) = FX (b) FX (a ) Since {a X b} = {b} {a X < b}, we have: P (a X < b) = P (a X b) P (X = b) = FX (b ) FX (a ) Since {a X < b} = {a} {a < X < b}, we have: P (a < X < b) = P (a X < b) P (X = a) = FX (b ) FX (a) NOTE: if the CDF is continuous at the endpoints of an interval, then the endpoints have zero probabilities, and therefore, they can be included in, or excluded from, the interval without aecting the probabilities.

EXAMPLE 5 (back to example 1) Consider example 1 and let A = {1 < X 2}, B = {0.5 X < 2.5}, C = {1 X < 2}. Find P (A), P (B ), P (C ).

EXAMPLE 6 (back to example 3) Find P (0.5 < X < 0.25), P (0.3 < X < 0.65), P (|X 0.4| > 0.2).

Page 6