Chapter 03 Random Variables Part A

Definition of a Random Variable
Random Variable [m-w.org] : a variable that is itself a function of the result of a statistical experiment in which each outcome has a definite probability of occurrence
Copyright Syed Ali Khayam 2009

A random variable is a mapping from an outcome s of a random experiment to a real number
X : S SX
domain range SX is called the image of X

X : S SX
Random R d Experiment Sample Space head tail 0 1 R X(s) Random Variable
Sx

X : S SX
X(s)
1 2 3 4 5 6
Random Experiment Sample Space, S
Sx Random Variable Image courtesy of www.buzzle.com/

More than one outcomes can be mapped to the same real number
X : S SX
X(s)
0 Random Experiment E i t Sample Space
1 Sx
Random Variable Image courtesy of www.buzzle.com/
Types of Random Variables

Discrete random variables: have a countable (finite or infinite) image
Sx = {0, 1} Sx = {, -3, -2, -1, 0, 1, 2, 3, }
Continuous random variables: have an uncountable image

Sx = (0, 1] Sx = R
Mixed random variables: have an image which contains continuous and discrete parts
Sx = {0} U (0, 1]
We will mostly focus on discrete and continuous random variables i bl
Discrete R d Di Random V i bl Variables
Probability Mass Function

The Probability Mass Function (pmf) or the discrete probability density function provides the probability of a particular point in the sample space of a discrete random variable (rv) For a countable SX={a0, a2, , an}, the pmf is the set of probabilities
pX (ak ) = Pr {X = ak }, k = 1, 2, , n
pX( k) (a
SX={a0=0, a1=1, , a5=5}, 0 1 2 3 4 5
X
10
Properties of a PMF
P1: 0 pX (ak ) 1 P2:
ak S X
(ak ) = 1
pX(ak)
SX={a0=0, a1=1, , a5=5}, 0 1 2 3 4 5
11
Cumulative Distribution Function (CDF) of Discrete Random Variable

The Cumulative Distribution Function (CDF) for a discrete rv is defined as:
FX (t ) = Pr {X t } = pX (x )
x t
pX(x) pmf
FX(x) CDF
12
Cumulative Distribution Function (CDF) of Discrete Random Variable

CDF can be used to find the probability of a range of values in a rvs image:
Pr {a < X b } = Pr {X b } Pr {X a } = FX (b ) FX (a )
pX(x) pmf FX(x) CDF
X
13
Properties of a CDF
F1: 0 FX (x ) 1 F2: a b
pX(x) pmf
< x < FX (a ) FX (b )
FX(x) CDF
14
Properties of a CDF
F3:
limx FX (x ) = 0 lim li x FX (x ) = 1
F4: FX (x i +1 ) = FX (x i ) + pX (x i +1 )
pX(x) pmf FX(x) CDF
X
15
Expected Value of a Discrete Random Variable

The expected value, expectation or mean of a discrete rv is the average value of the random variable What is the average value of a random variable whose image is SX={1, 6, 7, 9, 13}? { , , , , }
16

What is the average value of the following random variable whose image is SX={1, 6, 7, 9, 13}? If your answer is 7.2 then you assumed that all of the values in the rvs image have equal weights g q g
1 1 1 1 1 1 1 + 6 + 7 + 9 + 13 = (1 + 6 + 7 + 9 + 13) = 7.2 5 5 5 5 5 5
17

Mathematically, the expected value of a discrete random variable is:
{X } = X =
ak S X
Pr {X = ak }
In some cases the expected value does not converge cases, In such cases, we say that the expected value does not exist
19
Variance of a Random Variable

Variance of a rv is a measure of the amount of variation of a rv around its mean Intuitively, which of the following discrete rvs has a higher variance?
pX(x) 0.5 0.4 E{X}=3.87
qX(x) 0.4 0.2 E{X}=5.2
0.033 1 6 7 9 13 X
0.1 1 6 7 9 13 X
20

Intuitively, which of the following discrete rvs has a higher variance? qX(x) has a higher variance because it varies more around its mean than pX(x)
pX(x) 0.5 0 0.4 E{X} 3.87 E{X}=3.87 qX(x) 0.4 04 0.2 0.033 1 6 7 9 13 X 0.1 1 6 7 9 13 X E{X}=5.2
21

Mathematically, the variance of a discrete rv is defined as:
var {X } = =
2 X
ak S X
(a
{X }) P {X = ak } Pr
22

pX(x) ( ) 0.5 0.4 E{X}=3.87 qX(x) ( ) 0.4 0.2 0.1 1 6 7
2
E{X}=5.2 var{X}=15.36
var{X}=9.91 0.033 9 13 X
9
2
13
var {X } = (1 3.87) 0.5 +
var {X } = (1 5.2) 0.4 + (6 5.2) 0.2 + (7 5.2) 0.2 + (9 5.2) 0.1 + (13 5.2) 0.1 = 15 36 15.36
2 2 2 2
(6 3.87) 0.4 + (7 3.87) 0.033 + (9 3.87) 0.033 + (13 3.87) 0.033 = 9 91 9.91
2 2
23
Standard Deviation of a Random Variable

In many scenarios, we use the square root of the variance called its standard deviation
X = var {X }
24
Standard Deviation of a Random Variable
pX(x) 05 0.5 0.4 E{X}=3.87
qX(x) 0 0.4 0.2 0.1 01 E{X}=5.2 X=3.92
X=3.14 0.033 1 6 7 9 13 X
13
X = var {X } = 9.91 = 3.14
X = var {X } = 15.36 = 3.92
25
Discrete Uniform Random Variable

A discrete uniform rv, D, has a finite image and all the elements of the image have equal probabilities
Pr{D=k}
1/n x1 x2 x3 xn
What do you think are the expected value and standard deviation of this random variable?
26
Bernoulli Random Variable

A Bernoulli Random Variable is defined on a single event A This rv is based on an experiment called a Bernoulli trial
The experiment is performed and the event A either happens or does not happen
Thus the sample space of a Bernoulli rv is binary

Sample Space
A=head Ac = Not head = tail
X(s)
Bernoulli Tr B rial: Toss a a coin
Sx
27

A Bernoulli Random Variable is defined on a single event A This rv is based on a the experiment called a Bernoulli trial

Bernoulli Random Variable Pa akistan cric cket team plays Au ustralia
X(s)
A=Pak wins
R
Ac=Pak losses
Sample Space
Image Courtesies of http://www.tribuneindia.com and images.google.com
28

A Bernoulli Random Variable is defined on a single event A This rv is based on a the experiment called a Bernoulli trial
Of course the Pr{A} = 0 for this Bernoulli Random p experiment

X(s)
A=Pak wins
Variable
Pakistan cricket te n eam plays Australia s a
R
Ac=Pak losses
Sample Space
Image Courtesies of http://www.tribuneindia.com and images.google.com
29

Typical examples of Bernoulli rvs in communication:
Transmit a bit over a wireless channel
Outcomes:
0 > bit is received error-free 1 > bit received is not received error-free => bit is received with errors
Transmit a packet over the Internet

Outcomes:
0 > packet is received 1 > packet is not received => p p packet is lost en-route due to congestion g
30

Sample space of a Bernoulli rv, I, is binary Both outcomes are mapped to real numbers,
Traditionally: I(A) = 1 and I(Ac) = 0 are used to represent a Bernoulli rvs outcomes
The pmf of I is:

Pr{I = 1} = p Pr{I = 0} = 1 p
Pr{I=k} p 1-p I
0
31

Example: Consider the experiment of a fair coin toss. What is the expected value and the variance of this Bernoulli rv?
Pr{I=k} 0.5 I
32
Discrete Random Variables

Example: Consider the experiment of a fair coin toss. What is the expected value and th variance of thi B l d the i f this Bernoulli rv? lli ? Since the coin toss is fair: Pr{I = 1} = 0.5 and Pr{I = 0} = 0.5 E{I} = (1)x(0 5) + (0)x(0 5) = 0 5 (1)x(0.5) (0)x(0.5) 0.5 var{I} = (10.5)2x(0.5) + (00.5)2x(0.5) = 0.25
Pr{I=k} 0.5 I=0.5 I=0.5 0.5
0
33

Example: Consider a binary symmetric channel with probability of bit-error 0.1. What is the expected value and the variance of this Bernoulli rv?
Pr{I=k} 0.9
0.1 0 1 I
34

Example: Consider a binary symmetric channel with probability of bit-error 0.1. What is the expected value and the variance of this Bernoulli rv? p The event of interest here is a bit-error: => Pr{I = 1} = 0.9 and Pr{I = 0} = 0.1 E{I} = (1)x(0 9) + (0)x(0 1) = 0 9 (1)x(0.9) (0)x(0.1) 0.9 var{I} = (10.9)2x(0.9) + (00.9)2x(0.1) = 0.09 Note that the variance of this pmf is smaller than the variance of the coin toss pmf f
Pr{I=k} 0.9 I=0.3 I=0.9 0.1 0
I
35
Binomial Random Variable

Consider a collection of n independent Bernoulli trials A Binomial Random Variable is the total number of occurrences of an event A in this independent Bernoulli collection
Send n bits, count the number of bits that are received with errors Send n packets, count the number of packets that are not lost
36

If Ij(A)=1 and Ij(Ac)=0, j=1,2,,n, are used to represent the outcomes of the Bernoulli trials then the Binomial Random Variable, X, is V i bl X i
X = Ij
j =1
So what is the image of X? g
37

If Ij(A)=1 and Ij(Ac)=0, j=1,2,,n, are used to represent the outcomes of the Bernoulli trials then the Binomial Random Variable, X, Variable X is
X = Ij
j =1
So what is the image of X?

SX = {0, 1, 2, 3, , n}
38

Pr{Ij(A)=1} = p and Pr{Ij(A)=0} = 1-p Then a Binomial rv X is defined as:
X = Ij
j =1
And h A d the pmf of a bi f f binomial rv i i l is
n k p (1 p )n k Pr P {X = k } = k
39

The pmf of a Binomial rv X is:
n k p (1 p )n k b (k ; n, p ) = P {X = k } = Pr k
This pmf gives the probability that exactly k out of a total of n Bernoulli trials were successes Be very careful about the definition of a success
40

Pr{X=k}
k
Image courtesy of Wikipedia article on Binomial Distribution
Copyright Syed Ali Khayam 2009 41

Example: Consider a binary symmetric channel with probability of bit-error p. Find the Fi d th probability th t a packet of n bit i received with one b bilit that k t f bits is i d ith or more errors?
42

Example: Consider a binary symmetric channel with probability of bit-error p. Find the Fi d th probability th t a packet of n bit i received with one b bilit that k t f bits is i d ith or more errors? Pr{Ij = 1} = 0.1 and Pr{Ij = 0} = 0.9, j=1,2,,n Then the probability that a packet is received with errors is
Pr {(X = 1) (X = 2) (X = 3) (X = n )}
43

Pr {pkt with errs} = Pr {(X = 1) (X = 2) (X = 3) (X = n )} = Pr {X = 1} + Pr {X = 2} + Pr {X = 3} + + Pr {X = n } n n n n 1 p (1 p )n 1 + p 2 (1 p )n 2 + p 3 (1 p )n 3 + + p n (1 p )n n = n 1 2 3 n n n i = p i (1 p ) i i =1
44

n i n i Pr kt ith P {pkt with errs} = p (1 p ) i i =1
n
There is an easier way to compute the same probability by noting that:

Pr {pkt with errs} = Pr {X > 0} = 1 Pr {X < 0} = 1 Pr {X = 0} n 0 n 0 = 1 p (1 p ) 0 = 1 (1 p )
45
Connection Between Bernoulli and Binomial RVs s

I(A)=1, I(A)=0 S A Pr{A} SI Pr{I=1}=p Pr{I=0}=1-p Bernoulli Trial Bernoulli RV n Bernoulli trials SX
n k n k Pr {X = k } = p (1 p ) k
Binomial RV
46
Geometric Random Variable

A Geometric Random Variable, M, is the number of Bernoulli trials until the first occurrence of an event A The experiment is stopped as soon as event A is observed The image of a Geometric random variable is infinite but countable
SM = {1 2 3 } {1, 2, 3, }
48

Also call the mo led odified geo ometric pm mf
The pmf of a Geometric Random Variable, Z, is
pZ (k ) = Pr {Z = k } = (1 p )
OR
k 1
pZ (k ) = P {Z = k } = (1 p ) p Pr
Depending on whether the success trial is included in the total count or not
49
Image courtesy of Wikipedia article on Geometric Distribution

Copyright Syed Ali Khayam 2009 50
Connection between Geometric and Binomial RVs s

Can we find the probability of a Geometric random variable using the Binomial random variable?
Pr {Z = k } = (1 p ) p n k p (1 p )n k Pr {Z = k } = k
Geometric
Binomial
51
Connection between Geometric and Binomial RVs

Can we find the probability of a Geometric random variable using the Binomial random variable?
k p1 1 p k 1 Pr {X = k } = ( ) 1 Pr {X = k } = (1 p )
k 1
k 1
p = Pr {Z = k }
52

In k Bernoulli trials, there are k ways in which you can have 1 success and k-1 failures Since the Binomial random variable counts successes and failures, it sums and considers all the k outcomes together , g
53

For k=4, a Binomial random variable X jointly considers the outcomes 0001, 0010, 0100, 1000 Pr{X = 1} = Pr{(0001) U (0010) U (0100) U (1000)} Pr{X = 1} = Pr{0001} + Pr{0010} + Pr{0100} + Pr{1000} Since the underlying Bernoulli trials are independent: Pr{X = 1} = (k)Pr{one success in k trials} For the Geometric random variable we are only interested in variable, one of these outcomes, 0001 => Pr{Z = 1} = Pr{X = 1} / k
54
Memoryless Property
It can be shown that the Geometric rv satisfies the memoryless property The memoryless property is satisfied when:
Pr {Z = j + k Z k } = Pr {Z = j }
This property is also called the Markov Property
55
Memoryless Property
The memoryless property is satisfied when:
Pr {Z = j + k Z k } = Pr {Z = j }
For the Geometric rv, RHS of the above equation is:
Pr P {Z = j } = (1 p ) p
Thus to prove that the Geometric rv satisfies the memoryless property, we need to show that
Pr {Z = j + k Z k } = (1 p ) p
j
56
Memoryless Property
To prove that the Geometric rv satisfies the memoryless property, we need to show that
Pr {Z = j + k Z k } = (1 p ) p
j
Lets expand the LHS using the definition of conditional probability

Pr {Z = j + k Z k } = Pr {Z = j + k Z k } Pr {Z k }
Pr {Z = j + k } = Pr {Z k }
(1 p ) p = k k +1 k +2 (1 p ) p + (1 p ) p + (1 p ) p +
57
j +k
Memoryless Property
Continued from last page
(1 p ) p Pr {Z = j + k Z j } = k k +1 k +2 (1 p ) p + (1 p ) p + (1 p ) p +
=
j +k
(1 p )
k 0 j +k
j +k 1
p
2
(1 p ) ((1 p ) p + (1 p ) p + (1 p ) p + )
(1 p ) p = k (1 p )
= (1 p ) p
Summation over all possible values of the Geometric pmf
j
58
Memoryless Property
It can also be shown that the Geometric rv is the only discrete random variable that satisfies the memoryless property Because of the memoryless property, the Geometric rv can be thought of as the number of failures between two successes OR the inter-arrival time between successes
59

Chapter 03 Random Variables Part A

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Chapter 03 Random Variables Part A

Hochgeladen von

Copyright:

Verfügbare Formate

Definition of a Random Variable

Copyright Syed Ali Khayam 2009

Definition of a Random Variable

Copyright Syed Ali Khayam 2009

Definition of a Random Variable

Copyright Syed Ali Khayam 2009

Definition of a Random Variable

Random Experiment Sample Space, S

Sx Random Variable Image courtesy of www.buzzle.com/

Copyright Syed Ali Khayam 2009

Definition of a Random Variable

0 Random Experiment E i t Sample Space

Random Variable Image courtesy of www.buzzle.com/

Copyright Syed Ali Khayam 2009

Types of Random Variables

Continuous random variables: have an uncountable image

We will mostly focus on discrete and continuous random variables i bl

Copyright Syed Ali Khayam 2009

Discrete R d Di Random V i bl Variables

Copyright Syed Ali Khayam 2009

Probability Mass Function

SX={a0=0, a1=1, , a5=5}, 0 1 2 3 4 5

Copyright Syed Ali Khayam 2009

SX={a0=0, a1=1, , a5=5}, 0 1 2 3 4 5

Copyright Syed Ali Khayam 2009

Cumulative Distribution Function (CDF) of Discrete Random Variable

Copyright Syed Ali Khayam 2009

Cumulative Distribution Function (CDF) of Discrete Random Variable

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Expected Value of a Discrete Random Variable

Copyright Syed Ali Khayam 2009

Expected Value of a Discrete Random Variable

Copyright Syed Ali Khayam 2009

Expected Value of a Discrete Random Variable

Copyright Syed Ali Khayam 2009

Variance of a Random Variable

pX(x) 0.5 0.4 E{X}=3.87

qX(x) 0.4 0.2 E{X}=5.2

Copyright Syed Ali Khayam 2009

Variance of a Random Variable

Copyright Syed Ali Khayam 2009

Variance of a Random Variable

Copyright Syed Ali Khayam 2009

Variance of a Random Variable

var {X } = (1 3.87) 0.5 +

Copyright Syed Ali Khayam 2009

Standard Deviation of a Random Variable

Copyright Syed Ali Khayam 2009

Standard Deviation of a Random Variable

pX(x) 05 0.5 0.4 E{X}=3.87

qX(x) 0 0.4 0.2 0.1 01 E{X}=5.2 X=3.92

X = var {X } = 9.91 = 3.14

X = var {X } = 15.36 = 3.92

Copyright Syed Ali Khayam 2009

Discrete Uniform Random Variable

Copyright Syed Ali Khayam 2009

Bernoulli Random Variable

Thus the sample space of a Bernoulli rv is binary

Bernoulli Random Variable

Copyright Syed Ali Khayam 2009

Bernoulli Tr B rial: Toss a a coin