ECNG 6703 - Principles of Communications

Random Variable Fundamentals

Sean Rocke
September 9
th
, 2013
ECNG 6703 - Principles of Communications 1 / 23
Outline
1
Random Variable Denition
2
Cumulative Distribution Functions (CDFs)
3
Discrete & Continuous Random Variables
4
Expectation of a Random Variable
5
Conditional Distributions
6
Multiple Random Variables
7
Computing Probabilities with MATLAB
8
Conclusion
ECNG 6703 - Principles of Communications 2 / 23
Random Variable Denition
Random Variables Denition
Denition:
A real-valued function X() dened for sample points in a
sample space S.
Examples:
Number of heads obtained when tossing a coin
Outcome of rolling a sixsided die
Number of visits to a website
Measured voltage in a circuit
R
X
X()

S
ECNG 6703 - Principles of Communications 3 / 23
Random Variable Denition
Random Variables Denition
Example 1:
Consider the number of heads in a sequence of three coin tosses.
Sample space, := {TTT, TTH, THT, HTT, . . . , HHH}.
Number of heads, X() =
_

_
0, = TTT,
1, {TTH, THT, HTT},
2, {THH, HTH, HHT},
3, = HHH.
ECNG 6703 - Principles of Communications 4 / 23
Random Variable Denition
Random Variables Denition
Example 2:
Now dene a random variable to describe the event that the number of
heads in three tosses is even.
ECNG 6703 - Principles of Communications 5 / 23
Cumulative Distribution Functions (CDFs)
Cumulative Distribution Functions (CDF)
Denition:
For a RV, X, the CDF is the function dened by:
F
X
(x) = P(X x), < x <
Properties:
0 F
X
(x) 1
F
X
(x
1
) F
X
(x
2
), if x
1
< x
2
lim
x
F
X
(x) = F
X
() = 1
lim
x
F
X
(x) = F
X
() = 0
lim
xa
+ F
X
(x) = F
X
(a
+
) = F
X
(a), a
+
= lim
0<0
a +
ECNG 6703 - Principles of Communications 6 / 23
Cumulative Distribution Functions (CDFs)
CDFs
Example 3:
Determine and sketch the CDFs for Examples 1 & 2.
ECNG 6703 - Principles of Communications 7 / 23
Discrete & Continuous Random Variables
Discrete RVs & Probability Mass Functions (PMFs)
Discrete RV Denition:
X is a discrete RV if there exist distinct real numbers x
i
such that

i
P(X = x
i
) = 1
PMF Denition:
When X is a discrete RV taking distinct values x
i
, we dene its
PMF by p
X
(X = x
i
)
Properties:
0 p
X
(x
i
) 1
p
X
(x) = 0, if x = x
i
(i = 1, 2, . . .)

i
p
X
(x
i
) = 1
ECNG 6703 - Principles of Communications 8 / 23
Discrete & Continuous Random Variables
Discrete RVs & Probability Mass Functions (PMFs)
Example 4:
Identify which are discrete RVs: Birthdays of everyone who ever
existed, voltage measurement, heights of men/women, RGB values of
screen pixel.
Example 5:
Determine and sketch the PMFs for examples 1 & 2.
ECNG 6703 - Principles of Communications 9 / 23
Discrete & Continuous Random Variables
Continuous RVs & Probability Density Functions
(PDFs)
Continuous RV Denition:
X is a continuous RV if P(X B) =
_
B
f (x) dx, where the range
of X contains an interval of the real number line.
PDF Denition:
When X is a continuous RV taking values x R, we dene its
PDF by f
X
(X = x) =
dF
X
(x)
dx
.
Properties:
0 f
X
(x) 1
_

f
X
(x) dx = 1
f
X
(x) is piecewise continuous
P(a X b) =
_
b
a
f
X
(x) dx
F
X
(x) = P(X x) =
_
x

f
X
() d
ECNG 6703 - Principles of Communications 10 / 23
Discrete & Continuous Random Variables
Continuous RVs & Probability Density Functions
(PDFs)
Example 6:
The PDF of a continuous RV, X, is given by
f
X
(x) =
_

_
1
3
, 0 < x < 1,
2
3
, 1 < x < 2,
0, otherwise
Determine the corresponding CDF,
F
X
(x), and sketch f
X
(x) and F
X
(x).
Example 7:
The PDF of a continuous RV, X, is given by
f
X
(x) =
_
kx, 0 < x < 1,
0, otherwise
Where k is a constant.
1
Determine k and sketch f
X
(x).
2
Find and sketch the corresponding CDF, F
X
(x).
3
Find P(
1
4
< X < 2).
ECNG 6703 - Principles of Communications 11 / 23
Discrete & Continuous Random Variables
Special Distributions
Discrete:
Bernoulli
Binomial
Poisson
Geometric (0 & 1)
Continuous:
Uniform
Exponential
Normal or Gaussian
Laplace/doublesided exponential
ECNG 6703 - Principles of Communications 12 / 23
Expectation of a Random Variable
Expectation a RV, X
n
th
Moment:
E(X
n
) =
_

k
x
n
k
p
X
(x
k
), X : discrete,
_

x
n
f
X
(x)dx, X : continuous
Mean or Expected Value, 1
st
Moment :

X
= E(X) =
_

k
x
k
p
X
(x
k
), X : discrete,
_

xf
X
(x)dx, X : continuous
Variance, 2
nd
Central Moment :

2
X
= Var (X) = E{[X E(X)]
2
}

2
X
=
_

k
(x
k

x
)
2
p
X
(x
k
), X : discrete,
_

(x
X
)
2
f
X
(x)dx, X : continuous
ECNG 6703 - Principles of Communications 13 / 23
Expectation of a Random Variable
Expectation of a RV, X
Expectation of a Function:
E[g(X)] =
_

k
g(x
k
)p
X
(x
k
), X : discrete,
_

g(x)f
X
(x)dx, X : continuous
Example 8:
If X Uniform[a, b], nd
1
E[X]
2
E[X
2
]
3
Var (X)
4
E[e
sX
]
ECNG 6703 - Principles of Communications 14 / 23
Conditional Distributions
Conditional Distributions
Recall:
Conditional probability of A given that event B has occurred,
P(A|B) =
P(A B)
P(B)
, P(B) > 0
Denition:
Conditional CDF of a RV X given that B has occurred
F
X
(x|B) = P{(X x)| B} =
P{(X x) B}
P(B)
Conditional PMF of a discrete RV X given that B occurred
p
X
(x
k
|B) = P{(X = x
k
)| B} =
P{(X x
k
) B}
P(B)
Conditional PDF of a continuous RV X given that B occurred
f
X
(x|B) =
dF
X
(x|B)
dx
ECNG 6703 - Principles of Communications 15 / 23
Conditional Distributions
Conditional Distributions
Example 9:
To transmit message i using an optical communication system, light of
intensity
i
is directed at a photodetector. When light of intensity
i
strikes the photodetector, the number of photoelectrons generated is a
Poisson(
i
) random variable.
1
Find the conditional probability that the number of photoelectrons
observed at the photodetector is less than 2 given that message i
was sent.
2
Find the average number of photoelectrons observed at the
photodetector given that message i was sent.
3
Find the joint probability if the message to be sent is a
Geometric
0
(p) distribution.
Note: If X Poisson() then p
X
(k) =

k
e

k!
, k = 0, 1, . . .
If X Geometric
0
(p) then p
X
(k) = (1 p)p
k
, k = 0, 1, . . .
ECNG 6703 - Principles of Communications 16 / 23
Multiple Random Variables
Multiple Discrete RVs
Denition:
For multiple RVs, X
1
, X
2
, . . ., the joint PMF is given by
p
X
1
,...,X
n
(x
1
, . . . , x
n
) = P(X
1
= x
1
, . . . , X
n
= x
n
).
Important Relationships:
1
Joint CDF:
F
X
1
,...,X
n
(x
1
, . . . , x
n
) =

X
1
x
1

X
n
x
n
_
p
X
1
,...,X
n
(x
1
, . . . , x
n
)

2
Marginal PMF:
P(X
1
= x
1
, X
3
= x
3
, . . .) =

X
2
_
p
X
1
,...,X
n
(x
1
, . . . , x
n
)

3
Conditional PMF:
p
X
1
,..., X
n
|Y
(x
1
, . . . , x
n
, y) :=
P(X
1
= x
1
,..., X
n
= x
n
, Y = y)
P(Y = y)
, P(y = y) = 0
ECNG 6703 - Principles of Communications 17 / 23
Multiple Random Variables
Multiple Continuous RVs
Denition:
For multiple RVs, X
1
, . . . , X
n
, the joint distribution function is
given by F
X
1
,...,X
n
(x
1
, . . . , x
n
) = P(X
1
x
1
, . . . , X
n
x
n
).
Important Relationships:
1
Joint PDF:
f
X
1
,...,X
n
(x
1
, . . . , x
n
) =

n
F
X
1
,...,X
n
(x
1
,...,x
n
)
x
1
...x
n
2
Joint CDF:
F
X
1
,...,X
n
(x
1
, . . . , x
n
) =
_
x
1

_
x
n

f
X
1
,...,X
n
(
1
, . . . ,
n
) d
1
d
n
3
Marginal PDF:
f
X
1
, X
3
,...
(x
1
, x
3
, . . .) =
_

f
X
1
,...,X
n
(x
1
, . . . , x
n
) dx
2
,
f
X
1
(x
1
) =
_

f
X
1
,...,X
n
(x
1
, . . . , x
n
) dx
2
dx
n
4
Conditional PDF:
f
X
1
,...,X
n
|Y
(x
1
, . . . , x
n
, y) :=
f
X
1
,...,X
n
, Y
(x
1
,...,x
n
, y)
f
Y
(y)
, f
Y
(y) > 0
ECNG 6703 - Principles of Communications 18 / 23
Computing Probabilities with MATLAB
Computing Probabilities with MATLAB
Single RVs:
1
Using column vectors (When RVs take nitely many values)
p
X
(x
i
) =
_

_
p
X
(x
1
)
p
X
(x
2
)
.
.
.
p
X
(x
n
)
_

_
e.g., A = [0.1; 0.4; 0.2; 0.3];
2
Using MATLAB functions (For special functions e.g., poisspdf,
randn, . . . )
e.g., B = poisspdf(1,4); computes the Poisson pdf at x = 1.
e.g., C = poissrnd(4); generates a random number from
X Poisson(4).
ECNG 6703 - Principles of Communications 19 / 23
Computing Probabilities with MATLAB
Computing Probabilities with MATLAB
Multiple RVs:
1
Using matrices (When RVs take nitely many values)
p
XY
(x
i
, y
j
) =
_

_
p
XY
(x
1
, y
1
) p
XY
(x
1
, y
2
) p
XY
(x
1
, y
n
)
p
XY
(x
2
, y
1
) p
XY
(x
2
, y
2
) p
XY
(x
2
, y
n
)
.
.
.
.
.
.
.
.
.
.
.
.
p
XY
(x
n
, y
1
) p
XY
(x
n
, y
2
) p
XY
(x
n
, y
n
)
_

_
e.g., D = [7 2 8 5 4; 4 2 5 5 9; 2 4 8 5 1]/71;
Marginal Probabilities:
1
Just sum along rows or columns depending on what is required.
e.g., sum(D); produces a row vector, p
Y
(y).
e.g., sum(D,2); produces a column vector, p
X
(x).
ECNG 6703 - Principles of Communications 20 / 23
Computing Probabilities with MATLAB
Computing Probabilities with MATLAB
Example 9:
If X Geometric
0
(p) with p = 0.8, compute the probability that X
takes the value of an odd integer between 5 and 13.
We need to compute (1 p)[p
5
+ p
7
+ p
7
+ p
9
+ p
11
+ p
13
].
Example 10:
Let P denote the matrix whose ij entry is p
XY
(i , j ), and suppose that
P =
1
71
_
_
7 2 8 5 4
4 2 5 5 9
2 4 8 5 1
_
_
Determine the marginal PMFs, p
X
(i ) and p
y
(j ).
ECNG 6703 - Principles of Communications 21 / 23
Conclusion
Conclusion
We covered:
Random Variable Fundamentals
Computing probabilities with MATLAB
Your goals for next class:
Continue working with MATLAB and Simulink
Complete HW
Review required readings on RVs and Source Coding
Review notes on Channel Coding in prep for next class
ECNG 6703 - Principles of Communications 22 / 23
Q & A
Thank You
Questions????
ECNG 6703 - Principles of Communications 23 / 23