DC Digital Communication PART3

A property of MVG_OMALLOOR 11/25/2008
ELEC 303 – Random Signals
Lecture 20 – Random processes

Dr. Farinaz Koushanfar
ECE Dept., Rice University
Nov 24, 2008
Lecture outline
• Basic concepts
• Statistical averages,
• Autocorrelation function
• Wide sense stationary (WSS)
• Multiple random processes
1
PDF processed with CutePDF evaluation edition www.CutePDF.com
Random processes
• A random process (RP) is an extension of a RV
• Applied to random time varying signals
• Example: “thermal noise” in circuits caused by
the random movement of electrons
• RP is a natural way to model info sources
• RP is a set of possible realizations of signal
waveforms governed by probabilistic laws
• RP instance is a signal (and not just one number
like the case of RV)
Example 1
• A signal generator generates six possible
sinusoids with amplitude one and phase zero.
• We throw a die, corresponding to the value F,
the sinusoid frequency = 100F
• Thus, each of the possible six signals would be
realized with equal probability
• The random process is X(t)=cos(2π × 100F t)
2
Example 2
• Randomly choose a phase Θ ~ U[0,2π]
• Generate a sinusoid with fixed amplitude (A)
and fixed freq (f0) but a random phase Θ
• The RP is X(t)= A cos(2πf0t + Θ)
X(t)= A cos(2πf0t + Θ)
3
Example 3
• X(t)=X
• Random variable
X~U[-1,1]
Random processes
• Corresponding to each ωi in the sample space
Ω, there is a signal x(t; ωi) called a sample
function or a realization of the RP
• For the different ωI’s at a fixed time t0, the
number x(t0; ωi) constitutes a RV X(t0)
• In other words, at any time instant, the value
of a random process is a random variable
4
Example: sample functions of a

random process
Example 4
• We throw a die, corresponding to the value F,
the sinusoid frequency = 100F
• Thus, each of the possible six signals would be
realized with equal probability
• The random process is X(t)=cos(2π × 100F t)
• Determine the values of the RV X(0.001)
• The possible values are cos(0.2π), cos(0.4π),
…, cos(1.2π) each with probability 1/6
5
Example 5
• Ω is the sample space

for throwing a die
• For all ωi let x(t; ωi)=
ωi e-1
• X is a RV taking values
e-1, 2e-1, …, 6e-1, each
with probability 1/6
Example 6
• Example of a discrete-time random process
• Let ωi denote the outcome of a random
experiment of independent drawings from
N(0,1)
• The discrete–time RP is {Xn}n=1 to ∞, X0=0, and
Xn=Xn-1+ ωi for all n≥1
6
Statistical averages
• mX(t) is the mean, of the random process X(t)
• At each t=t0, it is the mean of the RV X(t0)
• Thus, mX(t)=E[X(t)] for all t
• The PDF of X(t0) denoted by fX(t0)(x)
∞
E [ X ( t0 )] = m X ( t0 ) = ∫ xf x ( t0 ) ( x )dx
−∞
Mean of a random process
7
Example 7
• Randomly choose a phase Θ ~ U[0,2π]
• Generate a sinusoid with fixed amplitude (A)
and fixed freq (f0) but a random phase Θ
• The RP is X(t)= A cos(2πf0t + Θ)
• We can compute the mean
• For θ∈[1,2π], fΘ(θ)=1/2π, and zero otherwise
• E[X(t)]= ∫{0 to 2π} A cos(2πf0t+Θ)/2π.dθ = 0
Autocorrelation function
• The autocorrelation function of the RP X(t) is
denoted by RX(t1,t2)=E[X(t1)X(t2)]
• RX(t1,t2) is a deterministic function of t1 and t2
8
Example 8
• The autocorrelation of the RP in ex.7 is
• We have used
Example 9
• X(t)=X
• Random variable X~U[-1,1]
• Find the autocorrelation function
9
Wide sense stationary process

• A process is wide sense stationary (WSS) if its
mean and autocorrelation do not depend on
the choice of the time origin
• WSS RP: the following two conditions hold
– mX(t)=E[X(t)] is independent of t
– RX(t1,t2) depends only on the time difference τ=t1-
t2 and not on the t1 and t2 individually
• From the definition, RX(t1,t2)=RX(t2,t1) If RP
is WSS, then RX(τ)=RX(-τ)
Example 8 (cont’d)
• The autocorrelation of the RP in ex.7 is
• Also, we saw that mX(t)=0

• Thus, this process is WSS
10
Example 10
• Randomly choose a phase Θ ~ U[0,π]
• Generate a sinusoid with fixed amplitude (A) and
fixed freq (f0) but a random phase Θ
• The new RP is Y(t)= A cos(2πf0t + Θ)
• We can compute the mean
• For θ∈[1,π], fΘ(θ)=1/π, and zero otherwise
• MY(t) = E[Y(t)]= ∫{0 to π} A cos(2πf0t+Θ)/π.dθ
= -2A/π sin(2πf0t)
• Since mY(t) is not independent of t, Y(t) is
nonstationary RP
Multiple RPs
• Two RPs X(t) and Y(t) are independent if for all t1
and t2, the RVs X(t1) and X(t2) are independent
• Similarly, the X(t) and Y(t) are uncorrelated if for
all t1 and t2, the RVs X(t1) and X(t2) are
uncorrelated
• Recall that independence uncorrelation, but
the reverse relationship is not generally true
• The only exception is the Gaussian processes
(TBD next time) were the two are equivalent
11
Cross correlation and joint stationary

• The cross correlation between two RPs X(t)
and Y(t) is defined as
RXY(t1,t2) = E[X(t1)X(t2)]
clearly, RXY(t1,t2) = RXY(t2,t1)
• Two RPs X(t) and Y(t) are jointly WSS if both
are individually stationary and the cross
correlation depends on τ=t1-t2
for X and Y jointly stationary, RXY(τ) = RXY(-τ)
12
A property of MVG_OMALLOOR
Outline
Lecture #4
Stochastic Process z Stochastic Process
z Counting Process
z Poisson Process
ดร. อนันต ผลเพิ่ ม z Brownian Motion Process
Anan Phonphoem, Ph.D.
anan@cpe.ku.ac.th
z Autocovarience and Autocorrelation
z Random Sequence
http://www.cpe.ku.ac.th/~anan
Computer Engineering Department
Kasetsart University, Bangkok, Thailand
z Stationary Process
z Wide-sense Stationary Process
7/9/2003 2 7/9/2003
Definition Random Variable

X is a function that maps each outcome, s, in S
to a real number X(s), x
Stochastic Process
S X(s) = x
s
Random Function of Time
Random Process -2.0 -1.5 -1.0 -0.5
x
0 0.5
SX
3 7/9/2003 4 7/9/2003
Random Process Example 1

X(t) is a function that maps each outcome, s, in S z Taking temperature at the
to a time function x(t,s)
surface of a space shuttle
X(t,s) = x(t,s) x(t,s1) z Starting at launch time t = 0
S
z X(t) = temp in degree
s x(t,s2) Celsius on the surface
Ensemble z Each launch, record x(t,s)
x(t,sn) (Sx)
Source:www.inventorsmuseum.com
5
Sample Functions 6
7/9/2003 7/9/2003
1
Example 1 Example 1
x(t,e1)
x(t,e1) Average temp at e1

x(t,e2) for completed mission is
5000 C Î E[x(t, e1)]
2500.10
Time Average
…
Source:www.analyticalsci.com/Astronomy/ Hansen
x(t,en) Average temp of engine
At time t = 2500.10 sec at t = 2500.10 sec is
x(2500.10, e1) = 1200 C 1320 C Î E[X(2500.10)]
Ensemble Average
e1 = 1st launch measuring
Sample functions
7 7/9/2003 8 7/9/2003
Example 2 Example 2
f(t,y1)
z Measure the rain fall in a day @Songkla A sample function of rain fall
province every day. in y1 = year 1990 (1 ≤ t ≤ 365)
z Let F(t) = random process
f(t,y2)
A sample function of rain fall
z f(t,y) = a sample function for measuring in y2 = year 1991 (1 ≤ t ≤ 365)
at day “t” of the year “y”
…
f(t,yn)
A sample function of rain fall
in yn = year 2001 (1 ≤ t ≤ 365)
Sample functions
9 7/9/2003 10 7/9/2003
Example 2 Types of Stochastic Process

f(t,y1) W(t)
Therefore, we might Continuous Time,
want to know Continuous value Process
f(t,y2) X(n)
– The average rain Discrete Time,
fall in year 2001 Continuous value Process
– The average rain Y(t)
…
fall for Sep 3 Continuous Time,

f(t,yn) Discrete value Process
Z(n)
Discrete Time,
Discrete value Process
Sample functions
11 7/9/2003 12 7/9/2003
2
Stochastic Process Examples IID Random Sequence
W(t)
Record temperature as z Independent, Identically Distributed Random
a continuous time Sequence
X(n) z Independent trials of an experiment at a
Record round(temperature) constant rate
as a continuous time
Y(t) z Discrete / Continuous
Record temperature Theorem:
every T seconds k
Z(n) PXn … Xn (x1,…,xk) = PX(x1)…PX(xk) = ∏ PX(xi)
Record round(temperature) 1 k
i=1
every T seconds
13 7/9/2003 14 7/9/2003
Counting Process Counting Process

Definition: A Stochastic Process is a # of customers arrive at (0,t]
Counting Process N(t) if N(t)
4
z n(t,s) = 0 for t < 0
z n(t,s) = integer valued and non-decreasing 3
2
1
t
S1 S2 S3 S4
X1 X2 X3 X4
15 7/9/2003 16 7/9/2003
Counting Process Counting Process

N(t)
4 Binomial Process
For a small step ∆,
3
only one arrival (Xn = 1) ( mn) (λT/m)n(1- λT/m)m-n n = 0,1,2,..
2 PNm(n) =
1 Success Prob. of Xn= λ ∆ 0 Otherwise
t = λ T/m
S1 S2 S3 S4
m Æ ∞ or ∆ Æ 0
X1 X2 X3 X4 Poisson Process
Binomial PMF
T = m∆
(λT)n e- λT n = 0,1,2,..
m PN(t)(n) = n!
( ) n
(λT/m)n(1- λT/m)m-n n = 0,1,2,..
PNm(n) = 0 Otherwise
0 Otherwise
17 7/9/2003 18 7/9/2003
3
Poisson Process Poisson Process
z Poisson Process is N(t)
4
a Counting Process that the # of Arrival
during any interval is Poisson RV 3
nth arrival time
z An arrival during any instant is 2
independent of the past history of the 1
process Æ Memoryless t
S1 S2 S3 S4
z Xn is called Interarrival Time X1 X2 X3 X4 nth interarrival time
19 7/9/2003 20 7/9/2003
Poisson Process Poisson Process

Definition: z Process rate (λ) = E[N(t)] / t
A Counting Process N(t) is a Poisson Process z M = N(t1) – N(t0) = Poisson RV
N(t) if
[λ(t1-t0)]m e- λ (t1-t0) m = 0,1,2,..
z # of arrivals in (t0,t1], N(t1) – N(t0), is a Poisson PM(m) = m!
RV with expected value λ(t1-t0)
0 Otherwise
z # of arrivals in each interval are independent

random variable
21 7/9/2003 22 7/9/2003
Joint PMF Example

Theorem: Poisson Process N(t) of rate λ, A mobile station transmits data packet as
Joint PMF of N(t1),…,N(tk), t1 <…<tk Poisson process with rate 12
packets/sec
P N(t1),…,N(tk)(n1,…,nk)
– Find # of packets transmitted in the kth hour
α1n1 e- α1 α2(n2-n1) e- α…
2 α k(nk-nk-1) e- αk – Find Joint PMF of # of packets transmitted
0 ≤ n1 ≤… ≤ nk
= n1! (n2 – n1)! (nk – nk-1)! in the kth hour and zth hour
0 Otherwise
α i = λ(ti-ti-1)
23 7/9/2003 24 7/9/2003
4
Example Example
z Let Nk = # of packets transmitted in kth hour z Joint PMF of # of packets transmitted in the kth
hour and zth hour
z # packets in each hour is IID
αknk e- αk αznz e- αz nk = 0,1,…
PNk,Nz(nk,nz) = nk! n z! nz = 0,1,…
[12(3600-0)]n e -12 (3600-0) n = 0,1,2,..
PNi(n) = n! 0 Otherwise
0 Otherwise α(nk+nz) e- 2α nk = 0,1,…
[43200]n e -43200 n = 0,1,2,.. = nk! nz! nz = 0,1,…
= n! 0 Otherwise
0 Otherwise
α = αk = αZ = λ T = 12(3600-0)] = 43200
25 7/9/2003 26 7/9/2003
Interarrival Time Interarrival Time

Theorem: Theorem:
Poisson Process of rate λ, the interarrival A Counting Process with independent
times X1, X2, … are an iid random sequence exponential interarrival time X1, X2, … with
with Exponential PDF E[Xi] = 1/ λ is a Poisson Process of rate λ
λe -λx x≥0
fX(x) =
0 Otherwise
27 7/9/2003 28 7/9/2003
Brownian Motion Process Brownian Motion Process
z A continuous time, continuous value Gaussian RV –(x - µ)2

random process fX(x) =
1
e 2σ2
√2πσ2
Definition: A Brownian Motion Process X(t)
has the property that E[X(t)] = 0 and Var[X(t)] = ατ
z X(0) = 0 –x2
1
z For τ > 0 , X(t + τ) – X(t) = Gaussian RV fX(t)(x) =
√2πατ
e 2 ατ
with E[X(t)] = 0 and Var[X(t)] = ατ
that is independent of X(t’) for t’ ≤ t
29 7/9/2003 30 7/9/2003
5
Joint PDF Expected Value
z Theorem: For the Brownian motion X(t): X(t1) Æ f X(t 1 ) (x) Æ E[X(t1)]
process X(t), joint PDF of X(t1),…,X(tk)
Definition: The expected value of a
f X(t1),…,X(tk) (x1,…,xk) stochastic process X(t) is the
–(xn – xn-1)2 deterministic function
k
1 2 α(tn – tn-1)
= ∏ e µx(t) = E[X(t)]
n=1 √2πα(tn – tn-1)
31 7/9/2003 32 7/9/2003
Covariance of X and Y Autocovariance

Definition: Cov[X,Y] = E[(X-µx)(Y-µY)] z Auto = self = same process
z For a same process X(t), in 2 different times t1
Cov[X,Y] = E[XY - µxY - µYX + µx µy]
and t2 = t + τ
= E[XY] - µxE[Y] - µYE[X] + µx µy
= E[XY] - µx µy - µx µy + µx µy z For high covariance = a sample function, is
unlikely to change in τ interval
Theorem: Cov[X,Y] = E[XY] - µx µy z For near zero covariance = rapid change
• High covariance = an observation of X provides How much the sample function

an accurate indication of Y is likely to change in the τ interval after t
33 7/9/2003 34 7/9/2003
Autocovarience Correlation
Definition: Definition: The correlation of X and Y is rX,Y

The autocovariance function of a stochastic rX,Y = E[XY]
process X(t) is
CX(t,τ) = Cov[ X(t), X(t+τ) ] Theorem: Cov[X,Y] = rX,Y - µx µy
Note
• For τ = 0 Î CX(t,t) = Var[X(t)]
35 7/9/2003 36 7/9/2003
6
Autocorrelation Autocovariance & Autocorrelation
z Definition: The autocorrelation function Theorem: The autocovariance and

of a stochastic process X(t) is autocorrelation functions of a process X(t)
satisfy
RX(t,τ) = E [ X(t) X(t+τ) ] CX(t,τ) = RX(t,τ) – µx(t) µx(t+ τ)
Note:
Autocovariance Î use X(t) to predict a future X(t+τ)
Autocorrelation Î describe the power of a random signal
37 7/9/2003 38 7/9/2003
Autocovariance & Autocorrelation

Random Sequence of a Random Sequence
For a discrete time process, the sample function Definition: The autocovariance of a random
is described by the ordered sequence of
sequence Xn is
random variable Xn = X(nT)
CX[m,k] = Cov[Xm, Xm+k]
z Definition: A random sequence Xn is
an ordered sequence of random Definition: The autocorrelation of a random
variable X0,X1,… sequence Xn is
RX[m,k] = E[Xm Xm+k]
m and k are integers

39 7/9/2003 40 7/9/2003
Autocovariance & Autocorrelation

of a Random Sequence
X(t)
CX[m,k] = CX[4,1]
Stationary Process
= Cov[X4,X4+1]
T X4 X5
t
0 2 4 6 8
X0 = (0T)
X1 = (1T)
X2 = (2T)
41 7/9/2003 7/9/2003
7
Stationary Process Stationary Process
• For a random process X(t), normally,
Definition: A stochastic process X(t) is stationary
at t1: X(t1) has pdf = fX( t )(x) [depends on t1] iif for all sets of time t1,…,tm and any time
1
• For a random process X(t), different τ,
at t1: X(t1) has pdf = fX( t )(x) [not depend on t1] f X( t ),… ,X( tm ) (x1,…,xm) =
1 1
Stationary Process fX( t τ ),…, X( tm + τ ) (x1,…,xm)

1+
= same random variable at all time
= no statistical properties change with time
fX(t1 )(x) = fX(t1 + τ )(x) = fX(x)
43 7/9/2003 44 7/9/2003
Stationary Random Sequence Stationary Process
Definition: A random sequence Xn is stationary Theorem: A stationary process X(t),

iif for any finite sets of time instants n1,…,nm
and any time different k, µX(t) = µX
f X( n ),… ,X( nm) (x1,…,xm) = RX(t,τ) = RX(0,τ) = RX(τ)
1
CX(t,τ) = RX(τ) – µ2X = CX(τ)
fX( n +k ),…, X( n ) (x1,…,xm)
1 m+k
45 7/9/2003 46 7/9/2003
Stationary Random Sequence Example
Theorem: A stationary random sequence Xn, for z Telegraph Signal, X(t) take value ± 1
all m z X(0) = ± 1 with probability = 0.5
E[Xm] = µX
z Let X(t) toggles the polarity with each
RX[m.k] = RX[0,k] = RX[k]
CX[m,k] = RX[k] – µ2X = CX[k] occurrence of an event in a Poisson
process rate α
47 7/9/2003 48 7/9/2003
8
Example Example
z Find PMF of X(t), fX(t)(x)
z P[X(t)] = P[X(t) | X(0) = 1] P[X(0) = 1]
+1
+ P[X(t) | X(0) = –1 ] P[X(0) = –1]
• P[X(t) | X(0) = 1] = P[N(t) = even]
∞
(αt)2j
–1 =∑ e –αt
X1 X2 X3 X4 X5 j=0 (2j)!
= e–αt (1/2) (e–αt + e–αt )
= (1/2) (1 + e-2αt)
49 7/9/2003 50 7/9/2003
Example Example
• P[X(t) | X(0) = –1] = P[N(t) = odd] • P[X(t) = 1]
∞
(αt)2j+1 = P[X(t) | X(0) = 1] P[X(0) = 1]
=∑ e –αt
j=0 (2j+1)! + P[X(t) | X(0) = –1] P[X(0) = –1]
= (1/2) (1 + e-2αt)(1/2) + (1/2) (1 + e-2αt)(1/2)
= e–αt (1/2) (e–αt – e–αt )
= 1/2
= (1/2) (1 – e-2αt) • P[X(t) = -1]
= 1 – P[X(t) = 1] = 1/2
½ X(t) = –1 , 1
fX(t)(x) =
0 Otherwise
51 7/9/2003 52 7/9/2003
Example Wide Sense Stationary

z µX(t) = 1 (1/2) + (– 1)(1/2) = 0 Definition: X(t) is a wide sense stationary random
process iff for all t,
z Var [X(t)] = E[X2(t)] = 12(1/2) + (-1)2(1/2) =1 E[X(t)] = µX
RX(t,τ) = RX(0,τ) = RX(τ)
Autocovariance, CX(t,τ) = e -2ατ
Definition: Xn is a wide sense stationary random
f X( t ),… ,X( tm ) (x1,…,xm) =
1 sequence iff for all n,
fX( t ),…, X( tm + τ ) (x1,…,xm) E[Xn] = µX
1+ τ
RX[n,k] = RX[0,k] = RX[k]
53 7/9/2003 54 7/9/2003
9
Wide Sense Stationary Example
z For every stationary process or sequence, it z Let Xn = ± 1 with prob = ½ (n = even)
is also wide sense stationary. z For n = odd
Xn = -1/3 with prob = 9/10
z However, if it is a wide sense stationary it
Xn = 3 with prob = 1/10
may or may not be stationary.
z Stationary ?
– No
z Wide sense stationary ?
– Mean = 0 for all n
– CX(t,τ) = 0 for τ > 0
– CX(t,τ) = 1 for τ = 0
– Yes , it’s wide sense stationary
55 7/9/2003 56 7/9/2003
Wide Sense Stationary Wide Sense Stationary

Theorem: For a wide sense stationary process Theorem: For a wide sense stationary sequence
X(t), Xn,
RX(0) ≥ 0 RX[0] ≥ 0
RX(τ) = RX(– τ) RX[k] = RX[– k]
|RX(τ)| ≤ RX(0) |RX[k]| ≤ RX[0]
57 7/9/2003 58 7/9/2003
10
ECE 244/444
DIGITAL COMMUNICATIONS
Alireza Seyedi
Fall 2008
2 I
Introduction
d i
S ti 1
Section 1.1
1
U of R, ECE244/444, Alireza Seyedi, Fall '08

Digital
g Communications
3
General p problem of communications: We are asked to

transport the a source signal (analog or digital) from one point
to another (or many others).
I di
In digital
i l communications,
i i we digitize
di i i (sample
( l and d quantize)
i )
the source and we transport the digital data (bits).
Some advantages of digital communications compared to
analog communications are:
We can use digital signal processing
We can use storage
We can regenerate digital data

Elements of Digital
g Communications
4
Elements of digital communications system:

Source: Signal that we would like to transport (e.g. Music/Text
M / file)
f
Source encoder: Represents source signal with (minimal amount of)
digital data

Elements of Digital
g Communications
5
Channel encoder: Adds redundancy to the data to protect again

errors caused by channel
Digital Modulator: Generates a waveform to be transmitted over the
channel from the digital
g data
Channel: Represents the medium through which communication takes
place
Di it l Demodulator:
Digital D d l t Recovers
R digital
di it l d
data
t from
f the
th received
i d
waveform
Channel Decoder: Recovers the information from the received bits
Source Decoder: reconstructs the source from the received
information

6 R d Processes
Random P (A quick Review)
S ti 2
Section 2.7,2.7-1
727 1

Random processes
p
7
A stochastic or random pprocess (RP)

( ) is a set of random
variables (RV) with a time index:
Discrete-time random process: X[n], where each X[n=n0] is a random
variable.
variable
Continuous-time random process: X(t), where each X(t=t0) is a random
variable.
In general, each random variable X(t) may have its own pdf,
fX(t)(x).
As a result,, all statistics of X(t)
( ) will be a function of t, ((i.e.
μX(t), σX2(t)).

Random processes
p
8
Example:
p X1 where X1((t)) = Y,, where Y is an RV with uniform
pdf over [0,1]
Example: X2 where X2(t) = Cos(ωt), where ω an RV with
uniform
f pdf [0 2π]
df over [0,2
Example: X3 where X3(t) are independent and have pdf
N(t,1)
Example: X4 where X4(t) are iid with pdf N(0,1)
Example:
p X5 where X5((t)=X
) 4((t)+X
) 4((t-1))

Joint density
y and autocorrelation
9
Since each X(t)

( ) is an RV,, g
given N times t1,,…,t
, N, we can write
their joint pdf
f X ( t1 ),...,
) X ( t N ) ( x1 ,..., x N )
Using this joint pdf, we can define statistical averages
E[g ( X (t1 ),..., X (t N ))] = ∫ g ( X (t1 ),..., X (t N )) f X (t1 ),...,X (t N ) ( x1 ,..., xN )dx1...dxN
Th autocorrelation
The t l ti function
f ti off an RP X(t) is
i ddefined
fi d as
[
R X (t1 , t 2 ) = E X (t1 ) X * (t 2 ) ]

Autocorrelation
10
Example:
p Autocorrelation of X2
Example: Autocorrelation of X4

Autocorrelation
11
Example:
p Autocorrelation of X5

Stationaryy random processes

p
12
A stationaryy p
process X(t)
( ) is one that
f X ( t1 ),..., X ( t N ) ( x1 ,..., x N ) = f X ( t1 +τ ),..., X ( t N +τ ) ( x1 ,..., x N )

A Wide-Sense Stationary (WSS) process is one that:
1. μX(t) = μX (t+τ)
2. RX(t1,t2) = RX(t1+τ,t2+τ) = RX(t1-t2,0)=RX(t1-t2) =RX(t)

Stationaryy random processes

p
13
Example:
p
Is X2 WSS? Why?
A X4 and
Are d X5 WSS? Why?
Wh ?

Filtering
g an RP
15
Assume we filter an RP,, X(t),

( ), with a linear time-invariant filter,,
with impulse response h(t) and frequency response H(f), and
we get Y(t). Then:
1. μY(t)=
(t) μX(t)H(0)
2. SY (t) = SX (t)|H(f)|2
Example:
p
Find the impulse response of a filter h(t) that produces X5 from X4
Find the frequency response H(f)

Power spectral
p densityy (PSD)
( )
14
Power Spectral
p Densityy (PSD)
( ) of a WSS RP is the Fourier
transform of the RX(t), that is SX(f)= F {RX(t)}
PSD tells us “How much power the RP has at each frequency”.
Example: PSD of X4
Note: An RP with flat PSD is called a “white” RP. This is due to the
fact that white light has “all the frequencies” in the visible light
range.
g
Example: PSD of X5

Filtering
g an RP
16
Find PSD of X5 from PSD of X4

A property of MVG_OMALLOOR 國立台灣海洋大學
國立台灣海洋大學 National Taiwan Ocean University
National Taiwan Ocean University Announcement

Course webpage:
通訊與導航工程學系碩士班 http://dcstl.cge.ntou.edu.tw/DCSTL/Web/dc2008.htm
數位通訊 Textbook webpage:
http://www.mhhe.com/engcs/electrical/proakis/
Digital Communications
Reading Assignment:
Fall 2008 Chapter 1, 2, and 3
吳家琪助理教授 Homework No.1: 2.3, 2.5, 2.11, 2.14, 2.20
Due Sept. 29, 2008
Lecture 2: Probability and Random Processes
國立台灣海洋大學國立台灣海洋大學
National Taiwan Ocean University National Taiwan Ocean University
A Communication System with Gaussian Noise The Q-function

S ∈{±a} The Q-function is a standard form for expressing error
R = S+N
< probalities without a closed form:
Transmitter + Receiver R 0
> ∞ 1 −u 2 / 2 1 ⎛ x ⎞
Q( x) = ∫ e du = erfc⎜ ⎟
N ~ N(0,σn2)
x
2π 2 ⎝ 2⎠
Numerical Calculation of Q-function:
The probability that the receiver will make an error is: 1 ⎡ 1 1⋅ 3 (−1)n ⋅1⋅ 3 ⋅L(2n −1) ⎤
Q( x) = e− x / 2 ⎢1 − 2 + 4 + L+
2
( x+a ) 2 ⎥
∞ 1 − ⎛ a ⎞ x 2π ⎣ x x x 2n ⎦
P( R > 0 | S = −a) = ∫ e 2σ n2
= Q⎜⎜ ⎟⎟
0
2πσ n2 ⎝σn ⎠ ≈
1
e− x / 2 , for x ≥ 3
2
x 2π
A property 國立台灣海洋大學
of MVG_OMALLOOR 國立台灣海洋大學
The Q-function and its Approximation Example #3 - Rayleigh pdf

Q-function and Analytical Approximation
102 Let R = X 12 + X 22
-
- -
100 - - where X1 and X2 are Gaussian with mean 0 and variance σ2
- - Then R is a Rayleigh random variable with pdf:
10-2 - -
r / 2σ 2
PR (r ) = e −r
2
- -
σ2
Q(x)
10-4 - -
- -
10-6 - - Rayleigh pdf’s are frequently used to model fading when
- - no line of site signal is present
10-8 - -
- -
10-10
-
-
0 1 2 3 4 5 6
Rayleigh pdf Probability Mass Functions (pmf)

0.7
A discrete random variable can be described a pdf if we
-
0.6 - - allow impulse functions

0.5 - - We usually use probability mass functions (pmf):
0.4 - -
p(x)= P(X=x)
p(r)
Properties are analogous to pdf:

0.3 - -
p(x) ≥ 0
0.2 - -
∑X p(x) = 1
0.1 - - P(a ≤ x≤ b) = ∑a p(x)
b
0
-
0 1 2 3 4 5
r
Example #1: Binary Distribution Example #2: Binomial Distribution

⎧1 / 2, x = 0 Let Y = ∑i=1 Xi where {Xi , i = 1, 2, …, n} are
n

p( x) = ⎨
⎩1 / 2, x = 1 independent binary RVs with:

⎧1 − p, x = 0
This is frequently used to model binary data p X ( x) = ⎨
Mean: mx =∑X x•p(x) = 0×(1/2) +1×(1/2) = 1/2 ⎩ p, x =1
Variance: Then ⎛n ⎞ ⎛n ⎞ n!
pY ( y ) = ⎜⎜ ⎟⎟ p y (1 − p ) n − y , ⎜⎜ ⎟⎟ =

σx2 =∑X (x- mx)2p(x)=(1/2)2×(1/2)+(1/2)2×(1/2) = 1/4 ⎝ y⎠ ⎝ y ⎠ y!(n − y )!

If X1 and X2 are independent binary random variables, Mean: mx = n • p
then
Variance: σx2 = n • p • (1-p)
pX1X2(0,0) = pX1(0) • pX2(0) = (1/2)×(1/2) = 1/4
Example #2 (continued) Central Limit Theorem

Suppose that we transmit a 31 bit sequence with error Let X1, X2,…,XN be a set of independent random
correction capable of correcting up to 3 errors. variables with identical pdfs
If the probability of a bit error is p=0.001, what is the Let: Y = ∑i=1 Xi
N
probability that the codeword is received in error?
Then as N→∞ , the distribution of will tend towards a
P(codeword error) = 1- P(correct codeword)
Gaussian distribution
3
⎛ 31⎞
= 1 − ∑ ⎜⎜ ⎟⎟(0.999) 31−i (0.001) i ≈ 3 × 10 −8 In practice, N=10 is usually enough to see this effect
i =0 ⎝ i ⎠
Thermal noise results from the random movement of
If no error correction is used, the error probability is: many electrons - it is well modeled by a Gaussian
1-(1-0.001)31 = 0.0305 distribution
Example of Central Limit Theorem: Definition of a Source

A source is a box that produces ”events”, or symbols.
Symbol A
Source
Each value of A comes from a set of possible values,

A∈{a1, a2, a3, …, aN}
Each value has an assigned probability, {p1, p2, …, pN}
Binary source:
An∈{0, 1} p0 = Pr(An = 0), p1 = Pr(An = 1) = 1 – p0
Random Processes Random Processes

A random variable has a single value. However, actual In Information Theory, we model an information source
probabilistically, namely, as a probability measure on a set
signals change with time.
of outcomes. For example:
Random variables model unknown events. A book (Harry Potter or Star War) is an outcome from
Random processes model unknown signals. the sample space of all possible books.
A random process is just a collection of random An motion video is a point in a high-dimensional
variables. sample space.
If X(t) is a random process then X(1), X(1.5),and X(37.5) Engineers like to break large problems into smaller pieces.
are all random variables for any specific time t book Æ words Æ letters
movie Æ frames Æ 8 x 8 blocks
In this way, the overall outcome is described a collection
of random variables Xi: i ∈ I
Random Processes Random Processes

Definition: A random process is an indexed set of random Kolmogorov showed in Foundations of Probability
variables with a joint pdf. that if the pdf of any finite number of random
Examples of commonly used index sets: variables from the random process is defined and
N = {1, 2, …}: in this case, {X0, X1, X2 ,… } has an initial consistent, then the random process is well defined.
value, which is randomly chosen according to pdf p(x0). We will not need this big result for the time series
Z = integers: infinite past, when we do not want to specify that we study, since for index set I = N the random
a starting time. process is obviously defined by the pdfs
R or R+: continuous time (more difficult to define and
discuss).
p(x0, x1,…, xn) n = 1, 2, 3, …
176 x 144 x n: low bit rate video conferencing. The Many real world sources are (or at least we hope
dimensions are x, y, and time. they are) time invariant.
Stationary Terminology Describing Random Processes

A random process is stationary if its statistics do not A stationary random process has statistical properties
change with time: which do not change at all with time (i.e., all joint pdfs
do not change)
Pr(Xi1=x1, ……, Xin=xn)= Pr(Xj+i1=x1,……, Xj+in=xn) A wide sense stationary (WSS) process has a mean and
j = 1, 2, 3, … autocorrelation function which do not change with time
(this is usually sufficient)
A random process is ergodic if the time average always
converges to the statistical average.
Unless specified, we will assume that all random
processes are WSS and ergodic.
Description of Random Processes Autocorrelation

Knowing the pdf of individual samples of the random Autocorrelation measures how a random process
process is not sufficient. We also need to know how changes with time.
how individual samples are related to each other. Intuitively, X(1) and X(1.1) will be more strongly
Two tools are available to describe this relationship: related than X(1) and X(100000) (although it is possible
Autocorrelation function to construct counterexamples). The autocorrelation
Power spectral density function function quantifies this.
Definition (for WSS random processes):
φX(τ)=E[X(t)X(t+τ)]
Note that Power = φX(0)
Power Spectral Density Gaussian Random Processes

Φ(f) tells us how much power is at each frequency Gaussian Random Processes have several special
Wiener-Khinchine Theorem: Φ(f) = F[φ(τ)] properties:
Power spectral density and autocorrelation are a Fourier If a Gaussian random process is wide-sense stationary,
Transform pair! then it is also stationary.
Any sample point from a Gaussian random process is a
Properties of Power Spectral Density:
Gaussian random variable
Φ(f) ≥ 0 If the input to a linear system is a Gaussian random
Φ(f) = Φ(-f) process, then the output is also a Gaussian random
∞
Power = ∫−∞ Φ(f)df process
Linear Systems Computing the Output of Linear Systems

Input: x(t) Deterministic Signals:
Impulse Response: h(t) Time Domain: y(t) = h(t) * x(t)
Output: y(t) Frequency Domain: Y(f) = F{y(t)} = X(f)H(f)
For a random process, we can still relate the statistical
x(t) h(t) y(t) properties of the input and output signal
Time Domain: φY(τ) = φX(τ) * h(τ) * h(-τ)
Frequency Domain: ΦY(f) = ΦX(f)• |H(f)|2
Channels Digital Communication System

Physical channels: the atmosphere, wirelines, optical
fibers, computer hard disks, compact disks, . . .
Channel models: We need good models of the random Information Input Source Channel Digital
Source Transducer Encoder Encoder Modulator
phenomena introduced by the physical channel!
Examples:
A discrete channel A linear additive noise channel Channel
Linear r(t) Output Output Source Channel Digital

s(t) +
X Y filter c(t) Signal Transducer Decoder Decoder Demodulator
n(t) Information and Coding Theory Modulation and Detection Theory

channel
Some Lectures on … Some Lectures on …

Information and Coding Theory Modulation and Detection
Information sources and source coding Modulation: Transform digital data into analog signals
¾ Introduction to Information theory: Entropy, information that can be transmitted or stored (the “real world” is
measures, limits, … analog, not digital). . .
¾ Source coding: Use fewer bits to efficiently represent Demodulation/Detection: The received signal contains
source data in digital form … information about the transmitted data but is corrupted by
Channel capacity and coding noise (etc.).
¾ More on Information Theory: Channel capacity, limits,. . . ¾ Estimate what data was sent, aiming at minimum possible
¾ Channel coding: Use some clever redundant bits coding to probability of making mistakes. . .
detect and correct transmission errors. . .
Why digital communications? Information Sources and Source Coding

Some sources are inherently digital (e.g. text). . . Sources, information and entropy
Better “control” and more flexibility. . . Typical sequences and the source coding theorem
Bits can be detected and regenerated. Noise does not Mutual information and differential entropy
propagate additively. Lossless source coding
Easier to remove redundancy and reduce bandwidth. . . Definitions...
Digital ICs are inexpensive to manufacture. A single The source coding theorem. . .
chip can be mass produced at low cost, no mater how Huffman codes
complex Lempel-Ziv codes
Digital communications allows integration of voice,
video, and data on a single system (ISDN)
……
Randomness in Introductory
Computing
• Understanding randomness requires
mathematical precision
– How to, and how not to, generate random
Richard Anderson permutations
Random thoughts on • Randomness can be used generate realistically

appearing outputs
Randomness for SIGCSE 2005
– Many practical applications
– Very engaging assignments
– Example: generating random text
The cost of not getting things right Randomness is Subtle

• Intel Division Bug • Reasoning about random processes is
• Ariane 5 Rocket often incorrect
• Windows security holes • Formal arguments often needed
• Central problem for Computer Science • Example problem: Generate a random

Education: Give students tools and sense permutation: given an array A of length n,
to “get things right” permute its elements so each permutation
has probability exactly 1/n!
Text book code Incorrectness Proof

Random generator = new Random();
• Consider an array of size 3
for (int i = 0; i < 52; i++){
int j = Math.abs(generator.nextInt()) % 52; • Random sequence generated (x1,x2,x3)
// swap the two card values • Number of possible sequences 33=27
Object temp = thePile.elementAt(i);
• Number of permutations 6
thePile.setElementAt(thePile.elementAt(j), i);
thePile.SetElementAt(temp, j);
} 123 132 213 231 312 321
4/27 5/27 5/27 5/27 4/27 4/27
1
Java Library Code Use of Randomness for Generation

public static void shuffle(List list, Random rnd){ • Many applications for generating realistic
int size = list.size(); outputs
for (int i = size; i > 1; i--) – Animation
swap(list, i-1, rnd.nextInt(i)); – Games
} – Test programs
– Illustrations
• Nifty homework assignment – generate a

random X
Random Text Questions

• Generate random “words” for a language – • What is a random monster name?
words that look real
• Generate random “text” for a language
• What methods were used to generate the
• Class exercise (Index cards provided) name?
– Generate a random name for a new class of
monster
Approaches to word generation Generating random text

• Template based A Stance. Sured for at stic. Stanger flice, to a clearts, will was he
experpen spoke almostly. A might, the room, an every live was
• Production rules heaving ins, busybody of the sing hand morth can quire, time and
abore that, answered to my dring given you and end, drived his
• Generate and test away they were could of there view detainelses had into for
surmistling. To the which fount every ver. Drebber have roughter
• Finite state machine soon be swell. It with he must mout the circume longer, you with
that that hard, the was thoughten came is vorations no comes.
• Random selection from list Fourwhen curied manionsible rooms to fine. Not be diss. You
articall neithe minut upon over way. I ejacult Lation the the was
with you secomes it ever hung room, and step inties irong. The
lone old far peoplet of to the his so who hat I to had ling hearnes. I
led. Ther Mr. No publity abr
2
Random German Random Latin

Phnomentgegentworingt, ung kannteils das Ersche sage, die
Publatoniunguias, animum fiduo proculos mem dolii stu, quis.
Gebornehmeckenlose Werder sche alleichkeits eins hat sich gen
Turnia culiqui priminis omnipatris et laceae fectus saxa donatis.
Willendem Dasein absolches Bestig an in ist ein die
Adium ad arre silva res circum eductum domo armos infando sus
Umfangenwrtier Gegen, unen darin die ist nur als bleicheit, das
harentem Itale Corris impo, Titatit acis habeat ungentem per
Anterganissen Einzelnderlicht von und das Leeres Ich in Geser
Achisaequi que rebant, lausa Zacyntis inclami iuxtaque ac tibus,
von ist die Beziehung, enterschiede, darin des teils absolute Wurz
innostro secabat amor urbesaucialido currus efficis atque
erhltnischie, das Gedankonstndividualischwinn erzeugung des
omnemunum currundam, quid tem spes gens in temque gens, quo
allgemeine Subststndividualitat dernung deres Gemerzeugung ind
domus et per ad licet aud notusque pugnis constisque rum,
and so fllt die Norden es hatte Forteigente dennbar als aus sein
quantque fidum Sistigat, homina subi ausi clas tem nectibi aut
ist, in Selbare Damige Weser Natur das rastandenn es liegen er
cerdor haud tantibus pugnavissi nivertunam manta fortantque et,
Erschafte des in philosenheit dem Unglitte, dem eben hattfinden.
quoque Abantiamul longa oppitor extraemi cornus, quod te
Dies Gegens offen Ansibilitt istigen die Muttelbe bestigen Prfung
regemitam exsti mille tibus sub ardanimillo indus atque hae, loca
dem, wesen erkehren Gegen Untens nimmensosehr als fas
haudibus ad tutatus sivello dignem. Mihi formiseserum crum
Random Esperanto Non-random Esperanto

La viro, kies sxipo subakvigxis, diris kun grava mienoJunulo, ne
La unu mortiri insulo tio naj portigxulongxi nun manojn. Malsen
iru plu surmaron tiu ne estas la vivmaniero por vi. Kial do, sinjoro,
sklavo, ke laux de li malgxo metigxis. Sed miajnojn tremo kaj
vi mem iros plu surmaron Tiu estas alia afero. Mi estas elnutrita
amikoj. Jen zorgauxebla talegis li al Zuro, ke Jafojektre ili aux kaux
por la maro, sed vi ne estas vi venis sur mian sxipon por eltrovi, la
domoj, panarolitan atingis iom da estu juna ankonto plu reveniajn
staton de vivo surmara, kaj vi povas diveni tion, kio okazos al vi,
ventis tute gxis, ke iris sia fara, ke ili birdon timando. Majo.
se vi ne reiros hejmon. Dio ne benos vin, kaj eble vi kauxzis
Vilhelmontegmenas la terojn por mecon povus timili aux auxdis
tiuncxi tutan malbonon al ni. Mi ne parolis alian vorton al li, kiun
prenligi tiujncxi murmultereokazo den. La kam ne ekpreniradis, Cxu
vojon li iris, mi nek scias, nek deziris sciigxi, cxar mi estis ofendita
vi sed varmis, ke de la longtempo esti iris havu sklarmilo ne por
pro tiucxi malgxentila parolado. Mi multe pensis cxu iri hejmon,
spacete, ni al Dio, kie da prenoj bruo, lokonte nigrandonaton, ellas
aux cxu iradi surmaron. Honto detenis min pri iri hejmon, kaj mi ne
nun, kaj mia grenis, la certe tio devas ke la sovas mi krii, tiam da
povis decidi la vivkuron, kiun mi estis ironta. Kiel estis mia sorto
sur la voki li ol tiel la Skoto. Dio kaj mortigxojn la trajdis. Fine, legaj
travive, cxiam elekti la plej malbonon tiel same mi nun faris. Mi
sed nin aparte la jam Vendredos fine, estis la sed ni diris, Je
Generating Random Text Letter distribution

• Letter distribution should match language A of rs deveastorere ghinswalmporthokioutt mmede hug thed I fte
g blagrs h thtr. htho t is nd ppatiestath. o I sis te t. s hoomooure t
• Word length, punctuation should match pryoucly, polat hir h I fuetamede s hed a on h s d t s hy imaillabe
c o ted tathe hedenirngh wintht Le f inced a ly I t pe byose ollfrnd
language inofaps ron a I h is Leantond as, t oasir meromisebe omider
ceshed hiaidinte waritingithe t t hofon Papoug bthe as I o clld
stoapithraureninto waredrkerd t Cow, oreed todviron td connce,
• Simple attempt – random strings with avily d. thicos, I wis barem J. n d he tha be rerid m asethaler
onorrid icik tolk imene, heewny wiccere ondy. ongut lwaghes t wo
probability derived from language t w he vemied hay I sst d t a this. cis, t catimby d gusithai he w tlef
rewile thed w s mng fagrio thued apo d the menck ld iong s h
hous whe id I fenowe t Yotig obad Whep iody, d abry. cound rad
ily thicthes the
3
Bigram models Doyle (2)

A muld pince whispe. Froser reed. Docklid. Salmeted an and to
• Generate strings with correct conditional trattly the he ch roccolve tomme, look hick Hoposper swerected,
distribution P(a|b) wily. He alfs forectit wer of dis weringeaver thison is ruche to broo,
sat havenithat enee lodereen, alcurectictles the be byean wes
youghathis the overying be yettelsoter his upoly he had frile und
wit torcile to he has saw of son mong. Loonlecing, hichabox if
• Record distribution on ordered pairs of therebbegras antive had ison. Yould ing got theres ention ity ling.
Wat led im, hor of cally bect the inswelin lith aressidaugh my oung.
characters I his didead to the facriverebad of the pon. Yeles, I cre wrind frathe
doothe sed cong rat Dr. Prot inglace was brown thathere seve,
sainvidealle obectlestratempargaidich It th and it theart acry
createep teureas nousee layin ass wo he the he the gue my he on
beer deartied a
Generalize Doyle (3)

• K-Gram probability A Stance. Sured for at stic. Stanger flice, to a clearts, will was he
experpen spoke almostly. A might, the room, an every live was
• Construct state machine to generate text heaving ins, busybody of the sing hand morth can quire, time and
abore that, answered to my dring given you and end, drived his
• P(xk | x1x2…xk-1) away they were could of there view detainelses had into for
• Get probabilities from a corpus of text surmistling. To the which fount every ver. Drebber have roughter
soon be swell. It with he must mout the circume longer, you with
• www.gutenberg.org that that hard, the was thoughten came is vorations no comes.
– Free ebook download Fourwhen curied manionsible rooms to fine. Not be diss. You
articall neithe minut upon over way. I ejacult Lation the the was
– 13,000 books with you secomes it ever hung room, and step inties irong. The
• Use hash tablet – not a k dimensional array lone old far peoplet of to the his so who hat I to had ling hearnes. I
led. Ther Mr. No publity abr
Doyle (4) Doyle (5)

A Study improve of livid farms. Lestrainworking the been such ran A Study of stronomy. Nil. Philosophy. Nil. Philipped eyes swelled for
the door was new life, asked, gave beat also accounciling from the I saw him who to them on the solemn word Rache written in his
other killed foundless often he him probably founder to that it bedrooms of the door, who had sat stolidly and them which now
states inted see hed to doubt of Montpelled dog among angular and from the thought. The two ran out by the magazine downstairs
fling another may affortunes. That it to his no name count not a and sobbed him when I enumerous aspectfully, the mountaining the
yards bed its ghastly. I gathere, he follow where rapidly a break, passin. It seemed to the had growing habit they have mischievous
he protecting or face highest grey down there are with was scrapbook I picketed. It depends of analytical laborator having over
suspectfully high, be suburbs, and Ill thands in his bait you what husband, but I guess you actually terrors had render he had so
that politary. The two the you, I got society. Its in young girl dry exact caravan, pass what there taken will be oppose that there is no
wered it communicate wild, cab, and bright the lost unravely, newspaper have done. Next moment, and wrung man who
fright we weak from the every fury. The preciating intentired fact accustomed as I have missed his heavily tire me. You shoulder, he
show much the on plast of the stranting toward of the mothere no apparently over that them as took made every scent. Then India at
be rain Indian said, the easy matter the paper
4
Doyle (7) Doyle (10)

A Study In Scarlet. Part I. Being a reprint from the Ravine where to
A Study In Scarlet. Part I. Being a popular hymn. Both of his fate, look and what hed say if he come ome and found that he should
are quite exciting, exclaimed what it with an inarticular crony of my fly with his ridingwhip from the tropics, for his temper is violent,
natural tint of asking him occasion to prop him up. Ill tell you what and he had frequently before harm comes on her I guess there will
that face had produced such employer. I did not act, however, but not now abandon His own chosen people, and that the criminal
still lying upon the sofa in their last spring up at the Solar System I annals. The ring, man, the ring of his name and address Lestrade
laughed any course, Doctor Ill lay you two to one against the sight are the pick of a bad lot. They are both from the smouldering fire,
of it had a hurried onwards frantically, dropping abruptly and while the two castaways. If we take you with us, he said. Gregson
mother Mothers beady eyes glistened face, and he heard in the and Lestrade succeeded, and how quickly I took advantage of
breast of his name is My name is a wellknown Scotland Yard some early love affair, and of a fairhaired detectives smiled
officiallooking allusion that Mr. Drebber has been said my derisively, by no means a new thing in criminal was probably be
company, I answered. It seldom is in a way when the undeniable here in a few seconds enveloped into the hall with the avowed
fancy kind of your late boards, with a motive, and her daughter is intention had become so ac
very clever,
Marx (3) Dostoevsky

Manists weal as mean mustruggles social diffect. In idical life are Crimself, losomething again fool. He face axe a duty ready.
Will themself und accommunicalcults of in have, in the Berlothen supposit in gladov, when my home of Marfull healittle to
wordepidly, the mon it has now class colled, into begin. It polisie that the suddenly becall we he breasuall the werental depeak, I
had yourgeoisie intry abourgeoisie nation to the writs of Willus she backing, even, it is in to comptible conce, lood of Damn meet
the ecommunistitical and to a workings of la Mass a patriculcated. hous that in and ally of last ind repeachin, of tely convict laugh a
They way by that fore asses like ulture into with it bases. Wages, gethis did Raskolnikov, does on up andrage it suddenturned why
builds of been, they class sting the acquire is that the Owing hers broundrest go in anovitch, had his to ult at I persburious
cases. The icy was classolete aims, revolutionsistical sect has by dying inquity is void hangerstate. Justly I hand ally what and her
the at past alway, as, as to as be that more, the dist had nation to lay chool. And to be built a sing can shed set, cottonia Ive alm.
only, have nobilise pare from unish seen, attain away, and perty What a musing with the said Raskolka Alexating to him the be
sectuationserywhen part time lar, thanges of the sharactionary secream and scall bear the me, utterrow, not rent as flectreethin
dest. Our out or a conomic strientagonisms stend Commonal expectable ally the me into befor you forth he day litiveta,
bourgeois priety, and develo meallushe thats time. O
King James Bible (3) King James (7)

The Old Testament in under in the wilderness forty men of God is
The that the city, shall places, and shall the king trees writtle. I a found no place toward mouth. Three times his strength is like
sons and ourted upon of thou shally brought shorn by are fath ther deep to fall before the sabbath through sanctified them. This is the
beloversons in havent, the bond Hesh yea, the with their hold, heart lion had a name of them so they return upon whither the Jew and I
of they sit store all all no man she eare Bles one livere. For a grave in my hearts, that I may planted them all the daughters of one
their with him them and be of Jehold, the to hild. Brink of that the man, or wherein and thy servant among them. If thou brought him
prish unto may for their saying come. The Heshbib kicked if ther the They that day that greatest among women ravished and the sons
been they wearth. The shall bettether wall drinkhoring of Baal dwell of Israel. And after a while he yet he knoweth no respecteth him
the Lord, thithee, and of the said decause. And the shallent of things go And Moses. But if he thither unto me, and pursued after
his the word, ye fined unto Gent of trimmel an and with, will he right. thereof. In that there no gods. How long will yet planted in me.
The friesting saying the praim, and befor mout shorselve house, Then said he unto the land. So they shall fall by them for comfort
said, and the that down as in their him be sabbah was by wentere a thine hand is sprung up and fled for the second month Sebat, in
grave press not appoine of the cup. This order of thy whole months to
5
Mystery 1 Mystery 2
Jims faced around that was all. I was. He cold camp in the first Hwt he fescean hine, Hrot ic, monigongsele brothgars gestrles his
time whether time and settle druther saw his headingscow, or two r dyde, hres siomia num bite le inna nga e ofer mg inga t nall
plantation, Aint you, too bluff bank every well follow it little place. onned, searon monda Getan. Of The and lestre gende berse cw h
All right between up the hugged me stands after him. They was hine hwere, eorcum and on ban Hrla dre urh freodon gladufond
blooded these two thing one, and pick up to his plan, I wasnt have geheall runces fde nfrendel lenga, sylf wisthle onne gehws geatolc
confidence of itand went aloft In the said not to it was to steal that one orfan can mdes forbearn sprc, eard gd hinder. Hle hie riht
come yet aint, Mars Tom, and the raft did youd killed. Youd see geflmes, and fyreferas nicel hlw u eaf, on nefng giddan, fre, his
them betwixt them says these grave, and so he tribute before she faran scealdan, t his lefulofter rst wereard ges weallenrce, t ymb
was for from Ohioa mulatter the doors shoot But Well, Mars Sid, A wealsgen, grdig, gum, git eorhten, holde scold. Sige byrge ingum
dog. Why, what was making come afternoon, and carriages again fen hrurond Hroterda hogoe t gewrc, him dyrne t den u in beora
bundle of raft and, but coffin, and red flapping along begun in fren grtten he Joy Attas, rce gewealh is ford forlscaa ror, Hrmanna
slopped a few minute and jumped it out a whacks till cyst rnste, Gren eaxe genele fingene, byrhten Heaoscip oferhe
Erra. Hoard bodon he hsa ge
Mystery 3 Mystery 4
The day, Into hele swete hem here As she hires thus it wist, I Morning all years dismember. Their reclined or dear what is or
lengthe of myn hertes, I woful Troilus, And what is here. Thus good trousers perament. Hooray Seven the who it acrostitute Blushes
entencioun, And lest, Ayeins than hete Is that fresh entente. What good might threw the bore that would youd my fear him I will not.
echone, doun of trouthe affyle, She hard that I spek your wordes of Haines lie wide of osier. Millicence of the Peering fools in the didnt
thou art amis helm to grone. Rys, taken me I smiten worst I nede, In before his time. Saluting. Twopence. He was widower, wait. It was
alle Gan pulle, or sharpe Than can biholde I love, quod Pandarus not in your loud By Its Cordial refresh from began Mr Nannett cup.
another cominge, as is to my laste. Among the lettre that I seye, This was on two dark sea. Where weak eyes, made them on. They
Was al my parlement. And sith hede three, That in the goost draggin. Would give picture sad in the must simply, reclined out
departe first hir owne And, and woundemen of the day thought with and a smeared Of The jady could only to be original mirth,
devyse every lawe I here folk of Troilus, no preye Acursed be, That both waterment anything in a weeks, render, well What reproduced
felawe, That this swere to levest lere. But of Pandarus on the love a there hanker coach. Stars face, says Bob Doran, which it hes men
wey ye, that ther though the stalke, And if that swich routhe, and he any morning and ribbons. Her Leg Of The gr
gan t
6
Random
Processes
1/24
Random Process
Other Names : Random Signal
Stochastic Process
A Random Process is an extension of the concept

of a Random variable (RV)
Simplest View : A Random Process is a RV that

is a Function of time
Could be Discrete-time or Continuous-time

2/24
Examples of Discrete-Time
Random Processes
1. x[k] = value on the kth roll of two dice

2. x[k] = lottery payout on day k
3. x[k] = temperature at noon on day k
4. x[k] = sequence of 1’s and 0’s in a
binary file (also a coin flip)
3/24
Examples of Continuous-Time
Random Processes
1. X(t) = temperature at time t

2. X(t) = speed of an Aircraft at time t
3. X(t) = thermal noise voltage on resistor
at time t
4/24
Two Views of a Random

Process
1) Viewing RP as a sequence of RV’s
At each instant of time view x(t) as a RV:
DT Example #1: at each k you roll 2 dice

and get a random value
CT Example #3: at each time t you

measure the thermal noise voltage to
get a value
5/24

Process
2) Alternate view: RP is a collection of functions of
Time…the one you get is determined by some
single “Probabilistic Event”
Ex: Random Bits – Here are just 2 possible functions for this process:
x[k]
… …
k
x[k]
… …
k
6/24

Process
Each of the possible waveforms is called a
“Realization” or “Sample Function” of the RP.
The collection of all possible realizations of a RP

is called the “Ensemble” of the process
Of the two views, #1 is the most useful

i.e. a sequence of RV’s
7/24
Random Processes
How do we probabilistically characterize a RP?
View #1 provides one clue!
At each time ti we have a RV and it can be
described by a PDF: P(x;ti). It is possible that the
PDF changes with time.
DT Example #3 – temp @ noon on day k.

P(x;k1) k = winter day P(x;k2) k2 = summer day
1
-10° 20° 50° x 60° 80°100° x
8/24
Random Processes
So to describe a RP, we need P(x;t) for all t!
Is that enough? NO!
RECALL: To describe two RV’s you need a PDF for each

and need to know how they are related
⇒how they are Correlated ⇒need their joint PDF!
But we have more than 2 RV’s in our RP!

⇒To completely describe a RP we need
all possible joint PDF’s over the Time Points:
9/24
Random Processes
To Completely Describe a RP…
Need P(x1,x2,…, xn; t1,t2,…tn)

For every choice of t1,…,tn
For all n (up to ∞ !)
t1 t2 tn
Realization #1
Realization #2
Realization #3
x1 x2 ….. xn 10/24
Random Process
This complete description of RP is virtually
impossible to use for practical applications!
Usually make do with 1st and 2nd order PDF’s:
P(x;t) & P(x1,x2; t1,t2)
Q: What do the 1st and 2nd order PDF’s tell us ?
Ans #1 1st order P(x;t) tells, as a function of time, what
values are likely and unlikely to occur
P(x;t) can be characterized (but not necessarily

completely) by two parameters: Mean & Variance
11/24
Random Process
1) Mean or Average or “Expected Value” of x(t)
∞
E{x(t )} = ∫ x P( x; t ) dx
−∞
Other Notations:
x(t ) = E{x(t )}
mx (t ) = E{x(t )}
Shows “center of concentration” of possible values
of x(t) as a Function of time in general.
12/24
Random Process
2) Variance of X(t)
{ }
Deviation from mean
σ x2( t ) = E [x (t ) − x (t )] 2
∞
= ∫ [x − x (t )] P ( x; t ) dx
2
−∞
Shows a measure of expected range of variation

around the mean, as a function of time in general
13/24
Random Process
Example: let x[k] be temperature at noon on kth

day (at a particular fixed location).
At each day k we view the ensemble of temperature

realizations x[k] as occurring over a collection of
“parallel universes” and the values occur with
probability according to the PDF P(x;k)
14/24
Random Processes
This ensemble view is theoretical and is used to

model reality – don’t confuse these theoretical
ensemble averages with the empirical averages
used in data analysis.
N
∑ score ( i )
1
Test Average =
N i =1
15/24
Time-Varying PDF of RP
Mean
PDF p(x;k)
Varies
0.03 For This
0.025 Example
0.02
0.015
0.01
0.005
0
150
100
50
x (Temp in F)
400
0 350
300
250
200
150
100
-50 50 k (Day of the Year)
0
16/24
Sample Functions
of This TV RP
Five Sample Functions of Temp RP
100 Can See
Temp in F Temp in F Temp in F Temp in F Temp in F
50 The
0 Varying
100 0 50 100 150 200 250 300 350 400
Mean
50
0
100 0 50 100 150 200 250 300 350 400
50
0
100 0 50 100 150 200 250 300 350 400
50
0
100 0 50 100 150 200 250 300 350 400
50
0
0 50 100 150 200 250 300 350 400
k (Day of year)
17/24
Different Example
Time-Varying PDF of RP
Variance
Varies
PDF: p(x;k)
For This
0.08 Example
0.06
0.04
0.02
0
50
400
0 300
200
x (value) 100 k (time index)
-50 0
18/24
Sample Functions
of this TV RP
Five Sample Functions of RP Can See
100
The
0 Varying
x
-100 Variance
50 0 50 100 150 200 250 300 350 400
0
x
-50
50 0 50 100 150 200 250 300 350 400
0
x
-50
100 0 50 100 150 200 250 300 350 400
0
x
-100
50 0 50 100 150 200 250 300 350 400
0
x
-50
0 50 100 150 200 250 300 350 400
k index
19/24
Random Processes
Q: What do the 1st and 2nd order PDF’s tell us ?

Recall : 1st order PDF of RP tells the
Ans #2 likelihood of values occurring at each
given time
2nd Order PDF Characterizes “Probabilistic Coupling”

between RP values at each pair of times t1 and t2
Example : What is the probability x(t1) and x(t2) are…
…. both Positive?
…. both Negative?
…. of Opposite Signs?
20/24
Random Processes
As with mean and variance for the 1st order PDF,
we want something that captures most of the
essence of the 2nd order PDF
Auto correlation function (ACF) of a RP
R x ( t1 , t 2 ) = E {x ( t1 ) x ( t 2 )}
Correlates process at pairs
of times t1, t2
∞ ∞
R x (t1 , t2 ) = ∫ ∫ x1 x2 P( x1, x2 ; t1, t2 ) dx1, dx2
−∞ −∞
21/24
Auto Correlation Function
Positive value if x(t1)& x(t2) are highly likely to

have the same sign
RX(t1, t2) = Negative value if x(t1)& x(t2) are highly likely to

have opposite signs
Near Zero if Product x(t1)x(t2) is ≈ equally

likely to be positive or neg.
22/24
Comparing ACF’s of 2 RP’s
Note : Both x(t) & y(t) have the same 1st Order PDF,
….yet they appear to be very different!
Four Realizations of x(t) Four Realizations of y(t)

t1
τ t2= t1+τ t1 t2= t1+τ
τ
t t
t t
t t
t t
x(t1) x(t2) y(t1) y(t1) 23/24
Four Realizations of x(t) Four Realizations of y(t)
τo τo
t t
t t
t t
t t
t1 t2= t1+τo τo t1 t2= t1+τo
Ry (t1, t1+τ)
RX (t1, t1+τ)
τ
For t2= t1+ τ, R x ( t1 , t 2 ) = R x ( t1 , t1 + τ )
View as a function of τ for each fixed t1 24/24

Baseband Data Transmission I
After this lecture, you will be able to

– describe the components of a digital transmission system
• Information source, transmitter, channel, receiver and
destination
– calculate the signaling rate and bit rate of a system
– design the matched filter of a receiver

• derive the condition for maximum signal-to-noise ratio at
the receiver
– determine the error rate
• Error rate versus received signal energy per bit per hertz
of thermal noise
D.1
Reference
Reference
– Chapter 4.1 - 4.3, S. Haykin, Communication Systems, Wiley.
D.2
Introduction
Digital communication system

Information Transmitter Destination
source channel Receiver
Transmitted Received
Noise
message message
Information source
– produces a message (or a sequence of symbol) to be
transmitted to the destination.
– Example 1
• Analog signal (voice signal): sampling, quantizing and
encoding are used to convert it into digital form
D.3
Introduction (1)
– Sampling and quantizing
87
76
65
5
4
Quantization 4
noise 3 Digits
3
2
2
1
1
00
t
D.4
Introduction (2)
– encoding
Digits Binary code Return-to-zero
00 000
21 001
32 010
43 011
54 100
65 101
76 110
87 111
D.5
Introduction (3)
– Example 2
• digital source from a digital computer
Transmitter
– operates on the message to produce a signal suitable for
transmission over the channel.
D.6
Introduction (4)
Channel
– medium used to transmit the signal from transmitter to the
receiver
– Attenuation and delay distortions
– Noise
Receiver
– performs the reverse function of the transmitter
• determine the symbol from the received signal
¾Example: 1 or 0 for a binary system
Destination
– the person or device for which the message is intended.
D.7
Signaling Rate
Digital message
– An ordered sequence of symbols drawn from an alphabet of
finite size µ.
• Example
¾Binary source: µ=2 for alphabet 0,1 where 0 and 1 are
symbols
¾A 4 level signal has 4 symbols in its alphabet such as ±1, ±3
Signaling Rate
– The symbols are suitably shaped by a shaping filter into a sequence
of signal-elements. Each signal-element has the same duration of T
second and is transmitted immediately one after another, so that the
signal-element rate (signaling rate) is 1/T elements per second
(bauds).
D.8
Bit Rate
Binary digit: 0 and 1
Symbols: 1,2,3 and 4

D.9
Bit Rate
Bit Rate
– The bit rate is the product of signaling rate and no of
bit/symbol.
– Example
• A 4-level PAM with a signaling rate = 2400 bauds/s.
• Bit rate (Data rate) =2400 X log2(4) = 4800 bits/s (bps)
D.10
Matched Filter (1)
– A basic problem that often arises in the study of

communication systems is that of detecting a pulse
transmitted over a channel that is corrupted by channel noise
t t
Square pulse Signal at the receiving end
D.11
Matched Filter (2)
– A matched filter is a linear filter designed to provide the

maximum signal-to-noise power ratio at its output. This is
very often used at the receiver.
x(t ) = g (t ) + w(t ) y (t ) = g o (t ) + n(t )
– Consider that the filter input x(t) consists of a pulse signal

g(t) corrupted by additive noise w(t). It is assumed that the
receiver has knowledge of the waveform of the pulse signal
g(t). The source of uncertainty lies in the noise w(t). The
function of receiver is to detect the pulse signal g(t) in an
optimum manner, given the received signal x(t).
D.12
Matched Filter (3)
– The purpose of the circuit is to design an impulse response

h(t) of the filter such that the output signal-to-noise ratio is
maximized.
Signal Power
Let g(f) and h(f) denoted the Fourier Transform of g(t) and
h(t).
∞
g 0 (t ) = ∫ H ( f )G ( f ) exp( j 2πft )df
−∞
∞
∫ H ( f )G( f ) exp( j 2πft )df

2 2
The signal power = g 0 (t ) =
−∞
D.13
Matched Filter (4)
Noise Power
N0
– Since w(t) is white with a power spectral density , the
2
spectral density function of Noise is
N0 2
SN ( f ) = H( f )
2
∞
N0
∫
2
– The noise power = E[n (t )] =
2
H( f ) df
2 −∞
D.14
Matched Filter (5)
S/N Ratio
– Thus the signal to noise ratio become
∞ 2
∫ H ( f )G( f ) exp( j 2πfT )df

η= −∞
∞ ...….(1)
N0
∫
2
H ( f ) df
2 −∞
• (the output is observed at t = T )
D.15
Matched Filter (6)
– Our problem is to find, for a given G(f), the particular form of the
transfer function H(f) of the filter that makes η at maximum.
Schwarz’s inequality:
∞ ∞
∫ φ1 ( x) dx < ∞ and ∫ φ 2 ( x) dx < ∞ ,

2 2
If
−∞ −∞
∞ 2 ∞ ∞
∫ ∫ φ1 ( x) dx ∫ φ 2 ( x) dx
2 2
φ1 ( x)φ 2 ( x)dx ≤
−∞ −∞ −∞
This equality holds, if and only if, we have φ1 ( x) = kφ 2* ( x)

where k is an arbitrary constant, and * denotes complex
conjugation.
D.16
Matched Filter (7)
Applying the schwarz’s inequality to the numerator of

equation (1), we have
∞ ∞ ∞
∫ H ( f )G( f ) exp( j 2πfT )df ∫ ∫ G( f ) df

2 2 2
≤ H ( f ) df
−∞ −∞ −∞
……(2)
(Note: e j 2πfT = 1)
D.17
Matched Filter (8)
Substituting (2) into (1) ,
∞
2
∫ G( f )
2
The S/N ratio η ≤ df
N0 −∞
or
2E
η≤ ……(3)
N0
∞
where the energy E= ∫ G ( f ) df is the input signal energy
2
−∞
D.18
Matched Filter (9)
Notice that the S/N ratio does not depend on the transfer
function H(f) of the filter but only on the signal energy.
The optimum value of H(f) is then obtained as
H ( f ) = kG * ( f ) exp(− j 2πfT )
D.19
Matched Filter (10)
Taking the inverse Fourier transform of H(f) we have

∞
h(t)=k ∫ G * ( f ) exp[− j 2πf (T − t )]df
−∞
and G * ( f ) = G (− f ) for real signal g (t )

∞
h(t)=k ∫ G ( − f ) exp[− j 2πf (T − t )]df
−∞
h(t)=kg(T-t) …..(4)
Equation (4) shown that the impulse response of the filter is

the time-reversed and delayed version of the input signal
g(t). ”Matched with the input signal”
D.20
Matched Filter (11)
Example:
The signal is a rectangular pulse.
g (t )
T t
The impulse response of the matched filter has exactly

the same waveform as the signal.
h(t )
kA
T t D.21
Matched Filter (12)
The output signal of the matched filter has a triangular

waveform.
g o (t )
kA2T
T t
D.22
Matched Filter (13)
In this special case, the matched filter may be

implemented using a circuit known as integrate-and-
dump circuit.
T
r (t ) ∫0
Sample at t = T
D.23
Realization of the Matched filter (1)
Assuming the output of y(t) = r(t) ⊗ h(t)

t
=
∫ r (τ )h(t − τ )dτ ...(5)
Substitute (4) into (5) we have

t
y (t ) = ∫ r (τ ) g[T − (t − τ )]dτ
0
When t = T
T
y(T)= ∫ r (τ )g (τ )dτ
0
D.24
Realization of the Matched filter (2)
T
r (t ) ∫0
g (t )
Correlator
D.25
Error Rate of Binary PAM (1)
Signaling
– Consider a non-return-to-zero (NRZ) signaling (sometime
called bipolar). Symbol 1 and 0 are represented by positive
and negative rectangular pulses of equal amplitude and
equal duration.
Noise
– The channel noise is modeled as additive white Gaussian
noise of zero mean and power spectral density No/2. In the
signaling interval 0 ≤ t ≤ Tb , the received signal is
+ A + w(t ) symbol 1 was sent

x(t ) = 
− A + w(t ) symbol 0 was sent
• A is the transmitted pulse amplitude
• Tb is the bit duration
D.26
Receiver
T y Decision 1 if y > λ
x(t ) ∫0 device 0 if y < λ
Sample at t = Tb
λ
– It is assumed that the receiver has prior knowledge of the

pulse shape, but not its polarity.
– Given the noisy signal x(t), the receiver is required to make a
decision in each signaling interval
D.27
– In actual transmission, a decision device is used to determine

the received signal. There are two types of error
• Symbol 1 is chosen when a 0 was actually transmitted
• Symbol 0 is chosen when a 1 was actually transmitted
Case I
– Suppose that a symbol 0 is sent then the received signal is
x(t) = -A + n(t)
If the signal is input to a bandlimited low pass filter
(matched filter implemented by the integrate-and-dump
circuit), the output y(t) is obtained as:
t Tb
1
∫
y(t)= x (t ) dt = − A +
∫
Tb 0
n(t )dt
0 D.28
As the noise n(t ) is white and Gaussian, we may

characterize y (t ) as follows:
• y (t ) is Gaussian distributed with a mean of –A
N0
• the variance of y(t) can be shown as σ y =
2
2Tb
(Proof refers to p.254, S. Haykin, Communication
Systems)
D.29
The Probability density function of a

Gaussian distributed signal is given as
1 ( y − y)2
f y ( y 0) = exp(− )
2π σ y 2σ y
2
1 ( y + A) 2
∴ f y ( y 0) = exp(− )
πN 0 / Tb N 0 / Tb
where f y ( y | 0) is the conditional probability density

function of the random variable y, given that 0 was sent
D.30
– Let p10 denote the conditional probability of error, given that

symbol 0 was sent
• This probability is defined by the shaded area under the
curve of f y ( y | 0) from the threshold λ to infinity, which
corresponds to the range of values assumed by y for a
decision in favor of symbol 1
D.31
The probability of error, conditional on sending symbol 0 is

defined by
P10 = P( y > λ Symbol 0 was sent )
∞
= ∫ f y ( y 0)dy
λ
∞
1 ( y + A) 2
=
πN 0 / Tb ∫ exp(−
N 0 / Tb
)dy
λ
D.32
• Assuming that symbols 0 and 1 occur with equal

probability, i.e.
P0 = P1 = 1 / 2
• If no noise, the output at the matched filter will be –

A for symbol 0 and A for symbol 1. The threshold λ
is set to be 0.
D.33
y+ A
Define a new variable z =
N o / Tb
N0
and then dy = dz.
Tb
∞
1
We have
P10 =
π ∫ exp(− z 2 )dz
Eb / N o
where Eb is the transmitted signal energy per bit,

defined by Eb = A 2Tb
D.34
At this point we find it convenient to introduce the definite

integration called complementary error function.
∞
2
erfc(u ) = ∫ − 2
exp( z )dz
π u
Therefore, the conditional probability of error

1 Eb
P10 = erfc( )
2 N0
u
2
( Note: erf(u ) = ∫ exp(− z 2
)dz and erfc(u)=1-erf(u) )
π 0
D.35
In some literature, Q function is used instead of erfc

function.
∞
1 u2
Q( x) = ∫
2π x
exp(− ) du
2
1 x
Q( x) = erfc( ) and erfc( x) = 2 Q( x 2 )
2 2
D.36
Case II
Similary, the conditional probability density function of Y
given that symbol 1 was sent, is
1 ( y − A) 2
f y ( y 1) = exp(− )
πN 0 / Tb N 0 / Tb
λ
1 ( y − A) 2
P01 =
πN 0 / Tb ∫
−∞
exp(−
N 0 / Tb
)dy
D.37
By setting λ =0 and putting

y−A
= −z
N 0 / Tb
we find that P01 = P10
The average probability of symbol error Pe is obtained as

Pe = P0 P10 + P1P01
If the probability of 0 and 1 are equal and equal to ½

1 Eb
Pe = erfc( )
2 N0
D.38
D.39
Maximum SNR Property

Assume AWGN
Matched Filter We want the filter that yields the
highest signal-to-noise ratio (SNR) at its
output
Instructor: M.A. Ingram
Next, show that the optimum filter is
ECE4823 the matched filter
Set Up Signal and Noise Parts

Suppose r (t ) = s (t ) + n(t ) , where n(t) is The signal part of the output is
WGN with spectral height No/2, and s(t) t
is a signal with a finite duration T ys (t ) = ∫ s(u )h(t − u )du

0
Let r(t) be the input to a filter with
impulse response h(t) The noise part of the output is
t
Let y(t) be the output
yn (t ) = ∫ n(u ) h(t − u )du

r (t ) h(t ) y (t ) 0
Signal-to-noise Ratio Denominator

ys2 (t )
SNR =
[
E yn2 (t ) ] [
E yn2 (t ) ]
2  t
 t  
t  = E  ∫ n(u )h(t − u )du   ∫ n(v)h(t − v)dv  
 ∫ s (u )h(t − u )du   0  0  
= 0  t t
= ∫ ∫ E{n(u )n(v)}h(t − u )h(t − v)dudv
2
t 
E  ∫ n(u )h(t − u )du  0 0
0 
1
Invoke White Noise Model SNR So Far

To optimize the SNR, choose h(u) to
[ ]

E yn2 (t ) maximize the numerator

t t 2
N0 t 
= ∫∫ δ (u − v)h(t − u )h(t − v)dudv
2  ∫ s (u )h(t − u )du 
SNR =  0 t 
0 0
t
N0 2 N0 2
2 ∫0
= h (t − u )du
2 ∫0
h (t − u )du
Cauchy-Schwarz Inequality Cauchy-Schwarz for Signals

Say S and Q are two points in a Hilbert Let S and Q be points in the Hilbert
space, then space of square-integrable functions
2 2 2 Then,
S,Q ≤S Q 2
t  t t
∫  ∫
≤ ∫
2
with equality when Q=cS s (u ) q (u ) du s (u ) du q 2 (u )du
0  0 0
Equality is reached when cs(u ) = q (u )
Apply Cauchy-Schwartz Simplify Optimal SNR

Recall numerator of SNR Substitute h(t-u)=cs(u)
2
t 
 ∫ s(u )h(t − u )du  2
0   t 2  t
 ∫
c s (u ) du  ∫ s 2 (u ) du
Pick h(t-u) to be equal to cs(u) SNR opt (t ) =  0 2 t  =0
N 0c N0
2 0 ∫ s 2 (u ) du
2
2
Max SNR Filter = Matched

Optimize t Filter
If s(t) has finite duration T , then SNR Matched filter s(u)
is maximized by setting t=T impulse response is a
0 T
u
“flipped in place”
T
version of signal h(T-u)
∫s
2
(u )du
2ε s h(T − u ) = cs(u ) u
SNR opt = 0
= 0 T
N0 N0 or h(u)
2 h(u ) = cs(T − u )
0 T
u
Inner Product Interpretation Matched Filter Frequency

of Matched Filter Response
T
y (T ) = ∫ h(u )r (T − u )du
Take Fourier Transform of the Matched
0 Filter impulse response
T
T
= ∫ cs(T − u )r (T − u )du
0
H ( f ) = ∫ h(u )e − j 2πfu du
T 0
= c ∫ s (u )r (u )du = c s, r T
0
= c ∫ s (T − u )e − j 2πfu du
h(t ) =
r (t ) y (t ) 0
cs(T − t )
Let r=T-u Matched in Frequency Domain

T The magnitude of the matched filter
H ( f ) = c ∫ s (T − u )e − j 2πfu du response is just a scaled version of the
0 signal’s F.T.
0
= c ∫ s (r )e − j 2πf (T − r ) (− dr ) H ( f ) = c S( f )
T
T
= ce − j 2πfT ∫ s (r )e j 2πfr dr = ce − j 2πfT [S ( f )]
*
3
Relation to Optimum Alternative Form of Minimum

Detection Distance Detector
Recall optimum detector is the Expand the signal space distance
minimum distance detector between the received vector r and the
noiseless signal point sm
choose s1
2 2 2
Signal Space r − sm = r − 2 r, s m + s m
choose s2
The black dots are mˆ opt = arg min [− 2 r, s m + ε m ]

m
the noiseless points
If Signals are Equal Energy Summary

The minimum distance receiver can be When the input is signal plus WGN, then the
filter that maximizes the SNR is the matched
implemented as a bank of matched filter
filters The proof is an application of the Cauchy-
Schwarz Inequality
mˆ opt = arg min [− 2 r, s m + ε m ] The filter “matches” (has the same shape as)
m
the signal in magnitude in the frequency
domain
= arg max r, s m
m
T
The minimum-distance receiver can be
implemented as a bank of matched filters
= arg max ∫ r (t ) sm (t )dt
m
0
4
MATCHED FILTERS
•The matched filter is the optimal linear filter
for maximizing the signal to noise ratio (SNR)
in the presence of additive stochastic noise.
•Matched filters are commonly used in radar,
in which a signal is sent out, and we measure
the reflected signals, looking for something
similar to what was sent out.
•Two-dimensional matched filters are
commonly used in image processing, e.g., to
improve SNR for X-ray pictures
1
• A general representation for a matched

filter is illustrated in Figure 3-1
FIGURE 3-1 Matched filter
2
• The filter input x(t) consists of a pulse signal g(t) corrupted by

additive channel noise w(t), as shown by
(3.0)
• where T is an arbitrary observation interval. The pulse signal g(t)
may represent a binary symbol I or 0 in a digital communication
system.
• The w(t) is the sample function of a white noise process of zero
mean and power spectral density No/2.
• The source of uncertainty lies in the noise w(t).
• The function of the receiver is to detect the pulse signal g(t) in an
optimum manner, given the received signal x(t).
• To satisfy this requirement, we have to optimize the design of the
filter so as to minimize the effects of noise at the filter output in some
statistical sense, and thereby enhance the detection of the pulse
signal g(t).
3
• Since the filter is linear, the resulting output y(t) may be

expressed as
• (3.1)
• where go(t) and n(t) are produced by the signal and

noise components of the input x(t), respectively.
• A simple way of describing the requirement that the
output signal component go(t) be considerably greater
than the output noise component n(t) is to have the filter
make the instantaneous power in the output signal go(t),
measured at time t = T, as large as possible compared
with the average power of the output noise n(t). This is
equivalent to maximizing the peak pulse signal-to-noise
ratio, defined as
(3.2)
4
White Noise
• For the case of white noise, the
description of the matched filter is
simplified as follows: For white noise,
= No / 2. Thus equation becomes,
(3.3)
• From this equation, the following theorem

is obtained
5
Example 3.1
(3.4)
Proof
6
• But s(t) is a real signal; hence, let

C=2K/No so that the impulse response is
equivalent to equation (3.4).
• Equation (3.4) shows that the impulse
response of the matched filter (white-noise
case) is simply the known signal
waveshape that is "played backward" and
translated by an amount to. Thus, the filter
is said to be "matched" to the signal.
7
8
FIGURE 3-2 Waveforms associated with the match filter of Example 3-1
Error rate due to Noise

• To proceed with the analysis, consider a binary PCM
system based on polar nonreturn-to-zero (NRZ)
signaling.
• symbols 1 and 0 are represented by positive and

negative rectangular pulses of equal amplitude and
equal duration.
• The channel noise is modeled as additive white

Gaussian noise w(t) of zero mean and power spectral
density NO/2; the Gaussian assumption is needed for
later calculations. In the signaling interval O < t < Tb the
received signal written as:
9
• where Tb is the bit duration, and A is the transmitted

pulse amplitude.
• The receiver has acquired knowledge of the starting and

ending times of each transmitted pulse;
• Given the noisy signal x(t), the receiver is required to

make a decision in each signaling interval as to whether
the transmitted symbol is a 1 or a 0.
10
• The structure of the receiver used to perform this decision-making

process is shown in Figure 3.3. It consists of a matched filter
followed by a sampler, and then finally a decision device.
FIGURE 3.3 Receiver for baseband transmission of binary-encoded

PCM wave using polar NRZ signaling.
The filter is matched to a rectangular pulse of amplitude A and duration Tb,

exploiting the bit-timing information available to the receiver.
The resulting matched filter output is sampled at the end of each
signaling interval.
The presence of channel noise w(t) adds randomness to the matched filter
output. 11
• The sample value y is compared to a preset threshold A

in the decision device.
• If the threshold is exceeded, the receiver makes a

decision in favor of symbol 1; if not, a decision is made in
favor of symbol 0.
• There are two possible kinds of error to be considered:
– 1. Symbol 1 is chosen when a 0 was actually transmitted; we

refer to this error as an error of the first kind.
– 2. Symbol 0 is chosen when a 1 was actually transmitted; we

refer to this error as an error o f the second kind.
12
Raised Cosine Spectrum

• Overcome the practical difficulties encountered with the
ideal Nyquist channel by extending the bandwidth from
the minimum value W = Rb/2 to an adjustable value
between W and 2W.
• Restrict the frequency band of interest to [- W, W], as

shown by
13
• A particular form of P(f) that embodies

many desirable features is provided by a
raised cosine spectrum.
• This frequency response consists of a flat
portion and a rollo f f portion that has a
sinusoidal form, as follows:
14
• The frequency parameter f, and bandwidth W are related

by
• The parameter a is called the rolloff factor; it indicates

the excess bandwidth over the ideal solution, W.
Specifically, the transmission bandwidth BT is defined by
• The frequency response P(f), normalized by multiplying it

by 2W, is plotted in Figure 3.4a for three values of a,
namely, 0, 0.5, and 1.
15
FIGURE 3-4 Responses for different roll-off factors. 16

(a) Frequency response. (b) Time response.
17
Implementation of the optimum receiver:

Correlation receiver
145
To implement the MAP rule, we need to:

1. Find r from r(t)
2. Correlate with sm and add ηm
3. Compare and decide


Correlation receiver
146
Alternatively, we can use correlation between r(t)

and sm(t), instead of r.sm.
Comparative complexity depends on N and M.


Matched filter receiver
147
In both implementation
p of the correlation receiver
we calculate
where x(t) is either sm(t) or φl(t).

Instead we can use a filter with h(t)=x(Ts-t), or
h(t) is called a matched filter.


148


149
In frequency domain we have H ( f ) = X * ( f )e − j 2πfT

s
Matched filter maximizes SNR at its output:

Upper bound on Probability of Error: Union

Bound
150
The SER for an ML receiver is
where
L us define
Let d fi
we have

Upper bound on Probability of Error: Union

Bound
151
Consequently
Example: 16QAM

Lower bound on Probability of Error

152
The SER for an ML receiver is
We have

AT77.11 Digital Modulation Techniques
II. DIGITAL TRANSMISSION THROUGH AN

ADDITIVE WHITE GAUSSIAN NOISE CHANNEL
2.1 PULSE MODULATION SIGNALS AND THEIR GEOMETRIC
REPRESENTATION
The pulse modulation signals considered include
(1) Pulse Amplitude Modulated Signals
(2) Pulse Position Modulated (orthogonal) Signals
(3) Bi-orthogonal Signals
(4) Simplex Signals
(5) Signals Generated from Binary Code Sequences
2.1.1 Pulse Modulation Signals
In Pulse Amplitude Modulation (PAM), the information is conveyed by the
amplitude of the pulse. Bit 1 is represented by a pulse of amplitude A and bit
0 is represented by a pulse of amplitude –A.
Pulse are transmitted at a bit rate Rb = 1 Tb per second. Tb is the bit interval.
g1 (t ) Tb
A g 2 (t )
Figure 2.1.1 Example of signals
Tb for binary PAM.
0 −A
2-1
In Pulse Position Modulation (PPM), the information is conveyed by the time

interval in which the pulse is transmitted.
In binary PPM, the bit interval is divined into the two time slot of Tb/2 seconds
each.
The information bit 1 is represented by a pulse of amplitude A in the 1st slot,
the information bit 0 is represented by a pulse of amplitude A in the 2nd slot.
The pulse shape within a time slot need not be rectangular.
g1 (t ) g 2 (t )
A A
0 Tb 2 Tb t 0 Tb 2 Tb t
Figure 2.1.2 Example of signals for binary PPM
Another pulse modulation method for transmitting binary information is called

On-Off Keying (OOK).
The transmitted signal for transmitting a 1 is pulse of duration Tb .
the transmitted signal for transmitting a 0 is no pulse of duration Tb .
2-2
2.1.2 The Generalization of PAM and PPM to Non-binary (M-ary)

Pulse Transmission
The binary information sequence is sub-divined into block of k bits, called
symbols, and each block (symbol) is represented by one of M = 2 pulse
k
amplitude values for PAM and pulse position values for PPM.
The symbol interval is
k
T= = kTb (2.1.1)
Rb
0 Tb 2Tb kTb
T is the symbol interval. Tb is the bit interval.
Figure 2.1.3 Relationship between the symbol interval and the bit interval
2-3
2.1.2.1 M-ary PAM Signal

The M-ary PAM signal waveform can be expressed as
sm (t ) = Am gT (t ), m = 1, 2, …, M , 0 ≤ t ≤ T (2.1.2)
Where gT (t ) is a pulse of some arbitrary shape.

gT (t ) Figure 2.1.4 Signal pulses
gT (t )
1
0 T 0 T M
Signal pulses for PAM Signal pulses for PPM
Another important feature of these signals is their energies. They can be
expressed as
T T
Em = ∫ s (t )dt = A
2
m
2
m 0 ∫ gT2 (t )dt , m = 1, 2,…, M (2.1.3)
0
Note: the signals have different energies.

2-4
2.1.2.2 M-ary PPM Signal

The signals can be expressed as
s m ( t ) = Ag T ( t − ( m − 1) T M ) , m = 1, 2, … , M
( m − 1) T M ≤ t ≤ m T M (2.1.4)
Where g T (t ) is a pulse of duration T/M and of arbitrary shape.
A major distinguish characteristics of these waveforms is that they are non-

overlapping. Consequently,
T
∫0
sm (t ) sn (t )dt = 0, m ≠ n (2.1.5)
Such signal waveforms are said to be orthogonal.

A second distinguishing feature of PPM signals is that all the signals have
equal energy, i.e.,
T mT M
∫ s ( t ) dt = A ∫ g 2 ( t − ( m − 1) T M ) dt
2 2
0 m ( m − 1) T M
(2.1.6)
T M
= A2 ∫ g 2 ( t ) dt = E s , ∀m
0
2-5
The Difference between a set of M PAM and a set of M PPM Signals

A major difference between a set of M PAM signals and a set of M PPM
signals is the channel bandwidth required for their transmission.
The channel bandwidth required to transmit the PAM signals is determined by

the frequency characteristics of the pulse g T (t ) . This basic pulse has a
duration T.
The basic pulse gT (t ) in a set of M PPM signals has a duration T/M . It is

clear that the spectrum of the PPM pulse is M times wider than the spectrum
of a PAM pulse.
- Consequently, the PAM signals do not require an expansion of the
channel bandwidth as M increases.
- On the other hand, the PPM signals require an increase in the
channel bandwidth as M increases.
2-6
PAM and PPM are two examples of a variety of different types of signal sets
that can be constructed for transmission of digital information over base-band
channels.
For example, if we take a set of M/2 PPM signals and construct the M/2 negative
signal pulses, the combined set of M signals constitute a set of M biorthogonal
signals.
- All the M signals have equal energy.
Example: M=4 - The channel bandwidth required to transmit the M signals
is just one-half of that required to transmit M PPM
signals.
A A
t t
0 T 2 0 T 2 T
0 T 2 t 0 T 2 T t
Figure 2.1.5 A set of M=4

−A Biorthogonal Signals −A
2-7
2.1.2.3 Simplex Signals

We can construct another set of M signals (Simplex Signal) from any set of M
orthogonal signals by subtracting the average of the M signals from the M
orthogonal signals, thus
M
1
′
sm (t ) = sm (t ) −
M
∑ s (t )
k =1
k (2.1.7)
The energy of these signals sm′ (t ) is
⎛ 1 ⎞
Es′ = ∫ [ sm′ (t ) ] dt = ⎜ 1 − (2.1.8)
T 2
⎟ Es
0
⎝ M ⎠
T 1 1
and ∫0 sm′ (t)sn′ (t)dt = − M Es = − ( M −1) Es′, m ≠ n (2.1.9)
Where Es is the energy of each of the orthogonal signals and Es′ is the energy
of each of the signals in the simplex signal set.
Note: (1) the signals in the simplex signal set have smaller energy than the
signals in the orthogonal signal set.
(2) The simplex signals are not orthogonal. They have negative
correlation, which is equal for all pairs of signals.
2-8
2.1.2.4 Signals Generated from Binary Code Sequence

Let us consider a set of M binary code words of the form
cm = ( cm1 , cm 2 ,…, cmN ) , m = 1, 2,…, M (2.1.10)
Where cmj=0 or 1 for all m and j.

N is called the block length or dimension of the code words.
Given M code words, we can construct M signals by mapping a code bit
cmj=1 into a pulse gT(t) of duration T/N and a code bit cmj=0 into the
negative pulse –gT(t).
Example 2.1.1: Given the code words
c1 = [1 1 1 1] Construct a set of M = 4 signals,

as described above,
c2 = [1 1 0 0] using a rectangular pulse gT (t )
c3 = [1 0 1 0]
c4 = [ 0 1 0 1]
2-9
Solution:
As indicated before, a code bit 1 is mapped into the rectangular pulse gT (t ) of
duration T/4 and a code bit 0 is mapped into the rectangular pulse − gT (t ) .
Thus, we construct the four signals shown as follows that correspond to the
four code words.
s1 (t ) s2 (t )
A A
t T t
0 T 0
−A
s3 (t ) s4 (t )
A A
t t
0 T 4 T 2 3T 4 T 0 T 4 T 2 3T 4 T
−A −A
Figure E2.1.1 Generated signals
Note: the first three signals are mutually orthogonal, but the fourth signals is
the negative of the third.
2-10
2.1.3 Geometric Representation of Signals

Suppose we have a set of M signals sm (t ), 1 ≤ m ≤ M which are to be used for
transmitting information over a communication channel.
From the set of M signals, we first construct a set of N ≤ M orthonormal
signals, where N is the dimension of the signal space. We use Gram-Schmidt
orthogonalization procedure.
2.1.3.1 Gram-Schmidt Orthogonalization Procedure
We begin with the first signals s1(t), which is assumed to have energy E1.
● Step 1: The first signal of the orthonormal set is constructed simply as
ψ 1 (t ) = s1 (t ) E1 (2.1.11)
Thus,ψ 1 (t ) is simply s1(t) normalized to unit energy.
● Step 2: The second signal is constructed from s2(t) by first computing the
projection of s2(t) onto ψ 1 (t ) which is
∞
(2.1.12)
c12 = ∫−∞
s2 (t )ψ 1 (t ) dt
Then, c12ψ 1 (t ) is subtracted from s2(t) to yield
d 2 (t ) = s2 (t ) − c12ψ 1 (t ) (2.1.13)
2-11
Now, d2(t) is orthogonal to ψ 1 (t ) , but it does not posses unit energy.

If E2 denotes the energy in d2(t), then the energy normalized signal that is
orthogonal to ψ 1 (t ) is
ψ (t ) = d (t ) E
2 2 2
(2.1.14)
In general, the orthogonalization of the kth function leads to
ψ k (t ) = d k (t ) Ek (2.1.15)
where k −1
d k (t ) = sk (t ) − ∑c
i =1
ik ψ i (t ) (2.1.16)
∞
and cik = ∫ sk (t )ψ i (t )dt , i = 1, 2,…, k − 1 (2.1.17)
−∞
Thus, the orthogonalization process is continued until all the M signals {sm (t )}
have been exhausted and N ≤ M orthonormal signals have been constructed.
The N orthonormal signals {ψ n (t )} form a basis in the N-dimensional signal space.
The dimensionality N of the signal space will be equal to M if all the M signals
are linearly independent (if none of the signals is a linear combination of the
other signals).
2-12
Once, we have constructed the set of orthogonal signals {ψ n (t )} , we can

express the M signals {sm (t )} as exact linear combinations of the {ψ n (t )}.
Hence, we may write
N
sm (t ) = ∑ smnψ n (t ), m = 1, 2, … , M (2.1.18)
n =1
where T
smn = ∫ sm (t )ψ n (t )dt
0
∞ N
and Em = ∫ s (t )dt = ∑ smn

2 2 (2.1.19)
−∞ m
n =1
Based on the expression in (2.1.18), each signal may be represented by the
sm = ( sm1 , sm 2 ,…, smN )
vector
(2.1.20)
or equivalently, as a point in N-dimensional signal space with coordinates
{smi, i=1,2,…,N}.
The energy of the mth signal is simply the square of the length of the vector or,
equivalently, the square of the Euclidean distance from the origin to the point
in the N-dimensional space.
Thus, any N-dimensional signal can be represented geometrically as a point in
the signal space spanned by the N orthonormal functions {ψ n (t )} .
2-13
Example 2.1.2: Let us apply the Gram-Schmidt procedure to the set of four
signals illustrated in Figure E2.1.2 (a).
s1 (t ) s3 (t )
1 1
t
t
0 0 1 3
2
-1
s2 (t ) s4 (t )
1 1
t t
0 1 2 0 3
-1
Figure E2.1.2 (a) Original signal set
Step 1: ψ 1 (t ) = s1 (t ) E1 Step 2: ψ 2 (t ) = d 2 (t ) E2
2 2 s1 (t )
2 c12 = ∫ s2 (t )ψ 1 (t )dt = ∫ s2 (t ) dt = 0
E1 = ∫ s12 (t )dt = 2 0 0
2
0
d 2 (t ) = s2 (t ) − c12ψ 1 (t ) = s2 (t )
ψ 1 (t ) = s1 (t ) 2 2 2
E2 = ∫ d 22 (t )dt = ∫ s22 (t )dt = 2
0 0
ψ 2 (t ) = s2 (t ) 2
2-14
Step 3: ψ 3 (t ) = d3 ( t ) E3 Step 4: ψ 4 (t ) = d 4 ( t ) E4
s1 ( t )
s1 ( t ) c14 = ∫ s4 ( t )ψ 1 ( t ) dt = ∫ s4 ( t )
3 3
dt = 2
c13 = ∫ s3 ( t )ψ 1 ( t ) dt = ∫ s3 ( t )
3 3
dt = 0 0 0
2
0 0
2
3 3 s2 (t )
s2 ( t ) c24 = ∫ s4 (t )ψ 2 (t )t = ∫ s4 (t ) dt = 0
c23 = ∫ s3 ( t )ψ 2 ( t ) dt = ∫ s3 ( t )
3 3
dt = − 2 0 0
2
0 0
2
c34 = ∫ s4 (t )ψ 3 (t )dt = ∫ s4 (t ) ( s3 (t ) + s2 (t ) ) dt = 1
3 3
d3 (t ) = s3 (t ) − c13ψ 1 ( t ) − c23ψ 2 ( t )
0 0
= s3 ( t ) + 2ψ 2 ( t ) d 4 (t ) = s4 (t ) − c14ψ 1 ( t ) − c24ψ 2 ( t ) − c34ψ 3 ( t )

= s3 ( t ) + s2 ( t ) = s4 ( t ) − 2ψ 1 ( t ) − ( s3 (t ) + s2 (t ) )
= s4 ( t ) − s3 ( t ) − s2 ( t ) − s1 ( t )
E3 = ∫ d (t )dt = ∫ ( s3 (t ) + s2 (t ) ) dt = 1
3 3 2
2
0 3 0 =0
ψ 3 (t ) = d3 (t )
ψ 4 (t ) = 0
= s3 (t ) + s2 (t )
2-15
Thus, s4 (t ) is a linear combination of ψ 1 (t ) and ψ 3 (t ).
Consequently, the dimensionality of the signal set is N=3.
The functions ψ 1 (t ),ψ 2 (t ) and ψ 3 (t ) are shown in Figure E2.1.2 (b).
ψ 1 (t ) ψ 3 (t )
1
1 2
t t
0 2 2 3
ψ 2 (t )
1 2
t
1 2 Figure E2.1.2 (b) Orthonormal signals
−1 2
2-16
Example 2.1.3: Let us determine the vector representation of the four

signals shown in the Figure E2.1.2 (a) by using the orthonormal set of
functions in Figure E2.1.2 (b)
Since, the dimensionality of the signal space is N=3, each signal is described
by three components, which are obtained by projecting each of the four
signals on the three orthonormal basis functions ψ 1 (t ),ψ 2 (t ) and ψ 3 (t ) .
Thus, we obtain s1 = ( ) ( ) ( )
2, 0, 0 , s2 = 0, 2, 0 , s3 = 0, − 2,1 , s4 = ( 2, 0,1)
These signal vectors are shown in Figure E2.1.3
s1 (t ) = 2ψ1 (t )
ψ 2 (t )
s2 s2 (t ) = 2ψ 2 (t )
s3 (t ) =ψ 3 (t ) − s2 (t ) =ψ 3 (t ) − 2ψ 2 (t )
2 0
2 s1 ψ 1 (t )
s4 (t ) = d4 (t ) + 2ψ1 (t ) +ψ 3 (t )
3
ψ 3 (t ) 3
s4 = 2ψ1 (t ) +ψ 3 (t )
s3
Figure E2.1.3 Signal vectors corresponding to the signals si (t ), i = 1, 2,3, 4
2-17
Observation: the set of basis functions {ψ n (t )} obtained by the Gram-

Schmidt procedure is not unique
For example, another set of basis functions that span the three-dimensional
space as shown in the Figure below.
For this basis, the signal vectors are
s1 = (1,1, 0 ) , s2 = (1, −1, 0 ) , s3 = ( −1,1,1) , s4 = (1,1,1)
ψ 1 (t ) ψ 3 (t )
1 1
t t
0 0 2 3
1
ψ 2 (t )
1 Figure. 2.1.6 Alternate set of basis functions
t
0 1 2
Note: The change in the basis functions has not changed the lengths (energies)
of the signal vectors.
2-18
2.1.3.2 Geometric Representation of M-ary Pulse Modulation Signals
The signals for M-ary PAM may be expressed as
sm (t ) = Am gT (t ), 0 ≤ t ≤ T , m = 1, 2, … , M (2.1.21)
Where M=2k and gT(t) is a pulse of unit amplitude.

The M-ary PAM signals are one-dimensional signals, which may be expressed as
sm (t ) = smψ ( t ) , m = 1, 2,…, M (2.1.22)
Where the basis function ψ ( t ) is defined as

1
ψ (t ) = gT (t ), 0 ≤ t ≤ T (2.1.23)
Eg
Eg is the energy of the signal pulse gT(t) , and the signal coefficients (one-
dimensional vectors) are simply
sm = E g Am , m = 1, 2, … , M (2.1.24)
2-19
The Euclidean distance between two signal points is
d mn = sm − sn = Eg ( Am − An )
2 2
(2.1.25)
If we select the signal amplitudes { Am } to be symmetrically spaced about zero

and equally distant between adjacent signal amplitudes, we obtain the signal
points for symmetric PAM, as shown below.
2d 2d 2d 2d 2d
Figure 2.1.7 Signal points (constellation) for symmetric PAM
We observe that the PAM signals have different energies. In particular, the
energy of the mth signal is
Em = sm2 = Eg Am2 (2.1.26)
For equally probable signals, the average energy is

1 M Eg M
Eav =
M
∑ Em =
m =1 M
∑ m
A 2
m =1
(2.1.27)
2-20
Next, we consider a set of M PPM signals which are represented as in (2.1.4),

where g(t) is a pulse of duration T/M.
We recall that these M signals are orthogonal and have equal energy Es.
Therefore, the representation of these signals requires M basis functions.
We define the basis functions as
⎧ 1
⎪ g ( t − ( m − 1) T M ) , ( m − 1) T M ≤ t ≤ mT M
ψ m (t ) = ⎨ Es (2.1.28)
⎪
⎩ 0, otherwise
For m=1,2,…,M.
Hence, M-ary PPM signals are represented geometrically by the M-dimensional
( E , 0, 0,…, 0 )
vectors,
s1 = s
s2 = ( 0, E , 0, … , 0 )
s
(2.1.29)
(
sM = 0, 0, 0, … , Es )
2-21
Clearly, these vectors are orthogonal, i.e., si ⋅ s j = 0 when i ≠ j.

The M signal vectors are mutually equidistant, i.e.,
(2.1.30)
d mn = sm − sn = 2 Es , ∀m ≠ n
2
Hence, the minimum distance between signal points is 2 Es .

Figure 2.1.8 shows an example of M=3 orthogonal signals.
ψ 2 (t )
s2
2 Es
2 Es
s1 ψ 1 (t )
s3 2 Es
ψ 3 (t )
Figure. 2.1.8 Orthogonal signals for M=N=3
2-22
2.1.3.3 The geometric representation of M-ary biorthogonal signals

{
s (t ), 1 ≤ m ≤ M 2, − s (t ), M 2 + 1 ≤ m ≤ M .
m m }
We begin with M/2 orthogonal vectors in N=M/2 dimensions and then append
their negatives. Thus we obtain
s1 = ( E , 0, 0,… , 0 )
s s2
s2 = ( 0, E , 0, … , 0 )
s
sM 2 (
= 0, 0, 0, … , E s ) − s1 s1
sM 2 +1 (
= − E s , 0, 0, … , 0 ) − s2
(
s M = 0, 0, 0, … , − E s ) (2.1.31)
Figure 2.1.9 Signal constellation for
M=4 biorthogonal signals
We note that the distance between any pairs of signal vectors is either 2 E s
or 2 E . Hence, the minimum distance between pairs of signal vectors is 2 Es
s
2-23
2.1.3.4 The geometric representation of a set of M simplex signals

It is obtained by subtracting the mean signal vector from a set of m
orthogonal vectors.
Thus, we have 1 M
sm′ = sm −
M
∑s ,
k =1
k m = 1, 2, … , M (2.1.32)
The effect of subtracting the mean signal

M
1
s=
M
∑s
k =1
k
(2.1.33)
from each orthogonal vector is to translate the origin of the M orthogonal

signals to the point s and to minimize the energy in the signal set sm′ { }
If the energy per signal for the orthogonal signals is Es = sm , then the
2
energy for the simplex signals is
Es′ = sm′ = sm − s
2 2
⎛ 1 ⎞ (2.1.34)
= ⎜1 − ⎟ Es
⎝ M ⎠
2-24
The distance between any two signal points is not changed by the transition of
the origin, i.e., the distance between signal points remains at d = 2 Es .
Finally, the M simplex signals are correlated.
The cross-correlation coefficient (normalized cross-correlation) between the
mth and nth signals is
sm′ ⋅ sn′
γ mn =
sm′ sn′
−1 M 1
= =− (2.1.35)
(1 − 1 M ) ( M − 1)
Hence, all the signals have the same pair-wise.
s2
Figure 2.1.10 Signal constellation for
M=4 simplex signals
s3 s1
s4 2-25
2.1.3.5 The geometric representation of a set of M signals generated

from a set of M binary words
We consider the geometric representation of set of M signals generated from

a set of M binary words of the form
cm = ( cm1 , cm 2 ,…, cmN ) , m = 1, 2,…, M (2.1.36)
Where cmj=0 or 1 for all m and j, as described previously.
The M signals are of dimension N ≤ M and are represented geometrically in

vector form as
m (
s = s , s ,…, s , m = 1, 2,…, M
m1 m2 mN ) (2.1.37)
Where smj = ± Es N for all m and j .
We observe that there are 2 N possible signals that can be constructed from the
possible 2 N binary code words.
We may select a subset of M < 2 N signals for transmission of the information.
2-26
We also observe that the 2 N possible signal points correspond to the vertices of
an N-dimensional hypercube with its center at the origin.
The M signals constructed in this manner have equal energy Es.
The cross-correlation coefficient between any pair of signals depends on how we
select the M signals from the 2 N possible signals.
It is apparent that any adjacent signal points have a cross-correlation coefficient
of N −2
γ= (2.1.38)
N
and a corresponding Euclidean distance
ψ 2 (t ) d = 2 Es N (2.1.39)
s2 s1 ψ 2 (t )
ψ 1 (t ) s5 s3
s6
s3 s4 N=2 s4 ψ 1 (t )
ψ 3 (t ) s7
Figure 2.1.11 Signal-space diagrams for s2 N=3
signals generated from binary codes s8
s1 2-27
2.2 OPTIMUM RECEIVER FOR PULSE-MODULATED SIGNALS IN

ADDITIVE WHITE GAUSSIAN NOISE
Let us consider a digital communication system that transmits digital information
by use of any one of the M-ary signals.
Thus, the input sequence to the modulator is subdivided into k-bit blocks or
symbols and each of the M=2k symbols is associated with a corresponding
signal from the set {sm (t ), m = 1, 2,… , M } .
Each signal is transmitted within the symbol (signaling) interval or time slot of
duration T.
To be specific, we consider the transmission of information over the interval
0≤t ≤T
The channel is assumed to corrupt the signal by the additive White Gaussian
Noise as shown in Figure 2.2.1
Figure 2.2.1 Model for received signal passed
through an AWGN channel
Transmitted signal + Received signal
sm (t ) r (t ) = sm (t ) + n(t )
Noise n(t )
2-28
The received signal in the interval 0 ≤ t ≤T may be expressed as
r (t ) = sm (t ) + n(t ), 0 ≤ t ≤ T (2.2.1)
Where n(t) denotes the sample function of Additive White Gaussian Noise
(AWGN) process with power-spectral density Sn(f)=N0/2 W/Hz.
Based on the observation of r(t) over the signal interval, we wish to design a
receiver that is optimum in the sense that it minimizes the probability of
making error.
It is convenient to subdivide the receiver into two parts:
- The signal demodulator: to convert the received signal r(t) into an
( )
N-dimensional vector r = r1 , r2 ,… , rN , where N is the dimension
of the transmitted signals.
- The detector: to decide which of the M possible signals was
transmitted based on observation of the vector r.
Two realization:
- based on the use of signal correlators.
- based on the use of matched filters.
2-29
2.2.1 Correlation-type Demodulator
A correlation-type demodulator decomposes the received signal and the noise

into N-dimensional vectors.
In other words, the signal and the noise are expanded into a series of linearly
weighted orthonormal basis functions {ψ n (t )}.
It is assumed that the N basis functions {ψ n (t )} span the signal space, so that
every one of the possible transmitted signals of the set {sm (t ),1 ≤ m ≤ M } can
be represented as a weighted linear combination of {ψ n (t )} .
In the case of the noise, the function {ψ n (t )} do not span the noise space.
However, the noise terms that fall outside the signal space are irrelevant to
the detection of the signal.
Suppose the received signal r(t) is passed through a parallel bank of N cross-
correlators which basically compute the projection of r(t) onto the N basis
functions {ψ n (t )}, as illustrated in Figure 2.2.2.
2-30
ψ 1 (t ) Figure 2.2.2 Correlation-type Demodulator

r1
∫ ( ) dt
T
X 0
ψ 2 (t )
Received r2
X ∫0 ( ) dt
T
To detector
signal
r (t )
ψ N (t )
rN
X ∫ ( ) dt
T
0
Sample at t=T
r (t )ψ k (t )dt = ∫ [ sm (t ) + n(t )]ψ k (t )dt

T T
We have
∫ 0 0
rk = smk + nk , k = 1, 2,… , N (2.2.2)
T
where smk = ∫ sm (t )ψ k (t ) dt , k = 1, 2,… , N
0
T
nk = ∫ n (t )ψ k (t ) dt , k = 1, 2,… , N (2.2.3)
0 2-31
The signal is now represented by the vector sm with components smk, k=1,2,…,N.
Their values depend on which of the M signals was transmitted.
The components {nk } are random variables that arise from the presence of
additive noise.
In fact, we can express the received signal r(t) in the interval 0 ≤ t ≤ T as
N N
r (t ) = ∑ smkψ k (t ) + ∑ nkψ k (t ) + n′(t )
k =1 k =1
N
= ∑ rkψ k (t ) + n′(t ) (2.2.4)
k =1
The term n′(t ) , defined as

N
n′(t ) = n(t ) − ∑ nkψ k (t ) (2.2.5)
k =1
is a zero-mean Gaussian noise process that represents the difference between

the original noise process n(t) and that part which corresponds to the projection
of n(t) onto the basis functions {ψ k (t )} .
2-32
We will show below that n′(t ) is irrelevant to the decision as to which signal was
transmitted.
Consequently, the decision my be based entirely on the correlator output signal
and noise components rk = smk + nk , k = 1, 2,… , N .
{ }
Since the signals sm (t ) are deterministic, the signal components are
deterministic. The noise components {nk } are Gaussian.
Their mean values are
E [ nk ] = E [ n ( t ) ]ψ k ( t ) dt = 0,
T
∫0
∀n (2.2.6)
Their covariance are
E [ nk nm ] = E [ n ( t ) n (τ ) ]ψ k ( t )ψ m (τ ) dtd τ
T T
∫ ∫
0 0
N0 T T
=
2 ∫0 ∫0
δ ( t − τ )ψ k ( t )ψ m (τ ) dtd τ
N0 T N0 (2.2.7)
=
2 ∫0
ψ k ( t )ψ m ( t ) dt =
2
δ mk
Where δ mk = 1 when m = k and zero otherwise.
Therefore, the N noise components {nk } are zero-mean uncorrelated Gaussian
random variables with a common variance σ n = N 0 2 .
2
2-33
From the previous development, it follows that the correlator outputs

conditioned on the mth signal being transmitted are Gaussian random
variables with mean
E [ rk ] = E [ smk + nk ] = smk (2.2.8)
and equal variance

σ r2 = σ n2 = N 0 2 (2.2.9)
Since the noise components{nk } are uncorrelated Gaussian random variables,

they are also statistically independent.
As a consequence, the correlator outputs {rk } conditioned on the mth signal
being transmitted are statistically independent Gaussian variables.
Hence, the conditional Probability Density Functions (PDF) of the random
variables ( r1 , r2 ,… , rN ) = r are simply
(2.2.10)
f ( r sm ) = Π f ( rk smk ) , m = 1, 2,… , M
N
k =1
f ( rk smk ) =
1 −( rk − smk )
2
where , k = 1, 2,… , N (2.2.11)
N0
e
π N0
2-34
By substituting (2.2.11) into (2.2.10), we obtain the joint conditional PDF as
⎡ N ⎤
f ( r sm ) =
1
exp ⎢ −∑ ( rk − smk ) N 0 ⎥ , m = 1, 2,… , M (2.2.12)
2
(π N 0 )
N 2
⎣ k =1 ⎦
As a final point, we wish to show that the correlator outputs (r1,r2,…,rN) are
sufficient statistics for reaching a decision on which of the M signals was
transmitted, i.e., that no additional relevant information can be extracted from
the remaining noise process n′(t ) .
Indeed, n′(t ) is uncorrelated with the N correlator outputs {rk }, i.e.,
E [ n′(t )rk ] = E [ n′(t ) ] smk + E [ n′(t )nk ] = E [ n′(t )nk ]
⎧⎪ ⎡ N ⎤ ⎫⎪
= E ⎨ ⎢ n(t ) − ∑ n jψ j (t ) ⎥ nk ⎬
⎩⎪ ⎣ j =1 ⎦ ⎭⎪
N
= ∫ E [ n(t )n(τ ) ]ψ k (τ )dτ − ∑ E ⎡⎣ n j nk ⎤⎦ψ j (t )
T
0
j =1
N0 N (2.2.13)
= ψ k (t ) − 0 ψ k (t ) = 0
2 2
2-35
Since n′(t ) and {rk } are Gaussian and uncorrelated, they are also statistically
independent.
Consequently, n′(t ) does not contain any information that is relevant to the
decision as to which signal was transmitted.
All the relevant information is contained in the correlator outputs {rk } .

Hence, n′(t ) may be ignored.
2-36
Example 2.2.1: Consider an M-ary PAM signal in which the basic pulse shape
gT (t ) is rectangular as shown in Figure 2.2.3.
The additive noise is a zero-mean white Gaussian noise process.
Determine the basis function ψ (t ) and the output of the correlation-type
demodulator.
gT (t )
a Figure 2.2.3 Signal pulse
t
0
T
Solution: The energy in the rectangular pulse is

T T
Eg = ∫ g (t )dt = ∫ a 2 dt = a 2T
2
T
0 0
Since the PAM signal set has a dimension N=1, there is only one basis
function ψ (t ) .
2-37
This obtained from (2.1.23) and given as

⎧ 1
1 ⎪ , 0≤t ≤T
ψ (t ) = gT (t ) = ⎨ T
⎪ 0, otherwise
2
aT
⎩
The output of the correlation-type demodulator is
T 1 T
r=∫ r (t )ψ (t )dt = ∫ r (t )dt
0 0
T
Note: the correlator becomes a simple integrator when ψ (t ) is rectangular.
If we substitute for r(t), we obtain
1 ⎡ T ⎤ dt
r=
⎢
⎣ ∫ 0
[ s m ( t ) + n ( t ) ] ⎥⎦
T
1 ⎡ T T
⎤
= ∫ m
T ⎢⎣ 0
s ψ ( t ) dt + ∫0 n ( t ) dt
⎥⎦
= sm + n
2-38
Where the noise term E [ n ] = 0 and
⎡1 T T ⎤
σ = E ⎢ ∫ ∫ n(t )n(τ )dtdτ ⎥
2
n
⎣T 0 0 ⎦
1 T T
= ∫ ∫ E [ n(t )n(τ ) ] dtdτ
T 0 0
N0 T T N0
=
2T ∫0 ∫0
δ (t − τ )dtdτ =
2
The Probability Density Function for the sampled output is
f ( r sm ) =
1 − ( r − sm )
2
N0
e
π N0
2-39
2.2.2 Matched-Filter Type demodulator

In stead of using a bank of N correlators to generate the variables {rk } , we
may use a bank of N linear filters.
Let us suppose that the impulse responses of the N filters are
hk (t ) = ψ k (T − t ), 0 ≤ t ≤ T (2.2.14)
Where {ψ k (t )} are the N basis functions and hk(t)=0 outside of the interval
0 ≤ t ≤ T.
The outputs of these filters are
t
yk (t ) = ∫ r (τ )hk (t − τ )dτ
0
t
= ∫ r (τ )ψ k (T − t + τ )dτ , k = 1, 2,… , N (2.2.15)
0
Now, if we sample the outputs of the filters at t=T, we obtain

T
yk (T ) = ∫ r (τ )ψ k (τ )dτ = rk , k = 1, 2,… , N (2.2.16)
0
2-40
Hence, the sampled outputs of the filters at time t=T are exactly the set of
values {rk } obtained from the N linear correlators.
A filter whose impulse response h(t)=s(T-t), where s(t) is assumed to be
confined to the time interval 0 ≤ t ≤ T , is called the matched filter to the
signal s(t).
The response of h(t) = s(T-t) to the signal s(t) is

t
y (t ) = ∫ s(τ ) s(T − t + τ )dτ
0
Which is basically the time-autocorrelation function of the signal s(t).
We note that the autocorrelation function y (t) is an even function of t , which

attains a peak at t=T.
2-41
s (t ) h(t ) = s (T − t ) (b) Impulse response

A A of filter matched to s(t)
t t
0 0 T
T t
y (t ) = ∫ s(τ ) s(T − t + τ )dτ
(a) Signal s(t) y (T ) 0
Figure 2.2.4 Signal s(t)

and filter matched to s(t)
Figure 2.2.5 matched t

filter output is the 0 T 2T
autocorrelation function
of s(t)
ψ1(T −t)
r1
Received
r2
ψ2 (T −t) Figure 2.2.6
signal r (t )
matched filter-
rN type demodulator
ψN (T −t)
2-42
Properties of the Matched Filter

The most important property, which may be stated as follows: If a signal s(t)
is corrupted by AWGN, the filter with impulse response matched to s(t)
maximizes the output Signal-to-Noise Ratio (SNR).
Proof: Assume that r (t ) = s (t ) + n(t ) and
E [ nk ] = 0, Sn ( f ) = N 0 2
● The signal r(t) is passed through a filter with impulse response h(t),
0 ≤ t ≤ T , and its output is sampled at time t=T.
The filter response to the signal and noise components is
t
y (t ) = ∫ r (τ )h(t − τ )dτ
0
t t
= ∫ s (τ )h(t − τ )dτ + ∫ n(τ )h(t − τ )dτ (2.2.17)
0 0
At the sampling instant t=T, the signal and noise components are
T T
y (T ) = ∫ s(τ )h(T − τ )dτ + ∫ n(τ )h(T − τ )dτ
Signal 0 0
component = ys (T ) + yn (T ) (2.2.18)
Noise component
2-43
Problem: To select the filter impulse response that maximizes the output
Signal-to-Noise Ratio (SNR) defined as
⎛ S ⎞ y s2 (T ) (2.2.19)
⎜ ⎟ =
⎝ N ⎠ 0 E ⎡⎣ y n (T ) ⎤⎦
2
The denominator in (2.2.19) is simply the variance of the noise term at the
output of the filter.
Let us evaluate E ⎡⎣ yn (T ) ⎤⎦ . We have
2
∫ E [ n(τ )n(t )] h(T − τ )h(T − t )dtdτ

T T
E ⎡⎣ y (T ) ⎤⎦ = ∫
2
n 0 0
N T T
= 0
2 ∫ ∫ 0 0
δ (t − τ )h(T − τ )h(T − t )dtdτ
N T
= 0
2 ∫ 0
h 2 (T − t )dt (2.2.20)
Note: the variance depends on the power-spectral density of the noise and
the energy in the impulse response h(t).
2-44
By substituting for ys (T ) and E ⎡⎣ yn2 (T ) ⎤⎦ into (2.2.19), we obtain the expression

for the output SNR as
2 2
⎡ T s (τ )h(T − τ )dτ ⎤ ⎡ T h(τ ) s (T − τ )dτ ⎤
⎛S⎞ ⎢⎣ ∫0 ⎥⎦ ⎢⎣ ∫0 ⎥⎦
⎜ ⎟ = = (2.2.21)
⎝ N ⎠0 N0 T 2 N0 T 2
2 ∫0
h (T − t )dt
2 ∫0
h (T − t )dt
Since the denominator of the SNR depends on the energy in h(t), the maximum
output SNR over h(t) is obtained by maximizing the numerator of (S/N)0 subject
to the constraint that the denominator is held constant.
The maximization of the numerator is most easily performed by use of the
Cauchy-Schwarz inequality.
Cauchy-Schwarz inequality
If g1(t) and g2(t) are finite-energy signals, then
2
⎡ g (t ) g (t )dt ⎤ ≤ ∞ g 2 (t )dt ∞ g 2 (t )dt
∞
⎢⎣ ∫−∞ 1 ⎥⎦ ∫−∞ 1 ∫−∞ 2

2
(2.2.22)
Where equality holds when g1(t)=Cg2(t) for any arbitrary constant C.

2-45
If we set g1 (t ) = h(t )
and g 2 (t ) = s (T − t )
it is clear that the (S/N)0 is maximized when h(t)=Cs(T-t), i.e., h(t) is matched
to the signal s(t).
The scale factor C2 drops out of the expression for (S/N)0, since it appears in
both the numerator and the denominator.
The output (maximum) SNR obtained with the matched filter is
⎛S⎞ 2 T 2
⎜ ⎟ =
⎝ N ⎠0 N 0
∫ 0
s (t )dt
2 Es
= (2.2.23)
N0
Note: the output SNR from the matched filter depends on the energy of the
signal s(t) but not on the detailed characteristics of s(t).
2-46
Frequency Domain Interpretation of the Matched Filter
Since h(t)=s(T-t), the Fourier transform of this relationship is

T
H ( f ) = ∫ s (T − t )e − j 2π ft dt
0
= ∫ s (τ )e j 2π f τ dτ ⎤ e − j 2π fT
⎡ T
⎢⎣ 0 ⎥⎦
(2.2.24)
= S ∗ ( f )e − j 2π fT
We observe that the matched filter has a frequency response which is the
complex conjugate of the transmitted signal spectrum multiplied by the phase
− j 2π fT
factor e , which represents the sampling delay of T.
In other words, H ( f ) = S ( f ) , so that the magnitude response of the

matched filter is identical to the transmitted signal spectrum.
On the other hand, the phase of H(f) is the negative of the phase of S(f).
2-47
Now, if the signal s(t) with spectrum S(f) is passed through the matched filter,
− j 2π fT
the filter output has a spectrum Y ( f ) = S ( f ) e
2
.
Hence, the output signal is
∞
ys (t ) = ∫ Y ( f )e j 2π ft df
−∞
∞
=∫ S ( f ) e − j 2π fT e j 2π ft df
2
(2.2.25)
−∞
By sampling the output of the matched filter at t=T, we obtain
∞ T
ys (T ) = ∫ S ( f ) df = ∫ s 2 (t )dt = Es
2
(2.2.26)
−∞ 0
Where the last step follows from the Parseval’s relation.
2-48
The noise of the output of the matched filter has a power-spectral density
S0 ( f ) = H ( f ) N 0 2
2
(2.2.27)
Hence, the total noise power at the output of the matched filter is
∞ N0 ∞
Pn = ∫ S0 ( f )df = ∫
2
H ( f ) df
−∞ 2 −∞
N0 ∞ Es N 0 (2.2.28)
2 ∫−∞
= S ( f ) df =
2
The output SNR is simply the ratio of the signal power Ps given by
Ps = ys2 (T ) (2.2.29)
to the noise power Pn . Hence
⎛S⎞ Ps Es2 2 Es (2.2.30)

⎜ ⎟ = = =
⎝ N ⎠0 Pn Es N 0 2 N 0
which agrees with the result given by (2.2.23)
2-49
Example 2.2.2: Consider the M=4 biorthogonal signals shown in the Figure
2.1.5 for transmitting information over an AWGN channel.
The noise is assumed to have zero mean and power-spectral density N0/2.
Determine the basis functions for this signal set, the impulse response of the
matched-filter demodulators, and the output signals of the matched-filter
demodulators when the transmitted signal is s1(t).
Solution: The M=4 biorthogonal signals have dimension N=2.
Hence, two basis functions are needed to represent the signals.
From Figure 2.1.5, we choose ψ 1 (t ) and ψ 2 (t ) as
⎧ 2 T , ⎧ 2 T (2.2.31)
⎪ , 0≤t ≤ ⎪ , ≤t ≤T
ψ 1 (t ) = ⎨ T 2 ψ 2 (t ) = ⎨ T 2
⎪ 0, otherwise ⎪ 0, otherwise
⎩ ⎩
The impulse responses of the two matched filters are
⎧ 2 T ⎧ 2 T
⎪ , ≤t ≤T ⎪ , 0≤t ≤
h1 (t ) = ψ 1 (T − t ) = ⎨ T 2 , h2 (t ) = ψ 2 (T − t ) = ⎨ T 2
⎪ 0, otherwise ⎪ 0, oherwise (2.2.32)
⎩ ⎩
2-50
ψ 1 (t ) ψ 2 (t )
2T 2T
t t
0 T 2 T 3T 2 0 T 2 T 3T 2
Figure E2.2.2 (a) Basis functions
h1 (t ) = ψ 1 (T − t ) h2 (t ) = ψ 2 (T − t )
2T 2T
t t
0 T 2 T 3T 2 0 T 2 T 3T 2
Figure E2.2.2 (b) Impulse responses
y1s (t ) y2 s (t )
2
AT 2 A2T 2
t t
0 T 2 T 3T 2 0 T 2 T 3T 2
Figure E2.2.2 (c) Matched filter response

2-51
If s1 (t ) is transmitted, the (noise-free) responses of the two matched filters

are shown in Figure E2.2.3 (c).
A2T
Since y1(t) and y2(t) are sampled at t=T, we observe that y1s (T ) = and
2
y2s(T)=0.
Note that A2T/2=Es, the signal energy.
Hence, the received vector formed from the two matched filter outputs at the
sampling instant t=T is
( 1 2)
r = r ,r = E + n ,n ( s 1 2 ) (2.2.33)
Where n1=y1n(T) and n2=y2n(T) are the noise components at the outputs of
the matched filter, given by
T
ykn (T ) = ∫ n(t )ψ k (t )dt , k = 1, 2 (2.2.34)
0
Clearly, E [ nk ] = E [ ykn (T ) ] = 0 .
2-52
Their variance is
∫ E [ n(t )n(τ )]ψ

T T
σ = E ⎡⎣ y (T ) ⎤⎦ = ∫
2
n
2
kn k (t )ψ k (τ )dtdτ
0 0
N
∫ ∫ δ ( t − τ )ψ
T T
= 0 k (τ )ψ k (t )dtdτ
2 0 0
N T N0
= 0
2 ∫
0
ψ k2 (t )dt =
2
(2.2.35)
Observe that the (S/N)0 for the first matched filter is
( )
2
⎛S⎞ Es 2 Es
⎜ ⎟ = =
⎝ N ⎠0 N0 2 N0
Which agrees with our previous result.
Note: the four possible outputs of the two matched filters, corresponding to
the four possible transmitted signals in Figure 2.1.5 are
( r1 , r2 ) = ( )( )(
Es + n1 , n2 , n1 , Es + n2 , − Es + n1 , n2 and n1 , − Es + n2 ) ( )
2-53
2.2.3 The Optimum Detector

To design a signal detector that makes decision on the transmitted signal in
each signal interval based on the observation of the vector r=(r1,r2,…,rN)
in each interval such that the probability of a correct decision is maximized.
Assumption: no memory in signals transmitted in successive signal intervals.
We consider a decision rule based on the computation of the posterior
probabilities defined as
P ( signal sm was transmitted r ) , m = 1, 2,… , M
The decision criterion is based on selecting the signal corresponding to the

maximum of the set of posterior probabilities P (sm|r), which is called
maximum a posteriori probability (MAP) criterion.
Using Bayes rule, the posterior probabilities may be expressed as
f ( r sm ) P ( sm )
P ( sm r ) = (2.2.36)
f (r )
Where f (r|sm) is the conditional PDF of the observed vector given sm ,
P(sm) is the a priori probability of the mth signal being transmitted.
2-54
The denominator of (2.2.36) may be expressed as

M
f ( r ) = ∑ f ( r sm ) P ( sm ) (2.2.37)
m =1
From (2.2.36) and (2.2.37) we observe that the computation of the posterior
probabilities P (sm|r) requires knowledge of the a priori probabilities P (sm)
and the conditional PDF f (r|sm) for m=1,2,…, M.
When the M signals are equally probable, a priori, i.e., P(sm)=1/M for all M.
Furthermore, the denominator in (2.2.36) is independent of which signal is
transmitted.
Consequently, the decision rule based on finding the signal that maximizes
P(sm|r) is equivalent to finding the signal that maximizes f (r|sm).
The conditional PDF f (r|sm) is usually called the likelihood function.
The decision criterion based on the maximum of f (r|sm) over the M signals is
called the maximum-likelihood (ML) criterion.
A detector based on the MAP criterion and one that is based on the ML
criterion make the same decisions as long as the priori probabilities P (sm) are
all equal, i.e., the signals {sm} are equiprobable.
2-55
In the case of an AWGN channel, the likelihood function f (r|sm) is given by
f ( r sm ) = Π f ( rk smk ) , m = 1, 2,… , M
N
k =1
f ( rk smk ) =
1 − ( rk − smk )2
where e N0
, k = 1, 2,… , N
π N0
The natural logarithm of f (r|sm), which is a monotonic function, is
N
ln f ( r sm ) = − ln (π N 0 ) −
N 1
∑ ( rk − smk ) (2.2.38)
2
2 N0 k =1
The maximum of f (r|sm) over sm is equivalent to finding the signal sm that

minimizes the Euclidean distance
N
D ( r, sm ) = ∑ ( rk − smk )
2
(2.2.39)
k =1
We call D (r,sm), m=1,2,…, M the distance metrics.
Hence, for the AWGN channel, the decision rule based on the ML criterion
reduces to finding the signal sm that is closest in distance to the received
signal vector r.
This decision rule is referred to minimum distance detection. 2-56
Another interpretation of the optimum decision rule based on the ML criterion

is obtained by expanding the distance metrics in (2.2.39) as
N N N
D ( r, sm ) = ∑ r − 2∑ rn smn + ∑ smn
n
2 2
n =1 n =1 n =1
= r − 2r.sm + sm , m = 1, 2,… , M
2 2
(2.2.40)
2
The term r is common to all decision metrics, and hence, it may be ignored in
the computations of the metrics.
The result is a set of modified distance metrics
D′ ( r , sm ) = −2r.sm + sm
2
(2.2.41)
To select the signal sm that minimizes D′ ( r , s m ) is equivalent to selecting the

signal that maximizes the metric C ( r , s m ) = − D′(r , sm )
C ( r , sm ) = 2r.sm − sm
2
(2.2.42)
The term r.s m represents the projection (or the correlation) of the received
signal vector onto each of the M possible transmitted signal vectors.
We call C (r,sm ), m = 1, 2,… , M the correlation metrics for deciding which of the
M signals was transmitted.
2-57
The term |sm|2=Em, m=1,2,…, M may be viewed as bias terms that serve as
compensation for signal sets that have unequal energies (PAM).
2
If all signals have the same energy, s m may also be ignored in the
computation of the correlation metrics C ( r,sm ) and the distance metrics
D ( r,sm ) or D′ ( r,sm )
Conclusion:
We have demonstrated that the optimum ML detector computes a set of M
distances D ( r,sm ) or D′ ( r,sm ) and selects the signal corresponding to the
smallest (distance) metric.
Equivalently, the optimum ML detector computes a set of M correlation
metrics C (r,sm) and selects the signal corresponding to the largest
correlation metric.
In this case (the signals are equally probable), the MAP criterion is
equivalent to the ML criterion.
But, when the signals are not equally probable, the optimum MAP detector
bases it decision on the probabilities P (sm|r), m=1,2,…, M, given by (2.2.36)
or, equivalently, on the metrics
PM ( r,sm ) = f ( r sm ) P ( sm ) (2.2.43)
2-58
Example 2.2.3: Consider the case of binary PAM signals in which the two
possible signal points are s1 = − s2 = Eb , where Eb is the energy per bit.
The priori probabilities are P(s1)=p and P(s2)=1-p.
Determine the metrics for the optimum MAP detector when the transmitted
signal is corrupted with AWGN.
Solution: The received signal vector (one dimensional) for binary PAM is
r = ± Eb + yn (T ) (2.2.44)
Where yn (T ) is a zero-mean Gaussian random variable with variance σ n2 = N 0 2 .

Consequently, the conditional PDF f (r|sm) for the two signals are
( )
f ( r s1 ) =
2
1 − r − Eb 2σ n2
e (2.2.45)
2πσ n
( )
f ( r s2 ) =
2
1 − r + Eb 2σ n2
e (2.2.46)
2πσ n
2-59
Then the metrics PM (r,s1) and PM (r,s2) defined by (2.2.43) are
( ) (2.2.47)
PM ( r,s1 ) = pf ( r s1 ) =
2
p − r − Eb 2σ n2
e
2πσ n
(1 − p ) −( r + )
PM ( r,s 2 ) = (1 − p ) f ( r s2 ) =
2
Eb 2σ n2 (2.2.48)
e
2πσ n
If PM (r,s1) > PM (r,s2), we select s1 as the transmitted signal; otherwise,

we select s2.
This decision rule may be expressed as
s1
PM ( r,s1 ) >
< 1 (2.2.49)
PM ( r,s 2 ) 2
s
But PM ( r,s1 ) ⎢⎣(

⎡ r+
) −( r − ) ⎤⎥⎦
2 2
p Eb Eb 2σ n2
= e (2.2.50)
PM ( r,s 2 ) (1 − p )
2-60
So that (2.2.50) may be expressed as
( ) ( ) s1
2 2
r + Eb − r − Eb > (1− p)
2σ 2 < ln
p
(2.2.51)
n s2
Or equivalently,
s1
> σ n2 (1 − p ) N 0 (1 − p )
r Eb < ln = ln (2.2.52)
s2 2 p 4 p
This is the final form for the optimum detector.

It computes the correlation metric C ( r,s1 ) = r Eb and compares it with the
threshold ( N 0 4 ) ln (1 − p ) p
Note: in the case of unequal priori probabilities, its is necessary to know not
only the values of the priori probabilities but also the value of the power-
spectral density N0 in order to compute the threshold.
When p=1/2, the threshold is zero, and knowledge of N0 is not required by the
detector.
2-61
Proof: the decision rule based on the maximum-likelihood criterion minimizes

the probability of error when the M signals are equally probable a priori.
Let us denote by Rm the region in the N-dimensional space for which we decide
that signal sm(t) was transmitted when the vector r=(r1,r2,…,rN) is received.
The probability of a decision error, given that sm(t) was transmitted, is
P ( e sm ) = ∫ f ( r s m ) dr (2.2.53)
Rmc
Where Rmc is the complement of Rm. The average probability of error is

M M
P ( e ) = ∑ P ( e sm ) = ∑ ∫ c f ( r sm ) dr
1 1
m =1 M m =1 M
Rm
M
= ∑ ⎡1 − ∫ f ( r sm ) dr ⎤
1 (2.2.54)
m =1 M ⎣
⎢ Rm ⎦⎥
Note: P(e) is minimized by selecting the signal sm if f (r|sm) is larger than
f (r|sk) for all m ≠ k .
When the M signals are not equally probable, the proof given above can be
generalized to show that the MAP criterion minimizes the average probability of
error.
2-62
2.3 PROBABILITY OF ERROR FOR SIGNALS IN ADDITIVE WHITE

GAUSSIAN NOISE
2.3.1 Probability of Error for Binary Modulation
Let us consider binary PAM signals as follows:
s1 (t ) = gT (t ) ⎫
⎬ called antipodal
s2 (t ) = − gT (t ) ⎭
where
⎧arbitrary non − zero pulse, 0 ≤ t ≤ Tb
gT (t ) = ⎨
⎩ 0, elsewhere
The energy in the pulse gT(t) is Eb.
PAM signals are one-dimensional, their geometric representation is simply

the one-dimensional vector s1 = Eb , s2 = − Eb Figure 2.3.1
Signal points for
− Eb Eb arbitrary
antipodal signals
s2 0 s1
2-63
Let us assume that the two signals are equally likely and that signal s1(t) was
transmitted.
Then, the received signal from the (matched filter or correlation-type)
demodulator is (2.3.1)
r = s1 + n = Eb + n
Where n represents the Additive White Gaussian Noise component which
has zero mean and variance σ n = N 0 2 .
2
In this case, the decision rule based on the correlation metric given by (2.2.42)
C ( r, sm ) = 2r.sm − sm
2
(2.2.42)
compares r with the threshold zero.
If r > 0 , the decision is made in favor of s1(t), and if r < 0, the decision is
made that s2(t) was transmitted.
Clearly, the two conditional PDF of r are
−( r − )
p ( r s1 ) =
2
1 Eb N0
(2.3.2)
e
π N0
−( r + )
p ( r s2 ) =
2
1 Eb N0
(2.3.3)
e
π N0
2-64
These two conditional PDF’s are shown in Figure 2.3.1
p ( r s2 ) p ( r s1 )
r
− Eb 0 Eb
Figure 2.3.1 Conditional PDF’s two signals
Given that s1(t) was transmitted, the probability of error is simply the probability
that r < 0, i.e.,
( )
p ( e s1 ) = ∫ p ( r s1 ) dr
2
0 1 0 − r− E
∫
N0
= e dr
−∞
π N0 −∞
set x =
2
N0
(r − Eb )
1 − 2 Eb N 0 1 ∞
∫ ∫
− x2 2
= dx = e − x 2 dx
2
e
2π −∞
2π 2 Eb N 0 ∞
⎛ 2 Eb ⎞ Marcum’s Q-function Definition

= Q ⎜⎜ ⎟⎟ (2.3.4) ∞ 1 −u2 2
⎝ N 0 ⎠ Q ( x) ∫ e du
Where Q ( x ) is the Q-function.
x
2π
2-65
Definition of Q-Marcum Function
The two mostly used upper bounds
1 − x2 2
(1) Q ( x) ≤ e for all x ≥ 0
2
1
(2)
Q ( x) < e − x2 2
for all x ≥ 0
2π x
The frequently used lower bound is
1 ⎛ 1 ⎞ − x2 2
(1) Q ( x) > ⎜1 − 2 ⎟ e for all x ≥ 0
2π x ⎝ x ⎠
2-66
Similarly, if we assume that s2(t) was transmitted, r = − Eb + n and the

probability that r > 0 is also ⎛ 2E ⎞
p ( e s2 ) = Q ⎜⎜ b
⎟⎟
⎝ N 0 ⎠
Since the signals s1(t) and s2(t) are equally likely to be transmitted, the
average probability of error is
⎛ 2 Eb ⎞
pb = p ( e s1 ) + p ( e s2 ) = Q ⎜⎜
1 1 (2.3.5)
⎟⎟
2 2 ⎝ N0 ⎠
Two important characteristics of this performance measure
1. The probability of error depends only on the ratio Eb/N0 and not on any
other detailed characteristics of the signals and noise.
2. 2Eb/N0 is also the output SNR from the matched filter (and correlation-
type) demodulator. Eb/N0 is usually called the signal-to-noise ratio.
The probability of error may be expressed in terms of the distance between the
two signals s1 and s2.
From Figure 2.3.1, the two signals are separated by the distance d12 = 2 Eb .
By substituting Eb = d122 4 in (2.3.5), we obtain
(2.3.6)
⎛ d
2 ⎞
pb = Q ⎜ 12
⎟
⎜ 2 N0 ⎟
⎝ ⎠ 2-67
Evaluation of the Error Probability for Binary Orthogonal Signals
Binary PPM is considered as an example of binary orthogonal signaling.

The signal vectors s1 and s2 are two-dimensional, as shown in Figure 2.3.3 ,
s2 2 Eb
s1
Figure 2.3.3 Signal points for binary orthogonal signals
and may be expressed, according to (2.1.29), as
s1 = ( E , 0)
b
= ( 0, E )
(2.3.7)
s2 b
Where Eb denotes the energy for each of the signals.
2-68
To evaluate the probability of error, assume that s1 was transmitted.

Then, the received vector at the output of the demodulator is
r = ⎡⎣ Eb + n1 , n2 ⎤⎦ (2.3.8)
We can now substitute for r into the correlation metrics given by (2.2.42) to
obtain C(r,s1) and C(r,s2).
Then the probability of error is the probability that C(r,s2)>C(r,s1). Thus
P ( e s1 ) = P ⎡⎣C ( r,s 2 ) > C ( r,s1 ) ⎤⎦ = P ⎡⎣ n2 − n1 > Eb ⎤⎦ (2.3.9)
Since n1 and n2 are zero mean statistically independent Gaussian random

variables each with variance N0/2, the random variable x = n2-n1 is zero mean
Gaussian with variance N0.
( ) 1 ∞
∫
−x 2N 2
P n2 − n1 > Eb = e dx0
2π N 0 E b
1 ∞ ⎛ Eb ⎞
= ∫ e − x 2 dx = Q ⎜⎜
2
⎟⎟ (2.3.10)
2π Eb N0
⎝ N 0 ⎠
2-69
Proof (2.3.9) C (r, sm ) = 2r.sm − sm

2
C (r, s 2 ) = 2r.s 2 − s 2
2
C (r, s1 ) = 2r.s1 − s1
2
P ⎡⎣C ( r, s 2 ) > C ( r, s1 ) ⎤⎦ = P ⎡⎣C ( r, s 2 ) − C ( r, s1 ) > 0 ⎤⎦
= P ⎡ 2r.s 2 − s 2 − 2r.s1 + s1 > 0 ⎤

2 2
⎣ ⎦
= P ⎡⎣ 2r. ( s 2 - s1 ) > 0 ⎤⎦
r = ⎡⎣ Eb + n1 , n2 ⎤⎦
s1 = ⎡⎣ Eb , 0 ⎤⎦ , s 2 = ⎡⎣ 0, Eb ⎤⎦ ⇒ ( s 2 − s1 ) = ⎡⎣ − Eb , Eb ⎤⎦
( )(
P ⎡⎣C ( r, s 2 ) > C ( r, s1 ) ⎤⎦ = P ⎡ 2 Eb + 2n1 , 2n2 . − Eb , Eb > 0 ⎤
⎣ ⎦ )
= P ⎡⎣ −2 Eb − 2n1 Eb + 2n2 Eb > 0 ⎤⎦
= P ⎡⎣( n2 - n1 ) Eb > Eb ⎤⎦
= P ⎡⎣( n2 - n1 ) > Eb ⎤⎦
2-70
Proof Variance of x=N0

N0 N
σ n2 = , σ n22 = 0
1
2 2
x = n2 − n1 ⇒ x(t ) = n2 (t ) − n1 (t )
⎡1 T T ⎤
σ = E ⎢ ∫ ∫ n(t )n(τ )dtdτ ⎥
2
n
⎣T 0 0 ⎦
⎡1 T T ⎤
σ = E ⎢ ∫ ∫ x(t ) x(τ )dtdτ ⎥
2
x
⎣T 0 0 ⎦
= E ∫ ∫ ( n2 (t ) − n1 (t ) )( n2 (τ ) − n1 (τ ) ) dtdτ ⎤
1 ⎡ T T
T ⎢⎣ 0 0 N0/2
⎥⎦
N0/2
0 0
= E ⎡ ∫ ∫ ( n2 (t )n2 (τ ) − n1 (t )n2 (τ ) − n2 (t )n1 (τ ) + n1 (t ) n1 (τ ) ) dtdτ ⎤
1 T T
T ⎢⎣ 0 0 ⎥⎦
N N
= 0+ 0
2 2
= N0
2-71
Due to the symmetry, the same error probability is obtained when we

assume that s2 is transmitted.
Consequently, the average error probability for binary orthogonal signals is
⎛ Eb ⎞
Pb = Q ⎜⎜ ⎟⎟ (2.3.11)
⎝ N 0 ⎠
Conclusions:
- At any given error probability, the
Eb/N0 required for orthogonal signals
is 3 dB more than that for antipodal
signals.
-The difference of 3 dB is simply due to
the distance between the two signal
points, which is d12 = 2 Eb for
2
orthogonal signals, whereas d12 = 4 Eb

2
for antipodal signals.
Figure 2.3.4 Probability of error for

binary signals
2-72
2.3.2 Probability of Error for M-ary Modulation

2.3.2.1 M-ary PAM signals
Recall that M-ary PAM signals are represented geometrically as M one-
dimensional signal points with values
sm = Eg Am , m = 1, 2,… , M (2.3.12)
Where Eg is the energy of the basic signal pulse gT (t ) .
The Amplitude: Am = ( 2m − 1 − M ) , m = 1, 2,… , M (2.3.13)
Where the distance between adjacent signal points is 2 E g .

The Average Energy: 1 M Eg M
Eav = ∑ Em = ∑ ( 2m − 1 − M )
2
M m =1 M m =1
Eg M ( M − 1)
2
⎛ M 2 −1 ⎞
= =⎜ ⎟ Eg (2.3.14)
M 3 ⎝ 3 ⎠
Where Em denotes the energy of the PAM signals.
The Average Power: Eav ⎛ M 2 − 1 ⎞ Eg

Pav = =⎜ ⎟ (2.3.15)
T ⎝ 3 ⎠T
2-73
The average probability of error for M-ary PAM can be determined from the
decision rule that maximizes the correlation metrics given by (2.2.42).
Equivalently, the detector compares the demodulator output r with a set of
M-1 thresholds, which are placed at the midpoints of successive amplitude
levels as shown in Figure 2.3.5.
si si +1 si + 2 si + 3 si + 4 si + 5
τi τ i +1 τ i+2 τ i +3 τ i+4
si - Signal point 2 Eg
τ i - Thresholds
Figure 2.3.5 Placement of Thresholds at midpoints of successive amplitude levels
If the mth amplitude level is transmitted, the demodulator output is
r = sm + n = Eg Am + n (2.3.16)
Where the noise variable n has zero-mean and varianceσ n = N 0 2.

2
2-74
On the basis that all amplitude levels are equally likely a priori, the average
probability of a symbol error is simply the probability that the noise variable n
exceeds in magnitude one-half of the distance between levels.
However, when either one of the two outside levels ± ( M − 1) is transmitted, an
error can occur in one direction only.
Thus, we have
M −1
PM =
M
(
P r − sm > Eg ) xold
M −1 2 ∞
∫
− x2 N0
= e dx
M π N0 g E
xnew = 2 N 0 xold
Set
M −1 2 ∞
∫
− x2 2
= e dx
M 2π 2 Eg N 0
xnew
2 ( M − 1) ⎛ 2 Eg ⎞
= Q⎜ ⎟ (2.3.17)
M ⎜ N0 ⎟
⎝ ⎠
3
From (2.3.15) we note that E g = PavT (2.3.18)
M −1 2
2-75
By substituting Eg of (2.3.18) into (2.3.17) , we obtain the average probability

of a symbol error for PAM in terms of the average power as
2 ( M − 1) ⎛ 6 PavT ⎞
PM = Q⎜ ⎟ (2.3.19)
M ⎜
⎝ ( M − 1) N 0 ⎟⎠
2
Or equivalently,
2 ( M − 1) ⎛ 6 Eav ⎞
PM = Q⎜ ⎟
M ⎜
⎝ ( )
M − 1 N0 ⎟
2
⎠
(2.3.20)
Where Eav=PavT is the average energy.
Since T=kTb and k=log2M, (2.3.20) may be expressed as
2 ( M − 1) ⎛ 6 ( log 2 M ) Ebav ⎞ (2.3.21)

PM = Q⎜ ⎟
M ⎜ ( M 2 − 1) N 0 ⎟
⎝ ⎠
Eav = PavT = Pav kTb
Where Ebav=PavTb is the average bit energy and = ( log 2 M ) PavTb
Ebav/N0 is the average SNR per bit.
= ( log 2 M ) Ebav
2-76
Conclusions:
The SNR/bit increases by over 4 dB for

every factor of two increment in M.
For large M, the additional SNR/bit

required to increase M by a factor of
two approaches 6 dB.
Note: the case M=2 corresponds to

the error probability for binary
antipodal signals.
Figure 2.3.6 Probability of a symbol error for PAM

2-77
2.3.2.2 M-ary Orthogonal Signals

For equal energy orthogonal signals, the optimum detector selects the signal
resulting in the largest crosscorrelation between the received vector r and
each of the M possible transmitted signal vectors sm , i.e., { }
M
C (r,sm ) = r.sm = ∑ rk smk , m = 1, 2,… , M (2.3.22)
k =1
To evaluate the probability of error, suppose that the signal s1 is transmitted.
Then the received signal vector is
r= ( Es + n1 , n2 , n3 ,… , nM ) (2.3.23)
Where r1,r2,…,rM are zero-mean, mutually statistically independent Gaussian

random variables with equal varianceσ n = N 0 2 .
2
In this case, the outputs from the bank of M correlators are

C ( r,s1 ) = Es ( Es + n1 )
C ( r,s 2 ) = Es n2
C ( r,s M ) = Es nM (2.3.24)
2-78
Note: the scale factor Es may be eliminated from the correlator outputs by
dividing each output by Es .
Then, with this normalization, the PDF of the first correlator output r1 = Es + n1 ( )
is 1 −( x1 − Es ) N 0
2
f r1 ( x1 ) = e (2.3.25)
π N0
and the PDF of the other M-1 correlator outputs are
1
f rm ( xm ) = e − xm
2
N0
, m = 2,3,… , M (2.3.26)
π N0
It is mathematically convenient to first derive the probability that the detector
makes a correct decision.
This is the probability that r1 is larger than each of the other M-1 correlator
outputs n2,n3,…,nM. This probability may be expressed as
Pc = ∫ P ( n2 < r1 , n3 < r1 ,… , nM < r1 r1 ) f ( r1 ) dr1

∞
−∞ (2.3.27)
Where P ( n2 < r1 , n3 < r1 ,… , nM < r1 r1 ) denotes the joint probability that
n2,n3,…,nM are all less than r1, conditioned on any given r1.
Then this joint probability is averaged over all r1.
2-79
Since the{rm } are statistically independent, the joint probability factors into a
product of M-1 marginal probabilities of the form
P ( nm < r1 r1 ) = ∫ f rm ( xm ) dxm , m = 2,3,… , M
r1
−∞
1 2 N 0 r1 ⎛ 2 ⎞
∫
− x2 2
= e dx = Q ⎜⎜ − r1 ⎟⎟ (2.3.28)
2π −∞
⎝ N0 ⎠
These probabilities are identical for m=2,3,…,M and hence, the joint probability
under consideration is simply the result in (2.3.28) raised to the (M-1) power.
Thus, the probability of a correct decision is
M −1
∞ ⎡ ⎛ 2 ⎞⎤
Pc = ∫ ⎢Q ⎜⎜ − r1 ⎟⎟ ⎥ f ( r1 ) dr1 (2.3.29)
−∞
⎢⎣ ⎝ N 0 ⎠ ⎥⎦
and the probability of a (k-bit) symbol error is PM = 1 − Pc
(2.3.30)
⎧ ⎡ ⎛ M −1
⎫
1 ∞ ⎪ x ⎞ ⎤ ⎪ −( x − )
2
∫−∞ ⎨1 − ⎢⎢Q ⎜⎜
2 Es N 0 2
where PM = ⎟⎟ ⎥ ⎬ e dx (2.3.31)
2π ⎪⎩ ⎣ ⎝ N 0 ⎠ ⎦⎥ ⎪
⎭
The same expression for the probability of error is obtained when any one of the
other M-1 signals is transmitted.
2-80
Since all the M signals are equally likely, the expression for PM given in (2.3.31)
is the average probability of a symbol error.
This expression can be evaluated numerically.
In comparing the performance of various digital modulation methods, it is
desirable to have the probability of error expressed in terms of the SNR per bit,
Eb/N0, in stead of the SNR per symbol Es/N0.
With M=2k, each symbol conveys k bits of information, and hence, Es=kEb.
Thus, (2.3.31) may be expressed in terms of Eb/N0 by substituting for Es.
For equi-probable orthogonal signals, all symbol errors are equi-probable and
occur with probability PM P
= kM (2.3.32)
⎛k ⎞
M −1 2 −1
Furthermore, there are ⎜ n ⎟ ways in which n bits out of k may be in error. Hence,
⎝ ⎠
the average number of bit errors per k-bit symbol is
k
⎛ k ⎞ PM 2k −1 (2.3.33)
∑ ⎜ ⎟ k
n =1 ⎝ n ⎠ 2 − 1
=k k
2 −1
PM
And the average bit error probability is just the result in (2.3.33) divided by k,
the number of bits per symbol. Thus
2k −1 P
Pb = k PM ≈ M , k 1 (2.3.34)
2 −1 2
2-81
Conclusions:
This figure illustrates that by increasing
the number M of signals, one can
reduce the SNR/bit required to achieve
a given probability of a bit error.
For example, to achieve a Pb=10-5, the

required SNR/bit is a little more than 12
dB for M=2, but if M is increased to 64
signals (k=6 bits/symbol), the required
SNR/bit is approximately 6 dB.
Thus, a saving of over 6 dB (a factor of

four reduction) is realized in transmitter
power (or energy) required to achieve a
Pb=10-5 by increasing M from M=2 to
M=64.
What is the minimum required Eb/N0
to achieve an arbitrary small probability
Figure 2.3.7 Probability of bit error for
of error as M → ∞ ?
coherent detection of orthogonal signals
2-82
A Union Bound on the Probability of Error

Let us investigate the effect of increasing M on the probability of error for
orthogonal signals.
To simplify the mathematical development, we first derive an upper bound on
the probability of a symbol error which is much simpler than the exact form given
in (2.3.31).
Recall that the probability of error for binary orthogonal signals is given by
(2.3.11).
Now, if we view the detector for M orthogonal signals as one that makes M-1
binary decisions between the correlator output C(r,s1) that contains the signal
and the other M-1 correlator outputs C(r,sm),m=2,3,…,M, the probability of
error is upper bounded by the union bound of the M-1 events.
That is, if Ei represents the event that C(r,sm)>C(r,s1) for m ≠ 1 , then we have
PM = P ( ∪ Ei ) ≤ ∑ P ( Ei )
n
n
i =1
Hence, i =1
PM ≤ ( M − 1) P2 = ( M − 1) Q ( )
Es N 0 < MQ ( Es N 0 ) (2.3.35)
P2 Probability for orthogonal signal

2-83
This bound can be simplified further by upper-bounding Q ( )

Es N 0 .
We have
( )
Q Es N 0 < e − Es 2 N0 (2.3.36)
Thus,
PM < Me − Es 2 N0
= 2 k e − kEb 2 N0
− k ( Eb N0 − 2ln 2 ) 2 (2.3.37)
PM < e
As k → ∞ , or equivalently, as M → ∞ , the probability of error approaches
zero exponential, provided that Eb N 0 is greater than 2 ln 2 ,i.e.,
Eb (2.3.38)
> 2 ln 2 = 1.39 (1.42dB )
N0
The simple upper bound on the probability of error given by (2.3.37) implies
that as long as SNR>1.42 dB, we can achieve an arbitrary low PM.
However, this union bound is not a very tight upperbound at a sufficiently low
SNR due to the fact that the upper bound for the Q-function in (2.3.36) is
loose.
2-84
In fact, by more elaborate bounding techniques, it is shown that the upper

bound in (2.3.37) is sufficiently tight for Eb N 0 < 4 ln 2 .
For Eb N 0 < 4 ln 2 , a tighter upper bound on PM is
( )
2
−k Eb N 0 − ln 2 (2.3.39)
PM < 2e
Consequently, PM → 0 as k → ∞ , provided that
Eb
> ln 2 = 0.693 (−1.6dB ) (2.3.40)
N0
Hence, -1.6 dB is the minimum required SNR/bit to achieve an arbitrary small

probability of error in the limit as k → ∞ ( M → ∞ ) .
This minimum SNR/bit (-1.6 dB) is called the Shannon limit for an Additive
White Gaussian Noise channel.
2-85
2.3.2.3 Biorthogonal Signals

A set of M=2k biorthogonal signals are constructed from M/2 orthogonal signals
by including the negatives of the orthogonal signals.
Thus, we achieve a reduction in the complexity of the demodulator for the
biorthogonal signals relative to that for the orthogonal signals, since the former
is implemented with M/2 crosscorrelators or matched filters, whereas the latter
requires M matched filters or crosscorrelators.
To evaluate the probability of error for the optimum detector, assume that the
( )
signal s1(t) corresponding to the vector s1 = Es + n1 , n2 ,… , nM 2 was
transmitted. Then, the received signal vector is
(
r = E + n , n ,… , n
s 1 2 M 2 ) (2.3.41)
Where the {nm } are zero-mean, mutually statistically independent and identically
distributed Gaussian random variables with variance σ n2 = N 0 2.
The optimum detector decides in favor of the signal corresponding to the
largest in magnitude of the crosscorrelators
M 2
C ( r,sm ) = r.sm = ∑ rk smk , m = 1, 2,… , M 2 (2.3.42)
k =1
While the sign of this largest term is used to decide whether sm(t) or –sm(t) was
transmitted.
2-86
According to this decision rule, the probability of a correct decision is equal to the
probability that r1 = Es + n1 > 0 and r1 exceeds rm = nm for m = 2,3,… , M 2 .
But,
P ( nm < r1 r1 > 0 ) =
1 r1 1 r1 N0 2
∫ ∫
− x2 N0 − x2 2
e dx = e dx (2.3.43)
π N0 − r1
2π − r1 N0 2
Then, the probability of a correct decision is

M
−1
∞⎡ 1 ⎤
f ( r1 ) dr1
r1 N0 2 2
Pc = ∫ ⎢ ∫ e − x 2 dx ⎥
2
0
⎣ 2π − r1 N0 2
⎦
Which, upon substitution for f (r1), we obtain

M
−1
∞ ⎡ 1 ( v + 2 Es N 0 ) − x 2 2 ⎤ 2
Pc = ∫ ∫ e dx ⎥ e − v2 2
dv (2.3.44)
N0 ⎢ − ( v + 2 Es N 0 )
− 2 Es
⎣ 2π ⎦
Where we have used the PDF of r1 given in (2.3.25).
Finally, the probability of a symbol error PM=1-Pc.
2-87
Conclusions:
We observe that this graph is similar to

that for orthogonal signal.
However, in this case, the probability of
error for M=4 is greater than that for
M=2.
This is due to the fact that we have
plotted the signal error probability PM
in Figure 2.3.8.
If we plot the equivalent bit error
probability we would find that the -
graphs for M=2 and M=4 coincide.
As in the case of orthogonal signals, as
M → ∞ (or k → ∞ ) , the minimum
required Eb/N0 to achieve an arbitrary
small probability or error is -1.6 dB, the
Shannon limit.
Figure 2.3.8 Probability of symbol error
for biorthogonal signals
2-88
2.3.2.4 Simplex Signals
Simplex signals are a set of M equally correlated signals with mutually

crosscorrelation coefficient γ mn = −1 ( M − 1) .
These signals have the same minimum separation of 2 Es between adjacent

signal points in M-dimensional space as orthogonal signals.
They achieve this mutual separation with a transmitted energy of Es ( M − 1) M

which is less than that required for orthogonal signals by a factor of ( M − 1) M
Consequently, the probability of error for simplex signals is identical to the

probability of error for orthogonal signals, but this performance is achieved with
a savings of
M
10 log (1 − γ mn ) = 10 log dB (2.3.45)
M −1
in SNR.
2-89
2.3.3 Comparison of Modulation Methods

The digital modulation methods can be compared in many ways;
(1) On the basis of the SNR required to achieve a specified probability of error.
- This would not be very meaningful, unless it were made on the basis of some
constraint, such as fixed data rate of transmission.
Suppose that the bit rate Rb is fixed and let us consider the channel bandwidth
required to transmit the various signals.
M-ary PAM
The symbol interval is T=k/Rb, where M=2k.

A signal pulse of duration T has a spectral characteristic of width 1/2T
(approximately).
The channel bandwidth required to transmit the M-ary PAM signal is
W = Rb 2k = Rb 2 log 2 M Hz (2.3.50)
2-90
PPM
The symbol interval T is subdivided into M subintervals of duration and pulses
of width T/M are transmitted in the corresponding subintervals.
The spectrum of each pulse is approximately M/2T wide.
The channel bandwidth required to transmit the PPM signals is
W = M 2T = M 2 ( k Rb ) = MRb 2 log 2 M Hz (2.3.51)
Biorthogonal and Simplex Signals
These signals result in similar relationships as PPM (orthogonal).
In the case of biorthogonal signals, the required bandwidth is one-half of that
for orthogonal signals.
(2) Based on the normalized data rate Rb/W (bits/second per hertz of
bandwidth) versus the SNR per bit (Eb/N0) required to achieve a given error
probability.
For PAM and orthogonal signals, we have
Rb
PAM: = 2 log 2 M (2.3.52)
W
Rb 2 log 2 M
PPM (orthogonal): = (2.3.53)
W M
2-91
Figure 2.3.9 Comparison of several modulation methods at 10-5

symbol error probability
2-92
Conclusions:
In the case of PAM, increasing the number of amplitudes M results in a
higher bit rate to bandwidth ratio Rb/W.
However, the cost of achieving the higher data rate is an increment in the
SNR/bit.
Consequently, M-ary PAM modulation is appropriate for communication

channels that are bandwidth limited, where we desire a bit rate-to-bandwidth
ratio Rb/W>1 and where there is sufficiently high SNR to support multiple
amplitude levels.
In contrast, M-ary orthogonal signals yield a bit rate-to-bandwidth ratio of

Rb W ≤1
As M increases, Rb/W decreases due to an increment in the required channel
bandwidth.
However, the SNR/bit required to achieve a given error probability (in this
case, PM=10-5) decreases as M increases.
2-93
Conclusions:
Consequently, M-ary orthogonal signals are appropriate for power-limited
channels that have sufficiently large bandwidth to accommodate a large
number of signals.
In this case, as M → ∞, the error probability can be made as small as

desired, provided that Eb/N0>0.693 (-1.6 dB).
This is the minimum SNR/bit required to achieve reliable transmission in the

limit as the channel bandwidth W → ∞ and the corresponding bit rate-to-
bandwidth ratio Rb W → 0.
2-94
Modulation,
A property ofDemodulation
MVG_OMALLOOR and Coding
Modulation, Demodulation and

Coding Course
Period 3 - 2005
Sorour Falahati
Lecture 1
Course information
Scope of the course
Course material
Schedule
Staff
Grading
More information on:

http://www.signal.uu.se/Courses/CourseDirs/ModDemKod/2005/main.html
2005-01-21 Lecture 1 2
Scope of the course

Learning fundamental issues in designing a
digital communication system (DCS):
Utilized techniques
Formatting and source coding
Modulation (Baseband and bandpass signaling)
Channel coding
Equalization
Synchronization
Design goals
Trade-offs between various parameters
2005-01-21 Lecture 1 3
Modulation,
Course material
Course text book:
“Digital communications: Fundamentals and Applications”
by Bernard Sklar,Prentice Hall, 2001,ISBN: 0-13-084788-7
Additional recommended books:
“Communication systems engineering”, by John G. Proakis
and Masoud Salehi, Prentice Hall, 2002, 2nd edition, ISBN:
0-13-095007-6
“Communication Systems, Analysis and design”, by
H.P.E.Stern and S. A. Mahmoud, Prentice Hall, 2004,
ISBN: 0-13-121929-4
Material accessible from course homepage:
Lecture slides
Laboratory syllabus (Lab. PM)
Set of exercises and formulae
Home assignments and solutions
2005-01-21 Lecture 1 4
Schedule
12 lectures (from week 3 to week 8)
10 tutorials (from week 4 to week 8)
4 mandatory graded home assignments
1 mandatory lab. work (weeks 8-9)
Final written exam 14th March 2005
2005-01-21 Lecture 1 5
Staff
Course responsible and lecturer: Sorour
Falahati.
Email: sorour.falahati@signal.uu.se
Office: Magistern 2112A
Tel.: 018-471 1077
Tutorial and laboratory assistant: Daniel
Aronsson.
Email: daniel.anorsson@signal.uu.se
Office: Magistern 2140B
Tel.: 018-471 3071
2005-01-21 Lecture 1 6
Modulation,
Grading
To obtain grade 3, a student has to:
1. To complete the laboratory work
2. To pass all the home assignments (HA)
3. To pass the written final exam
The final grade (3,4,5) is calculated as
Final grade:
0.8(grade on final exam)+0.2(average grade on HAs)
Exam and home assignments have each 60 points.
0-29 30-39 40-49 50-60
Fail Grade 3 Grade 4 Grade 5
2005-01-21 Lecture 1 7
Today, we are going to talk about:

What are the features of a Digital
communication system (DCS)?
Why “digital” instead of “analog”?
What do we need to know before taking off
toward designing a DCS?
Classification of signals
Random process
Autocorrelation
Power and energy spectral densities
Noise in communication systems
Signal transmission through linear systems
Bandwidth of signal
2005-01-21 Lecture 1 8
Block diagram of a digital

communication system
Noise
Transmitted Received Received

Info. signal signal info.
SOURCE
Source Transmitter Channel Receiver User
Transmitter
Source Channel
Formatter Modulator
encoder encoder
Receiver
Source Channel
Formatter Demodulator
decoder decoder
2005-01-21 Lecture 1 9
Modulation,
Digital communication system …

Important features of a DCS:
Transmitter sends a waveform from a finite
set of possible waveforms during a limited
time
Channel distorts, attenuates the transmitted
signal and adds noise to it.
Receiver decides which waveform was
transmitted from the noisy received signal
Probability of erroneous decision is an
important measure for the system
performance
2005-01-21 Lecture 1 10
Digital versus analog

Advantages of digital communications:
Regenerator receiver
Original Regenerated
pulse pulse
Propagation distance
Different kinds of digital signal are treated

identically.
Voice
Data A bit is a bit!
Media
2005-01-21 Lecture 1 11
Classification of signals
Deterministic and random signals
Deterministic signal: No uncertainty with
respect to the signal value at any time.
Random signal: Some degree of uncertainty
in signal values before it actually occurs.
Thermal noise in electronic circuits due to the
random movement of electrons
Reflection of radio waves from different layers of
ionosphere
2005-01-21 Lecture 1 12
Modulation,
Classification of signals …
Periodic and non-periodic signals
A periodic signal A non-periodic signal
Analog and discrete signals
A discrete signal
Analog signals
2005-01-21 Lecture 1 13
Classification of signals ..
Energy and power signals
A signal is an energy signal if, and only if, it has
nonzero but finite energy for all time:
A signal is a power signal if, and only if, it has

finite but nonzero power for all time:
General rule: Periodic and random signals are power

signals. Signals that are both deterministic and non-
periodic are energy signals.
2005-01-21 Lecture 1 14
Random process
A random process is a collection of time functions, or
signals, corresponding to various outcomes of a
random experiment. For each outcome, there exists a
deterministic function, which is called a sample
function or a realization.
Random
variables
Real number
Sample functions
or realizations
(deterministic
function)
time (t)
2005-01-21 Lecture 1 15
Modulation,
Random process …
Strictly stationary: If none of the statistics of the random process
are affected by a shift in the time origin.
Wide sense stationary (WSS): If the mean and autocorrelation

function do not change with a shift in the origin time.
Cyclostationary: If the mean and autocorrelation function are

periodic in time.
Ergodic process: A random process is ergodic in mean and

autocorrelation, if
and
, respectively.
2005-01-21 Lecture 1 16
Autocorrelation
Autocorrelation of an energy signal
Autocorrelation of a power signal
For a periodic signal:
Autocorrelation of a random signal
For a WSS process:
2005-01-21 Lecture 1 17
Spectral density
Energy signals:
Energy spectral density (ESD):
Power signals:
Power spectral density (PSD):
Random process:
Power spectral density (PSD):
2005-01-21 Lecture 1 18
Modulation,
Properties of an autocorrelation
function
For real-valued (and WSS in case of
random signals):
1. Autocorrelation and spectral density form
a Fourier transform pair.
2. Autocorrelation is symmetric around zero.
3. Its maximum value occurs at the origin.
4. Its value at the origin is equal to the
average power or energy.
2005-01-21 Lecture 1 19
Noise in communication systems
Thermal noise is described by a zero-mean Gaussian

random process, n(t).
Its PSD is flat, hence, it is called white noise.
[w/Hz]
Power spectral
density
Autocorrelation
function
Probability density function
2005-01-21 Lecture 1 20
Signal transmission through

linear systems
Input Output
Linear system
Deterministic signals:
Random signals:
Ideal distortionless transmission:

All the frequency components of the signal not only arrive
with an identical time delay, but also are amplified or
attenuated equally.
2005-01-21 Lecture 1 21
Modulation,
Signal transmission … - cont’d
Ideal filters:
Non-causal!
Low-pass
Band-pass High-pass
Realizable filters:
RC filters Butterworth filter
2005-01-21 Lecture 1 22
Bandwidth of signal
Baseband versus bandpass:
Baseband Bandpass
signal signal
Local oscillator
Bandwidth dilemma:
Bandlimited signals are not realizable!
Realizable signals have infinite bandwidth!
2005-01-21 Lecture 1 23
Bandwidth of signal – cont’d

Different definition of bandwidth:
a) Half-power bandwidth d) Fractional power containment bandwidth
b) Noise equivalent bandwidth e) Bounded power spectral density
c) Null-to-null bandwidth f) Absolute bandwidth
(a)
(b)
(c)
(d)
(e)50dB
2005-01-21 Lecture 1 24
TAPR Error Control Coding Seminar
Phil Karn
KA9Q
March 1995
Copyright 1995 Phil Karn
KA9Q FEC - 1
Overview of Seminar
This seminar will:
– Present the theoretical limits to communication over noisy channels

– Give you a feel for what coding can and cannot do
– Describe the basic kinds of codes and the more important examples of
each kind along with their applications
This seminar will not:
Make you an expert. Dozens of textbooks and thousands of journal articles

and PhD theses have been written on this subject!
Disclaimer: I'm not an expert either. But teaching is a good way to learn a
subject...
KA9Q FEC - 2
Limits to Communication
The fundamental limits on communication over noisy channels were

published by Claude Shannon in 1948 in the classic paper, "A
Mathematical Theory of Communication" (Bell System Technical
Journal, July 1948)
Shannon showed that error-free communication is possible on a noisy
channel provided that the data rate is less than the channel capacity
Channel capacity depends on the bandwidth and the S/N ratio
KA9Q FEC - 3
Channel Capacity Formula
Channel capacity is given by the formula:
S
C = B log 2 (1 + )
N
where:
C = channel capacity, bits/sec
B = channel bandwidth, Hz
S = signal power, W
N = noise power, W
KA9Q FEC - 4
Comments on Shannon
Random data bits are assumed. If not, compress them. This is the subject
of another branch of information theory called source coding (not the
subject of this seminar).
The formula applies only to the additive white gaussian noise (AWGN)
channel. Fading, interference, distortion, etc are not considered
Channel capacity is a theoretical limit only; it describes the best that can
possibly be done with any code and modulation method. Beating this limit
is like building a perpetual motion machine
You can send data faster than capacity, but it won't be error free. If you add
an error-correcting code to deal with the errors, the corrected data rate will
always be less than the channel capacity, but you may still be ahead of
where you started
Shannon does not actually show how to build a practical system that
reaches the limit. This is what everybody has been working on since.
KA9Q FEC - 5
Another Look at Capacity
In the standard form of the Shannon equation

S
C = B log 2(1 + )
N
the noise power is
N = N 0B
where B is the bandwidth and N0 is the noise spectral density in watts/Hz.
Doubling the bandwidth doesn't quite double the capacity because the
noise also doubles. So what's the net effect?
KA9Q FEC - 6
Capacity Restated
Let's define
Eb = energy per bit, joules (watt-seconds)

R = relative data rate, bits/sec/Hz (so-called "spectral efficiency")
Then after some (omitted) algebra, we find that for a system to obey the
Shannon limit, it must satisfy the inequality
Eb 2R −1
≥
N0 R
The ratio Eb/N0, pronounced "ebb-know", describes the overall power
efficiency of the system. It is usually expressed in dB.
Large R (many bits/sec/Hz) requires a large Eb/N0; conversely, small R

allows a small Eb/N0. I.e.,
.Bandwidth can be traded off against power-
KA9Q FEC - 7
Minimum Eb/N0 vs Bandwidth
Eb (dB)
20
N0
15
10
spectral density (b/s/Hz)

0
0.0 1 0.1 1 10
Limit at zero: -1.6dB -5
"power "bandwidth
limited" limited"
KA9Q FEC - 8
Potential Coding Gains
Eb (dB)
20
N0
15
Ideal uncoded BPSK 5.9x

●
(BER=10-5) ●
10
QPSK
5
10.8dB
0.0 1 0.1 1 10
-5
KA9Q FEC - 9
What is Error Control Coding?
Error Control Coding is the addition of carefully designed redundancy to a

data stream before modulation to help the receiver detect and/or correct
errors introduced by the transmission process
When used for error correction, this is called Forward Error Correction (FEC)
The redundant information is computed according to an algorithm that is

known to the receiver ahead of time
The encoded information consists of symbols, as distinguished from the

original data bits
In some systems, ECC and modulation/demodulation are very closely

intertwined
KA9Q FEC - 10
Why Use Coding?
Reduce power requirements on power-limited channels,

e.g., radio links (emphasis of this seminar)
Besides the obvious benefits of lower power (cheaper RF hardware,

longer battery life, reduced biohazards, etc), interference resistance is
increased. This is important on shared channels
Reduce bandwidth on bandwidth-limited channels,

e.g., telephone circuits
Shape signal spectrum
Usually to remove low frequency components (Manchester coding on

Ethernet, EFM on Compact Disc)
KA9Q FEC - 11
Power-Limited Coding
Power-reduction coding generally uses extra bandwidth.

Example:
–Uncoded QPSK requires Eb/N0 =10dB for a bit error rate of about 10-5.
–QPSK with rate 1/2, K=7 Viterbi-decoded convolutional coding requires
Eb/N0 =4dB for the same BER, only 25% as much power for the same data
rate.
But there's a catch: twice as much bandwidth is required
KA9Q FEC - 12
Bandwidth-Limited Coding
The bandwidth "penalty" of coding can be relieved by combining it with

higher-level modulation schemes (more bits/sec/Hz).
This may seem counterproductive, but it works. Done right, the S/N increase
required by the higher-level modulation method is more than compensated
for by the S/N decrease allowed by coding.
The most common example is trellis coding, a generalization of
convolutional coding, which is extremely popular in telephone modems
(V.32, V.32bis, V.34)
KA9Q FEC - 13
Coding as Spread Spectrum

Coding for a power-limited channel is really a form of spread spectrum:
P=SB
spectral density S
uncoded signal
f
r=1/2 K=7
coded signal P=S/4 spectral density=S/8
f
2B
KA9Q FEC - 14
ECC Types
There are two major types of error control codes:
Block codes
CRC
Orthogonal (Reed-Mueller, etc)
Hamming
Golay
BCH
Reed-Solomon
etc
Convolutional codes
Sequentially decoded
Viterbi decoded
KA9Q FEC - 15
Block Coding
k input bits
(n,k) block coder
channel errors n channel symbols
(n,k) block decoder
k output bits erasure indicate
KA9Q FEC - 16
Error Detection Coding
The lowly CRC is a block code used for error detection only. I.e., the
decoder produces the k output bits only if there are no channel errors. If
even a single error occurs, the decoder signals an erasure. A higher level
ARQ (Automatic Request-Repeat) then retransmits the block
More sophisticated codes can correct some number of errors. Only if the
number of errors exceeds the code's correction ability will it signal an
erasure
KA9Q FEC - 17
Systematic Codes
Most block codes are systematic. That is, the k input bits appear as-is in the
n output symbols. The additional n-k symbols are parity symbols
In a nonsystematic code, every output symbol is a function of more than
one input data bit, i.e., the input does not appear imbedded in the output
Nonsystematic linear block codes have no particular advantage over
systematic codes, so systematic block codes are generally used in
practice. This lets decoders cut corners (at a price in lost performance)
KA9Q FEC - 18
Hamming Distance
The Hamming distance (or just "distance") between two equal-length bit
strings is the number of places in which they disagree.
For example, these strings have distance 2:
001101001
100101001
KA9Q FEC - 19
Distance Properties
A (n,k) block code has 2k possible inputs, so only 2k of the possible 2n

encoded output blocks (codewords) are valid.
The code designer tries to maximize the distance between every pair of
valid codewords as this maximizes the number of errors that can be
detected and/or corrected. This implies a fairly uniform distribution of
codewords
A code used only for error detection declares an erasure if the received
codeword isn't valid
A code used for error correction finds the valid codeword nearest the one
that was received and assumes that it was sent. I.e., it assumes that a few
errors are more likely than a lot of errors
KA9Q FEC - 20
Distance Properties - 2
KA9Q FEC - 21
Distance Properties - 3
All other things being equal, bigger code blocks provide better performance for
the same relative overhead - at the expense of added delay
Tradeoffs exist between the erasure rate and the undetected error rate. For
example, the designer might specify that if a received codeword is "too far"
from a valid codeword, an erasure is declared even though the error could
have been corrected. This reduces the undetected error rate but increases the
erasure rate
In most block codes, the distribution of errors in a block is irrelevant; only the
total number of errors in a block matters. This makes large block codes good
for burst errors, e.g., against pulsed interference
Resistance to error bursts can be further increased by interleaving (more later)
KA9Q FEC - 22
Orthogonal Signalling
Signal sets with elements that can be independently detected

without mutual interference are orthogonal
The number of elements in the set, m, is usually a power of 2
This can be seen as a family of block codes of the form (m,log2m),
e.g., (2,1); (4,2); (8,3); etc
Ordinary binary (2-ary) FSK is an orthogonal signal set with 2
elements
Given enough bandwidth, orthogonal signal sets can be
constructed with as many elements as you like. As m increases,
performance for both coherent and noncoherent detection (slowly)
approaches the Shannon limit of Eb/N0= -1.6dB.
Besides sine waves, other orthogonalizing functions include Walsh
functions, the rows of a Hademard matrix. These are Reed-Mueller
codes
KA9Q FEC - 23
Performance
KA9Q FEC - 24
Uses of Orthogonal Signals
Orthogonal codes were among the first to be analyzed according to

Shannon
The big problem with orthogonal signalling is the slow Eb/N0
improvement with bandwidth. But it found use as an error control
code in the first NASA deep space probes to use digital
transmission; a (32,6) biorthogonal code was used on Mariner 6 and
9 at Mars
Orthogonal codes are now used in spread-spectrum systems,
usually in combination with other, more powerful coding (e.g., with
Viterbi-decoded convolutional coding in the Qualcomm CDMA
reverse link)
The benefit of orthogonal coding is its performance with
noncoherent demodulation. This is an important advantage in
severe multipath fading where there is no stable carrier phase
reference (HF and EME modem designers take note)
KA9Q FEC - 25
Some Block Codes

CRC (Cyclic Redundancy Check)
– Extremely popular for error detection
– Adaptable to varying block sizes with a constant number of parity
symbols n-k
– Distance properties poor for large n - size should be limited
Hamming
– One of the first block codes; not particularly strong

– A family of codes with n = 2m - 1, k=n-m; m>2.
E.g., (7,4) (15,11) (31,26) (63,57) (127,120)
– Minimum distance between valid codewords = dmin = 3
– Error correcting ability =(dmin-1)/2 = 1
– Now used mainly to correct errors in RAMs (e.g., Microsat)
KA9Q FEC - 26
Some More Block Codes

Golay
– A (23,12) block code with dmin=7 (corrects 3 errors)
– Often extended to (24,12) with dmin=8
– Fairly easy to decode
– Better than Hamming, but not the best
– Used in Kantronics' G-TOR
BCH
– A large class of codes usually with block length 2m-1 (m>0)
– Can be binary or nonbinary
– Used in AMPS cellular
KA9Q FEC - 27
Reed-Solomon Codes
Reed-Solomon codes are a subclass of non-binary BCH codes. That is,
instead of individual binary digits (bits), R-S codes operate on symbol
alphabets of more than two values. These alphabet sizes are usually powers
of 2 and are written GF(2m)
A R-S code block size is one less than the alphabet size. E.g., a R-S code over
GF(256) will have a block size of 255 symbols or 2040 bits. There is an
"extended" R-S code with a block size equal to the alphabet size (256 symbols
or 2048 bits for our example)
The number of data symbols within a R-S codeword can be chosen according
to the desired error correcting ability. To correct up to E errors in a block, you
need 2E parity symbols, with the rest being data. E.g., if you want to correct
up to 10 errors in a R-S code over GF(256), we will need 20 parity symbols.
This leaves 255-20 = 235 data symbols per block. This results in a (255,235)
code over GF(256)
R-S codes can correct as many erasures (known errors) as parity symbols, ie.,
twice as many erasures as errors
KA9Q FEC - 28
R-S Codes - 2
A Reed-Solomon code achieves the best performance possible for any
linear code with the same block size. They are also relatively easy to
implement at large block sizes (better performance)
This has made them extremely popular; the R-S code is now arguably the
single most important block code used for error correction
Applications of R-S codes include digital audio (Compact Disc, DAT, etc),
deep space communications (Voyager and Galileo, both in combination with
convolutional coding), and the HAL Clover II HF modem
The multi-bit R-S symbol is a natural fit to an m-ary orthogonal modulation

scheme. E.g., a GF(256) R-S coder could send each symbol as one of 256
orthogonal carriers in a 256-ary FSK system designed for HF channels with
severe fading
KA9Q FEC - 29
KA9Q FEC - 30
Interleaving
Interleaving is a technique that allows a designer to construct a big block
code out of a smaller one. I.e., given a (n,k) block code, we can construct a
(L*n,L*k) block code
A common way to do this is to allocate a block of memory as a 2-
dimensional matrix. The encoded symbols of each block are written into
memory in rows. Then they are read out and transmitted in columns. The
receiver reverses the procedure, decoding each row in turn.
transmit order
code block 1 S11 S12 S13 S14 S15 A burst now must
S21 S22 S23 S24 S25 be more than 5
code block 2
symbols long to
code block 3 S31 S32 S33 S34 S35 affect more than
S41 S42 S43 S44 S45 one symbol in
code block 4
each code block.
code block 5 S51 S52 S53 S54 S55
KA9Q FEC - 31
The Compact Disc

The Compact Disc uses two concatenated Reed-Solomon codes over
GF(256) with an interleaver between them
(28,24) (32,28)
Interleaver disc
RS coder RS coder
(32,28) De- (28,24)

disc
decoder interleave decoder
(This is a simplified diagram. The EFM channel coding isn't shown,

and there is additional interleaving before the first coder and after the
second.)
KA9Q FEC - 32
CIRC (complete)
KA9Q FEC - 33
The CD - 2
By itself, the inner (32,28) RS code isn't very strong. More than (32-28)/2 =2
symbol errors to the decoder causes it to fail and flag an erasure on all 28
of its output symbols.
But the deinterleaver distributes these 28 erasures across 28 different
blocks of the outer (28,24) RS code, one erasure per block. And because
these are erasures, not errors, the outer code can correct up to 28-24=4 of
them in a block. So the outer code easily fills in the missing symbols.
The concatenated system is so strong that it can completely correct an
error burst of up to 3,874 bits - assuming no more bursts occur until the
interleavers have flushed!
The two CD codes are examples of "shortened" R-S codes. An 8-bit
symbol size would ordinarily imply a 255-byte block, but the encoder sets
the unused symbols to zero and the decoder takes this into account.
KA9Q FEC - 34
Convolutional Coding
Convolutional codes (also known as tree codes) operate on continuous
streams of data, rather than on fixed size blocks
A convolutional encoder is extremely simple: a shift register, some XOR
gates and a multiplexer:
r=1/2, K=7
NASA Standard code
+
coded
data
symbols
input
out
+
shift
KA9Q FEC - 35
Code Parameters
Convolutional codes are described by two parameters:
K = the constraint length, the length of the shift register+1

r = the “rate” of the code; the number of data bits per encoded
channel symbol. Usually expressed as a rational fraction b/v. In
most common codes, b=1. The most popular rates are 1/2, 1/3 and
1/4.
Coding gain increases directly with K and inversely with r;

diminishing returns are reached
KA9Q FEC - 36
Convolutional Decoding
This is the fun part!

There are two main types of algorithms:
Sequential decoding - complexity insensitive to K, so large
values are generally used (e.g., 32)
Maximum Likelihood (Viterbi) decoding - complexity increases
as 2k , so k>9 is rare (Galileo uses k=14!)
Both are relatively insensitive to r
Unlike block codes, nonsystematic convolutional codes outperform
systematic codes, so nonsystematic codes are more often used
KA9Q FEC - 37
Example: r=1/2, K=3
KA9Q FEC - 38
Code Tree
KA9Q FEC - 39
Sequential Decoding
Sequential decoding looks for the path through the code tree that
most closely matches the received sequence of symbols
Two sequential decoding algorithms are commonly used: Fano and
stack. Though they look quite different, they accomplish the same
thing
The decoder measures the “goodness” of the match. This is called
the metric. As long as it continues to increase, it moves forward
If the metric decreases, the decoder assumes it may have made a
mistake earlier due to noise. It backs up and sequentially examines
alternate paths until it finds one whose metric increases again
The decoder may look at the “quality” of each received symbol and
give it an appropriate weight in the total metric. This is called soft
decision decoding and is a significant advantage of convolutional
codes over block codes (2 dB on AWGN channels). Just a few bits
of precision give almost all of this gain; 3 bit A/D sampling is
common
KA9Q FEC - 40
Sequential Decoding Performance
The performance of a sequential decoder is strongly dependent on

the number and distribution of errors. Burst errors are particularly
nasty as the decoding time is exponential with the burst length. This
makes interleaving almost mandatory
When there are few errors in the symbol stream, sequential
decoding is very fast; it rarely examines more than one branch per n
data bits
But when the error rate is high, the decoder can take an impractially
long time to finish. Practical decoders use timers; if a timeout
occurs, the entire packet is erased
The decoding time of a sequential decoder is therefore a random
variable. If the signal is too weak, the average decoding time goes
to infinity
KA9Q FEC - 41
Example Performance
KA9Q FEC - 42
Viterbi Decoding
Despite its excellent coding gain, the random speed of sequential

decoding is a problem for some real-time applications (e.g., digital
voice, synchronous transmission links)
Sequential decoding, as the name implies, is also not well suited to
parallel implementation in VLSI
An alternative is Maximum-Likelihood Decoding, also known as
Viterbi Decoding
A Viterbi decoder exploits the finite length of the encoder shift
register. Data more than K bits old “falls off” the end of the shift
register and can no longer influence the encoded symbols
Because of the finite shift register, branches in the code tree
eventually remerge, forming a trellis with a number of states equal
to the number of possible shift register states, i.e., 2k-1
KA9Q FEC - 43
Trellis Diagram
Number of trellis states = 2k-1
KA9Q FEC - 44
Decoding Example
KA9Q FEC - 45
Viterbi Decoder Performance
KA9Q FEC - 46
Performance vs k
KA9Q FEC - 47
Sequential vs Viterbi
Sequential decoding allows the use of very long constraint length

codes. This gives high coding gain with a steep “waterfall” curve.
The probability of an uncorrected error is small compared to the
probability that the decoder will time out. And the decoding process
is relatively fast when the signal is not too noisy
These properties makes sequential decoding well suited for medium
speed packet communications, especially when implemented in
software
Viterbi decoding operates at a constant rate, regardless of the

channel error rate. The algorithm is also ideally suited to parallel
implementation in VLSI. This makes it well suited to synchronous,
real-time communications at high speeds
KA9Q FEC - 48
Sequential vs Viterbi
KA9Q FEC - 49
Block vs Convolutional
Block
Practical at very large block sizes for improved performance
against burst errors
Inherently difficult to adapt to soft-decision decoding (Chase
method)
Requires external synchronization
Convolutional
Outperforms block codes at comparable implementation
complexity
Constraint lengths have practical limits. Interleaving to protect
against burst errors
Easily adapted to soft-decision decoding
Viterbi decoders can self-synchronize
KA9Q FEC - 50
Short Block vs Convolutional
KA9Q FEC - 51
Concatenated Coding
The properties of block and convolutional codes are

complementary. This suggests combining the two in order to get
the best of both worlds
For example, Viterbi decoders are ideally suited for the power-
limited AWGN channel (e.g., a deep-space link) because of their
ability to use soft decisions. But the decoder output error rate may
not be low enough to satisfy all applications (e.g., compressed
images).
Furthermore, when a Viterbi decoder makes a mistake, it generally
emits a burst of errors until it recovers. This suggests combining it
with an interleaver and a Reed-Solomon decoder good at correcting
bursts
These concatenated codes have been widespread in deep space
communications since Voyager
KA9Q FEC - 52
Voyager Code Performance
KA9Q FEC - 53
Galileo Coding
The failure of the high-gain antenna on Galileo gave coding a

chance to “pull out the stops” to make the most of the low gain
antenna
The high gain antenna has a r=1/4, K=14 convolutional encoder. A
hardware Viterbi decoder was built for this code. The low gain
antenna has a standard r=1/2, K=7 convolutional encoder. Both are
hardwired to their respective antennas
JPL designed a convolutional coder that executes in software on
the onboard computer. When concatenated with the existing K=7
encoder on the low gain antenna, it gives a (different) r=1/4, K=14
convolutional code. This code is Viterbi decoded in software on a
Sparc 1000, which is fairly easy at only 10 bps!
Concatenated with the convolutional code is a GF(256) Reed-
Solomon block code and interleaver. The RS blocks have varying
amounts of parity
KA9Q FEC - 54
Galileo-2
The variable overhead in the RS blocks means that some of the

blocks may decode successfully while others do not.
When a RS block decodes successfully, this information is fed back
to the Viterbi decoder
The Viterbi decoder uses this information to “pin down” the states
it may consider in a second pass over the original received
message
The improved Viterbi decoder performance results in better data
being fed to the RS decoder, which in turn is able to correct even
more erasures, feeding even more information back to the Viterbi
decoder
After 4 or 5 iterations, things stop improving. The result is a
phenominally low Eb/N0 requirement of only 0.56 dB!
KA9Q FEC - 55
Coding Over Fading Channels
So far the discussion has dealt almost solely with nonfading

channels; wideband thermal noise is the only problem
This is true for satellite and deep space communications, where
much of the early work in coding was done. But it is not true for
many practical terrestrial channels, e.g., mobile radio, HF
Coding is often even more powerful on a fading channel than it is
on the AWGN channel. This is especially true for short, deep fading
or pulsed interference (e.g., radar). Without coding, the waterfall
curves can flatten out an an unacceptably high error rate for any
practical Eb/N0
On HF, coding can be seen as a more efficient form of diversity.
Equivalently, simple diversity can be seen as a trivial form of coding
(repetition coding)
KA9Q FEC - 56
Diversity vs Coding
KA9Q FEC - 57
Hybrid ARQ-FEC Protocols
Error control coding is highly effective at optimizing the use of

bandwidth and/or power when the channel characteristics are
relatively stable and the required error rate is not too low
But computer networking applications have extremely tight error
requirements; essentially no errors are acceptable. And real
channels often vary in quality
This explains the traditional popularity of ARQ (Automatic Request-
Repeat) protocols in computer communications
The ideal scheme combines FEC and ARQ to give the advantages of
both. Two hybrid ARQ-FEC types exist: Type I and Type II
KA9Q FEC - 58
Type I ARQ-FEC
In a Type I Hybrid ARQ-FEC system, an ARQ protocol is layered on

(computer-speak) or concatenated with (code-speak) a FEC scheme
such as Viterbi decoded convolutional coding
What errors remain after FEC decoding are detected (e.g., with a
CRC) and corrected by ARQ retransmissions. The inner FEC
scheme reduces the error rate seen by the ARQ protocol so the
latter functions reasonably efficiently, i.e., without too many
retransmissions.
The two layers operate more or less independently, though the ARQ
layer may be able to ask the FEC layer to change its coding rate to
optimize overall performance as the channel changes
The V.32/V.32bis/V.34/V.42bis modem is a very good example of a
Type I hybrid. LAPM (in V.42bis) is the ARQ component and the
trellis coding in V.32/V.32bis/V.34 is the FEC component.
KA9Q FEC - 59
Type II ARQ-FEC
In a Type II hybrid ARQ-FEC system, the two layers are more closely
coupled.
A low rate FEC code that can be punctured or inverted is chosen.
That is, the decoder is capable of decoding a message with a large
number of missing symbols
The transmitter first sends only enough of the coded symbols to
allow successful decoding on a “clean” channel. If this is
successful, an ACK signals the transmitter to proceed to the next
data block; the remaining symbols from the acknowledged block
are discarded unsent
If the receiver cannot decode the packet, it sets it aside without
discarding it. The transmitter then sends additional symbols from
the encoded packet. The receiver combines these with the previous
transmission and tries again to decode the result
The process repeats until decoding is successful. If all of the coded
symbols have been sent and the receiver still can’t decode the
packet, the symbols are sent again
KA9Q FEC - 60
Type II - 2
As more of the symbols of a packet are sent, two things happen: the
total received Eb/N0 increases, and the effective code rate
decreases, increasing the coding gain.
Eventually the packet can be decoded successfully, assuming that
the synchronization mechanism still works
Type II hybrid ARQ-FEC schemes are more complex, so they aren’t
as popular as Type I schemes. But they are highly efficient and
adapt quickly to changing channel conditions, which makes them
interesting for radio
Kantronics’ G-TOR is a Type II hybrid. The FEC code in G-TOR is the
Golay block code
Pactor resembles a Type II hybrid, but the “FEC” scheme (simple
repetition of the same uncoded packet) has no coding gain. Pactor
relies solely on Eb/N0 accumulation for its performance
KA9Q FEC - 61
Summary
Error control coding has been around a long time. Most of the
important codes and algorithms were discovered over 25-30 years
ago. But until relatively recently, strong FEC was used mainly in
deep space communications where the benefits outweighed the
considerable computational costs
But the computer revolution has changed all this. It is now possible
to implement strong FEC at reasonable speeds and at reasonable
prices, even in software on general purpose computers
Inexpensive consumer products (CD players, V.32 modems) now
incorporate strong FEC that brings these systems remarkably close
to the Shannon limit
It’s time we brought FEC to ham radio as well!
KA9Q FEC - 62
For Further Reading
Odenwalder, J.P., “Error Control Coding Handbook (Final Report)”,

15 July 1976, Linkabit Corporation, San Diego CA (Excellent
practical overview of the field, though slightly dated)
Viterbi, A.J., Omura, J.K., “Principles of Digital Communication and
Coding”, McGraw-Hill, 1979. ISBN 0-07-067516-3 (The Viterbi
algorithm analyzed by its inventor)
Lindsey, W.C., Simon, M.K., “Telecommunications Systems
Engineering”, Prentice-Hall 1973. ISBN 0-13-902429-8. (Good
discussion of orthogonal modulation)
Lin, S., Costello, D.J., “Error Control Coding: Fundamentals and
Applications”, Prentice-Hall 1983. ISBN 0-13-283796-X. (A very
popular course textbook. Good section on ARQ-FEC hybrids)
Proakis, J. “Digital Communications” (2nd Edition), McGraw-Hill
1989. ISBN 0-07-050937-9 (very good all-around reference)
Yuen, J.H., ed., “Deep Space Telecommunications Systems
Engineering”, Plenum Press 1983. ISBN 0-306-41489-9. (Good
reference for NASA/JPL practices)
KA9Q FEC - 63
3G Wireless Cellular Communications:

Improving Scrambled CDMA Downlink
Receivers
Adam Margetts
Electrical Engineering Dept.
The Ohio State University
Link to Real World
• What is 3G?
– Next generation cellular standards
– based on code division multiple access (CDMA)
(~2 Mbps)
– W-CDMA (ETSI)—Upgrade from GSM (sort-of)
– cdma2000 (TIA)—Upgrade from IS-95
• Where does my research fit in?
– Improving the 3G Downlink (base station to
mobile unit)
– Mobile unit’s receiver combats time-varying
channel impairments
Transmitted Multiuser CDMA Downlink Signal
Scrambling
Spread Code
Bitstream 1
(SF = N1)
multiuser
+ x chips
Spread
Bitstream K Pilot
(SF = NK)
• Chip-rate is constant.
• Orthogonal Variable Spreading Factor (OVSF) codes.
• Pilot is all 1’s code.
• Scrambling whitens transmitted signal.
Channel Impairments In the Downlink
• Symbol period is shorter than delay spread of echoes

(Channel modeled as FIR filter)
– Multipath ⇒ ICI ⇒ loss of orthogonality ⇒ MAI.
• Mobility implies a time-varying system
– Fading ⇒ Tracking ⇒ Adaptation.
• Ubiquitous additive white Gaussian noise
Two-Stage Receiver
Stage 1 • All bitstreams are

received detected and fed
Linear forward
signal
• DD adaptation
Delay
Despread improves tracking
Detect
Respread
Linear to despreader
Stage 2
Second Stage: Decision “Feedback”
Equalization
Received signal To despreader Fed-forward chips
-
FIR + FIR z-1
+
-
• Decisions are fed-forward from 1st stage.
• Post-cursor ICI cancellation.
• Decision-directed adaptation.
• Improvement over linear MMSE.
Second Stage: Inter-chip-Interference (ICI)
Cancellation
Received signal To despreader Fed-forward chips
- Re-channel
+ (sans Cursor)
- +
• Re-channel multiuser chip-rate signal.
• Pre- and post-cursor ICI cancellation.
• Decision-directed adaptation.
• MRC many ICI cancellation branches for diversity gain
Simulation Results
• DD enhances
adaptive
performance
• Second-stage
improves BER
• IC out-performs
DFE.
• Max-SINR+IC
fails to reach MF
bound due to
errors fed
forward.
Adaptive
Algorithms
Publications
• Receiver first stage:

– A. R. Margetts and P. Schniter, `Àdaptive Chip-Rate Equalization of Downlink
Multirate Wideband CDMA,'' submitted to IEEE Transactions on Signal Processing.
– P. Schniter and A.R. Margetts, `Àdaptive Chip-Rate Equalization of Downlink Multirate
Wideband CDMA,'' Proc. Asilomar Conf. on Signals, Systems, and Computers, (Pacific
Grove, CA), Nov. 2002
• Receiver second stage:

– A.R. Margetts and P. Schniter, ``Chip-Rate Adaptive Two-Stage Receiver for
Scrambled Multirate CDMA Downlink,'' to appear in Asilomar Conf. on Signals,
Systems, and Computers, (Pacific Grove, CA), Nov. 2003.
– A.R. Margetts and P. Schniter, `Àdaptive Inter-chip Interference Cancellation of
Multirate Scrambled Downlink CDMA,''' Proc. Allerton Conf. on Communication,
Control, and Computing, (Monticello, IL), Oct. 2003.
• Future Research:
– On the MF bound of scrambled DS-SS in Rayleigh Fading
– Detection of the Active OVSF codes in the Downlink
– MIMO Equalization for the CDMA downlink
My Background
• Courses related to my research

– Adaptive Equalization (EE 894a)
– Random Variables & Processes, Detection and Estimation
(EE 804-806)
– Linear Algebra (Math 601-602)
– DSP (EE 600, EE 700)
– Digital Communications (EE 803, EE 807, EE 809, EE 894i)
• My funding
– DAGSI Fellowship 2000-2001
– Teacher Assistant 2001-2002
– NASA OSGC Fellowship 2002-2005

DC Digital Communication PART3

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

DC Digital Communication PART3

Hochgeladen von

Copyright:

Verfügbare Formate

A property of MVG_OMALLOOR 11/25/2008

ELEC 303 – Random Signals

Lecture 20 – Random processes

Example: sample functions of a

• Ω is the sample space

Mean of a random process

Wide sense stationary process

• Also, we saw that mX(t)=0

Cross correlation and joint stationary

Definition Random Variable

Random Process Example 1

x(t,e1) Average temp at e1

Example 2 Types of Stochastic Process

fall for Sep 3 Continuous Time,

Counting Process Counting Process

Counting Process Counting Process

independent of the past history of the 1

Poisson Process Poisson Process

z # of arrivals in each interval are independent

Joint PMF Example

Interarrival Time Interarrival Time

Brownian Motion Process Brownian Motion Process

z A continuous time, continuous value Gaussian RV –(x - µ)2

Covariance of X and Y Autocovariance

• High covariance = an observation of X provides How much the sample function

Definition: Definition: The correlation of X and Y is rX,Y

z Definition: The autocorrelation function Theorem: The autocovariance and

Autocovariance & Autocorrelation

m and k are integers

Autocovariance & Autocorrelation

Stationary Process fX( t τ ),…, X( tm + τ ) (x1,…,xm)

Stationary Random Sequence Stationary Process

Definition: A random sequence Xn is stationary Theorem: A stationary process X(t),

Stationary Random Sequence Example

Example Wide Sense Stationary

Wide Sense Stationary Wide Sense Stationary

U of R, ECE244/444, Alireza Seyedi, Fall '08

General p problem of communications: We are asked to

U of R, ECE244/444, Alireza Seyedi, Fall '08

Elements of digital communications system:

U of R, ECE244/444, Alireza Seyedi, Fall '08

 Channel encoder: Adds redundancy to the data to protect again

U of R, ECE244/444, Alireza Seyedi, Fall '08

U of R, ECE244/444, Alireza Seyedi, Fall '08

A stochastic or random pprocess (RP)

U of R, ECE244/444, Alireza Seyedi, Fall '08

U of R, ECE244/444, Alireza Seyedi, Fall '08

Since each X(t)

Using this joint pdf, we can define statistical averages

U of R, ECE244/444, Alireza Seyedi, Fall '08

U of R, ECE244/444, Alireza Seyedi, Fall '08

U of R, ECE244/444, Alireza Seyedi, Fall '08

Stationaryy random processes

f X ( t1 ),..., X ( t N ) ( x1 ,..., x N ) = f X ( t1 +τ ),..., X ( t N +τ ) ( x1 ,..., x N )

U of R, ECE244/444, Alireza Seyedi, Fall '08

Stationaryy random processes

U of R, ECE244/444, Alireza Seyedi, Fall '08

Assume we filter an RP,, X(t),

 Find the frequency response H(f)

U of R, ECE244/444, Alireza Seyedi, Fall '08

U of R, ECE244/444, Alireza Seyedi, Fall '08

 Find PSD of X5 from PSD of X4

U of R, ECE244/444, Alireza Seyedi, Fall '08

National Taiwan Ocean University Announcement

Channel encoder: Adds redundancy to the data to protect again

Find the frequency response H(f)

Find PSD of X5 from PSD of X4

Properties are analogous to pdf:

Each value of A comes from a set of possible values,