Communications DSP

Digital Transmission Through Bandlimted Channels

CONTENTS
Characterization of Bandlimited Channels
Characterization of Intersymbol Interference
Signal design for bandlimited Channels
Linear Equalizers
Adaptive Linear Equalizers
Non-Linear Equalizers
Textbook : J. Proakis and M. Salehi: Contemporary communication systems
using MATLAB, 1st Edition. Brooks/Cole, Thomson Learning.
2000. (2nd Edition available on 2004).
1

Communications DSP

1. Intersymbol Interference

Many communication channels, including telephone channels and some

radio channels, may be characterized as bandlimited linear filters with
frequency response
C ( f ) = A( f )e j ( f )

(2-1)

where A( f ) and ( f ) are the amplitude and phase responses, respectively.

A channel is non-distorting or ideal within the bandwidth W if,

A( f ) = Ac , and ( f ) = c f , f W ,

(2-2)

where Ac and c are constants (i.e. constant amplitude and linear-phase

responses).

Communications DSP

If A( f ) is not constant, the distortion is called amplitude distortion, and

if c is not constant, there will be phase distortion.

A measure of the phase linearity or phase distortion of the system is the

envelope delay or group delay

( f ) =

1 d ( f )

.
2
df

(2-3)

Due to the amplitude and phase distortion caused by non-ideal channel, a

succession of pulses transmitted through the channel at a rate
comparable to the bandwidth W are smeared.

Individual pulses might not be distinguishable at the receiver and we

have intersymbol interference (ISI).

Communications DSP

Note the received pulse

for a non-ideal channel
does

not

have

zero

crossings at T , 2T , and
so on.

It is possible to
compensate for the nonideal frequency response
characteristic of the
channel by use of a filter
or equalizer at the
receiver.

Communications DSP

2. CHARACTERIZATION OF BANDLIMITED CHANNELS

Usable band of the channel :

300Hz to 3200Hz.

Impulse response duration of

an average channel is ~ 10 ms
= L.

If the transmitted symbol rates

is Rs=2500 pulses or symbols
per second, the intersymbol
interference might extend over
20 to 30 symbols
( Rs L = 2500 10 10 3 ).
Telephone Channels
5

Communications DSP

Time dispersion, and hence ISI, is

the result of multiple propagation
paths with different path delays.

The number of paths and the

relative time delays can vary with
time.
called

For this reason, they are

time-variant

multipath

channels.

The channels can be

Time-dispersive wireless channels:
e.g. short-wave ionospheric propagation
(HF), tropospheric scatter, and mobile

characterized by the scattering

function: a 2D representation of
the average received signal power

cellular radio.

as a function of relative time delay

and Doppler frequency spread.
6

Communications DSP

The total time duration (multipath spread) of the channel response is

approximately 0.7 s on the average. If transmission occurs at a rate of 107
symbols/sec over the channel, the multipath spread of 0.7 s will result in
intersymbol interference that spans about 7 symbols ( 0.7 10 6 107 ).

Exercise: GO THROUGH ILLUSTRATIVE PROBLEM 6.5 (ON MULTIPATH

CHANNEL SIMULATION) AND THE M-FILE.

Communications DSP

2.3

Eye Diagram

The amount of ISI and noise can

be viewed in an oscilloscope:
Display the received signal y (t ) on
the

vertical

input

with

the

horizontal sweep rate set to 1/T

(the symbol rate).

ISI causes the eye to close:

1) reducing the margin for additive
noise (higher detection errors).
2) distorting the position of the
zero-crossings and causes the
system

more

sensitive

synchronization errors.
8

to

Communications DSP

3 Signal Design for Bandlimited Channels

Necessary and sufficient condition

for a signal x(t ) to have zero ISI is

0 n = 0
x(nT ) =

1 n 0

X(f +

m =

m
)=T
T
(3-1)

where 1/T is the symbol rate.

One commonly used signals has a

raised-cosine-frequency
characteristics.

The sampled waveform x (t ) can be written as

9

response

Communications DSP

T
f
,
0

2T
T

1 1
1+

< f
X rc ( f ) = 1 + cos f
,

2T
2T
2T
2
1+

>
f
0
,

2T

(3-2)

where 0 1 is called the roll-off factor, range and 1/T is the symbol
rate.

When = 0 , X rc ( f ) reduces to an ideal brick wall physical

nonrealizable response with bandwidth occupancy 1/(2T), called the
Nyquist frequency.

For > 0 , the bandwidth occupied by X rc ( f ) beyond the Nyquist

frequency is called the excess bandwidth, usually expressed as a
percentage of the Nyquist frequency.

For = 1 / 2 , the excess

bandwidth is 50%, and when = 1, the excess bandwidth is 100%.

10

Communications DSP

The signal pulse xrc (t ) having the raised-cosine spectrum is

xrc (t ) =

sin(t / T ) cos(t / T )
(t / T ) 1 4 2t 2 / T 2

(3-3)

In absence of channel distortion, there is no ISI from adjacent symbols.

In presence of channel distortion, a channel equalizer is needed to
minimize its effect on system performance.
Exercise: GO THROUGH ILLUSTRATIVE PROBLEM 6.7 (DESIGN OF
TRANSMIT AND RECEIVE FILTER).
This method is called frequency sampling. Suppose the filter length is 2N+1.
The desired analog frequency response is H d () . The desired frequency
~
~
response of the discrete-time filter is then H d (e j ) = H d (e jTs ) = H d () . [For
designing the pulse shaping filter, to avoid aliasing, we choose

11

1 4
= .]
Ts T

Communications DSP

~
Sample H d (e j ) at 2N+1 equally spaced points over the unit circle:

k = {2k /( 2 N + 1) : k = N ,...,0,..., N } , we get a set of points H d (e j ) = H [k ] . If the

k

DT-FT of the filter pass through these points then,

H (e

j k

)=

h[n]e

jn k

= H [k ]

n= N

Taking the inverse, one gets h[n] =

H [k ]e

k = N

jk n

= H [k ]e j 2kn /( 2 N +1) .
k = N

Other methods for designing the transmit and receive filters (Nyquist filters)
include semidefinite programming (SDP) and eigenfilter methods.
Normally, X RC ( f ) will be designed first. It is then factored by a method
called spectral factorization to obtain GT ( f ) and G R ( f ) .

12

Communications DSP

4 Linear Equalizers
The most common type of channel equalizer used in practice to reduce ISI is
a linear FIR filter with adjustable coefficients {ck } .

The ISI is usually negligible

beyond a certain number
of symbols.

The number of terms in the

ISI term is thus finite.

The linear filter is therefore

usually implemented as
finite-duration impulse
response (FIR) filter, with
adjustable tap coefficients.

13

Communications DSP

4.1

Symbol- and Fractionally-spaced Linear Equalizers

The time delay between adjacent tap may be selected as large as T, and
the equalizer is called a symbol-spaced equalizer.

The input to the

equalizer is y (kT ) . However, the excessive bandwidth of the signal causes

those components above the Nyquist frequency 1/(2T) to aliase with those
below.

The equalizer only compensates for the aliased channel-distorted

signal.

If is shorten to T/2, i.e. the received signal is sampled at 1/(2T) Hz, then
even for 100% excess bandwidth, there will not be any aliasing in the
received signal. The channel equalizer is said to have fractionally spaced
taps, and it is called a fractionally spaced equalizer.
operating speed is needed.

14

However, higher

Communications DSP

For zero ISI, the product of the various transfer functions should equal to
a Nyquist channel, say X rc ( f ) , the raised cosine spectrum
GT ( f )C ( f )G R ( f )G E ( f ) = X rc ( f ) .

(4-1)

The receive pulse shaping filter G R ( f ) are usually matched to transmit

pulse shaping filter GT ( f ) with
GT ( f )G R ( f ) = X rc ( f ) ,

(4-2)

i.e. it forms a Nyquist channel with no ISI in absence of channel distortion.

The frequency response of the equalizer should be

GR ( f ) GE ( f ) =

1
1
=
e j c ( f ) ,
C( f ) C( f )

(4-3)

In this case, the equalizer is said to be the inverse channel filter to the
channel response. It might not be stable and realizable.
15

Communications DSP

4.2

Zero-forcing equalizers (ZF equalizers)

The impulse response of the FIR equalizer is

g E (t ) =

c (t n ) ,

n= K

(4-4)

and the corresponding frequency response is

GE ( f ) =

c e

n= K

j 2fn

(4-5)

where {cn } are the 2K+1 equalizer coefficients, and K is chosen sufficiently
large so that the equalizer spans the length of the ISI.

Zero-forcing assumes the received pulse shape (or channel) is known

and finds an equalizer to minimize the ISI at time instants, nT, n=1,..

16

Communications DSP

Let GT ( f )C ( f )G R ( f ) = X ( f ) and x(t ) the time waveform corresponding to

X ( f ) . The equalized output signal pulse q(t) is

x(t ) * g E (t ) = q (t ) =

n= K

x(t n ) ,

(4-6)

The samples of q (t ) taken at times t = mT , are given by

q (mT ) =

n= K

x(mT n ) , m = 0,1,...., K .

(4-7)

For zero ISI, q(mT ) should ideally be zero except at m=0.

Since there are 2K+1 equalizer coefficients, we can control only 2K+1
sampled values of q (t ) .

We therefore impose the zero-forcing

conditions to determine the ZF-equalizer coefficient {cn }

m=0
1,
q (mT ) =
.
0
,
1
,
2
,...,
=

m
K

17

(4-8)

Communications DSP

(4.7) is a system of linear equation in unknown {c n } which can be written

as
Xc = q ,

(4-9)

where X is an (2 K + 1) (2 K + 1) matrix with elements x(mT n ) , c is the

(2 K + 1) coefficient vector, and q is the (2 K + 1) column vector with one

nonzero element.

The equalizer coefficient is given by

c = X 1q ,

(4-10)

FIR zero-forcing equalizer does not completely eliminate the ISI

because it has a finite length. As K is increased, the residual ISI can be
reduced, and as K , ISI is completely eliminated under noise free
condition.

18

Communications DSP

A drawback of zero-forcing equalizer is that it ignores the presence of

additive noise. It might lead to significant noise enhancement.
frequency

range

where C ( f ) is

small,

the

channel

In a

equalizer

G E ( f ) = 1 / C ( f ) compensates by placing large gain in that frequency

range. The noise in that frequency range is greatly enhanced.

Example: Suppose that the received output is equal to

x(t ) =

1
,
2
1 + (2t / T )

(4-11)

where 1 / T is the symbol rate. The pulse is sampled at the rate 2 / T and is
equalized by a zero-forcing equalizer.
five-tap zero-forcing equalizer.

19

Determine the coefficients of a

Communications DSP

Solution
Since the pulse is sampled at the rate 2 / T , it is a fractionally-spaced equalizer
with = T / 2 . Also, as the equalizer is of 5 taps, K = (5 1) / 2 = 2 .

The zero-forcing condition in (4.8) becomes

q(mT ) =

n= K

c n x(mT

nT
) , m = 0,1,....,2 .
2

(4-12)

From (4.7), the matrix X and vector q are given by

0
1 / 5 1 / 10 1 / 17 1 / 26 1 / 37
1
0
1 / 2 1 / 5 1 / 10 1 / 17

X = 1/ 5 1/ 2
1
1 / 2 1 / 5 , and q = 1 .

1
/
17
1
/
10
1
/
5
1
/
2
1

0
1 / 37 1 / 26 1 / 17 1 / 10 1 / 5
0

20

(4-13)

Communications DSP

Solving for linear equation Xc = q yields

c opt

c 2
2 .2
c
4 .9
1

1
= c0 = X q = 3 . (4-14)

c
4
.
9
1

c 2
2.2

21

Communications DSP

4.3

Minimum Mean-Square Error (MMSE) Equalizers

Let z (t ) be the noise-corrupted output of the FIR equalizer

z (t ) =

n= K

y (t n ) ,

(4-14)

where y (t ) is the input to the equalizer given by (2.3).

The equalizer is sampled at times t = mT , and we have
z (mT ) =

n= K

y (mT n ) .

(4-15)

The desired response at the output of the equalizer at t = mT is the

transmitted symbol a m (assume to be known during training of the
equalizer).

The error is defined as the difference between a m and z (mT ) .

22

Communications DSP

The mean-square error (MSE) between the actual output sample z (mT ) and
the desired values a m is
MSE = E[ z ( mT ) a m

c c E [y
K

n= K k = K

n k

K
] = E c n y (mT n ) a m
n = K

(mT n ) y (mT k )]
(4-16)

2 c k E a m* y (mT k ) + E (| a m |2 ) .
k = K

c c

n= K n= K

where

n k

R y (n k ) 2 c k Ray (k ) + E (| a m | ) .
2

k = K

R y (n k ) = E[ y * (mT n ) y ( mT k )] , Ray (k ) = E[am* y (mT k )] ,

The expectation E [] is taken with respect to the random information

sequence {a m } and the additive noise.

23

Communications DSP

The minimum MSE solution is obtained by differentiating (4.15) with

respect to the equalizer coefficients {cn }.

The condition for minimum MSE is

K

c R

n= K

(n k ) = Ray (k ) , k = 0,1,2,..., K ,

(4-17)

which is a system of linear equation with (2 K + 1) equations in (2 K + 1)

unknown {c n }.

In practice, the autocorrelation sequence R y (n) and the cross-correlation

sequence Ray (n) are unknown a priori. They have to be estimated using
the time-average estimates
K
K
1
1
*
R y (n) = y (kT n ) y (kT ) , R ay (n) = y * (kT n )a k*
K k =1
K k =1

24

(4-18)

Communications DSP

The symbols ak are assumed known, which are transmitted to the

receivers during the so-called training mode.

In contrast to the zero-forcing solution, these equations depend on the

statistical properties (the autocorrelation) of the noise as well as the ISI
through the autocorrelation R y (n) .

25

Communications DSP

5 Adaptive Linear Equalizers

On channels whose frequency response characteristics are unknown

but time-invariant (does not change with time, such as telephone lines),
we may measure the channel characteristics by sending known
symbols (training symbols) to the receiver and adjust the parameters of
the equalizer. Once adjusted, the parameters remain fixed during the
transmission of data. Such equalizers are called preset equalizers.

Adaptive equalizers update their parameters by sending periodically

training symbols to the receivers during the transmission of data, so
they are capable of tracking a slowly time-varying channel response.

26

Communications DSP

Both the zero-forcing and MMSE equalizers require the solution of a system
of linear equation of the form
(5-1)

Bc = d

where B is a (2 K + 1) (2 K + 1) matrix, c is a column vector representing the

(2 K + 1) equalizer coefficients, and d is an (2 K + 1) column vector.

The solution is

c opt = B 1d

(5-2)

Solving (5.2) directly will require very high arithmetic complexity:

O((2 K + 1) 3 ) . In practical implementation, it is solved using iterative

methods such as the Least mean squares (LMS) or recursive least squares
(RLS) algorithms.

27

Communications DSP

5.1

LMS algorithm
The LMS algorithm is based on the method of steepest descent:

1. One begins with an arbitrarily chosen coefficient vector say c0 .

2. Each tap coefficient is changed in the direction opposite to its
corresponding gradient component in the gradient vector g , which is the
derivative of the MSE with respect to the (2 K + 1) filter coefficients:
c k +1 = c k g k

where is the step-size parameter.

28

(5-3)

Communications DSP

In the LMS algorithm, g is estimated continuously. A commonly used

method is to approximate the MSE by the instantaneous error
(ek ) 2 = (a k c kT y k* ) 2 . Thus, the estimated gradient is

g k = ck (ek ) 2 = 2ek ck ek = 2ek y k*

(5-4)

The LMS or stochastic gradient algorithm is given by

c k +1 = c k + ek y k* ,
ek = a k c kT y k* .

(5-5)

Step-size selection:
One commonly used step-size parameter in order to ensure
convergence and good tracking capabilities in slowly varying channels is

1
=
5(2 K + 1) PR

(5-6)

PR denotes the received signal-plus-

noise power, which can be estimated

from the received signal.
29

Communications DSP

The optimal solution can be approached after a few hundred iterations.

As the equalizer is updated at the symbol rate, it corresponds to a
fraction of a second.

Initially, the adaptive equalizer is trained by the transmission of a

known pseudorandom sequence {am } over the channel.

At the

demodulator, the equalizer employs the known sequence to adjust its

coefficients.

Upon initial adjustment, the adaptive equalizer switches from a

training mode to a decision-directed mode, in which case the
decisions at the output of the detector are sufficiently reliable so that
the error signal is formed by computing the difference between the
detector output and the equalizer output
ek = a k z k

(5.7)

a k is the output of the detector.

30

Communications DSP

In general, decision errors at the output of the detector occur infrequently.

Such errors have little effect on the performance of the tracking algorithm.

EXERCISE: GO THROUGH ILLUSTRATIVE PROBLEM 6.12 AND RUN THE MFILE.

31

Communications DSP

EXAMPLE
The LMS algorithm is used to identify the following channel

x = [0.05,0.063,0.088,0.126,0.25,0.9047,0.25,0,0.126,0.038,0.088]
with (2 K + 1) = 11.

Smaller stepsize
leads to faster
convergence but
higher errors.

RLS has faster

convergence but a
high complexity.

32

Communications DSP

6 Nonlinear Equalizers
The linear filter equalizers are very effective on channels, such as wire line
telephone channels, where ISI is not severe.
The severity of the ISI is directly related to the spectral characteristics of the
channel and not necessarily to the time span of the ISI.

There is a spectral null in channel B at f=1/2T (more severe ISI). Channel A

does not have a channel null and has a large span of ISI.
33

Communications DSP

The energy of the

total

response

is

normalized to unity
for both channels.

The time span of the ISI in channel A is 5 symbol intervals on each side of
the desired signal component, which has a value of 0.72.

The time span for the ISI in channel B is one symbol interval on each side
of the desired signal component, which has a value of 0.815.

In spite of the shorter ISI, channel B results in more severe ISI. A linear
equalizer will introduce a large gain in its frequency response to
compensate for the channel null in channel B at f=1/2T, noise is enhanced.

34

Communications DSP

6.1

Decision Feedback Equalizers (DFE)

A DFE is a nonlinear equalizer that employs previous decisions to

eliminate the ISI caused by previously detected symbols on the current
symbol to be detected.

35

Communications DSP

It consists of two filters. The first filter is a feedfoward filter, which is

generally a fractionally-spaced FIR filter with adjustable tap coefficients.
The second one is called a feedback filter, which is an FIR filter with
symbol-spaced taps having adjustable coefficients. Its input is the set
of previously detected symbols.

The output of the feedback filter is subtracted from the output of the
feedforward filter to form the input to the detector.
N1

N2

z m = c n y (mT n ) bn a~m n
n =1

n =1

36

(6.1)

Communications DSP

where {cn } and {bn } are the adjustable coefficients of the feedforward
and feedback filters, respectively; a~m n , n = 1,2,.., N 2 , are the previously
detected symbols; N1 and N 2 are the length of the feedforward filter
and feedback filters, respectively.

The tap coefficients are usually selected to minimize the MSE criterion
using the stochastic gradient (LMS) algorithm or RLS algorithm.

Decision errors from the detector that are fed to the feedback filter
have a small effect on the performance of the DFE. A small loss in
performance of 1 to 2dB is possible at error rates below 10 2 , but the
decision errors in the feedback filters are not catastrophic.

37

Communications DSP

In a digital communication system that transmit information over a

channel that causes ISI, the optimum detector is a maximum-likelihood
symbol detector (MLSD) that produces at its output the most probable symbol
sequence {a~ } for the given received sampled sequence { y } .
k

38

Communications DSP

That is, the detector finds the sequence

{a~k } that maximizes the

likelihood function.
({a k }) = ln p ({ y k } | {a k })

(6.2)

where p ({ y k } | {a k }) is the joint probability of the received sequence

{ y k } conditioned on {a k } .

The Viterbi algorithm can be used to

implement the MLSD, but its complexity grows exponentially with the
span of the ISI. They are suitable for short channel with severe ISI,
such as mobile channels.

39