ITG-Fachbericht 257: Photonische Netze 07. 08.05.

2015 in Leipzig

Equalization in Fiber-Optic Transmission Systems:

Theoretical Limits and Lattice-Reduction-Aided Techniques
Robert F.H. Fischer, Shayan Hassanpour
Institut fr Nachrichtentechnik, Universitt Ulm, Albert-Einstein-Allee 43, 89081 Ulm, robert.fischer@uni-ulm.de

Abstract
Fiber-optic transmission systems are studied where linear distortions due to chromatic dispersion are dominant. We give
an overview on the theoretical performance of linear equalization and decision-feedback equalization, both for coherent
schemes and those employing direct detection. Moreover, the use of lattice-reduction-aided equalization is discussed and
assessed. Its advantages over the classical schemes are enlightened.

Introduction

In general, the optical ber channel can be modeled as a

dispersive and nonlinear system. If the launch power is
small enough nonlinear distortions due to the Kerr effect
can be neglected; in these scenarios the chromatic dispersion is dominant. Since nowadays digital signal processing
is widely available for the required speeds, the dispersions
are preferably handled in the digital domain rather than installing dispersion compensating bers.
In the literature, to the most extent linear equalization
(LE) (either optimized according to the zero-forcing (ZF)
or minimum mean-squared error (MMSE) criterion) are
studied for handling linear dispersions, cf. [11]. Meanwhile also decision-feedback equalization (DFE) and its
transmitter-side counterpart Tomlinson-Harashima precoding (THP) have been investigated for the ber-optic channel, e.g., [15]. Furthermore, discrete multitone schemes,
e.g., [2], may be employed.
However, the performance of these schemes is usually
assessed via numerical simulations and under specic restrictions (e.g., xed receiver front end or nite number of
equalization taps); analytic studies can be found, e.g., in
[8, 11]. In this contribution, we rst give an overview on
the theoretical performance of the respective equalization
techniques. This is done for coherent optical schemes using
complex-valued signaling (one polarization), as well as for
schemes employing real-valued modulation in connection
with a strong optical carrier component and direct detection. Based on the theoretical insight, the effect of different
receiver front ends on the performance is discussed.
Moreover, lattice-reduction-aided (LRA) equalization
initially introduced for multiple-input/multiple-output systems [19, 17] and then transferred to intersymbol-interference channels [7]is applied to ber-optic schemes. The
optimization and the achievable performance are given. It
is shown that in particular in schemes using direct detection
where power fading [16, 14] is an inherent effect, LRA
schemes can signicantly improve performance; unstable
behavior for critical ber lengths which are derived an-

System Model

We consider coherent ber-optic transmission system and

those employing direct detection. The symbol interval is
denoted by T ; the baud rate is thus 1/T . The (linear) system model in (complex) baseband representation, ignoring
any nonlinear effect in the ber, is shown in Fig. 1.
a[k]

T hT (t)
T HT (f )

s(t)

hC (t)
HC (f )

n(t)
r(t)

Figure 1 Linear system model (equivalent (complex) baseband)

of the considered ber-optic transmission system.

The data symbols a[k] (discrete time index k) are drawn

uniformly from a signal set A. Here, a QAM constellation
with cardinality M = |A| and variance a2 = E{|a|2 } is
used.
=
Following [9, 10], the transmit pulse shape gT (t) def
T hT (t), is modeled as a cos-roll-off impulse in the time
domain (roll-off factor t )

t)

|t| < T (1
1,
2


/2)
gT (t) = ce 12 1 sin (|t|T
, else
t T

t)
0,
|t| > T (1+
2

(1)

GT (f ) = ce T

sin(f T ) cos(t f T )
,
f T 1 (2t f T )2

where the constant ce is chosen such that the pulse has a

This work was funded by the German Federal Ministry of Education,

Science, Research and Technology (BMBF) within the framework of the
SASER project ADVAntage-Net under grant 16 BP 12406.

ISBN 978-3-8007-3938-7

alytically, is avoided. Via integer precoding (i.e., generalized partial-response coding) LRA schemes can readily be
combined with channel coding schemes but, in contrast to
THP, preserve the discrete nature of the transmit symbols.
The paper is organized as follows: In Sec. 2, the system
model is introduced. The theoretical performance of equalization schemes is given in Sec. 3, followed by numerical
examples and a discussion in Sec. 4. The appendix briey
discusses the equivalent lowpass model for direct detection.

145

VDE VERLAG GMBH Berlin Offenbach

ITG-Fachbericht 257: Photonische Netze 07. 08.05.2015 in Leipzig

given (and xed) energy Eg , i.e.,


|gT (t)|2 dt .
Eg =

(2)

We only consider the linear effects of the ber; nonlinear distortions are neglected by assuming a sufciently low
launch power. Moreover, since we are only interested in
the dispersive effects the attenuation is ignored.1 Assuming the chromatic dispersion to be dominant, the channel
transfer function in equivalent complex baseband notation2
reads for coherent transmission
HC (f ) = ejL

2
2

(2f )2

(3)

where f is the frequency in Hz and the coefcient 2 of

the quadratic phase term is related to the dispersion parameter D (usually given in units ps/km/nm) of the ber
via D = 2c2 /2L . Noteworthy, since HC (f ) does not
obey the symmetry condition of an odd phase, the corresponding impulse response hC (t) ` HC (f ) is complexvalued.
The additive noise n(t) (either generated by distributed
or lumped ampliers) is assumed to be white and Gaussian
with power spectral density (PSD) nn (f ) = N0 (complex
baseband).
Using direct detection, the system model of Fig. 1 still
holds. However, all signals and the data symbols are
real-valued and the equivalent channel impulse response is
given by (cf. [14, 16] and the Appendix)
hDD (t) = 2 Re{hC (t)}

(4)

which, taking (3) into account, corresponds to the transfer

function
HDD (f ) = 2 cos(L 22 (2f )2 ) .

(5)

Here, the noise n(t) is real-valued and is assumed to be

white and Gaussian with PSD nn (f ) = N0 /2.

Since the ber (ignoring the attenuation) is a pure allpass lter, i.e., |HC (f )| = 1, f , for coherent de j2f T ) =
tection the
folded spectral SNR reads SNR(e
2
2
(T a /N0 ) |HT (fT )| , and hence only depends on
the choice of the transmit pulse. If, additionally, the
transmit
pulse has square-root Nyquist characteristics, i.e.,

2
|HT (fT )| = Eg /T , the spectral SNR reduces to the
2
constant a Eg /N0 . In this case, an additive white Gaussian noise (AWGN) (dispersionless) end-to-end discretetime channel with noise variance n2 = N0 /Eg is present.
No further equalization is required and the SNRs (7), (8)
are identical and equal to a2 Eg /N0 .
If the transmit pulse does not have square-root Nyquist
characteristics (which is the case for the above choice (1)),
the loss in SNR compared to the AWGN channelequal
to the loss in optical SNRcan be quantied by the loss
factor3
() def
=

3.2

The performance of the transmission system can be characterized via the folded spectral signal-to-noise ratio [5]
T a2

j2f T ) def
SNR(e
=
|HT (fT )HC (fT )|2 . (6)
N0

r(t)

Therefrom, the performance of ideal (zero-forcing) linear

equalization (LE), and that of ideal (zero-forcing) decisionfeedback equalization (DFE), respectively, can be calculated as [5]
 1/(2T
1
 )
1
(iLE)
j2f
T

SNR(e
= T
)
df
, (7)
SNR

SNR

1/(2T
 )

= exp T

log SNR(e

j2f T

) df

(10)

hR (t)
HR (f )

y[k]
kT

F (z)

d[k]

If the transmit spectrum, and hence the useful part of the

receive signal has spectral components for |f | > 1/(2T )
(for |f | > 1/T in case of T /2-spaced processing), alias
components are generated which cannot be removed (and
which are not constructively used as in case of ideal equalization). The subsequent T -spaced feedforward ltering
has to equalize the resulting situation, usually leading to a
loss in performance.
Employing a specic receiver front-end lter hR (t) `
HR (f ), the T -spaced equivalent system model (from the


.(8)

1/(2T )
1 This assumption is equivalent to compare transmission schemes
based on equal receive power and not based on equal transmit power.
2 Transformation frequency equal to the carrier frequency f = c/
L
L
of the laser, where c = 2.99792 108 m/s is the speed of light in vacuum.

ISBN 978-3-8007-3938-7

T
h (t) ,
Eg T

Figure 2 Simplied receiver front end.

1/(2T )

(9)

followed by T -spaced sampling and discrete-time post processing (transfer function F (z)), cf. the block diagram in
Fig. 2. Based on d[k], decisions are taken (uncoded case)
or channel decoding is performed.

Theoretical Limits

(iDFE)

Please note that both, ideal LE and ideal DFE, require a

continuous-time receiver front end consisting of a matched
lter which is adapted to the cascade of transmit lter and
actual channel, i.e., hR (t) = const.hT (t)hC (t) would
have to be used. Thereby, in case of pure chromatic dispersion, the channel matched lter is nothing else than an ideal
dispersion compensating ber, i.e., completely compensating the phase response of the channel. Ideally, an adaptation of this analog receiver front-end lter to the actual
channel would have to be done.4
A more practical way is to implement the matched lter
w.r.t. the transmitter pulse, i.e., simply use

Equalization

3.1

N0 SNR()

Classical Procedures

hR (t) =

a2 Eg

3 ()

indicates the respective equalization strategy.

can be approximated by T /2-spaced sampling.

4 This

146

VDE VERLAG GMBH Berlin Offenbach

ITG-Fachbericht 257: Photonische Netze 07. 08.05.2015 in Leipzig

data symbols a[k] at the input of the pulse shaping (Fig. 1)

to the samples y[k] at the output of the receiver frontend lter (Fig. 2)) has the following discrete-time end-toend transfer function and PSD of the additive discrete-time
noise sequence n[k]

HT (fT )HC (fT )HR (fT )

H(e j2f T ) =

nn (e

j2f T

(11)

2
N0

HR (fT ) .
)=
T

3.2.1 Linear Equalization

1
,
H(z)

(e j2f T ) df .

= T

(14)
3.2.2 Decision-Feedback Equalization
Applying decision-feedback equalization the task of the
feedforward lter F (z) is to generate i) white noise and
ii) a (monic) causal and minimum-phase end-to-end impulse response Heq (z) = F (z)H(z). Via a feedback lter,
the interference of the already available decided symbols
is canceled in the current symbol leading to an improved
detection. In this case, the loss factor calculates to


1/(2T
 )

(DFE) = exp T



log (e j2f T ) df


(15)

1/(2T )

Noteworthy, the receiver-side DFE can be moved to

the transmitter side in form of Tomlinson-Harashima precoding (THP), which has (almost) the same performance
[5, 15] but straightforwardly enables the use of channel
coding.
Moreover, an optimally loaded [3] discrete multitone
(DMT) scheme performs (under some idealized assumptions) as well as DFE [4]. Hence, the theoretical result
(DFE) is also valid for ber-optic multitone scheme as discussed, e.g., in [2, 1, 13].

3.3

Lattice-Reduction-Aided Equalization

1/(2T
 )

Zopt (z) = argmin T




 Z(e j2f T ) 2


 H(e j2f T )  df . (20)

1/(2T )

An efcient algorithm based on lattice decoding for solving

this problem has been given in [7].
Choosing the best-matching integer impulse response,
the loss factor reads
1/(2T
 )

(LRA) def

= T

Zopt (e j2f T )(e j2f T ) df . (21)

1/(2T )

The main advantages of lattice-reduction-aided equalization are as follows:

A smaller noise enhancement (better performance) compared to linear equalization (both using the same receiver
front end) is present.
We assume an M -ary, M a square number, QAM constellation
where

I and Q components are drawn from the set 1, 3, . . ., M 1.

6 mod (): modulo reduction of each quadrature component to the
M

interval [0, M ).
5

In [6, 7], the principle of lattice-reduction-aided equalization [19, 17] has been applied to intersymbol-interference channels. The main idea is to factor the end-to-end

ISBN 978-3-8007-3938-7

d[k]

As in THP, a slicer, taking the modulo congruence of the

signal points into account has to be used at the receiver.
Lattice-reduction-aided equalization can be seen as a
generalization of partial-response signaling [12], where
the target response, Z(z), is chosen in dependency of the
Z(z)
1
(z) = H(z)
causes least
actual channel, such that Hred
noise enhancement, i.e.,

Z(z)

1
(z)
Hred

For data symbols a[k] drawn from an M -ary, M a square

number, QAM constellation5 the precoding is performed as
follows6

a
[k] = a[k] + (1 + j)( M 1)/2
(17)


[k]
[]
x[k ]
(18)
x
[k] = modM a
>0

(19)
x[k] = x[k] (1 + j)( M 1)/2 .

(13)

with the inverse normalized SNR after discrete-time equalization


 
2

HR (fT )
j2f T def
(e
) = 
2 .

HT (f )HC (f )HR (f )
T

H(z)

Figure 3 Lattice-reduction-aided equalization of intersymbolinterference channels with equalization of the integer part via
(Tomlinson-Harashima-type) precoding.

1/(2T )

x[k]

Z(z)1

(12)

1/(2T
 )

(16)

where Z(z) [k] corresponds to an impulse response

(monic, causal, minimum phase) with only coefcients
drawn from the Gaussian integers, i.e., [k] Z + jZ,
k. Then, only Hred (z) is linearly equalized (i.e, F (z) =
1
Hred
(z)), followed by threshold decision. The integer part
is either equalized via DFE on the already decided symbols, or, preferably, via Tomlinson-Harashima-type precoding preequalized at the transmitter side, cf. Fig. 3.
a[k]

where H(z) is the analytic continuation of H(e j2f T ). For

this setting, the loss factor (cf. (9)) calculates to [5]
(LE)

H(z) = Hred (z) Z(z) ,

n[k]

In case of (ZF) linear equalization, the feedforward lter

F (z) removes all (residual) intersymbol interferences, i.e.,
= F (z)H(z) = 1. This is achieved by
Heq (z) def
F (z) =

discrete-time channel transfer function as

147

VDE VERLAG GMBH Berlin Offenbach

ITG-Fachbericht 257: Photonische Netze 07. 08.05.2015 in Leipzig

Employing precoding of the integer part, any channel

coding scheme can be applied immediately (as in THP).
Only very few integer coefcients have to be communicated to the transmitter.
The transmit symbols x[k] are still discrete-valued and
drawn from the same constellation as in conventional
signaling. The laser does not have to be modulated with
a continuous-valued signal as in case of THP, hence only
a low-resolution DAC is required.
However, compared to optimally adjusted DFE/THP, some
performance degradation has to be accepted.

We now present numerical examples covering the performance of the respective equalization techniques. For that
we assume a laser wavelength of L = 1550 nm (carrier
frequency fL = 193.41 THz) and a ber with a secondorder propagation coefcient7 2 = 21.668 ps2 /km. A
single polarization is studied. The symbol rate is 1/T =
10 GHz; the cos-roll-off in the time domain pulse (1) with
t = 0.4 (unless otherwise stated) is used.

4.1
3.4

Direct Detection

If the receive lter has a cut-off frequency smaller than 1/T

(but larger than 1/(2T )), as is usually the case, sampling
after the receive lter leads to a superposition of only two
spectral components. If, thereby, two spectral zeros overlap, the useful transfer function H(z) (or equivalently the
folded spectral SNR) has a spectral zero. In turn, linear
equalization would lead to an innite noise enhancement
and hence is unstable. Two spectral zeros overlap if
!

1
fzero,n2 ,
T

n 1 , n2 N ,

which occurs at ber lengths

2


2n1 + + 2n2 +

Lcrit =
.
1
(2)2 |2 |
T

(23)

(24)

For the settings in the numerical examples, we have the

critical ber lengths (not exhaustive) tabulated in Tab. 1.
Table 1 Critical ber lengths (selection of the six smallest) in
direct detection for the used scenario. 1/T = 10 GHz and |2 | =
21.668 ps2 /km.

n1
0
0
0
1
0
1

ISBN 978-3-8007-3938-7

n2
0
1
2
1
3
2

Coherent Transmission

SNR Loss over Fiber Length

The above described equalization techniques can be applied to either coherent detection, as well as to direct detection. In coherent schemes, the channel transfer function
HC (f ) according to (3) is present; all signals and systems
are complex-valued.
Using direct detection, as only the instantaneous receiver
power but not amplitudes is detected, only real-valued signaling can be used. Even worse, the end-to-end model (see
the Appendix) is given by the transfer function HDD (f )
according to (5).
A well-known characteristic of this transfer function is
the effect called power fading [16, 14], i.e., HDD (f ) has
zeros at



fzero,n = (2)21L|2 | 2n + , n N . (22)

fzero,n1 =

Numerical Examples

Lcrit
146.9 km
274.1 km
384.6 km
440.7 km
488.1 km
578.3 km

First, the loss in SNR over the ber length is assessed for
different receive lters and equalization techniques. Noteworthy, the losses of ideal equalizationwhich is independent of the ber length and caused since the transmit
 0.100 dB;
pulse is not square-root-Nyquistare (iLE) =
 0.046 dB for t = 0.7, and (iLE) =
 0.027 dB;
(iDFE) =
(iDFE) =
 0.013 dB for t = 0.4, respectively, and hence
can be ignored. Again, it should be emphasized that this
performance would require an analog receiver front end
compensating completely the chromatic dispersion of the
ber.
The performance of practical receiver front ends/equalization, specically the matched lter corresponding to the
transmit pulse shape, cf. (10), is shown in Figs. 4 and 5. As
can be seen, losses in the range of 1 to 2 dB for DFE/THP
and up to 4 to 5 dB for LE occur. For long ber lengths,
LRA equalization (the integer polynomial Z(z) is optimized for each ber length individually) can limit the loss
of LE but still a 2 dB difference to DFE/THP is present.
However, channel coding can be used very easily.
The loss is mainly caused by aliasing, as the spectrum
of the transmit pulse is close to a sinc-function. The useful
spectrum after the matched lter is still very broad (in theory innite bandwidth) leading to aliasing due to T -spaced
sampling. This aliased spectrum with its uctuations over
the frequency has to be equalized leading to a loss. For
t = 0.7 the losses are somewhat smaller than for t = 0.4
since the spectrum is more compact and less aliasing is
caused. The oscillations are caused by constructive and
destructive aliasing.8
In Fig. 6 the receive lter is a fth-order Butterworth
lter with 3 dB cut-off frequency 5 GHz. If the receive
lter would be an ideal lowpass lter with a bandwidth of
5 GHz (half the baud rate), no aliasing would be caused after the receive lter, and the quadratic phase of the channel
could be equalized without any noise enhancement. Hence,
no dependency on the ber length would be present. The
loss would only be caused by the non-Nyquist end-to-end
impulse response and (to the most extent) by the loss in
energy due to cutting off all spectral components beyond
7 This corresponds to D = 17 ps/km/nm, which is a typical value
for a standard single-mode ber at a wavelength of 1550 nm, which is
the center wavelength of the third optical window commercially used in
optical transmission.
8 Constructive and destructive interference of the side lobes, when fold
!
ing a sinc-spectrum occur if (cf. (3)) L 22 (2 2/T )2 = 2 hence for
2
L = T /(22 ), which, in our scenario, gives L 73.5 km.

148

VDE VERLAG GMBH Berlin Offenbach

ITG-Fachbericht 257: Photonische Netze 07. 08.05.2015 in Leipzig

10 log10 (() ) [dB]

Contrary, for the Butterworth lter still some aliasing is

caused which has to be equalized. In turn, the loss increases
with the ber length. However, compared to the matched
lter (w.r.t. hT (t)), much less aliasing is present and hence
the losses are smaller. Here, LRA equalization cannot provide any gain (the blue line completely lies under the red
one).

LE
LRA
DFE

Optimization of the Receive Filter

When using a compromise receiver front-end lter, its
parameters have to be adjusted suitably. To this end, in
Fig. 7, the loss is shown over the 3 dB bandwidth of the
fth-order Butterworth lter.
For small cut-off frequencies, (almost) no aliasing is
caused leading to a small loss. If the 3 dB cut-off frequency
is larger than approx. 5 GHz, aliasing occurs and the loss
rapidly increases. For 3 dB cut-off frequencies larger than
approx. 6 GHz, LRA equalization outperforms LE.
Noteworthy, all results are valid for innite lter length,
i.e., neither the number of taps of the feedforward lter
F (z), nor the number of taps of the feedback lter in the
DFE are restricted. If this is done (as usual in practice)
especially for small cut-off frequencies (where more intersymbol interference in the end-to-end discrete-time response H(z) is caused) the loss increases, too, leading to
an optimum bandwidth, cf., e.g., [15]. This effect strongly
depends on the used front-end lter, the point where the
sampling takes place, and the used discrete-time lter approach.

500

1000

1500

2000

L [km]
Figure 4 Loss due to intersymbol interference over the ber
length for different equalization techniques. Transmit pulse: cosroll-off in time domain, t = 0.7; Receive lter: matched lter
w.r.t. transmit pulse.
LE
LRA
DFE

500

1000

1500

2000

10 log10 (() ) [dB]

10 log10 (() ) [dB]

L [km]
Figure 5 Loss due to intersymbol interference over the ber
length for different equalization techniques. Transmit pulse: cosroll-off in time domain, t = 0.4; Receive lter: matched lter
w.r.t. transmit pulse.

10 log10 (() ) [dB]

LE
LRA
DFE

10

f3 dB [GHz]

Figure 7 Loss due to intersymbol interference over the 3 dB

cut-off frequency of the fth-order Butterworth receive lter for
different equalization techniques. Transmit pulse: cos-roll-off in
time domain, t = 0.4; Fiber length: 1000 km.

LE
LRA
DFE

500

1000

1500

Numerical simulations (not shown here due to lack of

space) of the bit error rate are in very good agreement to
the theoretically derived performance measure.

2000

L [km]
Figure 6 Loss due to intersymbol interference over the ber
length for different equalization techniques. Transmit pulse: cosroll-off in time domain, t = 0.4; Receive lter: fth-order Butterworth lter with 3 dB cut-off frequency 5 GHz.

4.2

For to direct detection, the pulse shape and the parameters of the ber are the same as in the coherent case. However, only real-valued signaling is possible as only the receive power is evaluated.

5 GHz. For such a scenario, (LE) =

 1.095 dB and
(DFE) =
 0.905 dB can be calculated.

ISBN 978-3-8007-3938-7

Direct Detection

149

VDE VERLAG GMBH Berlin Offenbach

ITG-Fachbericht 257: Photonische Netze 07. 08.05.2015 in Leipzig

15
10
5

30

15
10
5

100

200

300

400

500

L [km]
Figure 8 Loss due to intersymbol interference over the ber
length for different equalization techniques. Transmit pulse: cosroll-off in time domain, t = 0.4; Receive lter: matched lter
for HT (f )HDD (f ).

200

300

400

500

LE
LRA
DFE

25
20
15
10
5
0

100

200

300

400

500

L [km]

As before, a fth-order Butterworth lter as receiver

front-end lter may be a practical solution. In Figs. 9
and 10, the loss over the ber length is shown for 3 dB
bandwidths of 4 GHz and 8 GHz, respectively. Moreover,
no clear/unique optimum of the receive bandwidth is visible. For LE, the higher bandwidth is advantageous, for
DFE (or equivalently THP or optimally loaded DMT) a
smaller bandwidth should be used. The advantages of
DFE/THP/DMT over LE are clearly visible. For the critical ber lengths derived above (dashed black vertical lines)
LE does not work stably as the present spectral zeros cannot be equalized linearly; this increasing loss has already
been observed, cf. [18]. DFE/THP and DMT with optimum loading (which omits the respective tones) are robust
against spectral zeros and work stably.
In direct detection schemes, LRA equalization can provide signicant gains (the integer polynomial is optimized
for each ber length individually). A performance close to
that of DFE/THP or DMT is provided. The operation of the
different equalization schemes is visualized in Fig. 11 for a
ber of length 140 km. In the top plot, the signal transfer
function H(e j2f T ) and the noise PSD nn (e j2f T ) after the receive lter (Butterworth lowpass lter with 3 dB
bandwidth 8 GHz) and sampling (cf. (11)) is shown. This
transfer function has to be equalized. In the second row
LE is assumed; the signal transfer function is equalized

ISBN 978-3-8007-3938-7

100

L [km]

20

20

Figure 9 Loss due to intersymbol interference over the ber

length for different equalization techniques. Transmit pulse: cosroll-off in time domain, t = 0.4; Receive lter: fth-order Butterworth lter with 3 dB cut-off frequency 4 GHz.

LE
LRA
DFE

25

LE
LRA
DFE

25

10 log10 (() ) [dB]

10 log10 (() ) [dB]

30

30

10 log10 (() ) [dB]

SNR Loss over Fiber Length

Again, we start with the discussion of the loss over the
ber length. In Fig. 8 the loss over the ber length is shown
when employing a matched lter for HT (f )HDD (f ). Although this receiver lter cannot be implemented in a
causal fashion, the losses can serve as reference for other
front ends. Even in this case, due to the equivalent channel HDD (f ), signicant losses occur. Compared to coherent detection, much higher losses are visible. The dashed
black vertical lines mark the critical ber lengths according to Tab. 1. As HT (f ) is not strictly bandlimited to
|f | < 1/T slight shifts in the critical lengths are noticeable. LRA equalization can provide gains; specically, the
unstable behavior of LE can be avoided.

150

Figure 10 Loss due to intersymbol interference over the ber

length for different equalization techniques. Transmit pulse: cosroll-off in time domain, t = 0.4; Receive lter: fth-order Butterworth lter with 3 dB cut-off frequency 8 GHz.

to one (Heq (e j2f T ) = F (e j2f T )H(e j2f T )) but at the

same time the PSD of the additive noise (assuming white
noise prior to the continuous-time receive lter) is boosted
(eq (e j2f T ) = |F (e j2f T )|2 nn (e j2f T )). When applying DFE (bottom row), the noise is whitened (constant
PSD); the non-constant signal transfer function is resolved
via the feedback loop. Using LRA schemes, equalization is
done w.r.t. the optimized target polynomial; in the present
case Zopt (z) = 1 + z 1 . As is clearly visible, much less
noise enhancement compared to LE is generated when using LRA equalization. Please note that for each ber length
the target polynomial has to be optimized individually; e.g.,
for 300 km we obtain Zopt (z) = 1 + z 4 .
Optimization of the Receive Filter
The optimization of the receive lter (3 dB bandwidth
of the Butterworth lter) is shown in Fig. 12 (solid: L =
140 km; dashed: L = 300 km). As can be seen, for smaller
ber lengths, smaller bandwidths are preferable; LE, LRA
equalization, and DFE/THP may require different optimum
bandwidths.

VDE VERLAG GMBH Berlin Offenbach

nn (ej2f T )

2
1
0

LE

30

10 log10 (() ) [dB]

H(ej2f T )

ITG-Fachbericht 257: Photonische Netze 07. 08.05.2015 in Leipzig

eq (ej2f T )

Heq (ej2f T )

1
0

LRA

2
1
0

DFE

0.1

25

10 log10 (() ) [dB]

20
15
10
5

100

200

300

400

500

L [km]
0.2

0.3

f T

0.4

0.5

0.1

0.2

0.3

f T

0.4

Figure 13 Loss due to intersymbol interference over the ber

length for different equalization techniques. Transmit pulse: cosroll-off in time domain, t = 0.4; Receive lter: fth-order
Butterworth lter with (individually per ber length/equalization
strategy) optimized cut-off frequency.

0.5

Figure 11 Visualization of signal transfer function and PSD

when performing equalization. Left: signal transfer function;
Right: noise PSD. Transmit pulse: cos-roll-off in time domain,
t = 0.4; Receive lter: fth-order Butterworth lter with 3 dB
cut-off frequency 4 GHz. L = 140 km. Top to bottom: signal transfer function (left) and PSD after the receive lter and
sampling (right); LE; LRA with target polynomial Zopt (z) =
1 + z 1 ; DFE.

LE
LRA
DFE

20

tion is shown in Fig. 14. From that, following closely the

derivations given in [14] but taking the additive noise into
account, the equivalent baseband system can be derived.
After pulse shaping of the real-valued data symbols a[k],
a constant s is added9 to s(t); this signal modulates the
laser (carrier frequency fL ). In the optical domain, we
hence have (with obvious denitions)


so (t) = s(t) + s 2 cos(2fL t)
= so (t) + so cos(2fL t) .
(25)

L = 140 km
L = 300 km

15

This (optical) signal is fed into the ber with (real-valued)

impulse response (in the optical domain) hC,o (t). From the
bandpass/lowpass transform a substitute (valid within the
desired transmission band, i.e., around fL for the optical
signals and around f = 0 for equivalent lowpass signals)
channel model is given by

10

10

f3 dB [GHz]

hC (t) = hC,o (t)ej2fL t

HC (f ) = HC,o (f + fL )

Figure 12 Loss due to intersymbol interference over the 3 dB

cut-off frequency of the fth-order Butterworth receive lter for
different equalization techniques. Transmit pulse: cos-roll-off in
time domain, t = 0.4; Fiber length: solid: 140 km; dashed:
300 km.

Finally, in Fig. 13, the loss over the ber length is shown
where the 3 dB bandwidth of the Butterworth receive lter
is optimized individually for each ber length and receive
strategy. LE fails for the critical ber lengths, whereas
all other strategies show good performance. LRA equalization is able to perform close to DFE/THP/optimally
loaded DMT but (via precoding of the integer part) can
be combined directly with channel coding and preserve the
discrete-amplitude nature of the transmit signal.

25

1
0

LE
LRA
DFE

Appendix

W.l.o.g. we assume that HC (0) = HC,o (fL ) = 1 (which

can be guaranteed by suitably scaling and delaying the
time-domain signal). The ber output signal is thus given
by
ro (t) = so (t) hC,o (t) + no (t) ,

(27)

where no (t) is real-valued white Gaussian noise with PSD

N0,o /2.
The detector operates on the optical signal (in the band
around fL ) and determines its instantaneous power; some
form of lowpass ltering and DC suppression is assumed to
be inherent to the power
detection. Assuming a strong cars  1), the relevant (ignoring the
rier component (
so = 2
DC offset and components at twice the carrier frequency
9 This is equivalent to adding a constant a
= s/GT (0) to the data
symbols, if GT (/T ) = 0, Z \ {0}, i.e., a DC offset in the discretetime sequence leads to a DC offset in the continuous-time signal but not
to harmonics of 1/T .

The block diagram showing the system-theoretical operations in the optical domain (index o) for direct detec-

ISBN 978-3-8007-3938-7

(26)

151

VDE VERLAG GMBH Berlin Offenbach

ITG-Fachbericht 257: Photonische Netze 07. 08.05.2015 in Leipzig

s
a[k]

T hT (t)
T HT (f )

2 cos(2fL t)

s(t)

so (t)

no (t), no no (f ) = N0,o /2
hC,o (t)
HC,o (f )

ro (t)

()2
s

r(t)

n(t), nn (f ) = N0 /2
s(t)

hDD (t)
HDD (f )

r(t)

Figure 14 System model (top: optical domain; bottom: equivalent baseband system) of the ber-optic transmission system using
direct detection.

and those weighted by 1/

so ) terms of the normalized instantaneous power ro2 (t)/
s read after some manipulations,
cf. [14]
 


r(t) = s(t) hC (t) + s(t) hC (t)

+ 2ej2fL t no (t) + 2ej2fL t no (t)

= 2Re{hC (t)} s(t) + n(t) .
(28)
Since no (t) is real-valued white Gaussian noise with PSD
N0,o /2 in the optical domain, the total noise in r(t) is realvalued and white with PSD 4 N0,o /2. Hence, the equivalent noise n(t) in (28) is real-valued white Gaussian with
PSD N0 /2 = 2N0,o .
In summary, the PAM signal s(t) obtained by pulse
shaping the data sequence a[k] with gT (t) experiences
the end-to-end dispersive channel with description (using
2
2
hC (t) ` HC (f ) = ejL 2 (2f ) )
hDD (t) = 2 Re{hC (t)}

HDD (f ) = 2 cos(L 22 (2f )2 ) .

(29)

Acknowledgment
The authors thank Werner Rosenkranz and Roi Rath from
the University of Kiel and Helmut Grieer from ADVA Optical Networking SE for valuable discussions.

References
[1] A. Dochhan, H. Grieser, L. Nadal, M. Eiselt, M. Svaluto
Moreolo, J.-P. Elbers. Discrete Multitone Transmission for
Next Generation 400G Data Center Inter-Connections. In
2014 OptoElectronics and Communication Conf. and Australian Conf. on Optical Fibre Technology, July 2014.
[2] A. Dochhan, H. Griesser, L. Nadal, M. Eiselt, M. Svaluto
Moreolo, J.-P. Elbers. Discrete Multitone Transmission in
the Presence of Optical Noise, Chromatic Dispersion and
Narrow-band Optical Filtering. In 15. ITG Symposium on
Photonic Networks, Leipzig, Germany, May 2014.
[3] R.F.H. Fischer, J.B. Huber. A New Loading Algorithm for
Discrete Multitone Transmission. In IEEE Global Telecommunications Conf., pp. 724728, London, UK, Nov. 1996.
[4] R.F.H. Fischer, J.B. Huber. On The Equivalence of Singleand Multicarrier Modulation: A New View. In International
Symposium on Information Theory, Ulm, Germany, pp. 197,
June 1997.
[5] R.F.H. Fischer. Precoding and Signal Shaping for Digital
Transmission, John Wiley & Sons, New York, 2002.

ISBN 978-3-8007-3938-7

152

[6] R.F.H. Fischer, C. Siegl. Lattice-Reduction-Aided Equalization for Transmission over Intersymbol-Interference Channels. IET Electronics Letters, vol. 41, pp. 969970, Aug.
2005.
[7] R.F.H. Fischer, C. Siegl. On the Relation between LatticeReduction-Aided Equalization and Partial-Response Signaling. In International Zurich Seminar, Zurich, Switzerland,
pp. 3437, Feb. 2006.
[8] M. Franceschini, G. Ferrari, R. Raheli, G. Bongiorni. Fundamental Limits of Electronic Dispersion Compensation in
Optical Communications with Direct Photodetection. IET
Electronics Letters, vol. 42, no. 15, pp. 874875, July 2006.
[9] K.-P. Ho, J.M. Khan. Sectrum of Externally Modulated
Optical Signals Journal of Lightwave Technology, vol. 22,
no. 2, Feb. 2004.
[10] E. Ip, J.M. Kahn. Power Spectra of Return-to-Zero Optical
Signals. Journal of Lightwave Technology, vol. 24, no. 2,
pp. 16101618, Mar. 2006.
[11] E. Ip, J.M. Kahn. Digital Equalization of Chromatic Dispersion and Polarization Mode Dispersion. Journal of Lightwave Technology, vol. 25, no. 8, pp. 20332043, Aug. 2007.
[12] P Kabal, S. Pasupathy. Partial-Response Signaling. IEEE
Transactions on Communications, vol. 23, no. 9, pp. 921
934, Sept. 1975.
[13] L. Nadal, M. Svaluto Moreolo, J.M. Fabrega, A, Dochhan,
H. Griesser, M. Eiselt, J.-P. Elbers. DMT Modulation With
Adaptive Loading for High Bit Rate Transmission Over Directly Detected Optical Channels. Journal of Lightwave
Technology, vol. 32, no. 21, pp. 41434153, Nov. 2014.
[14] H. Paul, K.-D. Kammeyer. Equivalent Baseband Channels of Systems Using Envelope Detection. International
Journal of Electronics and Communications (AE), vol. 63,
no. 7, pp. 533540, July 2009.
[15] R. Rath, W. Rosenkranz. Tomlinson-Harashima Precoding for Fiber-Optic Communication Systems. In European
Conf. on Optical Communications, paper We.2.C.2, London, United Kingdom, 2013.
[16] J. Wang, K. Petermann. Small Signal Analysis for Dispersive Optical Fiber Communication Systems. Journal of
Lightwave Technology, vol. 10, no. 1, pp. 96100, Jan. 1992.
[17] C. Windpassinger, R.F.H. Fischer. Low-Complexity NearMaximum-Likelihood Detection and Precoding for MIMO
Systems using Lattice Reduction. In IEEE Information Theory Workshop, pp. 345348, Paris, France, Mar./Apr. 2003.
[18] C. Xia. Advanced Electronic Distortion Equalization for
High-Speed Optical SMF and MMF Communications. Dissertation, University Kiel, 2008.
[19] H. Yao, G.W. Wornell. Lattice-Reduction-Aided Detectors for MIMO Communication Systems. In IEEE Global
Telecommunications Conf., Taipei, Taiwan, Nov. 2002.

VDE VERLAG GMBH Berlin Offenbach