To appear in Problemy Pereda

hi Informatsii.

M. Lentmaier, D. V. Truha hev, and K. Sh. Zigangirov

On the Theory of Low{Density

Convolutional Codes II.

1

Abstra t| A theoreti al and experimental analysis of iterative de oding of low{density

onvolutional (LDC) odes is given. Two families are investigated: homogeneous LDC
odes and the onvolutional ode version of turbo{ odes.
x1. INTRODUCTION

In the rst part of this work, published in [1℄, a mathemati al denition of low{density
onvolutional (LDC) odes was given, together with general methods to onstru t them. A
de oding algorithm was also presented. In the present se ond part we analyze the asymp-
toti properties of iterative de oding of LDC odes. In parti ular, we derive an upperbound
on the de oding bit error probability as a fun tion of the number of iterations and a bound
on the so alled iterative limit, introdu ed in [2℄ in onne tion with an investigation of Gal-
lager's low{density (LD) blo k odes [3℄. We onsider two parti ular families of LDC odes:
homogeneous LDC odes, whi h are the onvolutional analog of low{density blo k odes
( onstru tion A), and the onvolutional version of the famous turbo{ odes [4℄ ( onstru tion
B).
The main result of this paper is to prove the existen e of and derive bounds on iterative
limits for onstru tion A and onstru tion B odes with dierent parameters. The results
we obtain for odes of onstru tion B an easily be arried forward to the standard blo k
turbo{ odes and therefore are espe ially interesting.
In order to obtain the bounds we onsider a spe ial sequen e of LDC odes, fC k g1k , ( )
=1

and propose a de oding algorithm whi h is espe ially tailored to permit an asymptoti
analysis. The sequen e of odes is hosen su h, that for ode C k the a posteriori prob-
( )

abilities al ulated in the de oding pro ess are al ulated \ orre tly" during the rst k
de oding iterations (the denition of \ orre tness" in the al ulation of the a posteriori
1 This work was supported in part by Swedish Resear h Coun il for Engineering S ien es (Grant 98-216).

1
probabilities is given in [1℄). This allows us to estimate the bit error probability and show
that it goes to zero at least exponentially with the de oding iterations if the signal{to{noise
ratio of the hannel is above some level.
It is worth to mention that the proof of the existen e itself of odes, whose de oding error
probability de reases exponentially or almost exponentially with the de oding omplexity,
is not new. In the little{known paper of Zyablov and Pinsker [5℄, for example, the existen e
of binary odes, whose de oding error probability de reases exponentially with the ode
length N at a de oding omplexity of order N log N , was proved for all transmission rates
up to the hannel apa ity. This makes an asymptoti analysis of on rete LDC odes,
instead of an existen e proof only, espe ially attra tive.
The arti le is organized as follows: In x2 we present some notations and onne tions to
related previous papers. x3 and x4 are devoted to the analysis of LDC odes of onstru tion
A and onstru tion B, respe tively. x3.1 and x4.1 deal with the onstru tion of spe ial se-
quen es of odes for the two onsidered families. Using results given in [2℄, we formulate in
x3.2 the results of an asymptoti analysis of iterative de oding for homogeneous LDC odes
( onstru tion A). To obtain orresponding results for onvolutional turbo{ odes ( onstru -
tion B), whi h we present in x4.5, we introdu e in x4.2 a two{phase de oding algorithm.
The rst phase is analyzed numeri ally in x4.3, the se ond phase analyti ally in x4.4, using
the weight enumerator fun tion of the omponent en oders. Con lusions are nally given
in x5.
x2. PRELIMINARIES

We will shortly re all the main denitions from the rst part of this work, published in
[1℄. The ode sequen e
v = :::;v 1 ; v 0 ; v1 ; : : : ; v t ; : : : ; vt = vt ; : : : ; v t
( +1) 1 ; (1)
of a onvolutional ode is dened by the relation vH = 0, where H is an innite T T

transposed parity{ he k matrix, also alled syndrome former, and dened by formula (4)
from [1℄. A syndrome former of an LDC ode satises the onstraint (9) from [1℄. From
2
now on we assume that the sequen e v starts with the subblo k v , i.e. vt = 0 if t < 0, 0

that the onditions (5) and (6) from [1℄ are fullled, and that the rst b symbols of ea h
subblo k vt are information symbols. The ode is hara terized by it's syndrome former
memory m and rate R = b= .
s

In the syndrome former H of a homogeneous LDC ode ( onstru tion A) all rows
T

have equal Hamming weight, and the same applies for all olumns. One way to onstru t
syndrome formers H of homogeneous LDC odes is the Jimenez{Zigangirov method [6℄.
T

The syndrome former of a onvolutional turbo{ ode ( onstru tion B) has in general a
more ompli ated stru ture. In this work we will onsider a parti ular ase of onstru tion
B, where the en oder onsists of two parallel re ursive systemati binary onvolutional
en oders with memory m and rate R = 1=2. The resulting rate of the turbo{ ode is

R = 1=3. In this ase, the information sequen e u = (u ; u ; : : : ; ut; : : :), ut 2 f0; 1g, enters
0 1

the rst omponent en oder dire tly, the se ond en oder through a (single) onvolutional
permutor . A onvolutional permutor is an innite permutation matrix  = jji;j jj, having
2

exa tly one one in ea h row and olumn, while all other elements are zeros. The matrix 
also satises the ausality ondition, i;j = 0, i > j . The quantity Æ = max i;j 2Z
fj iji;j = 1g
is alled the delay of the permutor. In pra ti al appli ations permutors are often hosen
periodi with some period T , i.e. i;j = i T;j T , 8i; j 2 Z. If the sequen e u enters
+ +

the permutor, then the sequen e u0 = u appears at the permutor's output, whi h at the
same time forms the input sequen e of the se ond omponent en oder. The rst omponent
en oder generates the parity{ he k sequen e v = (v , v , : : :, vt , : : :), the se ond
(1) (1)
0
(1)
1
(1)

en oder the parity{ he k sequen e v = (v , v , : : :, vt , : : :). By denition, the ode

(2) (2)
0
(2)
1
(2)

sequen e of a onstru tion B LDC ode is the sequen e

 
v = u0 ; v ; v ; u1 ; v ; v ; : : : ; ut ; vt ; vt ; : : : ;
(1)
0
(2)
0
(1)
1
(2)
1
(1) (2)
(2)
2 The term \interleaver" is usually used in the te hni al literature to des ribe a devi e onsisting of two
tables whi h are lled and read alternately. We will denote by \permutor" a general devi e that hanges
the order of the symbols in the input information sequen e. We used the term \s rambler" in the rst
part of the work but sin e this term determines other devi es in ommuni ation theory and ryptography
we de ided to hange it, following the advi e of J. Massey.

3
where ut, vt and vt are symbols of the information sequen e, the parity{ he k sequen e
(1) (2)

of the rst en oder, and the parity{ he k sequen e of the se ond en oder, respe tively.
As in [2℄, we will onsider transmission over the binary symmetri hannel (BSC) and
the additive white Gaussian noise (AWGN) hannel. Let r denote the sequen e, re eived
after the hannel.
Consider iterative de oding of odes from an arbitrary ode family C , used for trans-
mission over the AWGN hannel. Let P (C; I ) denote the de oding bit error probability
b

if C 2 C is de oded using a total number of iterations equal to I . We demand from an

iterative de oding algorithm, that the following two onditions are satised:
1. the de oding omplexity per bit, C , grows at most linearly with the number of
b

iterations I , C = O(I ).
b

2. there exists a , su h that for any E =N greater than there exists a sequen e
b 0

of odes fC k g1k , C k 2 C , with orresponding de oding bit error probabilities

( )
=1
( )

fP (C k ; I = k)g1k , su h that P (C k ; I = k) ! 0 at least exponentially with I .

b
( )
=1 b
( )

The iterative limit SNR for the family C is the inmum over all su h 's and iterative
0

de oding algorithms.
For other hannels the iterative limit of a ode family an be dened analogously. In
this paper we will present bounds on the iterative limits for the ase of the AWGN hannel
and the BSC.
Iterative de oding of homogeneous LD blo k odes and blo k turbo{ odes is des ribed
in [2℄ and [7℄. The same prin iples an substantially be applied to iterative de oding of
LDC odes. One signi ant dieren e is, that the re urrent nature of onvolutional odes
en ourages the use of a pipeline realization, whi h ontinuously operates on the streaming
ode sequen e. In this ase, the de oding iterations an be performed in parallel by inde-
pendent pro essors. A pipeline realization for onstru tion A is des ribed in [6℄ and [1℄.
For onstru tion B a pipeline realization an be obtained in an analogous manner. In this
latter ase, the pro essors use either the BCJR algorithm [8℄ for the sequential al ulation
of the a posteriori probabilities of the information symbols or the \max{path" algorithm
4
[7℄. As in [7℄, we investigate the appli ation of a modi ation of the BCJR algorithm,
operating on a sliding window of size (2
+ 1). It an be shown, that the performan e
of this window de oder for
! 1 approa hes the performan e of the standard BCJR
de oder.
For the investigation of an iterative de oding algorithm for homogeneous LD blo k odes
in [2℄, a tree{like graph representation of the odes was used. Analogously, the de oding
of blo k turbo{ odes was studied using a
{graph [7℄, where the value
was determined
by the size of the de oding window. Sin e the graph{representation of onstru tion A and
onstru tion B LDC odes does not dier from the representation in the papers [2℄ and [7℄,
we will leave it out here.
A ru ial role within our analysis of iterative de oding of low{density odes has the
on ept of l {nondegeneration ([2℄ and [7℄). In an l {nondegenerated ode, all symbol
0 0

nodes in the de oding tree up to the l th generation represent dierent ode symbols. For
0

su h a ode, the a posteriori probabilities al ulated by the iterative de oder stay \ orre t"
during the rst l iterations.
0

x3. ANALYSIS OF CONSTRUCTION A LDC CODES

x3.1 Design of l0 {Nondegenerated Codes

In this se tion we will show, that for any positive integer l it is possible to onstru t an
0

l {nondegenerated homogeneous LDC ode, for whi h the a posteriori probabilities, al u-

0

lated by the iterative de oder, stay \ orre t" during the rst l iterations. This immediately
0

implies, that we an onstru t a sequen e fC k g1k , where C k is a k{nondegenerated LDC

( )
=1
( )

ode. The existen e of su h a sequen e of odes is of great importan e in the analysis of

iterative limits of LDC odes, given in x3.2.
The onditions, needed for the al ulated a posteriori probabilities to stay \ orre t",
an be represented in form of onstraints on the graph des ribing the LDC ode. A or-
responding analysis for homogeneous low{density (LD) blo k odes was done by Gallager
[3℄, who obtained upper and lower bounds for the maximal number of \ orre t" de oding

5
iterations. The lower bound was later strengthened by Margulis [9℄[10℄[11℄. The results of
Gallager and Margulis were used in [2℄ for an asymptoti analysis of iterative de oding of
LD blo k odes. In the following we derive orresponding results for LDC odes.
Consider a homogeneous blo k LD (N; J; K ){ ode, dened by the transposed parity{
he k matrix He = jjehi;j jj, i = 0; 1; : : : ; N 1, j = 0; 1; : : : ; NJ=K 1. Assume that N is
T

a multiple of some positive integer > 1.

Denition 1 A y le of length l in the transposed parity{ he k matrix He T of a homo-
geneous blo k LD (N; J; K ){ ode is a sequen e of indi es i1 , j1 , i2 , j2 , : : :, il , jl , 0
ik  N 1, 0  jk  NJ=K 1, k = 1; : : : ; l, su h that all elements in ea h of the sets
fi ; i ; : : : ; il g and fj ; j ; : : : ; jl g are dierent and ehik ;jk = 1; ehik ;jk+1 = 1; k = 1; : : : ; l 1,
1 2 1 2

hi ;j
e
l l = 1; ehi1;j = 1.
l

A y le in the syndrome former of a homogeneous LDC ode an be dened analogously.

Let us hoose a matrix He onsisting of KJ square N=K  N=K submatri es Hei;j ,
T

0 1
B H0;0
e He 0;1


He ;J0 1
C
B C
B He He 1;1


He ;J C
He T = B 1;0 C
(3)
1 1
B ... ... ... C ;
B C
B C
 A
He K 1;0 He K 1;1


HeK 1;J 1

and satisfying the following onditions:

1. All elements in the main diagonal and the diagonal below of the submatri es Hei;i; i =
0; : : : ; J 1 are ones, and other elements of these submatri es are zeros.
2. The element in the upper right orner of the submatri es Hei ;i; i = 0; : : : ; J 2 +1

and He ;J is one, and other elements of these submatri es are zeros.

0 1

3. All other submatri es are permutations of the N=K  N=K identity matrix.
Lemma 1 For any positive integers l, J , K and N , su h that K > J > 1 and


N  4K
( K 1)(J 1) l
; (4)
KJ K J
6
there exists a matrix He T without y les of length less than l + 1, satisfying the three on-
ditions, listed above.

A proof of this lemma an be obtained by analyzing a pro edure whi h lls the subma-
tri es with permutations of the identity matrix without forming y les of length less than
l + 1. The pro edure is analogous to Gallager's pro edure [3℄. The only dieren e to the
problem, onsidered by Gallager, is that the submatri es Hei;i; i = 0; : : : ; J 1 ontain
T

two ones, instead of one, in ea h olumn and ea h row (ex ept the rst row and the last
olumn). Therefore, the bound on the required size of the matrix should be doubled, sin e
the maximal number of una eptable positions in a row and in a olumn of a submatrix is
two times as large.
Consider two positive integers b and su h that b= = (K J )=K , and onstru t
another N  NJ=K transposed parity{ he k matrix Hb = jjbhi;j jj, i = 0; 1; : : : ; N 1,
T

j = 0; 1; : : : ; NJ=K 1, of a blo k LD (N; J; K ){ ode by permuting the rows of the matrix

He . Let us denote by q (j ) and r(j ) the quotient and remainder from the division of j by
T

. Let h
e j denote the j th row of the matrix H e . The j th row of the matrix H
T b is dened
T

by 8
>
q j b r j ; if r (j ) < b ;
>
<heN
hj = (5)
1 ( ( ) + ( ))
b
>
:hq j b r j b ; if r (j )  b :
> e( )( )+ ( )

Lemma 2 Assume that the matrix He T does not ontain y les of length less than l; then
the matrix Hb T , onstru ted from He T by a permutation of rows, does not ontain y les of
length less than l, either.

To prove this lemma it is enough to noti e, that a y le of length k maps again into a y le
of length k after a permutation of the rows of the matrix.
Using the Jimenez{Zigangirov pro edure [6℄ we transform the matrix Hb into a syn-
T

drome former of a periodi homogeneous LDC (m ; J; K ){ ode, having rate R = b= and

s

period m + 1 = N= . The matrix Hb onsists of submatri es of size  ( b). We ut

s
T

it into two parts, as shown in Figure 1 (a), and then move the lower part as shown in

7
Figure 1 (b). Completing the rows by zeros we get one fragment of the innite matrix H T

of the homogeneous LDC (m ; J; K ){ ode. Repeating this fragment periodi ally we obtain
s

the whole matrix H . T

As expe ted, this matrix denes a homogeneous LDC (m ; J; K ){ ode of rate R = b= . s

The submatri es H (t) are in the form

T
0

0 1
Q(t)
H0T (t) =  A ; (6)
P
where Q(t) are arbitrary b  ( b) matri es, all the elements in the main diagonal and the
diagonal below of the ( b)  ( b) matrix P are equal to one, and all other elements
in P are zeros. Therefore rank H (t) = b; t 2 Z, and the last b rows of H (t) are
T
0
T
0

linearly independent. The submatri es Hms (t) are nonzero, sin e they ontain a one in the
T

position (b; b 1). The matrix H ontains exa tly J ones in ea h row and K ones in
T

ea h olumn.
We denote by r (i) the remainder from the division of i 2 Z by (m + 1) and by r (i)
1 s 2

the remainder from the division of i 2 Z by (m + 1)( b). The following simple lemma
s

is valid.
Lemma 3 For any i; j 2 Z su h that hi;j = 1, we have bhr1 (i);r2 (j ) = 1.

The following two lemmas are less trivial.

Lemma 4 Assume that the matrix Hb T does not ontain y les of length less than l; then
the syndrome former H T of the rate R = b= , period ms +1 homogeneous LDC (ms ; J; K ){
ode, onstru ted from Hb T using the pro edure, des ribed above, does not ontain y les of
length less than l, either.

Proof: Assume that the matrix H ontains a y le i ; j ; i ; j ; : : : ; ik ; jk , of length k

T
1 1 2 2

(k < l). Consider the sequen e r (i ), r (j ), r (i ), r (j ), : : :, r (ik ), r (jk ). It follows

1 1 2 1 1 2 2 2 1 2

from Lemma 3 that bhr1 i ;r2 j = 1, bhr1 i ;r2 j +1 = 1, p = 1; : : : ; k 1 and bhr1 i ;r2 j = 1,
( p) ( p) ( p) ( p ) ( k) ( k)

hr1 i1 ;r2 j = 1. If all elements in ea h of the sets fr (i ), r (i ), : : :, r (ik )g and fr (j ),

b ( ) ( k) 1 1 1 2 1 2 1

8
r2 (j2 ), : : :, r2 (jk )g are dierent, the onsidered sequen e forms a losed y le of length k < l
in the matrix Hb . This is in ontradi tion with the onditions of the lemma. If there are
T

ommon elements in these sets then it is possible to hoose a sub{sequen e r (iq ), r (jq ), 1 2

r (iq ), r (jq ), : : :, r (in ), r (jn ), su h that all elements in the sets fr (iq ), r (iq ), : : :,
1 +1 2 +1 1 2 1 1 +1

r (in )g and fr (jq ), r (jq ), : : :, r (jn )g are dierent and bhr1 i ;r2 j = 1. This is a y le
1 2 2 +1 2 ( q) ( n)

of length n q + 1 < l, whi h is in ontradi tion with the onditions of the lemma. The
lemma is proved.
Lemma 5 If the syndrome former H T of a homogeneous LDC (ms ; J; K ){ ode does not
ontain y les of length less than 2l0 + 1, the orresponding ode is l0 {nondegenerated.

The proof of Lemma 5 is the same as the proof of Lemma C.1 in [3℄.
We will formulate the main result of this paragraph in form of a theorem:
Theorem 1 For any positive integers ms , J , K , b, and , su h that > b, J > 1,
b= = (K J )=K , and (ms +1) is a multiple of K , there exists an l0 {nondegenerated rate
R = b= homogeneous LDC (ms ; J; K ){ ode, where l0 satises

l >
log ( m + 1) + log KJ KK J
2
1:s 2
(
4
)

(7)
2 log (K 1)(J 1)
0
2

Proof: Consider the positive integer N = (m + 1) . Choose l su h that s 0

4K ( K 1)(J 1) l0
 N < 4K ( K 1)(J 1) l0
2

:
2 +2

(8)
KJ K J KJ K J
It follows from Lemma 1 that there exists a blo k LD (N; J; K ){ ode, dened by the
transposed parity{ he k matrix He , whi h does not ontain y les of length less than
T

2l + 1, and satises the onditions 1{3. Using the pro edure des ribed above, we an
0

onstru t a syndrome former H of a rate R = b= homogeneous LDC (m ; J; K ){ ode

T
s

from the matrix He . Lemma 4 states, that the matrix H does not ontain y les of
T T

length less than 2l + 1, either. It follows from Lemma 5 that the onstru ted LDC ode
0

is l {nondegenerated. From the right hand side of inequality (8) we nd that l satises
0 0

(7). The theorem is proved.

9
 x3.2 Iterative Limits of Constru tion A LDC Codes

The analysis of iterative de oding for onstru tion A is very similar to the orresponding
analysis for LD blo k odes, as given in [2℄ and [12℄. Instead of the Gallager{Margulis
theorem, used in [2℄, we rely on Theorem 1 from the previous se tion, when analyzing
onstru tion A. This theorem states that it is possible for any positive integer l to onstru t
0

an l {nondegenerated LDC ode if the syndrome former memory m is hosen suÆ iently
0 s

large. The tree, whi h is used to analyze the de oding of a parti ular symbol, has the
same stru ture for both blo k and onvolutional LD odes. Therefore, we an apply the
same analysis for blo k and onvolutional odes and as a result the bounds on the iterative
limits for homogeneous LD blo k odes are also valid for onstru tion A LDC odes.
Unfortunately, we do up to now not know an analyti al way to exa tly analyze Gal-
lager's iterative de oding algorithm [3℄. Even if a relatively a urate analyti al estimation
of the algorithm's performan e would be found, the bounds on the iterative limit ould be
too rough. Therefore, the only pra ti al method for the analysis involves numeri al al-
ulations, whi h do not allow an asymptoti analysis. For this reason, in [2℄ a two{phase
de oding algorithm was introdu ed. Consider transmission over the BSC with rossover
probability p. In the rst phase (the rst I iterations) the de oder performs soft de od-
1

ing, in the se ond phase hard majority de oding. From the analysis of the rst de oding
phase we an numeri ally al ulate the bit error probability "I1 (p) after I iterations. An
1

analysis of the se ond de oding phase shows, that there exists some value " (J; K; p) r

su h that when l ! 1 (and, onsequently m ! 1) the de oding error probability P

0 s b

of l {nondegenerated LDC odes is going to zero at least exponentially with l = I , if

0 0

" (J; K; p)  "I1 (p).

r

From this analysis we an obtain lower bounds p on the iterative limit p of the BSC.
0 0

Analogously we an get upper bounds SNR on the iterative limit SNR of the AWGN
0 0

hannel.
Simulation results of the de oding bit error probability for onstru tion A LDC odes
for transmission over the BSC and AWGN hannel are given in Figure 2 and Figure 3,

10
respe tively. The syndrome former memory of the LDC odes varies from 129 to 2049
for (3; 6){ odes and from 129 to 513 for (4; 8){ odes. The orresponding bounds on the
iterative limits are shown as verti al dashed lines.
x4. ANALYSIS OF CONSTRUCTION B LDC CODES

x4.1 Design of (l0 ;
){Nondegenerated Codes

Analogously to x3.1 we will show in this se tion, that for any arbitrary large number
l and any
we an onstru t a onvolutional turbo{ ode suitable for I = l \ orre t"
0 0

de oding iterations in a window of size 2
+ 1. The existen e of a sequen e fC k g1k of ( )

=1

su h odes, where C k is (l ;
){nondegenerated with l = k, is further used in the analysis

( )
0 0

of iterative limits of onvolutional turbo{ odes.

In [7℄ a spe ial method of analysis, developed by Gallager for homogeneous low{density
blo k odes, was extended in order to both analyze and onstru t blo k turbo{ odes.
Parti ularly, it was shown that for blo k turbo{ odes the number of " orre t" de oding
iterations is growing logarithmi ally with the length of the ode if the permutor is hosen
properly. In this paragraph we extend those results to onvolutional turbo{ odes.
First we onsider a size N permutor of a blo k turbo{ ode, analogous to those studied
in [7℄, des ribed by the matrix e = jj~i;j jj; i; j = 0; : : : ; N 1. Let us introdu e a fun tion
(i) = j , su h that i; i = 1. We introdu e the Lee metri on the set N = f0; 1; : : : ; N 1g
( )

of indi es of symbols un: d (i; j ) = min(ji j j; N ji j j), i; j 2 N .

L

Denition 2 For any even l we all a sequen e  = i1 , i2 , j2 , j3 , i3 , i4 , : : :, jl , j1 ,

0  ik ; jk  N 1, k = 1; : : : ; l, su h that all ik are dierent and jk = (ik ); k = 1; : : : ; l,
a y le of length l in the permutor matrix  e.

For a y le  we onstru t a sequen e of distan es (in Lee metri ): d1 = d (i ; i ), d =

L 1 2 2

d (j ; j ), d = d (i ; i ), : : :, dl = d (jl ; j ).
L 2 3 3 L 3 4 L 1

Denition 3 Assume that  is a y le of length l in the permutor matrix e and the

sequen e of distan es fdk glk=1 orresponds to  . Assume 9k1 ; k2 2 f1; : : : ; lg, su h that
11
1  dk1 ; dk2  2
and that 1  dk 
, 8k 2 f1; : : : ; lg n fk ; k g. Then  is alled a rst
1 2

type
{ y le of length l in the permutor matrix 

e.

Denition 4 Assume that  is a y le of length l in the permutor matrix e and the

sequen e of distan es fdk glk orresponds to  . Assume 9k 2 f1; : : : ; lg su h that 2
<
=1 1

dk1  3
, and that 1  dk 
, 8k 2 f1; : : : ; lg n fk g. Then  is alled a se ond type

1

{ y le of length l in the permutor matrix e .

Lemma 6 ([7℄) For any positive integers
, l0 and N , satisfying the inequality

16l (2
) l0  N
3
0
2
; (9)
it is possible to onstru t a permutor matrix 
e , whi h does not ontain rst type
{ y les

of length less than 2l0 + 1 and se ond type
{ y les of length less than l0 .

Let us onstru t a (single) onvolutional permutor  with period T = N , again using

the Jimenez{Zigangirov pro edure [6℄. Cutting the matrix e below the main diagonal,
moving the lower part and ompleting all rows by zeros we get one fragment of a (single)
onvolutional permutor matrix . Repeating this fragment periodi ally we obtain the
whole matrix . We noti e that the delay of the onstru ted permutor is Æ  T .
We denote by i the remainder from the division of i 2 Z by T . It is easy to he k the
statements of the following lemma.
Lemma 7 i) For any i; j 2 Z : i;j = 1 ) ~i;j = 1.

ii) For any i; j 2 Z : ji j j 
) dL (i; j ) 
.

Consider some (un;
){ lan of a onvolutional turbo{ ode, having a permutor  with
period T . Consider two arbitrary verti es of the lan. Let i and i0 denote the indi es of
symbols in the sequen e u = (: : :, u , u , : : :, un, : : :), asso iated with these verti es. If i
0 1

and i0 are not ongruent modulo T for any two verti es of the lan up to the lth generation,
the (un;
){ lan is alled nondegenerated in the stri t sense up to the lth generation.

12
 A lan is said to be degenerated in the stri t sense in the lth generation, if it is non-
degenerated in the stri t sense up to the (l 1)th generation and there exist two verti es
in the kth and lth generation (k  l), asso iated with the symbols ui and ui , su h that
k l

ik  il (mod T ).
A ode is alled (l ;
){nondegenerated in the stri t sense if the lans of all symbols of
0

the ode are nondegenerated in the stri t sense up to the l th generation. If a turbo{ ode
0

is (l ;
){nondegenerated in the stri t sense, then it obviously is (l ;
){nondegenerated.

0 0

Lemma 8 Assume that a periodi permutor matrix  with period T of a onvolutional

turbo{ ode is onstru ted using the pro edure des ribed above from a T  T permutor matrix
e of a blo k turbo{ ode. Assume that the matrix e does not ontain rst type
{ y les of
length less than 2l +1 or se ond type
{ y les of length less than l . Then the onvolutional
0 0

turbo{ ode des ribed by the matrix  is (l ;
){nondegenerated in the stri t sense.
0

The proof of Lemma 8 is given in Appendix. Lemma 6 and Lemma 8 imply

Theorem 2 For any positive T and
there exists an (l0 ;
){nondegenerated onvolutional
turbo{ ode having a permutor with period T and delay Æ  T , su h that

l0 
1 (log T 3 log (log T ) 4) 1 : (10)
2 log (2
)
2
2 2 2

x4.2 A Two{Phase Iterative De oding Algorithm

The iterative de oding algorithm for LDC odes of onstru tion B mostly oin ides
with the iterative de oding algorithm for blo k turbo{ odes [7℄. It omprises the sequen-
tial al ulation of a posteriori probabilities for information symbols using a symbolwise
de oding algorithm for the omponent onvolutional odes.
Figure 4 shows simulation results of the de oding bit error probability for onstru tion
B LDC odes in omparison with onventional blo k turbo{ odes. The blo k turbo{ odes
are based on blo k permutors of size N = 500, 1000 and 2000. The LDC odes are based
on onvolutional permutors, onstru ted from these blo k permutors like des ribed in x4.1.
13
 Like in ase of onstru tion A odes, whi h we onsidered in the previous paragraph, the
authors do not know any eÆ ient way to analyze this pra ti ally used algorithm without
numeri al methods. Sin e numeri al methods annot give results about the asymptoti
behavior of the algorithm, we introdu e a two{phase algorithm, as we did in our analysis
of onstru tion A. In the rst de oding phase (the rst I iterations) the omponent odes
1

are de oded by the onventional window BCJR algorithm, as des ribed in [7℄, operating
on a sliding window of size 2
+ 1.
In the se ond de oding phase, we use a suboptimal de oding algorithm, alled \max{
path" algorithm, also des ribed in [7℄. In the rst iteration of phase two, i.e. iteration
I + 1, the de oder uses the a posteriori probabilities resulting from the last step of the
1

rst de oding phase, but then it operates with the probabilities al ulated by the \max{
path" de oder. After the last iteration step the obtained statisti s are used to make a
de ision on the information symbols. The omponent de oder in the se ond phase an
be seen as a suboptimal version of the BCJR de oder and the two{phase algorithm gives
a pessimisti estimation of the performan e of an algorithm that uses the rst de oding
phase through all iterations.
The advantage of the two{phase algorithm is the possibility to use analyti al methods
to obtain asymptoti de oding performan es.
As it was shown in [7℄, the al ulated a posteriori probabilities are \ orre t" up to the
l th iteration if the de oder operates with a onstru tion B (l ;
){nondegenerated ode.
0 0

Theorem 2 implies the existen e of an (l ;
){nondegenerated LDC ode of onstru tion

0

B with syndrome former memory m , su h that l grows at least logarithmi ally with m .
s 0 s

In the remaining part of this paragraph we will analyze the two{phase iterative de oding
algorithm operating on an (l ;
){nondegenerated onstru tion B LDC ode, where the
0

number of iterations I equals l . In this ase all probabilities al ulated by the de oder are
0

\ orre t" probabilities.

To avoid doubling, we onsider only the AWGN hannel in detail and present for the
BSC only the results of the analysis, together with some omments. We start with the

14
analysis of the rst phase.
x4.3 Analysis of the First De oding Phase

In this se tion we des ribe a numeri al method to analyze the rst de oding phase. As
a result of this analysis we will obtain the Bhatta haryya parameter of the information
symbols after the rst phase, whi h will be used in x4.5 to evaluate bounds on the iterative
limits.
When de oding the symbol un, in the rst phase (the rst I iterations) the de oder
1

operates in the lower part of the tree, des ribing the (un;
){ lan, namely generations
l = l ; : : : ; l I . Let Ml denote the set of indi es of symbols within the lth generation
0 0 1

of the (un;
){ lan and R (um) denote the set of re eived symbols orresponding to all
1

des endants of symbol um , m 2 Ml , in the lan and the related parity{ he k symbols, up
to the l th generation. The de oder re ursively al ulates the log{likelihood ratios
0



P um = 0jR (um ); rm ; r~m P R (um ); rm ; r~m jum = 0
zm = ln 1

= ln
1

; (11)
P um = 1jR (um ); rm ; r~m
1 P R (um ); rm ; r~m jum = 1
1

for information symbols um , m 2 Ml, where l = l 1; l 2; : : : ; l I is de reased after

0 0 0 1

ea h iteration. Here rm and r~m denote the re eived symbols orresponding to um itself and
the related parity{ he k symbol, respe tively. By denition, the log{likelihood ratio for
symbols within the l th generation is equal to the intrinsi information, i.e.
0

P (r ju = 0)
zm = ln m m = m ; 8m 2 Ml0 :
def
(12)
P (rm jum = 1)
To obtain the log{likelihood ratios zm for the other levels, in ea h iteration a windowed APP
omponent de oder al ulates the values zm, m 2 Ml, from the intrinsi informations m
and the log{likelihood ratios of the hildren of um. In the rst iteration, where l = l 1,
0

the latter are given by (12), and for l < l 1 they result from the previous iteration.
0

Let f l (z) and f l (z) be the probability density fun tions of the log{likelihood ratio zm
( )
0
( )
1

obtained in the (l l)th iteration, l = l 1; : : : ; l I , given that um = 0 and um = 1,

0 0 0 1

respe tively. Be ause of the symmetry of the problem

f0(l) (z ) = f1(l) ( z ) ; (13)
15
and it is suÆ ient to regard the probability density fun tion f l (z), l = l 1; : : : ; l I , ( )
0 0 0 1

only. Moreover, sin e the ode is linear we an, for the analysis of the de oding error
probability, without loss of generality, onsider the ase where the all{zero sequen e was
transmitted over the hannel, i.e. u = v = v = 0. (1) (2)

In our investigation of the rst de oding phase we use a Monte Carlo te hnique. First
we al ulate an estimate f^ l0 (z) of the density fun tion f l0 (z) for the log{likelihood
(
0
1) (
0
1)

ratios zm , m 2 Ml0 , after the rst iteration step . This estimate is obtained from a
1
3

histogram of independently drawn out omes of the random variable zm , whi h are ob-
tained by applying the omponent APP window de oder to a sequen e of independent
randomly hosen input log{likelihood ratios generated a ording to the distribution given
by the hannel. Then, using f^ l0 (z), we an by the same prin iple obtain the estimation
(
0
1)

f^ l0 (z ) of the probability density fun tion for zm , m 2 Ml0 , after the se ond itera-
(
0
2)
2

tion, and so on. Finally, we get the estimate f^ l0 I1 (z) for the log{likelihood ratios zm ,
(
0
)

m 2 Ml0 I1 , after the I th iteration. 1

In two{hypothesis testing problems, the Bhatta haryya parameter plays an important

role. Parti ularly, we use in our analysis of the se ond de oding phase a Monte Carlo
estimation of the Bhatta haryya parameter DI1 for the distribution of zm after the I th 1

step of the rst phase of the de oding,

Z1  1 Z1  1
DI1 = (z )f (z ) dz = (z )f ( z) (14)
(l 0 I1 ) (l0 I1 ) 2 (l 0 I1 ) (l 0 I1 ) 2
f 0 1 f 0 0 dz :
1 1
The behavior of DI1 as a fun tion of the number of iterations I in the rst de oding phase 1

is shown in Figure 5 for a onstru tion B LDC ode with memory m = 3 omponent

onvolutional en oders and several values of E =N . We will show in the following, that
b 0

if DI1 falls below some riti al value D = D (E =N ), then the bit error probability P
r r b 0 b

goes to zero at least exponentially with the number of de oding iterations when l ! 1. 0

x4.4 Analysis of the Se ond De oding Phase

In our study of the se ond de oding phase we onsider a generalization of a well known
3 The density fun tions are, be ause of the symmetry in the tree, independent on the index m
16
 hara teristi of onvolutional odes, known as the path weight enumerator [13℄. With help
of this, we derive an upper bound on the Bhatta haryya parameters of the information
symbols after the dierent de oding iterations. In x4.5 we will use this bound to formulate
onditions under whi h the bit error probability P of the two{phase algorithm is de reasing
b

during the iterations.

Suppose, that some statisti s m orresponding to symbols of the output sequen e of
a onvolutional en oder enter the de oder. These m an, for example, be the output
symbols rm of the hannel, but in general they an also in lude information obtained from
somewhere else. Let us assume now, that the statisti s m for information and parity{ he k
symbols have dierent probability density fun tions. Let fu; ( ) and fu; ( )=fu; (  ) de-
0 1 0

note the onditional probability density fun tions of the statisti m orresponding to infor-
mation symbols, onditioned on um = 0 and um = 1, respe tively. A ordingly, let fv; ( ) 0

and fv; ( )=fv; (  ) denote the onditional probability density fun tion of the statisti s
1 1

orresponding to parity{ he k symbols. The Bhatta haryya parameters are dened as

 1   Z1

1
U = E e 2 zu um =0 = fu;0 ( )fu;1( ) 2 d (15)
1
 1   Z1

1
V = E e 2 zv vm =0 = fv;0 ( )fv;1( ) 2 d ; (16)
1
where the log{likelihood ratios zu and zv are dened as
f ( ) f ( )
zu = ln u; 1
zv = ln v; : 1
(17)
fu; ( )
0 fv; ( ) 0

Consider two binary ode sequen es whi h dier in du information symbols and dv

parity{ he k symbols. Let pd ;d be the pairwise probability of error when one of them is
u v

transmitted over a memoryless hannel and the de ision is made on base of the statisti s
 , having dierent probability density fun tions for information and parity{ he k symbols.

Lemma 9 The probability pdu ;dv is bounded by the inequality

pdu ;dv < U du V dv ; (18)

where U and V are the Bhatta haryya parameters dened by (15) and (16).

17
Proof: Assume, that one of the ode sequen es of length du + dv = n was transmitted over
the hannel, su h that the rst du symbols are information and the last dv are parity{ he k
symbols. Let z = (z ; : : : ; zn), z = z ( ), denote the log{likelihood ratios orresponding to
1

the re eived statisti s  = ( ; : : : ; n). A ne essary ondition for an error is

1

L(z ( )) =
du
Y fu;1 i ( ) Y n
fv;1 (i )
1 : (19)
i=1
fu; (i ) i d
0
= u +1
f v; 0 (i )

Using Cherno bounding arguments we get

Z1
1

pdu ;dv < L 2 z ( ) fu;0 (1)


fu;0 (du )
fv;0 (du +1 )


fv;0 (n) d1


dn
1
0 1 1du 0 1 1dv
Z Z
1 1 1 1
=  fu;2 0 (  ) u; ( )
f2 0 d A  fv;2 0 (  ) v; ( )d A = U d V d
f2 0
u v
; (20)
1 1
whi h nishes the proof of the lemma.
Consider an arbitrary rate 1=2 systemati re ursive onvolutional en oder with ex-
tended path enumerator T (X; Y; Z ), where X , Y and Z are dummy variables to generate
the total weight, the length and the information weight of ode sequen es, respe tively.
Then 
 XX
T (X; Y; Z )  = a(du; dv )(XZ )d X d ; (21) u v

Y =1 du dv
where a(du; dv ) is the number of paths with information weight du and parity{ he k weight
dv . We all a(du ; dv ) the modied spe trum of the en oder. If the distributions of the \re-
eived" information and ode symbols are hara terized by the Bhatta haryya parameters
U and V , respe tively, then the burst error probability P and the bit error probability P
B b

are upperbounded by the inequalities

 
 

T (X; Y; Z ) 
PB < T (X; Y; Z ) 
 X=V ; Pb < Z 
 X=V : (22)

 Y =1 Z 
 Y =1
Z = U=V Z = U=V

Let us suppose that this en oder is used as a omponent en oder of a onstru tion B
LDC ode, used over the AWGN hannel. Then V = exp( RE =N ) and U is a fun tion of b 0

the iteration number and depends on both the intrinsi and extrinsi information. The path
18
enumerator T (X; Y; Z ) des ribes the behavior of a Viterbi de oder operating on an innite
trellis. Sin e the omponent de oder operates on a nite trellis with 2
+ 1 se tions it's
behavior is, stri tly speaking, not expressed by the fun tion T (X; Y; Z ). Nevertheless, it is
possible to prove, that the performan e of the window de oder onverges to the performan e
of a de oder without window restri tion if
! 1. Further on we assume that T (X; Y; Z )
des ribes with suÆ ient exa tness the behavior of a Viterbi de oder operating on a nite
but long enough trellis.
Now we will use the extended path weight enumerator to analyze the windowed max{
path algorithm [7℄. This algorithm al ulates the probability of the most probable path in
the
{trun ated trellis orresponding to um = 0 and the probability of the most probable
path orresponding to um = 1. The ratio of these two probabilities is an approximation of
the likelihood ratio for the symbol um, al ulated by the BCJR algorithm.
Let um, m 2 Ml , be an arbitrary symbol in the lth generation, l 2 f0; : : : ; l I 1g. 0 1

Let V (um) be the set onsisting of symbol um, all its des endants up to the (l I 1)th
2 0 1

generation and the related parity{ he k symbols, R (um) be the set of re eived symbols
2

orresponding to V (um), and R (um) be the set of re eived symbols orresponding to the
2 1

des endants of um in generations l ; l 1; : : : ; l I and the related parity{ he k symbols.

0 0 0 1

Let p(R (um); R (um)jV (um)) denote the probability density fun tion for the re eived sets
1 2 2

R (um) and R (um) onditioned that the set V (um) was transmitted. Let
1 2 2

p1R (um); R (um) =
1 2 max p R (um); R (um)jV (um)
V2 u u ( m ): m =1
1 2 2 (23)



p R (um ); R (um ) = max p R (um ); R (um )jV (um ) (24)
0 1 2
V2 u u ( m ): m =0
1 2 2

p R (um ); R (um )
zm = ln 1 1


2
(25)
p R (um ); R (um )
0 1 2


p R (um ); R (um )
z~m = ln 1 1

;
2
(26)
p R (u ); R (u )j0
1 m 2 m

where 0 is the all{zero sequen e and the maxima are taken over all possible sets V (um) 2

with um = 0 and um = 1, respe tively. Obviously, z~m  zm , and

 1   
DI l = Um = E
def def
e 2 zm  E e 21 zm = U~m ; m 2 Ml :
~ def
(27)
19
 The expe tations in (27) are al ulated onditioned on the ase where the all{zero
sequen e is transmitted. Formula (27) generalizes the denition of the Bhatta haryya
parameter (15) from the ase where the de ision statisti s were given by (17) to the ase
with de ision statisti s (25) and (26).
Let us rst onsider the rst iteration of the se ond de oding phase. For any sets
R (um) and R (um), m 2 Ml0 I1 , we have
1 2 1

! 12
! 21
1z
e2 =
m
p R
1 ( um ); R (um )
1


1z p R ( u m ); R (um )
 e 2 = p R (u ); R (u )j0

2 ~m 1
(28)
1 2

p R (um ); R (um )
0 1 2 m m 1 2

!1
X p R (um ); R (um )jV (um ) 2
 p R ( u ) ; R ( u ) j 0

: 1 2 2

V2 u u all
m m
( m ): m =1
1 2

Averaging over all R (um) and R (um), onditioned that the all{zero sequen e was trans-
1 2

mitted, gives
XX
DI1 = U  +1 a (du; dv )DId1 V d :
def
(29) u 1 v +1

du dv
Here a(du; dv ) is the number of paths, orresponding to um = 1 and having the information
weight du and parity{ he k weight dv . It an be shown that
a (du ; dv ) = du a(du ; dv ) ; (30)
where a(du; dv ) is the modied spe trum of the en oder, as dened earlier (21).
The following iterations an be analyzed similarly. Then, analogously to (29), we get
XX
Di  dua(du ; dv )Did V d ; i = I + 1; : : : ; I: u 1
1 (31) v +1
1
du dv
The right hand side of (31) is equal to 

T (X; Y; Z ) 
Z


 X=V : (32)
 Y =1
Z = Di 1 =V

We have proved the following

Lemma 10 The values of the Bhatta haryya parameter, dened by (27) for the ith itera-
tion, i = I1 + 1; : : : ; I , satises the system of re urrent inequalities
2 3 

! 12 
= E4
Di def
p1 R (um); R (um)

1 2
5  T (X; Y; Z ) 

; um 2 Ml0 ; (33)

p0 R (um); R (um)
1 2 Z 

X=V
Y =1
i
Z = Di 1 =V

20
where DI1 is dened by (14), and V = exp( REb =N0 ).

x4.5 Iterative Limits of Constru tion B LDC Codes

Using the results from x4.2 and x4.3 we are now ready to derive bounds on the iterative
limits of onstru tion B LDC odes. We will present values of the bounds and ompare
them with simulation results.
Let us introdu e the fun tion

F (Z ) =
1 T (X; Y; Z ) 
 : (34)
V X=V
X Z  Y =1

Lemma 11 Let the fun tion FV (Z ) satisfy the onditions

8
>
>
>
>
>
>
FV (Z )jZ = 0 ;
=0
< 
>  = FV (Z ) < 1 ;
0
def
(35)
>
> Z =0
>
>
: Z FV (Z ) ! 1; when Z ! 1
> 1
:
Then there exists a positive root Z = Z to the equation
r

FV (Z ) = Z : (36)
Proof: The onditions assure, that the fun tion FV (Z ) has a root Z = 0, lies for small
Z > 0 below f (Z ) = Z and in reases for large Z faster than linearly with Z . From this
follows immediately that FV (Z ) rosses f (Z ) = Z for some Z > 0. The lemma is proved.
The rst and the third ondition in (35) are satised due to the stru ture of the path
weight enumerator T (X; Y; Z ). The se ond ondition requires, that the Bhatta haryya
parameter V does not ex eed a ertain value, depending on the omponent en oder. If the
Bhatta haryya parameter DI1 , given in (14), satises DI1 < V Z = D , it follows from
r
def
r

Lemma 10 and (35) that

Di < Di 1 8i 2 fI + 1; : : : ; I g :
1 (37)
If l is suÆ iently large, the linear term of the fun tion FV (Z ) = Z + O(Z ) is dominating
0
2

and we have
P < (V )l0 I1 ;
b (38)
21
Table 1
m 2 3 4 5


g (D ); g (D )
1 2 (7,5) (15,17) (31,35) (76,65)
SNR 0 0.05 dB -0.08 dB 0.00 dB 0.01 dB
V 0.71377 0.72090 0.71653 0.71598
D r 0.03077 0.08025 0.10813 0.11709

where is some onstant. Hen e, the bit error probability is de reasing with l = I at least0

exponentially. We have proved

Theorem 3 Consider a sequen e of rate R = 1=3 (l0 ;
){nondegenerated onstru tion B
LDC odes, where l0 ! 1 and
! 1. Suppose these odes are used for the transmis-
sion over an AWGN hannel and the extended path weight enumerator of the omponent
en oders satises (35). Let the two{phase algorithm des ribed in x4.2 be used for de oding,
having a total number of iterations I =l 0 and I1 < I iterations in the rst phase. Then,
if DI1 < V Z r = D r , where Z r is a positive root to equation (36), the bit error probability
Pb for the given sequen e of odes goes to zero at least exponentially with I , when I and l0
go to innity.

Any signal{to{noise ratio SNR that omplies with the requirements of Theorem 3, gives
us an upper bound SNR = SNR on the iterative limit SNR . In Table 1 values of SNR
0 0 0

are given for some rate R = 1=3 onstru tion B turbo{ odes with dierent memory m

omponent en oders, together with the orresponding values V and D . The oeÆ ients r
2

of the generator polynomials (g (D); g (D)) of the en oders are given in o tal form, where
1 2

g (D) is the feedba k polynomial. It is easy to see that the riti al values D are not
1 r

signi antly smaller than V . This allows us to use relatively small values of I in the 1

numeri al analysis.
2 The upper bound values given here are not ne essarily the tightest that ould be obtained by the
analysis des ribed above. Furthermore, the hosen omponent en oders are not ne essarily the ones that
lead to the best iterative limit values.

22
Table 2
m 2 3 4 5


g (D); g (D) (7,5)
1 2 (15,17) (31,35) (76,65)
p0 0.1553 0.1614 0.1612 0.1610
V 0.7244 0.7358 0.7354 0.7351
D r 0.0190 0.0679 0.0945 0.1025

An analysis of the two{phase algorithm for the ase of transmission over the BSC,
analogous to the one we have done for the AWGN hannel, gives lower bounds p on the 0

iterative limits p . The orresponding results are given in Table 2.

0

Simulation results of iterative de oding for LDC odes of onstru tion B used for the
transmission over the AWGN hannel are presented in Figure 6. The syndrome former
memory equals m = 25000, the memory of the omponent en oders m = 2 (right urve)
s

and m = 3 (left urve). The upper bounds on the iterative limits SNR are shown by
0

the dashed verti al lines.

x5. CONCLUSIONS

This paper is devoted to the analysis of iterative de oding of LDC odes. The key results
of the arti le are the proofs of the theorems about nondegeneration of LDC odes, whi h
generalize theorems about nondegeneration of LD blo k odes, and the development of
analysis methods for iterative de oding of turbo{ odes based on the study of the extended
path weight enumerator fun tion of the omponent en oders. As a result of the analysis,
upper bounds on the error probability for iterative de oding of some onstru tions of LDC
odes were derived and upper (lower) bounds on iterative limits for the AWGN hannel
(BSC) were obtained. We studied well{known onstru tions only, but a generalization of
the analysis methods to re ently proposed onstru tions, su h as dierent nonhomogeneous
(\nonregular") LDC odes and serial on atenated odes is, in prin iple, not diÆ ult and
is planned to be done in the future.

23
 APPENDIX

The proof of Lemma 8 is analogous to the proof of Lemma 1 in [7℄. Assume that the
onvolutional turbo{ ode des ribed by the matrix  is degenerated in the lth generation,
l  l . This implies that there exists an (ui0 ;
){ lan that is degenerated in the stri t
0

sense in the lth generation, i.e. there exists a pair of symbols ui and ui from the lth l k

and kth generation of the lan, su h that il  ik (mod T ). The rest of the proof, like
the proof of Lemma 1 in [7℄, onsists of the onsideration of three ases: one of the two
symbols is an an estor of the other, both symbols are des endants of the lan head from
dierent marriages, or both symbols are des endants of some other symbol without being
des endants of the lan head from dierent marriages or being des endant of one another.
Case 1: The symbol ui is an an estor of the symbol ui (k < l). Let us denote the
k l

symbols along the path from ui down to the symbol ui by ui ; ui +1 ; : : : ; ui 1 ; ui . Let

k l k k l l

fr ; fr 2 f1; 2g, be the type of the family of the symbol ui ; r = k; : : : ; l 1 (the head of
r

the family), and jr satisfy i ;j = 1; r = k; : : : ; l. Consider the following two sub ases:
r r

Case 1.1: fk = 1; fl = 2. Then jik ik j 
, jjk

1 jk j 
, : : :, jjl
+1 +1 jl j 
+2 1

. Consider a sequen e  = fik , ik , jk , jk , : : :, jl , jl g. Let us proof that this

+1 +1 +2 1

is a rst type
{ y le of length l k  l in the matrix . 0

e In fa t, subse tion i) of

Lemma 7 implies that ~i ;j = 1; r = k; : : : ; l, and subse tion ii) implies d (ik ; ik ) 
,
r r L +1

d (jk ; jk ) 
; : : : ; d (jl ; jl ) 
. Sin e the (ui0 ;
){ lan is nondegenerated in

L +1 +2 L 1

the stri t sense up to the (l 1)th generation all indi es ik ; ik ; : : : ; il are dierent. +1 1

The equality il = ik follows from the degeneration in the stri t sense of the lan in the lth
generation. Therefore, the matrix e ontains a rst type
{ y le of length less than 2l +1. 0

This is in ontradi tion with the onditions of the lemma. (The ase fk = 2; fl = 1 an 1

be onsidered analogously.)
Case 1.2: fk = 1; fl = 1. Analogously to Case 1.1 we study the sequen e  =
1

fil ; ik ; jk , jk ; : : : ; jl g. Subse tion i) of Lemma 7 implies that ~i ;j = 1, r =

1 +1 +1 +2 1 r r

k; : : : ; l, and subse tion ii) implies d (ik ; ik ) 
, d (jk ; jk ) 
, : : : ; d (il ; il ) 

L +1 L +1 +2 L 1

24

. Sin e il = ik we have d (ik ; ik ) 
, d (ik ; il ) 
) d (ik ; il )  2
.
L +1 L 1 L +1 1

Hen e,  is a rst type
{ y le of length l k 1  l 1 in the matrix .

0
e This is

in ontradi tion with the onditions of the lemma. (The ase fk = 2; fl = 2 an be

1

onsidered analogously.)
The se ond and the third ase of degeneration an be onsidered using the same s heme
as in the proof of Lemma 1 in [7℄. All of them lead to the appearan e of a rst type
{ y le
of length less than 2l + 1 or a se ond type
{ y le of length less than l . The lemma is
0 0

proved.

25
Referen es

[1℄ Engdahl K., Zigangirov K. Sh., On the Theory of Low{Density Convolutional Codes I.
Probl. Pereda h. Inform., 1999, vol. 35, no. 4, pp. 12{27.

[2℄ Zigangirov K. Sh., Lentmaier M., Mathemati al Analysis of an Iterative Algorithm for
Low{Density Code De oding. Probl. Pereda h. Inform., 2000, vol. 36, no. 4, pp. 35{46.
[3℄ Gallager R. G., Low{Density Parity{Che k Codes, Cambridge, Massa husetts: M.I.T.
Press, 1963.
[4℄ Berrou C., Glavieux A., Thitimajshima P., Near Shannon Limit Error Corre ting Cod-
ing and De oding: Turbo{Codes. Pro eedings IEEE Int. Conf. on Communi ations,
Geneva, Switzerland, 1993, pp. 1064{1070.
[5℄ Zyablov V. V., Pinsker M. S., Reliable transmission with low de oding omplexity.
Pro eedings VI Conferen e on Coding and Information Transmission, Mos ow-Tomsk,
1975, pp. 64-69.
[6℄ Jimenez A., Zigangirov K.Sh., Periodi Time-Varying Convolutional Codes with Low{
Density Parity{Che k Matri es, IEEE Trans. on Inform. Theory, 1999, vol. IT-45, no.
5, pp. 2181-2190.
[7℄ Truha hev D. V., Lentmaier M., Zigangirov K. Sh., Some Results Con erning Design
and De oding of Turbo{Codes, to appear in Probl. Pereda h. Inform., 2001, vol. 37,
no. 3.
[8℄ Bahl L., Co ke J., Jelinek F., Raviv J., Optimal De oding of Linear Codes for Min-
imizing Symbol Error Rate, IEEE Trans. Inform. Theory, 1974, vol. 20, no. 1, pp.
284{287.
[9℄ Margulis G. A., Expli it Constru tions of Graphs without Short Cy les and Low Den-
sity Codes, Combinatori a, 2(1):71{78, 1982.

26
[10℄ Margulis G. A., Some Expli it Constru tions of Low{Density Codes, Pro . III Inter-
national Seminar on Information Theory "Convolutional Codes; Communi ation with
many users", So hy 1987, pp. 120{123.

[11℄ Margulis G. A., Expli it Group{Theoreti Constru tions of Combinatorial S hemes

and their Appli ation to the Design of Expanders and Con entrators. Probl. Pereda h.
Inform., 1988. vol. 24, no. 1, pp. 51{60.

[12℄ Ri hardson T., Urbanke R., The Capa ity of Low{Density Parity Che k Codes under
Message-Passing De oding, IEEE Trans. on Inform. Theory, 2001, vol. IT-47, no. 2,
pp. 599-618.
[13℄ Johannesson R., Zigangirov K.Sh., Fundamentals of Convolutional Coding, Pis at-
away, N.J.: IEEE Press, 1999.

27
Figure Captions

Figure 1: Constru tion of a syndrome former of a homogeneous LDC ode from the
parity{ he k matrix of a homogeneous LD blo k ode.
Figure 2: BSC: Simulation results for iterative de oding of LDC (3; 6){ odes, ms =
129; 257; 513; 1025; 2049 (from top to bottom), and of (4; 8){ odes, ms = 129; 257; 513
(from top to bottom). Lower bounds on the iterative limits p (3; 6) and p (4; 8) are
0 0

shown by the dashed lines.

Figure 3: AWGN: Simulation results for iterative de oding of LDC (3; 6){ odes, ms =
129; 257; 513; 1025; 2049 (from top to bottom), and of (4; 8){ odes, ms = 129; 257; 513
(from top to bottom). Upper bounds on the iterative limits p (3; 6) and p (4; 8) are
0 0

shown by the dashed lines.

Figure 4: AWGN: Simulation results for iterative de oding of onstru tion B LDC odes
(solid) and blo k turbo{ odes (dashed), with memory m = 2 omponent en oders.

Figure 5: The behavior of DI1 as a fun tion of the number of iterations I for E =N =-0.06
b 0

dB, -0.08 dB and -0.1 dB.

AWGN: Simulation results for iterative de oding of onstru tion B LDC odes,
Figure 6:
m = 25000, with omponent en oders of memory m = 2 (right) and m = 3 (left).
s

Upper bounds on the iterative limits SNR are shown by the dashed verti al lines.
0

28
 Hb 0;0 Hb 0;1 Hb 0;2 : : : Hb 0;ms Hb 0;0 Hb 0;1 Hb 0;2 : : : Hb 0;ms
Hb 1;0 Hb 1;1 Hb 1;2 Hb 1;1 Hb 1;2 Hb 1;0
Hb ms ;ms 1

Hb 2;0 Hb 2;1 Hb 2;2 Hb 2;2 Hb 2;0 Hb 2;1

... ... ... ...
Hb ms ;0 Hb ms ;ms Hb ms ;ms Hb ms ;0

(a) (b)

Figure 1:

29
 (3; 6){ odes (4; 8){ odes
−1 ms=129 −1 ms=129
10 10
m =257 m =257
s s
ms=513 ms=513
ms=1025
m =2049
s
−2 −2
10 10

−3 −3
10 10
Pb

Pb

−4 −4
10 10

g repla ements −5 −5
10 10

−6 −6
10 10
0.09 0.08 0.07 0.06 0.09 0.08 0.07 0.06
p p

Figure 2:

30
 0
(3; 6){ odes 0
(4; 8){ odes
10 10
m =129 m =129
s s
m =257 m =257
s s
m =513 m =513
−1 s −1 s
10 m =1025 10
s
m =2049
s

−2 −2
10 10

−3 −3
10 10
Pb

Pb

−4 −4
10 10

−5 −5
10 10
g repla ements
−6 −6
10 10

−7 −7
10 10

Eb =N0 in dB Eb =N0 in dB
1 1.2 1.4 1.6 1.8 2 1 1.2 1.4 1.6 1.8 2

Figure 3:

31
 −1
10
N=500
N=1000
N=2000
−2
10

−3
10
Pb

−4
10

−5
10

−6
10

g repla ements
−7
10

Eb =N0 in dB
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Figure 4:

32
 0
10
0:10 dB

0:08 dB
−1
10

D r
DI1

−2
10
repla ements

0:06 dB
I1

−3
10

−4
10
0 10 20 30 40 50 60 70 80 90 100
I1

Figure 5:

33
 −1
10

−2
10

−3
10
Pb

−4
10

−5
10

rag repla ements

−6
10
−0.25 −0.2 −0.15 −0.1 −0.05 0 0.05 0.1 0.15 0.2 0.25
Eb =N0 in dB

Figure 6:

34