Channel Coding-Mini Subject-Turbo Code

Master IST
Vo Khanh Nam and Nguyen Minh Thang
CHANNEL CODING Mini-Subject: Turbo Code

1. Background
Turbo code was introduced first time by Berrou, Glavieux and Thitimajshima in 1993. Turbo codes consist of two binary convolutional codes concatenated in parallel, decoded by an iterative decoding algorithm. These codes obtain an excellent bit error rate (BER) performance by making use of three main components. They are constructed using two systematic convolutional encoders that usually known as recursive systematic convolutional (RSC) encoders, which are concatenated in parallel. In this parallel concatenation, a random interleaver plays a very important role as the randomizing constituent part of the coding technique. This coding scheme is decoded by an iterative decoder that makes the resulting BER performance closed to the Shannon limit.[1] As two recursive convolutional encoders are arranged in parallel concatenation, so that each input element is encoded twice, but the input to the second encoder first passes through a random interleaver. This interleaving procedure is designed to make the encoder output sequences be statistically independent from each other. The systematic encoders have code rate Rc = 1/2. As a result of the systematic form of the coding scheme, and the double encoding of each input bit, the resulting code rate is Rc = 1/3. In order to improve the rate, another useful technique included in a turbo coding scheme is puncturing of the convolutional encoder outputs. The decoding algorithm for the turbo coding scheme involves the corresponding decoders of the two convolutional codes iteratively exchanging soft-decision information, so that the information can be passed from one decoder to the other. The decoders operate in a softinputsoft-output mode; that is, both the input applied to each decoder, and the resulting output generated by the decoder, should be soft decisions or estimates. Both decoders operate by utilizing what is called a priori information, and together with the channel information provided by the samples of the received sequence, and information about the structure of the code, they produce an estimate of the message bits. They are also able to produce an estimate called the extrinsic information, which is passed to the other decoder, information that in the following iteration will be used as the a priori information of the other decoder. Thus the first decoder generates extrinsic information that is taken by the second decoder as its a priori information. This procedure is repeated in the second decoder, which by using the a priori information, the channel information and the code information generates again an estimation of the message information, and also an extrinsic information that is now passed to the first decoder. The first decoder then takes the received information as its a priori information for the new iteration, and operates in the same way as described above, and so on. The iterative passing of information between the first and the second decoders continues until given number of iterations is reached. With each iteration the estimates of the message bits improve, and they usually converge to a correct estimate of the message. The number of errors corrected increases as the number of iterations increases. However, the improvement of
Channel Coding: Mini-Subject Report Page 1
Master IST
the estimates does not increase linearly, and so, in practice, it is enough to utilize a reasonable small number of iterations to achieve acceptable performance. [2]
2. Turbo Encoder
Figure 1: A Turbo Encoder A turbo encoder constructed using two RSC encoders arranged in parallel, and combined with a random interleaver, together with a multiplexing and puncturing block, is seen in Figure 1. In the original structure of a turbo encoder, the encoders E1 and E2 are RSC encoders of rate Rc = 1/2, such that c1 = c1, c2= c2 and the lengths of the sequences m, c1 and c2, and c1 and c2 are all the same. Then the overall turbo code rate is Rc = 1/3. To improve the overall rate of the code, Puncturing technique is applied. The puncturing selection process is performed by periodically eliminating one or more of the outputs generated by the constituent RSC encoders. Thus, for instance, the parity bits generated by these two encoders can be alternately eliminated so that the redundant bit of the first encoder is first transmitted, eliminating that of the second decoder, and in the following time instant the redundant bit of the second encoder is transmitted, eliminating that of the first. In this way, the lengths c1 and c2 are half the lengths of c1 and c2, respectively, and the resulting overall rate becomes Rc = 1/2. Puncturing is not applied to the message (systematic) bits, because this causes a BER performance loss.
Channel Coding: Mini-Subject Report
Page 2
Master IST
Figure 2. Punctured Rate R=1/2 Turbo Coder There are two important components of a turbo encoder whose parameters have a major influence on the BER performance of a turbo code: the first is the interleaver, especially its length and structure, and the second is the use of RSC as constituent encoders. The excellent BER performance of these codes is enhanced when the length of the interleaver is significantly large, but also important is its pseudo-random nature. The interleaving block, and its corresponding de-interleaver in the decoder introduce a significant delay in the system, which in some cases can be a strong drawback.
3. Turbo Decoder
Figure 3: A Turbo Decoder
Page 3
Master IST
Turbo codes are so named because of their iterative soft-decision decoding process, which enables the combination of relatively simple RSC codes to achieve near-optimum performance. Turbo decoding involves iterative exchange between the constituent decoders of progressively better estimates of the message bits, in a decoding procedure that is helped by the statistical independence of the two code sequences generated by each input bit. In the decoding procedure, each decoder takes into account the information provided by the samples of the channel, which correspond to the systematic (message) and parity bits, together with the a priori information that was provided by the other decoder, which was calculated as its extrinsic information in the previous iteration. However, instead of making a hard decision on the estimated message bits, as done for instance in the traditional decoding of convolutional codes using the Viterbi algorithm, the decoder produces a soft-decision estimate of each message bit. This soft-decision information is an estimate of the corresponding bit being a 1 or a 0; that is, it is a measure of the probability that the decoded bit is a 1 or a 0. This information is more conveniently evaluated in logarithmic form, by using what is known as a log likelihood ratio (LLR), to be defined below.
4. Decoding Algorithms
The two main types of decoder are Maximum A Posteriori (MAP) and the Soft Output Viterbi Algorithm (SOVA). MAP looks for the most probable value for each received bit by calculating the conditional probability of the transition from the previous bit, given the probability of the received bit. The focus on transitions, or state changes within the trellis, makes LLR a very suitable probability measure for use in MAP. SOVA uses a trellis to establish a surviving path, compares this with the sequences that were used to establish the non-surviving paths. Where surviving and non-surviving paths overlap the likelihood of that section being on the correct path is reinforced. Where there are differences, the likelihood of that section of the path is reduced. At the output of each decoding stage the values of the bit sequence are scaled by a channel reliability factor, calculated from the likely output sequence, to reduce the probability of over-optimistic soft outputs. The sequence and its associated confidence factors are then presented to the interleaver for further iterations. After the prescribed number of iterations, the SOVA decoder will output the sequence with the maximum likelihood.
5. Interleaver:
Interleaving is a widely used technique in digital communication and storage systems. An interleaver takes a given sequence of symbols and permutes their positions, arranging them in a different temporal order. The basic goal of an interleaver is to randomize the data sequence. When used against burst errors, interleavers are designed to convert error patterns that contain long sequences of serial erroneous data into a more random error pattern, thus distributing errors among many code vectors. Burst errors are characteristic of some channels, like the wireless channel, and they also occur in concatenated codes, where an inner decoder overloaded with errors can pass a burst of errors to the outer decoder.
Master IST
In general, data interleavers can be classified into block, convolutional, random, and linear interleavers. In a block interleaver, data are first written in row format in a permutation matrix, and then read in column format. A pseudo-random interleaver is a variation of a block interleaver where data are stored in a register at positions that are determined randomly. Convolutional interleavers are characterized by a shift of the data, usually applied in a fixed and cumulative way. Linear interleavers are block interleavers where the data positions are altered by following a linear law.
6. Other concepts that used/mentioned in Turbo Codes:

a. The Shannon limit: Shannons Channel Coding Theorem states that if the information rate, R (rH bits/s) is equal to or less than the channel capacity, C, (i.e. R < C) then there is, in principle, a coding technique which enables transmission over the noisy channel with no errors.
C is the channel capacity, B is the channel band-width in hertz, S is the signal power and N is the noise power. The channel capacity, C, increases as the available band-width increases and as the signal to noise ratio increases (improves). For a given average transmitted power S and channel bandwidth, B, we can transmit information at the rate C bits/s with no error, by employing sufficiently com-plex coding systems. It is not possible to transmit at a rate higher than C bits/s by any coding system without a definite probability of error. Hence the channel capacity theorem defines the fundamental limit on the rate of error-free transmission for a power-limited, band-limited channel [2]. b. Probabilities and Estimates The probability distribution is a useful description of a discrete random variable X whose values are taken from a discrete alphabet of symbols AX. The distribution (or histogram when plotted graphically) is a function that assigns to each value of the variable the probability of occurrence of that value. In the case of continuous random variables, the histogram becomes the so-called probability density function. A probability distribution for a discrete random variable is of the form
where a non-negative number p(x) is assigned to each value of the random variable x AX . An equivalent quantity often utilized in decoding algorithms is the probability measure or metric (x) of the event x AX. A measure or estimate of an event x AX of a discrete random variable is a generalization of the probability distribution, where the restriction that the sum over all the probabilities in the distribution does not necessarily have to be equal to 1. Then the relationship between measures (x) and probabilities p(x) is given by
Page 5
Master IST
Measures have properties similar to those of probabilities. The marginal measure of an event x AX , conditioned on a random variable Y , is obtained by summing over all the events of the associated random variable Y :
where (x, y) is the joint measure for a pair of random variables X and Y . The Bayes rule is also applicable to joint measures:
It is usually more convenient to convert products into sums by using measures in logarithmic form. In this case the measure is called a metric of a given variable:
It is also true that
and
c. Likelihood Functions The mathematical foundations of hypothesis testing rest on Bayes theorem. For communications engineering, where applications involving an AWGN channel are of great interest, the most useful form of Bayes theorem expresses the a posteriori probability (APP) of a decision in terms of a continuous-valued random variable x in the following form:
(1) And
Page 6
Master IST
(2) where P(d = i|x) is the APP, and d = i represents data d belonging to the ith signal class from a set of M classes. Further, p(x|d = i) represents the probability density function (pdf) of a received continuous-valued data-plus-noise signal x, conditioned on the signal class d = i. Also, P(d = i), called the a priori probability, is the probability of occurrence of the ith signal class. Typically x is an observable random variable or a test statistic that is obtained at the output of a demodulator or some other signal processor. Therefore, p(x) is the pdf of the received signal x, yielding the test statistic over the entire space of signal classes. In Equation (1), for a particular observation, p(x) is a scaling factor, since it is obtained by averaging over all the classes in the space. Lowercase p is used to designate the pdf of a continuous-valued random variable, and uppercase P is used to designate probability (a priori and APP). Determining the APP of a received signal from Equation (1) can be thought of as the result of an experiment. Before the experiment, there generally exists (or one can estimate) an a priori probability P(d = i). The experiment consists of using Equation (1) for computing the APP, P(d = i|x), which can be thought of as a refinement of the prior knowledge about the data, brought about by examining the received signal x. [3] d. Hard decision and soft decision Symbol detection is performed in receivers and decoders, and it is applied in order to determine the value of a given discrete random variable X by observing events of another random variable Y, which is related to the variable X. Thus, for instance, a given discrete random variable X that takes values from the range of a discrete alphabet AX = {0, 1} can be the input of a binary symmetric channel with transition or error probability p, and the corresponding output of this channel can be the discrete random variable Y . If in this channel the output discrete random variable Y takes values from the discrete alphabet AY = {0, 1}, it is said that the channel is a hard-decision channel. Another model is the Gaussian channel, where a given discrete random variable X with values taken from the discrete polar format alphabet AX = {+1,1} generates a continuous random variable Y , which after being affected by white and Gaussian noise takes values from the set of real numbers AY = . In general terms, a hard decision over X is said to occur when after observing the variable Y, the decoder or receiver makes a firm decision that selects one of the two possible values of X, AX = {+1,1}, a decision that will be denoted as x. This is the case of a decoder or receiver that takes samples of the received signal and compares these samples with a voltage threshold in order to decide for a 1 or a 0 depending on this comparison. A soft decision of X is said to happen when after observing the variable Y the decoder or receiver assigns a measure, metric or estimate (x) of X based on the observed value of Y. There are many ways of assigning a measure (x) of X, that is, to make a soft decision of X, but the more significant ones are those provided by the maximum likelihood (ML) and the maximum a posteriori (MAP) decoding methods.
Page 7
Master IST
In the ML decoding method, the soft decision of X based on the event of the variable y AY is given by the following conditional distribution function
In the MAP decoding method, the soft decision of X based on the event of the variable y AY is given by the following conditional probability distribution function:
The ML measure is not a probability distribution function because the normalizing condition, the sum over the alphabet of X, AX , should be equal to 1, is not obeyed in this case. However, it is obeyed in the case of the MAP estimation, which is indeed a probability density function. The MAP measure is proportional to the joint probability p(x, y). Since:
Then
e. The Log Likelihood Ratio (LLR): The LLR is the most common information measure or metric used in iterative decoding algorithms, and it is usually the measure or the extrinsic estimate that each decoder communicates to the other in a turbo decoder. The LLR for a bit bi is denoted as L(bi ), and it is defined as the natural logarithm of the quotient between the probabilities that the bit is equal to 1 or 0. Since this is a signed number, this sign can be directly considered as representative of the symbol which is being estimated, and so it is more convenient to define it as the quotient of the probability that the bit is equal to +1 or 1, using the polar format. This is the same as saying that the decision is taken over the transmitted signal alphabet in the range {1}, rather than over the binary information alphabet {0, 1}. This estimate is then defined as
(1) This definition will be found more convenient in the description of the decoding algorithms, where the sign of the LLR is directly used as the hard decision of the estimate, and its value is
Master IST
utilized as the magnitude of the reliability of the estimate. Thus, a soft decision can be understood as a weighted hard decision.
Figure 4: LLR as a function of the bit probability of the symbol +1 Figure 4 shows the LLR as a function of the bit probability of the symbol +1, which is positive if P(bi = +1) > 0.5 (symbol 1 is more likely than symbol 0), and it is negative if P(bi = +1) < 0.5 (symbol 0 is more likely than symbol 1). The magnitude of this amount is a measure of the probability that the estimated bit adopts one of these two values. From (1), and as P(bi = +1) = 1 P(bi = 1), then
And also
Page 9
Master IST
Can be summarised as
since the bit bi = 1.
References [1] Dr. Aoife Moloney., Channel Capacity, Lecture No. 11, School of Electronics and Communications, Dublin Institute of Technology [2] Jorge Castineira Moreira, Patric Guy Farrel., Essentials of Error Control Coding, John Wiley & Sons, 2006 [3] Bernard Sklar., Fundamentals of Turbo Codes

Channel Coding-Mini Subject-Turbo Code

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Channel Coding-Mini Subject-Turbo Code

Hochgeladen von

Copyright:

Verfügbare Formate

Master IST

Vo Khanh Nam and Nguyen Minh Thang