Lecture 1

Introduction to Error Control

Coding

2/03/2009

Introduction

Introduction
1.1 Why Coding?


To achieve reliable data communication


To achieve reliable data storage


To reduce the required transmit power


To reduce hardware costs of transmitters


To improve bandwidth efficiency


To increase channel utilisation


To increase storage density

2/03/2009

Introduction

Introduction
1.2 Applications Of Coding

2/03/2009

Block codes for error detection in data networks

Reed Solomon (RS) codes in Deep-Space Standard
RS codes in Compact Disc (CD) systems

Introduction

Introduction

2/03/2009

RS and BCH codes in satellite networks

Trellis codes in fax modems
Convolutional codes in mobile phones
Turbo codes in WI-Fi networks
Low Density Parity Check (LDPC) codes - Wi-Max

Introduction

Introduction
1.3 How Coding Can Be Achieved?

2/03/2009

Coding is achieved by adding properly designed redundancy

to each message. These redundant digits are used for
detecting and/or correcting transmission (or storage)
errors.

Introduction

Digital Channel

Information
Source

Source
Encoder

Channel
Encoder

Modulator
(Writing Unit)

Digital Data Source

Transmission
Channel

Noise

(Storage Medium)
Digital Data Destination

Destination

Source
Decoder

Channel
Decoder

Demodulator
(Reading Unit)

Fig. 3.3 Block diagram of a typical data transmission or

storage system
2/03/2009

Introduction

Introduction
1.4 Types of Coding

2/03/2009

Block and convolutional coding.

Block coding: A message of k digits is mapped into a
structured sequence of n digits, called a codeword. The
mapping operation is called encoding. Each encoding
operation is independent of the past encodings. That is, the
encoder has no memory of history of past encodings. The
collection of all codewords is called a block code.
In general, both message and code symbols are binary
symbols, 0 and 1. In this case, there are 2k distinct
messages. Corresponding to these 2k distinct messages,
there are 2k binary codewords.
Introduction

Introduction

2/03/2009

The parameters, k and n, are called the message and code

lengths respectively. In general n > k.
The ratio R = k/n is called the code rate.
n - k redundant digits are added to each massage for
protection against errors.
Example 1.1: Let k = 3 and n = 6. The following table gives a
block code of length 6. The code rate is R = 1/2.

Introduction

Introduction
Message

Codeword

( 0 0 0 0 0 0)
(0 1 1 1 0 0)
(1 0 1 0 1 0)
(1 1 0 1 1 0)
(1 1 0 0 0 1)
(1 0 1 1 0 1)
(0 1 1 0 1 1)
(0 0 0 1 1 1)

(0 0 0)
(1 0 0)
(0 1 0)
(1 1 0)
(0 0 1)
(1 0 1)
(0 1 1)
(1 1 1)

2/03/2009

Introduction

Introduction

2/03/2009

Convolutional coding: An information sequence is divided into

(short) blocks of k digits each. Each k-digit message is
encoded into an n-digit coded block. The n-digit coded block
depends not only on the input k-digit message block but also
on m ( 1) previous message blocks. That is, the encoder has
memory of order m.
The encoder has k inputs and n outputs.
An information sequence is encoded into a code sequence.
The collection of all possible code sequences is called an (n,
k, m) convolutional code.
The parameters, k and n, are normally small, say 1 k 8
and 2 n 9. Again k < n and the ratio R = k/n is called the
code rate.
Introduction

10

Introduction

Example 1.2: Let k = 1, n = 2 and m = 2. The following

circuit generates a (2,1,2) convolutional code.

cl

cl 1

cl 2

2/03/2009

vl(1)

Introduction

vl( 2)

11

Introduction
Input:

ct

Outputs:

vl(1) = cl + cl 2
vl( 2) = cl + cl 1 + cl 2 .

Shift register stage

+

exclusive-OR gate

2/03/2009

Introduction

12

Introduction
1.5 Types of Errors

Random and burst errors.

1.6 Types of Channels

2/03/2009

Random error channels: deep space channels, many satellite

channels, line-of-sight transmission facilities, etc.
Burst error channels: radio links, wire and cable
transmission.

Introduction

13

Introduction
1.7 Types of Codes

2/03/2009

Classification based on structure

a) block codes - linear codes, cyclic codes
b) convolutional codes
Classification based on the types of errors which they
correct
a) random-error-correcting codes
b) burst-error-correcting codes
Classification based on the types of code symbols
a) binary codes
b) nonbinary codes
Introduction

14

Introduction

2/03/2009

Classification based on applications

a) error-correction codes
b) error-detection codes

Introduction

15

Introduction
1.8 Types of Error Control Schemes

2/03/2009

Forward-error-correction (FEC): An error - correction code

is used. After error correction, the decoded message is
delivered to the user.
Automatic-repeat-request (ARQ): An error - detection
code is used. If the presence of error is detected in a
received word (or sequence), a retransmission is requested.
The request signal is sent to the transmitter through a
feedback channel. Retransmission continues until no errors
being detected.
Hybrid ARQ: A proper combination of FEC and ARQ.

Introduction

16

Introduction
1.9 FEC vs. ARQ

ARQ

Advantage: simple, easy to achieve high reliability.

Disadvantage: feedback channel is needed, variable
throughput

FEC

Advantage: no feedback channel is needed, constant

throughput.
Disadvantage: complex, hard to achieve high reliability.

2/03/2009

Introduction

17

Introduction
1.10 Decoding

2/03/2009

Suppose a codeword corresponding to a certain message is

transmitted over a noisy channel.
Let r be the corresponding received sequence.
The receiver (or decoder), based on r, the encoding rules
and the noise characteristics of the channel, makes a
decision which message was actually transmitted.

Introduction

18

Introduction

2/03/2009

This decision making operation is called decoding.

The device which performs the decoding operation is called
a decoder.
There are two types of decodings based on the decisions
made by the decoder.

Introduction

19

Introduction
1.11 Hard and Soft Decision Decodings

2/03/2009

Hard-Decision: When binary coding is used, the modulator

has only binary inputs (M = 2). If binary demodulator output
quantisation is used (Q = 2), the decoder has only binary
inputs. In this case, the demodulator is said to make hard
decisions. Decoding based on hard decisions made by the
demodulator is called hard-decision decoding.
Soft-Decision: If the output of the demodulator consists
of more than two quantisation levels (Q > 2) or is left
unquantised, the demodulator is said to make soft decisions.
Decoding based on soft decision made by the demodulator
is called soft-decision decoding.

Introduction

20

Introduction

2/03/2009

Hard-decision decoding is much easier to implement than

soft-decision decoding. However, soft-decision decoding
offers significant performance improvement over harddecision decoding.

Introduction

21

Introduction
1.12 Some channel models

In a binary coded digital communication system, if the

channel is an additive white Gaussian noise (AWGN)
channel, hard decision made by the demodulator results in a
binary symmetric channel (BSC).
1-p
0

1
1-p

2/03/2009

Introduction

22

Introduction

Suppose the demodulator makes soft decision and has 8

output quantisation levels (Q = 8). Then we have a binaryinput, 8-ary output discrete channel. The 8-level
quantisation scheme is most frequently used in the softdecision decoding systems.
0
1
0
2
3
4
1
5
6
7

2/03/2009

Introduction

23

Introduction
1.13 Maximum Likelihood Decoding
Optimum Decoding

2/03/2009

Suppose the codeword v corresponding to a certain message

c is transmitted.
Let r be the corresponding output of the demodulator.
The decoder must produce an estimate of the message
based on r.
A decoding rule is a strategy for choosing an estimated
message for each possible received sequence .
Obviously, we would like to devise a decoding rule such that
the probability of a decoding error, p(e), is minimised. Such
a decoding rule is called an optimum decoding rule.
Introduction

24

Introduction
Maximum-Likelihood Decoding

Suppose all the messages are equally likely. An optimum

decoding can be done as follows. For every codeword
compute the conditional probability
P( r | v j ).

2/03/2009

The codeword with the largest conditional probability is

chosen as the estimate for the transmitted codeword .
Then decode into the estimate for the transmitted
message c based on the encoding rule. This decoding rule is
called the maximum likelihood decoding (MLD).

Introduction

25

Introduction
1.14 MLD for a BSC

Let a=(a1,a2,,an) and b=(b1,b2,,bn) be two binary sequences

of n components. The Hamming distance between a and b,
denoted d(a,b), is defined as the number of places where
a and b, differ.
For example, let a=(1 0 1 1 0 1 1) and b=(0 1 1 0 1 0 1). Then

d(a,b)=5.

2/03/2009

In coding for a BSC, every codeword and every received

sequence are binary sequences.
Suppose some codeword is transmitted and the received
sequence is r=(r1,r2,,rn).

Introduction

26

Introduction

For a codeword vi, the conditional probability P(r|vi) is

P( r | vi ) = p d ( r ,vi ) (1 p) n d ( r ,vi )

For p <1/2, P(r|vi) is a monotonically decreasing function

of d (r,vi).
Then
P( r | v i ) > P( r | v j )
if and only if d(r,vi)<d(r,vj).

2/03/2009

Introduction

27

Introduction

2/03/2009

MLD:
1) Compute d(r,vj) for all vj.
2) vi is taken as the transmitted codeword if
d(r,vi)<d(r,vj) for ji.
3) Decode vi into message ci.
The received vector r is decoded into the closest
codeword.
This is also called the minimum distance (nearest
neighbour) decoding.

Introduction

28

Introduction
1.15 Coding and Modulation-Demodulation

2/03/2009

In most of coded digital communication systems, coding is

performed separately from modulation - demodulation.
Error control is provided by transmitting additional
redundant bits in the code, which has the effect of
lowering the information bit rate per channel bandwidth.
In this case, bandwidth efficiency is traded for increased
power efficiency.
This is suitable for power-limited systems.
However, when bandwidth efficiency is a major concern
(such as in bandwidth-limited systems), the most effective
method for error control is to combine coding and
modulation as a single entity.
Introduction

29

Introduction

2/03/2009

In such an approach, coding is redefined as a process of

imposing certain patterns on the transmitted signal.
This definition obviously includes the traditional idea of
redundancy.
The combined modulation coding is called coded modulation.
This error control technique is most suitable for bandwidth - limited systems where the available bandwidth must
be utilised effectively.

Introduction

30