Coding Theory – Main Problems

Dr D D Karunaratna
Properties of a good code
 A good code (n,M,d)-code should have the following
properties(n- length of a code word, M – number of code
words in the code, d- minimum distance of the code)
 Small code length n
 Large number of code words M
 Large Minimum Distance D

 Why these requirements made the code good?

Properties of a good code
 A good code (n-M,d)-code should have the following
properties
 Small code length n - For fast transmission
 Large number of code words M – Enables large number of
messages.
 Large Minimum Distance D – Enables to detect and correct
large number of errors.
Properties of a good code ….
 A (n,M,d)-code cannot be designed to satisfy all three
properties (small code length(n), large number of cord
words(M), large minimum distance (d)) at the same time as
they are conflicting needs.
 Thus the main aim of coding is to optimize one parameter
given values of the other two.
 Typically, in coding theory one of the main objectives is to
find a code with the largest number of code words (M), given
the length of a code word (n) and the minimum distance (d).
Properties of a good code ….
 Let Aq(n,d) denotes the largest value of M if q-arry(n,M,d)-
code exists.
 Theorem :
Aq(n,1) = qn

How to prove this result ?

Properties of a good code ….
 Aq(n,1) = ?

Each code-word should be distinct.

Thus the maximum number of code-words possible
= |(Fq)n |= qn
Properties of a good code ….
 Theorem :
Aq(n,n) = q

How can we prove this?

Properties of a good code ….
 Aq(n,n) = ?

Two different code-words in the code should be different in all n positions.

Thus, a symbol appearing in any fixed position must be distinct.
Example : The code words should looks like
1 ……..
2 …….
……….
……….
q

Therefore Aq(n,n) = q
Properties of a good code ….
 Consider the following code.
Code C3
00000
01101
10110
11011

d(c) = 3
Can correct 1 error.

 This is a binary (5,4,3)-code.

 Is it possible to have a code with A2(5,3) ≥ 4 ?
Binomial coefficients
 If n and m are integers with 0 ≤ m ≤ n, then the Binomial
n
coefficient  
m is defined by

n n!
 
 m  m!(n  m)!

 The number of unordered selection of m distinct objects

n
from a set of n distinct objects is  
m
 Binomial coefficients
 Let F2 = {0,1}
 (F2)5 = set of all vectors over the alphabet
Then the order of (F2)5 = | (F2)5 |= 25 = 32

Question : How many different binary codes with 5 codewords

(M) can be constructed from the vectors in (F2)5 ?
 Binomial coefficients
 Let F2 = {0,1}
 (F2)5 = set of all vectors over the alphabet
Then the order of (F2)5 = | (F2)5 |= 25 = 32

Question : How many different binary codes with 5 codewords (M) can be
constructed from the vectors in (F2)5 ?

 32  32!
 
 5  5!(32  5)!
= 201,376
 Some of these codes may be equivalent
 To find a better code than the binary (5,4,3) code, the minimum distance
of all possible codes has to be computed – Time consuming task.
Permutation of a set
 A permutation of a set S = {x1, x2, …., xn } is a one-to-one
mapping f, from S to itself.

Example :
S = {x1, x2, x3 , x4, x5 }

S = {x1, x2, x3 , x4, x5 }

f(x1) = x3, f(x2) = x1, f(x3) = x4, f(x4) = x2, f(x5) = x5

f is a permutation on S
Permutation of a set ….
 Let S = {x1, x2, …., x5 }
 Example 1:
Let f be a mapping on S defined as below.
f(x1) = x3, f(x2) = x1, f(x3) = x4, f(x4) = x2, f(x5) = x3
Is f a permutation on S ?
Permutation of a set ….
 Let S = {x1, x2, …., xn }
 Example 1:
Let f be a mapping on S defined as below.
f(x1) = x3, f(x2) = x1, f(x3) = x4, f(x4) = x2, f(x5) = x3
Is f a permutation on S ?
Answer :
f is not 1-1. Thus it is not a permutation on S
Permutation of a set ….
 Let S = {x1, x2, …., xn }
 Example 2:
Let f be a mapping on S defined as below.
f(x1) = x3, f(x2) = x1, f(x1) = x4, f(x4) = x2, f(x5) = x5
Is f a permutation on S ?
Permutation of a set ….
 Let S = {x1, x2, …., xn }
 Example 2:
Let f be a mapping on S defined as below.
f(x1) = x3, f(x2) = x1, f(x1) = x4, f(x4) = x2, f(x5) = x5
Is f a permutation on S ?
Answer :
f is not 1-1. Thus it is not a permutation on S
Permutation of a set ….
 Let S = {x1, x2, …., xn }. How many permutations can be
defined on S?
 Number of permutations = n x (n-1)x…..x2x1 = n!
Code equivalence
 Two q-array codes are called equivalent if one can be
obtained from other by a combination of operations of the
following types:
P1: Permutation of the columns of the code matrix.
 Rearrangement of the columns of the code matrix.
P2: Permutation of the symbols appearing in a fixed
position (a column - when the code is represented as
an matrix)
 Re-labelling of the symbols appearing in a given
column.
Code equivalence …
 Example :
0 0 1 0 0
0 1 0 0 1
C2 
1 0 0 1 0
1 1 1 1 1

Apply the permutation f(1) = 1, f(2) = 4,f(3) = 3, f(4) =

2,f(5) = 5 on the columns of C2 –P1
0 0 1 0 0
0 0 0 1 1
C3 
1 1 0 0 0
1 1 1 1 1

C2 and C3 are all equivalent.

Code equivalence …
 Example :
0 0 0 0 0
0 1 1 0 1
C1 
1 0 1 1 0
1 1 0 1 1
Apply the permutation f(0) = 1, f(1) = 0 to the 3rd column
of C1 – P2
0 0 1 0 0
0 1 0 0 1
C2 
1 0 0 1 0
1 1 1 1 1

C1 and C2 are equivalent

Code equivalence …
 Let C be a code and x and y be any two codewords in C.
How does d(x,y) and d(C ) change when any combination of
P1 and P2 permutations applied on C?
Code equivalence …
 Let C be a code and x and y be any two codewords in C. How
does d(x,y) and d(C ) change when any combination of P1 and P2
permutations applied on C?

 Conclusion : The distances between codewords would not change

by applying P1 and P2 permutaions on a code and also will end up
with a code
 Having the same parameters (n,M,d).
 Having the ability to correct the same number of errors.
 Also, under the assumption of q-array symmetric channel, the
performance of equivalent codes will be identical in terms of
probabilities of error correction.
 Code equivalence …
 Can we create a code equivalent to the following with all zeros
code word in the code?
0 0 1 0 0
0 1 0 0 1
C
1 0 0 1 0
1 1 1 1 1
 Code equivalence …
 Example :
0 0 1 0 0
0 1 0 0 1
C
1 0 0 1 0
1 1 1 1 1 Apply f(1) = 0,f(0) = 1 to the third column.

0 0 0 0 0
0 1 1 0 1
C
1 0 1 1 0
1 1 0 1 1
Code equivalence …
 Deduce an equivalent code to the following code which
contains the all-zero vector.

012
120
201
Code equivalence …
 Deduce an equivalent code to the following code which
contains the all-zero vector.
012 apply the permutation f(0) = 2, 002
120 f(1) = 0, f(2) =1 to the 2nd 110
201 column 221

002 apply the permutation f(0) = 1, 000

110 f(1) = 2, f(2) =0 to the 3nd 111
column
221 222
A2(n,d) = ?
 Computation of A2(n,d) is resource intensive and the value of
A2(n,d) is known only for a limited number of n and d
values.
 What is A2(5,5) ?
A2(n,d) = ?
 What is A2(5,5) = 2
00000
11111
All these codes are
equivalent
01010
10101

……………
A2(n,d) = ?
 What is A2(5,3) ?
A2(n,d) = ?
 What is A2(5,3) ?

 We can start with the code word 0 0 0 0 0.

 Then any other code word in the code should have at least three
1’s to have the minimum distance 3. Thus we can have the
following code words.
00000
11100
00111
A2(n,d) = ?
 What is A2(5,3) ?

 The only other code possible is 11011

00000
11100
00111
11011

Therefore A2(5,3) = 4
Code equivalence …
 Any q-array(n,M,d)-code over an alphabet {0,1,…..,q-1} is
equivalent to an (n,M,d)- code which contains the all-zero vector
0= 0 0 0 …..0.

Proof : Take any codeword x1x2….xn and for each xi ≠ 0 apply the
permutation f(xi) = 0, f(0) = xi, f(j) = j for all other values to the
column in which xi appears. Repeat this operation for all xi s
where xi ≠ 0.
Weight of a vector
Lets define the weight of a vector x in (F2)n as the number
of 1’s in x and denoted by w(x).

Example :
w(11100) = 3
w(00011) = 2
 Binomial coefficients
 Question : How many vectors with weight i are in (F2) n ?
 Binomial coefficients
 Question : How many vectors with weight i are in (F2) n ?
 = Select any different i positions from n positions
 assign those positions the value 1 and the other positions 0.
 =  
n

i

Question : How many vectors with weight 2 are in (F2) 3 ?

(F2) 3 = {000,100,010,001,110,101,011,111}
Vectors with weight 2 are in (F2) 3 = {110,101,011}
3 3!
 
 2  2!(3  2)!
=3
Computation of Hamming Distance
 Hamming distance d(x,y) : The number of places in which x and y
differ.

 Let F2 = {0,1} and x = x1x2….xn, y = y1y2….yn be two vectors in

(F2)n.
Define two operations + and ∩ on as below;
x + y = = l1l2….ln where li= xi  yi
x ∩ y = = m1m2….mn where mi= xi  yi
Where  and  are two modulo operations (without carrying) defined
as below
 0 1  0 1
0 0 1 0 0 0
1 1 0 1 0 1
Computation of Hamming Distance …..
Example :
Compute
1. 11100 +00111
2. 11100 ∩ 00111
Computation of Hamming Distance …..
Example :
1. 11100 + 00111 = 11011
2. 11100 ∩ 00111 = 00100
Computation of Hamming Distance …..
Relating d(x,y), w(x), x +y and x ∩ y

d(11100,00111) = 4
w(11100 + 00111) = w(11011) = 4
d(11100,00111) = w(11100 + 00111)

w(11100 ∩ 00111) = w(00100) = 1

d(11100,00111) = w(11100) + w(00111) –
2 w(11100 ∩ 00111)
Computation of Hamming Distance …..
Relating d(x,y), w(x), x +y and x ∩ y

Lemma : if x and y are in (F2)n

 d(x,y) = w(x+y)
 d(x,y) = w(x) + w(y) – 2w(x ∩ y)
Existence of codes
 Theorem : Suppose d is odd. Then a binary (n,M,d)-code
exists iff a binary (n+1,M,d+1)-code exists.
 If d is odd A2(n,d) = A2(n+1,d+1)
 Equivalently, if d is even then A2(n,d) = A2(n-1,d-1)

If we know that a binary (n,M,d)-code exists and d is odd, how

can be obtain a binary (n+1,M,d+1)-code?
Existence of codes
If we know that a binary (n,M,d)-code exists, how can be
obtain a binary (n+1,M,d+1)-code?

x = x1x2x3….. xn → x1x2x3….. xn0 if w(x) is even

x1x2x3….. xn1 if w(x) is odd

The construction of this new (n+1,M,d+1)-code from

(n,M,d)-code is called “adding an overall parity check” to
the (n,M,d)-code.
Existence of codes
x = x1x2x3….. xn → x1x2x3….. xn0 if w(x) is even
x1x2x3….. xn1 if w(x) is odd

How can we prove that the minimum distance of this

code is d+1?
Existence of codes
x = x1x2x3….. xn → x1x2x3….. xn0 if w(x) is even
x1x2x3….. xn1 if w(x) is odd

How can we prove that the minimum distance of this code (C’) is
d+1?
 w(x) is even for any x in C’
 Since d(x,y) = w(x) + w(y) – 2w(x ∩ y), the distance
between any two code words of C’ should be even
 d ≤ d(C’) ≤ d + 1 and d is odd
 Since d(C’) has to be even d(C’) should be d+1
Example
Add overall parity check to the following code

00000
01101
10110
11011
Example
Add overall parity check to the following code

00000 000000
01101 → 011011
10110 101101
11011 110110
Example ….
 If d is odd, how can we obtain a binary (n,M,d)-code from a
binary (n+1,M,d+1)-code?
 How can we obtain (5,4,3)-code from the following (6,4,4)-
code?
000000
011011
101101
110110
Example ….
 Choose code words x and y such that d(x,y) = 4.
 Choose a position in which x and y differ and delete that
column from all code words.
000000 00000
011011 01101
101101 10111
110110 11010