CS 2011 3 Comm Channels PDF

Communication Channels
Professor A. Manikas
Imperial College London
EE303 - Communication Systems

An Overview of Fundamentals
Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 1 / 48

Table of Contents
1 Introduction
2 Continuous Channels
3 Discrete Channels
4 Converting a Continuous to a Discrete Channel
5 More on Discrete Channels
Backward transition Matrix
Joint transition Probability Matrix
6 Measure of Information at the Output of a Channel
Mutual Information of a Channel
Equivocation & Mutual Information of a Discrete Channel
7 Capacity of a Channel
Shannos’s Capacity Theorem
Capacity of AWGN Channels
Capacity of non-Gaussian Channels
Shannon’s Channel Capacity Theorem based on Continuous Channel
Parameters
8 Bandwidth and Channel Symbol Rate
9 Criteria and Limits of DCS
Digital Communications - Introduction
ENERGY UTILIZATION EFFICIENCY (EUE)
Bandwidth Utilisation E¢ ciency (BUE)
Visual Comparison of Comm Systems
Theoretical Limits on the Performance of Dig. Comm. Systems
10 Other Comparison-Parameters
11 Appendix-A: SNR at the output of an Ideal Comm System
Introduction
Introduction
With reference to the following block structure of a Dig. Comm. System (DCS), this topic
is concerned with the basics of both continuous and discrete communication channels.
Block structure of a DCS

Introduction
Just as with sources, communication channels are either

I discrete channels, or
I continuous channels
1 wireless channels (in this case the whole DCS is known as a Wireless
DCS)
2 wireline channels (in this case the whole DCS is known as a Wireline
DCS)
Note that a continuous channel is converted into (becomes) a discrete
channel when a digital modulator is used to feed the channel and a
digital demodulator provides the channel output.
Examples of channels - with reference to DCS shown in previous page,
I discrete channels:
F input: A2 - output: b (alphabet: levels of quantiser - Volts)
A2
F input: B2 - output: b
B2 (alphabet: binary digits or binary codewords)
I continuous channels:
F b (Volts) - continuous channel (baseband)
input: A1 - output: A1,
F input: T, - output: Tb (Volts) - continuous channel (baseband),
F b (Volts) - continuous channel (bandpass).
input: T1 - output: T1
Continuous Channels
Continuous Channels
A continuous communication channel (which can be regarded as an
analogue channel) is described by
I an input ensemble (s (t ), pdf (s )) and PSDs (f )
s
I an output ensemble, (r (t ), pdf (r ))
r
I the channel noise (AWGN) n (t ) and β,
i
I the channel bandwidth B and channel capacity C .
()

Discrete Channels
Discrete Channels
A discrete communication channel has a discrete input and a discrete
output where
I the symbols applied to the channel input for transmission are drawn
from a …nite alphabet, described by an input ensemble (X , p ) while
I the symbols appearing at the channel output are also drawn from a
…nite alphabet, which is described by an output ensemble (Y , q )
I the channel transition probability matrix F.

Discrete Channels
In many situations the input and output alphabets X and Y are

identical but in the general case these are di¤erent. Instead of using
X and Y , it is common practice to use the symbols H and D and
thus de…ne the two alphabets and the associated probabilities as
,p 1 ,p 2 ,p M
z }| { z }| { z }| {
input: H = fH1 , H2 , ..., HM g p = [Pr(H1 ), Pr(H2 ), ..., Pr(HM )]T
,q 1 ,q 2 ,q K
z }| { z }| { z }| {
output: D = fD1 , D2 , ..., DM g q = [Pr(D1 ), Pr(D2 ), ..., Pr(DK )]T
where pm abbreviates the probability Pr(Hm ) that the symbol Hm

may appear at the input while qk abbreviates the probability Pr(Dk )
that the symbol Dk may appear at the output of the channel.

Discrete Channels
The probabilistic relationship between input symbols H and output

symbols D is described by the so-called channel transition probability
matrix F, which is de…ned as follows:
2 3
Pr(D1 jH1 ), Pr(D1 jH2 ), ..., Pr(D1 jHM )
6 Pr(D2 jH1 ), Pr(D2 jH2 ), ..., Pr(D2 jHM ) 7
F =6
4
7
5 (1)
..., ..., ..., ...
Pr(DK jH1 ), Pr(DK jH2 ), ..., Pr(DK jHM )
Discrete Channels
Pr(Dk jHm ) denotes the probability that symbol Dk 2 D will appear

at the channel output, given that Hm 2 H was applied to the input.
The input ensemble H, p , the output ensemble D, q and the
matrix F fully describe the functional properties of the channel.
The following expression describes the relationship between q and p
q = F.p (2)
Note that in a noiseless channel
D = H (3)
q = p
i.e the matrix F is an identity matrix
F = IM (4)

Converting a Continuous to a Discrete Channel

A continuous channel is converted into (becomes) a discrete channel
when a digital modulator is used to feed the channel and a digital
demodulator provides the channel output.
A digital modulator is described by M di¤erent channel symbols .
These channel symbols are ENERGY SIGNALS of duration Tcs .
Digital Modulator:
Digital Demodulator:
If M = 2 )Binary Digital Modulator )Binary Comm. System

If M > 2 )M-ary Digital Modulator )M-ary Comm. System
..01..101..00..
Tcs Tcs
0..00 s1(t)= : H1 , Pr(H1)
t
s(t)= t
0..01 s2(t)= : H2 , Pr(H2)
t
Tcs Tcs
t : H , Pr(H )
1..11 sM(t)= M M s(t)
Channel
..00..101..10..
r(t)=s(t)+n(t)
0..00 s1(t)= : D1 , Pr(D1)
t Tcs Tcs
0..01 s2(t)= : D2 , Pr(D2) D Detector r(t)=
with a
t Decision
Device
t
t : D , Pr(D ) Tcs Tcs
1..11 sM(t)= M M (Decision
Rule)

More on Discrete Channels Backward transition Matrix
More on Discrete Channels

Backward transition Matrix
There are also occassions where we get/observe the output of a

channel and then, based on this knowledge, we refer to the input .
In this case we may use the concept of an imaginary "backward "

channel and its associated transition matrix, known as backward
transition matrix de…ned as follows:
2 3T
Pr(H1 jD1 ), Pr(H1 jD2 ), ..., Pr(H1 jDK )
6 Pr(H2 jD1 ), Pr(H2 jD2 ), ..., Pr(H2 jDK ) 7
B =6
4
7
5 (5)
..., ..., ..., ...
Pr(HM jD1 ), Pr(HM jD2 ), ..., Pr(HM jDK )

More on Discrete Channels Joint transition Probability Matrix
Joint transition Probability Matrix

The joint probabilistic relationship between
input channel symbols H = fH1 , H2 , ..., HM g and output channel
symbols D = fD1 , D2 , ..., DM g,
is described by the so-called joint-probability matrix,
2 3T
Pr(H1 , D1 ), Pr(H1 , D2 ), ..., Pr(H1 , DK )
6 Pr(H2 , D1 ), Pr(H2 , D2 ), ..., Pr(H2 , DK ) 7
J ,64
7
5 (6)
..., ..., ..., ...
Pr(HM , D1 ), Pr(HM , D2 ), ..., Pr(HM , DK )
J is related to the forward transition probabilities of a channel with
the following expression (compact form of Bayes’Theorem):
2 3
p1 0 ... 0
6 0 p2 ... 0 7
J = F.6 7
4 ... ... ... ... 5 = F.diag(p ) (7)
0 0 ... pM
| {z }
,diag (p )
More on Discrete Channels Joint transition Probability Matrix
Note: This is equivalent to a new (joint) source having alphabet
f(H1 , D1) , (H1 , D2 ) , ..., (HM , DK )g
and ensemble (joint ensemble) de…ned as follows
8 9
>
> (H1 , D1 ), Pr(H1 , D1 ) >
>
>
> (H1 , D2 ), Pr(H1 , D2 ) >
>
>
> >
>
< =
...
(H D, J) = (8)
>
> (Hm , Dk ), Pr(Hm , Dk ) >
>
>
> >
>
>
> ... >
>
: ;
(HM , DK ), Pr(HM , DK )
80 1 9
>
< >
=
B C
= @(Hm , Dk ), Pr(Hm , Dk )A , 8mk : 1 m M, 1 k K
>
: | {z } >
;
=Jkm

Measure of Information at the Output of a Channel
Measure of Information at the Output of a Channel
In general three measures of information are of main interest:
1 the Entropy of a Source - in (info) bits per source symbol
2 the Mutual Entropy (or Multual Information) of a Channel , in

(info) bits per channel symbol
3 the Discrimination of a Sink
Next we will focus on the Mutual Information of a Channel Hmut

Measure of Information at the Output of a Channel Mutual Information of a Channel
Mutual Information of a Channel
The mutual information measures the amount of information that

the output of the channel (i.e. received message) gives about
the input to the channel (transmitted message).
That is, when symbols or signals are transmitted over a noisy

communication channel, information is received. The amount of
information received is given by the mutual information,
Hmut = 0 (9)

Measure of Information at the Output of a Channel Mutual Information of a Channel
M K
qk
Hmut , Hmut (p, F) = ∑ ∑ Fkm .pm log2 Fkm
(10)
m =1 k =1
M K
pm .qk
= ∑ ∑ Jkm log2 Jkm
(11)
m =1 k =1
0 1
B h iC
= 1TK B
@J log2 F.p.p T ?J CA 1M
bits
symbol
| {z }
K M matrix
(12)
where
1M = a column vector of M ones
, ? = Hadamard operators (mult. and div.)
Note that
1T A1 = adds all elements of A (13)
Measure of Information at the Output of a Channel Equivocation & Mutual Information of a Discrete Channel
Equivocation & Mutual Information of a Discrete Channel

Consider a discrete source (H, p ) followed by a discrete channel, as
shown below
The average amount of information gained (or uncertainty
removed) about the H source (channel input) by observing the
outcome of the D source (channel output), is given by the conditional
entropy HH jD which is de…ned as follows:
M K
Jkm
HH jD , HH jD (J) = ∑ ∑ Jkm . log2 qk
(14)
m =1 k =1
0 1
B 0 B
1C
B z }| { C
B
T B B 1 CC
= 1K BJ log2 @diag (q ) JA C
C 1M
bits
symbol
B C
@ A
| {z }
K M matrix
(15)
A similar expression can be also given for the average information

gained about the channel output D by observing the channel input H,
i.e.
M K
Jkm
HD jH , HD jH (J) = ∑ ∑ Jkm . log2 pm
(16)
m =1 k =1
0 1
B 0 F
1C
B z }| { C
B
T B B CC
= 1K BJ log2 @J.diag (p ) 1 ACC 1M
bits
symbol
B C
@ A
| {z }
K M matrix
(17)
The conditional entropy HH jD is also known as equivocation and it

is the entropy of the noise or, otherwise, the uncertainty in the
input of the channel from the receiver’s point of view.

Notes
1 for a noiseless channel:

HH j D = 0 (18)
2 For a discrete memoryless channel,
Hmut , Hmut (p, F) = HH HH j D (19)

= HD HD jH (20)

Capacity of a Channel Shannos’s Capacity Theorem
Capacity of a Channel
Shannon’s Capacity Theorem
There is a theoretical upper limit to the performance of a speci…ed

digital communication system with the upper limit depending on the
actual system speci…ed.
However, in addition to the speci…c upper limit associated with each
system, there is an overall upper limit to the performance which no
digital communication system, and in fact no communication system
at all, can exceed.
This bound (limit) is important since it provides the performance level
against which all other systems can be compared.
The closer a system comes, performance wise, to the upper limit the
better.

Capacity of a Channel Shannos’s Capacity Theorem
The theoretical upper limit was given by Shannon (1948) as an upper

bound to the maximum rate at which information can be transmitted
over a communication channel.
This rate is called channel capacity and is denoted by the symbol C .
Shannon’s capacity theorem states:
C , max (Hmut ) symbol
bits
(21)
or
C , rcs max (Hmut ) bits
sec (22)
where rcs denotes the channel-symbol rate (in channel-symbols per
sec) with
1
rcs = (23)
Tcs
rcs
B (24)
2
i.e. if Hmut (p, F) is maximised with respect to the input probabilities
p, then it becomes equal to C , the channel capacity (in bits/symbol)
Capacity of a Channel Capacity of AWGN Channels
Capacity of AWGN Channels

In the case of a continuous channel corrupted by additive white
Gaussian noise the capacity is given by
1 bits
C = log2 (1 + SNRin ) symbol (25)
2
or (26)
bits
C = B log2 (1 + SNRin ) sec (27)
where
B = baseband band width of channel

Ps
SNRin =
Pn
Ps b
= Power of the signal at point T
Pn b = N0 B
= Power of the noise at point T

Capacity of a Channel Capacity of non-Gaussian Channels
Capacity of non-Gaussian Channels

If the pdf of the noise is arbitrary (non-Gaussian) then it is very
di¢ cult to estimate the capacity .
However, it can be proved [Shannon 1948] that in this case the
capacity is bounded as follows:
Ps + Nn Ps + Pn bits
B log2 C B log2 sec (28)
Nn Nn
where
Ps : is the average received signal power,

Nn : is the entropy power of the noise, and
Pn : is the power of the noise
Equation 28 is important in that it can be used to provide bounds for

any kind of channel.
Shannon’s Channel Capacity Theorem based on Continuous
Capacity of a Channel Channel Parameters
Shannon’s Channel Capacity Theorem based on

Continuous Channel Parameters
Consider a time-continuous channel which comprises of a linear time
invariant …lter with transfer function H (f ), the output of which is
corrupted by an additive zero mean stationary noise n (t ) of PSDn (f ).
I if the average power of the channel input signal is constraint to be Ps ,
then
Z∞
( )
PSDn (f )
Ps = max 0, θ 2
.df (29)
∞
j H ( f )j
Z∞
( !)
1 θ. jH (f )j2
and C max 0, log2 .df (30)
2 PSDn (f )
∞
with the equality holding if the noise is Gaussian.
I Further, if the channel noise is white Gaussian with PSDn (f ) = N20
then Equation 30 simpli…es to the well known result
C = B log2 (1 + SNRin ) bits
sec (31)
Bandwidth and Channel Symbol Rate
Bandwidth and Channel Symbol Rate
The following expression are given without any proof:

channel symbol rate
Baseband Bandwidth (32)
2
channel symbol rate
Bandpass Bandwidth 2 (33)
2
The equality is known as Nyquist Bandwidth.
In this course, except if it is de…ned otherwise,
I the word "bandwidth" will mean "Nyquist bandwidth"
I the carrier will be ignored and thus "bandwidth" by default will refer to
"baseband bandwdith"

Criteria and Limits of DCS Digital Communications - Introduction
Critteria and Limits of DCS

Digital Communications - Introduction
Digital Communications provide excellent message-reproduction and

greatest Energy (EUE) and Bandwidth (BUE) Utilization E¢ ciency
through e¤ective employment of two fundamental techniques:
I source compression coding (to reduce the transmission rate for a
given degree of …delity)
I error control coding and digital modulation (to reduce the SNR
and bandwidth requirements)
With reference to the general structure of a DCS given in the next
page,
I the source compression coding is implemented by the blocks "Source
Encoder" and "Source Decoder"
I the error control coding is implemended by the "Discrete Channel
Endoder", "Interleaver", "DeInterleaver" and "Discrete Channel
Decoder".


Let us focus on the Discrete Channel: We have seen that a digital

modulator is described by M = 2γcs di¤erent channel symbols which
are ENERGY SIGNALS of duration Tcs .

Criteria and Limits of DCS ENERGY UTILIZATION EFFICIENCY (EUE)
Energy Utilisation E¢ ciency (EUE)

The parameter EUE is a measure of how e¢ ciently the system utilises
the available energy in order to transmit information in the presence
of additive white Gaussian noise of double-sided power spectral
density PSDn (f ) = N0 /2 and it is de…ned as follows:
Eb
EUE , (34)
N0
Note that EUE is directly related to the received signal power. It will
be appreciated of course that this is, in turn, directly related to the
transmitted power by the attenuation factor introduced by the
channel.
Clearly, a question of major importance is how large EUE needs to
be in order to achieve communication at some speci…c bit error
probability pe .
Obviously the smaller EUE to achieve a speci…ed error probability
the better .
Criteria and Limits of DCS Bandwidth Utilisation E¢ ciency (BUE)
Bandwidth Utilisation E¢ ciency (BUE)
The BUE measures how e¢ ciently the system utilises the bandwidth
B available to send information and it is de…ned as follows:
B
BUE , (35)
rb
where rb denotes the bit rate.
Speci…cally, the BUE indicates how much bandwidth is being
used per transmitted information bit and hence, for a given level
of performance, the smaller BUE the better since this means that
less bandwidth is being used to achieve a given rate of data
transmission.
N.B.:
rb
signaling speed , = BUE 1
(36)
B

Criteria and Limits of DCS Visual Comparison of Comm Systems
Visual Comparison of Comm Systems

By using EUE and BUE the SNRin can be expressed as follows
Eb Eb
Ps Tb Eb Eb N0 EUE
SNRin = Pn = = N 0 BT b = 1
N 0 B rb
= B
= BUE (37)
N0 B rb
By determining the EUE and BUE of any particular system, that
system can be represented as a point in the plane (EUE,BUE).
It is desirable for this point to be as close to the origin as possible
EUE bits
C = B log2 1 + BUE sec (38)
bits
EUE sec
C /B = log2 1 + BUE Hz (39)

Criteria and Limits of DCS Visual Comparison of Comm Systems
N.B.:
I a line from origin represents those points (systems) in the plane for
which the SNRin =constant
I By comparing points representing one system with those representing
another ) VISUAL COMPARISON !
I It can be observed that

F CS1 beter than CS2 which is better than CS3
F CS2 and CS3 have the same SNRin

Criteria and Limits of DCS Theoretical Limits on the Performance of Dig. Comm. Systems
Theoretical Limits on the Performance of Dig. Comm.

Systems
We have seen that the capacity of a white Gaussian channel of
bandwidth B is
C = B log2 (1 + SNRin ) bits
sec (40)
Please don’t forget that the above equation refers to bandlimited
white-noise channel with a constraint on the average transmitted
power.
Question: if B =" (and in particluar if B = ∞) then C =?
Answer :
I From the capacity-equation (Equ 40) it can be seen that B "=) C "
I However, when B tends to ∞ then
Ps
C∞ = 1.44 (41)
N0

LIMIT-1 : limit on bit rate

I when binary information is transmitted in the channel, rb should be
limited as follows:
rb C (42)
I ideal case:
rb = C (43)
LIMIT-2 : limit on EUE
I the best Energy E¢ ciency is EUE=0.693. This is the ultimate limit
below which no physical channel can transmit without errors
i.e
EUE 0.693
LIMIT-3 : Shannon’s threshold channel capacity curve
I f
This is the curve EUE= fBUEg for a bit rate rb equal to its maximum
value,
i.e.
2BUE 1
rb = C ) EUE = (44)
BUE 1
Plot of Equation 44:
No physical realizable CS could occupy a point in the plane

(EUE,BUE) lying below this theoretical channel capacity curve .

EUEdata EUEinf
SNRin = = (45)
BUEdata BUEinf
data rate : rb ,data = rcs log2 M (46)
info rate : rb ,info = rcs Hmut (47)
Point B always should be above the Shannon’s thr. capacity curve.

Point A may be above or below the Shannon’s thr. capacity curve.
The following table gives the corresponding equivalent parameters
between Analogue and Digital Comm Systems which can be used to
place on the (EUE,BUE) not only digital but also analogue
communication systems
Digital CS Analogue CS
Eb Ps Ps
EUE= N0 = N 0 rb , SNRin-mb = N 0 .F g
rb , Fg
BUE= B
rb , β, B
Fg
where:
I Fg denotes the maximum frequency of the message signal g (t ), i.e. it
represents the bandwidth of the message
I β is known as "bandwidth expansion factor",
e.g. SSB: β = 1;AM: β = 2
Comparison of various Digital and Analogue CS are shown below (for a …xed SNR out )

Other Comparison-Parameters
Other Comparison-Parameters
SPECTRAL CHARACTERISTICS of transmitted signal

(rate at which spectrum falls o¤).
INTERFERENCE RESISTANCE
I it may be necessary to increase (EUE,BUE) in order to increase interf.
resistance
FADING
I Fading problem pe ="
I
BUE ="
Note that, if then Fading=#
EUE =#
DELAY DISTORTION
I Try to avoid this problem by selecting appropriate signals
COST and COMPLEXITY

Appendix-A: SNR at the output of an Ideal Comm System
Appendix: SNR at the output of an Ideal Comm System
In this section a so-called ideal system of communication will be

considered and it will be shown that bandwidth can be exchanged for
signal-to-noise performance.
The ideal system forms a benchmark against which other

communication systems can be compared.
An ideal system has been de…ned as one that transmits data at a bit
rate
r =C (48)
where C is the channel capacity i.e. C = B log2 (1 + SNRin ) bits/sec

Furthermore we have seen that for an ideal communication system:
SNRin
EUE = ) lim EUE = 0.693 (49)
log2 (1 + SNRin ) SNR in !0
1
BUE = EUE
(50)
log2 (1 + BUE )
1
2BUE 1
) EUE = 1
) lim EUE = 0.693 (51)
BUE BUE !∞

Block Diagram of an Ideal Communication System:

The previous …gure shows the elements of a basic ideal

communication system.
An input analogue message signal g (t ), which is of bandwidth Fg , is
applied to a signal mapping unit which, in response to g (t ), produces
an analogue signal s (t ) of bandwidth B and this signal is transmitted
over an analogue channel having a similar bandwidth, B.
I The channel is corrupted by additive white Gaussian noise of double
sided power spectral density N0 /2 which is bandlimited to the channel
bandwidth B.
I Let the signal-to-noise ratio at the input of the receiver be SNRin .
I Assume further that the received signal, plus noise, is then fed to a
detector having a bandwidth Fg , equal to the message bandwidth.
I Let the signal-to-noise ratio at the output of the detector be SNRout.

Now the capacity of the analogue transmission system (channel) is
C = B log2 (1 + SNRin ) bits/s (52)
Also, the "mapping-unit/channel/detector" can be regarded as a

channel having a signal-to-noise ratio SNRout and hence it too can
be regarded as an AWGN channel and its capacity is given by
C 0 = Fg log2 (1 + SNRout ) bits/s (53)
If, in order to avoid information loss (ideal case), the capacities are
set equal then it can be seen, after simple mathematical
manipulation, that
B
C 0 = C ) ... ) SNRout,ideal = (1 + SNRin ) Fg 1 (54)
SNRin mb β
= (1 + ) 1 (55)
β
where β is the bandwidth expansion factor.
The above expression is fundamentally important since it shows that

the overall system performance, SNRout , can be improved by using
more channel bandwidth.
The …gure below shows, as a function of the bandwidth expansion
factor β, typical curves of SNRout versus SNRin mb for the ideal
communication system.
Note that all other known communication systems should be

compared with this optimum performance provided by Equation .
From the previous …gure it can be seen that:

I if SNRin mb is small then on increasing β the e¤ect on the SNRout is
small (i.e. very little increase in the SNRout is obtained).
I If, however, SNRin mb is large then a small increase in the bandwidth
expansion factor results in a large increase in the SNRout .
I A practical consequence of this is that if the SNRin mb is small then
there is little to be gained from using more channel bandwidth.
Case-1:SNRin-mb =small Case-2:SNRin-mb =large=10

SNRin-mb =1 SNRin-mb =10
β=1 SNRout =1 SNRout =10
β=2 SNRout =1.25 SNRout =35
β = 100 SNRout =1.7048 SNRout =13780

CS 2011 3 Comm Channels PDF

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

CS 2011 3 Comm Channels PDF

Hochgeladen von

Copyright:

Verfügbare Formate

Communication Channels

Imperial College London

EE303 - Communication Systems

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 1 / 48

Block structure of a DCS

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 3 / 48

Just as with sources, communication channels are either

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 5 / 48

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 6 / 48

In many situations the input and output alphabets X and Y are

where pm abbreviates the probability Pr(Hm ) that the symbol Hm

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 7 / 48

The probabilistic relationship between input symbols H and output

Pr(Dk jHm ) denotes the probability that symbol Dk 2 D will appear

Note that in a noiseless channel

i.e the matrix F is an identity matrix

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 9 / 48

Converting a Continuous to a Discrete Channel

If M = 2 )Binary Digital Modulator )Binary Comm. System

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 11 / 48

More on Discrete Channels

There are also occassions where we get/observe the output of a

In this case we may use the concept of an imaginary "backward "

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 12 / 48

Joint transition Probability Matrix

Note: This is equivalent to a new (joint) source having alphabet

f(H1 , D1) , (H1 , D2 ) , ..., (HM , DK )g

and ensemble (joint ensemble) de…ned as follows

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 14 / 48

Measure of Information at the Output of a Channel

In general three measures of information are of main interest:

1 the Entropy of a Source - in (info) bits per source symbol

2 the Mutual Entropy (or Multual Information) of a Channel , in

3 the Discrimination of a Sink

Next we will focus on the Mutual Information of a Channel Hmut

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 15 / 48

Mutual Information of a Channel

The mutual information measures the amount of information that

That is, when symbols or signals are transmitted over a noisy

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 16 / 48

Equivocation & Mutual Information of a Discrete Channel

A similar expression can be also given for the average information

The conditional entropy HH jD is also known as equivocation and it

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 19 / 48

1 for a noiseless channel:

2 For a discrete memoryless channel,

Hmut , Hmut (p, F) = HH HH j D (19)

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 20 / 48

There is a theoretical upper limit to the performance of a speci…ed

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 21 / 48

The theoretical upper limit was given by Shannon (1948) as an upper

Capacity of AWGN Channels

B = baseband band width of channel

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 23 / 48

Capacity of non-Gaussian Channels

Ps : is the average received signal power,

Equation 28 is important in that it can be used to provide bounds for

Shannon’s Channel Capacity Theorem based on

Bandwidth and Channel Symbol Rate

The following expression are given without any proof:

Prof. A. Manikas (Imperial College) EE303: Communication Channels 2011 26 / 48

Critteria and Limits of DCS

Digital Communications provide excellent message-reproduction and