Beruflich Dokumente
Kultur Dokumente
5
About the Classes
• You’re welcome to ask questions.
– You can interrupt
p me at any y time.
– Please don’t disturb others in the class.
• Our responsibility is to facilitate you to learn.
You have to make the effort
effort.
• Spend time reviewing lecture notes afterwards.
• If yyou have a question
q on the lecture material
after a class, then
– Look up a book! Be resourceful.
– Try to work it out yourself.
yourself
– Ask me during the problem class or one of scheduled times of availability.
6
References
• C. Ling, Notes of Communication Systems,
Imperial Collage.
• S Haykin & M
S. M. Moher
Moher, Communication
Systems, 5th ed., International Student
Version, Wiley, 2009 (£43.99 from Wiley)
• y , Communication Systems,
S. Haykin, y , 4th ed.,,
Wiley, 2001
– Owns the copyright of many figures in these
slides.
– For
F convenience,
i th note
the t “ 2000 WilWiley,
Haykin/Communication System, 4 ed.” is not
th
shown for each figure.
• B.P.
B P Lathi,
Lathi Modern Digital and Analog
Communication Systems, 3rd ed., Oxford
University Press, 1998
• J.G. Proakis and M. Salehi,, Communication
Systems Engineering, Prentice-Hall, 1994
• L.W. Couch II, Digital and Analog
Communication Systems, 6th ed., Prentice-
H ll 2001
Hall,
7
Multitude of Communications
• Telephone network
• Internet
• Radio and TV broadcast
• Mobile communications
• Wi-Fi
• Satellite and space communications
• Smart power grid, healthcare…
• Analogue communications
– AM, FM
• Digital communications
– Transfer of information in digits
– Dominant technology today
– Broadband, 3G, DAB/DVB
8
What’s Communications?
• Communication involves the transfer of information from
one point to another.
• Three basic elements
– Transmitter: converts message into a form suitable for
transmission
– Channel: the physical medium, introduces distortion, noise,
interference
– Receiver:
R i reconstruct
t t a recognizable
i bl fform off the
th message
Speech
Music
Pictures
Data
…
9
Communication Channel
• The channel is central to operation of a communication
y
system
– Linear (e.g., mobile radio) or nonlinear (e.g., satellite)
– Time invariant (e.g., fiber) or time varying (e.g., mobile radio)
• The information-carrying capacity of a communication
system is proportional to the channel bandwidth
• Pursuit for wider bandwidth
– Copper wire: 1 MHz
– Coaxial cable: 100 MHz
– Microwave: GHz
– Optical fiber: THz
• Uses light as the signal carrier
• Highest capacity among all practical signals
10
Noise in Communications
• Unavoidable presence of noise in the channel
– Noise refers to unwanted waves that disturb communications
– Signal is contaminated by noise along the path.
• External noise: interference from nearby channels, human-
made noise, natural noise...
• Internal noise: thermal noise, random emission... in
electronic devices
• Noise is one of the basic factors that set limits on
communications.
i ti
• A widely used metric is the signal-to-noise (power) ratio
(SNR)
signal power
SNR= noise p
power
11
Transmitter and Receiver
• The transmitter modifies the message signal into a form
suitable for transmission over the channel
• This modification often involves modulation
– Moving the signal to a high-frequency carrier (up-conversion) and
varying some parameter of the carrier wave
– Analog: AM, FM, PM
– Digital: ASK
ASK, FSK,
FSK PSK (SK: shift keying)
• The receiver recreates the original message by
demodulation
– Recovery is not exact due to noise/distortion
– The resulting degradation is influenced by the type of modulation
• Design of analog communication is conceptually simple
g
• Digital communication is more efficient and reliable; design
g
is more sophisticated
12
Objectives of System Design
• Two primary resources in communications
– Transmitted p
power ((should be g
green))
– Channel bandwidth (very expensive in the commercial market)
• In certain scenarios, one resource may be more important
than the other
– Power limited (e.g. deep-space communication)
– Bandwidth limited (e.g.
( telephone circuit))
• Objectives of a communication system design
– The message is delivered both efficiently and reliably
reliably, subject to
certain design constraints: power, bandwidth, and cost.
– Efficiency is usually measured by the amount of messages sent in
unit power, unit time and unit bandwidth.
– Reliability is expressed in terms of SNR or probability of error.
13
Information Theory
• In digital communications, is it possible to operate at
zero error rate even though the channel is noisy?
• Poineers: Shannon, Kolmogorov…
– The maximum rate of reliable transmission is
calculated.
– The famous Shannon capacity formula for a channel
with bandwidth W (Hz)
C = W log(1+SNR) bps (bits per second)
Shannon
– Zero error rate is possible as long as actual signaling
rate is less than C.
• Many concepts
M t were fundamental
f d t l and
d paved
d th
the
way for future developments in communication
theory.
– Provides a basis for tradeoff between SNR and
bandwidth, and for comparing different communication
schemes.
Kolmogorov
14
Milestones in Communications
• 1837, Morse code used in telegraph
• 1864 Maxwell formulated the eletromagnetic (EM) theory
1864,
• 1887, Hertz demonstrated physical evidence of EM waves
• 1890’s-1900’s
1890 s-1900 s, Marconi & Popov,
Popov long-distance radio
telegraph
– Across Atlantic Ocean
– From Cornwall to Canada
16
Cellular Mobile Phone Network
• A large area is partitioned into cells
• Frequency reuse to maximize capacity
17
Growth of Mobile Communications
• 1G: analog communications
– AMPS
• 2G: digital communications
– GSM
– IS-95
• 3G: CDMA networks
– WCDMA
– CDMA2000
– TD-SCDMA
TD SCDMA
• 4G: data rate up to
1 Gbps (giga bits per second)
– Pre-4G technologies:
WiMax, 3G LTE
18
Wi--Fi
Wi
• Wi-Fi connects “local” computers (usually within 100m
g )
range)
19
IEEE 802.11 Wi-
Wi-Fi Standard
• 802.11b
– Standard for 2.4GHz ((unlicensed)) ISM band
– 1.6-10 Mbps, 500 ft range
• 802.11a
– Standard for 5GHz band
– 20-70 Mbps, variable range
– Similar to HiperLAN in Europe
• 802.11g
– Standard in 2.4 GHz and 5 GHz bands
– Speeds up to 54 Mbps, based on orthogonal frequency division
multiplexing (OFDM)
• 802.11n
– Data rates up
p to 600 Mbpsp
– Use multi-input multi-output (MIMO)
20
Satellite/Space Communication
• Satellite communication
– Cover very large areas
– Optimized for one-way transmission
• Radio (DAB) and movie (SatTV)
broadcasting
– Two-way systems
• The only choice for remote-area and
maritime communications
• Propagation delay (0.25 s) is
uncomfortable in voice
communications
• Space communication
– Missions to Moon, Mars, …
– Long distance, weak signals
– High-gain antennas
– Powerful error-control coding
21
Future Wireless Networks
Ubiquitous Communication Among People and Devices
Wireless
Wi l IInternet
t t access
Nth generation Cellular
Ad Hoc Networks
Sensor Networks
Wireless Entertainment
Smart Homes/Grids
Automated Highways
All this and more…
•Hard Delay Constraints
•Hard Energy Constraints
22
Communication Networks
• Today’s communications networks are complicated
systems
– A large n
number
mber of users
sers sharing the medi
medium
m
– Hosts: devices that communicate with each other
– Routers: route date through the network
23
Concept of Layering
• Partitioned into layers, each doing a relatively simple task
• Protocol stack
Network
Application
Transport
Network
Link
Ph i l
Physical
Physical
TCP/IP protocol
stack (Internet)
OSI Model 2-layer model
25
Outline
• Probability
– How probability is defined
– cdf and pdf
– Mean and variance
– Joint distribution
– Central limit theorem
• Random processes
p
– Definition
– Stationary random processes
– Power spectral density
• References
– Notes of Communication Systems,
y , Chap.
p 2.3.
– Haykin & Moher, Communication Systems, 5th ed., Chap. 5
– Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap 11
Chap.
26
Why Probability/Random Process?
• Probability is the core mathematical tool for communication
y
theory.
• The stochastic model is widely used in the study of
communication systems.
• Consider a radio communication system where the received
signal is a random process in nature:
– Message is random. No randomness, no information.
– Interference is random.
– Noise is a random process.
process
– And many more (delay, phase, fading, ...)
• Other real-world applications of probability and random
processes include
– Stock market modelling,
g g
gambling
g ((Brown motion as shown in the
previous slide, random walk)…
27
Probabilistic Concepts
• What is a random variable (RV)?
– It is a variable that takes its values from the outputs
p of a random
experiment.
• What is a random experiment?
– It is an experiment the outcome of which cannot be predicted
precisely.
– All possible identifiable outcomes of a random experiment
constitute its sample space S.
– An event is a collection of possible outcomes of the random
experiment.
i
• Example
– For tossing a coin
coin, S = { HH, T }
– For rolling a die, S = { 1, 2, …, 6 }
28
Probability Properties
• PX(xi): the probability of the random variable X taking on
the value xi
• The probability of an event to happen is a non-negative
number, with the following properties:
– The probability of the event that includes all possible outcomes of
the experiment is 1.
– The probability of two events that do not have any common
outcome is the sum of the probabilities of the two events
separately.
• Example
– Roll a die: PX(x = k) = 1/6 for k = 1, 2, …, 6
29
CDF and PDF
• The (cumulative) distribution function (cdf) of a random variable X
is defined as the probability of X taking a value less than the
argumentt x:
FX ( x) P( X x)
• Properties
FX () 0, FX () 1
FX ( x1 ) FX ( x2 ) if x1 x2
• The probability density function (pdf) is defined as the derivative of
the distribution function:
dF X ( x )
f X ( x) dx
x
FX ( x )
f X ( y ) dy
b
P ( a X b ) FX (b ) FX ( a )
a
f X ( y ) dy
dF X ( x )
f X ( x) dx 0 since F X ( x ) is non - decreasing
30
Mean and Variance
• If x is sufficiently small,
x x
P ( x X x x ) f X ( y ) dy f X ( x ) x
x
f X ( y) Area
f X ( x)
x
y
x
• Mean (or expected value DC level):
E[ ]: expectation
E[ X ] X
x f X ( x ) dx operator
• Variance ( power for zero-mean signals):
X2 E[( X X ) 2 ]
( x X ) 2 f X ( x)dx E[ X 2 ] X2
31
Normal (Gaussian) Distribution
f X ( x)
x
0 m
( x m )2
f X ( x) 1 e 2 2 f x
for
2 E[ X ] m
( y m )2
x 2 2
X2 2
FX ( x ) 1
2 e dy : rm s v a lu e
32
Uniform Distribution
fX(x)
1
a xb ab
f X ( x) b a E[ X ]
0 elsewhere 2
0 xa (b a ) 2
xa X2
12
FX ( x ) a xb
b a
1 xb
33
Joint Distribution
• Joint distribution function for two random variables X and Y
FXY ( x , y ) P ( X x , Y y )
• Joint probability density function
2 FXY ( x , y )
f XY ( x, y ) xy
• Properties
1) FXY (, )
d d 1
f XY (u , v)dudv
2) f X ( x)
y
f XY ( x, y ) dy
3) fY ( x )
x
f XY ( x, y ) dx
4) X , Y are independent f XY ( x, y ) f X ( x) fY ( y )
5) X , Y are uncorrelated E[ XY ] E[ X ]E[Y ]
34
Independent vs. Uncorrelated
• Independent implies Uncorrelated (see problem sheet)
• Uncorrelated does not imply Independence
• For normal RVs (jointly Gaussian), Uncorrelated implies
Independent (this is the only exceptional case!)
• An example of uncorrelated but dependent RV’s
Let be uniformly
y distributed in [0,2 ]
Y Locus of
f ( x) 21 for 0 x 2 X and Y
35
Joint Distribution of n RVs
• Joint cdf
FX1 X 2 ... X n ( x1 , x2 ,...xn ) P( X 1 x1 , X 2 x2 ,... X n xn )
• Joint pdf
n FX1X 2 ... X n ( x1 , x2 ,... xn )
f X 1 X 2 ... X n ( x1 , x2 ,...xn ) x1x2 ...xn
• Independent
FX 1 X 2 ... X n ( x1 , x2 ,...xn ) FX 1 ( x1 ) FX 2 ( x2 )...FX n ( xn )
f X 1 X 2 ... X n ( x1 , x2 ,...xn ) f X 1 ( x1 ) f X 2 ( x2 )...
) f X n ( xn )
• i.i.d. (independent, identically distributed)
– The random variables are independent and have the same
distribution.
– Example: outcomes from repeatedly flipping a coin.
36
Central Limit Theorem
• For i.i.d. random variables,
z = x1 + x2 +· · ·+ xn
x1 x1 + x2
tends to Gaussian as n
goes to infinity.
• Extremely useful in
communications.
• That’s why noise is usually x1 + x2 x1 + x2+
+ x3 x3 + x4
Gaussian. We often say
“Gaussian noise or
Gaussian noise”
“Gaussian channel” in
communications.
Illustration of convergence to Gaussian
distribution
37
What is a Random Process?
• A random process is a time-varying function that assigns
the outcome of a random experiment to each time instant:
X(t).
• For a fixed (sample path): a random process is a time
varying function, e.g., a signal.
• For fixed t: a random pprocess is a random variable.
• If one scans all possible outcomes of the underlying
random experiment, we shall get an ensemble of signals.
38
An Ensemble of Signals
39
Statistics of a Random Process
• For fixed t: the random process becomes a random
variable, with mean
X (t ) E[ X (t )] x f X ( x; t )dx
– In general
general, the mean is a function of t.
t
• Autocorrelation function
RX (t1 , t2 ) E[ X (t1 ) X (t2 )] xy f X ( x, y; t1 , t2 )dxdy
40
Stationary Random Processes
• A random process is (wide-sense) stationary if
– Its mean does not depend
p on t
X (t ) X
– Its autocorrelation function only depends on time difference
RX (t, t ) RX ( )
RX (t , t ) E[ X (t ) X (t )]
E[ A2 cos(c t ) cos(c t c )]
A2 A2
E[cos(2c t c 2)] E[cos(c )]
2 2
A2 2 1 A2
2 0
cos(2c t c 2 )
2
d
2
cos(c )
A2
RX ( ) cos(c )
2
42
Power Spectral Density
• Power spectral density (PSD) is a function that measures
the distribution of power of a random process with
frequency.
• PSD is only defined for stationary processes.
• Wiener-Khinchine
Wiener Khinchine relation: The PSD is equal to the
Fourier transform of its autocorrelation function:
S X ( f ) R X ( )e j 2f d
43
Passing Through a Linear System
• L
Lett Y(t) obtained
bt i d bby passingi random d process X(t) th
through
h
a linear system of transfer function H(f). Then the PSD of
Y(t) 2
SY ( f ) H ( f ) S X ( f ) (2.1)
44
EE2--4: Communication Systems
EE2
Lecture 3: Noise
45
Outline
• What is noise?
• White noise and Gaussian noise
• Lowpass noise
• Bandpass noise
– In-phase/quadrature representation
– Phasor representation
p
• References
– Notes of Communication Systems, Chap. 2.
– Haykin & Moher, Communication Systems, 5th ed., Chap. 5
– Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap. 11
46
Noise
• Noise is the unwanted and beyond our control waves that
g
disturb the transmission of signals.
• Where does noise come from?
– External sources: e.g., atmospheric, galactic noise, interference;
– Internal sources: generated by communication devices themselves.
• This type of noise represents a basic limitation on the performance of
electronic communication systems
systems.
• Shot noise: the electrons are discrete and are not moving in a
continuous steady flow, so the current is randomly fluctuating.
• Thermal
Th l noise:
i caused
dbby th
the rapid
id andd random
d motion
ti off electrons
l t
within a conductor due to thermal agitation.
• Both are often stationaryy and have a zero-mean Gaussian
distribution (following from the central limit theorem).
47
White Noise
• The additive noise channel
( ) models all types
– n(t) yp of noise
– zero mean
• White noise
– Its power spectrum density (PSD) is constant over all frequencies,
i.e., N
SN ( f ) 0 , f
2
– Factor 1/2 is included to indicate that half the power is associated
with positive frequencies and half with negative.
– The term white is analogous to white light which contains equal
amounts of all frequencies (within the visible band of EM wave).
– It
It’ss only defined for stationary noise
noise.
• An infinite bandwidth is a purely theoretic assumption.
48
White vs. Gaussian Noise
• White noise PSD
SN(f) Rn()
N
– Autocorrelation function of n(t ) : Rn ( ) 0 ( )
2
– Samples
S l att diff
differentt time
ti iinstants
t t are uncorrelated. l t d
• Gaussian noise: the distribution at any time instant is
Gaussian Gaussian
– Gaussian noise can be colored PDF
• White noise Gaussian noise
– White noise can be non-Gaussian
• Nonetheless, in communications, it is typically additive
white Gaussian noise (AWGN)
(AWGN).
49
Ideal Low
Low--Pass White Noise
• Suppose white noise is applied to an ideal low-pass filter
of bandwidth B such that
N0
, | f | B Power PN = N0B
SN ( f ) 2
0, otherwise
• By
B Wiener-Khinchine
Wi Khi hi relation,
l i autocorrelation
l i ffunction
i
Rn() = E[n(t)n(t+)] = N0B sinc(2B) (3.1)
where sinc(x) = sin(x)/x.
x
• Samples at Nyquist frequency 2B are uncorrelated
Rn() = 0,
0 = k/(2B),
k/(2B) k = 1,
1 2,2 …
50
Bandpass Noise
• Any communication system that uses carrier modulation will typically
have a bandpass filter of bandwidth B at the front-end of the receiver.
n(t)
• Any y noise that enters the receiver will therefore be bandpass
p in nature:
its spectral magnitude is non-zero only for some band concentrated
around the carrier frequency fc (sometimes called narrowband noise).
51
Example
• If white noise with PSD of N0/2 is passed through an ideal
bandpass
p filter,, then the PSD of the noise that enters the
receiver is given by
N0
, f fC B
SN ( f ) 2
0, otherwise
Power PN = 2N0B
• Autocorrelation function
Rn() = 2N0Bsinc(2B)cos(2fc)
– which follows from (3.1) by g ( t ) G ( )
applying
l i ththe ffrequency-shift
hift
g ( t ) 2 cos 0 t [ G ( 0 ) G ( 0 )]
property of the Fourier transform
• Samples taken at frequency 2B are still uncorrelated.
uncorrelated
Rn() = 0, = k/(2B), k = 1, 2, …
52
Decomposition of Bandpass Noise
• Consider bandpass noise within f fC B with any PSD
((i.e.,, not necessarilyy white as in the previous
p example)
p )
• Consider a frequency slice ∆f at frequencies fk and −fk.
• For ∆f small:
n k ( t ) a k cos( 2 f k t k )
– θk: a random p phase assumed independent
p and uniformly
y
distributed in the range [0, 2)
– ak: a random amplitude.
∆f
-ffk fk
53
Representation of Bandpass Noise
• The complete bandpass noise waveform n(t) can be
constructed by summing up such sinusoids over the entire
band ii.e.,
band, e
n(t ) nk (t ) ak cos(2 f k t k ) f k f c k f (3.2)
k k
nc(t) nc(t)
ns(t) ns(t)
55
Properties of Baseband Noise
• If the noise n(t) has zero mean, then nc(t) and ns(t) have
zero mean.
• If the noise n(t) is Gaussian, then nc(t) and ns(t) are
Gaussian.
• If the noise n(t) is stationary, then nc(t) and ns(t) are
stationary.
• If the noise n(t) is Gaussian and its power spectral density
S( f ) is symmetric with respect to the central frequency fc,
th nc(t)
then ( ) and
d ns(t)
( ) are statistical
t ti ti l independent.
i d d t
• The components nc(t) and ns(t) have the same variance (=
power) as n(t). )
56
Power Spectral Density
• Further, each baseband noise waveform will have the
same PSD:
S N ( f f c ) S N ( f f c ), | f | B (3.4)
Sc ( f ) S s ( f )
0, otherwise
• This is analogous to
g (t ) G ( )
g (t )2 cos 0t [G ( 0 ) G ( 0 )]
– A rigorous
g p
proof can be found in A. Papoulis,
p , Probability,
y, Random
Variables, and Stochastic Processes, McGraw-Hill.
– The PSD can also be seen from the expressions (3.2) and (3.3)
where each of nc(t) and ns(t) consists of a sum of closely spaced
base-band sinusoids.
57
Noise Power
• For ideally filtered narrowband noise, the PSD of nc(t)
and ns((t)) is therefore g
given by
y Sc(f)=Ss(f)
N 0 , | f | B
Sc ( f ) S s ( f ) (3.5)
0, otherwise
ns ( t )
– (t ) tan 1 : the p
phase of the noise
nc (t )
θ( ) ≡ 2 fct + (t)
θ(t) ()
59
Distribution of Envelop and Phase
• It can be shown that if nc(t) and ns(t) are Gaussian-
distributed,, then the magnitude
g ( ) has a Rayleigh
r(t) y g
distribution, and the phase (t) is uniformly distributed.
60
Summary
• White noise: PSD is constant over an infinite bandwidth.
• Gaussian noise: PDF is Gaussian
Gaussian.
• Bandpass noise
– In-phase
In phase and quadrature compoments nc(t) and ns(t) are low-pass
low pass
random processes.
– nc(t) and ns(t) have the same PSD.
– nc(t) and ns(t) have the same variance as the band-pass noise n(t).
– Such properties will be pivotal to the performance analysis of
bandpass communication systems.
• The in-phase/quadrature representation and phasor
representation
p are not onlyy basic to the characterization of
bandpass noise itself, but also to the analysis of bandpass
communication systems.
61
EE2--4: Communication Systems
EE2
62
Outline
• SNR of baseband analog transmission
• Revision of AM
• SNR of DSB-SC
• References
– Notes of Communication Systems
Systems, Chap
Chap. 3
3.1-3.3.2.
1-3 3 2
– Haykin & Moher, Communication Systems, 5th ed., Chap. 6
– Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap. 12
63
Noise in Analog Communication Systems
• How do various analog modulation schemes perform in
the presence of noise?
• Which scheme performs best?
• How can we measure its performance?
64
SNR
• We must find a way to quantify (= to measure) the
performance of a modulation scheme.
p
• We use the signal-to-noise ratio (SNR) at the output of the
receiver:
average power of message signal at the receiver output PS
SNRo
average power of noise at the receiver output PN
66
A Baseband Communication System
• It does not use
modulation
• It is suitable for
transmission over wires
• The power it transmits
is identical to the
message power: PT = P
• No attenuation: PS = PT =
P
• The results can be
extended to band
band-pass
pass
systems
67
Output SNR
• Average signal (= message) power
P = the area under the triangular curve
• Assume: Additive, white noise with power spectral
density PSD = N0/2
• A
Average noise
i power att ththe receiver
i
PN = area under the straight line = 2W N0/2 = WN0
• SNR at the receiver output:
PT
SNRbaseband
N 0W
– Note: Assume no propagation loss
• Improve the SNR by:
– increasing the transmitted power (PT ↑),
– restricting the message bandwidth (W ↓),
– making the channel/receiver less noisy (N0 ↓).
↓)
68
Revision: AM
69
Signal Recovery
n(t)
Receiver model
1) 1 A m p : use an envelopel d
detector.
t t
This is the case in almost all commercial AM radio
receivers
receivers.
Simple circuit to make radio receivers cheap.
2) Otherwise: use synchronous detection = product
detection = coherent detection
Th terms
The t detection
d t ti andd demodulation
d d l ti are used
d iinterchangeably.
t h bl
70
Synchronous Detection for AM
• Multiply the waveform at the receiver with a local carrier of
the same frequency (and phase) as the carrier used at the
t
transmitter:
itt
2 cos(2 f c t ) s (t ) AM [ A m(t )]2 cos 2 (2 f c t )
[ A m(t )][1 cos(4 f c t )]
A m(t )
71
DSB--SC
DSB
• Double-sideband suppressed carrier (DSB-SC)
s (t ) DSB SC Am(t ) cos(2 f c t )
• Signal recovery: with synchronous detection only
• The received noisy signal is
x (t ) s (t ) n (t )
s(t ) nc (t ) cos(2f c t ) ns (t ) sin(2f c t )
Am(t ) cos(2f c t ) nc (t ) cos(2f c t ) ns (t ) sin(2f ct )
A (t ) nc (t )] cos((2f c t ) ns (t ) sin(
[ Am i (2f c t )
y(t)
2
72
Synchronous Detection for DSB-
DSB-SC
• Multiply with 2cos(2fct):
y (t ) 2 cos(2 f c t ) x(t )
Am(t )2 cos 2 (2 f c t ) nc (t )2 cos 2 (2 f c t ) ns (t ) sin(4 f c t )
Am(t )[1 cos(4 f c t )] nc (t )[1 cos(4 f c t )] ns (t ) sin(4 f c t )
• Use a LPF to keep
~
y Am(t ) nc (t )
• Signal power at the receiver output:
73
Comparison
• SNR at the receiver output:
A2 P
SNRo
2 N 0W
• To which transmitted power does this correspond?
A2 P
PT E{ A m(t ) cos (2 f c t )}
2 2 2
2
• So
PT
SNRo SNRDSB SC
N0W
• Comparison with
PT
SNRbaseband SNRDSB SC SNRbaseband
N 0W
• Conclusion: DSB
DSB-SC
SC system has the same SNR performance as a
baseband system.
74
EE2--4: Communication Systems
EE2
75
Outline
• Noise in SSB
• Noise in standard AM
– Coherent detection
(of theoretic interest only)
– Envelope detection
• References
– Notes of Communication Systems, Chap. 3.3.3
3.3.3-3.3.4.
3.3.4.
– Haykin & Moher, Communication Systems, 5th ed., Chap. 6
– Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap. 12
76
SSB Modulation
• Consider single (lower) sideband AM:
A A
s(t )SSB m(t ) cos 2f ct mˆ (t ) sin
i 2f ct
2 2
where mˆ (t ) is the Hilbert transform of m(t).
• mˆ (t ) is obtained by passing m(t) through a linear filter
with transfer function −jsgn(f ).
• mˆ (t ) and m(t) have the same power P .
• The average power is A2P/4.
77
Noise in SSB
• Receiver signal x(t) = s(t) + n(t).
• Apply a band-pass
band pass filter on the lower sideband
sideband.
• Still denote by nc(t) the lower-sideband noise (different
from the double
double-sideband
sideband noise in DSB).
• Using coherent detection:
y (t ) x(t ) 2 cos(2 f c t )
A A
m(t ) nc (t ) m(t ) nc (t ) cos(4 f c t )
2 2
A
mˆ (t ) ns (t ) sin(4 f c t )
2
• After low-pass filtering,
A
y (t ) m(t ) nc (t )
2
78
Noise Power
• Noise power for nc(t) = that for band-pass noise =
N0W ((halved compared
p to DSB)) ((recall ((3.4))
))
SN(f)
N0/2
f
-ffc -ffc+W 0 fc-W
W fc
Lower-sideband noise
SNc(f)
N0/2
f
-W
W 0 W
Baseband noise 79
Output SNR
• Signal power A2P/4
• SNR at output
A2 P
SNRSSB
4 N 0W
• For a baseband system with the same transmitted power
A2P/4
A2 P
SNRbaseband
4 N 0W
• Conclusion: SSB achieves the same SNR performance
as DSB-SC (and the baseband model) but only requires
half the band-width.
80
Standard AM: Synchronous Detection
• Pre-detection signal:
x(t ) [ A m(t )]cos(2f ct ) n(t )
[ A m(t )]cos(2f ct ) nc (t ) cos(2f ct ) ns (t ) sin(2f ct )
[ A m(t ) nc (t )]cos(2f ct ) ns (t ) sin(2f ct )
• LPF
~
y A m(t ) nc (t )
81
Output SNR
• Signal power at the receiver output:
PS E{m2(t)} P
• Noise power:
PN 2 N 0W
• SNR at the receiver output:
P
SNRo SNRAM
2 N0W
• Transmitted power
A2 P A2 P
PT
2 2 2
82
Comparison
• SNR of a baseband signal with the same transmitted
power:
p A2 P
SNRbaseband
2 N 0W
• Thus:
P
SNRAM 2 SNRbaseband
A P
• Note:
P
1
A P
2
83
Model of AM Radio Receiver
84
Envelope Detection for Standard AM
• Phasor diagram of the signals present at an AM
receiver
x(t)
• Envelope
y (t ) envelope of x (t )
[ A m(t ) nc (t )]2 ns (t ) 2
85
Small Noise Case
• 1st Approximation: (a) Small Noise Case
n ( t ) [ A m ( t )]
• Then
ns (t ) [ A m(t ) nc (t )]
• Then Identical to the post-
y (t ) [ A m(t ) nc (t )] detection signal in the
case off synchronous
h
detection!
• Thus P
SNRo SNRenv
2 N 0W
• And in terms of baseband SNR:
P
SNRenv 2 SNRbaseband
A P
• Valid for small noise onl
only!!
86
Large Noise Case
• 2nd Approximation: (b) Large Noise Case
n(t ) [ A m(t )]
• Isolate the small quantity:
y2 (t ) [ A m(t) nc (t )]2 ns2 (t)
( A m(t))2 nc2 (t ) 2( A m(t ))nc (t ) ns2 (t )
( A m(t ))2
2( A m(t))nc (t )
[nc (t ) ns (t )]1 2
2 2
2
nc (t) ns (t ) nc (t) ns (t )
2 2 2
2[ A m(t )]nc (t )
y (t ) [n (t ) n (t )]1 2
2 2
c
2
s
nc (t ) ns (t )
2
2[ A m(t )]nc (t )
E (t )1
2
n 2
En (t ) nc2 (t ) ns2 (t )
En (t
(t )
87
Large Noise Case: Threshold Effect
• From the phasor diagram: nc(t) = En(t) cosθn(t)
• Then:
2[ A m(t)]cosn (t)
y(t) En (t) 1
En (t)
x
• U
Use 1 x 1 for x 1
2
[ A m(t )] cos n (t )
y (t ) En (t )1
En ( t )
En (t ) [ A m(t )] cos n (t )
• Noise is multiplicative here!
• No term proportional to the message!
• R
Result:
lt a threshold
th ff t as below
h ld effect, b l some carrier
i power llevell ((very
low A), the performance of the detector deteriorates very rapidly.
88
Summary
A: carrier amplitude
amplitude, P: power of message signal
signal, N0: single-sided PSD of noise
noise,
W: message bandwidth.
89
EE2--4: Communication Systems
EE2
90
Outline
• Recap of FM
• PSD of noise
• References
– Notes of Communication Systems, Chap. 3.4.1-3.4.2.
– Haykin & Moher, Communication Systems, 5th ed., Chap. 6
– Lathi,
Lathi Modern Digital and Analog Communication Systems
Systems,
3rd ed., Chap. 12
91
Frequency Modulation
• Fundamental difference between AM and FM:
1 d i ( t )
f i (t )
2 dt
93
FM
• In FM: the instantaneous frequency of the carrier varies
linearlyy with the message:g
fi(t) = fc + kf m(t)
– where kf is the frequency sensitivity of the modulator.
• Hence (assuming i(0)=0):
t t
i (t ) 2 fi ( )d 2 f c t 2 k f m( )d
0 0
• Modulated signal:
s(t ) A cos 2f c t 2k f m( )d
t
0
• Note:
– (a) The envelope is constant
– (b) Signal s(t) is a non-linear function of the message signal m(t).
94
Bandwidth of FM
• mp = max|m(t)|: peak message amplitude.
• fc − kf mp < instantaneous frequency
q y < fc + kf mp
• Define: frequency deviation = the deviation of the
instantaneous frequency from the carrier frequency:
∆f = kf mp
• Define: deviation ratio:
f / W
– W: the message bandwidth.
β: FM bandwidth 2x message
– Small β g bandwidth ((narrow-band FM))
– Large β: FM bandwidth >> 2x message bandwidth (wide-band FM)
• Carson’s rule of thumb:
BT = 2W(β+1) = 2(∆f + W)
– β <<1 ֜ BT ≈ 2W (as in AM)
– β >>1 ֜ BT ≈ 2∆ff
95
FM Receiver
n(t)
96
Linear Argument at High SNR
• FM is nonlinear (modulation & demodulation), meaning superposition
doesn’t hold.
• Nonetheless, it can be shown that for high SNR, noise output and
message signal are approximately independent of each other:
Output ≈ Message + Noise (i.e., no other nonlinear terms).
97
Output Signal Power Without Noise
• Instantaneous frequency of the input signal:
fi fc k f m(t)
• Output of discriminator:
k f m((t )
• So, output signal power:
PS k 2f P
– P : the average power of the message signal
98
Output Signal with Noise
• In the presence of additive noise, the real
predetection signal
p g is
x(t ) A cos 2 f c t 2 k f m( )d
t
0
nc (t ) cos(2 f c t ) ns (t ) sin(2 f c t )
99
Phase Noise
Phasor diagram of
the FM carrier and
ns(t)
noise signals
A nc(t)
• Instantaneous
I t t phase
h noise:
i
ns (t )
i (t) tan1
A nc (t )
• For large carrier power (large A):
ns (t ) ns (t )
i (t ) tan 1
A A
• Discriminator output = instantaneous frequency:
1 di (t ) 1 dns (t )
f i (t )
2 dt 2 A dt
100
Discriminator Output
• The discriminator output in the presence of both
g
signal and noise:
1 dn
d s (t )
k f m( t )
2A dt
101
Noise PSD
• It follows from (2.1) that
So ( f ) | H ( f ) |2 Si ( f )
– Si(f ): PSD of input signal
– So(f ): PSD of output signal
– H(f )): transfer
t f function
f ti off the
th system t
• Then: PSD of nd (t ) | j 2 f |2 PSD of ns (t )
PSD of ns (t ) N 0 within band
BT
2
PSD of nd (t ) | 2 f |2 N 0 f BT / 2
2
1 dns (t ) 1 f2
PSD of fi (t ) | 2 f | N 0 2 N 0
2
2 A dt 2 A A
• Aft
After the
th LPF,
LPF the
th PSD off noise
i output
t t no(t)
( ) is
i restricted
t i t d iin
the band ±W
f2
S No ( f ) 2 N 0 f W (6 1)
(6.1)
A
102
Power Spectral Densities
SNs(f)
(a) Power spectral density of quadrature component ns(t) of narrowband noise n(t).
(b) Power spectral density of noise nd(t) at the discriminator output.
(c) Power spectral density of noise no(t) at the receiver output.
103
EE2--4: Communication Systems
EE2
Lecture 7: Pre/de
Pre/de--emphasis for FM and
Comparison of Analog Systems
Dr. Cong Ling
Department of Electrical and Electronic Engineering
104
Outline
• Derivation of FM output SNR
• Comparison with AM
• References
– Notes of Communication Systems, Chap. 3.4.2-3.5.
– Haykin & Moher, Communication Systems, 5th ed., Chap. 6
– Lathi,
Lathi Modern Digital and Analog Communication Systems
Systems,
3rd ed., Chap. 12
105
Noise Power
• Average noise power at the receiver output:
W
PN S No ( f )df
W
• Thus,
Thus from (6
(6.1)
1)
W f2 2 N 0W 3
PN 2
N 0 df (7.1)
W A 3 A2
• Average noise power at the output of a FM receiver
1
carrier power A2
• A ↑ ֜ Noise↓,
Noise↓ called the quieting effect
106
Output SNR
• Since PS k 2f P , the output SNR
2 2
PS 3 A k f P
SNRO 3
SNRFM
PN 2 N 0W
• Transmitted power of an FM waveform:
A2
PT
2
PT k f mp
• From SNRbaseband and :
N 0W W
3k 2f P 2 P
SNR FM 2
SNR baseband 3 2
SNRbaseband
W mp
2 SNRbaseband (could be much higher than AM)
• Valid when the carrier power is large compared with the
noise power
107
Threshold effect
• The FM detector exhibits a more pronounced threshold
p detector.
effect than the AM envelope
• The threshold point occurs around when signal power is
10 times noise power:
A2
10, BT 2W ( 1)
2 N 0 BT
• Below the threshold the FM receiver breaks (i.e.,
significantly deteriorated).
• Can be analyzed by examining the phasor diagram
ns(t)
A nc(t)
108
Qualitative Discussion
• As the noise changes randomly, the point P1 wanders
around P2
– High SNR: change of angle is small
– Low SNR: P1 occasionally sweeps around origin, resulting in
changes
h off 2
2 in
i a short
h t ti
time
109
Improve Output SNR
111
Improvement Factor
• Assume an ideal pair of pre/de-emphasis filters
Hde ( f ) 1/ Hpe ( f ),
) f W
• PSF of noise at the output of de-emphasis filter
f2 2 f2
2
N 0 H de ( f ) , f BT / 2, recall S No ( f ) 2 N 0
A A
• Average power of noise with de
de-emphasis
emphasis
W f2
PN
2
2
H de ( f ) N 0 df
W A
• Improvement factor (using (7.1))
2 N0W 3
3
PN without pre / de - emphasis 2 2W
I W 2 3A 2 W
PN with p
pre / de - emphasis
p W A2 H de ( f ) N0dff 3W f H de
f 2
( f )
2
df
d
112
Example Circuits
• (a) Pre-emphasis filter
H pe ( f ) 1 jf / f 0
f 0 1/ (2 rC ), R r , 2 frC 1
• (b) D
De-emphasis
h i filt
filter
1
H de ( f )
1 jf / f 0
• Improvement
2W 3
I W
3 f 2 / (1 f 2 / f 0 2 )df
W
(W / f 0 )3
3[(W / f 0 ) tan 1 (W / f 0 )]
• In commercial FM,
FM W = 15 kHzkHz, f0 = 2.1
2 1 kHz
I = 22 13 dB (a significant gain)
113
Comparison of Analogue Systems
• Assumptions:
– single-tone
g modulation,, i.e.: m(t) ( fmt);
( ) = Am cos(2 );
– the message bandwidth W = fm;
– for the AM system, µ = 1;
– for the FM system, β = 5 (which is what is used in commercial FM
transmission, with ∆f = 75 kHz, and W = 15 kHz).
• With these assumptions
assumptions, we find that the SNR
expressions for the various modulation schemes become:
SNRDSB SC SNRbaseband SNRSSB
1
SNRAM SNRbaseband
3
3 75 without pre/de-
SNRFM 2 SNRbaseband SNRbaseband emphasis
2 2
114
Performance of Analog Systems
115
Conclusions
• (Full) AM: The SNR performance is 4.8 dB worse than a
baseband system, and the transmission bandwidth is BT =
2W .
• DSB
DSB: Th
The SNR performance
f iis id
identical
ti l tto a b
baseband
b d
system, and the transmission bandwidth is BT = 2W.
117
Outline
• Introduction to digital communication
• PCM
• Companding
• References
– Notes of Communication Systems, Chap. 4.1-4.3
– Haykin & Moher
Moher, Communication Systems,
Systems 5th ed
ed., Chap
Chap. 7
– Lathi, Modern Digital and Analog Communication Systems,
3rd ed., Chap. 6
118
Block Diagram of Digital Communication
119
Why Digital?
• Advantages:
– Digital
g signals
g are more immune to channel noise by y using
g channel
coding (perfect decoding is possible!)
– Repeaters along the transmission path can detect a digital signal
and retransmit a new noise
noise-free
free signal
– Digital signals derived from all types of analog sources can be
represented using a uniform format
– Digital signals are easier to process by using microprocessors and
VLSI (e.g., digital signal processors, FPGA)
– Digital systems are flexible and allow for implementation of
sophisticated functions and control
– More and more things are digital…
• For digital communication: analog signals are converted to
digital.
120
Sampling
• How densely should we sample an analog signal so that
p
we can reproduce its form accurately?
y
• A signal the spectrum of which is band-limited to W Hz,
can be reconstructed exactly from its samples, if they are
taken uniformly at a rate of R ≥ 2W Hz.
• Nyquist frequency: fs = 2W Hz
121
Quantization
• Quantization is the process of transforming the sample
p
amplitude into a discrete amplitude
p taken from a finite set
of possible amplitudes.
• The more levels, the better approximation.
• Don’t need too many levels (human sense can only detect
finite differences).
• Quantizers can be of a uniform or nonuniform type.
122
Quantization Noise
• Quantization noise: the error between the input signal and
p signal
the output g
123
Variance of Quantization Noise
• ∆: gap between quantizing levels (of a uniform quantizer)
• q: Quantization error = a random variable in the range
q
2 2
• If ∆ is sufficiently small,
small it is reasonable to assume that q is
uniformly distributed over this range:
1
, q
fQ (q) 2 2
0, otherwise
• Noise variance
1 /2 2
PN E{e } q fQ (q)dq q dq
2 2
/2
3 /2
1q 1 3 ()3 2
3
/2
24 24 12
124
SNR
• Assume: the encoded symbol has n bits
– the maximum number of qquantizing
g levels is L = 2n
– maximum peak-to-peak dynamic range of the quantizer = 2n∆
• P: power of the message signal
• mp = max |m(t)|: maximum absolute value of the message
signal
• Assume: the message signal fully loads the quantizer:
1
m p 2n 2n 1 (8.1)
2
• SNR at the quantizer output:
PS P 12 P
SNRo 2 2
PN / 12
125
SNR
• From (8.1) mp 2m p 4m 2p
n 1
n
2
2 2 22 n
• 12 P 3P 2 n
SNRo 22 (8.2)
4 m2p mp
•
22 n
• I dB,
In dB
3P
SNRo (dB) 10 log10 (2 ) 10 log10 2
2n
mp
3P
20n log10 2 10 log10 2
m
p
3P
6n 10 log10 2 (dB)
m
p
• Hence, each extra bit in the encoder adds 6 dB to the output SNR
of the q
quantizer.
• Recognize the tradeoff between SNR and n (i.e., rate, or bandwidth).
126
Example
• Sinusoidal message signal: m(t) = Am cos(2 fmt).
• Average signal power:
Am2
P
2
• Maximum signal value: mp = Am.
• Substitute into (8
(8.2):
2):
3 Am2 2 n 3 2 n
SNRo 2
2 2
2 Am 2
• In dB
SNRo (dB) 6n 1.8
1 8 dB
• Audio CDs: n = 16 ֜ SNR > 90 dB
127
Pulse--Coded Modulation (PCM)
Pulse
129
Problem With Uniform Quantization
• Problem: the output SNR is adversely affected by peak to
g p
average power ratio.
• Typically small signal amplitudes occur more often than
large signal amplitudes.
– The signal does not use the entire range of quantization levels
available with equal probabilities.
– Small amplitudes are not represented as well as large amplitudes
amplitudes,
as they are more susceptible to quantization noise.
130
Solution: Nonuniform quantization
• Nonuniform quantization uses quantization levels of
p g, denser at small signal
variable spacing, g amplitudes,
p ,
broader at large amplitudes.
uniform
if nonuniform
if
quantized quantized
131
Companding = Compressing + Expanding
• A practical (and equivalent) solution to nonuniform quantization:
– Compress the signal first
– Quantize it (using a uniform quantizer)
– Transmit it
– Expand
E d it
• Companding is the corresponding to pre-emphasis and de-emphasis
scheme used for FM.
• Predistort a message signal in order to achieve better performance in
the presence of noise, and then remove the distortion at the receiver.
• The
Th exactt SNR gaini obtained
bt i d with
ith companding
di depends
d d on the
th exactt
form of the compression used.
• With pproper
p companding,
p g, the output
p SNR can be made insensitive to
peak to average power ratio.
132
µ-Law vs. A
A--Law
134
Demo
• Music playing in Windows Sound Recorder
135
Summary
• Digitization of signals requires
– Sampling:
p g a signal
g of bandwidth W is sampled
p at the Nyquist
yq
frequency 2W.
– Quantization: the link between analog waveforms and digital
representation.
representation
• SNR (under high-resolution assumption)
3P
SNRo (dB) 6n 10 log
l 10 2 (dB)
m
p
• Companding can improve SNR.
• PCM is a common method of representing audio signals.
– In a strict sense, “pulse coded modulation” is in fact a (crude)
source coding technique (i (i.e,
e method of digitally representing
analog information).
– There are more advanced source coding (compression)
techniques in information theory.
136
EE2--4: Communication Systems
EE2
137
Outline
• Line coding
• Performance of baseband
digital transmission
– Model
– Bit error rate
• References
– Notes of Communication
Systems, Chap. 4.4
– Haykin & Moher, Communication
y
Systems, , 5th ed.,, Chap.
p 8
– Lathi, Modern Digital and Analog
Communication Systems, 3rd ed.,
Chap. 7
138
Line Coding
• The bits of PCM, DPCM etc need to be converted into
g
some electrical signals.
139
Line Codes
Unipolar nonreturn-to-zero
(NRZ) signaling (on-off
signaling)
Bipolar RZ signaling
Manchester code
140
Analog and Digital Communications
• Different goals between analog and digital communication
y
systems:
– Analog communication systems: to reproduce the transmitted
waveform accurately. ֜ Use signal to noise ratio to assess the
quality of the system
– Digital communication systems: the transmitted symbol to be
identified correctly by the receiver ֜ Use the probability of error of
the receiver to assess the quality of the system
141
Model of Binary Baseband
C
Communication
i ti System
S t
T
Lowpass
Lowpass
filter
filter
T
n(t)
n(t) T
144
Errors
• Two cases of decision error:
– ((i)) a symbol
y 0 was transmitted,, but a symbol
y 1 was chosen
– (ii) a symbol 1 was transmitted, but a symbol 0 was chosen
145
Case (i)
• Probability of (i) occurring = (Probability of an error,
given symbol
g y 0 was transmitted)) × ((Probabilityy of a 0 to
be transmitted in the first place):
p(i) = Pe0 × p0
where:
– p0: the a priori probability of transmitting a symbol 0
– Pe0: the conditional probability of error
error, given that symbol 0 was
transmitted:
1 n2
Pe 0 exp dn
2
T 2 2
146
Case (ii)
• Probability of (ii) occurring = (Probability of an error,
given symbol
g y 1 was transmitted)) × ((Probabilityy of a 1 to
be transmitted in the first place):
p(ii) = Pe1 × p1
where:
– p1: the a priori probability of transmitting a symbol 1
– Pe1: the conditional probability of error
error, given that symbol 0 was
transmitted:
T 1 ( n A)2
Pe1 exp dn
2
2 2
147
Total Error Probability
• Total error probability:
Pe (T ) p(i ) p(ii )
p1 Pe1 (1 p1 ) Pe 0
T 1 (n A) 2 1 n2
p1 exp dn (1 p1 ) T exp 2 dn
2 2 2
2 2
148
Derivation
• Leibnitz rule of differentiating an integral with respect
to a p
parameter: if
b( )
I ( ) f ( x; )dx
a ( )
• then
dI ( ) db( ) da ( ) b ( ) f ( x; )
f (b( ); ) f ( a ( ); ) dx
d d d a ( )
• Therefore
dPe (T ) 1 (T A)2 1 T2
p1 exp (1 p1 ) exp 2 0
dT 2 2 2
2 2
p1 T 2 (T A) 2
exp
1 p1 2 2
T 2 T 2 A2 2TA A(2T A)
exp exp
2 2
2 2
149
Optimum Threshold
p1 A(2T A) p1
ln 2 ln2
A(2T A)
1 p1 2 2 1 p1
2 2 p1 2 p1 A
ln 2T A T ln
A 1 p1 A 1 p1 2
• Equi-probable symbols (p1 = p0 = 1 − p1) ֜ T = A/2.
• For
F equi-probable
i b bl symbols,
b l it can b
be shown
h th
thatt Pe0 =
Pe1.
• Probability of total error:
Pe p(i ) p(ii ) p0 Pe 0 p1 Pe1 Pe 0 Pe1
since p0 = p1 = 1/2, and Pe0 = Pe1.
150
Calculation of Pe
• Define a new variable of integration
n 1
z dzd dn d dz
d dn d
– When n = A/2, z = A/(2σ).
– When n = ∞, z = ∞.
1
z 2 /2
• Then Pe 0 e dz
2 A /(2 )
1
z 2 /2
e dz
2 A /(2 )
152
Probability
y of Bit Error
153
Q-function
• Upper bounds and good
pp
approximations:
1 x2 / 2
Q( x) e , x0
2 x
1 x2 / 2
Q( x) e ,x0
2
154
Applications to Ethernet and DSL
• 100BASE-TX
– One of the dominant forms of fast Ethernet,, transmitting
g up
p to 100
Mbps over twisted pair.
– The link is a maximum of 100 meters in length.
– Line
Li codes:
d NRZ NRZ IInvertt MLT-3.
MLT 3
• Digital subscriber line (DSL)
– Broadband over twist pair
pair.
– Line code 2B1Q achieved bit error rate of 10-7 at 160 kbps.
– ADSL and VDSL adopt p discrete multitone modulation ((DMT)) for
higher data rates.
155
EE2--4: Communication Systems
EE2
156
Outline
• References
– Notes of Communication Systems, Chap. 4.5
– Haykin & Moher, Communication Systems, 5th ed., Chap. 9
– Lathi, Modern Digital and Analog Communication Systems,
3rd ed., Chap. 13
157
Digital Modulation
• Three Basic Forms of Signaling Binary Information
(a) Amplitude-shift
keying (ASK).
(c) Frequency
Frequency-shift
shift
keying (FSK).
158
Demodulation
161
Bit Error Rate
• Reminder: The in-phase noise component nc(t) has the
g
same variance as the original band-pass
p noise n(t)
()
– The received signal is identical to that for baseband digital
transmission
~ (t )
– The
Th sample l values
l off y (t will
ill h
have PDF
PDFs that
th t are id
identical
ti l tto
those of the baseband case
• For ASK the statistics of the receiver signal are identical to
those of a baseband system
• Theep
probability
obab y o of e
error
o for
o ASK
S is
s the
e sa
same e as for
o the
e
baseband case
• Assume equiprobable
q p transmission of 0s and 1s.
• Then the decision threshold must be A/2 and the
probability of error is given by: A
Pe, ASK Q
2
162
PSK
• Phase shift keying (PSK)
s(t) = A(t) cos(2 fct), A(t) { א−A, A}
• Use
U coherent
h td
detection
t ti again, i tto eventually
t ll gett th
the
detection signal:
y (t ) A(t ) nc (t )
• Probability density functions for PSK for equiprobable 0s
and 1s in noise (use threshold 0 for detection):
– (a): symbol 0 transmitted
– ((b):
) symbol
y 1 transmitted
163
Analysis
• Conditional error probabilities:
1 (n A)2
Pe0 exp dn
2 2
0 2
0 1 (n A)2
Pe1 exp dn
2 2
2
165
FSK
• Frequency Shift Keying (FSK)
( f0t),
s0((t)) = A cos(2 ), if symbol
y 0 is transmitted
s1(t) = A cos(2 f1t), if symbol 1 is transmitted
• Symbol recovery:
– Use two sets of coherent detectors, one operating at a frequency f0
and the other at f1.
166
Output
• Each branch = an ASK detector
A + noise
i if symbol
b l presentt
LPF output on each branch =
noise if symbol not present
167
Bit Error Rate for FSK
• Set detection threshold to 0
• Difference from PSK: the noise term is now n1(t) − n0(t).
168
The Sum of Two R.V.
• Noise is the sum or difference of two independent zero
mean random variables:
– x1: a random variable with variance σ12
– x2: a random variables with variance σ22
• What is the variance of y ≡ x1 ± x2?
• By definition
y2 E{ y 2 } E{ y}2 E{( x1 x2 ) 2 }
E{x12 2 x1 x2 x22 } E{x12 } E{x1 x2 } E{x22 }
• For independent variables: E{x1x2} = E{x1}E{x2}
• For zero-mean random variables:
E{x1} = E{x2} = 0 ֜ E{x1x2} = 0
• So
y2 E{x12 } E{x22 } 12 22
169
Comparison of Three Schemes
170
Comment
• To achieve the same error probability (fixed Pe):
171
Q-function
172
Examples
• Consider binary PSK modulation. Assume the carrier
p
amplitude A = 1 v,, and noise standard deviation = 1/3.
Determine the bit error probability.
– Answer: Pe = 1.03 10-3.
173
Application: GMSK
• Gaussian minimum shift keying (GMSK), a special form of
FSK p preceded byy Gaussian filtering,
g, is used in GSM
(Global Systems for Mobile Communications), a leading
cellular phone standard in the world.
– Also known as digital FM, built on some
of FM-related advantages of AMPS,
g
the first-generation analogg system
y
(30 KHz bandwidth).
– Binary data are passed through a
Gaussian filter to satisfy stringent
requirements of out-of-band radiation.
– Minimum Shift Keying: its spacing between the two frequencies of
FSK is minimum in a certain sense (see problem sheet).
– GMSK is allocated bandwidth of 200 kHz, shared among 32 users.
This provides a (30/200)x32
(30/200)x32=4 4.8
8 times improvement over AMPS
AMPS.
174
EE2--4: Communication Systems
EE2
175
Outline
• References
– Haykin & Moher, Communication Systems, 5th ed., Chap. 9
– Lathi, Modern Digital and Analog Communication Systems,
3 d ed.,
3rd d ChChap. 13
176
Noncoherent Demodulation
• Coherent demodulation assumes perfect synchronization.
– Needs a p
phase lock loop.
p
• However, accurate phase synchronization may be difficult
in a dynamic channel.
– Phase synchronization error is due to varying propagation delays,
frequency drift, instability of the local oscillator, effects of strong
noise ...
– Performance of coherent detection will degrade severely.
• When
e the
e ca
carrier
e pphase
ase is
suunknown,
o ,o one
e must
us rely
eyo on non-
o
coherent detection.
– No provision is made for carrier phase recovery.
• The phase is assumed to be uniformly distributed on [0,
2].
• Circuitry is simpler, but analysis is more difficult!
177
Noncoherent Demodulation of ASK
Signal
plus Bandpass Envelope Threshold Decision
noise filter detector device
in t = nTb
r ( r 2 A2 ) /( 2 2 ) Ar
f (r) e I 0 2 , r 0
2
179
Rayleigh Distribution
• Define a random variable R X Y where X and Y
2 2
are independent
p Gaussian with zero mean and variance 2
• R has Rayleigh distribution:
r 2 / ( 2 2 )
f R (r ) r
2
e , r0
• Hence x2 y 2 r2
r r
f R , ( r , ) e 2 2
e 2 2
2 2
2 2
• pdf of R:
r2
2 r
f R (r ) f R , (r , )d e 2 2
, r0
0 2
• pdf of 1
f ( ) f R , (r , )dr , [0, 2 ]
0 2
181
Rician Distribution
• If X has nonzero mean A, R has Rician distribution:
(r2 A2 )/(2 2 )
fR (r) 2 e
r
I0 (Ar
A
2 ), r 0
where 2
I 0 ( x) 1
2 0
e x cos d
is the modified zero-order Bessel function of the first kind.
182
Derivation
• Similarly,
f R , (r , )drd f X ,Y ( x, y )rdrd
( x A )2 y 2
1
e 2 2
rdrd
2 2
r 2 A2 2 Ar cos
1
e 2 2
rdrd
2 2
• Hence
r 2 A2 2 Ar cos
r
f R , ( r , ) e 2 2
2 2
Bessel function
• pdf of R:
r 2 A2 Ar cos
2 r 1 2
f R (r) f R , ( r, )d e 2 2
e 2
d , r0
0 2
2 0
183
Error Probability
• Let the threshold be A/2 for simplicity.
• The error pprobability
y is dominated by
y symbol
y 0,, and
is given by
1 r r 2 /(2 2 )
Pe e dr
2 A /2 2
Bandpass
filter
centered at f1
R1
If R1 > R2,
choose 0
If R1 < R2,
choose 1
R2
Bandpass
filter
centered at f2
185
Distribution of Envelope
• When a symbol 1 is sent, outputs of the BPFs
y1((t)) = n1((t))
y2(t) = n2(t) + A cos(2 f2t)
• Again, the first branch has Rayleigh distribution
2 2
r1 r1 /(2 )
f R1 (r1 ) 2 e , r1 0
186
Error Probability
• Error occurs if Rice < Rayleigh
Pe P( R2 R1 )
2
r2
e ( r2 2 A2 )/(2 2 ) Ar2
I0 ( 2 ) 2 e r1 r12 /(2 2 )
dr1dr2
0 r2
( r2 2 A2 )/(
)/(2 2 )
I0 ( 2 )
2 2
r2 Ar2 r1 /( )
r1 /(2
2
e 2 e dr
d 1dr
d2
0 r2
r2
2 e (2 r2 2 A2 )/(2 2 )
I 0 ( Ar22 )dr2
0
A2 /(4 2 ) ( x 2 2 )/(2 2 )
e
1
2
x
2 e I 0 ( x2 )dx x 2r2 , A / 2
0
bn
input DPSK
an signal
bn-1
Delay Tb cos(ct)
188
Differential Demodulation
signal
plus Bandpass Lowpass Threshold
noise filter filter device
Decision
D i i
on an
Delay Tb
{bn-1} 1 1 -1 1 1 -1 1 1
Differentially encoded
1 1 -1 1 1 -1 1 1 1
sequence {bn}
Transmitted phase
0 0 0 0 0 0 0
(radians)
Output of lowpass filter
+ – – + – – + +
(polarity)
ec s o
Decision 1 -1 -1 1 -1 -1 1 1
190
Summary and Comparison
Scheme Bit Error Rate
Noncoherent
ASK, FSK
Coherent ASK Q(A/2)
Coherent
Coherent FSK Q(A/2) ASK, FSK
• Non-coherent
Non coherent demodulation has error performance slightly
worse than coherent demodulation, but approaches
coherent performance at high SNR.
192
Application: DPSK
• WLAN standard IEEE 802.11b
• Bluetooth2
• Digital audio broadcast (DAB): DPSK + OFDM (orthogonal
frequency division multiplexing)
• Inmarsat (International Maritime Satellite Organization):
now a London-based mobile satellite company
193
EE2--4: Communication Systems
EE2
194
Outline
• What is information theory?
• Entropy
– Definition of information
– Entropy of a source
• Source coding
– Source coding theorem
– Huffman coding
• References
– N
Notes
t off Communication
C i ti S Systems,
t Ch
Chap. 5
5.1-5.4
154
– Haykin & Moher, Communication Systems, 5th ed., Chap. 10
– Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap. 15
195
Model of a Digital Communication System
196
What is Information Theory
• C. E. Shannon, ”A mathematical theory of
communication,”, Bell System
y Technical
Journal, 1948.
• Two fundamental questions in
communication theory:
– What is the ultimate limit on data compression?
– What is the ultimate transmission rate off reliable
communication over noisy channels?
• Shannon showed reliable (i(i.e.,
e error-free)
error free)
communication is possible for all rates below
channel capacity (using channel coding).
• Any source can be represented in bits at any
rate above entropy (using source coding).
– Rise of digital information technology
197
What is Information?
• Information: any new knowledge about something
• How can we measure it?
• Messages containing knowledge of a high probability of
occurrence ֜ Not very informative
• Messages containing knowledge of low probability of
occurrence ֜ More informative
• A small change in the probability of a certain output should
not change the information delivered by that output by a
large amount.
198
Definition
• The amount of information of a symbol s with
probability
p y p:
1
I (s) log
p
• Properties:
– p = 1 ֜ I(s)
( ) = 0: a symbol
y that is certain to occur contains no
information.
199
Example
• In communications, log base 2 is commonly used,
g in unit bit
resulting
• Example: Two symbols with equal probability p1 = p2
= 0.5
• Each symbol represents I(s) = log2(1/0.5) = 1 bit of
information.
1
• In summary: I ( s ) log 2 bits
p
200
Discrete Memoryless Source
• Suppose we have an information source emitting a
q
sequence of symbols
y from a finite alphabet:
p
S = {s1, s2, . . . , sK}
201
Source Entropy
• If symbol sk has occurred, then, by definition, we have received
1
I (sk ) log2 log2 pk
pk
bits of information.
202
Meaning of Entropy
• What information about a source does its entropy give us?
• It is the amount of uncertainty before we receive itit.
• It tells us how many bits of information per symbol we
expect to get on the average.
203
Example: Entropy of a Binary Source
• s1: occurs with probability p
• s0: occurs with probability 1 − p.
• Entropy of the source:
H ( S ) (1 p ) llog 2 (1 p ) p log
l 2 p H ( p)
H(p)
Maximum uncertainty
when p = 1/2.
p
204
Example: A Three-Symbol Alphabet
– A: occurs with probability 0.7
– B: occurs with probability 0.2
– C: occurs with probability 0.1
• Source entropy:
H ((SS ) 0.7 log2 (0.7) 0.2 log2 (0.2) 0.1 log2 (0.1)
0.7 0.515 0.2 2.322 0.1 3.322
1.157 bits/symbol
• How can we encode these symbols in order to
transmit them?
• We need 2 bits/symbol if encoded as
A = 00, B = 01, C = 10
• Entropy prediction: the average amount of information is
only 1.157 bits per symbol.
• We
W are wasting ti bits!
bit !
205
Source Coding Theory
• What is the minimum number of bits that are required to
transmit a p
particular symbol?
y
206
Average Codeword Length
• lk: the number of bits used to code the k-th symbol
• K: total number of symbols
• pk: the probability of occurrence of symbol k
• Define the average codeword length:
K
L p k lk
k 1
• It represents the average number of bits per symbol in the
source alphabet.
alphabet
• An idea to reduce average codeword length: symbols that
occur often should be encoded with short codewords;;
symbols that occur rarely may be encoded using the long
codewords.
207
Minimum Codeword Length
• What is the minimum codeword length for a particular
p
alphabet of source symbols?
y
• In a system with 2 symbols that are equally likely:
– Probability of each symbol to occur: p = 1/2
– Best one can do: encode each with 1 bit only: 0 or 1
208
The General Case
• S = {s1, . . . , sK}: an alphabet
• pk: the p
probability y of symbol
y sk to occur
• N : the number of symbols generated
• We expect Npk occurrences of sk
• Assume:
– The source is memoryless
– allll symbols
b l are iindependent
d d t
– the probability of occurrence of a typical sequence SN is:
p ( S N ) p1Np1 p2 Np2 ... pK NpK
209
Source Coding Theorem
• The number of bits required to represent a typical
q
sequence SN is
1
LN log 2 log 2 ( p1Np1 p2 Np2 ... pK NpK )
p(S N )
Np l 2 p1 Np
N 1 log l 2 p2 ... Np
N 2 log N K log
l 2 pK
K
N pk log 2 pk NH ( S )
k 1
211
Example
• The average
g codeword length:
g
212
Application: File Compression
• A drawback of Huffman coding is that it requires
knowledgeg of a pprobabilistic model,, which is not always
y
available a prior.
• Lempel-Ziv coding overcomes this practical limitation and
has become the standard algorithm for file compression.
– compress, gzip, GIF, TIFF,
PDF modem…
PDF, modem
– A text file can typically be
compressed to half of its
original size.
213
Summary
• The entropy of a discrete memoryless information source
K
H ( S ) pk log2 pk
k 1
214
EE2--4: Communication Systems
EE2
215
Outline
• Channel capacity
– Channel coding theorem
– Shannon formula
• Comparison with
practical systems
p y
• References
– Notes of Communication Systems, Chap. 5.5-5.6
– Haykin & Moher, Communication Systems, 5th ed., Chap. 10
– Lathi, Modern Digital and Analog Communication Systems,
3rd ed., Chap. 15
216
Channel Coding Theorem
• Shannon, “A mathematical theory of communication,”
1948
• For any channel, there exists an information capacity C
(whose calculation is beyond the scope of this course).
• Important implication:
– The basic limitation due to noise in a communication channel is not
on the reliability of communication, but rather, on the speed of
communication.
i ti
217
Shannon Capacity Formula
• For an additive white Gaussian noise (AWGN) channel,
p y is
the channel capacity
P
C B log 2 1 SNR B log 2 1 bps
N 0 B
– B: the bandwidth of the channel
– P: the average signal power at the receiver
– N0: the single-sided PSD of noise
• Important implication: We can communicate error free
t C bits
up to bit per second. d
• How can we achieve this rate?
– D
Design
i power error correcting
ti codes
d tto correctt as many errors as
possible (the next two lectures).
– Use the ideal modulation scheme that does not lose information in
the detection process.
218
Shannon Limit
• Tradeoff between power and bandwidth: to get the same
g capacity
target p y C, one mayy
– Increase power, decrease bandwidth; Unachievable
region
1/B
B
P 21/ B 1
B(2 1)
1/ B
N0 1/ B P/N0 (dB)
P 21/ B ln 2( B 2 )
lim lim 2
ln 2
B N
0
B B
0.693
0 693 1.6
1 6 dB
220
General Model of a Modulated
Communication System
221
Input/Output SNR
• The maximum rate at which information may arrive at the
receiver is
Cin B log
l 2 (1 SNR
S in )
– SNRin: the predetection signal-to-noise ratio at the input to the
demodulator
demodulator.
• The maximum rate at which information can leave the
receiver is
Co W log2 (1 SNRo )
– SNRo: the SNR at the output of the postdetection filter.
• For the ideal modulation scheme:
Cin = Co ֜ 1 + SNRo = [1 + SNRin]B/W
• For high SNR,
SNR
SNRo SNRinB/W
• It seems that spreading
p g the bandwidth would make the
output SNR increase exponentially (not true).
222
A More Elaborate Analysis
• If the channel noise has a double-sided white PSD of N0/2
֜ the average
g noise p power at the demodulator will be N0B.
• If the signal power is P :
P W P
SNRini
N 0 B B N 0W
P
SNRbaseband: the baseband SNR
N 0W
• Increasing B will reduce SNRin, and thus
SNRbaseband
W logg 2 ((1 SNRo ) B logg 2 1
B /W
B /W lim 1 e
x
x
e SNRbaseband as B / W
223
SNR vs. Bandwidth
dB) B/W=12
B/W 2
B/W=2
SNRo (d
B/W=1
SNRbaseband (dB)
224
Implications to Analog Systems
• A bandwidth spreading ratio of B/W=1 corresponds to both
SSB and baseband.
• A bandwidth spreading ratio of B/W=2 corresponds to
DSB and AM (the ideal system would
have SNRo SNRbase
2
/4 )).
• A bandwidth spreading ratio of B/W=12 corresponds to
commercial FM broadcasting g and AM ((the ideal system
y
would have SNR o e SNR base
).
• SSB and baseband systems provide noise performance
id ti l tto th
identical the id
ideall ((which
hi h iis ttrivial).
i i l)
• DSB has a worse performance because of its additional
bandwidth requirements.
requirements
• FM systems only come close to the ideal system near
threshold.
225
Noise Performance of Analog
Communication Systems
226
Discussion
• Known analogue communication systems do not achieve
the ideal p
performance in ggeneral.
• Something must be missing with analogue
communications.
• Similarly, it can be shown that simple digital modulation
schemes cannot achieve capacity too.
• One must resort to digital communications and coding to
approach the capacity.
• The idea of tradeoff between bandwidth and SNR is useful
in spread-spectrum communications (CDMA, 3G mobile
communications) and ultra wideband
wideband.
227
EE2--4: Communication Systems
EE2
228
Outline
• Introduction
• Block codes
• References
– Haykin & Moher, Communication Systems, 5th ed., Chap. 10
– Lathi, Modern Digital and Analog Communication Systems,
3rd ed
ed., Chap
Chap. 16
229
Taxonomy of Coding
Error Error
Correction Detection
Coding Coding
= FEC - used
- no feedback in ARQ
channel as in TCP/IP
- feedback channel
- retransmissions
k = 11, n = 22, L = 2
231
Noise and Errors
• Noise can corrupt the information that we wish to transmit.
232
Channel Model
• We can measure the effect of noise in different ways.
The most common is to specify an error probability,
probability p.
p
Consider the case of a Binary Symmetric Channel
with noise.
1 1
1-p
p p
1-p
0 0
233
Simple Error Checks
• If the error probability is small and the information is fairly fault tolerant,
it is possible to use simple methods to detect errors.
• Repetition – Repeating each bit in the message
– If the two symbols in an adjacent pair are different, it is likely that an error
has occurred.
– However, this is not very efficient (efficiency is halved).
– Repetition provides a means for error checking, but not for error correction.
• P it bit – Use
Parity U off a ‘parity
‘ it bit’ att the
th end
d off the
th message
– A parity bit is a single bit that corresponds to the sum of the other
message bits (modulo 2).
– This allows any odd number of errors to be detected, but not even
numbers.
– As with repetition,
p , this technique
q only y allows error checking,
g, not error
correction.
– It is more efficient than simple repetition.
234
Block Codes
• An important class of codes that can detect and correct
some errors are block codes
• The first error-correcting block code was devised by
Hamming around the same time as Shannon was working
on the foundation of information theory
– Hamming codes are a particular class of linear block code
• Block codes
– Encode a series of symbols from the source, a ‘block’, into a
longer string: codeword or code block
– Errors can be detected as the received coded block will not be
one of the recognized, valid coded blocks
– Error correction: To “decode” and associate a corrupted block to a
valid coded block by its proximity (as measured by the “Hamming
distance”)
d s a ce )
235
Binary Fields and Vectors
• We need to discuss some of mathematics that will be needed.
• Fortunately, we have restricted things to binary sources, so the
mathematics is relatively simple.
• The binary alphabet A={0,1} is properly referred to as a Galois field
with two elements
elements, denoted GF(2).
– Addition (XOR)
0 + 0 = 0, 0 + 1 = 1 + 0 = 1, 1 + 1 = 0
– Multiplication (AND)
0ڄ1 = 0ڄ0 = 1ڄ0 = 0, 1ڄ1 = 1
– This is also referred to as Boolean arithmetic in Digital Electronics or
modulo-2 arithmetic.
• A message is built up from a number of binary fields, and forms a
bi
binary vector,
t rather
th ththan a llarger bi
binary number.
b
– Hence, 101 ≠ 5 101={1}{0}{1}
236
Example
• Answers
i
i. 00111
ii. 00010
iii
iii. 1100 0011 = 0000
237
Hamming Distance
• Hamming Weight
– The Hamming weight of a binary vector, a (written as wH(a)), is the
number of non
non-zero
zero elements that it contains
contains.
– Hence,
001110011 has a Hamming g weight
g of 5.
000000000 has a Hamming weight of 0.
• Hamming Distance
– The Hamming Distance between two binary vectors, a and b, is
written dH(a,b) , and is equal to the Hamming weight of their
((Boolean)) sum.
dH(a,b) = wH(a+b)
– Hence, 01110011 and 10001011 have a Hamming distance of
dH = wH (01110011+10001011)
= wH (11111000) = 5
238
Linear Block Codes
• A binary linear block code that takes block of k bits of
source data and encodes them using n bits, is referred to
as a (n, k) binary linear block code.
– The ratio between the number of source bits and the number of
bits used in the code
code, R=k/n,
k/n is referred to as the code rate.
• The most important feature of a linear block code
– Linearity: the Boolean sum of any codewords must be another
codeword.
– This means that the set of code words forms a vector space,
within
ithi which
hi h mathematical
th ti l operations
ti can b
be d
defined
fi d andd
performed.
239
Generator Matrix
• To construct a linear block code we define a matrix, the
generator matrix G, that converts blocks of source
symbols
b l into
i t llonger bl
blocks
k corresponding
di tto coded words.
d
• G is a k×n matrix (k rows, n columns), that takes a
source block u (a binary vector of length k),
k) to a code
word x (a binary vector of length n),
x=uڄG
u x
240
Systematic Codes
codeword
241
Example
Given the generator matrix for a (7,4) systematic code
1 0 0 0 0 1 1
0 1 0 0 1 0 1
G
0 0 1 0 1 1 0
0 0 0 1 1 1 1
242
Error Detection
• To determine the number of errors a particular code can detect and
correct, we look at the minimum Hamming distance between any two
code
d words.
d
• From linearity the zero vector must be a code word.
• If we define the minimum distance between any two code words to be
dmin = min{dH(a,b), a,bאC} = min{dH(0,a+b), a,bאC}
= min{wH(c), cאC, c≠0}
where C is the set of code words.
• The number of errors that can be detected is then (dmin–1), since dmin
errors can turn an input code word into a different but valid code word
word.
Less than dmin errors will turn an input code word into a vector that is
not a valid code word.
243
Error Correction
• The number t of errors that can be corrected is simply
the number of errors that can be detected divided by two
and rounded down to the nearest integer, since any
output vector with less than this number of errors will
‘nearer’ to the input code word.
dmin 2t
2 +1 dmin< 2t
244
Parity Check Matrix
• To decode a block coded vector, it is more complicated.
• If the generator matrix is of the form
G = [Ik ×k ⋮ Pk ×(n −k )]
To check for errors, we define a new matrix, the parity check matrix,
H. The p parity
y check matrix is a (n-k)×n matrix that is defined so that
– It will produce a zero vector when no error in the received code
vector x,
x·HT = 0 (14 1)
(14.1)
where HT is the transpose of H.
• To satisfy this condition
condition, it is sufficient to write the parity check matrix
in the form
H = [(Pk ×(n-k))T ⋮ I(n −k ) ×(n −k )]
• The minimum Hamming distance is equal to the smallest number of
columns of H that are linearly dependent.
– This follows from the condition ((14.1).
) See Haykin,
y , Chap.
p 10 for p
proof.
245
Syndrome
• If the received coded block y contains errors, then the
product of the received block with the transpose of the
parity check matrix will not be zero,
y ڄHT ≠ 0
• Writing y = x + e which is the sum of the original coded
block x and the error e, we find that
y ڄHT = e ڄHT
• This value is referred to as the syndrome, s = y ڄHT = e ڄ
HT.
• The syndrome is a function of the error only, and contains
the information required to isolate the position (or positions)
of the error (or errors).
246
Syndrome Decoding
• s is an (n-k) row vector, taking 2(n-k) -1 possible values (the
p
zero vector corresponding g to no error).)
• This means that it is necessary to calculate and store 2(n-k)
-1 syndromes as a look-up table to be able to pin-point the
positions of the errors exactly.
• Problem: this is impractical for large values of (n – k).
Received Syndrome
vector
t y s
Parity-check Syndrome
matrix H table
Error
E Decoded
vector e codword x
247
Summary
• A linear block code can be defined by a generator matrix
G and the associated pparity-check
y matrix H.
• Every linear block code is equivalent to a systematic code.
• The key parameter of a linear block code is the minimum
Hamming distance dmin:
– Up to dmin – 1 errors can be detected;
– Up to (dmin – 1)/2 errors can be corrected.
• Syndrome decoding: compute the syndrome and then find
th error pattern.
the tt
– Only practical for short codes.
248
EE2--4: Communication Systems
EE2
249
Outline
• Hamming codes
– A special type of cyclic codes that
correct a single error
• Cyclic codes
– Can correct more errors
– The most important
p class of block codes
– Implementation takes advantage
of polynomial multiplication/division
• References
– Haykin
ay & Moher,
o e , Communication
Co u cat o Systems,
Syste s, 5t
5th ed
ed.,, C
Chap.
ap 10
0
– Lathi, Modern Digital and Analog Communication Systems,
3rd ed., Chap. 16
250
Hamming Codes
• Hamming codes are a class of linear block codes that
can correct a singleg error. Theyy satisfy y the condition
r = n – k = log2(n+1) n = 2r – 1, k = 2r – 1 – r.
• From this expression, it is easy to see that the first few
Hamming codes correspond to
(n, k) = (7,4), (15,11), (31,26),…
• They are easy to construct and are simple to use.
– For all Hamming codes, dmin = 3.
– All (correctable) error vectors have unit Hamming weight, and the
syndrome associated with an error in the i’th column of the vector
is the i’th row of HT .
• Columns of H are binary representations of 1, …, n.
251
Syndrome Table
• For the (7,4) Hamming code, the parity check matrix is
0 1 1 1 1 0 0 s e
H 1 0 1 1 0 1 0 000 0000000
1 1 0 1 0 0 1 001 0000001
• G
Generator
t matrix
ti 010 0000010
1 0 0 0 0 1 1
0 100 0000100
1 0 0 1 0 1
G 111 0001000
0 0 1 0 1 1 0
110 0010000
0 0 0 1 1 1 1
101 0100000
• The corresponding syndrome table is:
011 1000000
s eH T
• When a coded
Wh d d vectort iis received,
i d ththe syndrome
d iis calculated
l l t d and
d any
single error identified, and corrected by exchanging the relevant bit
with the other binary value – However, problems can occur if there is
more than one error.
252
Example
• Consider the (7,4) Hamming code. If the code vector
1000011 is sent while 1000001 and 1001100 are received,,
decode the information bits.
• Answer:
– The first vector
s=(1000001)HT=(010)
e=(0000010)
x=(1000001)+(0000010)=(1000011)
u=(1000) correct (as there is one error)
253
Cyclic Codes
• Cyclic codes are a subclass of linear block codes offering larger
Hamming distances, thereby stronger error-correction capability.
• Whereas the principal property of the simple linear block code is
that the sum of any two code words is also a code word, the cyclic
codes have an additional p property:
p y a cyclic
y shift of any
y code word
is also a code word.
• A cyclic shift of a binary vector is defined by taking a vector of
length n,
a = [a0, a1, …, an-1]
g g the elements,
and rearranging
a = [an-1, a0, a1, …, an-2]
• A code is cyclic if:
(c0, c1, …, cn-1) אC ֜ (cn-1, c0, …, cn-2) אC
254
Generator Matrices
• A cyclic code is still a linear block code, so all of the
properties
p p p
previously y discussed hold for cyclic
y codes.
• They are constructed by defining a generator matrix, and
an associated parity check matrix and are decoded using
syndromes in exactly the same way as the other linear
block codes that we have discussed.
• A generator matrix is defined in the same way as before,
except that the rows are now cyclic shifts of one n-
dimensional basis vector
vector.
g0 g1 g n k 0 0
0 g0 g1 g n k 0
G
0 0 g0 g1 g n k
255
Encoding
• The cyclic property leads to a very useful property: they can be
represented simply (mathematically and in hardware) by polynomial
operations.
operations
• We start by looking at the code word x generated by a source block
u,
x = u ڄG = [x0, x1,…, xn-1]
• With the generator matrix in the cyclic (non-systematic) form given in
the previous slide
slide, the elements of the code words are
are,
x0 = u0g0
x1 = u0g1+ u1g0
…
xn-1 = uk-1gn-k
These elements take the general form,
k 1
xl ui gl i
i 0
256
Polynomial Representation
• Taking a binary vector x, it is possible to represent this as a
y
polynomial in z ((over the binary
y field))
x = [ x0 , x1 ,… , xn−1 ] → x (z) = x0 zn−1 + x1 zn−2 +…+ xn−1
• Representing the original code word and the n-dimensional basis
vector for the cyclic code as polynomials,
polynomials
u = [u0 , u1 ,… , uk −1 ] → u (z) = u0 zk −1 + u1 zk −2 +…+uk −1
g = [g0 , g1,… , gn−k ,0,…,0]
→ g (z) = g0 zn−k + g1zn−k−1 +…+ gn−k
• We notice that the product of these two polynomials is exactly the
same as the h polynomial
l i l representation
i off the
h corresponding
di code d
vector.
Nonsystematic
y Encoding: g x(z) = u(z) ڄg(z)
• This means that the problem of matrix-vector multiplication can be
reduced to a problem of polynomial multiplication.
257
Advantage
• The main advantage is that it simplifies the cyclic shift
operation and thereby simplifies the hardware
operation,
implementation of the code considerably.
• Multiplication of the code word by z shifts all of the
coefficients along by one, and replacing the term (x0zn)
byy a term ((x0z0)), g
gives a cyclic
y shift.
• A simpler way to achieve the same result is to multiply
the polynomial by z, divide by zn-1, and take the
remainder term, which can be written as
(z x(z)) mod(zn −1)
258
Parity Check Polynomials
• One property important to cyclic codes is the ability to factorise
polynomials. Given a polynomial of the form a(z) = zn−1 = zn+1 (n > 1),
it is always possible to find two polynomials
polynomials, such that
that,
zn+1 = g(z) h(z)
• Taking g g(z) to be a g
generator p polynomial,
y , this condition is sufficient for
a resultant code to be an (n,k) cyclic code, where:
– g(z) is a polynomial of degree n−k = r.
– h(z) is
i a polynomial
l i l off d
degree k.
• The other factor, the polynomial h(z), turns out to be a parity check
polynomial
p y , playing
p y g the same role as the p parity
y check matrix.
• If we have a valid code word, x(z), the product of the code word with
h(z) is zero (modulo zn+1), and if we have a code word contained an
error,
error
(y(z)h(z))mod(zn+1) = [(x(z)+e(z))h(z)]mod(zn+1)
= [e(z)h(z)]
)]mod((zn+1))
which is called the syndrome polynomial, s(z).
259
Encoding/Decoding
• Encoding: x(z)=u(z)g(z);
• Parity check polynomial:
h(z) =[zn+1] /g(z)
• Decoding: (y(z) is the received vector)
– Calculate the syndrome polynomial:
s(z)=(y(z)h(z))mod(z
s(z) (y(z)h(z))mod(zn+1)
1)
– Look up the syndrome table to get e(z) from s(z);
– x(z)=y(z)+e(z);
– u(z)=x(z)/g(z);
• Thi
This iis th
the en/decoding
/d di procedure
d ffor non-systematic
t ti
codes. A modified version works for systematic codes
(non-examinable)
(non-examinable).
260
Hardware Implementation
• Binary polynomial multiplication: multiply-by-g(z)
a(z)
g0 g1 g2 g3
b(z)
g(z)=
) g0+g1z+g2z2+g3z3, g0=g3=1;
1
• Binary polynomial division: divide-by-g(z)
g0 g1 g2
a(z) b(z)
g(z)=g0+g1z+g2z2+g3z3, g0=g3=1;
261
Examples of Cyclic Codes
• Hamming code (used in computer memory)
• Cyclic redundancy check (CRC) code (used in Ethernet, ARQ etc.)
• B
Bose-Chaudhuri-Hocquenghem
Ch dh i H h (BCH) code
d
• Reed-Solomon (RS) code (widely used in CD, DVD, hard disks,
wireless, satellites etc.))
266
EE2--4: Communication Systems
EE2
Revision Lecture
267
Lectures
Introduction and background Digital communications
1. Introduction 8. Digital representation of signals
2. Probability and random 9. Baseband digital transmission
processes 10. Digital modulation
3 Noise
3. 11 Noncoherent
11. N h td
demodulation
d l ti
Effects of noise on analog Information theory
communications 12 Entropy and source coding
12.
4. Noise performance of DSB 13. Channel capacity
5. Noise performance of SSB and 14. Block codes
AM
15. Cyclic codes
6. Noise performance of FM
p
7. Pre/de-emphasis for FM and
comparison of analog systems
268
The Exam
• The exam paper contains 3 questions. All questions are
p y
compulsory.
• For example, the questions may look like
– Question 1 (40 marks): Basic knowledge of communication
systems, elements of information theory and/or coding, mostly
bookwork
– Question 2 (30 marks): Analog or digital communications
communications,
bookwork, new example/application/theory
– Question 3 (30 marks): Digital or analog communications or
i f
information
i theory/coding,
h / di b
bookwork,
k k new
example/application/theory
• Sample questions:
– Past papers
– Problems in classes
269
Introduction, Probability and Random
Processes
• Primary resources in communications: power, bandwidth,
cost
• Objectives of system design: reliability and efficiency
• Performance measures: SNR or bit error probability
• Probability distribution: Uniform distribution, Gaussian
distribution, Rayleigh
y g distribution, Ricean distribution
• Random process: stationary random process, auto-
correlation and power spectral density, Wiener-Khinchine
relation
S X ( f ) RX ( )e j 2f d
270
Noise
• Why is noise important in communications? How does
noise affect the p
performance? What types
yp of noise exist?
• White noise: PSD is constant over an infinite bandwidth.
• Gaussian noise: PDF is Gaussian.
• Additive white Gaussian noise
• Bandlimited noise, bandpass representation, baseband
noise nc(t) and ns(t), power spectral density
S ( f f c ) S N ( f f c ), | f | B
Sc ( f ) S s ( f ) N
0, otherwise
271
Noise performance of AM
PS
• Signal-to-noise ratio SNR
PN
PT
• Baseband communication model SNRbaseband
N 0W
272
Noise performance of FM
• FM modulator and demodulator
• Method to deal with noise in FM: linear argument at high
SNR
• Derivation of the output SNR, threshold effect
273
Digital communications
• PCM: sample, quantize, and encode
3P
• Quantization noise and SNR SNRo (dB) 6n 10 log10 2 (dB)
mp
274
Noise performance of bandpass digital
communications
• Modulation formats: ASK, FSK, PSK
• Q-function
Q function Q(x): computation by using the graph or
approximation
1 x2 / 2
Q( x) e ,x0
2 x
275
Performance of digital modulation
Scheme Bit Error Rate
Noncoherent
ASK, FSK
Coherent ASK Q(A/2)
Coherent
Coherent FSK Q(A/2) ASK, FSK
277
Channel coding theorem
• If the transmission rate R ≤ C, then there exists a
coding g scheme such that the output
p of the source can
be transmitted over a noisy channel with an arbitrarily
small probability of error. Conversely, it is not
possible to transmit messages without error if R > C.
• Shannon formula
P
C B log 2 1 SNR B log 2 1
N 0 B
278
Channel coding
• Block vs. convolutional codes
• Binaryy fields and vector space,
p , Hamming g distance/weight
g
• A linear block code can be defined by a generator matrix
G and the associated parity-check matrix H.
• Every linear block code is equivalent to a systematic code.
• The key parameter of a linear block code is the minimum
Hamming distance dmin:
– Up to dmin – 1 errors can be detected;
p to (d
– Up ( min – 1)/2
) errors can be corrected.
• Syndrome decoding: compute the syndrome and then find
the error pattern.
• Hamming codes
• Cyclic codes: (polynomial representation is not
examinable)
279
Appendix:
More Background on Probability
Probability
• Sample Space
– S : Set of all random experiment outcomes
– S = { s : s iis an outcome
t }
• Examples
– For tossing a coin, S = { H, T }
– Roll a die, S = { 1, 2, …, 6 }
• Event E S
– Roll a die twice: S = { (H,H), (H,T), (T,H), (T,T) }
– Event E = { (H,H), (T,T) }
– A collection of events, F. Obviously, F S
• A probability measure on F is a function P : F → [0,1]
[0 1] satisfying the
probability axioms:
– P(S) = 1
– P(F) ≥ 0
– For events A, B belonging to F, if A ∩ B=0, P(A U B) = P(A) + P(B)
281
Random Variables
• A random variable X(s) is a variable whose value depends on
the outcome s of a random experiment with the defined
probabilityy measure.
• For convenience, we simply denote the random variable by X.
• Consider a gamble in Macau (an example of discrete random
variables):
5 s W
Loss Draw Win X (s) 0 sD S W D L
P=3/8 P=1/4 P=3/8 5 sL
P ( X ( s ) 5) P ( s : s W ) P (W ) 3
8
P ( X ( s ) 0) P ( s : s D ) P ( D ) 1
4
X(s) P ( X ( s ) 5) P ( s : s L ) P ( L ) 3
-$5
$5 0 $5 8
FX ( )
f X ( y ) dyy 1
b
P ( a X b ) FX ( b ) FX ( a )
a
f X ( y ) dy
• If x is sufficiently small,
x x
P ( x X x x ) f X ( y ) dyy f X ( x ) x
x
f X ( y) Area
f X ( x)
x
y
x
• Expectation operator (expected value or average):
E[ X ] X
y f X ( y ) dy
• Variance:
X2 E[( X X )2 ] ( y X )2 f X ( y)dy E[ X 2 ] X2
284
Moments of a Random Variable
Average (mean) of X:
1 E [ X ] or m X
• Central moments,, centered on the mean
285
Examples of Discrete Distributions
• Let p + q = 1
• Bernoulli distribution
P ( X 1) p P ( X 0) q
• Bi
Binomial
i l distribution
di t ib ti
n
P (Y k ) p k q n k k 0 ,1, 2 ,..., n
k
Y represents the total number of successes, in an independent
p
trial of n Bernoulli experiments
• Exercise: Verify E [ X ] p , X2 pq
E [Y ] np , Y2 npq
286
Examples of Continuous Distributions:
Exponential Distribution
f X ( x)
x
0
f X ( x ) e x for x 0
• E
Exercise:
i V if
Verify E(X )
1
X2 1
2
287
Normal (Gaussian) Distribution
f X ( x)
x
0 m
( x m )2
f X ( x) 1 e 2 2 for x
2
( y m )2
x
2 2
FX ( x ) 1
2 e dy
• Exercise: Verify E ( X ) m
2 2
X
288
Rayleigh and Rice Distributions
289
Conditional Probability
Random variables : X ,Y
x y
FXY ( x , y ) f
XY ( u , v ) dudv
f XY ( u , v ) dudv
FY ( y | x X x x ) P ( YP (xy, xXXx xx ) x ) xx x
f X ( u ) du
x
y
f XY ( x ,v ) xdv
f X ( x ) x
y
f XY ( x ,v ) dv
As x 0, FY ( y | x )
fX (x)
fY ( y | x ) dFY ( y | x )
dy fY ( y | x ) f XY ( x , y )
fX (x)
292
Joint Distribution Function of Several Random Variables
Covariance of X and Y:
cov( X , Y ) E [( X m X )( Y mY )]
Correlation Coefficient:
E [( X m X )(Y mY )]
XY cov(
X ,Y )
XY
X Y
Property 1: 1 XY 1
Property 2: X and Y are linearly related ( i .e., Y aX b )
if and only if XY 1
294
Joint Gaussian distribution
f
y2
fY ( y ) XY ( x , y ) dx 1
2
exp{ 2 2
}
x
295
Independent vs. Uncorrelated
• Independent implies Uncorrelated
• Uncorrelated does not imply Independence
• For jointly Gaussian random variables, Uncorrelated
implies Independent (this is the only exceptional case!)
296
An example of uncorrelated but dependent random variables
n
• Define S n X i
i 1
e
y2 / 2
lim P ( R n x ) 1
2
dy
n
298