Quantization Concept

UMCP ENEE631 Slides (created by M.Wu 2001/2004)

L-level Quantization

tmin

tmax

Minimize errors for this lossy process What L values to use? Map what range of continuous values to each of L values?
tmin tmax
(tmaxtmax)/2L

MaxMax-Lloyd Quantizer

Uniform partition
over a dynamic range of 2V Best solution?
X X

Maximum errors = ( tmax - tmin ) / 2L = 2V / 2L

tk tk+1

quantization error

Consider minimizing maximum absolute error (min-max) vs. MSE what if the value between [a, b] is more likely than other intervals?

Consider the Probability Distribution

UMCP ENEE631 Slides (created by M.Wu 2001)

Derivation of MMSE (Max(Max-Lloyd) Quantizer

z

p.d.f pu(x) t1 z z tL+1

MSE of L-level quantizer Optimal (MMSE) quantizer necessary conditions

r1

rL

Allocate more reconstruct. values in more probable ranges Minimize error in a probability sense
MMSE (minimum mean square error)
X X

assign high penalty to large error and to likely occurring values squared error gives convenience in math.: differential, etc.

An optimization problem
What {tk} and {rk } to use? Necessary conditions: by setting partial differentials to zero

MMSE Quantizer (Lloyd(Lloyd-Max)

UMCP ENEE631 Slides (created by M.Wu 2001/2004)

Observations on the MMSE Quanitizer

Reconstruction and decision levels need to satisfy

Solve iteratively
Choose initial values of {tk}(0) , compute {rk}(0) Compute new values {tk}(1), and {rk}(1)

For large number of quantization levels

Approx. constant pdf within t[tk, tk+1), i.e. p(t) = p(tk) for tk=(tk+tk+1)/2
Reference: S.P. Lloyd: Least Squares Quantization in PCM, IEEE Trans. Info. Theory, vol.IT-28, March 1982, pp.129-137

Example: Uniform Density

UMCP ENEE631 Slides (created by M.Wu 2001/2004)

MMSE Quantizer for Uniform Distribution

z

Uniform quantizer
Is optimal for uniformly distributed r.v. in MMSE sense MSE = 2 / 12 with = 2V / L
t1 tL+1

SNR of uniform quantizer

SNR = 6n dB
X

with

L = 2n

p.d.f. of uniform distribution

1/(2V) t1 2V tL+1

1 bit is worth 6 dB.

Uniform quantizer is the optimal mean square quantizer for uniform density.

z
UMCP ENEE739M Slides (created by M.Wu 2002)

Note: Quantization is a Lossy Step in Source Coding

Quantization is a lossy way to data compression

Lloyd-Max quantizer minimizes MSE distortion with a number of output bits n (i.e. a give L=2n)