Digital Video Compression Fundamentals and Standards

Digital Video Compression
Fundamentals and Standards

Speaker: Wei-Yi Wei
Advisor: Prof. Jian-Jung Ding
Digital Image and Signal Processing
Lab
Outline
Introduction
Video Compression Standards
Simulation Reference Software
Future Work and Conclusions
2008/12/26
Digital Video Compression Fundamentals an
Outline
Introduction
2008/12/26
Introduction (1/2)
Why video compression technique is
important ?
One movie video without compression
720 x 480 pixels per frame
30 frames per second
Total 90 minutes
Full color
The total quantity of data = 167.96 G Bytes !!
2008/12/26
Introduction (2/2)
What is the difference between video
compression and image compression?
Temporal Redundancy
Coding method to remove redundancy

Intraframe Coding
Remove spatial redundancy
Interframe Coding
Remove temporal redundancy
2008/12/26
The most intuitive method to remove

Spatiotemporal redundancy
3-Dimensional DCT
Remove spatiotemporal correlation
Good for low motion video
Bad for high motion video
N 1 N 1 N 1
8
(2 x 1)u
(2 y 1)
v
F ( x, y, t ) 3 C (u )C (v)C ( w) ( x, y, t ) cos
cos
N
2 N
2 N
t 0 x 0 y 0
cos
(2t 1) w
2 N
for u 0,..., N 1 ,v 0,..., N 1 and w 0,..., N 1

1/ 2 for k 0
1 otherwise
where N 8 and C ( k )
2008/12/26
The most popular method to remove

temporal redundancy
The Block-Matching Algorithm
2008/12/26
Matching Function
The dissimilarity D( s, t ) between two blocks n and n1
p
D( s, t ) M [ n( x, y ), n 1( x Vx, y Vy )]
Vy 1 Vx , 1
The matching criteria

Mean square error (MSE)
M (u , v) (u - v) 2
High precision is needed
Mean absolute difference (MAD)
M (u, v) | u - v |
Low precision is enough

2008/12/26
The Exhaustive Block-Matching

Algorithm
Reference Frame
Current Frame
Motion
Vector
|A|=12
12 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
11 23 34 44 55 66 77 88
11 23 34 44 55 66 77 88
Search
11 23 34 44 55 66 77 88
Range
11 23 34 44 55 66 77 88
2008/12/26
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
1
1
1
1
0
0
0
0
0
0
0
0
1
1
1
1
0
0
0
0
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Fast Block-Matching Algorithms

EBMA needs Intensive computation
720 480
16
16 30 170 G instructions/sec
128 128
16 16
Fast Algorithm is needed
Find the possible local optimal

Reference Frame
2008/12/26
Current Frame
10
Fast Block-Matching Algorithms

The characteristics of fast algorithm
Not accurate as EBMA
Save large computation
Two famous fast algorithm

2-D logarithm Search Method
Three Steps Search Method
2008/12/26
11
2-D logarithm Search Method
2
4
2008/12/26
12
Three Step Search Method

2
1
2
2
2008/12/26
2
3 3 3
1 3 2 3
3 3 3
2
2
13
Multiresolution Motion Estimation (1/3)

The number of levels is L
l-th level images of the target frames
t ,l (x), x l , t 1, 2,...
where l is set of pixels at level L
At the l-th level, the MV is d( x)
%l (x) U (d (x))
d
l 1
At the l-th level, the estimated MV is
Determine update ql (x) such that error is minimized
error
%l (x) q (x)) (x)) | p

|
(
x
d
2,l
l
1,l
xl
%
The new motion vector is dl (x) d l (x) ql (x)
2008/12/26
14

Variable block size method
2008/12/26
15
2008/12/26
16
Outline
Introduction
2008/12/26
17
The Development of Video

Compression Standards
2008/12/26
18
The MPEG-1 Standard

Group of Pictures
Motion Estimation
Motion Compensation
Differential Coding
DCT
Quantization
Entropy Coding
2008/12/26
19
Group of Pictures (1/2)

I-frame (Intracoded Frame)
Coded in one frame such as DCT.
This type of frame do not need previous frame
P-frame (Predictive Frame)

One directional motion prediction from a previous frame
The reference can be either I-frame or P-frame
Generally referred to as inter-frame
B-frame (Bi-directional predictive frame)

Bi-directional motion prediction from a previous or future frame
The reference can be either I-frame or P-frame
Generally referred to as inter-frame
2008/12/26
20
Group of Pictures (2/2)

The distance between two nearest P-frame or P-frame and I-frame
denoted by M
The distance between two nearest I-frames
denoted by N
Forward Motion
Compensation
GOP
I
2008/12/26
N=9
M=3
Bidirectional Motion
Compensation
21
The MPEG-1 Encoder (1/4)

Intra-frame
DCT
Residue
Entropy
Coding
Inter-frame
Q-1
Motion
Compensation
Motion Vector
IDCT
Motion
Estimation
Frame
Memory
2008/12/26
22

Differential Coding
^
D(t ) (t ) (t )
(t ) is the input image
^
(t ) is the predictive image
DCT
N 1 N 1
2
(2 x 1)u
(2 y 1)
v
F (u , v) C (u )C (v) f ( x, y ) cos
cos
N
2 N
2 N
x 0 y 0
for u 0,..., N 1 and v 0,..., N 1
1/ 2 for k 0
where N 8 and C (k )
1 otherwise
2008/12/26
23

Quantization
Intra quantization matrix
Inter quantization matrix
Qintra
16
19
22
22
26
26
27
16 19 22 26 27 29 34
16 17 18 19
16 22 24 27 29 34 37
17 18 19 20
18 19 20 21
22 26 27 29 34 34 38
22 26 27 29 34 37 40
19 20 21 22
Qintra
20 21 22 23
26 27 29 32 35 40 48
27 29 32 35 40 48 58
21 22 23 24
22 23 24 26
27 29 34 38 46 56 69
29 35 38 46 56 69 83
23 24 35 27
2008/12/26
20 21 22 23
21 22 23 24
22 23 24 25
23 24 26 27
25 26 27 28
26 27 28 30
27 28 30 31
28 30 31 33
24

Motion Compensation
Exploit motion vector and the previous reconstructed frame to generate
the predictive frame
( x, y ) ( x v ( p, q), y v ( p, q)), ( x, y ) MB( p, q )
( x, y )
n
is the compensated image
n ( x, y ) is the previous image
v (vx ( p, q), v y ( p, q)) is the motion vector

Reference frame
2008/12/26
Target frame
25
The MPEG-2 Standard

Field/Frame DCT Coding
Field/Frame Prediction Mode Selection
Alternative Scan Order
Various Picture Sampling Formats
User Defined Quantization Matrix
2008/12/26
26
Progressive Scan and Interlaced

Scan
Progressive Scan
2008/12/26
Interlaced Scan
27
Field/Frame DCT Coding

The field type DCT
Fast motion video
The frame type DCT

Slow motion video
Field DCT Coding
2008/12/26
Luminance MB
Frame DCT Coding
28
Alternative Scan Order

Zigzag scan order
Frame DCT
Alternative scan order

Field DCT
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
2008/12/26
Zig-zag
1 1 1
0 0 0
1 1 1
0 0 0
1 1 1
0 0 0
1 1 1
0 0 0
scan
1 1
0 0
1 1
0 0
1 1
0 0
1 1
0 0
2D DCT
4
0.72
0
0.85
0
1.27
0
3.62
Alternate
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
scan
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0
0
0
0
0
0
0
0
29

Quantization
User can change the quantization if necessary
Intra quantization matrix
Inter quantization matrix
Qintra
16 16 16 16
8 16 19 22 26 27 29 34
16
16
22
24
27
29
34
37
16 16 16 16
16 16 16 16
19 22 26 27 29 34 34 38
22
22
26
27
29
34
37
40
Q 16 16 16 16
22 26 27 29 32 35 40 48 inter 16 16 16 16
26
27
29
32
35
40
48
58
16 16 16 16
26 27 29 34 38 46 56 69
16 16 16 16
27 29 35 38 46 56 69 83
16 16 16 16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
Various picture sampling formats

4:4:4
4:2:2
4:2:0
2008/12/26
30

SNR Enhanced Layer
Base Layer
Bits Enhance
Basic quality requirement

For SDTV
Enhanced Layer
High quality service
For HDTV
Entropy
Coding
Q-1
DCT
Entropy
Coding
Q
Q-1
Q-1
Motion
Compensation
Bits Base
Base
Layer
IDCT
Motion
Estimation
Frame
Memory
2008/12/26
31
H.264/AVC
Variable Block Size
Multiple Reference Frames
Integer Transform
Intra Prediction
In-loop Deblocking Filtering
1/4-pel Resolution Motion Estimation
CAVLC
2008/12/26
32
Variable Block Size

The fixed block size may not be suitable for all
motion objects
Improve the flexibility of comparison
Reduce the error of comparison
7 types of blocks for selection

16 x 16
16 x 8
0
8x8
1
8x4
0
2008/12/26
0
1
8 x 16
4x8
8x8
4x4
33
Multiple Reference Frames

The neighboring frames are not the most similar in some cases
The B-frame can be reference frame
B-frame is close to the target frame in many situations
2008/12/26
34
Integer Transform for Reducing The

Spatial Redundancy (1/2)
The transform matrix C
1 1 1 1
2
1
C
1 1 1 1
1 2 2 1
44 Block Size
Separable Integer Transform
The transform coefficients are CXCT
2008/12/26
35
Integer Transform for Reducing The

Spatial Redundancy (2/2)
0.5
0.6325 0.3162 0.3162 0.6325
T
Y AXA
0.5
0.5
0.5
0.5
0.3162
0.6325
0.6325
0.3162
0.5
0.5
1 1 1 1

2 1 1 2
=
1 1 1 1

1 2 2 1
x00
x
10
x20
x01
x11
x21
x02
x12
x22
x30
x31
x32
x00
x01
x02
x10
x20
x30
x11
x21
x31
x12
x22
x32
1 1 1 1

2 1 1 2
=
1 1 1 1

1 2 2 1
C
2008/12/26
x00
x
10
x01
x11
x02
x12
x20
x21
x22
x30
x31
x32
0.5
x03
x13
x23
x33
x03
x13
x23
x33
0.5
0.5
0.3162
0.3162 0.5 0.6325
0.5 0.3162 0.5 0.6325
0.5
0.6325
0.5
0.3162
0.6325
0.5
0.3162
0.5
0.5
0.5

0.5

0.5 0.3162 0.5 0.6325

0.3162 0.3162 0.3162 0.3162
0.5
0.5 0.3162 0.5 0.6325
0.5
0.5
0.5

0.5 0.6325 0.5 0.3162

0.3162 0.3162 0.3162 0.3162
1 2 1 1
0.25 0.1581 0.25 0.1581
1 1 1 2 0.1581
0.1
0.1581
0.1
1 1 1 2
0.25 0.1581 0.25 0.1581
1 2 1 1
0.1
0.1581
0.1
0.1581
0.5
x03
x13
x23
x33
0.6325
CT
0.5
36
Intra Prediction
Predict the similarity between the neighboring pixels in one
frame in advance, and exploit differential coding transform
coding to remove the redundancy.
Intra
Prediction
Vertical
Transform/
Quantization
DC
+ + + +
Horizontal
+
+
+
+
Vertical right
2008/12/26
Entropy
Coding
Diagonal down-left
Diagonal right
Mean
Horizontal down
Vertical left
Horizontal up
37
Remove Perceptual Redundancy

In-loop deblocking filtering
Remove blocking artifact
Result from block based motion compensation
Result from block based transform coding
q0 q1 q2 q3
QP
p3 p2 p1 p0
2008/12/26
38
1/4-pel Resolution Motion

Estimation
Integer search positions
Best integer match
Half-pel search positions
Best half-pel match
Quarter-pel search positions
Best quarter-pel match
2008/12/26
39
The H.264/AVC Encoder

Control Data
Coder
Controller
Transform/
Quantization
Motion
Compensation
Intra-frame
Prediction
Residue
Inverse
Transform/
De-Quantization
Entropy
Coding
Motion Vector
De-blocking
Filter
Motion
Estimation
2008/12/26
40
Outline
Introduction
2008/12/26
41
H.264 Reference Software

JM http://iphome.hhi.de/suehring/tml/
Current software version: JM 15
Benchmark
2008/12/26
42
Reference Software Demo
2008/12/26
43
Outline
Introduction
2008/12/26
44
Future Work
Fast Mode Decision Algorithm
Interpolation Filter Design
Deblocking Filter Design
DCT-Based Motion Estimation
Implementation Based on TI DSP
2008/12/26
45
Fast Mode Decision Algorithm

The computational cost of H.264 is large
Variable block-size ME
Multiple reference frames
Fast mode decision is needed for reducing the computation

time
2008/12/26
46
Interpolation Filter Design

In order to estimate and compensate the fractional-pel
displacements
Adaptive Interpolation filter for replacing the fixed coefficient
filter
2008/12/26
47
Deblocking Filter Design

Block based ME and Transform result in the annoying
blocking artifact
Reduce the blocking artifact can increase the quality of
compressed video
2008/12/26
48
DCT Based Motion Estimation

Robust even in noisy environment
Complexity comparison
DCT Based ME
O(M2)
Block Based ME
O(N2M2)
Concept
Pseudo Phase
j
F
{
x
(
t
)}
e
F {x(t )}
Similar to
2008/12/26
49
Implementation Based on TI DSP

TMS320C6416
TI DM642
2008/12/26
50
Conclusions
Thank You
Q&A
2008/12/26
51
References (1/2)
[1] Yun Q.Shi and Huifang Sun, Image and Video Compression for Multimedia
Engineering: Fundamentals, Algorithms, and Standards, CRC press, 2000.
[2] Yao Wand, Jorn Ostermann and Ya-Qin Zhang, Video Processing and
Communications, Prentice Hall, 2007.
[3] Richardson, Lain E. G., Video Codec Design: Developing Image and Video
Compression Systems, John Wiley & Sons Inc, 2002.
[4] Barry G, Haskell, Atul Puri and Arun N. Netravali, Digital Video : An
Introduction to MPEG-2, Boston : Kluwer Academic, 1999.
[5] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, Overview of the
H.264/AVC video coding standard, IEEE Trans. on Circuits and systems for
video Technology, vol. 13, no. 7, pp. 560-576, July 2003.
[6] G. Sullivan and T. Wiegand, Video Compression - From Concepts to the
H.264/AVC Standard, Proceedings of the IEEE, Special Issue on Advances in
Video Coding and Delivery, December 2004.
[7]
2004.
2008/12/26
52
References (2/2)
[8] Thomas Wedi, Adaptive Interpolation Filters and High-Resolution
Displacements For Video Coding, IEEE Trans. on Circuits and Systems For
Video Technology, vol.
[9] Dong-Hwan Kim, Hwa-Yong Oh, Oguzhan Urhan, Sarp Ertrk and Tae-Gyu
Chang, Optimal Post-Process/In-Loop Filtering for Improved Video
Compression Performance, IEEE Trans. on Consumer Electronics, vol. 53, no.
4, Nov. 2007.
[10] Shu-Fa Lin, Meng-Ting Lu, and Homer Chen, Fast Multi-Frame Motion
Estimation for H.264 and Its Applications to Complexity-Aware Streaming,
IEEE International Symposium on Circuits and Systems, 2005.
[11] Kai-Ting Cheng and Hsueh-Ming Hang, Acceleration and Implementation
of H.264 Encoder and Scalable Extension of H.264 Decoder on TI DSP
Platform, master thesis, June 2007
2008/12/26
53

Digital Video Compression Fundamentals and Standards

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Digital Video Compression Fundamentals and Standards

Hochgeladen von

Copyright:

Verfügbare Formate

Digital Video Compression

Fundamentals and Standards

Digital Video Compression Fundamentals an

Digital Video Compression Fundamentals an

Digital Video Compression Fundamentals an

Coding method to remove redundancy

Digital Video Compression Fundamentals an

The most intuitive method to remove

for u 0,..., N 1 ,v 0,..., N 1 and w 0,..., N 1

Digital Video Compression Fundamentals an

The most popular method to remove

Digital Video Compression Fundamentals an

The matching criteria

High precision is needed

Mean absolute difference (MAD)

Low precision is enough

Digital Video Compression Fundamentals an

The Exhaustive Block-Matching

Digital Video Compression Fundamentals an

Fast Block-Matching Algorithms

Fast Algorithm is needed

Find the possible local optimal

Digital Video Compression Fundamentals an

Fast Block-Matching Algorithms

Two famous fast algorithm

Digital Video Compression Fundamentals an

2-D logarithm Search Method

Digital Video Compression Fundamentals an

Three Step Search Method

Digital Video Compression Fundamentals an

Multiresolution Motion Estimation (1/3)

%l (x) q (x)) (x)) | p

Digital Video Compression Fundamentals an

Multiresolution Motion Estimation (2/3)

Digital Video Compression Fundamentals an

Multiresolution Motion Estimation (3/3)

Digital Video Compression Fundamentals an

Digital Video Compression Fundamentals an

The Development of Video

Digital Video Compression Fundamentals an

The MPEG-1 Standard

Digital Video Compression Fundamentals an

Group of Pictures (1/2)

P-frame (Predictive Frame)

B-frame (Bi-directional predictive frame)

Digital Video Compression Fundamentals an

Group of Pictures (2/2)

The MPEG-1 Encoder (1/4)

Digital Video Compression Fundamentals an

The MPEG-1 Encoder (2/4)

(t ) is the predictive image

Digital Video Compression Fundamentals an

The MPEG-1 Encoder (3/4)

Digital Video Compression Fundamentals an

The MPEG-1 Encoder (4/4)

n ( x, y ) is the previous image

v (vx ( p, q), v y ( p, q)) is the motion vector

Digital Video Compression Fundamentals an

The MPEG-2 Standard

Digital Video Compression Fundamentals an

Progressive Scan and Interlaced

Digital Video Compression Fundamentals an