Chair for

Chip Design for

Embedded Computing

Parallelism in High Efficiency Video Coding(HEVC)

Seminarvortrag

J
orn Hameyer
July 5, 2013

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Outline

Why is parallelism important in todays video encoding?

General problems (challenges) to parallelize video coding
Specific problems (challenges) to parallelize video coding
Brief introduction to HEVC
HEVC parallelism approaches
Parallel Context-Adaptive Arithmetic Coding(CABAC) in HEVC
Conclusions

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 2

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Why is parallelism important in todays video encoding?

Before the Multicore Processor Era special hardware-processors

needed
The latest Single Core Processor is able to decode a 1080p
H.264/AVC video in real time
100% load
High power consumption

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 3

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Why is parallelism important in todays video encoding?

Before the Multicore Processor Era special hardware-processors

needed
The latest Single Core Processor is able to decode a 1080p
H.264/AVC video in real time
100% load
High power consumption

High Definition(HD) currently is a standard for consumer

devices, also UHD(4k and 8k) are upcoming
Multitasking
Good performance gains; future proof

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 3

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

General problems (challenges) to parallelize video coding?

Data dependency
Load of the cores have to be maintained leading to overhead
Overhead
Heavy increase of computational complexity
Video coding processes are naturally sequential

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 4

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Specific problems (challenges) to parallelize video coding

Data-based parallelism
Frame Level parallelism
Slice Level parallelism
Macroblock Level parallelism

Task-based parallelism

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 5

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Problems in Frame Level parallelism

Was already in H.264/AVC where it offers low to no parallelism

Independent frame sequence needed for full frame level parallelism

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 6

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Problems in Frame Level parallelism

Was already in H.264/AVC where it offers low to no parallelism

Independent frame sequence needed for full frame level parallelism
However, frames are dependent on each other

[1]

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 6

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Problems in Slice Level Parallelism

[2]

[3]

Introduced already in H.264/AVC due to their independency

High overhead
Either packetization or parallelism
Bad implementations in most codecs
Increases the computational complexity
Deblocking filter still applies across boundaries
J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 7

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Problems in Macroblock Parallelism

A macroblock is dependent on
its adjacent macroblocks

[4]

Motion vectors for Inter-frame

prediction
Pixels for Intra-frame
prediction
Pixels for Deblocking filter
Entropy coding needs the
previously decoded macroblock

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 8

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Problems in Task-based parallelism

Load imbalance due to heuristics in the motion estimation and

non-preditable encoding time
Data dependency since task-based parallization borrows methods
from other parallization strategies

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 9

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Brief introduction to the High Efficiency Video Coding Standard

Most recent joint video project of the ITU-T VCEG and ISO/IEC
MPEG standardization organizations Joint Collaborative Team
on Video Coding (JCT-VC)
Has been finalized in January 2013 (First Version)
Successor of H.264/AVC, achieves a compression gain of 50% over
H.264/AVC
Can support 8k UHD and resolutions up to 8192x4320

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 10

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

HEVC decoder overview

[5]
Main Structure very similar to conventional video codecs
Many differences in terms of flexibility: significant gain

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 11

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Wavefront parallel processing (WPP)

A bitstream is divided into multiple rows of macroblocks

Decoding of a row can start when the dependency is met

[6]

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 12

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Wavefront parallel processing (WPP)

A bitstream is divided into multiple rows of macroblocks

Decoding of a row can start when the dependency is met

[6]
Cannot be used in conjunction with Tiles
J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 12

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Tiles

[7]
Reduced line buffer requirements for motion estimation
Slices can contain tiles and vice versa
Explicit division of frames
Multiple tiles use same header
J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 13

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Tiles

[7]
Reduced line buffer requirements for motion estimation
Slices can contain tiles and vice versa
Explicit division of frames
Multiple tiles use same header
Cannot be used in conjunction with WPP
J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 13

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Performance comparison of WPP and Tiles

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 14

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Performance comparison of WPP and Tiles

[8]
J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 14

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

CABAC overview

CABAC is a form of entropy coding

Entropy coding is a lossless compression used at the last stage of
video encoding
Arithmetic coding is a type of entropy coding
Symbols(syntax elements) are mapped to a number of bits
CABAC does this in three ways:
Binarization
Context modeling
Arithmetic coding

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 15

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

CABAC overview

CABAC is a form of entropy coding

Entropy coding is a lossless compression used at the last stage of
video encoding
Arithmetic coding is a type of entropy coding
Symbols(syntax elements) are mapped to a number of bits
CABAC does this in three ways:
Binarization
Context modeling
Arithmetic coding

Again: huge data dependencies conflicts with parallel processing

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 15

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

CABAC in H.264/AVC

[9]
CABAC was already implemented in H.264/AVC
Some serious bottlenecks
Because of data dependency between the bins it is very difficult to
parallalize them
J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 16

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

CABAC in HEVC

Reduced number of Context Coded Bins, instead Bypass Coded

Bins
Grouping of bypass coded bins
Reordering of context coded bins
Avoiding of redundant bits
Decouple CABAC from all other video coding processes

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 17

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Zig-Zag Scan vs Diagonal Scan

[10]
Zig-Zig Scan implemented in H.264/AVC has stronger data
dependency, but less computational complexity
Diagonal Scan implemented in HEVC has almost no data
dependency, but more computational complexity

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 18

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Zig-Zag Scan vs Diagonal Scan

[10]
Zig-Zig Scan implemented in H.264/AVC has stronger data
dependency, but less computational complexity
Diagonal Scan implemented in HEVC has almost no data
dependency, but more computational complexity
After all, the little increase in computational complexity is negligible
J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 18

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Conclusion

Reduce data dependency almost to a point where it is not possible

anymore to reduce it any further
Have a very good system similar to slices (tiles)
Have little increase in computational complexity in contrast to its
performance gain
WPP and Tiles both are great techniques as they can be used for
the whole video coding
Use its parallel abilities for enormous performance gain

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 19

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

Conclusion

[11]
J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 20

Chair for
Chip Design for
Embedded Computing

Thanks for your attention!

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 21

Chair for
Chip Design for
Embedded Computing

Importance General Problems Specific problems Brief introduction to HEVC HEVC parallelism approaches CABAC Conclusions

References

[1]:Chi Ching Chi; Alvarez-Mesa, M.; Juurlink, B.; Clare, G.; Henry, F.; Pateux, S.; Schierl, T., Parallel Scalability and
Efficiency of HEVC Parallelization Approaches, Circuits and Systems for Video Technology, IEEE Transactions on , vol.22,
no.12, pp.1827,1838, Dec. 2012 doi: 10.1109/TCSVT.2012.2223056
[2][3]:Sullivan, G.J.; Ohm, J.; Woo-Jin Han; Wiegand, T., Overview of the High Efficiency Video Coding (HEVC)
Standard, Circuits and Systems for Video Technology, IEEE Transactions on , vol.22, no.12, pp.1649,1668, Dec. 2012 doi:
10.1109/TCSVT.2012.2221191
[4][6]:Kiho Choi; Jang, Euee S., Leveraging Parallel Computing in Modern Video Coding Standards, MultiMedia, IEEE ,
vol.19, no.3, pp.7,11, July-Sept. 2012 doi: 10.1109/MMUL.2012.36
[5]: http://www.panoramaaudiovisual.com/wp-content/uploads/ 2012/01/H.265-605x289.jpg
[7]:Fuldseth, A.; Horowitz, M.; Shilin Xu; Misra, K.; Segall, A.; Minhua Zhou, Tiles for managing computational
complexity of video encoding and decoding, Picture Coding Symposium (PCS), 2012 , vol., no., pp.389,392, 7-9 May 2012
doi: 10.1109/PCS.2012.6213371
[8][11]:Chi Ching Chi; Alvarez-Mesa, M.; Juurlink, B.; Clare, G.; Henry, F.; Pateux, S.; Schierl, T., Parallel Scalability and
Efficiency of HEVC Parallelization Approaches, Circuits and Systems for Video Technology, IEEE Transactions on , vol.22,
no.12, pp.1827,1838, Dec. 2012 doi: 10.1109/TCSVT.2012.2223056
[9][10]:Sze, V.; Budagavi, M., High Throughput CABAC Entropy Coding in HEVC, Circuits and Systems for Video
Technology, IEEE Transactions on , vol.22, no.12, pp.1778,1791, Dec. 2012 doi: 10.1109/TCSVT.2012.2221526

J
orn Hameyer | Parallelism in High Efficiency Video Coding(HEVC) | 22

Chair for
Chip Design for
Embedded Computing