Beruflich Dokumente
Kultur Dokumente
Multimedia Processing
Spring 2008
Introduction
Technology Digital era Image/Video/Audio Huge amounts of Data Compression Compression lose data selectively Human Visual System (HVS) Visual Communication System (VCS) Limitations of Human Vision exploit
Basic Principle in Perceptual coding technique - consider all the data that humans cannot perceive as superfluous data, and discard them.
[2]
Generally inconsistent between individuals, needs impractical elaborate setups and lack objectivity. Best predictor of Visual Quality
[2]
Double Stimulus Impairment Scale Method (DSISM) Double Stimulus Quality Scale Method (DSQSM) Comparison Scale Method (CSM) Single Stimulus Method (SSM) Continuous Quality Evaluations (CQE)
[33]
[33]
Vision Models
[34]
Tektronix
Board design and MPEG vision-optimized encoder implementation
[2]
[8]
[8]
[8]
[2]
Image 1
JND Map
Image 2
[8]
JND - visibility threshold below which any change cannot be detected by the HVS. Determination of JND Complex and challenging
JND map JND values units of JNDs => represents the difference between two images.
1JND : differences are barely visible, cannot be distinguished, even when exact nature and location of differences are known in advance. 3JNDs : differences visible to detailed observation, not obvious only when an observer knows exactly where to look. 5JNDs : differences are clearly visible readily apparent.
Popularity of MPEG-2
Objective:
Study the Sarnoff Vision Optimized MPEG-2 Encoder which uses the Sarnoff JND Vision Model to improve the perceived image quality at low bitrate.
Motivation:
[2]
JND Metrics
JND parameters
[1]
Apply JND as a MPEG-friendly prefilter to maskable regions. Macroblock-level multi-pass JND. Frame-level feed-forward JND for real-time IPB quality equalization.
[1]
Preprocessing features 3:2 pull-down detection scene change detection flash frame detection noise reduction Fade-to-black correction
Fast hierarchical motion estimation ~1000 times faster than full search with equivalent video quality
Improved mode decision using motion vector bits favored skipped MB at low bit rate Improved rate control adaptive Rate-Distortion model
Side information can be stored on a storage device, then made available to an encoder.
The encoder utilizes side information to best select one or more coding parameters.
Degree of Motion Anomalous Frame Detection Fade-Out Detection Complexity of the Next N Frames 3:2 Pull-down Advice Bits needed to Encode this frame at Constant Quantization Scale/Quality
Noise Filtering
Adaptive Q Matrix
Adjust Q Matrix
The algorithm comprises two parts: (1) the shape adaptation of the quantization matrix and (2) the mean adjustment of the Q matrix.
DCT Map
Fig. illustrates the P pictures (P1 and P3) that are worst affected by scene cuts within an MPEG GoP [ref]
Reducing bits allocated to surrounding B pictures increase P frame bit allocation avoid VBV underflow B picture errors do not carry over to any other pictures within the group of pictures safely spread out over several B pictures.
Conclusions
Vision based optimization gives significant improvement in image quality below 2 Mbits/sec
[2]
Adaptive pre-processing to select features for bit allocation Control quality among frame types - IPB rate control Optimized bit allocation within a frame Mquant optimization
Balanced encoder incorporating improvements to all aspects of the encoder Most approaches used are directly applicable to MPEG-4 encoding
[15]
References
[1] H.R. Wu and K.R. Rao, Digital Video Image Quality and Perceptual Coding, Boca Raton, FL: CRC Press, 2006. [2] A. Pica, Making Every bit Count Vision Optimized Encoding, Sarnoff Corp. [3] K.R. Rao and J.J. Hwang Techniques and Standards for Image, Video, Audio Coding, Upper Saddle River, NJ: Prentice Hall, 1997. [4] Y. Jia, W. Lin and A.A. Kassim, Estimating Just-Noticeable Distortion for Video, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 16, pp. 820-829, July 2006. [5] MPEG-2 reference software ISO/IEC 13818-5:2005, http://standards.iso.org/ittf/licence.html [6] MPEG homepage - http://www.chiariglione.org/mpeg/ [7] MPEG reference website - http://www.mpeg.org [8] J. Lubin, A Human Vision System Model For Objective Picture Quality Measurements, International Broadcasting Convention, pp. 498-503, 12-16 Sept., 1997. [9] J. Lubin, Just Noticeable Difference Analysis: How and Why We Measure and Model the Visibility of Differences between Images, Sarnoff Corp. [10] Measuring Image Quality: Sarnoff's JNDmetrix Technology, Sarnoff Corp. [11] MPEG2 Overview - http://www.erg.abdn.ac.uk/research/future-net/digital-video/mpeg2.html [12] S. Winkler, Issues in vision modeling for perceptual video quality assessment, Signal processing, 78-231, 1999.
References
(contd)
[25]S. Daly, The visible difference predictor: An algorithm for the assessment of image fidelity, in Digital Images and Human Vision, A. B. Watson, ed., pp. 179-206, MIT Press, 1993. [26] J. Lubin, A Visual Discrimination Model for Imaging System Design and Evaluation, Vision Models for Target Detection and Recognition, Eli Peli, Editor, World Scientific, New Jersey, pp. 245-283, 1995. [27] Van den Branden Lambrecht and J. Farrell, Perceptual quality metric for digitally coded color images, Proceedings of the VIII European Signal Processing Conference EUSIPCO, pp. 1175-1178, 1996. [28] M. Masry, and S. S. Hemani, An Analysis of Subjective Quality in Low Bit Rate Video, Proc. IEEE ICIP Thessaloniki, Greece, pp. 465-468, 2001. [29] S. Winkler, C. van den Branden Lambrecht and M. Kunt, Vision and Video: Models and Applications, Ecublens : EPFL, 2001. [30] P. Lindh and C. van den Branden Lambrecht, Efficient spatio-temporal decomposition for perceptual processing of video sequences, IEEE Proceedings of International Conference on Image Processing ICIP'96, Vol. 3, pp. 331-334, 1996. [31] S. Winkler, Issues in vision modeling for perceptual video quality assessment, Signal Processing, Vol. 78, Nr. 2, pp. 231-252, 1999. [32] S. Winkler, Visual fidelity and perceived quality: Towards comprehensive metrics, in Proc. SPIE Human Vision and Electronic Imaging Conference, Vol. 4299, pp. 114-125, 2001.
References
(contd)
[13] S. Winkler infoscience.epfl.ch/record/61769/files/Winkler2000_653.pdf [14] Human Visual System - http://www.ecs.csun.edu/~dsalomon/DC2advertis/AppendH.pdf [15] J. Wang, VDM with H.264, a project proposal, UTA [16] JND - http://en.wikipedia.org/wiki/Just_noticeable_difference [17] Human visual systemimage formation http://vision.berkeley.edu/roordalab/Pubs/EISTChapterRoorda.pdf [18] The human visual system - http://www.dip.ee.uct.ac.za/~nicolls/lectures/eee401f/hvs.pdf [19]Z. Wang and A. C. Bovik, A Human Visual System-Based Objective Video Distortion Measurement System http://www.cns.nyu.edu/~zwang/files/papers/icmps.pdf [20] S. Winkler, Digital video quality : vision models and metrics, Hoboken, NJ : John Wiley & Sons, 2005. [21] G. Westheimer, The eye as an optical instrument. In K. R. Boff, L. Kaufman, J. P. Thomas (eds.), Handbook of Perception and Human Performance, vol. 1, chap. 4, John Wiley & Sons, 1986. [22] D. C. Hood and M. A. Finkelstein. Sensitivity to light. In K. R. Boff, L. Kaufman, J. P. Thomas (eds.),
Handbook of Perception and Human Performance, vol. 1, chap. 5, John Wiley & Sons, 1986.
[23] B. E. Rogowitz, The human visual system: A guide for the display technologist. In Proceedings of the Society for Information Display, vol. 24/3, pp. 235252, 1983. [24] Human Visual System -http://www.ecs.csun.edu/~dsalomon/DC2advertis/AppendH.pdf
References
(contd)
[33] S. Winkler, Quality metric design: A closer look, Proc. SPIE Human Vision and Electronic Imaging Conference, Vol. 3959, pp. pp. 37-44, 2000. [34] S. Winkler, A perceptual distortion metric for digital color video, Proceedings of the SPIE Conference on Human Vision and Electronic Imaging, Vol. 3644, pp. 175-184, 1999. [35] S. Winkler, Visual quality assessment using a contrast gain control model, Proceedings of the IEEE Workshop on Multimedia Signal Processing (MMSP), pp. 527-532, 1999. [36] S. Winkler and P. Vandergheynst, Computing isotropic local contrast from oriented pyramid decompositions, IEEE in Proceedings of the 6th International Conference on Image Processing (ICIP), Vol. 4, pp. 420-424, 1999.
[37] M. Rohaly et al, Video Quality Experts Group: Current results and future directions, Proc. SPIE Visual Communications and Image Processing, Vol. 4067, pp. 742-753, June 2000.
[38] I. D. Basso, F. Tobagi and C. van den Branden Lambrecht, Study of MPEG-2 coding performance based on a perceptual quality metric, Proceedings of the Picture Coding Symposium, pp. 263268, March 1996. [39] van den Branden Lambrecht and O. Verscheure, Perceptual quality measure using a spatiotemporal model of the human visual system, Proceedings of the SPIE, Vol. 2668, pp. 450-461, Jan. 1996. [40] T. Ebrahimi and C. Horne, MPEG-4 natural video coding - An overview, Signal Processing: Image Communication, Vol. 15, Nr. 4-5, pp. 365-385, 2000.
References
(contd)
[41] J. Lubin, A human vision system model for objective picture quality measurements, International Broadcasting Convention, pp. 498 503, Sept. 1997. [42] Sarnoff Corporation, JND Research - http://www.sarnoff.com/research-and-development/videocommunications-networking/video/just-noticeable-difference. [43] M. H. Brill and J. Lubin, Report : Sarnoff JND vision model for flat-panel design http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19980151087_1998151455.pdf. [44] J. Watkinson, MPEG handbook : MPEG-1, MPEG-2, MPEG-4 , 2nd ed., Focal Press, 2004. [45] K. Jack, Video demystified : a handbook for the digital engineer, 5th ed., Burlington,MA :Newnes, 2007. [46] I. E. G. Richardson, Video codec design : developing image and video compression systems , NJ : John Wiley & Sons, 2002. [47] F. Pereira and T. Ebrahimi, MPEG-4 book, Prentice Hall, 2002. [48] I. E. G. Richardson, H.264 and MPEG-4 Video Compression: Video Coding for Next Generation Multimedia, NJ : John Wiley & Sons, 2003. [49] H.264/AVC reference software JM12.1, http://iphome.hhi.de/suehring/tml/. [50] S.K. Kwon, A. Tamhankar and K.R. Rao Overview of H.264 / MPEG-4 Part 10 J. VCIR, Vol. 17, pp. 186-216, April 2006, Special Issue on "Emerging H.264/AVC Video Coding Standard,". [51] Test sequence, ftp://ftp.tnt.uni-hannover.de/pub/svc/testsequences/ [52] Decoder Block Diagram, http://www.altera.com/products/ip/ampp/amphion/images/video-decodercs6651-fig1-pop.pdf
List of Patents