Visual Media Coding and Transmission

Professor Ahmet Kondoz, University of Surrey, GuildfordProfessor Kondoz is a Deputy Director in the Centre for Communication Systems Research (CCSR) at the University of Surrey. His current research interests are low bit rate speech, image and video coding error resilient video transmission, mobile multimedia communications, robust wireless ATM, real-time terminal design and implementation for mobile communications. He is the author/co-author of more than 130 publications. His book entitled DIGITAL SPEECH: Coding for Low Bit Rate Communication Systems published by John Wiley & sons in 1994 has been accepted as a standard text in low bit rate speech coding by many engineers and universities.

• VISNET II Researchers xiiiPreface xvGlossary of Abbreviations xvii1 Introduction 12 Video Coding Principles 72.1 Introduction 72.2 Redundancy in Video Signals 72.3 Fundamentals of Video Compression 82.3.1 Video Signal Representation and Picture Structure 82.3.2 Removing Spatial Redundancy 92.3.3 Removing Temporal Redundancy 142.3.4 Basic Video Codec Structure 162.4 Advanced Video Compression Techniques 172.4.1 Frame Types 172.4.2 MC Accuracy 192.4.3 MB Mode Selection 202.4.4 Integer Transform 212.4.5 Intra Prediction 222.4.6 Deblocking Filters 222.4.7 Multiple Reference Frames and Hierarchical Coding 242.4.8 Error-Robust Video Coding 242.5 Video Codec Standards 282.5.1 Standardization Bodies 282.5.2 ITU Standards 292.5.3 MPEG Standards 292.5.4 H.264/MPEG-4 AVC 312.6 Assessment of Video Quality 312.6.1 Subjective Performance Evaluation 312.6.2 Objective Performance Evaluation 322.7 Conclusions 35References 363 Scalable Video Coding 393.1 Introduction 393.1.1 Applications and Scenarios 403.2 Overview of the State of the Art 413.2.1 Scalable Coding Techniques 423.2.2 Multiple Description Coding 453.2.3 Stereoscopic 3D Video Coding 473.3 Scalable Video Coding Techniques 483.3.1 Scalable Coding for Shape, Texture, and Depth for 3D Video 483.3.2 3D Wavelet Coding 683.4 Error Robustness for Scalable Video and Image Coding 743.4.1 Correlated Frames for Error Robustness 743.4.2 Odd–Even Frame Multiple Description Coding for Scalable H.264/AVC 823.4.3 Wireless JPEG 2000: JPWL 913.4.4 JPWL Simulation Results 943.4.5 Towards a Theoretical Approach for Optimal Unequal Error Protection 963.5 Conclusions 98References 994 Distributed Video Coding 1054.1 Introduction 1054.1.1 The Video Codec Complexity Balance 1064.2 Distributed Source Coding 1094.2.1 The Slepian–Wolf Theorem 1094.2.2 The Wyner–Ziv Theorem 1104.2.3 DVC Codec Architecture 1114.2.4 Input Bitstream Preparation – Quantization and Bit Plane Extraction 1124.2.5 Turbo Encoder 1124.2.6 Parity Bit Puncturer 1144.2.7 Side Information 1144.2.8 Turbo Decoder 1154.2.9 Reconstruction: Inverse Quantization 1164.2.10 Key Frame Coding 1174.3 Stopping Criteria for a Feedback Channel-based Transform Domain Wyner–Ziv Video Codec 1184.3.1 Proposed Technical Solution 1184.3.2 Performance Evaluation 1204.4 Rate-distortion Analysis of Motion-compensated Interpolation at the Decoder in Distributed Video Coding 1224.4.1 Proposed Technical Solution 1224.4.2 Performance Evaluation 1264.5 Nonlinear Quantization Technique for Distributed Video Coding 1294.5.1 Proposed Technical Solution 1294.5.2 Performance Evaluation 1324.6 Symmetric Distributed Coding of Stereo Video Sequences 1344.6.1 Proposed Technical Solution 1344.6.2 Performance Evaluation 1374.7 Studying Error-resilience Performance for a Feedback Channel-based Transform Domain Wyner–Ziv Video Codec 1394.7.1 Proposed Technical Solution 1394.7.2 Performance Evaluation 1404.8 Modeling the DVC Decoder for Error-prone Wireless Channels 1444.8.1 Proposed Technical Solution 1454.8.2 Performance Evaluation 1494.9 Error Concealment Using a DVC Approach for Video Streaming Applications 1514.9.1 Proposed Technical Solution 1524.9.2 Performance Evaluation 1554.10 Conclusions 158References 1595 Non-normative Video Coding Tools 1615.1 Introduction 1615.2 Overview of the State of the Art 1625.2.1 Rate Control 1625.2.2 Error Resilience 1645.3 Rate Control Architecture for Joint MVS Encoding and Transcoding 1655.3.1 Problem Definition and Objectives 1655.3.2 Proposed Technical Solution 1665.3.3 Performance Evaluation 1695.3.4 Conclusions 1715.4 Bit Allocation and Buffer Control for MVS Encoding Rate Control 1715.4.1 Problem Definition and Objectives 1715.4.2 Proposed Technical Approach 1725.4.3 Performance Evaluation 1775.4.4 Conclusions 1795.5 Optimal Rate Allocation for H.264/AVC Joint MVS Transcoding 1795.5.1 Problem Definition and Objectives 1795.5.2 Proposed Technical Solution 1805.5.3 Performance Evaluation 1815.5.4 Conclusions 1825.6 Spatio-temporal Scene-level Error Concealment for Segmented Video 1825.6.1 Problem Definition and Objectives 1825.6.2 Proposed Technical Solution 1835.6.3 Performance Evaluation 1875.6.4 Conclusions 1885.7 An Integrated Error-resilient Object-based Video Coding Architecture 1895.7.1 Problem Definition and Objectives 1895.7.2 Proposed Technical Solution 1895.7.3 Performance Evaluation 1955.7.4 Conclusions 1955.8 A Robust FMO Scheme for H.264/AVC Video Transcoding 1955.8.1 Problem Definition and Objectives 1955.8.2 Proposed Technical Solution 1955.8.3 Performance Evaluation 1975.8.4 Conclusions 1985.9 Conclusions 199References 1996 Transform-based Multi-view Video Coding 2036.1 Introduction 2036.2 MVC Encoder Complexity Reduction using a Multi-grid Pyramidal Approach 2056.2.1 Problem Definition and Objectives 2056.2.2 Proposed Technical Solution 2056.2.3 Conclusions and Further Work 2086.3 Inter-view Prediction using Reconstructed Disparity Information 2086.3.1 Problem Definition and Objectives 2086.3.2 Proposed Technical Solution 2086.3.3 Performance Evaluation 2106.3.4 Conclusions and Further Work 2116.4 Multi-view Coding via Virtual View Generation 2126.4.1 Problem Definition and Objectives 2126.4.2 Proposed Technical Solution 2126.4.3 Performance Evaluation 2156.4.4 Conclusions and Further Work 2166.5 Low-delay Random View Access in Multi-view Coding Using a Bit Rate-adaptive Downsampling Approach 2166.5.1 Problem Definition and Objectives 2166.5.2 Proposed Technical Solution 2166.5.3 Performance Evaluation 2196.5.4 Conclusions and Further Work 222References 2227 Introduction to Multimedia Communications 2257.1 Introduction 2257.2 State of the Art: Wireless Multimedia Communications 2287.2.1 QoS in Wireless Networks 2287.2.2 Constraints on Wireless Multimedia Communications 2317.2.3 Multimedia Compression Technologies 2347.2.4 Multimedia Transmission Issues in Wireless Networks 2357.2.5 Resource Management Strategy in Wireless Multimedia Communications 2397.3 Conclusions 244References 2448 Wireless Channel Models 2478.1 Introduction 2478.2 GPRS/EGPRS Channel Simulator 2478.2.1 GSM/EDGE Radio Access Network (GERAN) 2478.2.2 GPRS Physical Link Layer Model Description 2508.2.3 EGPRS Physical Link Layer Model Description 2528.2.4 GPRS Physical Link Layer Simulator 2568.2.5 EGPRS Physical Link Layer Simulator 2618.2.6 E/GPRS Radio Interface Data Flow Model 2688.2.7 Real-time GERAN Emulator 2708.2.8 Conclusion 2718.3 UMTS Channel Simulator 2728.3.1 UMTS Terrestrial Radio Access Network (UTRAN) 2728.3.2 UMTS Physical Link Layer Model Description 2798.3.3 Model Verification for Forward Link 2908.3.4 UMTS Physical Link Layer Simulator 2988.3.5 Performance Enhancement Techniques 3078.3.6 UMTS Radio Interface Data Flow Model 3098.3.7 Real-time UTRAN Emulator 3128.3.8 Conclusion 3138.4 WiMAX IEEE 802.16e Modeling 3168.4.1 Introduction 3168.4.2 WIMAX System Description 3178.4.3 Physical Layer Simulation Results and Analysis 3238.4.4 Error Pattern Files Generation 3248.5 Conclusions 3288.6 Appendix: Eb/No and DPCH_Ec/Io Calculation 329References 3309 Enhancement Schemes for Multimedia Transmission over Wireless Networks 3339.1 Introduction 3339.1.1 3G Real-time Audiovisual Requirements 3339.1.2 Video Transmission over Mobile Communication Systems 3359.1.3 Circuit-switched Bearers 3399.1.4 Packet-switched Bearers 3489.1.5 Video Communications over GPRS 3509.1.6 GPRS Traffic Capacity 3519.1.7 Error Performance 3549.1.8 Video Communications over EGPRS 3579.1.9 Traffic Characteristics 3579.1.10 Error Performance 3589.1.11 Voice Communication over Mobile Channels 3599.1.12 Support of Voice over UMTS Networks 3609.1.13 Error-free Performance 3619.1.14 Error-prone Performance 3629.1.15 Support of Voice over GPRS Networks 3629.1.16 Conclusion 3639.2 Link-level Quality Adaptation Techniques 3659.2.1 Performance Modeling 3659.2.2 Probability Calculation 3679.2.3 Distortion Modeling 3689.2.4 Propagation Loss Modeling 3689.2.5 Energy-optimized UEP Scheme 3699.2.6 Simulation Setup 3709.2.7 Performance Analysis 3729.2.8 Conclusion 3739.3 Link Adaptation for Video Services 3739.3.1 Time-varying Channel Model Design 3749.3.2 Link Adaptation for Real-time Video Communications 3799.3.3 Link Adaptation for Streaming Video Communications 3899.3.4 Link Adaptation for UMTS 3969.3.5 Conclusion 4029.4 User-centric Radio Resource Management in UTRAN 4039.4.1 Enhanced Call-admission Control Scheme 4039.4.2 Implementation of UTRAN System-level Simulator 4039.4.3 Performance Evaluation of Enhanced CAC Scheme 4109.5 Conclusions 411References 41310 Quality Optimization for Cross-network Media Communications 41710.1 Introduction 41710.2 Generic Inter-networked QoS-optimization Infrastructure 41810.2.1 State of the Art 41810.2.2 Generic of QoS for Heterogeneous Networks 42010.3 Implementation of a QoS-optimized Inter-networked Emulator 42210.3.1 Emulation System Physical Link Layer Simulation 42610.3.2 Emulation System Transmitter/Receiver Unit 42810.3.3 QoS Mapping Architecture 42810.3.4 General User Interface 43810.4 Performances of Video Transmission in Inter-networked Systems 44210.4.1 Experimental Setup 44210.4.2 Test for the EDGE System 44310.4.3 Test for the UMTS System 44510.4.4 Tests for the EDGE-to-UMTS System 44510.5 Conclusions 452References 45311 Context-based Visual Media Content Adaptation 45511.1 Introduction 45511.2 Overview of the State of the Art in Context-aware Content Adaptation 45711.2.1 Recent Developments in Context-aware Systems 45711.2.2 Standardization Efforts on Contextual Information for Content Adaptation 46711.3 Other Standardization Efforts by the IETF and W3C 47611.4 Summary of Standardization Activities 47911.4.1 Integrating Digital Rights Management (DRM) with Adaptation 48011.4.2 Existing DRM Initiatives 48011.4.3 The New ‘‘Adaptation Authorization’’ Concept 48111.4.4 Adaptation Decision 48211.4.5 Context-based Content Adaptation 48811.5 Generation of Contextual Information and Profiling 49211.5.1 Types and Representations of Contextual Information 49211.5.2 Context Providers and Profiling 49411.5.3 User Privacy 49711.5.4 Generation of Contextual Information 49811.6 The Application Scenario for Context-based Adaptation of Governed Media Contents 49911.6.1 Virtual Classroom Application Scenario 50011.6.2 Mechanisms using Contextual Information in a Virtual Collaboration Application 50211.6.3 Ontologies in Context-aware Content Adaptation 50311.6.4 System Architecture of a Scalable Platform for Context-aware and DRM-enabled Content Adaptation 50411.6.5 Context Providers 50711.6.6 Adaptation Decision Engine 51011.6.7 Adaptation Authorization 51411.6.8 Adaptation Engines Stack 51711.6.9 Interfaces between Modules of the Content Adaptation Platform 54411.7 Conclusions 552References 553Index 559