Fundamentals and Evolution of MPEG-2 Systems

Jan van der Meer, Independent Consultant, The NetherlandsJan van der Meer retired from Philips as Director of Standardization in July 2008; he is currently an independent consultant. Jan worked for Philips from 1979 until 2008 and is now an independent consultant. He has played a leading role in the MPEG standards committee almost from its very beginning, contributing not only to the developments of the MPEG-1, MPEG-2 and MPEG-4 standards, but also to their usage in specific application areas, such as specified by 3GPP, DAVIC, DVB, IETF and ISMA. Jan is worldwide acknowledged for his contributions to MPEG and other standard bodies and is (co-)inventor of 12 patents. He has chaired the MPEG Systems Group and discussed and promoted the use of MPEG technologies in a large variety of standardization bodies.

• Foreword xi Preface xiiiAbout the Author xviiAcknowledgements xxiPart One BACKGROUNDS OF MPEG-2 SYSTEMS 11 Introduction 31.1 The Scope of This Book 71.2 Some Definitions 7References 82 Technology Developments Around 1990 9References 113 Developments in Audio and Video Coding in MPEG 133.1 The Need for Compression 133.1.1 Compression Factors for Audio 143.1.2 Compression Factors for Video 143.2 MPEG Video 193.2.1 Introduction 193.2.2 MPEG-1 and MPEG-2 Video Essentials 203.2.3 Evolution of MPEG Video 393.3 MPEG Audio 473.3.1 MPEG-1 and MPEG-2 Audio Essentials 473.3.2 Evolution of MPEG Audio 53References 594 Other Important Content Formats 614.1 Metadata 614.2 Timed Text 644.3 Lossless and Scalable Lossless Audio 694.4 Multiview Video 694.5 3D Video 704.5.1 Left and Right Views in a Single Video Stream 734.5.2 Depth Information Associated to 2D Video 754.5.3 Use of MVC to Convey Left and Right Views 784.5.4 Further 3D Video Evolution 79References 805 Motivation for a Systems Standard 836 Principles Underlying the MPEG-2 Systems Design 876.1 Building an End-to-End System 876.1.1 Constant End-to-End Delay 876.1.2 Video Coding Delay 886.1.3 Audio Coding Delay 946.1.4 Delay Compensation 956.2 The Multiplex and Demultiplex Operation 976.3 Delivery Schedule of MPEG System Streams 1066.4 Synchronization of Audio and Video 1086.5 MPEG-2 System Streams and the STD Model 1136.6 Timing Issues 1186.6.1 Frequency and Tolerance of the STC in MPEG-1 Systems 1196.6.2 Regeneration of the STC in System Decoders 1216.6.3 Frequency and Tolerance of the STC in MPEG-2 Systems 1256.7 Quality of Service Issues 1276.8 Transport Layer Independence 131References 1327 MPEG-1 Systems: Laying the MPEG-2 Foundation 1337.1 Driving Forces 1337.2 Objectives and Requirements 1367.3 Structure of MPEG-1 System Streams 1387.4 The MPEG-1 System Target Decoder 1437.5 The MPEG-1 System Stream 1557.5.1 Data Structure and Design Considerations 1557.5.2 Constrained System Parameter Streams 1617.5.3 Compliancy Requirements of MPEG-1 System Streams 1667.6 MPEG-1 Applications 1687.6.1 Compact Disc 1687.6.2 Computers 1697.7 Conclusions on MPEG-1 169References 170Part Two THE MPEG-2 SYSTEMS STANDARD 1718 The Development of MPEG-2 Systems 1738.1 Driving Forces 1738.2 Objectives and Requirements 1768.3 The Evolution of MPEG-2 Systems 178References 1859 Layering in MPEG-2 Systems 1879.1 Need for Program Streams and Transport Streams 1879.2 PES Packets as a Common Layer 1889.3 Program Streams 1899.4 Transport Streams 1939.4.1 Transport Packets 1939.4.2 Conveying PES Packets in Transport Packets 1959.4.3 The Size of Transport Packets 1969.4.4 Multiple Programs, PSI, Descriptors and Sections 1999.4.5 Conveying Sections in Transport Packets 213References 21410 Conditional Access and Scrambling 21710.1 Support of Conditional Access Systems 21710.2 Scrambling in Transport Streams 21910.3 Improving the Interoperability between CA Systems 22410.4 Scrambling in Program Streams 225Reference 22611 Other Features of MPEG-2 Systems 22711.1 Error Resiliency 22711.2 Re-Multiplexing of Transport Streams 23011.3 Local Program Insertion in Transport Streams 23411.3.1 Usage of Local Program Insertions 23411.3.2 Associated PSI Issues 23511.3.3 Time Base Discontinuities 23611.4 Splicing in Transport Streams 23911.5 Variable Bitrate and Statistical Multiplexing 24511.6 Padding and Stuffing 24511.7 Random Access and Parsing Convenience 24811.8 Carriage of Private Data 25011.9 Copyright and Copy Control Support 25411.10 Playback Trick Modes 25511.11 Single Program and Partial Transport Streams 25511.12 Program Stream Carriage within a Transport Stream 25811.13 PES Streams 26011.14 Room for Future Extensions 260References 26112 The MPEG-2 System Target Decoder Model 26312.1 Introduction to the MPEG-2 STD 26312.2 The Program Stream STD: P-STD 26412.2.1 Description of P-STD 26412.2.2 Buffer Management in the P-STD 26712.2.3 CSPS: Constrained System Parameter Program Stream 26812.2.4 Usage of P-STD for PES-STD 27012.3 Transport Stream STD: T-STD 27512.3.1 Description of T-STD 27512.3.2 The Use of Transport Buffers 27912.3.3 System Data Processing and Buffer Management 28112.3.4 Processing of Elementary Stream Data 28412.3.5 T-STD Buffers for Elementary Stream Decoding 28812.3.6 Buffer Management for Elementary Stream Data 29012.4 General STD Constraints and Requirements 29012.5 Content Format Specific STD Issues 29212.5.1 Decoding of MPEG Audio Streams in STD Model 29212.5.2 Decoding of MPEG Video Streams in STD Model 29513 Data Structure and Design Considerations 29913.1 System Time Clock Samples and Time Stamps 29913.2 PES Packets 30113.3 Descriptors of Programs and Program Elements 30913.3.1 General Format of Descriptors 30913.3.2 Types of Descriptors 31113.3.3 System Orientated Descriptors 31113.3.4 General Content Descriptors 31513.4 Program Streams 31913.5 Sections 32613.6 Transport Streams and Transport Packets 329Reference 33114 Content Support in MPEG-2 Systems 33314.1 Introduction 33314.2 MPEG-1 33414.2.1 MPEG-1 Video 33414.2.2 MPEG-1 Audio 33414.2.3 MPEG-1 System Stream 33414.3 MPEG-2 33614.3.1 MPEG-2 Video 33614.3.2 MPEG-2 (BC) Audio 33814.3.3 MPEG-2 AAC 34014.3.4 MPEG-2 DSM-CC 34114.3.5 MPEG-2 System Stream 34214.3.6 MPEG-2 IPMP 34314.4 (ITU-T Rec.) H.222.1 34314.5 MHEG 34414.6 MPEG-4 34514.6.1 MPEG-4 Visual 34514.6.2 MPEG-4 Audio 34614.6.3 MPEG-4 Timed Text 34914.6.4 MPEG-4 Systems 35014.7 AVC 35414.8 SVC 36014.9 3D Video 36614.9.1 Service Compatible and Frame Compatible 3D Video 36614.9.2 Depth or Parallax Map as Auxiliary Video Stream 36914.9.3 MVC 37014.10 JPEG 2000 Video 37614.11 Metadata 37714.12 Overview of Assigned Stream-type Values 387References 38915 The Real-Time Interface for Transport Streams 391Reference 39616 Relationship to Download and Streaming Over IP 39716.1 IP Networks and MPEG-2 Systems 39716.2 Streaming Over IP 39716.3 Download 40016.4 Carriage of MPEG-2 Systems Across IP Networks 40016.5 Adaptive HTTP Streaming 401References 40117 MPEG-2 System Applications 40318 The Future of MPEG-2 Systems 407Reference 412Epilogue 413Annexes 423Index 427