Cover -- Title Page -- Copyright Page -- Contents -- About the Author -- Acknowledgements -- About the Companion Website -- Chapter 1 Introduction -- 1.1 Why Write This Book? -- 1.2 What Is in the Book? -- 1.3 How Should You Use This Book? -- References -- Chapter 2 Video Coding and Video Quality -- 2.1 Introduction -- 2.2 An Overview of Video Coding -- 2.2.1 Just How Much Data Are We Talking About? -- 2.2.2 Bandwidth Is Increasing Every Year, But... -- 2.3 Inputs and Outputs -- 2.3.1 Digital Video -- 2.3.2 Uncompressed Video Formats -- 2.3.2.1 Frames, Fields and Pixels -- 2.3.2.2 Colour and Colour Spaces -- 2.3.3 Side Information -- 2.3.4 Coded Video -- 2.4 Structural Elements -- 2.5 Prediction -- 2.5.1 Prediction in a Video Encoder -- 2.5.2 Prediction in a Video Decoder -- 2.5.3 Prediction Example -- 2.5.4 Choosing a Prediction -- 2.5.5 Inter and Intra Prediction -- 2.6 Transform and Quantisation -- 2.6.1 Transform -- 2.6.2 Quantisation -- 2.7 Bitstream Coding -- 2.7.1 Pre-processing -- 2.7.2 Entropy Encoding -- 2.8 The Coded Bitstream -- 2.9 Storing and Transmitting the Coded Bitstream -- 2.10 The Decoder -- 2.10.1 Entropy Decoding -- 2.10.2 Rescaling and Inverse Transform -- 2.10.3 Reconstruction -- 2.10.4 Decoder Output -- 2.11 The Video Codec Model -- 2.12 Video Codec Performance -- 2.12.1 Video Quality -- 2.12.2 Subjective Quality Measurement -- 2.12.2.1 Factors Influencing Subjective Quality -- 2.12.2.2 Subjective Quality Assessment Methods -- 2.12.3 Objective Quality Measurement -- 2.12.3.1 PSNR -- 2.12.3.2 Other Objective Quality Metrics -- 2.13 Conclusion -- References -- Chapter 3 A History of Video Coding and Video Coding Standards -- 3.1 Introduction -- 3.2 The Foundations of Video Coding, 1950-1990 -- 3.2.1 Entropy Coding: 1952-1987 -- 3.2.2 Frame Differencing: 1956-1972 -- 3.2.3 The Discrete Cosine Transform: 1974-1984.

3.2.4 The Development of the Video Codec Model: 1977-1985 -- 3.2.5 Early Advances in Motion-Compensated Prediction: 1989-1995 -- 3.2.5.1 Bidirectional Prediction -- 3.2.5.2 Variable Block Size Motion Compensation -- 3.2.5.3 Multiple Reference Frames -- 3.2.5.4 Motion Compensation with Fractional Pixel Accuracy -- 3.3 Video Coding Standards and Formats: 1990-2021 -- 3.3.1 How Are the Standards Developed? -- 3.3.1.1 MPEG and VCEG -- 3.3.1.2 The Development Process -- 3.3.1.3 Standards and Intellectual Property Rights -- 3.3.2 H.120, H.261 and MPEG-1: Early Video Coding Standards -- 3.3.3 MPEG-2: Digital TV and DVD-Video -- 3.3.4 H.263 and MPEG-4 Part 2: Internet and Mobile Video -- 3.3.5 H.264: Multi-Purpose Video Coding -- 3.3.6 H.265/HEVC -- 3.3.7 H.266/VVC -- 3.3.8 Other Standards and Formats -- 3.3.8.1 VC-1 -- 3.3.8.2 VP8, VP9 and AV1 -- 3.3.8.3 Essential Video Coding and Low-ComplexityEnhancement Video Coding -- 3.4 Comparing Video Coding Standards -- 3.5 Conclusions -- References -- Chapter 4 Structures -- 4.1 Introduction -- 4.1.1 How Does a Video Codec Use Structures? -- 4.1.2 Separating a Sequence into Processing Units -- 4.1.3 Regular or Irregular Structures -- 4.1.4 Transmission and Access -- 4.2 Coded Video: Sequence to Picture -- 4.2.1 Coded Sequence -- 4.2.2 Coded Frame or Picture -- 4.2.3 Multi-picture Structures -- 4.3 Coded Video: Picture to Basic Unit -- 4.3.1 Basic Units -- 4.3.2 Slices -- 4.3.3 Tiles -- 4.4 Coded Video: Basic Unit to Block -- 4.4.1 Coding Structures -- 4.4.2 Prediction Structures -- 4.4.3 Transform Structures -- 4.5 HEVC Coding Structures -- 4.5.1 Coded HEVC Sequence -- 4.5.2 Parameter Sets -- 4.5.3 Profiles and Levels -- 4.5.4 Reference Picture Sets -- 4.5.5 HEVC Picture Types and Random Access -- 4.5.5.1 Random Access Picture -- 4.5.5.2 Examples: RAPs, Trailing and Leading Pictures.

4.5.5.3 Random Access and Error Recovery -- 4.5.5.4 Bitstream Splicing Operations -- 4.5.6 HEVC-Coded Pictures -- 4.5.7 Slices and Slice Segments -- 4.5.8 Tiles -- 4.5.9 Combining Slices and Tiles in HEVC -- 4.5.10 HEVC Structures and Parallel Processing -- 4.5.10.1 Parallel Decoding Using Tiles -- 4.5.10.2 Parallel Decoding Using Slices -- 4.5.10.3 Wavefront Parallel Processing/Parallelising CTU Rows -- 4.5.11 Coding Tree Units and Coding Units -- 4.5.11.1 Quantisation Groups -- 4.5.12 Prediction Units and Transform Units -- 4.5.12.1 Prediction Unit Structures -- 4.5.12.2 Transform Unit, Transform Block and Transform Sub-Block -- 4.5.13 HEVC Structures Example -- 4.6 Structures in Versatile Video Coding/H.266 -- 4.6.1 Sequence Down to Picture -- 4.6.2 Picture Down to Coding Unit -- 4.7 Conclusion -- Reference -- Chapter 5 Intra Prediction -- 5.1 Introduction -- 5.2 The Intra Prediction Process -- 5.3 Intra Prediction Modes -- 5.3.1 DC Prediction -- 5.3.2 Planar Prediction -- 5.3.3 Directional Prediction -- 5.3.4 Sample Filtering -- 5.4 Prediction Block Sizes -- 5.5 Signalling Intra Prediction Choices -- 5.6 Choosing a Prediction -- 5.7 HEVC Intra Prediction -- 5.7.1 HEVC Intra Prediction Modes -- 5.7.2 HEVC Intra Prediction Filtering -- 5.7.3 Signalling HEVC Intra Prediction Modes -- 5.7.3.1 HEVC Luma Prediction Mode Signalling -- 5.7.3.2 HEVC Chroma Prediction Mode Signalling -- 5.7.4 HEVC Intra Prediction Examples -- 5.8 VVC Intra Prediction -- 5.8.1 Intra Prediction Modes -- 5.8.2 Intra Prediction of Rectangular Blocks -- 5.8.3 Intra Interpolation Filters -- 5.8.4 Cross-Component Prediction -- 5.8.5 Extra Reference Lines -- 5.8.6 Intra Sub-Partitions -- 5.8.7 VVC Intra-coding Performance -- 5.9 Conclusions -- References -- Chapter 6 Inter Prediction -- 6.1 Introduction -- 6.2 Inter Prediction -- the Basics.

6.2.1 Motion Vectors for Inter Prediction: The Terminology Problem -- 6.3 Forward, Backward and Biprediction -- 6.3.1 Picture or Slice Types -- 6.3.2 Forward Prediction from Past Frames -- 6.3.3 Backward Prediction from Future Frames -- 6.3.4 Biprediction from Two Frames -- 6.4 Inter Prediction Block Sizes -- 6.5 Motion Vectors -- 6.5.1 How Does the Encoder Choose Each Motion Vector? -- 6.6 Sub-Pixel Interpolation -- 6.6.1 What Does a Sub-Pixel Interpolated Reference Frame Look Like? -- 6.6.2 Interpolation Filters -- 6.6.3 Trade-Offs in Sub-Pixel Interpolation -- 6.7 Reference Pictures -- 6.7.1 Reference Pictures for Inter Prediction -- 6.7.2 Reference Picture Structures -- 6.7.2.1 Low-Delay Picture Structures -- 6.7.2.2 Random Access Picture Structures -- 6.7.3 Reference Picture Buffers -- 6.7.3.1 Adding Pictures to the Reference Picture Buffer -- 6.7.3.2 Removing Pictures from the Reference Picture Buffer -- 6.7.3.3 Ordering Reference Pictures -- 6.7.3.4 Communicating Reference Picture Structures -- 6.8 Signalling Inter Prediction Choices -- 6.8.1 Inter Prediction Parameters -- 6.8.2 Signalling the Base Motion Vector Parameters -- 6.8.3 Signalling the Delta Motion Vector Parameters -- 6.8.4 Signalling Reference Picture Choices -- 6.9 Skip Mode -- 6.10 Loop Filter -- 6.11 When Inter Prediction Does Not Find a Good Match -- 6.12 HEVC Inter Prediction -- 6.12.1 Overview -- 6.12.1.1 HEVC Inter Prediction Example -- 6.12.2 HEVC Inter prediction Unit Partitioning -- 6.12.2.1 Slice Types and PU Types -- 6.12.2.2 Symmetric PU Partitions -- 6.12.2.3 Asymmetric PU Partitions -- 6.12.2.4 CU Partitionings into Inter-PUs: Examples -- 6.12.3 HEVC Inter-PU Coding -- 6.12.3.1 Skip or Merge Coding -- 6.12.3.2 Inter-PU Coding: Advanced Motion Vector Prediction -- 6.12.3.3 HEVC Inter-PU Coding: Examples -- 6.12.4 HEVC Inter Prediction Sources.

6.12.5 HEVC Biprediction and Weighted Prediction -- 6.12.6 HEVC Sub-Pixel Interpolation -- 6.12.6.1 Luma Sample Interpolation -- 6.12.6.2 Chroma Sample Interpolation -- 6.12.7 HEVC Reference Pictures -- 6.13 Inter Prediction in VVC -- 6.13.1 Inter Prediction Partitions -- 6.13.2 Extended Merge Mode -- 6.13.3 Affine Motion-Compensated Prediction -- 6.13.4 Biprediction Enhancements -- 6.13.5 Sub-Pixel Interpolated Motion Compensation -- 6.14 Conclusions -- References -- Chapter 7 Transform and Quantisation -- 7.1 Introduction -- 7.1.1 Transform: An Overview -- 7.1.2 Quantisation: An Overview -- 7.2 Residual Blocks -- 7.3 Block Transforms -- 7.3.1 What Is the Purpose of a Block Transform? -- 7.3.2 Spatial Frequencies and Basis Patterns -- 7.3.3 What Does the Transform Do? -- 7.3.4 Types of Block Transform -- 7.3.4.1 The Karhunen-Loeve Transform -- 7.3.5 The Discrete Cosine Transform -- 7.3.6 The Hadamard Transform -- 7.3.7 Discrete Sine Transform -- 7.3.8 The Discrete Cosine Transform in Detail -- 7.3.9 The Discrete Cosine Transform: What It Means -- 7.3.10 The Discrete Cosine Transform: Significant Coefficients -- 7.4 Quantisation -- 7.4.1 Combining Transform and Quantisation -- 7.4.2 Designing the Quantiser -- 7.5 Transform and Quantisation in Practice -- 7.6 HEVC Transform and Quantisation -- 7.6.1 Overview of HEVC Transform and Quantisation -- 7.6.2 The HEVC Transforms -- 7.6.2.1 DCT-Based Transforms -- 7.6.2.2 DST-Based Transform -- 7.6.2.3 The HEVC Forward and Inverse Transform and Quantisation Processes -- 7.6.3 HEVC Quantisation and Rescaling -- 7.6.3.1 HEVC QP and Qstep -- 7.6.3.2 HEVC Forward Quantiser -- 7.6.3.3 HEVC Rescaling -- 7.6.3.4 HEVC Quantisation Matrices -- 7.6.4 Example: The Complete HEVC Transform and Quantise Processes -- 7.6.4.1 Forward Transform and Quantise -- 7.6.4.2 Rescale and Inverse Transform.