Color Science and Digital Imaging

Steve Wright, an award-winning VFX pioneer and renowned educator, is the published author of two popular books: Digital Compositing for Film and Video and Compositing Visual Effects. In 2005, his attention turned full-time to educating the next generation of VFX practitioners, imparting his vast technical knowledge gleaned from 30-plus years of production experience. His love of digital imaging technology and color science is the mainstay of his training website, FX Ecademy.

“This book is a treasure trove of knowledge about the technology and color science used for creating digital images. I wish this had been available when I was starting out.” - Grant McKean, Senior Flame VFX Artist, London“The author conducts the reader through a journey of the entire digital imaging pipeline providing essential information all the way. By the end of the book, the reader will have painlessly accumulated an astonishing depth of knowledge of color science.” - Eberhard Hasche, Senior Color Researcher at the Department of Computer Science and Media Technology at Technical University of Applied Sciences, Brandenburg, Germany“The topic of color science is difficult to teach but Steve Wright has a succinct way of communicating this complex subject matter making it easy even for the layperson to comprehend. The visual impact of the animated videos will prove to be an invaluable learning tool for my courses. As an educator, I highly recommend this book!” - Shaun Peyman, Senior Lecturer of VFX at the Media Design School of Auckland, New Zealand“As a supervisor of Visual Effects and CG artists, I see my team struggle with the issues covered by this book all the time. It will be a boon to their careers and the industry as a whole when everybody knows this stuff by heart.” - Isa Alsup, Senior VFX and CG Supervisor

• Foreword...........................................................................................xvPreface............................................................................................. xixAcknowledgments............................................................................ xxiiiIntroduction: The Five Spaces of the Digital Imaging Pipeline...........1Chapter 1 – World Space..............................................................51.1 The Nature of Light....................................................................61.1.1 Wavelength............................................................................61.1.2 Amplitude..............................................................................71.1.3 Phase....................................................................................81.1.4 Polarization............................................................................81.1.5 Monochromatic Light.............................................................91.1.6 Coherent Light.......................................................................91.1.7 Particles............................................................................... 101.1.8 The Inverse Square Law...................................................... 101.1.9 Interior vs. Exterior Lighting.................................................. 111.2 Measuring Light...................................................................... 111.2.1 Candela (cd): The Brightness of a Light Source................... 121.2.2 Lumen (lm): The Total Amount of Incident Light................... 121.2.3 Lux (lx): How Brightly a Scene is Lit..................................... 121.2.4 Other Units.......................................................................... 131.3 Dynamic Range........................................................................ 131.3.1 Brightness vs. Luminance.................................................... 141.3.2 What is Dynamic Range?..................................................... 141.3.3 Stops................................................................................... 161.4 A Color Science Moment........................................................ 161.4.1 Emission Spectrums............................................................ 171.4.2 Spectral Reflectance............................................................ 171.4.3 Color Stimulus..................................................................... 181.5 Light Sources......................................................................... 191.5.1 Emission Spectrums............................................................ 191.5.2 Color Temperature...............................................................211.5.3 Standard Illuminants............................................................231.6 Surface Interactions with Light................................................ 241.6.1 Spectral Reflectance............................................................ 241.6.2 Diffuse vs. Specular Reflections...........................................261.7 Exotic Surface Interactions with Light.....................................271.7.1 Scattering............................................................................271.7.2 Polarized Light.....................................................................291.7.3 The Fresnel Effect................................................................301.7.4 Color from Interference........................................................321.7.5 Dichroic Filters.....................................................................331.8 How We See the World.........................................................331.8.1 Color Perception..................................................................341.8.2 CIE xy-chromaticity Diagram...............................................351.8.3 Our Visual Response...........................................................371.8.4 Adaptation...........................................................................381.8.4.1 General Brightness Adaptation.............................391.8.4.2 Lateral Brightness Adaptation...............................391.8.4.3 Chromatic Adaptation...........................................401.8.5 Calculating Luminance.........................................................401.8.6 Metamers............................................................................421.9 Conclusion.......................................................................................43Chapter 2 – Camera Space....................................................................... 452.1 Camera Perceptions of Color...........................................................462.1.1 Spectral Sensitivity..............................................................462.1.2 Camera Gamuts..................................................................482.1.3 Converting Light to Digits.....................................................482.1.4 The Debayer Process..........................................................492.1.5 Camera Imaging Math......................................................... 512.1.6 Light Sources......................................................................522.1.6.1 Color Temperature................................................522.1.6.2 Spectral Emissions...............................................532.1.7 CCD vs. CMOS...................................................................542.2 Space and Time...............................................................................562.2.1 Resolution...........................................................................562.2.2 ISO......................................................................................572.2.3 Speed..................................................................................572.2.4 Sensor Crop........................................................................592.2.5 Sensor Size.........................................................................602.2.6 Anamorphic Formats...........................................................622.2.6.1 Cinemascope 2.39................................................622.2.6.2 HDTV 1.33............................................................632.3 Camera Dynamic Range..................................................................632.3.1 Clipping...............................................................................642.3.2 Bit Depth.............................................................................652.4 Video Standards..............................................................................662.4.1 Resolutions..........................................................................662.4.1.1 HDTV Standards ....................................................................662.4.1.2 UHDTV Standards ....................................................................672.4.2 Scan Modes........................................................................672.4.2.1 Progressive Scan..................................................672.4.2.2 Interlaced Scan.....................................................672.4.2.3 History..................................................................692.4.3 Frame Rates........................................................................702.5 Output Options................................................................................ 712.6 Camera Artifacts..............................................................................722.6.1 Noise...................................................................................722.6.2 Rolling Shutter..................................................................... 742.6.3 DSLR Rolling Shutter...........................................................752.6.4 Vignetting............................................................................752.7 Lens Artifacts...................................................................................762.7.1 Lens Distortion....................................................................782.7.2 Spherical Aberration............................................................782.7.3 Chromatic Aberration...........................................................792.7.4 Depth of Field......................................................................792.7.5 Veiling Glare.........................................................................802.7.6 Lens Flare............................................................................802.7.7 Bokeh..................................................................................812.7.8 The Swirly Bokeh.................................................................812.8 The DIT............................................................................................822.8.1 On-Set Support...................................................................822.8.2 Data Management...............................................................832.8.3 Color Grading......................................................................832.8.4 Post-Production Liaison.......................................................832.9 Virtual Production............................................................................842.9.1 How it Works.......................................................................842.9.2 The Promise........................................................................852.9.3 The Problems......................................................................862.10 What about 35 mm Film?................................................................872.10.1 Composition........................................................................872.10.2 Frame Rate..........................................................................882.10.3 Color Gamut........................................................................882.10.4 Grain....................................................................................892.10.5 A Softer Look......................................................................892.10.6 No Rolling Shutter................................................................902.10.7 Color Temperature...............................................................902.11 Conclusion.......................................................................................91Chapter 3 - Storage Space....................................................................... 933.1 Image Structure...............................................................................943.1.1 Pixels...................................................................................943.1.2 Bit Depth.............................................................................953.1.3 Channels.............................................................................973.1.4 Deep Data...........................................................................983.1.5 Metadata...........................................................................1003.2 Compression Strategies................................................................. 1013.2.1 Reduced Bit Depth............................................................ 1023.2.2 Color Subsampling............................................................ 1043.2.2.1 Luminance and Chrominance............................. 1053.2.2.2 Subsampling Schemes....................................... 1053.2.3 Discrete Cosine Transform (DCT)....................................... 1073.2.4 MPEG................................................................................ 1093.2.4.1 Spatial Compression........................................... 1093.2.4.2 Temporal Compression....................................... 1103.2.4.3 Summary............................................................ 1103.2.5 Graphics Lossless Compression........................................1113.2.5.1 RLE Compression................................................1113.2.5.2 LZW Compression.............................................. 1123.3 Codecs.......................................................................................... 1123.3.1 H.264................................................................................ 1133.3.2 H.265.................................................................................1143.3.3 JPEG 2000.........................................................................1143.3.4 ProRes.............................................................................. 1153.3.5 DNxHD.............................................................................. 1153.4 Containers..................................................................................... 1163.4.1 MOV.................................................................................. 1163.4.2 MP4................................................................................... 1163.4.3 AVI......................................................................................1173.4.4 Summary............................................................................1173.5 Log Images.....................................................................................1173.5.1 Why Log?.......................................................................... 1183.5.2 What is Log Data?............................................................. 1203.5.3 Data Point Distribution....................................................... 1213.5.4 How Log Images Are Created............................................ 1223.5.5 Log for Photography.......................................................... 1233.6 Cinema Export Options................................................................. 1253.6.1 Camera RAW..................................................................... 1253.6.2 Camera Log....................................................................... 1263.6.3 Compressed Exports........................................................ 1263.7 Cinema File Formats...................................................................... 1273.7.1 EXR................................................................................... 1283.7.2 DPX................................................................................... 1293.7.3 ProRes.............................................................................. 1293.7.4 DNxHR.............................................................................. 1303.7.5 Archival.............................................................................. 1303.8 Photography Export Options.......................................................... 1313.8.1 Camera Raw...................................................................... 1313.8.2 Log Recording (Camera Log)............................................. 1313.8.3 Compressed Exports........................................................ 1323.9 Photography File Formats.............................................................. 1323.9.1 DNG.................................................................................. 1333.9.2 JPEG................................................................................. 1333.9.3 PSD................................................................................... 1333.9.4 PNG.................................................................................. 1333.9.5 TIFF................................................................................... 1333.9.6 WebP................................................................................ 1343.9.7 HEIF.................................................................................. 1343.9.8 Do Not Use........................................................................ 1343.10 Conclusion..................................................................................... 135Chapter 4 – Work Space...........................................................................1374.1 Color Theory.................................................................................. 1384.1.1 Additive Color.................................................................... 1384.1.2 Subtractive Color............................................................... 1394.2 Color Models................................................................................. 1404.2.1 RGB.................................................................................. 1424.2.2 HSL................................................................................... 1424.2.3 YCbCr............................................................................... 1434.2.4 CMYK................................................................................ 1444.3 Color Spaces................................................................................. 1454.3.1 CIE Chromaticity Diagram.................................................. 1464.3.2 Primary Chromaticities....................................................... 1484.3.3 White Points...................................................................... 1484.3.4 The Transfer Function........................................................ 1494.3.5 Color Gamuts.................................................................... 1514.3.6 Pointer’s Gamut................................................................. 1534.3.7 CIEXYZ and CIELAB (CIE L*a*b*)........................................ 1534.4 Linear vs. Gamma Corrected......................................................... 1554.4.1 What is Gamma, Exactly?.................................................. 1574.4.2 Monitor Gamma................................................................. 1594.4.3 Gamma Correction............................................................ 1604.4.4 Gamma-Corrected Images................................................ 1604.4.5 Linear Images.................................................................... 1614.5 Integer vs. Float............................................................................. 1624.5.1 Integer............................................................................... 1634.5.2 Float.................................................................................. 1634.6 LDR vs. HDR Images..................................................................... 1654.7 Choosing Your Work Space........................................................... 1664.7.1 Working in Linear............................................................... 1674.7.2 Working in Display Space.................................................. 1684.7.3 Working in ACES............................................................... 1694.7.4 Working in Log.................................................................. 1704.7.4.1 Picking the Right Curve....................................... 1714.7.4.2 Going Negative................................................... 1724.7.4.3 Limited Dynamic Range...................................... 1724.8 Photography Work Spaces.............................................................1744.8.1 sRGB..................................................................................1744.8.2 Adobe RGB....................................................................... 1754.8.3 ProPhoto RGB................................................................... 1764.8.4 SWOP............................................................................... 1764.9 Archiving........................................................................................ 1784.9.1 OpenEXR.......................................................................... 1794.9.2 DPX................................................................................... 1794.9.3 Camera Raw...................................................................... 1794.10 Conclusion..................................................................................... 179Chapter 5 – Display Space.......................................................................1815.1 Display Attributes........................................................................... 1825.1.1 Spectral Emissions............................................................ 1825.1.2 Color Gamut...................................................................... 1835.1.3 White Point........................................................................ 1845.1.4 Transfer Function (Gamma)................................................ 1845.1.5 Brightness......................................................................... 1845.1.6 Contrast Ratio................................................................... 1855.1.7 Bit Depth........................................................................... 1865.1.8 Frame Rates...................................................................... 1875.1.9 Refresh Rates.................................................................... 1885.1.10 Calibration......................................................................... 1885.1.11 Viewing Surround.............................................................. 1895.2 SDR vs. HDR................................................................................. 1915.2.1 Cinema.............................................................................. 1925.3 HDR Display Formats..................................................................... 1925.3.1 Hybrid Log Gamma (HLG)................................................. 1935.3.2 The Perceptual Quantizer (PQ)........................................... 1935.4 Display Devices.............................................................................. 1955.4.1 Flat-Panel Display Construction......................................... 1955.4.2 Workstation Monitors......................................................... 1965.4.2.1 Specs................................................................. 1965.4.3 HDTV................................................................................. 1975.4.3.1 Specs................................................................. 1985.4.4 Digital Cinema................................................................... 1985.4.4.1 Specs................................................................. 1995.5 Coming Soon.................................................................................2005.5.1 UHDTV..............................................................................2005.5.1.1 Specs.................................................................2025.5.2 Digital Cinema 2.0..............................................................2035.5.2.1 RGB Lasers........................................................2035.5.2.2 Laser Phosphor..................................................2035.5.2.3 LED Screens.......................................................2045.5.2.4 High Frame Rate Movies.....................................2045.5.2.5 Specs.................................................................2055.6 Conclusion.....................................................................................205Chapter 6 - Color Management............................................................ 2076.1 Color Grading................................................................................2086.1.1 1D LUTs.............................................................................2096.1.2 3D LUTs.............................................................................2096.1.3 Grading Operations........................................................... 2106.1.4 Range Controls.................................................................. 2126.1.5 Secondary Color Grading.................................................. 2136.1.6 Scopes.............................................................................. 2156.1.6.1 Waveform Monitor............................................... 2166.1.6.2 Vectorscope........................................................ 2176.1.6.3 Histogram........................................................... 2176.1.7 Color Grading in Log......................................................... 2186.1.8 Using Histograms..............................................................2206.1.8.1 Color Matching...................................................2216.1.8.2 Increasing Contrast.............................................2216.2 Digital Intermediate (DI)..................................................................2226.2.1 Conforming........................................................................2226.2.2 The Mastering Process......................................................2236.2.3 Visual Effects..................................................................... 2246.2.4 Deliverable File Formats.....................................................2256.3 ICC Profiles....................................................................................2266.3.1 How ICC Profiles Work......................................................2276.3.2 Color Space Conversions..................................................2296.3.3 Gamut Mapping................................................................2296.4 Working with Supersized Gamuts.................................................. 2316.4.1 Viewing Supersized Gamuts.............................................. 2316.4.2 Remapping Supersized Gamuts........................................2326.4.2.1 Large to Small Gamuts.......................................2326.4.2.2 Small to Large Gamuts.......................................2336.4.3 Tone Mapping....................................................................2346.4.3.1 Soft Clip LUT......................................................2356.5 Color Pipelines...............................................................................2366.5.1 Display Space....................................................................2376.5.2 Color Managed..................................................................2376.6 OpenColorIO (OCIO)......................................................................2396.6.1 Implementation..................................................................2396.6.2 Color Space Conversions.................................................. 2406.6.3 Roles................................................................................. 2406.6.4 Display Transforms............................................................ 2416.6.5 Look Dev........................................................................... 2416.6.6 Custom LUTs..................................................................... 2416.6.7 Color Grading.................................................................... 2416.6.8 Discipline........................................................................... 2426.7 ACES Color Management.............................................................. 2426.7.1 The ACES Workflow.......................................................... 2446.7.2 ACES Gamuts................................................................... 2456.7.2.1 ACES 2065-1...................................................... 2466.7.2.2 ACEScg.............................................................. 2476.7.2.3 ACEScc.............................................................. 2486.7.2.4 ACESproxy......................................................... 2486.7.3 VFX with ACEScg.............................................................. 2486.7.4 OCIO and ACES................................................................ 2496.8 Historical Perspective.....................................................................2506.9 Conclusion..................................................................................... 251Glossary..................................................................................................... 253Index............................................................................................................271