From Photon to Pixel

Henri Maitre, Emeritus Professor at Telecom ParisTech, Institut Mines-Telecom.

• Acknowledgements xiChapter 1. First Contact 11.1. Toward a society of the image 11.1.1. A bit of vocabulary in the form of zoology 41.1.2. A brief history of photography 71.2. The reason for this book 101.3. Physical principle of image formation 111.3.1. Light 111.3.2. Electromagnetic radiation: wave and particle 121.3.3. The pinhole 131.3.4. From pinholes to photo cameras 151.4. Camera block diagram 21Chapter 2. The Photographic Objective Lens 252.1. Focusing 262.1.1. From focusing to blurring 262.1.2. Focusing complex scenes 282.2. Depth of field 342.2.1. Long-distance photography 352.2.2. Macrophotography 362.2.3. Hyperfocal 372.3. Angle of view 382.3.1. Angle of view and human visual system 382.3.2. Angle of view and focal length 392.4. Centered systems 412.4.1. Of the importance of glasses in lenses 422.4.2. Chromatic corrections 452.4.3. The choice of an optical system 472.4.4. Diaphragms and apertures 502.4.5. Zoom 532.4.6. Zoom and magnification 542.5. Fisheye systems 572.5.1. Projection functions 572.5.2. Circular and diagonal fisheyes 592.5.3. Fisheyes in practice 612.6. Diffraction and incoherent light 632.6.1. Coherence: incoherence 632.6.2. Definitions and notations 652.6.3. For a single wavelength 662.6.4. Circular diaphragm 682.6.5. Discussion 702.6.6. Case of a wide spectrum 712.6.7. Separation power 732.7. Camera calibration 742.7.1. Some geometry of image formation 742.7.2. Multi-image calibration: bundle adjustment 772.7.3. Fisheye camera calibration 782.8. Aberrations 792.8.1. Chromatic aberration 792.8.2. Geometrical aberrations 802.8.3. Internal reflections 822.8.4. Vignetting 832.8.5. The correction of the aberrations 86Chapter 3. The Digital Sensor 893.1. Sensor size 903.1.1. Sensor aspect ratio 903.1.2. Sensor dimensions 913.1.3. Pixel size 933.2. The photodetector 933.2.1. Image detection materials 933.2.2. CCDs 943.2.3. CMOSs 973.2.4. Back-side illuminated arrangement (BSI), stacked arrangement 1013.2.5. Stacked arrangements 1023.2.6. Influence of the choice of technology on noise 1033.2.7. Conclusion 1043.3. Integrated filters in the sensor 1043.3.1. Microlenses 1043.3.2. Anti-aliasing filters 1063.3.3. Chromatic selection filters 109Chapter 4. Radiometry and Photometry 1114.1. Radiometry: physical parameters 1124.1.1. Definitions 1124.1.2. Radiating objects: emissivity and source temperature 1164.1.3. Industrial lighting sources 1224.1.4. Reflecting objects: reflectance and radiosity 1234.2. Subjective aspects: photometry 1254.2.1. Luminous efficiency curve 1264.2.2. Photometric quantities 1284.3. Real systems 1284.3.1. Etendue 1294.3.2. Camera photometry 1304.4. Radiometry and photometry in practice 1344.4.1. Measurement with a photometer 1344.4.2. Integrated measurements 1374.5. From the watt to the ISO 1384.5.1. ISO sensitivity: definitions 1384.5.2. Standard output ISO sensitivity SOS 1434.5.3. Recommended exposure index 1434.5.4. Exposure value 144Chapter 5. Color 1455.1. From electromagnetic radiation to perception 1475.1.1. The color of objects 1475.1.2. Color perception 1495.2. Color spaces 1515.2.1. The CIE 1931 RGB space 1535.2.2. Other chromatic spaces 1605.2.3. The Lab space 1625.2.4. Other colorimetric spaces 1635.2.5. TV spaces 1645.2.6. The sRGB space 1655.2.7. ICC profile 1685.2.8. Chromatic thresholds 1695.3. The white balance 1705.3.1. Presettings 1715.3.2. Color calibration 1725.3.3. Gray test pattern usage 1735.3.4. Automatic white balance techniques 1735.3.5. The Retinex model 1755.4. Acquiring color 1785.4.1. “True color” images 1815.4.2. Chromatic arrays 1865.4.3. Chromatic selection of the arrays 1925.5. Reconstructing color: demosaicing 1955.5.1. Linear interpolation demosaicing 1965.5.2. Per channel, nonlinear interpolations 1995.5.3. Interchannel, non-linear interpolations 199Chapter 6. Image Quality 2056.1. Qualitative attributes 2066.1.1. The signal–noise ratio 2076.1.2. Resolution 2116.1.3. The modulation transfer function 2156.1.4. Sharpness 2216.1.5. Acutance 2216.2. Global image quality assessment 2266.2.1. Reference-based evaluations 2286.2.2. No-reference evaluation 2306.2.3. Perception model evaluation 2346.3. Information capacity 2376.3.1. The number of degrees of freedom 2386.3.2. Entropy 2436.3.3. Information capacity in photography 2456.4. What about aesthetics? 2526.4.1. Birkhoff’s measure of beauty 2536.4.2. Gestalt theory 2546.4.3. Shannon information theory, Kolmogorov Complexity and Computational Complexity theory 2546.4.4. Learning aesthetic by machine 254Chapter 7. Noise in Digital Photography 2577.1. Photon noise 2587.1.1. Fluctuations in the optical signal 2587.1.2. The Poisson hypothesis in practice 2617.1.3. From photon flux to electrical charge 2627.2. Electronic noise 2657.2.1. Dark current 2657.2.2. Pixel reading noise 2667.2.3. Crosstalk noise 2667.2.4. Reset noise 2677.2.5. Quantization noise 2677.3. Non-uniform noise 2687.3.1. Non-uniformity in detectors 2687.3.2. Salt-and-pepper noise 2687.3.3. Image reconstruction and compression noise 2687.4. Noise models for image acquisition 2697.4.1. Orders of magnitude 270Chapter 8. Image Representation: Coding and Formats 2738.1. “Native” format and metadata 2748.2. RAW (native) format 2758.2.1. Contents of the RAW format 2788.2.2. Advantages of the native format 2808.2.3. Drawbacks of the native format 2818.2.4. Standardization of native formats 2818.3. Metadata 2838.3.1. The XMP standard 2838.3.2. The Exif metadata format 2848.4. Lossless compression formats 2868.4.1. General lossless coding algorithms 2878.4.2. Lossless JPEG coding 2888.5. Image formats for graphic design 2898.5.1. The PNG format 2898.5.2. The TIFF format 2918.5.3. The GIF format 2928.6. Lossy compression formats 2928.6.1. JPEG 2948.6.2. JPEG 2000 2998.7. Tiled formats 3048.8. Video coding 3058.8.1. Video encoding and standardization 3068.8.2. MPEG coding 3078.9. Compressed sensing 310Chapter 9. Elements of Camera Hardware 3139.1. Image processors 3139.1.1. Global architecture and functions 3149.1.2. The central processing unit 3159.1.3. The digital signal processor 3189.1.4. The graphics processing unit 3209.2. Memory 3219.2.1. Volatile memory 3219.2.2. Archival memory cards 3219.3. Screens 3279.3.1. Two screen types 3279.3.2. Performance 3299.3.3. Choice of technology 3309.4. The shutter 3339.4.1. Mechanical shutters 3339.4.2. Electronic shutters 3339.5. Measuring focus 3359.5.1. Maximum contrast detection 3379.5.2. Phase detection 3409.5.3. Focusing on multiple targets 3419.5.4. Telemeter configuration and geometry 3429.5.5. Mechanics of the autofocus system 3439.5.6. Autofocus in practice 3449.6. Stabilization 3469.6.1. Motion sensors 3469.6.2. Compensating for movement 3499.6.3. Video stabilization 3529.7. Additions to the lens assembly: supplementary lenses and filters 3539.7.1. Focal length adjustment 3539.7.2. Infra-red filters 3569.7.3. Attenuation filters 3579.7.4. Polarizing filters 3589.7.5. Chromatic filters 3659.7.6. Colored filters 3669.7.7. Special effect filters 3679.8. Power cells 3679.8.1. Batteries 3689.8.2. Rechargeable Ni-Cd batteries 3689.8.3. Lithium-ion batteries 369Chapter 10. Photographic Software 37310.1. Integrated software 37410.1.1. Noise reduction 37410.1.2. Classic approaches 37510.1.3. Iterative methods 37610.1.4. Non-local approaches 37710.1.5. Facial detection 37910.1.6. Motion tracking 38210.1.7. Image rotation 38410.1.8. Panoramas 38510.2. Imported software 39410.2.1. Improving existing functions 39510.2.2. Creating new functions 39510.3. External software 39710.3.1. High-dynamic images (HDR) 39710.3.2. Plenoptic imaging: improving the depth of field 40210.3.3. Improving resolution: super-resolution 40810.3.4. Flutter-shutters 412Bibliography 417Index 439