Bayesian Approach to Inverse Problems

Jérôme Idier was born in France in 1966. He received the diploma degree in electrical engineering from the Ecole Superieure d'Electricité, Gif-sur-Yvette, France, in 1988, the Ph.D. degree in physics from the Universite de Paris-Sud, Orsay, France, in 1991, and the HDR (Habilitation a diriger des recherches) from the same university in 2001. Since 1991, he is a full time researcher at CNRS (Centre National de la Recherche Scientifique). He has been with the Laboratoire des Signaux et Systemes from 1991 to 2002, and with IRCCyN (Institut de Recherches en Cybernetique de Nantes (IRCCyN) since september 2002.His major scientific interest is in statistical approaches to inverse problems for signal and image processing. More specifically, he studies probabilistic modeling, inference and optimization issues yielded by data processing problems such as denoising, deconvolution, spectral analysis, reconstruction from projections. The investigated applications are mainly non destructive testing, astronomical imaging and biomedical signal processing, and also radar imaging and geophysics. Dr Idier has been involved in joint research programs with several specialized research centers: EDF (Electricite de France), CEA (Commissariat a l'Energie Atomique), CNES (Centre National d'Etudes Spatiales), ONERA (Office National d'Etudes et de Recherches Aerospatiales), Loreal, Thales, Schlumberger.

• Introduction 15Jérôme IDIERPART I. FUNDAMENTAL PROBLEMS AND TOOLS 23Chapter 1. Inverse Problems, Ill-posed Problems 25Guy DEMOMENT, Jérôme IDIER1.1. Introduction 251.2. Basic example 261.3. Ill-posed problem 301.3.1. Case of discrete data 311.3.2. Continuous case 321.4. Generalized inversion 341.4.1. Pseudo-solutions 351.4.2. Generalized solutions 351.4.3. Example 351.5. Discretization and conditioning 361.6. Conclusion 381.7. Bibliography 39Chapter 2. Main Approaches to the Regularization of Ill-posed Problems 41Guy DEMOMENT, Jérôme IDIER2.1. Regularization 412.1.1. Dimensionality control 422.1.2. Minimization of a composite criterion 442.2. Criterion descent methods 482.2.1.Criterion minimization for inversion 482.2.2. The quadratic case 492.2.3. The convex case 512.2.4. General case 522.3. Choice of regularization coefficient 532.3.1. Residual error energy control 532.3.2. “L-curve” method 532.3.3. Cross-validation 542.4. Bibliography 56Chapter 3. Inversion within the Probabilistic Framework 59Guy DEMOMENT, Yves GOUSSARD3.1. Inversion and inference 593.2. Statistical inference 603.2.1. Noise law and direct distribution for data 613.2.2. Maximum likelihood estimation 633.3. Bayesian approach to inversion 643.4. Links with deterministic methods 663.5. Choice of hyperparameters 673.6. A priori model683.7. Choice of criteria 703.8. The linear, Gaussian case 713.8.1. Statistical properties of the solution 713.8.2. Calculation of marginal likelihood 733.8.3. Wiener filtering 743.9. Bibliography 76PART II. DECONVOLUTION 79Chapter 4. Inverse Filtering and Other Linear Methods 81Guy LE BESNERAIS, Jean-François GIOVANNELLI, Guy DEMOMENT4.1. Introduction 814.2. Continuous-time deconvolution 824.2.1. Inverse filtering 824.2.2. Wiener filtering 844.3. Discretization of the problem 854.3.1. Choice of a quadrature method 854.3.2. Structure of observation matrix H 874.3.3. Usual boundary conditions 894.3.4. Problem conditioning 894.3.5.Generalized inversion 914.4. Batch deconvolution 924.4.1. Preliminary choices 924.4.2. Matrix form of the estimate 934.4.3. Hunt’s method (periodic boundary hypothesis) 944.4.4. Exact inversion methods in the stationary case 964.4.5. Case of non-stationary signals 984.4.6. Results and discussion on examples 984.5. Recursive deconvolution 1024.5.1. Kalman filtering 1024.5.2. Degenerate state model and recursive least squares 1044.5.3. Autoregressive state model 1054.5.4. Fast Kalman filtering 1084.5.5. Asymptotic techniques in the stationary case 1104.5.6. ARMA model and non-standard Kalman filtering 1114.5.7. Case of non-stationary signals 1114.5.8. On-lineprocessing: 2Dcase 1124.6. Conclusion 1124.7. Bibliography 113Chapter 5. Deconvolution of Spike Trains 117Frédéric CHAMPAGNAT, Yves GOUSSARD, Stéphane GAUTIER, Jérôme IDIER5.1. Introduction 1175.2. Penalization of reflectivities, L2LP/L2Hy deconvolutions 1195.2.1. Quadratic regularization 1215.2.2. Non-quadratic regularization 1225.2.3. L2LPorL2Hy deconvolution 1235.3. Bernoulli-Gaussian deconvolution 1245.3.1. Compound BG model 1245.3.2. Various strategies for estimation 1245.3.3. General expression for marginal likelihood 1255.3.4. An iterative method for BG deconvolution 1265.3.5. Other methods 1285.4. Examples of processing and discussion 1305.4.1. Nature of the solutions 1305.4.2. Setting the parameters 1325.4.3. Numerical complexity 1335.5. Extensions 1335.5.1. Generalization of structures of R and H 1345.5.2. Estimation of the impulse response . . . 1345.6. Conclusion 1365.7. Bibliography 137Chapter 6. Deconvolution of Images 141Jérôme IDIER, Laure BLANC-FÉRAUD6.1. Introduction 1416.2. Regularization in the Tikhonov sense 1426.2.1. Principle 1426.2.2. Connection with image processing by linear PDE 1446.2.3. Limits of Tikhonov’s approach 1456.3. Detection-estimation 1486.3.1. Principle 1486.3.2. Disadvantages 1496.4. Non-quadratic approach 1506.4.1. Detection-estimation and non-convex penalization 1546.4.2. Anisotropic diffusion by PDE 1556.5. Half-quadratic augmented criteria 1566.5.1. Duality between non-quadratic criteria and HQ criteria 1576.5.2. Minimization of HQ criteria 1586.6. Application in image deconvolution 1596.6.1. Calculation of the solution 1596.6.2. Example 1616.7. Conclusion 1646.8. Bibliography 165PART III. ADVANCED PROBLEMS AND TOOLS 169Chapter 7. Gibbs-Markov Image Models 171Jérôme IDIER7.1. Introduction 1717.2. Bayesian statistical framework 1727.3. Gibbs-Markov fields 1737.3.1. Gibbs fields 1747.3.2. Gibbs-Markov equivalence 1777.3.3. Posterior law of a GMRF 1807.3.4. Gibbs-Markov models for images 1817.4. Statistical tools, stochastic sampling 1857.4.1. Statistical tools 1857.4.2. Stochastic sampling 1887.5. Conclusion 1947.6. Bibliography 195Chapter 8. Unsupervised Problems 197Xavier DESCOMBES, Yves GOUSSARD8.1. Introduction and statement of problem 1978.2. Directly observed field 1998.2.1. Likelihood properties 1998.2.2. Optimization 2008.2.3. Approximations 2028.3. Indirectly observed field 2058.3.1. Statement of problem 2058.3.2. EM algorithm 2068.3.3. Application to estimation of the parameters of a GMRF 2078.3.4. EM algorithm and gradient 2088.3.5. Linear GMRF relative to hyperparameters 2108.3.6. Extensions and approximations 2128.4. Conclusion 2158.5. Bibliography 216PART IV. SOME APPLICATIONS 219Chapter 9. Deconvolution Applied to Ultrasonic Non-destructive Evaluation 221Stéphane GAUTIER, Frédéric CHAMPAGNAT, Jérôme IDIER9.1. Introduction 2219.2. Example of evaluation and difficulties of interpretation 2229.2.1. Description of the part to be inspected 2229.2.2. Evaluation principle 2229.2.3. Evaluation results and interpretation 2239.2.4. Help with interpretation by restoration of discontinuities 2249.3. Definition of direct convolution model 2259.4. Blind deconvolution 2269.4.1. Overview of approaches for blind deconvolution 2269.4.2. DL2Hy/DBGd econvolution 2309.4.3. Blind DL2Hy/DBG deconvolution 2329.5. Processing real data 2329.5.1. Processing by blind deconvolution 2339.5.2. Deconvolution with a measured wave 234 9.5.3. Comparison between DL2Hy and DBG 2379.5.4. Summary 2409.6. Conclusion 2409.7. Bibliography 241Chapter 10. Inversion in Optical Imaging through Atmospheric Turbulence 243Laurent MUGNIER, Guy LE BESNERAIS, Serge MEIMON10.1. Optical imaging through turbulence 24310.1.1. Introduction 24310.1.2. Image formation 24410.1.4. Imaging techniques 24910.2. Inversion approach and regularization criteria used 25310.3. Measurement of aberrations 25410.3.1. Introduction 25410.3.2. Hartmann-Shack sensor 25510.3.3. Phase retrieval and phase diversity 25710.4. Myopic restoration in imaging 25810.4.1. Motivation and noise statistic 25810.4.2. Data processing in deconvolution from wavefront sensing 25910.4.3. Restoration of images corrected by adaptive optics 26310.4.4. Conclusion 26710.5. Image reconstruction in optical interferometry (OI) 26810.5.1. Observation model 26810.5.2. Traditional Bayesian approach 27110.5.3. Myopic modeling 27210.5.4. Results 27410.6. Bibliography 277Chapter 11. Spectral Characterization in Ultrasonic Doppler Velocimetry 285Jean-François GIOVANNELLI, Alain HERMENT11.1. Velocity measurement in medical imaging 28511.1.1. Principle of velocity measurement in ultrasound imaging 28611.1.2. Information carried by Doppler signals 28611.1.3.Some characteristics and limitations 28811.1.4. Data and problems treated 28811.2. Adaptive spectral analysis 29011.2.1. Least squares and traditional extensions 29011.2.2. Long AR models – spectral smoothness – spatial continuity 29111.2.3. Kalman smoothing 29311.2.4. Estimation of hyperparameters 29411.2.5. Processing results and comparisons 29611.3. Tracking spectral moments 29711.3.1. Proposed method 29811.3.2. Likelihood of the hyperparameters 30211.3.3. Processing results and comparisons 30411.4. Conclusion 30611.5. Bibliography 307Chapter 12. Tomographic Reconstruction from Few Projections 311Ali MOHAMMAD-DJAFARI, Jean-Marc DINTEN12.1. Introduction 31112.2. Projection generation model 31212.3. 2D analytical methods 31312.4. 3D analytical methods 31712.5. Limitations of analytical methods 31712.6. Discrete approach to reconstruction 31912.7. Choice of criterion and reconstruction methods 32112.8. Reconstruction algorithms 32312.8.1. Optimization algorithms for convex criteria 32312.8.2. Optimization or integration algorithms 32712.9. Specific models for binary objects 32812.10. Illustrations 32812.10.1.2D reconstruction 32812.10.2.3Dreconstruction 32912.11. Conclusions 33112.12. Bibliography 332Chapter 13. Diffraction Tomography 335Hervé CARFANTAN, Ali MOHAMMAD-DJAFARI13.1. Introduction 33513.2. Modeling the problem 33613.2.1. Examples of diffraction tomography applications 33613.2.2. Modeling the direct problem 33813.3. Discretization of the direct problem 34013.3.1. Choice of algebraic framework 34013.3.2. Method of moments 34113.3.3. Discretization by the method of moments 34213.4. Construction of criteria for solving the inverse problem 34313.4.1. First formulation: estimation of x 34413.4.2. Second formulation: simultaneous estimation of x and φ 34513.4.3. Properties of the criteria 34713.5. Solving the inverse problem 34713.5.1. Successive linearizations 34813.5.2. Joint minimization 35013.5.3. Minimizing MAP criterion 35113.6. Conclusion 35313.7. Bibliography 354Chapter 14. Imaging from Low-intensity Data 357Ken SAUER, Jean-Baptiste THIBAULT14.1. Introduction 35714.2. Statistical properties of common low-intensity image data 35914.2.1. Likelihood functions and limiting behavior 35914.2.2. Purely Poisson measurements 36014.2.3. Inclusion of background counting noise 36214.2.4. Compound noise models with Poisson information 36214.3. Quantum-limited measurements in inverse problems 36314.3.1. Maximum likelihood properties 36314.3.2. Bayesian estimation 36614.4. Implementation and calculation of Bayesian estimates 36814.4.1. Implementation for pure Poisson model 36814.4.2. Bayesian implementation for a compound data model 37014.5. Conclusion 37214.6. Bibliography 372List of Authors 375Index 377