Adaptive Blind Signal and Image Processing

Andrzej Cichocki received the M.Sc. (with honors), Ph.D. and Dr.Sc. (Habilitation) degrees, all in electrical engineering, from Warsaw University of Technology in Poland. Since 1972, he has been with the Institute of Theory of Electrical Engineering, Measurement and Information Systems, Faculty of Electrical Engineering at the Warsaw University of Technology, where he obtain a title of a full Professor in 1995. He spent several years at University Erlangen-Nuerenberg in Germany, at the Chair of Applied and Theoretical Electrical Engineering directed by Professor Rolf Unbehauen, as an Alexander-von-Humboldt Research Fellow and Guest Professor. In 1995-1997 he was a team leader of the laboratory for Artificial Brain Systems, at Frontier Research Program RIKEN (Japan), in the Brain Information Processing Group.

Preface xxix 1 Introduction to Blind Signal Processing: Problems and Applications 1 1.1 Problem Formulations An Overview 2 1.1.1 Generalized Blind Signal Processing Problem 2 1.1.2 Instantaneous Blind Source Separation and Independent Component Analysis 5 1.1.3 Independent Component Analysis for Noisy Data 11 1.1.4 Multichannel Blind Deconvolution and Separation 15 1.1.5 Blind Extraction of Signals 19 1.1.6 Generalized Multichannel Blind Deconvolution State Space Models 20 1.1.7 Nonlinear State Space Models Semi-Blind Signal Processing 22 1.1.8 Why State Space Demixing Models? 23 1.2 Potential Applications of Blind and Semi-Blind Signal Processing 24 1.2.1 Biomedical Signal Processing 25 1.2.2 Blind Separation of Electrocardiographic Signals of Fetus and Mother 26 1.2.3 Enhancement and Decomposition of EMG Signals 28 1.2.4 EEG and MEG Data Processing 28 1.2.5 Application of ICA/BSS for Noise and Interference Cancellation in Multi-sensory Biomedical Signals 30 1.2.6 Cocktail Party Problem 35 1.2.7 Digital Communication Systems 36 1.2.8 Image Restoration and Understanding 38 2 Solving a System of Algebraic Equations and Related Problems 43 2.1 Formulation of the Problem for Systems of Linear Equations 44 2.2 Least-Squares Problems 45 2.2.1 Basic Features of the Least-Squares Solution 45 2.2.2 Weighted Least-Squares and Best Linear Unbiased Estimation 47 2.2.3 Basic Network Structure-Least-Squares Criteria 48 2.2.4 Iterative Parallel Algorithms for Large and Sparse Systems 49 2.2.5 Iterative Algorithms with Non-negativity Constraints 51 2.2.6 Robust Criteria and Iteratively Reweighted Least-Squares Algorithm 53 2.2.7 Tikhonov Regularization and SVD 57 2.3 Least Absolute Deviation (1-norm) Solution of Systems of Linear Equations 61 2.3.1 Neural Network Architectures Using a Smooth Approximation and Regularization 62 2.3.2 Neural Network Model for LAD Problem Exploiting Inhibition Principles 64 2.4 Total Least-Squares and Data Least-Squares Problems 68 2.4.1 Problems Formulation 68 2.4.2 Total Least-Squares Estimation 70 2.4.3 Adaptive Generalized Total Least-Squares 74 2.4.4 Extended TLS for Correlated Noise Statistics 76 2.4.5 An Illustrative Example - Fitting a Straight Line to a Set of Points 78 2.5 Sparse Signal Representation and Minimum 1-norm Solution 80 2.5.1 Approximate Solution of Minimum p-norm Problem Using Iterative LS Approach 81 2.5.2 Uniqueness and Optimal Solution for Sparse Representation 84 2.5.3 FOCUSS Algorithms 84 3 Principal/Minor Component Analysis and Related Problems 87 3.1 Introduction 87 3.2 Basic Properties of PCA 88 3.2.1 Eigenvalue Decomposition 88 3.2.2 Estimation of Sample Covariance Matrices 90 3.2.3 Signal and Noise Subspaces - Automatic Choice of Dimensionality for PCA 91 3.2.4 Basic Properties of PCA 94 3.3 Extraction of Principal Components 95 3.4 Basic Cost Functions and Adaptive Algorithms for PCA 99 3.4.1 The Rayleigh Quotient Basic Properties 99 3.4.2 Basic Cost Functions for Computing Principal and Minor Components 100 3.4.3 Fast PCA Algorithm Based on the Power Method 102 3.4.4 Inverse Power Iteration Method 105 3.5 Robust PCA 105 3.6 Adaptive Learning Algorithms for MCA 108 3.7 Unified Parallel Algorithms for PCA/MCA and PSA/MSA 111 3.7.1 Cost Function for Parallel Processing 112 3.7.2 Gradient of J(W) 113 3.7.3 Stability Analysis 114 3.7.4 Unified Stable Algorithms 117 3.8 SVD in Relation to PCA and Matrix Subspaces 118 3.9 Multistage PCA for BSS 120 4 Blind Decorrelation and SOS for Robust Blind Identification 129 4.1 Spatial Decorrelation - Whitening Transforms 130 4.1.1 Batch Approach 130 4.1.2 Optimization Criteria for Adaptive Blind Spatial Decorrelation 132 4.1.3 Derivation of Equivariant Adaptive Algorithms for Blind Spatial Decorrelation 133 4.1.4 Simple Local Learning Rule 136 4.1.5 Gram-Schmidt Orthogonalization 138 4.