Optimal and Robust State Estimation

YURIY S. SHMALIY, PhD, is a Professor with the Universidad de Guanajuato, Mexico. He serves as an Editorial Board Member in various scientific journals and is an IEEE Fellow. He also developed the theory of FIR state estimation, gave many keynote and plenary lectures, and his discrete orthogonal polynomials are called discrete Shmaliy moments. SHUNYI ZHAO, PhD, is a Professor with the Jiangnan University, China. His current research interests include statistical signal processing, Bayesian estimation theory, and fault detection and diagnosis.

• 1 Introduction 11.1 What is System State? 21.1.1 Why and How do We Estimate State? 21.1.2 What Model to Estimate State? 31.1.3 What are Basic State Estimates in Discrete Time? 51.2 Properties of State Estimators 61.2.1 Structures and Types 61.2.2 Optimality 101.2.3 Unbiased Optimality (Maximum Likelihood) 111.2.4 Suboptimality 141.2.5 Unbiasedness 171.2.6 Deadbeat 171.2.7 Denoising (Noise Power Gain) 171.2.8 Stability 181.2.9 Robustness 181.2.10 Computational Complexity 191.2.11 Memory Use 201.3 More About FIR State Estimators 201.4 Historical Overview and Most Noticeable Works 211.5 Summary 261.6 Problems 272 Probability and Stochastic Processes 312.1 Random Variables 312.1.1 Moments and Cumulants 332.1.2 Product Moments 392.1.3 Vector Random Variables 412.1.4 Conditional Probability. Bayes’ Rule 422.1.5 Transformation of Random Variables 452.2 Stochastic Processes 472.2.1 Correlation Function 482.2.2 Power Spectral Density 512.2.3 Gaussian Processes 532.2.4 White Gaussian Noise 552.2.5 Markov Processes 572.3 Stochastic Differential Equation 602.3.1 Standard Stochastic Differential Equation 612.3.2 Itˆo and Stratonovich Stochastic Calculus 612.3.3 Diffusion Process Interpretation 622.3.4 Fokker-Planck-Kolmogorov Equation 632.3.5 Langevin Equation 642.4 Summary 652.5 Problems 663 State Estimation 713.1 Lineal Stochastic Process in State Space 713.1.1 Continuous-Time Model 733.1.2 Discrete-Time Model 773.2 Methods of Linear State Estimation 813.2.1 Bayesian Estimator 823.2.2 Maximum Likelihood Estimator 853.2.3 Least Squares Estimator 863.2.4 Unbiased Estimator 873.2.5 Kalman Filtering Algorithm 883.2.6 Backward Kalman Filter 943.2.7 Alternative Forms of Kalman Filter 963.2.8 General Kalman Filter 983.2.9 Kalman-Bucy Filter 1103.3 Linear Recursive Smoothing 1133.3.1 Rauch-Tung-Striebel Algorithm 1133.3.2 Bryson-Frazier Algorithm 1143.3.3 Two-Filter (Forward-Backward) Smoothing 1153.4 Nonlinear Models and Estimators 1163.4.1 Extended Kalman Filter 1173.4.2 Unscented Kalman Filter 1193.4.3 Particle Filtering 1223.5 Robust State Estimation 1263.5.1 Robustified Kalman Filter 1273.5.2 Robust Kalman Filter 1283.5.3 H8 Filtering 1313.5.4 Game Theory H8 Filter 1323.6 Summary 1333.7 Problems 1344 Optimal FIR and Limited Memory Filtering 1394.1 Extended State-Space Model 1404.2 The a posteriori Optimal FIR Filter 1424.2.1 Batch Estimate and Error Covariance 1434.2.2 Recursive Forms 1454.2.3 System Identification 1494.3 The a posteriori Optimal Unbiased FIR Filter 1494.3.1 Batch OUFIR-I Estimate and Error Covariance 1504.3.2 Recursive Forms for OUFIR-I Filter 1514.3.3 Batch OUFIR-II Estimate and Error Covariance 1534.3.4 Recursion Forms for OUFIR-II Filter 1544.4 Maximum Likelihood FIR Estimator 1584.4.1 ML-I FIR Filtering Estimate 1584.4.2 Equivalence of ML-I FIR and OUFIR Filters 1594.4.3 ML-II FIR Filtering Estimate 1624.4.4 Properties of ML FIR State Estimators 1634.5 The a priori FIR Filters 1644.5.1 The a priori Optimal FIR Filter 1644.5.2 The a priori Optimal Unbiased FIR Filter 1654.6 Limited Memory Filtering 1654.6.1 Batch Limited Memory Filter 1664.6.2 Iterative LMF Algorithm using Recursions 1684.7 Continuous-Time Optimal FIR Filter 1694.7.1 Optimal Impulse Response 1694.7.2 Differential Equation Form 1714.8 Extended a posteriori OFIR Filtering 1724.9 Properties of FIR State Estimators 1744.10 Summary 1794.11 Problems 1825 Optimal FIR Smoothing 1875.1 Introduction 1875.2 Smoothing Problem 1885.3 Forward Filter/Forward Model q-lag OFIR Smoothing 1895.3.1 Batch Smoothing Estimate 1905.3.2 Error Covariance 1935.4 Backward OFIR Filtering 1955.4.1 Backward State-Space Model 1955.4.2 Batch Estimate 1965.4.3 Recursive Estimate and Error Covariance 1985.5 Backward Filter/Backward Model g-lag OFIR Smoother 2025.5.1 Batch Smoothing Estimate 2035.5.2 Error Covariance 2045.6 Forward Filter/Backward Model q-Lag OFIR Smoother 2055.6.1 Batch Smoothing Estimate 2055.6.2 Error Covariance 2085.7 Backward Filter/Forward Model q-Lag OFIR Smoother 2085.7.1 Batch Smoothing Estimate 2085.7.2 Error Covariance 2115.8 Two-Filter q-lag OFIR Smoother 2135.9 q-Lag ML FIR Smoothing 2145.9.1 Batch q-lag ML FIR Estimate 2155.9.2 Error Covariance 2165.10 Summary 2165.11 Problems 2176 Unbiased FIR State Estimation 2216.1 Introduction 2216.2 The a posteriori UFIR Filter 2226.2.1 Batch Form 2226.2.2 Iterative Algorithm Using Recursions 2246.2.3 Recursive Error Covariance 2266.2.4 Optimal Averaging Horizon 2286.3 Backward a posteriori UFIR Filter 2346.3.1 Batch Form 2356.3.2 Recursions and Iterative Algorithm 2366.3.3 Recursive Error Covariance 2396.4 The q-lag UFIR Smoother 2406.4.1 Batch and Recursive Forms 2406.4.2 Error Covariance 2426.4.3 Equivalence of UFIR Smoothers 2446.5 State Estimation using Polynomial Models 2456.5.1 Problems Solved with UFIR Structures 2466.5.2 The p-shift UFIR Filter 2476.5.3 Filtering of Polynomial Models 2506.5.4 Discrete Shmaliy Moments 2526.5.5 Smoothing Filtering and Smoothing 2526.5.6 Generalized Savitzky-Golay Filter 2546.5.7 Predictive Filtering and Prediction 2556.6 UFIR State Estimation under Colored Noise 2566.6.1 Colored Measurement Noise 2566.6.2 Colored Process Noise 2596.7 Extended UFIR Filtering 2626.7.1 First-Order Extended UFIR Filter 2636.7.2 Second-Order Extended UFIR Filter 2636.8 Robustness of UFIR Filter 2666.8.1 Errors in Noise Covariances and Weighted Matrices 2686.8.2 Model Errors 2716.8.3 Temporary Uncertainties 2746.9 Implementation of Polynomial UFIR Filters 2766.9.1 Filter Structures in z-Domain 2766.9.2 Transfer Function in DFT Domain 2826.10 Summary 2876.11 Problems 2887 FIR Prediction and Receding Horizon Filtering 2957.1 Introduction 2957.2 Prediction Strategies 2967.2.1 Kalman Predictor 2967.3 Extended Predictive State-Space Model 2987.4 UFIR Predictor 2987.4.1 Batch UFIR Predictor 2997.4.2 Iterative Algorithm using Recursions 2997.4.3 Recursive Error Covariance 3037.5 Optimal FIR Predictor 3047.5.1 Batch Estimate and Error Covariance 3057.5.2 Recursive Forms and Iterative Algorithm 3067.6 Receding Horizon FIR Filtering 3087.6.1 MVF-I Filter for Stationary Processes 3097.6.2 MVF-II Filter for Nonstationary Processes 3117.7 Maximum Likelihood FIR Predictor 3137.7.1 ML-I FIR Predictor 3147.7.2 ML-II FIR Predictor 3157.8 Extended OFIR Prediction 3157.9 Summary 3177.10 Problems 3188 Robust FIR State Estimation under Disturbances 3238.1 Extended Models under Disturbances 3248.2 The a posteriori H2 FIR Filtering 3268.2.1 H2-OFIR Filter 3288.2.2 Optimal Unbiased H2 FIR Filter 3308.2.3 Suboptimal H2 FIR Filtering Algorithms 3368.3 H2 FIR Prediction 3388.3.1 H2-OFIR Predictor 3398.3.2 Bias-constrained H2-OUFIR Predictor 3418.3.3 Suboptimal H2 FIR Predictive Algorithms 3418.3.4 Receding Horizon H2-MVF Filter 3438.4 H8 FIR State Estimation 3448.4.1 The a posteriori H8 FIR Filter 3468.4.2 H8 FIR Predictor 3508.5 H2{H8 FIR Filter and Predictor 3548.6 Generalized H2 FIR State Estimation 3558.6.1 Energy-to-Peak Lemma 3558.6.2 L2-to-L8 FIR Filter and Predictor 3598.7 L1 FIR State Estimation 3628.7.1 Peak-to-Peak Lemma 3638.7.2 L8-to-L8 FIR Filtering and Prediction 3658.8 Game Theory FIR State Estimation 3678.8.1 The a posteriori Energy-to-Power FIR Filter 3688.8.2 Energy-to-Power FIR Predictor 3708.9 Recursive Computation of Robust FIR Estimates 3718.9.1 Uncontrolled Processes 3728.9.2 Controlled Processes 3728.10 FIR Smoothing under Disturbances 3748.11 Summary 3748.12 Problems 3769 Robust FIR State Estimation for Uncertain Systems 3799.1 Extended Models for Uncertain Systems 3809.2 The a posteriori H2 FIR Filtering 3869.2.1 H2-OFIR Filter 3879.2.2 Bias-constrained H2-OFIR Filter 3929.3 H2 FIR Prediction 3949.3.1 Optimal H2 FIR Predictor 3959.3.2 Bias-constrained H2-OUFIR Predictor 3999.4 Suboptimal H2 FIR Structures using LMI 4009.4.1 Suboptimal H2 FIR Filter 4019.4.2 Bias-Constrained Suboptimal H2 FIR Filter 4029.4.3 Suboptimal H2 FIR Predictor 4039.4.4 Bias-Constrained Suboptimal H2 FIR Predictor 4049.5 H8 FIR State Estimation for Uncertain Systems 4059.5.1 The a posteriori H8 FIR Filter 4059.5.2 H8 FIR Predictor 4079.6 Hybrid H2{H8 FIR Structures 4109.7 Generalized H2 FIR Structures for Uncertain Systems 4119.7.1 The a posteriori L2-to-L8 FIR Filter 4129.7.2 L2-to-L8 FIR Predictor 4149.8 Robust L1 FIR Structures for Uncertain Systems 4169.8.1 The a posteriori L8-to-L8 FIR Filter 4179.8.2 L8-to-L8 FIR Predictor 4179.9 Summary 4189.10 Problems 41910 Advanced Topics in FIR State Estimation 42310.1 Distributed Filtering over Networks 42310.1.1 Consensus in Measurements 42410.1.2 Consensus in Estimates 42910.2 Optimal Fusion Filtering under Correlated Noise 43310.2.1 Error Covariances under Cross Correlation 43610.3 Hybrid Kalman/UFIR Filter Structures 43810.3.1 Fusing Estimates with Probabilistic Weights 43810.3.2 Fusing Kalman and Weighted UFIR Estimates 44210.4 Estimation under Delayed and Missing Data 44410.4.1 Deterministic Delays and Missing Data 44510.4.2 Randomly Delayed and Missing Data 44910.5 Summary 45310.6 Problems 45411 Applications of FIR State Estimators 45711.1 UFIR Filtering and Prediction of Clock States 45811.1.1 Clock Model 45811.1.2 Clock State Estimation over GPS-Based TIE Data 45911.1.3 Master Clock Error Prediction 46011.2 Suboptimal Clock Synchronization 46311.2.1 Clock Digital Synchronization Loop 46311.3 Localization over WSNs Using Particle/UFIR Filter 46811.3.1 Sample Impoverishment Issue 47011.3.2 Hybrid Particle/UFIR Filter 47111.4 Self-localization over RFID Tag Grids 47311.4.1 State Space Localization Problem 47411.4.2 Localization Performance 47611.5 INS/UWB-Based Quadrotor Localization 47811.5.1 Quadrotor State Space Model under CMN 47911.5.2 Localization Performance 48111.6 Processing of Biosignals 48111.6.1 ECG Signal Denoising using UFIR Smoothing 48211.6.2 EMG Envelope Extraction using UFIR Filter 48411.7 Summary 48711.8 Problems 488A Matrix Forms and Relationships 489A.1 Derivatives 489A.2 Matrix Identities 489A.3 Special Matrices 490A.4 Equations and Inequalities 491A.5 Linear Matrix Inequalities 493B Norms 495B.1 Vector Norms 495B.2 Matrix Norms 496B.3 Signal Norms 497B.4 System Norms 499References 501