High-Dimensional Covariance Estimation

MOHSEN POURAHMADI, PhD, is Professor of Statistics at Texas A&M University. He is an elected member of the International Statistical Institute, a Fellow of the American Statistical Association, and a member of the American Mathematical Society. Dr. Pourahmadi is the author of Foundations of Time Series Analysis and Prediction Theory, also published by Wiley.

• Preface xiPART I MOTIVATION AND THE BASICS1 Introduction 31.1 Least-Squares and Regularized Regression 41.2 Lasso: Survival of the Bigger 61.3 Thresholding the Sample Covariance Matrix 91.4 Sparse PCA and Regression 101.5 Graphical Models: Nodewise Regression 121.6 Cholesky Decomposition and Regression 131.7 The Bigger Picture: Latent Factor Models 141.8 Further Reading 162 Data, Sparsity and Regularization 212.1 Data Matrix: Examples 222.2 Shrinking the Sample Covariance Matrix 262.3 Distribution of the Sample Eigenvalues 292.4 Regularizing Covariances Like a Mean 302.5 The Lasso Regression 322.6 Lasso, Variable Selection and Prediction 362.7 Lasso, Degrees of Freedom and BIC 372.8 Some Alternatives to the Lasso Penalty 383 Covariance Matrices 453.1 Definition and Basic Properties 463.2 The Spectral Decomposition 493.3 Structured Covariance Matrices 523.4 Functions of a Covariance Matrix 553.5 PCA: The Maximum Variance Property 593.6 Modified Cholesky Decomposition 613.7 Latent Factor Models 653.8 GLM for Covariance Matrices 713.9 GLM via the Cholesky Decomposition 733.10 The GLM for Incomplete Longitudinal Data 763.11 A Data Example: Fruit Fly Mortality Rate 813.12 Simulating Random Correlation Matrices 853.13 Bayesian Analysis of Covariance Matrices 88PART II COVARIANCE ESTIMATION: REGULARIZATION4 Regularizing the Eigenstructure 954.1 Shrinking the Eigenvalues 964.2 Regularizing The Eigenvectors 1014.3 A Duality between PCA and SVD 1034.4 Implementing Sparse PCA: A Data Example 1064.5 Sparse Singular Value Decomposition (SSVD) 1084.6 Consistency of PCA 1094.7 Principal Subspace Estimation 1134.8 Further Reading 1145 Sparse Gaussian Graphical Models 1155.1 Covariance Selection Models: Two Examples 1165.2 Regression Interpretation of Entries of ∑-1 1185.3 Penalized Likelihood and Graphical Lasso 1205.4 Penalized Quasi-Likelihood Formulation 1265.5 Penalizing the Cholesky Factor 1275.6 Consistency and Sparsistency 1305.7 Joint Graphical Models 1305.8 Further Reading 1326 Banding, Tapering and Thresholding 1356.1 Banding the Sample Covariance Matrix 1366.2 Tapering the Sample Covariance Matrix 1376.3 Thresholding the Sample Covariance Matrix 1386.4 Low-Rank Plus Sparse Covariance Matrices 1426.5 Further Reading 1437 Multivariate Regression: Accounting for Correlation 1457.1 Multivariate Regression & LS Estimators 1467.2 Reduced Rank Regressions (RRR) 1487.3 Regularized Estimation of B 1507.4 Joint Regularization of (B;) 1527.5 Implementing MRCE: Data Examples 1557.5.1 Intraday Electricity Prices 1557.5.2 Predicting Asset Returns 1587.6 Further Reading 161