Linear Model Theory

"This text successfully offers a unified context for the theory of univariate, multivariate, and mixed modeling settings and may be useful supplemental text for individuals interested in multivariate modeling." (Journal of the American Statistician, December 2008) "I believe that this text provides an important contribution to the long-memory time series literature. I feel that it largely achieves its aims and could be useful for those instructors wishing to teach a semester-long special topics course ... .I strongly recommend this book to anyone interested in long-memory time series. Both researchers and beginners alike will find this text extremely useful." (Journal of the American Statistician, December 2008) "The book will certainly be useful for Ph.D. students and researchers in biostatistics who want to learn a little bit of theory of linear models." (Mathematical Reviews, 2007) "...stands out from the others...will certainly have its enthusiastic supporters." (Biometrics, March 2007) "...an excellent book for graduate students and professional researchers." (MAA Reviews, February 2007) "The focus of this book is on linear models with correlated observations and Gaussian errors." (Zentralblatt MATH, April 2007)

KEITH E. MULLER, PhD, is Professor and Director of the Division of Biostatistics in the Department of Epidemiology and Health Policy Research in the College of Medicine at the University of Florida in Gainesville, as well as Professor Emeritus of Biostatistics at The University of North Carolina at Chapel Hill where the book was written. PAUL W. STEWART, PhD, is Research Associate Professor of Biostatistics at The University of North Carolina at Chapel Hill.

Preface. PART I: MODELS AND EXAMPLES. 1. Matrix Algebra for Linear Models. 1.1 Notation. 1.2 Some Operators and Special Types of Matrices. 1.3 Five Kinds of Multiplication. 1.4 The Direct Sum. 1.5 Rules of Operations. 1.6 Other Special Types of matrices. 1.7 Quadratic and Bilinear Forms. 1.8 Vector Spaces and Rank. 1.9 Finding Rank. 1.10 Determinants. 1.11 The Inverse and Generalized Inverse. 1.12 Eigenanalysis (Spectral Decomposition). 1.13 Some Factors of Symmetric Matrices. 1.14 Singular Value Decomposition. 1.15 Projections and Other Functions of a Design matrix. 1.16 Special Properties of Patterned Matrices. 1.17 Functional Optimization and Matrix Derivatives. 1.18 Statistical Notation Involving Matrices. 1.19 Statistical Formulas. 1.20 Principal Components. 1.21 Special Covariance Patterns. 2. The General Linear Univariate Model. 2.1 Introduction. 2.2 Model Concepts. 2.3 The General Linear Univariate Linear Model. 2.4 The Univariate General Linear Hypothesis. 2.5 Tests about Variances. 2.6 The Role of the Intercept. 2.7 Population Correlation and Strength of Relationship. 2.8 Statistical Estimates. 2.9 Testing the General Linear Hypothesis. 2.10 Confidence Regions for . 2.11 Sufficient Statistics for the Univariate Model. Exercises. 3. The General Linear Multivariate Model. 3.1 Motivation. 3.2 Definition of the Multivariate Model. 3.3 The Multivariate General Linear Hypothesis. 3.4 Tests About Covariance Matrices. 3.5 Population Correlation. 3.6 Statistical Estimates. 3.7 Overview of Testing Multivariate Hypotheses. 3.8 Computing MULTIREP Tests. 3.9 Computing UNIREP tests. 3.10 Confidence Regions for . 3.11 Sufficient Statistics for the Multivariate Model. 3.12 Allowing Missing Data in the Multivariate Model. Exercises. 4. Generalizations of the Multivariate Linear Model. 4.1 Motivation. 4.2 The Generalized General Linear Univariate Model: Exact and Approximate Weighted Least Squares. 4.3 Doubly Multivariate Models. 4.4 Seemingly Unrelated Regression. 4.5 Growth Curve Models (GMANOVA). 4.6 The Relationship of the GCM to the Multivariate Model. 4.7 Mixed, Hierarchical, and Related Models. 5. The Linear Mixed Model. 5.1 Motivation. 5.2 Definition of the Mixed Model. 5.3 Distribution-Free and Noniterative Estimates. 5.4 Gaussian Likelihood and Iterative Estimates. 5.5 Tests about (Means, Fixed Effects). 5.6 Tests of Covariance Parameters, (random Effects). Exercises. 6. Choosing the Form of a Linear Model for Analysis. 6.1 The Importance of Understanding Dependence. 6.2 How Many Variables per Independent Sampling Unit? 6.3 What Types of Variables Play a Role? 6.4 What Repeated Sampling Scheme Was Used? 6.5 Analysis Strategies for Multivariate Data. 6.6 cautions and Recommendations. 6.7 Review of Linear Model Notation. PART II: MULTIVARIATE DISTRIBUTION THEORY. 7. General Theory of Multivariate Distributions. 7.1 Motivation. 7.2 Notation and Concepts. 7.3 Families of Distributions. 7.4 Cumulative Distribution Function. 7.5 Probability Density Function. 7.6 Formulas for Probabilities and Moments. 7.7 Characteristic Function. 7.8 Moment Generating Function. 7.9 Cumulant generating Function. 7.10 Transforming Random Variables. 7.11 Marginal Distributions. 7.12 Independence of Random Vectors. 7.13 Conditional Distributions. 7.14 (Joint) Moments of Multivariate Distributions. 7.15 Conditional Moments of Distributions. 7.16 Special Considerations for Random Matrices. 8. Scalar, vector, and Matrix Gaussian Distributions. 8.1 Motivation. 8.2 The Scalar Gaussian Distribution. 8.3 The Vector ("Multivariate") Gaussian Distribution. 8.4 Margi