Maximum Likelihood Estimation and Inference

Russell B. Millar is the author of Maximum Likelihood Estimation and Inference: With Examples in R, SAS and ADMB, published by Wiley.

“This book is well-presented and would suit applied scientists, researchers, graduate students and particularly anyone who uses likelihood and such methods to their studies and applications.”  (ISR, 2012)

• Preface xiii Part I PRELIMINARIES 11 A taste of likelihood 31.1 Introduction 31.2 Motivating example 41.3 Using SAS, R and ADMB 91.4 Implementation of the motivating example 111.5 Exercises 172 Essential concepts and iid examples 182.1 Introduction 182.2 Some necessary notation 192.3 Interpretation of likelihood 232.4 IID examples 252.5 Exercises 33Part II PRAGMATICS 373 Hypothesis tests and confidence intervals or regions 393.1 Introduction 393.2 Approximate normality of MLEs 403.3 Wald tests, confidence intervals and regions 433.4 Likelihood ratio tests, confidence intervals and regions 493.5 Likelihood ratio examples 543.6 Profile likelihood 573.7 Exercises 594 What you really need to know 644.1 Introduction 644.2 Inference about g(θ) 654.3 Wald statistics – quick and dirty? 754.4 Model selection 794.5 Bootstrapping 814.6 Prediction 914.7 Things that can mess you up 954.8 Exercises 985 Maximizing the likelihood 1015.1 Introduction 1015.2 The Newton-Raphson algorithm 1035.3 The EM (Expectation–Maximization) algorithm 1045.4 Multi-stage maximization 1135.5 Exercises 1186 Some widely used applications of maximum likelihood 1216.1 Introduction 1216.2 Box-Cox transformations 1226.3 Models for survival-time data 1256.4 Mark–recapture models 1346.5 Exercises 1417 Generalized linear models and extensions 1437.1 Introduction 1437.2 Specification of a GLM 1447.3 Likelihood calculations 1487.4 Model evaluation 1497.5 Case study 1: Logistic regression and inverse prediction in R 1547.6 Beyond binomial and Poisson models 1617.7 Case study 2: Multiplicative vs additive models of over-dispersed counts in SAS 1677.8 Exercises 1738 Quasi-likelihood and generalized estimating equations 1758.1 Introduction 1758.2 Wedderburn’s quasi-likelihood 1778.3 Generalized estimating equations 1818.4 Exercises 1879 ML inference in the presence of incidental parameters 1889.1 Introduction 1889.2 Conditional likelihood 1929.3 Integrated likelihood 1989.3.1 Justification 1999.3.2 Uses of integrated likelihood 2009.4 Exercises 20110 Latent variable models 20210.1 Introduction 20210.2 Developing the likelihood 20310.3 Software 20410.4 One-way linear random-effects model 21010.5 Nonlinear mixed-effects model 21710.6 Generalized linear mixed-effects model 22110.7 State-space model for count data 22710.8 ADMB template files 22810.9 Exercises 232Part III THEORETICAL FOUNDATIONS 23311 Cramer-Rao inequality and Fisher information 23511.1 Introduction 23511.2 The Cramer-Rao inequality for θ  RI 23611.3 Cramer-Rao inequality for functions of θ 23911.4 Alternative formulae for I (θ) 24111.5 The iid data case 24311.6 The multi-dimensional case, θ  RI s 24311.7 Examples of Fisher information calculation 24711.8 Exercises 25312 Asymptotic theory and approximate normality 25612.1 Introduction 25612.2 Consistency and asymptotic normality 25712.3 Approximate normality 27112.4 Wald tests and confidence regions 27612.5 Likelihood ratio test statistic 28012.6 Rao-score test statistic 28112.7 Exercises 28313 Tools of the trade 28613.1 Introduction 28613.2 Equivalence of tests and confidence intervals 28613.3 Transformation of variables 28713.4 Mean and variance conditional identities 28813.5 Relevant inequalities 28913.6 Asymptotic probability theory 29113.7 Exercises 29714 Fundamental paradigms and principles of inference 29914.1 Introduction 29914.2 Sufficiency principle 30014.3 Conditionality principle 30414.4 The likelihood principle 30614.5 Statistical significance versus statistical evidence 30914.6 Exercises 31115 Miscellanea 31315.1 Notation 31315.2 Acronyms 31515.3 Do you think like a frequentist or a Bayesian? 31515.4 Some useful distributions 31615.5 Software extras 32115.6 Automatic differentiation 323Appendix: Partial solutions to selected exercises 325Bibliography 337Index 345