Bruce E. Hansen is the Mary Claire Aschenbrener Phipps Distinguished Chair of Economics at the University of Wisconsin–Madison and one of the most cited econometricians in the world.

• PrefaceAcknowledgmentsMathematical PreparationNotation1 Basic Probability Theory1.1 Introduction1.2 Outcomes and Events1.3 Probability Function1.4 Properties of the Probability Function1.5 Equally Likely Outcomes1.6 Joint Events1.7 Conditional Probability1.8 Independence1.9 Law of Total Probability1.10 Bayes Rule1.11 Permutations and Combinations1.12 Sampling with and without Replacement1.13 Poker Hands1.14 Sigma Fields*1.15 Technical Proofs*1.16 Exercises2 Random Variables2.1 Introduction2.2 Random Variables2.3 Discrete Random Variables2.4 Transformations2.5 Expectation2.6 Finiteness of Expectations2.7 Distribution Function2.8 Continuous Random Variables2.9 Quantiles2.10 Density Functions2.11 Transformations of Continuous Random Variables2.12 Non-Monotonic Transformations2.13 Expectation of Continuous Random Variables2.14 Finiteness of Expectations2.15 Unifying Notation2.16 Mean and Variance2.17 Moments2.18 Jensen’s Inequality2.19 Applications of Jensen’s Inequality*2.20 Symmetric Distributions2.21 Truncated Distributions2.22 Censored Distributions2.23 Moment Generating Function2.24 Cumulants2.25 Characteristic Function2.26 Expectation: Mathematical Details*2.27 Exercises3 Parametric Distributions3.1 Introduction3.2 Bernoulli Distribution3.3 Rademacher Distribution3.4 Binomial Distribution3.5 Multinomial Distribution3.6 Poisson Distribution3.7 Negative Binomial Distribution3.8 Uniform Distribution3.9 Exponential Distribution3.10 Double Exponential Distribution3.11 Generalized Exponential Distribution3.12 Normal Distribution3.13 Cauchy Distribution3.14 Student t Distribution3.15 Logistic Distribution3.16 Chi-Square Distribution3.17 Gamma Distribution3.18 F Distribution3.19 Non-Central Chi-Square3.20 Beta Distribution3.21 Pareto Distribution3.22 Lognormal Distribution3.23 Weibull Distribution3.24 Extreme Value Distribution3.25 Mixtures of Normals3.26 Technical Proofs*3.27 Exercises4 Multivariate Distributions4.1 Introduction4.2 Bivariate Random Variables4.3 Bivariate Distribution Functions4.4 Probability Mass Function4.5 Probability Density Function4.6 Marginal Distribution4.7 Bivariate Expectation4.8 Conditional Distribution for Discrete X4.9 Conditional Distribution for Continuous X4.10 Visualizing Conditional Densities4.11 Independence4.12 Covariance and Correlation4.13 Cauchy-Schwarz Inequality4.14 Conditional Expectation4.15 Law of Iterated Expectations4.16 Conditional Variance4.17 H ölder’s and Minkowski’s Inequalities*4.18 Vector Notation4.19 Triangle Inequalities*4.20 Multivariate Random Vectors4.21 Pairs of Multivariate Vectors4.22 Multivariate Transformations4.23 Convolutions4.24 Hierarchical Distributions4.25 Existence and Uniqueness of the Conditional Expectation*4.26 Identification4.27 Exercises5 Normal and Related Distributions5.1 Introduction5.2 Univariate Normal5.3 Moments of the Normal Distribution5.4 Normal Cumulants5.5 Normal Quantiles5.6 Truncated and Censored Normal Distributions5.7 Multivariate Normal5.8 Properties of the Multivariate Normal5.9 Chi-Square, t,F , and Cauchy Distributions5.10 Hermite Polynomials*5.11 Technical Proofs*5.12 Exercises6 Sampling6.1 Introduction6.2 Samples6.3 Empirical Illustration6.4 Statistics, Parameters, and Estimators6.5 Sample Mean6.6 Expected Value of Transformations6.7 Functions of Parameters6.8 Sampling Distribution6.9 Estimation Bias6.10 Estimation Variance6.11 Mean Squared Error6.12 Best Unbiased Estimator6.13 Estimation of Variance6.14 Standard Error6.15 Multivariate Means6.16 Order Statistics∗6.17 Higher Moments of Sample Mean*6.18 Normal Sampling Model6.19 Normal Residuals6.20 Normal Variance Estimation6.21 Studentized Ratio6.22 Multivariate Normal Sampling6.23 Exercises7 Law of Large Numbers7.1 Introduction7.2 Asymptotic Limits7.3 Convergence in Probability7.4 Chebyshev’s Inequality7.5 Weak Law of Large Numbers7.6 Counterexamples7.7 Examples7.8 Illustrating Chebyshev’s Inequality7.9 Vector-Valued Moments7.10 Continuous Mapping Theorem7.11 Examples7.12 Uniformity Over Distributions*7.13 Almost Sure Convergence and the Strong Law*7.14 Technical Proofs*7.15 Exercises8 Central Limit Theory8.1 Introduction8.2 Convergence in Distribution8.3 Sample Mean8.4 A Moment Investigation8.5 Convergence of the Moment Generating Function8.6 Central Limit Theorem8.7 Applying the Central Limit Theorem8.8 Multivariate Central Limit Theorem8.9 Delta Method8.10 Examples8.11 Asymptotic Distribution for Plug-In Estimator8.12 Covariance Matrix Estimation8.13 t -Ratios8.14 Stochastic Order Symbols8.15 Technical Proofs*8.16 Exercises9 Advanced Asymptotic Theory*9.1 Introduction9.2 Heterogeneous Central Limit Theory9.3 Multivariate Heterogeneous Central Limit Theory9.4 Uniform Central Limit Theory9.5 Uniform Integrability9.6 Uniform Stochastic Bounds9.7 Convergence of Moments9.8 Edgeworth Expansion for the Sample Mean9.9 Edgeworth Expansion for Smooth Function Model9.10 Cornish-Fisher Expansions9.11 Technical Proofs*10 Maximum Likelihood Estimation10.1 Introduction10.2 Parametric Model10.3 Likelihood10.4 Likelihood Analog Principle10.5 Invariance Property10.6 Examples10.7 Score, Hessian, and Information10.8 Examples10.9 Cram ér-Rao Lower Bound10.10 Examples10.11 Cram ér-Rao Bound for Functions of Parameters10.12 Consistent Estimation10.13 Asymptotic Normality10.14 Asymptotic Cram ér-Rao Efficiency10.15 Variance Estimation10.16 Kullback-Leibler Divergence10.17 Approximating Models10.18 Distribution of the MLE under Misspecification10.19 Variance Estimation under Misspecification10.20 Technical Proofs*10.21 Exercises11 Method of Moments11.1 Introduction11.2 Multivariate Means11.3 Moments11.4 Smooth Functions11.5 Central Moments11.6 Best Unbiased Estimation11.7 Parametric Models11.8 Examples of Parametric Models11.9 Moment Equations11.10 Asymptotic Distribution for Moment Equations11.11 Example: Euler Equation11.12 Empirical Distribution Function11.13 Sample Quantiles11.14 Robust Variance Estimation11.15 Technical Proofs*11.16 Exercises12 Numerical Optimization12.1 Introduction12.2 Numerical Function Evaluation and Differentiation12.3 Root Finding12.4 Minimization in One Dimension12.5 Failures of Minimization12.6 Minimization in Multiple Dimensions12.7 Constrained Optimization12.8 Nested Minimization12.9 Tips and Tricks12.10 Exercises13 Hypothesis Testing13.1 Introduction13.2 Hypotheses13.3 Acceptance and Rejection13.4 Type I and Type II Errors13.5 One-Sided Tests13.6 Two-Sided Tests13.7 What Does “Accept ℍ0” Mean about ℍ0?13.8 t Test with Normal Sampling13.9 Asymptotic t Test13.10 Likelihood Ratio Test for Simple Hypotheses13.11 Neyman-Pearson Lemma13.12 Likelihood Ratio Test against Composite Alternatives13.13 Likelihood Ratio and t Tests13.14 Statistical Significance13.15 p-Value13.16 Composite Null Hypothesis13.17 Asymptotic Uniformity13.18 Summary13.19 Exercises14 Confidence Intervals14.1 Introduction14.2 Definitions14.3 Simple Confidence Intervals14.4 Confidence Intervals for the Sample Mean under Normal Sampling14.5 Confidence Intervals for the Sample Mean under Non-Normal Sampling14.6 Confidence Intervals for Estimated Parameters14.7 Confidence Interval for the Variance14.8 Confidence Intervals by Test Inversion14.9 Use of Confidence Intervals14.10 Uniform Confidence Intervals14.11 Exercises15 Shrinkage Estimation15.1 Introduction15.2 Mean Squared Error15.3 Shrinkage15.4 James-Stein Shrinkage Estimator15.5 Numerical Calculation15.6 Interpretation of the Stein Effect15.7 Positive-Part Estimator15.8 Summary15.9 Technical Proofs*15.10 Exercises16 Bayesian Methods16.1 Introduction16.2 Bayesian Probability Model16.3 Posterior Density16.4 Bayesian Estimation16.5 Parametric Priors16.6 Normal-Gamma Distribution16.7 Conjugate Prior16.8 Bernoulli Sampling16.9 Normal Sampling16.10 Credible Sets16.11 Bayesian Hypothesis Testing16.12 Sampling Properties in the Normal Model16.13 Asymptotic Distribution16.14 Technical Proofs*16.15 Exercises17 Nonparametric Density Estimation17.1 Introduction17.2 Histogram Density Estimation17.3 Kernel Density Estimator17.4 Bias of Density Estimator17.5 Variance of Density Estimator17.6 Variance Estimation and Standard Errors17.7 Integrated Mean Squared Error of Density Estimator17.8 Optimal Kernel17.9 Reference Bandwidth17.10 Sheather-Jones Bandwidth*17.11 Recommendations for Bandwidth Selection17.12 Practical Issues in Density Estimation17.13 Computation17.14 Asymptotic Distribution17.15 Undersmoothing17.16 Technical Proofs*17.17 Exercises18 Empirical Process Theory18.1 Introduction18.2 Framework18.3 Glivenko-Cantelli Theorem18.4 Packing, Covering, and Bracketing Numbers18.5 Uniform Law of Large Numbers18.6 Functional Central Limit Theory18.7 Conditions for Asymptotic Equicontinuity18.8 Donsker’s Theorem18.9 Technical Proofs*18.10 ExercisesAppendix: Mathematics ReferenceA.1 LimitsA.2 SeriesA.3 FactorialsA.4 ExponentialsA.5 LogarithmsA.6 DifferentiationA.7 Mean Value TheoremA.8 IntegrationA.9 Gaussian IntegralA.10 Gamma FunctionA.11 Matrix AlgebraReferencesIndex

Bruce E. Hansen is the Mary Claire Aschenbrener Phipps Distinguished Chair of Economics at the University of Wisconsin–Madison and one of the most cited econometricians in the world.

• PrefaceAcknowledgmentsMathematical PreparationNotation1 Basic Probability Theory1.1 Introduction1.2 Outcomes and Events1.3 Probability Function1.4 Properties of the Probability Function1.5 Equally Likely Outcomes1.6 Joint Events1.7 Conditional Probability1.8 Independence1.9 Law of Total Probability1.10 Bayes Rule1.11 Permutations and Combinations1.12 Sampling with and without Replacement1.13 Poker Hands1.14 Sigma Fields*1.15 Technical Proofs*1.16 Exercises2 Random Variables2.1 Introduction2.2 Random Variables2.3 Discrete Random Variables2.4 Transformations2.5 Expectation2.6 Finiteness of Expectations2.7 Distribution Function2.8 Continuous Random Variables2.9 Quantiles2.10 Density Functions2.11 Transformations of Continuous Random Variables2.12 Non-Monotonic Transformations2.13 Expectation of Continuous Random Variables2.14 Finiteness of Expectations2.15 Unifying Notation2.16 Mean and Variance2.17 Moments2.18 Jensen’s Inequality2.19 Applications of Jensen’s Inequality*2.20 Symmetric Distributions2.21 Truncated Distributions2.22 Censored Distributions2.23 Moment Generating Function2.24 Cumulants2.25 Characteristic Function2.26 Expectation: Mathematical Details*2.27 Exercises3 Parametric Distributions3.1 Introduction3.2 Bernoulli Distribution3.3 Rademacher Distribution3.4 Binomial Distribution3.5 Multinomial Distribution3.6 Poisson Distribution3.7 Negative Binomial Distribution3.8 Uniform Distribution3.9 Exponential Distribution3.10 Double Exponential Distribution3.11 Generalized Exponential Distribution3.12 Normal Distribution3.13 Cauchy Distribution3.14 Student t Distribution3.15 Logistic Distribution3.16 Chi-Square Distribution3.17 Gamma Distribution3.18 F Distribution3.19 Non-Central Chi-Square3.20 Beta Distribution3.21 Pareto Distribution3.22 Lognormal Distribution3.23 Weibull Distribution3.24 Extreme Value Distribution3.25 Mixtures of Normals3.26 Technical Proofs*3.27 Exercises4 Multivariate Distributions4.1 Introduction4.2 Bivariate Random Variables4.3 Bivariate Distribution Functions4.4 Probability Mass Function4.5 Probability Density Function4.6 Marginal Distribution4.7 Bivariate Expectation4.8 Conditional Distribution for Discrete X4.9 Conditional Distribution for Continuous X4.10 Visualizing Conditional Densities4.11 Independence4.12 Covariance and Correlation4.13 Cauchy-Schwarz Inequality4.14 Conditional Expectation4.15 Law of Iterated Expectations4.16 Conditional Variance4.17 H ölder’s and Minkowski’s Inequalities*4.18 Vector Notation4.19 Triangle Inequalities*4.20 Multivariate Random Vectors4.21 Pairs of Multivariate Vectors4.22 Multivariate Transformations4.23 Convolutions4.24 Hierarchical Distributions4.25 Existence and Uniqueness of the Conditional Expectation*4.26 Identification4.27 Exercises5 Normal and Related Distributions5.1 Introduction5.2 Univariate Normal5.3 Moments of the Normal Distribution5.4 Normal Cumulants5.5 Normal Quantiles5.6 Truncated and Censored Normal Distributions5.7 Multivariate Normal5.8 Properties of the Multivariate Normal5.9 Chi-Square, t,F , and Cauchy Distributions5.10 Hermite Polynomials*5.11 Technical Proofs*5.12 Exercises6 Sampling6.1 Introduction6.2 Samples6.3 Empirical Illustration6.4 Statistics, Parameters, and Estimators6.5 Sample Mean6.6 Expected Value of Transformations6.7 Functions of Parameters6.8 Sampling Distribution6.9 Estimation Bias6.10 Estimation Variance6.11 Mean Squared Error6.12 Best Unbiased Estimator6.13 Estimation of Variance6.14 Standard Error6.15 Multivariate Means6.16 Order Statistics∗6.17 Higher Moments of Sample Mean*6.18 Normal Sampling Model6.19 Normal Residuals6.20 Normal Variance Estimation6.21 Studentized Ratio6.22 Multivariate Normal Sampling6.23 Exercises7 Law of Large Numbers7.1 Introduction7.2 Asymptotic Limits7.3 Convergence in Probability7.4 Chebyshev’s Inequality7.5 Weak Law of Large Numbers7.6 Counterexamples7.7 Examples7.8 Illustrating Chebyshev’s Inequality7.9 Vector-Valued Moments7.10 Continuous Mapping Theorem7.11 Examples7.12 Uniformity Over Distributions*7.13 Almost Sure Convergence and the Strong Law*7.14 Technical Proofs*7.15 Exercises8 Central Limit Theory8.1 Introduction8.2 Convergence in Distribution8.3 Sample Mean8.4 A Moment Investigation8.5 Convergence of the Moment Generating Function8.6 Central Limit Theorem8.7 Applying the Central Limit Theorem8.8 Multivariate Central Limit Theorem8.9 Delta Method8.10 Examples8.11 Asymptotic Distribution for Plug-In Estimator8.12 Covariance Matrix Estimation8.13 t -Ratios8.14 Stochastic Order Symbols8.15 Technical Proofs*8.16 Exercises9 Advanced Asymptotic Theory*9.1 Introduction9.2 Heterogeneous Central Limit Theory9.3 Multivariate Heterogeneous Central Limit Theory9.4 Uniform Central Limit Theory9.5 Uniform Integrability9.6 Uniform Stochastic Bounds9.7 Convergence of Moments9.8 Edgeworth Expansion for the Sample Mean9.9 Edgeworth Expansion for Smooth Function Model9.10 Cornish-Fisher Expansions9.11 Technical Proofs*10 Maximum Likelihood Estimation10.1 Introduction10.2 Parametric Model10.3 Likelihood10.4 Likelihood Analog Principle10.5 Invariance Property10.6 Examples10.7 Score, Hessian, and Information10.8 Examples10.9 Cram ér-Rao Lower Bound10.10 Examples10.11 Cram ér-Rao Bound for Functions of Parameters10.12 Consistent Estimation10.13 Asymptotic Normality10.14 Asymptotic Cram ér-Rao Efficiency10.15 Variance Estimation10.16 Kullback-Leibler Divergence10.17 Approximating Models10.18 Distribution of the MLE under Misspecification10.19 Variance Estimation under Misspecification10.20 Technical Proofs*10.21 Exercises11 Method of Moments11.1 Introduction11.2 Multivariate Means11.3 Moments11.4 Smooth Functions11.5 Central Moments11.6 Best Unbiased Estimation11.7 Parametric Models11.8 Examples of Parametric Models11.9 Moment Equations11.10 Asymptotic Distribution for Moment Equations11.11 Example: Euler Equation11.12 Empirical Distribution Function11.13 Sample Quantiles11.14 Robust Variance Estimation11.15 Technical Proofs*11.16 Exercises12 Numerical Optimization12.1 Introduction12.2 Numerical Function Evaluation and Differentiation12.3 Root Finding12.4 Minimization in One Dimension12.5 Failures of Minimization12.6 Minimization in Multiple Dimensions12.7 Constrained Optimization12.8 Nested Minimization12.9 Tips and Tricks12.10 Exercises13 Hypothesis Testing13.1 Introduction13.2 Hypotheses13.3 Acceptance and Rejection13.4 Type I and Type II Errors13.5 One-Sided Tests13.6 Two-Sided Tests13.7 What Does “Accept ℍ0” Mean about ℍ0?13.8 t Test with Normal Sampling13.9 Asymptotic t Test13.10 Likelihood Ratio Test for Simple Hypotheses13.11 Neyman-Pearson Lemma13.12 Likelihood Ratio Test against Composite Alternatives13.13 Likelihood Ratio and t Tests13.14 Statistical Significance13.15 p-Value13.16 Composite Null Hypothesis13.17 Asymptotic Uniformity13.18 Summary13.19 Exercises14 Confidence Intervals14.1 Introduction14.2 Definitions14.3 Simple Confidence Intervals14.4 Confidence Intervals for the Sample Mean under Normal Sampling14.5 Confidence Intervals for the Sample Mean under Non-Normal Sampling14.6 Confidence Intervals for Estimated Parameters14.7 Confidence Interval for the Variance14.8 Confidence Intervals by Test Inversion14.9 Use of Confidence Intervals14.10 Uniform Confidence Intervals14.11 Exercises15 Shrinkage Estimation15.1 Introduction15.2 Mean Squared Error15.3 Shrinkage15.4 James-Stein Shrinkage Estimator15.5 Numerical Calculation15.6 Interpretation of the Stein Effect15.7 Positive-Part Estimator15.8 Summary15.9 Technical Proofs*15.10 Exercises16 Bayesian Methods16.1 Introduction16.2 Bayesian Probability Model16.3 Posterior Density16.4 Bayesian Estimation16.5 Parametric Priors16.6 Normal-Gamma Distribution16.7 Conjugate Prior16.8 Bernoulli Sampling16.9 Normal Sampling16.10 Credible Sets16.11 Bayesian Hypothesis Testing16.12 Sampling Properties in the Normal Model16.13 Asymptotic Distribution16.14 Technical Proofs*16.15 Exercises17 Nonparametric Density Estimation17.1 Introduction17.2 Histogram Density Estimation17.3 Kernel Density Estimator17.4 Bias of Density Estimator17.5 Variance of Density Estimator17.6 Variance Estimation and Standard Errors17.7 Integrated Mean Squared Error of Density Estimator17.8 Optimal Kernel17.9 Reference Bandwidth17.10 Sheather-Jones Bandwidth*17.11 Recommendations for Bandwidth Selection17.12 Practical Issues in Density Estimation17.13 Computation17.14 Asymptotic Distribution17.15 Undersmoothing17.16 Technical Proofs*17.17 Exercises18 Empirical Process Theory18.1 Introduction18.2 Framework18.3 Glivenko-Cantelli Theorem18.4 Packing, Covering, and Bracketing Numbers18.5 Uniform Law of Large Numbers18.6 Functional Central Limit Theory18.7 Conditions for Asymptotic Equicontinuity18.8 Donsker’s Theorem18.9 Technical Proofs*18.10 ExercisesAppendix: Mathematics ReferenceA.1 LimitsA.2 SeriesA.3 FactorialsA.4 ExponentialsA.5 LogarithmsA.6 DifferentiationA.7 Mean Value TheoremA.8 IntegrationA.9 Gaussian IntegralA.10 Gamma FunctionA.11 Matrix AlgebraReferencesIndex

Bruce E. Hansen is the Mary Claire Aschenbrener Phipps Distinguished Chair of Economics at the University of Wisconsin–Madison and one of the most cited econometricians in the world.

• PrefaceAcknowledgmentsMathematical PreparationNotation1 Basic Probability Theory1.1 Introduction1.2 Outcomes and Events1.3 Probability Function1.4 Properties of the Probability Function1.5 Equally Likely Outcomes1.6 Joint Events1.7 Conditional Probability1.8 Independence1.9 Law of Total Probability1.10 Bayes Rule1.11 Permutations and Combinations1.12 Sampling with and without Replacement1.13 Poker Hands1.14 Sigma Fields*1.15 Technical Proofs*1.16 Exercises2 Random Variables2.1 Introduction2.2 Random Variables2.3 Discrete Random Variables2.4 Transformations2.5 Expectation2.6 Finiteness of Expectations2.7 Distribution Function2.8 Continuous Random Variables2.9 Quantiles2.10 Density Functions2.11 Transformations of Continuous Random Variables2.12 Non-Monotonic Transformations2.13 Expectation of Continuous Random Variables2.14 Finiteness of Expectations2.15 Unifying Notation2.16 Mean and Variance2.17 Moments2.18 Jensen’s Inequality2.19 Applications of Jensen’s Inequality*2.20 Symmetric Distributions2.21 Truncated Distributions2.22 Censored Distributions2.23 Moment Generating Function2.24 Cumulants2.25 Characteristic Function2.26 Expectation: Mathematical Details*2.27 Exercises3 Parametric Distributions3.1 Introduction3.2 Bernoulli Distribution3.3 Rademacher Distribution3.4 Binomial Distribution3.5 Multinomial Distribution3.6 Poisson Distribution3.7 Negative Binomial Distribution3.8 Uniform Distribution3.9 Exponential Distribution3.10 Double Exponential Distribution3.11 Generalized Exponential Distribution3.12 Normal Distribution3.13 Cauchy Distribution3.14 Student t Distribution3.15 Logistic Distribution3.16 Chi-Square Distribution3.17 Gamma Distribution3.18 F Distribution3.19 Non-Central Chi-Square3.20 Beta Distribution3.21 Pareto Distribution3.22 Lognormal Distribution3.23 Weibull Distribution3.24 Extreme Value Distribution3.25 Mixtures of Normals3.26 Technical Proofs*3.27 Exercises4 Multivariate Distributions4.1 Introduction4.2 Bivariate Random Variables4.3 Bivariate Distribution Functions4.4 Probability Mass Function4.5 Probability Density Function4.6 Marginal Distribution4.7 Bivariate Expectation4.8 Conditional Distribution for Discrete X4.9 Conditional Distribution for Continuous X4.10 Visualizing Conditional Densities4.11 Independence4.12 Covariance and Correlation4.13 Cauchy-Schwarz Inequality4.14 Conditional Expectation4.15 Law of Iterated Expectations4.16 Conditional Variance4.17 H ölder’s and Minkowski’s Inequalities*4.18 Vector Notation4.19 Triangle Inequalities*4.20 Multivariate Random Vectors4.21 Pairs of Multivariate Vectors4.22 Multivariate Transformations4.23 Convolutions4.24 Hierarchical Distributions4.25 Existence and Uniqueness of the Conditional Expectation*4.26 Identification4.27 Exercises5 Normal and Related Distributions5.1 Introduction5.2 Univariate Normal5.3 Moments of the Normal Distribution5.4 Normal Cumulants5.5 Normal Quantiles5.6 Truncated and Censored Normal Distributions5.7 Multivariate Normal5.8 Properties of the Multivariate Normal5.9 Chi-Square, t,F , and Cauchy Distributions5.10 Hermite Polynomials*5.11 Technical Proofs*5.12 Exercises6 Sampling6.1 Introduction6.2 Samples6.3 Empirical Illustration6.4 Statistics, Parameters, and Estimators6.5 Sample Mean6.6 Expected Value of Transformations6.7 Functions of Parameters6.8 Sampling Distribution6.9 Estimation Bias6.10 Estimation Variance6.11 Mean Squared Error6.12 Best Unbiased Estimator6.13 Estimation of Variance6.14 Standard Error6.15 Multivariate Means6.16 Order Statistics∗6.17 Higher Moments of Sample Mean*6.18 Normal Sampling Model6.19 Normal Residuals6.20 Normal Variance Estimation6.21 Studentized Ratio6.22 Multivariate Normal Sampling6.23 Exercises7 Law of Large Numbers7.1 Introduction7.2 Asymptotic Limits7.3 Convergence in Probability7.4 Chebyshev’s Inequality7.5 Weak Law of Large Numbers7.6 Counterexamples7.7 Examples7.8 Illustrating Chebyshev’s Inequality7.9 Vector-Valued Moments7.10 Continuous Mapping Theorem7.11 Examples7.12 Uniformity Over Distributions*7.13 Almost Sure Convergence and the Strong Law*7.14 Technical Proofs*7.15 Exercises8 Central Limit Theory8.1 Introduction8.2 Convergence in Distribution8.3 Sample Mean8.4 A Moment Investigation8.5 Convergence of the Moment Generating Function8.6 Central Limit Theorem8.7 Applying the Central Limit Theorem8.8 Multivariate Central Limit Theorem8.9 Delta Method8.10 Examples8.11 Asymptotic Distribution for Plug-In Estimator8.12 Covariance Matrix Estimation8.13 t -Ratios8.14 Stochastic Order Symbols8.15 Technical Proofs*8.16 Exercises9 Advanced Asymptotic Theory*9.1 Introduction9.2 Heterogeneous Central Limit Theory9.3 Multivariate Heterogeneous Central Limit Theory9.4 Uniform Central Limit Theory9.5 Uniform Integrability9.6 Uniform Stochastic Bounds9.7 Convergence of Moments9.8 Edgeworth Expansion for the Sample Mean9.9 Edgeworth Expansion for Smooth Function Model9.10 Cornish-Fisher Expansions9.11 Technical Proofs*10 Maximum Likelihood Estimation10.1 Introduction10.2 Parametric Model10.3 Likelihood10.4 Likelihood Analog Principle10.5 Invariance Property10.6 Examples10.7 Score, Hessian, and Information10.8 Examples10.9 Cram ér-Rao Lower Bound10.10 Examples10.11 Cram ér-Rao Bound for Functions of Parameters10.12 Consistent Estimation10.13 Asymptotic Normality10.14 Asymptotic Cram ér-Rao Efficiency10.15 Variance Estimation10.16 Kullback-Leibler Divergence10.17 Approximating Models10.18 Distribution of the MLE under Misspecification10.19 Variance Estimation under Misspecification10.20 Technical Proofs*10.21 Exercises11 Method of Moments11.1 Introduction11.2 Multivariate Means11.3 Moments11.4 Smooth Functions11.5 Central Moments11.6 Best Unbiased Estimation11.7 Parametric Models11.8 Examples of Parametric Models11.9 Moment Equations11.10 Asymptotic Distribution for Moment Equations11.11 Example: Euler Equation11.12 Empirical Distribution Function11.13 Sample Quantiles11.14 Robust Variance Estimation11.15 Technical Proofs*11.16 Exercises12 Numerical Optimization12.1 Introduction12.2 Numerical Function Evaluation and Differentiation12.3 Root Finding12.4 Minimization in One Dimension12.5 Failures of Minimization12.6 Minimization in Multiple Dimensions12.7 Constrained Optimization12.8 Nested Minimization12.9 Tips and Tricks12.10 Exercises13 Hypothesis Testing13.1 Introduction13.2 Hypotheses13.3 Acceptance and Rejection13.4 Type I and Type II Errors13.5 One-Sided Tests13.6 Two-Sided Tests13.7 What Does “Accept ℍ0” Mean about ℍ0?13.8 t Test with Normal Sampling13.9 Asymptotic t Test13.10 Likelihood Ratio Test for Simple Hypotheses13.11 Neyman-Pearson Lemma13.12 Likelihood Ratio Test against Composite Alternatives13.13 Likelihood Ratio and t Tests13.14 Statistical Significance13.15 p-Value13.16 Composite Null Hypothesis13.17 Asymptotic Uniformity13.18 Summary13.19 Exercises14 Confidence Intervals14.1 Introduction14.2 Definitions14.3 Simple Confidence Intervals14.4 Confidence Intervals for the Sample Mean under Normal Sampling14.5 Confidence Intervals for the Sample Mean under Non-Normal Sampling14.6 Confidence Intervals for Estimated Parameters14.7 Confidence Interval for the Variance14.8 Confidence Intervals by Test Inversion14.9 Use of Confidence Intervals14.10 Uniform Confidence Intervals14.11 Exercises15 Shrinkage Estimation15.1 Introduction15.2 Mean Squared Error15.3 Shrinkage15.4 James-Stein Shrinkage Estimator15.5 Numerical Calculation15.6 Interpretation of the Stein Effect15.7 Positive-Part Estimator15.8 Summary15.9 Technical Proofs*15.10 Exercises16 Bayesian Methods16.1 Introduction16.2 Bayesian Probability Model16.3 Posterior Density16.4 Bayesian Estimation16.5 Parametric Priors16.6 Normal-Gamma Distribution16.7 Conjugate Prior16.8 Bernoulli Sampling16.9 Normal Sampling16.10 Credible Sets16.11 Bayesian Hypothesis Testing16.12 Sampling Properties in the Normal Model16.13 Asymptotic Distribution16.14 Technical Proofs*16.15 Exercises17 Nonparametric Density Estimation17.1 Introduction17.2 Histogram Density Estimation17.3 Kernel Density Estimator17.4 Bias of Density Estimator17.5 Variance of Density Estimator17.6 Variance Estimation and Standard Errors17.7 Integrated Mean Squared Error of Density Estimator17.8 Optimal Kernel17.9 Reference Bandwidth17.10 Sheather-Jones Bandwidth*17.11 Recommendations for Bandwidth Selection17.12 Practical Issues in Density Estimation17.13 Computation17.14 Asymptotic Distribution17.15 Undersmoothing17.16 Technical Proofs*17.17 Exercises18 Empirical Process Theory18.1 Introduction18.2 Framework18.3 Glivenko-Cantelli Theorem18.4 Packing, Covering, and Bracketing Numbers18.5 Uniform Law of Large Numbers18.6 Functional Central Limit Theory18.7 Conditions for Asymptotic Equicontinuity18.8 Donsker’s Theorem18.9 Technical Proofs*18.10 ExercisesAppendix: Mathematics ReferenceA.1 LimitsA.2 SeriesA.3 FactorialsA.4 ExponentialsA.5 LogarithmsA.6 DifferentiationA.7 Mean Value TheoremA.8 IntegrationA.9 Gaussian IntegralA.10 Gamma FunctionA.11 Matrix AlgebraReferencesIndex