Hypothesis Testing and Model Selection in the Social Sciences

David L. Weakliem, PhD, is Professor of Sociology at the University of Connecticut. He has been a fellow at the Center for Advanced Study in the Behavioral Sciences at Stanford University and at the Australian National University. Dr. Weakliem is Editor-in-Chief of Comparative Sociology and a past Deputy Editor of the American Sociological Review.

"Weakliem offers a principled discussion of statistical methods for model selection and demonstrates them on applied problems in the social sciences. This thoughtful work should influence both statistical theory and social science practice."--Andrew Gelman, PhD, Department of Statistics, Columbia University"One of the most difficult and complicated problems in any statistical analysis is model selection. In this comprehensive book, Weakliem provides a cogent and accessible presentation of existing thinking and methods. A 'must read' for any sociologist who is a serious applied quantitative researcher."--Christopher Winship, PhD, Diker–Tishman Professor of Sociology, Harvard University"I especially appreciate this book's careful treatment of the philosophical arguments underlying hypothesis testing and the historical approaches that have been taken to the model selection problem. The question addressed here is not 'Which statistical test or approach should I use?' but rather, 'How can model specification, estimation, and statistical estimation advance what is known about a particular problem?' The book makes a convincing case for the utility of both traditional and Bayesian approaches--instead of calling for a Bayesian revolution--and leads quite logically to a number of ways that conventional practice can be improved. Rich bibliographies at the end of each chapter provide sources for further reading."--Phillip K. Wood, PhD, Department of Psychological Sciences, University of Missouri -

• 1. Hypothesis Testing and Model Selection1.1. Introduction1.2. Standard Procedure of Hypothesis Testing1.3. Model Selection1.4. Purpose and Plan of the Book2. Hypothesis Testing: Criticisms and Alternatives2.1. Hypothesis Testing and Its Discontents2.2. Uses of Hypothesis Tests2.3. Criticisms of Conventional Hypothesis Testing2.4. Implications of the Criticisms2.5. Alternatives to Conventional Tests2.6. Examples2.7. Summary and ConclusionsRecommended Reading3. The Classical Approach3.1. Random Sampling and Classical Tests3.2. Two Approaches to Hypothesis Tests3.3. Confidence Intervals3.4. Choosing a Significance Level3.5. Comparison to Conventional Practice3.6. Implications of Choosing an α-level3.7. Other Kinds of Errors3.8. Example of Choosing an α-level3.9. Evaluation of Criticisms3.10. ConclusionsRecommended Reading4. Bayesian Hypothesis Tests4.1. Bayes's Theorem4.2. Bayesian Estimation4.3. Bayes Factors4.4. Bayesian Confidence Intervals and Bayes Factors4.5. Approaches to Bayesian Hypothesis Testing4.6. The Unit Information Prior4.7. Limits on Bayes Factors4.8. Bayes Factors for Multiple Parameters4.9. ConclusionsRecommended Reading5. The Akaike Information Criterion5.1. Information5.2. Prediction and Model Selection5.3. The AIC5.4. Consistency and Efficiency5.5. Cross-Validation and the AIC5.6. A Classical Perspective on the AIC5.7. A Bayesian Perspective on the AIC5.8. A General Class of Model Selection Criteria5.9. Summary and ConclusionsRecommended Reading6. Three-Way Decisions6.1. Substantive and Statistical Hypotheses6.2. Bayes Factors for Directional Hypotheses6.3. Bayes Factors for Three-Way Decisions6.4. Summary and ConclusionsRecommended Reading7. Model Selection7.1. Introduction7.2. Bayesian Model Selection7.3. The Value of Model Selection7.4. The Risks of Model Selection7.5. Examples of Model Selection7.6. ConclusionsRecommended Reading8. Hypothesis Tests8.1. Hypothesis Tests and the Strength of Evidence8.2. When Should Hypotheses Be Tested?8.3. The Role of Hypothesis Tests8.4. Overfitting8.5. Hypothesis Tests and the Development of Theory8.6. ConclusionsRecommended ReadingReferences