Psychometrics, Test Theory, and the Latent Factors Model

Petr Blahuš, PhD, was DrSc, Professor, Division of Methodology, in the Faculty of Physical Education and Sport, Charles University, Prague, Czech Republic. A distinguished international scholar, he published in Czech, English, and Russian. Bruce L. Brown, PhD, is Professor of Psychology at Brigham Young University and authored Multivariate Analysis for the Biobehavioral and Social Sciences (Brown, Hendrix, Hedges, and Smith, 2012, Wiley). C. Victor Bunderson, PhD, is emeritus professor of Instructional Psychology and Technology at Brigham Young University. He served on the Assessment Council of Western Governors University from 1997 through 2023. His experience includes VP Research Management at Educational Testing Service, and cofounder of several technology companies that advanced and applied concepts and methods explained in this book. Joseph A. Olsen, PhD, is Associate Research Professor, Assistant Dean, and Director of Research Support in the College of Family, Home, and Social Sciences at Brigham Young University. Dr. Olsen advises on methods and software in graduate research projects across many departments.

• About the Authors xviiForeword xviiiPreface xxAcknowledgements xxviiAbout the Companion Website xxxii1 The Psychometric Quest 11.1 Quantification in Psychometrics: Historical Origins 31.2 Variables, Scales, and Data 91.3 Statistical Methods: Location, Dispersion, and Statistical Inference 251.4 Summary 561.5 Study Questions 56References 622 Scores and Their Distributions 652.1 Quantification in Distributions: The Standard Deviation and Standard Scores 652.2 Combining and Using Standard Scores 822.3 Summary 912.4 Study Questions 93References 963 Validity, Explanation, and Constructs 973.1 Validity and Validation 993.2 Terms and Tools in External Validity 1113.3 Explanation Requires More Than Correlation 1213.4 Internal Construct Validity and Latent Factors 1323.5 Summary 1473.6 Study Questions 148References 1504 Measurement Error and Classical Test Theory 1534.1 Introduction and Overview of “Unfinished Business” from Chapter 3 1544.2 Measurement Functions that Provide Associative Links Between the Empirical World and the Informational World 1604.3 Reliability and Classical Test Theory 1714.4 Summary 1824.5 Study Questions 185References 1895 Content Validity, Item Response Theory, and Latent Variables 1915.1 Design and Initial Validation of Latent Factors and Domain Models 1925.2 Content Validity and Item Response Theory 1955.3 Latent Variable Modeling 2075.4 Summary 2185.5 Study Questions 220References 2226 The Latent Common Factor Model 2256.1 Latent Variables and the Common Factor Model 2266.2 Factor Scores 2436.3 Summary 2566.4 Study Questions 257References 2747 Analysis of Multiple Factors 2777.1 Modeling Multiple and Hierarchical Constructs 2777.2 Confirmatory and Exploratory 2877.3 Summary 3067.4 Study Questions 307References 3138 Metrology: Scientific, Applied, and Legal: An Introduction with Implications for Human and Social Sciences 3158.1 Purposes of Metrology, the Science of Measurement 3168.2 Brief History of International Metrology and Its Relevance to Human Sciences 3188.3 Scientific Metrology 3228.4 Applied and Legal Metrology 3278.5 A Manifesto for Establishing International Metrology in All of the Sciences 3298.6 Properties: Extensive and Intensive Metadomains 3328.7 Ostensive Metadomain 3348.8 Summary and Concluding Thoughts 339Acknowledgments 3428.9 Study Questions 343References 3459 Epilogue: Aspects of This Psychometrics Book 3479.1 Improbable and Unusual 3479.2 Unusual Scientific and Pedagogical Perspectives 3489.3 The Value of Design and Domain Theory 3519.4 New Approaches to Measurement, Validation, and Educational Evaluation 3539.5 Summary 357References 3581 Computational Guide 1 Classical Test Theory: A Model-Based Approach 3591 Classical Test Theory and the True Score Model 3601.1 Reliability with Multiple Test Components 3601.2 True Score Models for Multiple Items 3611.3 Latent Variable Analysis for True Score Models 3611.4 True Score Models as Factor Models 3631.5 Latent Variable Modeling of a Composite Score 3651.6 Inter-Item and Item-Total Correlations 3662 Estimating True Score Models as Latent Variable Models 3682.1 The Congeneric Model 3682.2 Tau-Equivalent and Parallel Models 3702.3 A Strictly Congeneric Model 3712.4 Item Reliability 3712.5 The Congeneric Model with Equally Reliable Indicators 3723 Composite Reliability 3743.1 Coefficient Alpha 3753.2 Bootstrapping Coefficient Alpha 3763.3 The Reliability Index for Congeneric Data 3763.4 Alternate Estimation of Coefficient Alpha 3783.5 Coefficient Omega 3793.6 Bootstrapping Omega 3803.7 Jointly Estimating and Comparing Alpha and Omega 3803.8 Omega If an Item Is Deleted 3823.9 Coefficient Rho 3833.10 Maximal Reliability and Optimally Weighted Indicators 3844 Standardized Reliability 3854.1 Standardized Alpha 3864.2 Standardized Omega 3874.3 Standardized Rho 3894.4 Standardized Maximal Reliability (Coefficient H) 3905 Summary 391References 3972 Computational Guide 2 Item Response Theory 3991 Item Response Theory (IRT) Models for Dichotomous Items 4002 Logistic (Logit) Item Response Theory Models 4002.1 The Rasch Model 4012.2 The One-Parameter Logistic (1PL) Model 4012.3 The Two-Parameter Logistic (2PL) Model 4023 Data Example 4033.1 Analyzing the Compressed Data 4033.2 Expanding the Compact Data and Analyzing the Expanded Data 4043.3 Estimating the Rasch Model 4053.4 Item Response Theory (IRT) and Categorical Item Factor Analysis (CIFA) for Dichotomous Items (1PL) 4064 Categorical Item Factor Analysis and Item Response Theory Models for Dichotomous Items 4074.1 The Rasch Model 4104.2 Estimated Ability Scores with the Rasch Model 4114.3 Relative Model Fit of the 1PL and 2PL Models 4124.4 Item Characteristic Curves (ICCs) and Item Information Functions (IIFs) 4125 Normal Ogive (Probit) IRT Models 4146 IRT Models for Dichotomous Items Using Mplus 4156.1 The Rasch Model 4156.2 The 1PL Model 4166.3 The 2PL Model 4176.4 IRT Parameterization for the 1PL and Logistic Rasch Models 4177 IRT Models for Ordered Categorical (Polytomous) Items 4177.1 Category Comparisons 4177.2 Model Specification 4187.3 Categorical Item Factor Analysis (CIFA) for Ordered Polytomous Categorical Items 4187.4 Model Equivalence, Nesting, and Comparisons 4198 Four Ordinal Polytomous IRT Models 4218.1 Model A: Item Discrimination Parameters and Step Thresholds 4218.2 Model B: Common Discrimination Parameter and Step Thresholds 4218.3 Model C: Item Discrimination Parameters with Item Location and Common Threshold Offsets 4228.4 Model D: Common Discrimination Parameter with Item Location and Common Threshold Offsets 4229 Software and Example Analyses 42210 Cumulative Probability Models Using Stata 42310.1 Model A: The Graded Response Model (GRM) 42310.2 Model B: The Graded Response Model with Common Discrimination (GRM-C) 42411 Adjacent Category Models Using Stata 42411.1 Model A: The Generalized Partial Credit Model (GPCM) 42411.2 Model B: The Partial Credit Model (PCM) 42511.3 Model C: The Generalized Rating Scale Model (GRSM) 42511.4 Model D: The Rating Scale Model (RSM) 42511.5 Summary 42512 Cumulative Probability Models Using Mplus 42612.1 Model A: The Graded Response Model (GRM) 42612.2 Model B: The Graded Response Model with Common Discrimination 42712.3 Model C: The Modified Graded Response Model (MGRM) 42712.4 Model D: The Modified Graded Response Model with Common Threshold Offsets (mgrm-c) 42813 Adjacent Category Models with Mplus 42913.1 Model A: The Generalized Partial Credit Model (GPCM) 42913.2 Model B: The Partial Credit Model (PCM) 43013.3 Model C: The Generalized Rating Scale Model (GRSM) 43113.4 Model d 43114 Model Implementation 43214.1 Cumulative Probability Models 43214.2 Adjacent Category Models 43215 Model Fit for the Example Data 433A. Appendix 434References 4353 Computational Guide 3 Factor Analysis 4371 Unidimensional and Multidimensional Factor Analysis 4381.1 Factor Analysis with Multiple Factors 4381.2 Exploratory and Confirmatory Factor Analysis 4391.3 The Independent Clusters Correlated Factors CFA Model 4401.4 The Correlated Factors Exploratory Factor Analysis (EFA) Model 4411.5 Higher-Order Factor Models 4411.6 Bifactor Models 4421.7 Data Example 4442 Correlated Factors Model 4452.1 The Correlated Factors Exploratory Factor Analysis (EFA) Model 4452.2 Communalities and Exploratory Structural Equation Modeling in Mplus 4462.3 The Correlated Factors Confirmatory Factor Analysis (CFA) Model 4483 Higher-Order Factor Models 4483.1 The Higher-Order Factor CFA 4483.2 The Higher Order CFA Model with Direct Indicators of the Higher Order Factor 4503.3 The Higher Order EFA/ESEM Model with Direct Indicators of the Higher Order Factor 4514 Bifactor Models 4524.1 The Symmetric or Complete Bifactor CFA 4524.2 The Bifactor CFA with a Reference Factor 4534.3 The Bifactor CFA Model with Direct Indicators of the General Factor 4544.4 The Orthogonal Bifactor EFA/ESEM with Direct Indicators of the General Factor 4554.5 Model Fit of Multidimensional Factor Analysis Models 4565 The Relationship Between the Higher Order and Bifactor Models 4575.1 The Higher Order Model as a Constrained Bifactor Model 4575.2 Loadings of the Indicators on the Higher-Order Factor Using the Higher Order Model 4585.3 The Bifactor Model as an Extended Higher Order Factor Model 4616 Reliability Estimation for Multidimensional Constructs 4626.1 Stratified Alpha 4626.2 Reliability Index with the Correlated Factors Model 4646.3 Reliability Index for the Higher Order Factor Model 4666.4 Reliability Estimation Via the Reliability Index in the Bifactor CFA 4676.5 Reliability Estimation for Higher-Order Factor Models Using Analytic Formulas 4697 Omega Reliability Estimation Via Reliability Formulas for Hierarchical Data 4717.1 Omega Hierarchical 4717.2 Omega Total 4717.3 Omega Hierarchical Subscale 4727.4 Omega Subscale 4727.5 Calculating Omega Reliability Coefficients from Factor Loadings and Residual Variances 4728 Expected Common Variance 4758.1 Expected Common Variance for the General Factor 4758.2 Expected Common Variance for Subscales 4758.3 Item Expected Common Variance 4759 Other Reliability Related Information 4779.1 Construct Replicability Index 4779.2 Average Relative Parameter Bias 4779.3 Factor Determinacy 47810 Special Situations in Reliability Assessment 48010.1 Negative Factor Loadings on the Specific Factors 48010.2 Correlated Indicator Residuals 481References 482Online Datasets and Services 483Appendix 1: Central Tendency: Mean, Median, and Mode 485Appendix 2: Calculation of Means, Variances, and Correlation Coefficients by the Algebra of Expectations 491Appendix 3: Matrix Multiplication 505Appendix 4: Bivariate Regression Analysis 519Appendix 5: Multiple Regression Analysis using Matrix Algebra and Stata 535References for the Five Mathematical Appendices 561Index 563