Robert L. Brennan - Böcker

In 1972 a monograph by Cronbach, Gleser, Nanda, and Rajaratnam was published entitled The Dependability of Behavioral Measurements. That book incorporated, systematized, and extended their previous research into what came to be called generalizability theory, which liberalizes classical test theory, in part through the application of analysis of variance proce­ dures that focus on variance components. Generalizability theory is perhaps the most broadly defined measurement model currently in existence, and the Cronbach et al. (1972) treatment of the theory represents a major con­ tribution to psychometrics. However, as Cronbach et al. (1972, p. 3) state, their book is "complexly organized and by no means simple to follow" and, of course, it is nearly 30 years old. In 1983, ACT, Inc. published my monograph entitled Elements of Gen­ eralizability Theory, with a slightly revised version appearing in 1992. That treatment is considerably less comprehensive than Cronbach et al. (1972) but still detailed enough to convey much ofthe richness of the theory and to facilitate its application. However, the 1983/1992 monograph is essen­ tially two decades old, it does not cover multivariate generalizability theory in depth, and it does not incorporate recent developments in statistics that bear upon the estimation of variance components. Also, of course, there have been numerous developments in generalizability theory in the last 20 years.

During the past two or three decades, research in cognitive science and psychology has yielded an improved understanding of the fundamental psychological nature of knowledge and cognitive skills that psychological testing attempts to measure. These theories have reached sufficient maturity, making it reasonable to look upon them to provide a sound theoretical foundation for assessment, particulary for the content of assessments. This fact, combined with much discontentedness over current testing practices, has inspired efforts to bring testing and cognitive theory together to create a new theoretical framework for psychological testing -- a framework developed for diagnosing learners' differences rather than for ranking learners based on their differences. This volume presents some initial accomplishments in the effort to bring testing and cognitive theory together. Contributors originate from both of the relevant research communities -- cognitive research and psychometric theory. Some represent collaborations between representatives of the two communities; others are efforts to reach out in the direction of the other community. Taking fundamentally different forms, psychometric test theory assumes that knowledge can be represented in terms of one or at most a few dimensions, whereas modern cognitive theory typically represents knowledge in networks -- either networks of conceptual relationships or the transition networks of production systems. Cognitively diagnostic assessment is a new enterprise and it is evident that many challenging problems remain to be addressed. Still, it is already possible to develop highly productive interactions between assessment and instruction in both automated tutoring systems and more conventional classrooms. The editors hope that the chapters presented here show how the reform of assessment can take a rigorous path.

During the past two or three decades, research in cognitive science and psychology has yielded an improved understanding of the fundamental psychological nature of knowledge and cognitive skills that psychological testing attempts to measure. These theories have reached sufficient maturity, making it reasonable to look upon them to provide a sound theoretical foundation for assessment, particulary for the content of assessments. This fact, combined with much discontentedness over current testing practices, has inspired efforts to bring testing and cognitive theory together to create a new theoretical framework for psychological testing -- a framework developed for diagnosing learners' differences rather than for ranking learners based on their differences. This volume presents some initial accomplishments in the effort to bring testing and cognitive theory together. Contributors originate from both of the relevant research communities -- cognitive research and psychometric theory. Some represent collaborations between representatives of the two communities; others are efforts to reach out in the direction of the other community. Taking fundamentally different forms, psychometric test theory assumes that knowledge can be represented in terms of one or at most a few dimensions, whereas modern cognitive theory typically represents knowledge in networks -- either networks of conceptual relationships or the transition networks of production systems. Cognitively diagnostic assessment is a new enterprise and it is evident that many challenging problems remain to be addressed. Still, it is already possible to develop highly productive interactions between assessment and instruction in both automated tutoring systems and more conventional classrooms. The editors hope that the chapters presented here show how the reform of assessment can take a rigorous path.

In 1972 a monograph by Cronbach, Gleser, Nanda, and Rajaratnam was published entitled The Dependability of Behavioral Measurements. That book incorporated, systematized, and extended their previous research into what came to be called generalizability theory, which liberalizes classical test theory, in part through the application of analysis of variance proce­ dures that focus on variance components. Generalizability theory is perhaps the most broadly defined measurement model currently in existence, and the Cronbach et al. (1972) treatment of the theory represents a major con­ tribution to psychometrics. However, as Cronbach et al. (1972, p. 3) state, their book is "complexly organized and by no means simple to follow" and, of course, it is nearly 30 years old. In 1983, ACT, Inc. published my monograph entitled Elements of Gen­ eralizability Theory, with a slightly revised version appearing in 1992. That treatment is considerably less comprehensive than Cronbach et al. (1972) but still detailed enough to convey much ofthe richness of the theory and to facilitate its application. However, the 1983/1992 monograph is essen­ tially two decades old, it does not cover multivariate generalizability theory in depth, and it does not incorporate recent developments in statistics that bear upon the estimation of variance components. Also, of course, there have been numerous developments in generalizability theory in the last 20 years.

This book provides an introduction to test equating, scaling and linking, including those concepts and practical issues that are critical for developers and all other testing professionals.