Paul W. Holland – författare

The issues surrounding the comparability of various tests used to assess performance in schools received broad public attention during congressional debate over the Voluntary National Tests proposed by President Clinton in his 1997 State of the Union Address. Proponents of Voluntary National Tests argue that there is no widely understood, challenging benchmark of individual student performance in 4th-grade reading and 8th-grade mathematics, thus the need for a new test. Opponents argue that a statistical linkage among tests already used by states and districts might provide the sort of comparability called for by the president''s proposal.

Public Law 105-78 requested that the National Research Council study whether an equivalency scale could be developed that would allow test scores from existing commercial tests and state assessments to be compared with each other and with the National Assessment of Education Progress.

In this book, the committee reviewed research literature on the statistical and technical aspects of creating valid links between tests and how the content, use, and purposes of education testing in the United States influences the quality and meaning of those links. The book summarizes relevant prior linkage studies and presents a picture of the diversity of state testing programs. It also looks at the unique characteristics of the National Assessment of Educational Progress.

Uncommon Measures provides an answer to the question posed by Congress in Public Law 105-78, suggests criteria for evaluating the quality of linkages, and calls for further research to determine the level of precision needed to make inferences about linked tests. In arriving at its conclusions, the committee acknowledged that ultimately policymakers and educators must take responsibility for determining the degree of imprecision they are willing to tolerate in testing and linking. This book provides science-based information with which to make those decisions.

The issues surrounding the comparability of various tests used to assess performance in schools received broad public attention during congressional debate over the Voluntary National Tests proposed by President Clinton in his 1997 State of the Union Address. Proponents of Voluntary National Tests argue that there is no widely understood, challenging benchmark of individual student performance in 4th-grade reading and 8th-grade mathematics, thus the need for a new test. Opponents argue that a statistical linkage among tests already used by states and districts might provide the sort of comparability called for by the president''s proposal.

Public Law 105-78 requested that the National Research Council study whether an equivalency scale could be developed that would allow test scores from existing commercial tests and state assessments to be compared with each other and with the National Assessment of Education Progress.

In this book, the committee reviewed research literature on the statistical and technical aspects of creating valid links between tests and how the content, use, and purposes of education testing in the United States influences the quality and meaning of those links. The book summarizes relevant prior linkage studies and presents a picture of the diversity of state testing programs. It also looks at the unique characteristics of the National Assessment of Educational Progress.

Uncommon Measures provides an answer to the question posed by Congress in Public Law 105-78, suggests criteria for evaluating the quality of linkages, and calls for further research to determine the level of precision needed to make inferences about linked tests. In arriving at its conclusions, the committee acknowledged that ultimately policymakers and educators must take responsibility for determining the degree of imprecision they are willing to tolerate in testing and linking. This book provides science-based information with which to make those decisions.

Kernel Equating (KE) is a powerful, modern and unified approach to test equating. It is based on a flexible family of equipercentile-like equating functions and contains the linear equating function as a special case. Any equipercentile equating method has five steps or parts. They are: 1) pre-smoothing; 2) estimation of the score-probabilities on the target population; 3) continuization; 4) computing and diagnosing the equating function; 5) computing the standard error of equating and related accuracy measures. KE brings these steps together in an organized whole rather than treating them as disparate problems.

KE exploits pre-smoothing by fitting log-linear models to score data, and incorporates it into step 5) above. KE provides new tools for diagnosing a given equating function, and for comparing two or more equating functions in order to choose between them. In this book, KE is applied to the four major equating designs and to both Chain Equating and Post-Stratification Equating for the Non-Equivalent groups with Anchor Test Design.

This book will be an important reference for several groups: (a) Statisticians and others interested in the theory behind equating methods and the use of model-based statistical methods for data smoothing in applied work; (b) Practitioners who need to equate tests—including those with these responsibilities in testing companies, state testing agencies and school districts; and (c) Instructors in psychometric and measurement programs. The authors assume some familiarity with linear and equipercentile test equating, and with matrix algebra.

Alina von Davier is an Associate Research Scientist in the Center for Statistical Theory and Practice, at Educational Testing Service. She has been a research collaborator at the Universities of Trier, Magdeburg, and Kiel, an assistant professor at the Politechnical University of Bucharest and a research scientist at the Institute for Psychology inBucharest.

Paul Holland holds the Frederic M. Lord Chair in Measurement and Statistics at Educational Testing Service. He held faculty positions in the Graduate School of Education, University of California, Berkeley and the Harvard Department of Statistics. He is a Fellow of the American Statistical Association, the Institute of Mathematical Statistics, and the American Association for the Advancement of Science. He is an elected Member of the International Statistical Institute and a past president of the Psychometric society. He was awarded the (AERA/ACT) E. F. Lindquist Award, in 2000, and was designated a National Associate of the National Academies of Science in 2002.

Dorothy Thayer currently is a consultant in the Center of Statistical Theory and Practice, at Educational Testing Service. Her research interests include computational and statistical methodology, empirical Bayes techniques, missing data procedures and exploratory data analysis techniques.

From the reviews:

"The book is nicely laid out, is extremely well written, and is an excellent text for a semester course or a short course…The book is highly recommended." Short Book Reviews of the International Statistical Institute, December 2004

"This book is well-written and the presentation is clear, rigorous, and concise...A rich set of applications is used to illustrate the methods...This book is a gem! I highly recommend it to any statistician or psychometrician who has even a passing interest in test equating." Pscyhometrika, March 2006

"This is a great book, and it is the first to focus on the kernel method of test equating." Applied Psychological Measurement, September 2005

Test fairness is a moral imperative for both the makers and the users of tests. This book focuses on methods for detecting test items that function differently for different groups of examinees and on using this information to improve tests. Of interest to all testing and measurement specialists, it examines modern techniques used routinely to insure test fairness. Three of these relevant to the book''s contents are: * detailed reviews of test items by subject matter experts and members of the major subgroups in society (gender, ethnic, and linguistic) that will be represented in the examinee population * comparisons of the predictive validity of the test done separately for each one of the major subgroups of examinees * extensive statistical analyses of the relative performance of major subgroups of examinees on individual test items.

Test fairness is a moral imperative for both the makers and the users of tests. This book focuses on methods for detecting test items that function differently for different groups of examinees and on using this information to improve tests. Of interest to all testing and measurement specialists, it examines modern techniques used routinely to insure test fairness. Three of these relevant to the book''s contents are: * detailed reviews of test items by subject matter experts and members of the major subgroups in society (gender, ethnic, and linguistic) that will be represented in the examinee population * comparisons of the predictive validity of the test done separately for each one of the major subgroups of examinees * extensive statistical analyses of the relative performance of major subgroups of examinees on individual test items.