George F. Madaus - Böcker

Evaluation Models is an up-to-date revision of the classic text first published in 1983. Organized in three sections, the first includes a historical perspective on the growth of evaluation theory and practice and two comparative analyses of the various alternative perspectives on evaluation. The second section includes articles representing the major schools of thought about evaluation written by the leaders who have developed these schools and models. The final section describes and discusses the Standards for Program Evaluation and the reformation of program evaluation.

Standardized testing in the United States has been increasing at a rapid pace in the last 25 years. The market for tests has not only been expanding rapidly, but has also been changing sharply in structure into a fractured marketplace. Indeed, one of the main features of this book is that the market for standardized testing is highly fractured with segments of the market facing monopoly conditions, others facing oligopoly conditions and still others where near free-market conditions exist. One of the main premises of the book is that the structures of markets have strong implications for how those markets perform. While this notion is widely accepted among economists, it is not widely appreciated in educational research. A second motivation for the book is that very little scholarly attention has been focused on the standardized testing industry. This topic - the structure of the testing industry and implications for the quality of tests and test use - affects how we evaluate the learning of students, the effectiveness of teaching, the quality of schools and the educational health of the nation.Of particular concern to the authors is one vital aspect of test quality: test validity.

I personally learned to know Ralph Tyler rather late in his career when, in the 1960s, I spent a year as a Fellow at the Center for Advanced Study in the Behavioral Sciences at Stanford. His term of office as Director of the Center was then approaching its end. This would seem to disqualify me thoroughly from preparing a Foreword to this "Classic Works. " Many of his colleagues and, not least, of his students at his dear Alma Mater, the University of Chicago, are certainly better prepared than I to put his role in American education in proper perspective. The reason for inviting me is, I assume, to bring out the influence that Tyler has had on the international educational scene. I am writing this Foreword on a personal note. Ralph Tyler's accomplishments in his roles as a scholar, policy maker, educational leader, and statesman have been amply put on record in this book, not least in the editors' Preface. My reflections are those of an observer from abroad but who, over the last 25 years, has been close enough to overcome the aloofness of the foreigner. Tyler has over many years been criss-crossing the North American con­ tinent generously giving advice to agencies at the federal, state, and local levels, lecturing, and serving on many committees and task forces that have been instrumental in shaping American education.

As a nation, we spend more than $1 billion a year on federally mandated educational tests that 30 million students must take each year. The country spends an additional $1.2 billion on test preparation materials designed to help students pass these tests. While test mandates were put in place with good intentions, increasingly educational leaders and policy makers are questioning these test based reform efforts. Some question whether these programs are doing more harm than good. Others call for the development of more and better tests.Given the vast amount of resources our nation pours into testing, is it time we pay closer attention to these testing programs? Is it time we hold the testing industry and policy makers accountable for the tests they make and use? Is it time we invest resources to develop new ways of testing our students? The Paradoxes of High-Stakes Testing explores these and other questions, as it helps parents, teachers, educational leaders, and policy makers better understand the complexities of educational policies that use tests as a lever for improving the quality of education.The book explores:-how testing is used to enable teachers and schools to be more effective and improve student learning,-why testing is so ingrained in the American psyche and why policy makers rely on testing policies to reform our educational system,-what we can learn from a long history of test-based reform efforts that have occurred over centuries and across continents,-what effects testing has on teaching and learning in our schools when it is used to solve political, social, or economic problems.Most importantly, the book describes several ways in which testing can be improved to provide more accurate and more useful measures of student learning. Many of these improvements capitalize on technology to provide teachers with more detailed, diagnostic information about student learning and measure skills that some leaders argue are essential for the 21st century work force. Exploring what is within reach is critical because current testing policies are hindering these improvements. Finally, given that testing is and will continue to be an integral part of our educational system, the book concludes that, like other sectors of our society, educational testing must be more closely monitored to ensure that high quality tests are used to measure student achievement and to minimize the negative effects that testing has on students, schools, and our society. Given the opportunity our nation has to rethink and redesign its testing policies, The Paradoxes of High-Stakes Testing presents a clear strategy to maximize the positive effects of educational testing.

As a nation, we spend more than $1 billion a year on federally mandated educational tests that 30 million students must take each year. The country spends an additional $1.2 billion on test preparation materials designed to help students pass these tests. While test mandates were put in place with good intentions, increasingly educational leaders and policy makers are questioning these test based reform efforts. Some question whether these programs are doing more harm than good. Others call for the development of more and better tests.Given the vast amount of resources our nation pours into testing, is it time we pay closer attention to these testing programs? Is it time we hold the testing industry and policy makers accountable for the tests they make and use? Is it time we invest resources to develop new ways of testing our students? The Paradoxes of High-Stakes Testing explores these and other questions, as it helps parents, teachers, educational leaders, and policy makers better understand the complexities of educational policies that use tests as a lever for improving the quality of education.The book explores:-how testing is used to enable teachers and schools to be more effective and improve student learning,-why testing is so ingrained in the American psyche and why policy makers rely on testing policies to reform our educational system,-what we can learn from a long history of test-based reform efforts that have occurred over centuries and across continents,-what effects testing has on teaching and learning in our schools when it is used to solve political, social, or economic problems.Most importantly, the book describes several ways in which testing can be improved to provide more accurate and more useful measures of student learning. Many of these improvements capitalize on technology to provide teachers with more detailed, diagnostic information about student learning and measure skills that some leaders argue are essential for the 21st century work force. Exploring what is within reach is critical because current testing policies are hindering these improvements. Finally, given that testing is and will continue to be an integral part of our educational system, the book concludes that, like other sectors of our society, educational testing must be more closely monitored to ensure that high quality tests are used to measure student achievement and to minimize the negative effects that testing has on students, schools, and our society. Given the opportunity our nation has to rethink and redesign its testing policies, The Paradoxes of High-Stakes Testing presents a clear strategy to maximize the positive effects of educational testing.

Attempting fonnally to evaluate something involves the evaluator coming to grips with a number of abstract concepts such as value, merit, worth, growth, criteria, standards, objectives, needs, nonns, client, audience, validity, reliability, objectivity, practical significance, accountability, improvement, process, pro­ duct, fonnative, summative, costs, impact, infonnation, credibility, and - of course - with the tenn evaluation itself. To communicate with colleagues and clients, evaluators need to clarify what they mean when they use such tenns to denote important concepts central to their work. Moreover, evaluators need to integrate these concepts and their meanings into a coherent framework that guides all aspects of their work. If evaluation is to lay claim to the mantle of a profession, then these conceptualizations of evaluation must lead to the conduct of defensible evaluations. The conceptualization of evaluation can never be a one-time activity nor can any conceptualization be static. Conceptualizations that guide evaluation work must keep pace with the growth of theory and practice in the field. Further, the design and conduct of any particular study involves a good deal of localized conceptualization.

Attempting fonnally to evaluate something involves the evaluator coming to grips with a number of abstract concepts such as value, merit, worth, growth, criteria, standards, objectives, needs, nonns, client, audience, validity, reliability, objectivity, practical significance, accountability, improvement, process, pro­ duct, fonnative, summative, costs, impact, infonnation, credibility, and - of course - with the tenn evaluation itself. To communicate with colleagues and clients, evaluators need to clarify what they mean when they use such tenns to denote important concepts central to their work. Moreover, evaluators need to integrate these concepts and their meanings into a coherent framework that guides all aspects of their work. If evaluation is to lay claim to the mantle of a profession, then these conceptualizations of evaluation must lead to the conduct of defensible evaluations. The conceptualization of evaluation can never be a one-time activity nor can any conceptualization be static. Conceptualizations that guide evaluation work must keep pace with the growth of theory and practice in the field. Further, the design and conduct of any particular study involves a good deal of localized conceptualization.

When George Bernard Shaw wrote his play, Pygmalion, he could hardly have foreseen the use of the concept of the self-fulfilling prophecy in debates about standardized testing in schools. Still less could he have foreseen that the validity of the concept would be examined many years later in Irish schools. While the primary purpose of the experimental study reported in this book was not to investigate the Pygmalion effect, it is inconceivable that a study of the effects of standardized testing, conceived in the 1960s and planned and executed in the 1970s, would not have been influenced by thinking about teachers' expectations and the influence of test information on the formation of those expectations. While our study did pay special attention to teacher expectations, its scope was much wider. It was planned and carried out in a much broader framework, one in which we set out to examine the impact of a standardized testing program, not just on teachers, but also on school practices, students, and students' parents.

Standardized testing in the United States has been increasing at a rapid pace in the last twenty-five years. The market for tests has not only been expanding rapidly, but has also been changing sharply in structure into a fractured marketplace. Indeed, one of the main features of this book is that the market for standardized testing is highly fractured - with segments of the market facing monopoly conditions, others facing oligopoly conditions and still others where near free-market conditions exist.One of the main premises of the book is that the structures of markets have strong implications for how those markets perform. While this notion is widely accepted among economists, it is not widely appreciated in educational research. A second motivation for the book is that very little scholarly attention has been focused on the standardized testing industry. This topic - the structure of the testing industry and implications for the quality of tests and test use - affects how we evaluate the learning of students, the effectiveness of teaching, the quality of schools and the educational health of the nation. Of particular concern to the authors is one vital aspect of test quality: test validity.This book is the most current and authoritative review and analysis of the market for standardized testing.

I personally learned to know Ralph Tyler rather late in his career when, in the 1960s, I spent a year as a Fellow at the Center for Advanced Study in the Behavioral Sciences at Stanford. His term of office as Director of the Center was then approaching its end. This would seem to disqualify me thoroughly from preparing a Foreword to this "Classic Works. " Many of his colleagues and, not least, of his students at his dear Alma Mater, the University of Chicago, are certainly better prepared than I to put his role in American education in proper perspective. The reason for inviting me is, I assume, to bring out the influence that Tyler has had on the international educational scene. I am writing this Foreword on a personal note. Ralph Tyler's accomplishments in his roles as a scholar, policy maker, educational leader, and statesman have been amply put on record in this book, not least in the editors' Preface. My reflections are those of an observer from abroad but who, over the last 25 years, has been close enough to overcome the aloofness of the foreigner. Tyler has over many years been criss-crossing the North American con­ tinent generously giving advice to agencies at the federal, state, and local levels, lecturing, and serving on many committees and task forces that have been instrumental in shaping American education.

Evaluation Models is an up-to-date revision of the classic text first published in 1983. Organized in three sections, the first includes a historical perspective on the growth of evaluation theory and practice and two comparative analyses of the various alternative perspectives on evaluation. The second section includes articles representing the major schools of thought about evaluation written by the leaders who have developed these schools and models. The final section describes and discusses the Standards for Program Evaluation and the reformation of program evaluation.