Advances in Program Evaluation – serie

This book is the first of its kind to address community arts and evaluation with an emphasis on the visual arts. This volume draws together a variety of international perspectives dealing with the common difficulty of conception related to both the mission of community centers and of the value of the visual arts programs offered within them. This volume provides a wide range of views solicited from scholars, expert in evaluation and community arts, who are situated in different cultural settings.

One of the more interesting perspectives for evaluation is 'narrative'. Narratives are the common vehicles people use to understand and to communicate the value of their actions and social practices. Given the valuational and action-oriented character of narrative it seems strange that evaluators have not yet discovered its value. In this volume we, an international and multidisciplinary group of practising evaluators and policy analysts in various policy fields such as mental health, education, social welfare and water management, explore what a narrative perspective can mean for the practice of program evaluation. We do so by showing and telling.

The field of programme evaluation is shaped by an ever-increasing range of approaches each of which, to varying degrees, reflects evaluation's dual role as a theoretical endeavour and a form of socio-political inquiry. There is an array of approaches, each emphasizing different purposes and endorsing different methodologies to guide practice. Yet, no matter which goals are pursued and which methods are employed, all evaluation involves an effort to conceptualize, comprehend, and convey the quality of the programme. This volume brings together the work of certain evaluators to explore the evaluation of programme quality. Through conceptual descriptions and applied examples they discuss the theoretical concerns and practical issues that give rise to their particular conceptions of quality, the methodologies they employ to pursue an understanding of these conceptions, and the representational forms they employ to convey their understanding to stakeholders.

Nothing is as fundamental to the quality of a school system as the quality of its teachers. For this reason many countries are seeking policies that meet the challenge of promoting quality teaching. The National Board for Professional Teaching Standards (NBPTS) in the USA is the most ambitious and successful attempt by any country to establish a certification system for recognizing and rewarding teachers who reach high professional standards. Established in 1987, the Boards mission has been driven by a simple principle; to place more value on accomplished teaching, we must learn how to evaluate teacher performance in ways that are valid, reliable and fair. By late 2007, over 60,000 teachers had become National Board Certified Teachers. As other countries place increasing importance on policies designed to attract, develop, and retain effective teachers there is growing interest in the extensive research and development work that has underpinned the Boards certification system and guided its implementation. The main purpose of this book is to bring together, for international as well as non-specialist audiences, papers written by the key researchers involved in the development of National Board assessments between 1987 and 1997.One of the many reasons why the Boards work is noteworthy is the extent to which, at every stage of development, it has subjected its standards and certification process to critique by leading figures in educational measurement, as the chapters in this volume illustrate. As a result, the NBPTS has made a major contribution to international understanding about how to develop valid standards and reliable methods for assessing teacher performance that gain both professional and public esteem. The NBPTS provides an example of a well researched certification scheme for measuring teacher quality that can provide a service to governments and employers seeking a reliable indicator of teacher quality. Standards are the gateway to greater professional self-direction, and a certification system is a means by which the teaching profession can build its own infrastructure for defining high quality teaching standards, promoting development toward those standards and providing recognition to those who meet them. This book also aims to provide some of the many lessons the Board has learned about how to make a professional certification system for teachers manageable and affordable, which is perhaps an even greater challenge. This is a special volume focussing on The National Board for Professional Teaching Standards (NBPTS).

The National Science Foundations (NSF) Advanced Technological Education (ATE) program is a diverse and dynamic set of projects intent on improving the USA's technical workforce. This book uses the ATE work as a means to focus on key issues for federally funded projects and all community colleges facing the difficult challenges of staying current and reaching constituents in positive ways. This book brings a wealth of information on key issues for community colleges. Topics addressed include: how to sustain innovative programs; how to develop effective instructional materials; how to promote and maintain collaboration among various institutions including business and industry; how to provide high quality professional development; how to strengthen and improve programs of study; how to recruit and retain diverse students; how to evaluate and use evaluation effectively; and how to engage advisory committees to effectively serve program needs. In developing these topics, the authors drew heavily on previous research and a common set of evaluation studies including site visits at 13 different projects and surveys of more than 100 ATE projects.

In our reforming public institutions it sometimes feels as though the very ground of social and political contracts is shifting. The economic revolution embraced by neo-liberals and neo-conservatives is paralleled by a governance revolution in those same institutions which were designed to protect us from historical swings and ideological roundabouts. Our public institutions - for the most part the public sector and its professional groups - in the eyes of some provided stability, while for others they were a brake on change. Now, however, they have become conduits for political change and reform. We live in an institutional world now dubbed the New Public Management (NPM). In this new landscape evaluators might have to think afresh about how to position ourselves in relation to institutional ethics and the pursuit of social justice. In this volume contributors give us a start in thinking through such a repositioning, some within the values framework of NPM, others as external observers.

This examination of advances in programme evaluation looks at issues such as deconstructing illuminative evaluation and new paradigm research; programme evaluation as a conceptual practice of power; and pragmatism and evaluative enquiry for organizational learning. It acknowledges the spread of postmodern doubt to the field of programme evaluation and the contributing authors discuss the theoretical and practical challenges postmodernism presents to programme evaluators.

A comprehensive and authoritative overview of issues relating to the evaluation of criminal justice/corrections 'interventions', this unique reference draws on a variety of theoretical, cultural and epistemological perspectives with authors from a range of disciplines and countries. It begins by looking at the purpose of evaluation within criminal justice systems as a historical and conceptual background. Methods outlined for evaluating criminal justice focus on educating readers about the design decisions they may face as evaluators, enabling them to make informed decisions when choosing designs that are not necessarily optimal. It raises the question of who evaluation is for, and a clearly informed discussion of the importance of the full range of stakeholders involved in evaluation and the potential impact of participating in evaluations on different stakeholders is presented. With insight into successful and unsuccessful evaluation from the perspective of those who are being evaluated, and a critical examination of the methodological and conceptual difficulties involved in identifying 'effects', this book concludes by looking ahead to the future of criminal justice evaluation.

Current conventions in school evaluation focus on accountability, control and compliance. New Zealand offers a distinctive, systemic alternative to school self-evaluation, with developmental and negotiated approaches ingrained throughout the education system, from school inspection to major government schooling improvement initiatives. In New Zealand there is no national testing, other than a Ministry-sponsored (voluntary) formative assessment system designed for school and teacher self-evaluation. This is a form of professional and program evaluation where there is shared power and responsibility between evaluators and those being evaluated. Through a detailed national case study of New Zealand, together with commentaries from international specialists, this volume examines the successes and challenges of this approach to programme evaluation and its generalizability to other educational and professional review settings, and show how education systems can recover a balance between an achievement agenda and a focus on educational quality.

In today's world, with its preoccupation with impact assessments and results-based management, program evaluation is all too often framed as an affirmation of an official narrative rather than as a source of alternatives. The power of case study is its insistence on opening up rather than suppressing the complexity of social programs, on documenting multiple voices and exploring contested viewpoints. In this way, case study resists the trend towards evaluations that simply focus on 'what works', that reduce the complexity of social life to a single narrative, and to formulations that strip out most of what matters. Now more than ever, as government policies and programs orientate to global economic crisis and its impact on the lives of citizens and communities, we require evaluations that resist information loss and produce richness.