Confirmative Evaluation

Joan Conway Dessinger is founder of and senior consultant with The Lake Group, a performance improvement consulting firm. She is the coauthor of two books with James L. Moseley and Darlene Van Tiem. The books won the 2001 and 2003 Award for Excellence in Instructional Communication from the International Society for Performance Improvement. She is also a contributor to Distance Training and Sustaining Distance Training, both from Jossey-Bass. James L. Moseley is an associate professor of community medicine at Wayne State University and is also on the faculty of the College of Education, in the instructional technology department. He has won numerous awards for teaching and mentoring and frequently consults with business and industry on program evaluation and needs assessment.

• List of Figures, Tables, and Performance Support Tools xv Acknowledgments xviiIntroduction: Getting the Most from This Resource xixPART 1 The ChallengeChapter 1: Full-Scope Evaluation: Raising the Bar 3Evaluation: The Full Scope 5Comparing the Four Types of Evaluation 11Evaluation: Full-Scope Model 15Challenges to Full-Scope Evaluation 17Chapter 2: Confirmative Evaluation: A Model Guides the Way 21Confirmative Evaluation Model 22Challenges to Implementing Confirmative Evaluation 29Why Bother? 33PART 2 Meeting the ChallengeChapter 3: Preplan: Assess Training Program Evaluability 41When to Plan Confirmative Evaluation 43How to Plan a Confirmative Evaluation 45Assess Evaluability 48Challenges to Evaluability Assessment 68Chapter 4: Plan: The Plan's the Thing 73What's in a Confirmative Evaluation Plan? 75Review, Validate, and Approve the Plan 101Chapter 5: Do: For Goodness' Sake 107Jump-Start Data Collection 108Focus Data Collection 112Collect the Data 116Train the Data Collectors 127Store the Data 127Manage the Data-Collection Process 128Chapter 6: Analyze: Everything Old Is New Again 133Get Ready, Get Set 134Prepare the Confirmative Evaluation Data 135Now Analyze 137Interpret Confirmative Evaluation Results 143Make Results-Based Recommendations 146Report Confirmative Evaluation Results 147Chapter 7: Improve: Now What? 157Focus on Utilization 158Assume the Role 162Accept the Challenge 163Alignment: The Last Word 168PART 3 Lessons from OzChapter 8: Case Study: Lions and Tigers and Bears, Oh My! 175The Case Study 176Meta Evaluation 186Final Thoughts 188Chapter 9: Conclusion: We’re Not in Oz Anymore 191Issues That Challenge Confirmative Evaluators 192Evaluation as an Emerging Discipline 197Improving the Process 198Put Yourself in the Picture 199Glossary 203References 211Index 221About the Authors 227About the Series Editors 229About the Advisory Board Members 233