Experimental Evaluation Design for Program Improvement

Laura R. Peck, Ph.D., is a principal scientist at Abt Associates and has spent her career evaluating social welfare and employment policies and programs, both in research and academic settings. A policy analyst by training, Dr. Peck specializes in innovative ways to estimate program impacts in experimental and quasi-experimental evaluations, and she applies this to many social safety net programs. Dr. Peck is currently the principal investigator, co-PI, or director of analysis for several major national evaluations for the U.S. Departments of Health and Human Services, Labor, and Housing and Urban Development and over her career has been part of more than 35 informative evaluations of nonprofit, local, state, and federal programs and policies. Peck is a co-author of a public policy textbook and is well published on program evaluation topics. Prior to her work at Abt Associates, Dr. Peck was a tenured professor at the Arizona State University School of Public Affairs and also served as the founding associate dean of the Barrett Honors College, Downtown Phoenix campus. She earned her PhD from the Wagner Graduate School at New York University.

Peck reminds us that the results from experiments only inform us about average effects, but more importantly provides us with the information necessary to look inside the "black box."

• List of Boxes, Figures, and TablesVolume Editors’ IntroductionAbout the AuthorAcknowledgmentsChapter 1 • IntroductionThe State of the FieldThe Ethics of ExperimentationWhat This Book CoversQuestions and ExercisesResources for Additional LearningChapter 2 • Conceptual Framework: From Program Logic Model to Evaluation Logic ModelProgram Logic ModelEvaluation Logic ModelConclusionQuestions and ExercisesResources for Additional LearningChapter 3 • The Basic Experimental Design DefinedRandom Assignment ExplainedThe Basic (Two-Armed) Experimental DesignTo Have a Control Group or Not to Have A Control Group?Questions and ExercisesResources for Additional LearningChapter 4 • Variants of the Experimental DesignMulti-Armed DesignsFactorial DesignsMultistage DesignsStaggered Introduction DesignsBlended DesignsAligning Evaluation Design Options With Program Characteristics and Research QuestionsConclusionQuestions and ExercisesChapter 5 • Practical Considerations and ConclusionSome Practical ConsiderationsRoad TestingPrinciples for Conducting High-Quality EvaluationQuestions and ExercisesResources for Additional LearningAppendix • Doing the Math and Other Technical ConsiderationsEstimating Treatment ImpactsHow to Interpret ResultsHandling Treatment Group No-Shows and Control Group CrossoversSubgroup AnalysesConclusionQuestions and ExercisesResources for Additional LearningReferencesGlossaryIndex