Evaluation and Measurement in Health Promotion

Manoj Sharma, MBBS, PhD, MCHES, is a tenured Full Professor & Chair of the Social & Behavioral Health Department at the University of Nevada, Las Vegas in the School of Public Health and an Adjunct Professor in the Department of Internal Medicine at the Kirk Kerkorian School of Medicine at UNLV.R. Lingyak Petosa, PhD, FAAHB, is a professor of health promotion at The Ohio State University, Columbus, Ohio

• Preface xiiiInstructor Resources xvAcknowledgments xviiAbout the Authors xixCHAPTER 1 INTRODUCTION TO EVALUATION IN HEALTH EDUCATION AND HEALTH PROMOTION 1Key Concepts 1Chapter Objectives 1The Field of Health Education and Health Promotion 1Defining Evaluation 3Evaluation Competencies for Health Educators 4Historical Milestones in the Evaluation of Health Education and Health Promotion 10FOCUS FEATURE 1.1 Hypothetical Account of a Health Educator's Evaluation Responsibilities 10Basic Terms in Evaluation 16Skill-Building Activity 19CHAPTER 2 PLANNING EVALUATIONS FOR HEALTH EDUCATION AND HEALTH PROMOTION PROGRAMS 25Key Concepts 25Chapter Objectives 25Evaluation and Continuous Health Program Improvement 25Logic Model for Health Education and Health Promotion Programs 26Logic Model and Comprehensive Evaluation 29Health Program Planning and Evaluation: Integration 30Political Climate and Evaluation 32Political Climate and Program Stakeholders 32Balance Between Evaluation Rigor and Utilization 35Conducting a Utilization-Focused Evaluation 36Conducting a Rigorous Evaluation 36FOCUS FEATURE 2.1 Planning a Health Education/Promotion Program Evaluation 38Critical Issues for an Evaluation Project 39Skill-Building Activity 39CHAPTER 3 BASICS OF MEASUREMENT 45Key Concepts 45Chapter Objectives 45Definition of Measurement 45Scales or Levels of Measurement 46Conceptual Paradigm of Measurement 48Types of Variables in Health Education and Health Promotion 51FOCUS FEATURE 3.1 Example of a Scale Measuring Knowledge 52FOCUS FEATURE 3.2 Example of a Scale Measuring Attitudes 54Skill-Building Activity 59CHAPTER 4 STEPS IN INSTRUMENT DEVELOPMENT 63Key Concepts 63Chapter Objectives 63Basic Principles of Instrument Development 63FOCUS FEATURE 4.1 Example of a Theory-Based Scale 71General Principles of Writing Items 78Types of Instruments Measuring Knowledge 78Types of Instruments Measuring Attitudes 81FOCUS FEATURE 4.2 Rosenberg Self-Esteem Scale 82Types of Instruments Measuring Health Behavior 85Physiological and Biomedical Instruments: Outcomes of Behavior 88Skill-Building Activity 89CHAPTER 5 RELIABILITY ASSESSMENT 93Key Concepts 93Chapter Objectives 93Definition of Reliability 93Types of Reliability 96FOCUS FEATURE 5.1 Using IBM-SPSS to Calculate Reliability Statistics 101Tips to Increase the Reliability of an Instrument 102Skill-Building Activity 102CHAPTER 6 VALIDITY ASSESSMENT 107Key Concepts 107Chapter Objectives 107Definition of Validity 107Types of Validity 108Assessing Face and Content Validity 108FOCUS FEATURE 6.1 Form Letter to Panel of Experts forFace and Content Validation 110Construct Validity and Factor Analysis 111Criterion Validity 115FOCUS FEATURE 6.2 Using IBM-SPSSfor Confirmatory Factor Analysis 116FOCUS FEATURE 6.3 Case Study of Confirmatory Factor Analysis 116Skill-Building Activity 121CHAPTER 7 MEASUREMENT ERRORS 127Key Concepts 127Chapter Objectives 127Definition of Measurement Error 127Classification of Measurement Errors 129FOCUS FEATURE 7.1 Strategies to Minimize Errors in Self-Report Instruments 136FOCUS FEATURE 7.2 Strategies to Minimize Errors in Interviews 137FOCUS FEATURE 7.3 Strategies to Minimize Errors in Observations 138Skill-Building Activity 138CHAPTER 8 PROCESS EVALUATION 143Key Concepts 143Chapter Objectives 143Defining Process Evaluation 144Targets of Process Evaluation 144Implementation Fidelity 148FOCUS FEATURE 8.1 Example of Implementation Fidelity Assessment 149Process Evaluation: Foundational to Impact and Outcome Evaluation 151Pragmatic Planning for a Process Evaluation 152Formative Evaluation 152Skill-Building Activity 155CHAPTER 9 DESIGNS FOR QUANTITATIVE EVALUATION 159Key Concepts 159Chapter Objectives 159Questions in Evaluation 159Evaluation Design 160Components of a Testable Hypothesis 162Internal Validity 162External Validity 165Evaluation Design Notation 169Types of Designs in Health Promotion and Education 169Barriers to Randomization 171FOCUS FEATURE 9.1 Considerations in Choosing a Design 172Skill-Building Activity 173CHAPTER 10 APPROACHES TO QUALITATIVE EVALUATION 177Key Concepts 177Chapter Objectives 177Introduction to Qualitative Evaluation and Research 177Quantitative versus Qualitative Evaluation 179Empowerment Evaluation 180Illuminative Evaluation 183Participatory Evaluation 184FOCUS FEATURE 10.1 Case Study of Participatory Evaluation in Vietnam 186CIPP Model of Evaluation 193Utilization-Focused Evaluation 194Goal-Free Evaluation 194Qualitative Data Analysis and Interpretation 195Skill-Building Activity 197CHAPTER 11 SAMPLING 203Key Concepts 203Chapter Objectives 203What Is a Sample? 203Probability Sampling 204Barriers to Probability Samples in Health Program Evaluations 206Nonprobability Samples 207Sample Size: How Many Participants Are Needed for the Evaluation? 208Response Rates 211FOCUS FEATURE 11.1 Calculating Sample Size Using G*Power 212Generalizing Evaluation Results 213Describing the Evaluation Sample and Population 214Skill-Building Activity 214CHAPTER 12 QUANTITATIVE DATA ANALYSIS 219Key Concepts 219Chapter Objectives 219Introduction 219Data Management 220Data Cleaning 221Describing the Sample 223Data Analysis: Descriptive Methods 224Frequency Distributions 227Measures of Central Tendency 227Measures of Variability 228Graphing Data 229Statistical Inference 232Hypothesis Testing 232Statistical Tests used in Health Evaluations 234FOCUS FEATURE 12.1 Using IBM-SPSS for a Paired t-Test 238FOCUS FEATURE 12.2 Using IBM-SPSS for Repeated Measures ANOVA 238Using Effect Sizes to Communicate Evaluation Results 239Skill-Building Activity 241CHAPTER 13 DATA INTERPRETATION AND REPORT WRITING 247Key Concepts 247Chapter Objectives 247Introduction 247Interpretation of Evaluation Results: Conceptual Tools 248Statistical Significance and Practical Importance 248Potential Failures in Interpretation 251Evaluation Standards: How Much Is Enough? 256Effectively Communicating Statistical Results 258Evaluation Report Writing 259Sections of the Evaluation Report 260FOCUS FEATURE 13.1 Example of an Evaluation Report inHealth Education 269Summary 280Review Question 281Websites to Explore 281References 282Glossary 285Index 303