Applied Research Methods in Public and Nonprofit Organizations

MITCHELL BROWN, PHD, is Associate Professor at Auburn University, Department of Political Science. Brown is Co-Director (with Kathleen Hale) of the Community Governance Research Project.KATHLEEN HALE, J.D., PHD, is Associate Professor and M.P.A. Director at Auburn University, Department of Political Science. Hale is author of How Information Matters: Networks and Public Policy Innovation.

• List of Figures and Tables xiiiAcknowledgments xviiThe Authors xixIntroduction 1Integrating Case Studies into Learning About Applied Research Methods 2The Cases 3Community Garden Initiative: Exploring Community Interest in Local Food Projects 4Statewide Needs Assessment: Understanding the Status of Aging Women 6Election Administration and Technology: Understanding the Implementation and Implications of Election Policy Decisions 8National Program Evaluation: Domestic Violence Prevention and Faith-Based and Community Organizations 11Criminal Justice Policy Reform: Exploring Innovation Across Sectors 13Structure of the Book and Message to Instructors 15Part One: Research Basics: What Do We Want to Know and Why? 19Chapter One Overview and Context of the Research Process 21The Research Process 21The Applied Research Context 25Public Administration, Nonprofit Studies, and Other Subfields 27Information Literacy 28Information Quality and Types of Sources 29Finding High-Quality Information 37Values, Beliefs, and Certainty in Applied Disciplines 38Grand Theory and Theories of the Middle Range 40Case Comparisons 41Ethics and Institutional Review Boards 43Specific Concerns for Public Administration and Nonprofit Studies 45Decision Trees 47Applied Research Project Planning 47Typical IRB Review Considerations 48Conclusion 49Overview of the Book 50Chapter Support Materials 51Chapter Two Applied Research Questions, Literature, and Theory 55Asking Good Research Questions 56Case Illustrations of Research Questions 57Research Questions and Sponsored Research 60Case Illustrations of Research Questions in Sponsored Research 60Using Literature and Building Theory 62The Literature Review 62Literature Sources 63Literature Searches 65A Note About Citations 69Theory Building 71Developing Research Expectations and Hypotheses 75Theories of Change and Logic Models 79Theories of Change 79Logic Models 82Steps for Development of Theories of Change 86Decision Tree 90Decisions About Applied Research and Using a Theory of Change 90Conclusion 91Chapter Support Materials 92Chapter Three Applied Research Designs 99General Parameters of Design 99Major Design Types 100Experimental Design 100Quasi-Experimental Design 105Nonexperimental Design 108Case Illustrations of Research Design Approaches 113Sampling 117Random or Probability Sampling 118Nonrandom or Nonprobability Sampling 121Sample Size 123Case Illustrations of Sampling Approaches 123Decision Trees 127Selecting a Research Design Type 127Choosing a Sampling Strategy 129Conclusion 130Chapter Support Materials 131Part two: Data Collection: Where and How Can We Gather Information? 135Chapter Four Qualitative Data Collection in the Field 137Validity, Reliability, and Error in Qualitative Research 138The Process of Conducting Field Research 142Interviews 146Writing Interview Questions 148Putting Respondents at Ease 149Focus Groups 151Direct Observation 152Content Analysis 154Case Studies 159Case Illustrations of Data Collection 161Statewide Needs Assessment 161Election Administration and Technology 162National Program Evaluation 163Decision Tree 163Data Collection Strategy 164Conclusion 165Chapter Support Materials 165Chapter Five Survey Research 171The Survey Research Process 172Survey Design 175Crafting Quality Questions 176Response Categories 181Demographic Questions 188Problems and Controversies in Survey Research 189Case Illustrations of Survey Use 191Statewide Needs Assessment 191Criminal Justice Policy Reform 192National Program Evaluation 192Decision Tree 193Decisions About Methods of Survey Distribution 193Conclusion 194Chapter Support Materials 195Part Three: Analyzing Data and Communicating About Them: What Do They Mean? How Can They Be Used? 199Chapter Six Data Analysis 201Context and Descriptive Analysis 202Qualitative Analysis 203Quantitative Analysis 207Coding 208Levels of Measurement 209Descriptive Statistics 210Statistical Significance 216Bivariate Statistics 219Multivariate Statistics 225Codebooks 232Case Applications of Qualitative and Quantitative Analysis 233Decision Tree 240Conclusion 242Chapter Support Materials 242Chapter Seven Writing and Presentations 249General Tips for Writing 250A Productive Writing Process 250Feedback and (Constructive) Criticism 251Readability and Appeal 252A Note About Style 254Types of Writing 256Needs Assessments and Asset Mapping 256Grant Applications 258Funder Reports 264Stakeholder Documents 266Journal Articles 268Applied Dissertations 270Presentations 271Presenting Material to Stakeholders 274Academic Presentations 275Case Illustrations of Writing and Presentation 277Decision Tree 277Decisions About Making Presentations 277Conclusion 280Chapter Support Materials 282Glossary 287Bibliography and Works Cited 303Index 3091.1 The Research Process 221.2 Inside the Steps of the Research Process 231.3 Top Ten Information Sources from Google Search for “Homeless Women” 371.4 Typical Research Roles Covered by IRB Requirements 441.5 Decision Tree for Planning a Research Process 481.6 Decision Tree for Typical IRB Review Process 491.7 Research Process Outline Worksheet 542.1 List of Selected Scholarly Journals for Applied Research in Public Service 662.2 Examples of Scholarly Sources Used in Case Studies 672.3 Journal Article Worksheet 682.4 Deductive and Inductive Approaches to Theory 722.5 Theory-Fact Relationships in Deduction and Induction 722.6 Time Horizons and Outcomes in the Theory of Change Model for the Community Garden Case 852.7 Logic Model/Theory of Change to Increase Democratic Functioning Through Voting 852.8 Program Evaluation Logic Model for Study of Community Organizations 892.9 Decision Tree for Using a Theory of Change Model 912.10 Annotated Bibliography Entry Worksheet 972.11 Logic Model Worksheet 983.1 Common Sources of Secondary Data 1123.2 Decision Tree for Selecting Research Design Type 1283.3 Decision Tree for Choosing a Sampling Strategy 1294.1 The Field Data Collection Process 1434.2 Decision Tree for Data Collection Strategy 1645.1 Approaches to Question Wording 1785.2 The Influence of Word Order and Response Set Choices 1835.3 Illustration of Ordered Scales 1845.4 Illustration of Likert Scale 1855.5 Decision Tree for Choosing Paper or Web-Based Survey Approach 1946.1 Qualitative Data Display Illustration: Discussion of Partners in Statewide Needs Assessment 2066.2 Illustration of Coding for Quantitative Analysis 2096.3 Graphic Display of Descriptive Statistics 2126.4 Codebook Example for Election Administration and Technology Project 2346.5 Proposed Theory of Change Model Developed from Statewide Needs Assessment Case 2396.6 Decision Tree for Approaching Qualitative and Quantitative Analyses 2417.1 SWOT Analysis Display for the Community Garden Case 2577.2 Gantt Chart Example for Statewide Needs Assessment 2607.3 Planning Process Example for National Replication of the Statewide Assessment 2617.4 Budget Summary and Budget Narrative 2637.5 Poster for Community Presentation on Statewide Needs Assessment (online only at  www.wiley.com/college/brownhale)7.6 Community Presentation on Community Garden (online only at www.wiley.com/college/brownhale)7.7 Decision Tree for Approaching a Presentation 2791.1 Association of College and Research Libraries Information Literacy Standards and Benchmarks 301.2 Top News Sources by Media Type 341.3 Case Comparison of Research Question, Theory, Reasoning, and Tools 422.1 Case Comparison of Approaches to Theory, Literature, and Hypotheses or Research Expectations 742.2 Case Comparison of Concepts and Operationalization 763.1 Measurement and Intervention Sequence for Major Forms of Experimental Design 1023.2 Major Forms of Quasi-Experimental Design 1073.3 Approaches to Nonexperimental Data Collection 1093.4 Case Comparison of Research Designs 1143.5 Case Comparison of Sampling Strategies 1244.1 Case Comparison of Case Selection Methods 1424.2 Illustration of Content Analysis Strategy for Examining Innovations in Voter Guides 1565.1 Index of Engaged Healthy Living Activities 1876.1 Frequency Distribution Example: State Requirements for Certification of Election Equipment 2116.2 Illustration of Calculation Matrix 2166.3 Measures of Central Tendency and Dispersion by Level of Measurement 2176.4 Bivariate Tests by Level of Measurement 2196.5 Illustration of Bivariate Crosstab Table for Soft Skills Class 2226.6 Multivariate Tests Based on Dependent Variable Measurement 2266.7 Relationship Between Soft Skills Course Participant Background and Course Attendance to Posttest Performance 2296.8 Case Comparison of Use of Qualitative Analysis 2366.9 Case Comparison of Use of Quantitative Analysis 2376.10 Codebook for Soft Skills Class 2467.1 Illustration of Table Format and Style 2557.2 Case Comparison of Types of Writing and Presentations 278