Case Study Research in Software Engineering

Dr. Austen Rainer is a Senior Lecturer in the School of Computer Science at the University of Hertfordshire. He is an active member of the Centre for Empirical Software Process Research (CESPR) and teaches several courses?in the BSc(Hons) and MSc degree programs. Dr. Per Runeson is leader of the Software Engineering Research Group at Lund University, as well as Research Director for EASE - Embedded Applications Software Engineering. He is a Senior Researcher at Sony Ericsson, where he fulfills his role as a verification advisor. His research focuses on efficient verification and validation processes. Dr. Martin Höst is Director of the M.Sc. Programme in Computer Science and Engineering at Lund University and a member of the Software Engineering Research Group (SERG). Dr. Björn Regnell is a professor of Software Engineering at Lund University's Department of Computer Science and Vice Dean of Research at the Faculty of Engineering, LTH. His research interests include market-driven software development, requirements engineering, software quality, software innovation, software product management, and empirical research methods.

• FOREWORD xiii PREFACE xvACKNOWLEDGMENTS xviiPART I CASE STUDY METHODOLOGY1 INTRODUCTION 31.1 What is a Case Study? 31.2 A Brief History of Case Studies in Software Engineering 51.3 Why a Book on Case Studies of Software Engineering? 61.4 Conclusion 92 BACKGROUND AND DEFINITION OF CONCEPTS 112.1 Introduction 112.2 Research Strategies 112.3 Characteristics of Research Strategies 132.3.1 Purpose 132.3.2 Control and Data 142.3.3 Triangulation 152.3.4 Replication 162.3.5 Inductive and Deductive Enquiries 162.4 What Makes a Good Case Study? 172.5 When is the Case Study Strategy Feasible? 192.6 Case Study Research Process 202.7 Conclusion 213 DESIGN OF THE CASE STUDY 233.1 Introduction 233.2 Elements of the Case Study Design 243.2.1 Rationale for the Study 243.2.2 Objective of the Study 243.2.3 Cases and Units of Analyses 263.2.4 Theoretical Framework 293.2.5 Research Questions 303.2.6 Propositions and Hypotheses 313.2.7 Concepts 323.2.8 Methods of Data Collection 323.2.9 Methods of Data Analysis 333.2.10 Case Selection 333.2.11 Selection of Data 353.2.12 Data Definition and Data Storage 363.2.13 Quality Control and Assurance 363.2.14 Maintaining the Case Study Protocol 373.2.15 Reporting and Disseminating the Case Study 383.3 Legal, Ethical, and Professional Issues 403.4 Conclusion 454 DATA COLLECTION 474.1 Introduction 474.2 Different Types of Data Source 474.2.1 Classification of Data Sources 474.2.2 Data Source Selection 494.3 Interviews 504.3.1 Planning Interviews 504.3.2 The Interview Session 524.3.3 Postinterview Activities 534.4 Focus groups 544.5 Observations 564.6 Archival Data 574.7 Metrics 584.8 Conclusion 605 DATA ANALYSIS AND INTERPRETATION 615.1 Introduction 615.2 Analysis of Data in Flexible Research 625.2.1 Introduction 625.2.2 Level of Formalism 645.2.3 Relation to Hypotheses 655.3 Process for Qualitative Data Analysis 655.3.1 Introduction 655.3.2 Steps in the Analysis 665.3.3 Techniques 685.3.4 Tool support 705.4 Validity 715.4.1 Construct Validity 715.4.2 Internal Validity 715.4.3 External Validity 715.4.4 Reliability 725.5 Improving Validity 725.6 Quantitative Data Analysis 745.7 Conclusion 766 REPORTING AND DISSEMINATION 776.1 Introduction 776.2 Why Report and Disseminate 786.3 The Audience for the Report 796.4 Aspects of the Case Study to Report and Disseminate 806.5 When to Report and Disseminate 816.6 Guidelines on Reporting 826.6.1 The Generic Content of an Academic Report 826.6.2 Reporting Recommendations from Evaluative Case Studies 846.6.3 Reporting to Stakeholders, Including Sponsor(s) 856.6.4 Reporting the Context of the Case Study 876.6.5 Reporting to Students 896.6.6 Ad Hoc and Impromptu Reporting 906.7 Formats and Structures for a Report 916.8 Where to Report 946.9 Ethics and Confidentiality 946.10 Conclusion 957 SCALING UP CASE STUDY RESEARCH TO REAL-WORLD SOFTWARE PRACTICE 977.1 Introduction 977.2 The Aims of Scaling up Case Studies 987.3 Dimensions of Scale 997.4 Longitudinal Case Studies 1007.5 Multiple Case Studies 1027.5.1 Multiple Cases and Replications 1027.5.2 Selecting the Cases 1047.6 Multiresearcher Case Studies 1057.7 Conclusion 1078 USING CASE STUDY RESEARCH 1098.1 Introduction 1098.2 Reading and Reviewing Case Studies 1098.2.1 Development of Checklists 1108.2.2 Checklists for Conducting Case Study Research 1118.2.3 Checklists for Reading and Reviewing Case Studies 1118.2.4 Development of Practice 1118.3 Identifying and Synthesizing Use Case Research 1118.3.1 Identifying Primary Studies 1128.3.2 Synthesis of Evidence from Multiple Case Studies 1138.3.3 Current State of Synthesis 1178.4 The Economics of Case Study Research 1188.4.1 Costs and Benefits of Evaluation Techniques 1198.4.2 Evaluation of the DESMET Methodology 1198.4.3 Frameworks for Organizing Methods of Evaluation 1198.5 Specializing Case Study Research for Software Engineering 1218.5.1 The Longitudinal Chronological Case Study Research Strategy 1228.5.2 Controlled Case Studies 1238.6 Case Studies and Software Process Improvement 1238.7 Conclusion 125PART II EXAMPLES OF CASE STUDIES9 INTRODUCTION TO CASE STUDY EXAMPLES 1299.1 Introduction 12910 CASE STUDY OF EXTREME PROGRAMMING IN A STAGE–GATE CONTEXT 13310.1 Introduction 13310.1.1 Methodological Status 13310.2 Case Study Design 13410.2.1 Rationale 13410.2.2 Objectives 13410.2.3 Cases and Units of Analysis 13510.2.4 Theoretical Frame of Reference 13610.2.5 Research Questions 13610.3 Planning 13610.3.1 Methods of Data Collection 13610.3.2 Selection of Data 13710.3.3 Case Selection Strategy 13710.3.4 Case Study Protocol 13710.3.5 Ethical Considerations 13710.4 Data Collection 13910.5 Data Analysis 13910.5.1 Threats to Validity 14410.6 Reporting 14410.6.1 Academics 14410.6.2 Practitioners 14410.7 Lessons Learned 14611 TWO LONGITUDINAL CASE STUDIES OF SOFTWARE PROJECT MANAGEMENT 14911.1 Introduction 14911.2 Background to the Research Project 14911.3 Case Study Design and Planning 15011.3.1 Rationale 15011.3.2 Objective 15011.3.3 Definition of the Case 15011.3.4 Units of Analyses 15111.3.5 Theoretical Frame of Reference and Research Questions 15111.3.6 Case Selection 15111.3.7 Replication Strategy 15211.3.8 Case Study Protocol 15211.3.9 Quality Assurance, Validity, and Reliability 15211.3.10 Legal, Ethical, and Professional Considerations 15311.4 Data Collection 15411.4.1 Sources of Data 15411.5 Data Analysis 15711.6 Reporting 15911.6.1 Internal Reporting of Results 16011.6.2 Dissemination of Artifacts 16011.7 Lessons Learned 16012 AN ITERATIVE CASE STUDY OF QUALITY MONITORING 16312.1 Introduction 16312.2 Case Study Design 16412.2.1 Objectives 16412.2.2 Cases and Units of Analysis 16512.2.3 Theoretical Frame of Reference 16512.2.4 Research Questions 16512.3 Planning 16512.3.1 Methods of Data Collection 16512.3.2 Case Selection Strategy 16712.3.3 Case Study Protocol 16712.3.4 Ethical Considerations 16712.3.5 Data Collection 16812.3.6 Exploratory Study 16812.3.7 Confirmatory Study 16812.3.8 Explanatory Study 16812.4 Data Analysis 16912.5 Reporting 16912.6 Lessons Learned 16913 A CASE STUDY OF THE EVALUATION OF REQUIREMENTS MANAGEMENT TOOLS 17113.1 Introduction 17113.2 Design of the Case Study 17213.2.1 Rationale 17213.2.2 Objective 17213.2.3 The Case and Its Context 17313.2.4 The Units of Analyses 17413.2.5 Theoretical Framework 17513.2.6 Research Questions 17513.2.7 Propositions, Concepts, and Measures 17513.2.8 Case Study Protocol 17513.2.9 Methods of Data Collection 17613.2.10 Methods of Data Analysis 17613.2.11 Case Selection Strategy 17713.2.12 Data Selection Strategy 17713.2.13 Replication Strategy 17713.2.14 Quality Assurance, Validity, and Reliability 17713.3 Data Collection 17813.4 Data Analysis 17913.5 Reporting and Dissemination 18013.6 Lessons Learned 18114 A LARGE-SCALE CASE STUDY OF REQUIREMENTS AND VERIFICATION ALIGNMENT 18314.1 Introduction 18314.2 Case Study Design 18414.2.1 Rationale 18414.2.2 Objectives 18414.2.3 Cases and Units of Analysis 18514.2.4 Theoretical Frame of Reference 18614.2.5 Research Questions 18714.3 Planning 18814.3.1 Methods of Data Collection 18914.3.2 Case Selection Strategy 19014.3.3 Selection of Data 19114.3.4 Case Study Protocol 19114.3.5 Ethical Considerations 19214.4 Data Collection 19214.5 Data Analysis 19314.6 Lessons Learned 19514.6.1 Effort Estimation Lessons 19514.6.2 Design and Planning Lessons 19614.6.3 Data Collection Lessons 19714.6.4 Data Analysis Lessons 19814.6.5 Reporting Lessons 19914.6.6 A General Lesson 199EPILOGUE 201Appendix A: CHECKLISTS FOR READING AND REVIEWING CASE STUDIES 203A.1 Design of the Case Study 203A.2 Data Collection 204A.3 Data Analysis and Interpretation 204A.4 Reporting and Dissemination 204A.5 Reader’s Checklist 205Appendix B: EXAMPLE INTERVIEW INSTRUMENT (XP) 207Appendix C: EXAMPLE INTERVIEW INSTRUMENT (REVV) 209Appendix D: EXAMPLE OF A CODING GUIDE 213D.1 Coding Instructions 213D.2 Codes 214D.2.1 High Level Codes: Research Questions 214D.2.2 Medium Level Codes: Categories 216D.2.3 Coding Example 216Appendix E: EXAMPLE OF A CONSENT INFORMATION LETTER 219REFERENCES 221INDEX 235