Software Engineering

Based on their own experiences of in-depth case studies of software projects in international corporations, in this book the authors present detailed practical guidelines on the preparation, conduct, design and reporting of case studies of software engineering. This is the first software engineering specific book on the case study research method.

FOREWORD xiii PREFACE xv
ACKNOWLEDGMENTS xvii
PART I CASE STUDY METHODOLOGY
1 INTRODUCTION 3
1. 1 What is a Case Study? 3
1. 2 A Brief History of Case Studies in Software Engineering 5
1. 3 Why a Book on Case Studies of Software Engineering? 6
1. 4 Conclusion 9
2 BACKGROUND AND DEFINITION OF CONCEPTS 11
2. 1 Introduction 11
2. 2 Research Strategies 11
2. 3 Characteristics of Research Strategies 13
2. 3. 1 Purpose 13
2. 3. 2 Control and Data 14
2. 3. 3 Triangulation 15
2. 3. 4 Replication 16
2. 3. 5 Inductive and Deductive Enquiries 16
2. 4 What Makes a Good Case Study? 17
2. 5 When is the Case Study Strategy Feasible? 19
2. 6 Case Study Research Process 20
2. 7 Conclusion 21
3 DESIGN OF THE CASE STUDY 23
3. 1 Introduction 23
3. 2 Elements of the Case Study Design 24
3. 2. 1 Rationale for the Study 24
3. 2. 2 Objective of the Study 24
3. 2. 3 Cases and Units of Analyses 26
3. 2. 4 Theoretical Framework 29
3. 2. 5 Research Questions 30
3. 2. 6 Propositions and Hypotheses 31
3. 2. 7 Concepts 32
3. 2. 8 Methods of Data Collection 32
3. 2. 9 Methods of Data Analysis 33
3. 2. 10 Case Selection 33
3. 2. 11 Selection of Data 35
3. 2. 12 Data Definition and Data Storage 36
3. 2. 13 Quality Control and Assurance 36
3. 2. 14 Maintaining the Case Study Protocol 37
3. 2. 15 Reporting and Disseminating the Case Study 38
3. 3 Legal, Ethical, and Professional Issues 40
3. 4 Conclusion 45
4 DATA COLLECTION 47
4. 1 Introduction 47
4. 2 Different Types of Data Source 47
4. 2. 1 Classification of Data Sources 47
4. 2. 2 Data Source Selection 49
4. 3 Interviews 50
4. 3. 1 Planning Interviews 50
4. 3. 2 The Interview Session 52
4. 3. 3 Postinterview Activities 53
4. 4 Focus groups 54
4. 5 Observations 56
4. 6 Archival Data 57
4. 7 Metrics 58
4. 8 Conclusion 60
5 DATA ANALYSIS AND INTERPRETATION 61
5. 1 Introduction 61
5. 2 Analysis of Data in Flexible Research 62
5. 2. 1 Introduction 62
5. 2. 2 Level of Formalism 64
5. 2. 3 Relation to Hypotheses 65
5. 3 Process for Qualitative Data Analysis 65
5. 3. 1 Introduction 65
5. 3. 2 Steps in the Analysis 66
5. 3. 3 Techniques 68
5. 3. 4 Tool support 70
5. 4 Validity 71
5. 4. 1 Construct Validity 71
5. 4. 2 Internal Validity 71
5. 4. 3 External Validity 71
5. 4. 4 Reliability 72
5. 5 Improving Validity 72
5. 6 Quantitative Data Analysis 74
5. 7 Conclusion 76
6 REPORTING AND DISSEMINATION 77
6. 1 Introduction 77
6. 2 Why Report and Disseminate 78
6. 3 The Audience for the Report 79
6. 4 Aspects of the Case Study to Report and Disseminate 80
6. 5 When to Report and Disseminate 81
6. 6 Guidelines on Reporting 82
6. 6. 1 The Generic Content of an Academic Report 82
6. 6. 2 Reporting Recommendations from Evaluative Case Studies 84
6. 6. 3 Reporting to Stakeholders, Including Sponsor(s) 85
6. 6. 4 Reporting the Context of the Case Study 87
6. 6. 5 Reporting to Students 89
6. 6. 6 Ad Hoc and Impromptu Reporting 90
6. 7 Formats and Structures for a Report 91
6. 8 Where to Report 94
6. 9 Ethics and Confidentiality 94
6. 10 Conclusion 95
7 SCALING UP CASE STUDY RESEARCH TO REAL-WORLD SOFTWARE PRACTICE 97
7. 1 Introduction 97
7. 2 The Aims of Scaling up Case Studies 98
7. 3 Dimensions of Scale 99
7. 4 Longitudinal Case Studies 100
7. 5 Multiple Case Studies 102
7. 5. 1 Multiple Cases and Replications 102
7. 5. 2 Selecting the Cases 104
7. 6 Multiresearcher Case Studies 105
7. 7 Conclusion 107
8 USING CASE STUDY RESEARCH 109
8. 1 Introduction 109
8. 2 Reading and Reviewing Case Studies 109
8. 2. 1 Development of Checklists 110
8. 2. 2 Checklists for Conducting Case Study Research 111
8. 2. 3 Checklists for Reading and Reviewing Case Studies 111
8. 2. 4 Development of Practice 111
8. 3 Identifying and Synthesizing Use Case Research 111
8. 3. 1 Identifying Primary Studies 112
8. 3. 2 Synthesis of Evidence from Multiple Case Studies 113
8. 3. 3 Current State of Synthesis 117
8. 4 The Economics of Case Study Research 118
8. 4. 1 Costs and Benefits of Evaluation Techniques 119
8. 4. 2 Evaluation of the DESMET Methodology 119
8. 4. 3 Frameworks for Organizing Methods of Evaluation 119
8. 5 Specializing Case Study Research for Software Engineering 121
8. 5. 1 The Longitudinal Chronological Case Study Research Strategy 122
8. 5. 2 Controlled Case Studies 123
8. 6 Case Studies and Software Process Improvement 123
8. 7 Conclusion 125
PART II EXAMPLES OF CASE STUDIES
9 INTRODUCTION TO CASE STUDY EXAMPLES 129
9. 1 Introduction 129
10 CASE STUDY OF EXTREME PROGRAMMING IN A STAGE-GATE CONTEXT 133
10. 1 Introduction 133
10. 1. 1 Methodological Status 133
10. 2 Case Study Design 134
10. 2. 1 Rationale 134
10. 2. 2 Objectives 134
10. 2. 3 Cases and Units of Analysis 135
10. 2. 4 Theoretical Frame of Reference 136
10. 2. 5 Research Questions 136
10. 3 Planning 136
10. 3. 1 Methods of Data Collection 136
10. 3. 2 Selection of Data 137
10. 3. 3 Case Selection Strategy 137
10. 3. 4 Case Study Protocol 137
10. 3. 5 Ethical Considerations 137
10. 4 Data Collection 139
10. 5 Data Analysis 139
10. 5. 1 Threats to Validity 144
10. 6 Reporting 144
10. 6. 1 Academics 144
10. 6. 2 Practitioners 144
10. 7 Lessons Learned 146
11 TWO LONGITUDINAL CASE STUDIES OF SOFTWARE PROJECT MANAGEMENT 149
11. 1 Introduction 149
11. 2 Background to the Research Project 149
11. 3 Case Study Design and Planning 150
11. 3. 1 Rationale 150
11. 3. 2 Objective 150
11. 3. 3 Definition of the Case 150
11. 3. 4 Units of Analyses 151
11. 3. 5 Theoretical Frame of Reference and Research Questions 151
11. 3. 6 Case Selection 151
11. 3. 7 Replication Strategy 152
11. 3. 8 Case Study Protocol 152
11. 3. 9 Quality Assurance, Validity, and Reliability 152
11. 3. 10 Legal, Ethical, and Professional Considerations 153
11. 4 Data Collection 154
11. 4. 1 Sources of Data 154
11. 5 Data Analysis 157
11. 6 Reporting 159
11. 6. 1 Internal Reporting of Results 160
11. 6. 2 Dissemination of Artifacts 160
11. 7 Lessons Learned 160
12 AN ITERATIVE CASE STUDY OF QUALITY MONITORING 163
12. 1 Introduction 163
12. 2 Case Study Design 164
12. 2. 1 Objectives 164
12. 2. 2 Cases and Units of Analysis 165
12. 2. 3 Theoretical Frame of Reference 165
12. 2. 4 Research Questions 165
12. 3 Planning 165
12. 3. 1 Methods of Data Collection 165
12. 3. 2 Case Selection Strategy 167
12. 3. 3 Case Study Protocol 167
12. 3. 4 Ethical Considerations 167
12. 3. 5 Data Collection 168
12. 3. 6 Exploratory Study 168
12. 3. 7 Confirmatory Study 168
12. 3. 8 Explanatory Study 168
12. 4 Data Analysis 169
12. 5 Reporting 169
12. 6 Lessons Learned 169
13 A CASE STUDY OF THE EVALUATION OF REQUIREMENTS MANAGEMENT TOOLS 171
13. 1 Introduction 171
13. 2 Design of the Case Study 172
13. 2. 1 Rationale 172
13. 2. 2 Objective 172
13. 2. 3 The Case and Its Context 173
13. 2. 4 The Units of Analyses 174
13. 2. 5 Theoretical Framework 175
13. 2. 6 Research Questions 175
13. 2. 7 Propositions, Concepts, and Measures 175
13. 2. 8 Case Study Protocol 175
13. 2. 9 Methods of Data Collection 176
13. 2. 10 Methods of Data Analysis 176
13. 2. 11 Case Selection Strategy 177
13. 2. 12 Data Selection Strategy 177
13. 2. 13 Replication Strategy 177
13. 2. 14 Quality Assurance, Validity, and Reliability 177
13. 3 Data Collection 178
13. 4 Data Analysis 179
13. 5 Reporting and Dissemination 180
13. 6 Lessons Learned 181
14 A LARGE-SCALE CASE STUDY OF REQUIREMENTS AND VERIFICATION ALIGNMENT 183
14. 1 Introduction 183
14. 2 Case Study Design 184
14. 2. 1 Rationale 184
14. 2. 2 Objectives 184
14. 2. 3 Cases and Units of Analysis 185
14. 2. 4 Theoretical Frame of Reference 186
14. 2. 5 Research Questions 187
14. 3 Planning 188
14. 3. 1 Methods of Data Collection 189
14. 3. 2 Case Selection Strategy 190
14. 3. 3 Selection of Data 191
14. 3. 4 Case Study Protocol 191
14. 3. 5 Ethical Considerations 192
14. 4 Data Collection 192
14. 5 Data Analysis 193
14. 6 Lessons Learned 195
14. 6. 1 Effort Estimation Lessons 195
14. 6. 2 Design and Planning Lessons 196
14. 6. 3 Data Collection Lessons 197
14. 6. 4 Data Analysis Lessons 198
14. 6. 5 Reporting Lessons 199
14. 6. 6 A General Lesson 199
EPILOGUE 201
Appendix A: CHECKLISTS FOR READING AND REVIEWING CASE STUDIES 203
A. 1 Design of the Case Study 203
A. 2 Data Collection 204
A. 3 Data Analysis and Interpretation 204
A. 4 Reporting and Dissemination 204
A. 5 Reader's Checklist 205
Appendix B: EXAMPLE INTERVIEW INSTRUMENT (XP) 207
Appendix C: EXAMPLE INTERVIEW INSTRUMENT (REVV) 209
Appendix D: EXAMPLE OF A CODING GUIDE 213
D. 1 Coding Instructions 213
D. 2 Codes 214
D. 2. 1 High Level Codes: Research Questions 214
D. 2. 2 Medium Level Codes: Categories 216
D. 2. 3 Coding Example 216
Appendix E: EXAMPLE OF A CONSENT INFORMATION LETTER 219
REFERENCES 221
INDEX 235