Cochrane Handbook for Systematic Reviews of Interventions

Julian P. T. Higgins is Professor of Evidence Synthesis at Bristol Medical School, University of Bristol, UK. He has worked in methods for systematic review and meta-analysis for over 25 years and acts as Senior Methods Advisor to Cochrane. James Thomas is Professor of Social Research and Policy at the EPPI-Centre, UCL Institute of Education. He has broad interests in systematic review methodology and tools and is Director of the Systematic Reviews Facility for the Department of Health, England.

• Contributors xiiiPreface xxiiiPart One Core methods 11 Starting a review 31.1 Why do a systematic review? 31.2 What is the review question? 41.3 Who should do a systematic review? 51.4 The importance of reliability 71.5 Protocol development 81.6 Data management and quality assurance 111.7 Chapter information 121.8 References 122 Determining the scope of the review and the questions it will address 132.1 Rationale for well-formulated questions 132.2 Aims of reviews of interventions 152.3 Defining the scope of a review question 162.4 Ensuring the review addresses the right questions 212.5 Methods and tools for structuring the review 242.6 Chapter information 292.7 References 293 Defining the criteria for including studies and how they will be grouped for the synthesis 333.1 Introduction 333.2 Articulating the review and comparison PICO 353.3 Determining which study designs to include 513.4 Eligibility based on publication status and language 603.5 Chapter information 613.6 References 614 Searching for and selecting studies 674.1 Introduction 684.2 General issues 684.3 Sources to search 704.4 Designing search strategies 794.5 Documenting and reporting the search process 904.6 Selecting studies 924.7 Chapter information 994.8 References 995 Collecting data 1095.1 Introduction 1095.2 Sources of data 1105.3 What data to collect 1145.4 Data collection tools 1255.5 Extracting data from reports 1305.6 Extracting study results and converting to the desired format 1365.7 Managing and sharing data 1365.8 Chapter information 1375.9 References 1376 Choosing effect measures and computing estimates of effect 1436.1 Types of data and effect measures 1436.2 Study designs and identifying the unit of analysis 1456.3 Extracting estimates of effect directly 1486.4 Dichotomous outcome data 1506.5 Continuous outcome data 1566.6 Ordinal outcome data and measurement scales 1686.7 Count and rate data 1706.8 Time-to-event data 1726.9 Conditional outcomes only available for subsets of participants 1736.10 Chapter information 1746.11 References 1747 Considering bias and conflicts of interest among the included studies 1777.1 Introduction 1777.2 Empirical evidence of bias 1807.3 General procedures for risk-of-bias assessment 1857.4 Presentation of assessment of risk of bias 1887.5 Summary assessments of risk of bias 1887.6 Incorporating assessment of risk of bias into analyses 1907.7 Considering risk of bias due to missing results 1927.8 Considering source of funding and conflict of interest of authors of included studies 1937.9 Chapter information 1997.10 References 1998 Assessing risk of bias in a randomized trial 2058.1 Introduction 2058.2 Overview of RoB 2 2068.3 Bias arising from the randomization process 2128.4 Bias due to deviations from intended interventions 2148.5 Bias due to missing outcome data 2178.6 Bias in measurement of the outcome 2208.7 Bias in selection of the reported result 2218.8 Differences from the previous version of the tool 2258.9 Chapter information 2268.10 References 2279 Summarizing study characteristics and preparing for synthesis 2299.1 Introduction 2299.2 A general framework for synthesis 2309.3 Preliminary steps of a synthesis 2319.4 Checking data before synthesis 2389.5 Types of synthesis 2389.6 Chapter information 2409.7 References 24010 Analysing data and undertaking meta-analyses 24110.1 Do not start here! 24210.2 Introduction to meta-analysis 24210.3 A generic inverse-variance approach to meta-analysis 24510.4 Meta-analysis of dichotomous outcomes 24610.5 Meta-analysis of continuous outcomes 25110.6 Combining dichotomous and continuous outcomes 25410.7 Meta-analysis of ordinal outcomes and measurement scales 25510.8 Meta-analysis of counts and rates 25510.9 Meta-analysis of time-to-event outcomes 25610.10 Heterogeneity 25710.11 Investigating heterogeneity 26510.12 Missing data 27210.13 Bayesian approaches to meta-analysis 27610.14 Sensitivity analyses 27710.15 Chapter information 27910.16 References 28011 Undertaking network meta-analyses 28511.1 What is network meta-analysis? 28511.2 Important concepts 28711.3 Planning a Cochrane Review to compare multiple interventions 29311.4 Synthesis of results 29711.5 Evaluating confidence in the results of a network meta-analysis 30411.6 Presenting network meta-analyses 30911.7 Concluding remarks 31511.8 Chapter information 31611.9 References 31612 Synthesizing and presenting findings using other methods 32112.1 Why a meta-analysis of effect estimates may not be possible 32112.2 Statistical synthesis when meta-analysis of effect estimates is not possible 32412.3 Visual display and presentation of the data 33012.4 Worked example 33312.5 Chapter information 34512.6 References 34613 Assessing risk of bias due to missing results in a synthesis 34913.1 Introduction 35013.2 Minimizing risk of bias due to missing results 35113.3 A framework for assessing risk of bias due to missing results in a synthesis 35413.4 Summary 36913.5 Chapter information 37013.6 References 37014 Completing ‘Summary of findings’ tables and grading the certainty of the evidence 37514.1 ‘Summary of findings’ tables 37514.2 Assessing the certainty or quality of a body of evidence 38914.3 Describing the assessment of the certainty of a body of evidence using the GRADE framework 39814.4 Chapter information 39914.5 References 39915 Interpreting results and drawing conclusions 40315.1 Introduction 40315.2 Issues of indirectness and applicability 40515.3 Interpreting results of statistical analyses 40815.4 Interpreting results from dichotomous outcomes (including numbers needed to treat) 41115.5 Interpreting results from continuous outcomes (including standardized mean differences) 41615.6 Drawing conclusions 42215.7 Chapter information 42715.8 References 428Part Two Specific perspectives in reviews 43316 Equity and specific populations 43516.1 Introduction to equity in systematic reviews 43516.2 Formulation of the review 43716.3 Identification of evidence 44116.4 Appraisal of evidence 44316.5 Synthesis of evidence 44316.6 Interpretation of evidence 44416.7 Concluding remarks 44516.8 Chapter information 44516.9 References 44517 Intervention complexity 45117.1 Introduction 45117.2 Formulation of the review 46117.3 Identification of evidence 46817.4 Appraisal of evidence 46917.5 Synthesis of evidence 46917.6 Interpretation of evidence 47217.7 Chapter information 47317.8 References 47418 Patient-reported outcomes 47918.1 Introduction to patient-reported outcomes 47918.2 Formulation of the review 48018.3 Appraisal of evidence 48218.4 Synthesis and interpretation of evidence 48518.5 Chapter information 48818.6 References 48919 Adverse effects 49319.1 Introduction to issues in addressing adverse effects 49319.2 Formulation of the review 49619.3 Identification of evidence 50019.4 Appraisal of evidence 50219.5 Synthesis and interpretation of evidence 50319.6 Chapter information 50419.7 References 50520 Economic evidence 50720.1 Introduction 50720.2 Formulation of the review 51220.3 Identification of evidence 51720.4 Appraisal of evidence 51920.5 Synthesis and interpretation of evidence 51920.6 Chapter information 52120.7 References 52221 Qualitative evidence 52521.1 Introduction 52521.2 Designs for synthesizing and integrating qualitative evidence with intervention reviews 52621.3 Defining qualitative evidence and studies 52721.4 Planning a qualitative evidence synthesis linked to an intervention review 52821.5 Question development 52921.6 Questions exploring intervention implementation 53021.7 Searching for qualitative evidence 53121.8 Assessing methodological strengths and limitations of qualitative studies 53221.9 Selecting studies to synthesize 53321.10 Selecting a qualitative evidence synthesis and data extraction method 53421.11 Data extraction 53421.12 Assessing the confidence in qualitative synthesized findings 53721.13 Methods for integrating the qualitative evidence synthesis with an intervention review 53721.14 Reporting the protocol and qualitative evidence synthesis 53821.15 Chapter information 53921.16 References 539Part Three Further topics 54722 Prospective approaches to accumulating evidence 54922.1 Introduction 54922.2 Evidence surveillance: active monitoring of the accumulating evidence 55022.3 Prospectively planned meta-analysis 55422.4 Statistical analysis of accumulating evidence 56122.5 Chapter information 56422.6 References 56523 Including variants on randomized trials 56923.1 Cluster-randomized trials 56923.2 Crossover trials 57623.3 Studies with more than two intervention groups 58523.4 Chapter information 59023.5 References 59124 Including non-randomized studies on intervention effects 59524.1 Introduction 59524.2 Developing criteria for including non-randomized studies of interventions 60124.3 Searching for non-randomized studies of interventions 60624.4 Selecting studies and collecting data 60824.5 Assessing risk of bias in non-randomized studies 61024.6 Synthesis of results from non-randomized studies 61124.7 Interpretation and discussion 61424.8 Chapter information 61724.9 References 61725 Assessing risk of bias in a non-randomized study 62125.1 Introduction 62225.2 Biases in non-randomized studies 62325.3 The ROBINS-I tool 62625.4 Risk of bias in follow-up (cohort) studies 63225.5 Risk of bias in uncontrolled before-after studies (including interrupted time series) 63525.6 Risk of bias in controlled before-after studies 63825.7 Chapter information 64025.8 References 64026 Individual participant data 64326.1 Introduction 64326.2 Collecting IPD 64726.3 Managing and checking IPD 65026.4 Analysis of IPD 65226.5 Reporting IPD reviews 65526.6 Appraising the quality of IPD reviews 65526.7 Chapter information 65526.8 References 655Index 659