Statistical Thinking for Non-Statisticians in Drug Regulation

Richard Kay, PhD is a Visiting Professor at the School of Pharmacy and Pharmaceutical Medicine, Cardiff University, UK, and a longtime statistical consultant for the pharmaceutical industry. He provides consultancy and training services for pharmaceutical companies and research institutions.

• Preface to the second edition, xvPreface to the first edition, xviiAbbreviations, xxi1             Basic ideas in clinical trial design, 11.1         Historical perspective, 11.2         Control groups, 21.3         Placebos and blinding, 31.4         Randomisation, 31.4.1      Unrestricted randomisation, 41.4.2      Block randomisation, 41.4.3      Unequal randomisation, 51.4.4      Stratified randomisation, 61.4.5      Central randomisation, 71.4.6      Dynamic allocation and minimisation, 81.4.7      Cluster randomisation, 91.5         Bias and precision, 91.6         Between- and within-patient designs, 111.7         Crossover trials, 121.8         Signal, noise and evidence, 131.8.1      Signal, 131.8.2      Noise, 131.8.3      Signal-to-noise ratio, 141.9         Confirmatory and exploratory trials, 151.10       Superiority, equivalence and non-inferiority trials, 161.11       Data and endpoint types, 171.12       Choice of endpoint, 181.12.1    Primary variables, 181.12.2    Secondary variables, 191.12.3    Surrogate variables, 201.12.4    Global assessment variables, 211.12.5    Composite variables, 211.12.6    Categorisation, 212             Sampling and inferential statistics, 232.1         Sample and population, 232.2         Sample statistics and population parameters, 242.2.1      Sample and population distribution, 242.2.2      Median and mean, 252.2.3      Standard deviation, 252.2.4      Notation, 262.2.5      Box plots, 272.3         The normal distribution, 282.4         Sampling and the standard error of the mean, 312.5         Standard errors more generally, 342.5.1      The standard error for the difference between two means, 342.5.2      Standard errors for proportions, 372.5.3      The general setting, 373             Confidence intervals and p-values, 383.1         Confidence intervals for a single mean, 383.1.1      The 95 per cent Confidence interval, 383.1.2      Changing the confidence coefficient, 403.1.3      Changing the multiplying constant, 403.1.4      The role of the standard error, 413.2         Confidence interval for other parameters, 423.2.1      Difference between two means, 423.2.2      Confidence interval for proportions, 433.2.3      General case, 443.2.4      Bootstrap Confidence interval, 453.3         Hypothesis testing, 453.3.1      Interpreting the p-value, 463.3.2      Calculating the p-value, 473.3.3      A common process, 503.3.4      The language of statistical significance, 533.3.5      One-sided and two-sided tests, 544             Tests for simple treatment comparisons, 564.1         The unpaired t-test, 564.2         The paired t-test, 574.3         Interpreting the t-tests, 604.4         The chi-square test for binary data, 614.4.1      Pearson chi-square, 614.4.2      The link to a ratio of the signal to the standard error, 644.5         Measures of treatment benefit, 644.5.1      Odds ratio, 654.5.2      Relative risk, 654.5.3      Relative risk reduction, 664.5.4      Number needed to treat, 664.5.5      Confidence intervals, 674.5.6      Interpretation, 684.6         Fisher’s exact test, 694.7         Tests for categorical and ordinal data, 714.7.1      Categorical data, 714.7.2      Ordered categorical (ordinal) data, 734.7.3      Measures of treatment benefit, 744.8         Extensions for multiple treatment groups, 754.8.1      Between-patient designs and continuous data, 754.8.2      Within-patient designs and continuous data, 764.8.3      Binary, categorical and ordinal data, 764.8.4      Dose-ranging studies, 774.8.5      Further discussion, 775             Adjusting the analysis, 785.1         Objectives for adjusted analysis, 785.2         Comparing treatments for continuous data, 785.3         Least squares means, 825.4         Evaluating the homogeneity of the treatment effect, 835.4.1      Treatment-by-factor interactions, 835.4.2      Quantitative and qualitative interactions, 855.5         Methods for binary, categorical and ordinal data, 865.6         Multi-centre trials, 875.6.1      Adjusting for centre, 875.6.2      Significant treatment-by-centre interactions, 875.6.3      Combining centres, 886             Regression and analysis of covariance, 896.1         Adjusting for baseline factors, 896.2         Simple linear regression, 896.3         Multiple regression, 916.4         Logistic regression, 946.5         Analysis of covariance for continuous data, 946.5.1      Main effect of treatment, 946.5.2      Treatment-by-covariate interactions, 966.5.3      A single model, 986.5.4      Connection with adjusted analyses, 986.5.5      Advantages of ANCOVA, 996.5.6      Least squares means, 1006.6         Binary, categorical and ordinal data, 1016.7         Regulatory aspects of the use of covariates, 1036.8         Baseline testing, 1057             Intention-to-treat and analysis sets, 1077.1         The principle of intention-to-treat, 1077.2         The practice of intention-to-treat, 1107.2.1      Full analysis set, 1107.2.2      Per-protocol set, 1127.2.3      Sensitivity, 1127.3         Missing data, 1137.3.1      Introduction, 1137.3.2      Complete cases analysis, 1147.3.3      Last observation carried forward, 1147.3.4      Success/failure classification, 1147.3.5      Worst-case/best-case classification, 1157.3.6      Sensitivity, 1157.3.7      Avoidance of missing data, 1167.3.8      Multiple imputation, 1177.4         Intention-to-treat and time-to-event data, 1187.5         General questions and considerations, 1208             Power and sample size, 1238.1         Type I and type II errors, 1238.2         Power, 1248.3         Calculating sample size, 1278.4         Impact of changing the parameters, 1308.4.1      Standard deviation, 1308.4.2      Event rate in the control group, 1308.4.3      Clinically relevant difference, 1318.5         Regulatory aspects, 1328.5.1      Power >80 per cent, 1328.5.2      Powering on the per-protocol set, 1328.5.3      Sample size adjustment, 1338.6         Reporting the sample size calculation, 1349             Statistical significance and clinical importance, 1369.1         Link between p-values and Confidence intervals, 1369.2         Confidence intervals for clinical importance, 1379.3         Misinterpretation of the p-value, 1399.3.1      Conclusions of similarity, 1399.3.2      The problem with 0.05, 1409.4         Single pivotal trial and 0.05, 14010           Multiple testing, 14210.1       Inflation of the type I error, 14210.1.1    False positives, 14210.1.2    A simulated trial, 14210.2       How does multiplicity arise?, 14310.3       Regulatory view, 14410.4       Multiple primary endpoints, 14510.4.1    Avoiding adjustment, 14510.4.2    Significance needed on all endpoints, 14510.4.3    Composite endpoints, 14610.4.4    Variables ranked according to clinical importance: Hierarchical testing, 14610.5       Methods for adjustment, 14910.5.1    Bonferroni correction, 14910.5.2    Hochberg correction, 15010.5.3    Interim analyses, 15110.6       Multiple comparisons, 15210.7       Repeated evaluation over time, 15310.8       Subgroup testing, 15410.9       Other areas for multiplicity, 15610.9.1    Using different statistical tests, 15610.9.2    Different analysis sets, 15610.9.3    Pre-planning, 15711           Non-parametric and related methods, 15811.1       Assumptions underlying the t-tests and their extensions, 15811.2       Homogeneity of variance, 15811.3       The assumption of normality, 15911.4       Non-normality and transformations, 16111.5       Non-parametric tests, 16411.5.1    The Mann–Whitney U-test, 16411.5.2    The Wilcoxon signed rank test, 16611.5.3    General comments, 16711.6       Advantages and disadvantages of non-parametric methods, 16811.7       Outliers, 16912           Equivalence and non-inferiority, 17012.1       Demonstrating similarity, 17012.2       Confidence intervals for equivalence, 17212.3       Confidence intervals for non-inferiority, 17312.4       A p-value approach, 17412.5       Assay sensitivity, 17612.6       Analysis sets, 17812.7       The choice of Δ, 17912.7.1    Bioequivalence, 17912.7.2    Therapeutic equivalence, 18012.7.3    Non-inferiority, 18012.7.4    The 10 per cent rule for cure rates, 18212.7.5    The synthesis method, 18312.8       Biocreep and constancy, 18412.9       Sample size calculations, 18412.10     Switching between non-inferiority and superiority, 18613           The analysis of survival data, 18913.1       Time-to-event data and censoring, 18913.2       Kaplan-Meier curves, 19013.2.1    Plotting Kaplan-Meier curves, 19013.2.2    Event rates and relative risk, 19213.2.3    Median event times, 19213.3       Treatment comparisons, 19313.4       The hazard ratio, 19613.4.1    The hazard rate, 19613.4.2    Constant hazard ratio, 19713.4.3    Non-constant hazard ratio, 19713.4.4    Link to survival curves, 19813.4.5    Calculating Kaplan-Meier curves, 19913.5       Adjusted analyses, 19913.5.1    Stratified methods, 20013.5.2    Proportional hazards regression, 20013.5.3    Accelerated failure time model, 20113.6       Independent censoring, 20213.7       Sample size calculations, 20314           Interim analysis and data monitoring committees, 20514.1       Stopping rules for interim analysis, 20514.2       Stopping for efficacy and futility, 20614.2.1    Efficacy, 20614.2.2    Futility and conditional power, 20714.2.3    Some practical issues, 20814.2.4    Analyses following completion of recruitment, 20914.3       Monitoring safety, 21014.4       Data monitoring committees, 21114.4.1    Introduction and responsibilities, 21114.4.2    Structure and process, 21214.4.3    Meetings and recommendations, 21415           Bayesian statistics, 21515.1       Introduction, 21515.2       Prior and posterior distributions, 21515.2.1    Prior beliefs, 21515.2.2    Prior to posterior, 21715.2.3    Bayes theorem, 21715.3       Bayesian inference, 21915.3.1    Frequentist methods, 21915.3.2    Posterior probabilities, 21915.3.3    Credible intervals, 22015.4       Case study, 22115.5       History and regulatory acceptance, 22215.6       Discussion, 22416           Adaptive designs, 22516.1       What are adaptive designs?, 22516.1.1    Advantages and drawbacks, 22516.1.2    Restricted adaptations, 22616.1.3    Flexible adaptations, 22716.2       Minimising bias, 22816.2.1    Control of type I error, 22816.2.2    Estimation, 22916.2.3    Behavioural issues, 23016.2.4    Exploratory trials, 23216.3       Unblinded sample size re-estimation, 23216.3.1    Product of p-values, 23216.3.2    Weighting the two parts of the trial, 23316.3.3    Rationale, 23416.4       Seamless phase II/III studies, 23416.4.1    Standard framework, 23416.4.2    Aspects of the p-value calculation, 23516.4.3    Logistical challenges, 23616.5       Other types of adaptation, 23616.5.1    Changing the primary endpoint, 23616.5.2    Focusing on a sub-population, 23716.5.3    Dropping the placebo arm in a non-inferiority trial, 23716.6       Further regulatory considerations, 23816.6.1    Impact on power, 23816.6.2    Non-standard experimental settings, 23917           Observational studies, 24117.1       Introduction, 24117.1.1    Non-randomised comparisons, 24117.1.2    Study types, 24117.1.3    Sources of bias, 24317.1.4    An empirical investigation, 24417.1.5    Selection bias in concurrently controlled studies: An empirical evaluation, 24517.1.6    Selection bias in historically controlled studies: An empirical evaluation, 24617.1.7    Some conclusions, 24617.2       Guidance on design, conduct and analysis, 24717.2.1    Regulatory guidance, 24717.2.2    Strengthening the Reporting of Observational Studies in Epidemiology, 24817.3       Evaluating and adjusting for selection bias, 24917.3.1    Baseline balance, 24917.3.2    Adjusting for imbalances using stratification and analysis of covariance, 25017.3.3    Propensity scores, 25017.3.4    Different methods for adjustment: An empirical evaluation, 25317.3.5    Some conclusions, 25617.4       Case–control studies, 25717.4.1    Background, 25717.4.2    Odds ratio and Relative risk, 25918           Meta-analysis, 26118.1       Definition, 26118.2       Objectives, 26318.3       Statistical methodology, 26418.3.1    Methods for combination, 26418.3.2    Confidence intervals, 26518.3.3    Fixed and random effects, 26518.3.4    Graphical methods, 26618.3.5    Detecting heterogeneity, 26618.3.6    Robustness, 26918.3.7    Rare events, 26918.3.8    Individual patient data, 26918.4       Case study, 27018.5       Ensuring scientific validity, 27118.5.1    Planning, 27118.5.2    Assessing the risk of bias, 27318.5.3    Publication bias and funnel plots, 27318.5.4    Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 27518.6       Further regulatory aspects, 27519           Methods for the safety analysis and safety monitoring, 27719.1       Introduction, 27719.1.1    Methods for safety data, 27719.1.2    The rule of three, 27819.2       Routine evaluation in clinical studies, 27919.2.1    Types of data, 28019.2.2    Adverse events, 28119.2.3    Laboratory data, 28419.2.4    ECG data, 28719.2.5    Vital signs, 28819.2.6    Safety summary across trials, 28819.2.7    Specific safety studies, 28919.3       Data monitoring committees, 28919.4       Assessing benefit–risk, 29019.4.1    Current approaches, 29019.4.2    Multi-criteria decision analysis, 29119.4.3    Quality-Adjusted Time without Symptoms or Toxicity, 29719.5       Pharmacovigilance, 29919.5.1    Post-approval safety monitoring, 29919.5.2    Proportional reporting ratios, 30019.5.3    Bayesian shrinkage, 30220           Diagnosis, 30420.1       Introduction, 30420.2       Measures of diagnostic performance, 30420.2.1    Sensitivity and specificity, 30420.2.2    Positive and negative predictive value, 30520.2.3    False positive and false negative rates, 30620.2.4    Prevalence, 30620.2.5    Likelihood ratio, 30720.2.6    Predictive accuracy, 30720.2.7    Choosing the correct cut-point, 30720.3       Receiver operating characteristic curves, 30820.3.1    Receiver operating characteristic, 30820.3.2    Comparing ROC curves, 30920.4       Diagnostic performance using regression models, 31020.5       Aspects of trial design for diagnostic agents, 31220.6       Assessing agreement, 31320.6.1    The kappa statistic, 31320.6.2    Other applications for kappa, 31421           The role of statistics and statisticians, 31621.1       The importance of statistical thinking at the design stage, 31621.2       Regulatory guidelines, 31721.3       The statistics process, 32121.3.1    The statistical methods section of the protocol, 32121.3.2    The statistical analysis plan, 32221.3.3    The data validation plan, 32221.3.4    The blind review, 32221.3.5    Statistical analysis, 32321.3.6    Reporting the analysis, 32321.3.7    Pre-planning, 32421.3.8    Sensitivity and robustness, 32621.4       The regulatory submission, 32721.5       Publications and presentations, 328References, 331Index, 339