Improving Surveys with Paradata

FRAUKE KREUTER is Associate Professor in the Joint Program in Survey Methodology at the University of Maryland; Professor of Statistics at Ludwig Maximilian University of Munich, Germany; and head of the Statistical Methods Research Department at the Institute for Employment Research (IAB) in Nuremberg, Germany.

• 1 Improving Surveys with Paradata: Introduction 1Frauke Kreuter 1.1 Introduction 1 1.2 Paradata and Metadata 31.3 Auxiliary Data and Paradata 41.4 Paradata in the Total Survey Error Framework 41.5 Paradata in Survey Production 51.6 Special Challenges in the Collection and Use of Paradata 71.7 Future of Paradata 8PART I PARADATA AND SURVEY ERRORS2 Paradata for Nonresponse Error Investigation 3Frauke Kreuter and Kristen Olson2.1 Introduction 32.2 Sources of Paradata 42.3 Nonresponse Rates and Nonresponse Bias 102.4 Paradata and Responsive Designs 202.5 Paradata and Nonresponse Adjustment 212.6 Issues in Practice 222.7 Summary and Take Home Messages 243 Collecting Paradata for Measurement Error Evaluations 33Kristen Olson and Bryan Parkhurst 3.1 Introduction 33 3.2 Paradata and Measurement Error 343.3 Types of paradata 383.4 Differences in Paradata by Modes 453.5 Turning paradata into data sets 513.6 Summary 55 4 Analyzing Paradata to Investigate Measurement Error 63Ting Yan and Kristen Olson4.1 Introduction 634.2 Review of Empirical Literature on the Use of Paradata for Measurement Error Investigation 644.3 Analyzing paradata 664.4 Four empirical examples 734.5 Cautions 814.6 Concluding Remarks 825 Paradata for Coverage Research 89Stephanie Eckman5.1 Introduction 895.2 Housing Unit Frames 935.3 Telephone Number Frames 1015.4 Household Rosters 1035.5 Population Registers 1055.6 Subpopulation Frames 1065.7 Web Surveys 1065.8 Conclusion 107PART II PARADATA IN SURVEY PRODUCTION6 Design and Management Strategies for Paradata-Driven Responsive Design 117Nicole G. Kirgis and James M. Lepkowski6.1 Introduction 1176.2 From Repeated Cross-Section to Continuous Design 1186.3 Paradata Design 1236.4 Key Design Change 1: A New Employment Model 1286.5 Key Design Change 2: Field Efficient Sample Design 1306.6 Key Design Change 3: Replicate Sample Design 1316.7 Key Design Change 4: Responsive Design Sampling of Nonrespondents in a Second Phase 1326.8 Key Design Change 5: Active Responsive Design Interventions 1346.9 Concluding Remarks 1357 Using Paradata-Driven Models to Improve Contact Rates 141James Wagner7.1 Introduction 1417.2 Background 1427.3 The Survey Setting 1447.4 Experiments: Data and Methods 1457.5 Experiments: Results 1577.6 Discussion 1628 Using Paradata to Study Response to Within-Survey Requests 169Joseph W. Sakshaug8.1 Introduction 1698.2 Consent to Link Survey and Administrative Records 1738.3 Consent to Collect Biomeasures in Population-Based Surveys 1778.4 Switching Data Collection Modes 1798.5 Income Item Nonresponse and Quality of Income Reports 1818.6 Summary 1859 Managing Data Quality Indicators with Paradata-Based Statistical Quality Control Tools 191Matt Jans, Robyn Sirkis and David Morgan9.1 Introduction 1919.2 Defining and Choosing Key Performance Indicators (KPIs) 1939.3 KPI Displays and the Enduring Insight of Walter Shewhart 2019.4 Implementation Steps for Survey Analytic Quality Control with Paradata Control Charts 2129.5 A Method for Improving Measurement Process Quality Indicators 2149.6 Reections on SPC, Visual Data Displays, and Challenges to Quality Control 2219.7 Some Advice on Using Charts 223Appendix 22510 Paradata as Input to Monitoring Representativeness and Measurement Profiles 233Barry Schouten and Melania Calinescu10.1 Introduction 23310.2 Measurement profiles 23510.3 Tools for monitoring nonresponse and measurement profiles 23810.4 Monitoring and improving response: a demonstration using the LFS 24310.5 Including paradata observations on households and persons 25410.6 General discussion 25610.7 Take home messages 257PART III SPECIAL CHALLENGES11 Paradata in Web Surveys 263Mario Callegaro11.1 Survey data types 26311.2 Collection of paradata 26411.3 Typology of paradata in web surveys 26511.4 Using paradata to change the survey in real time: adaptive scripting 27311.5 Paradata in online panels 27411.6 Software to collect paradata 27411.7 Analysis of paradata: levels of aggregation 27511.8 Privacy and ethical issues in collecting web survey paradata 27611.9 Summary and conclusions on paradata in web surveys 27712 Modeling Call Record Data: Examples from Cross-Sectional and Longitudinal Surveys 283Gabriele B. Durrant, Julia D'Arrigo and Gerrit Müller12.1 Introduction 28312.2 Call record data 28512.3 Modeling approaches 28712.4 Illustration of call record data analysis using two example datasets 29412.5 Summary 30513 Bayesian Penalized Spline Models for Statistical Process Monitoring of Survey Paradata Quality Indicators 311Joseph L. Schafer13.1 Introduction 31113.2 Overview of splines 31613.3 Penalized splines as linear mixed models 32313.4 Bayesian methods 32713.5 Extensions 33014 The Quality of Paradata: A Literature Review 341Brady T. West and Jennifer Sinibaldi14.1 Introduction 34114.2 Existing Studies Examining the Quality of Paradata 34214.3 Possible Mechanisms Leading to Error in Paradata 35414.4 Take Home Messages 35715 The Effects of Errors in Paradata on Weighting Class Adjustments: A Simulation Study 363Brady T. West15.1 Introduction 36315.2 Design of Simulation Studies 36715.3 Simulation Results 37215.4 Take Home Messages 38615.5 Future Research 388Topic Index 393