Agricultural Survey Methods

R. Benedetti, Department of Economics, University of Trento, Italia. M Bee,?Department of Economics, University of Trento, Italia. G Espa, Department of Economics, University of Trento, Italia. F Piersimoni, Italian Central Bureau of Statistics, Italy.

"All over the world, agricultural surveys are conducted to gather a large amount of information on the classic crops, yields, livestock, and other agricultural resources. The survey and analysis methods have tended to be locally devised to meet local or national conditions, cultures, and goals, but over the past few years, efforts have been made to establish methods that would allow comparison and evaluation across national and cultural boundaries. A summary of that effort is provided here in 22 methodology papers selected from presentations at the International Conference on Agricultural Statistics in 1998, 2001, 2004, and 2007. They address issues in census, frames, registers, and administrative data; sample design, weighting, and estimation; geographical information systems and remote sensing; data editing and quality assurance; and data dissemination and survey data analysis. Mathematicians and economists looking toward agriculture, agricultural scientists looking at statistics, and researchers and policy-making looking at the intersection could all find the volume to be a valuable reference." (SciTech Book News, December 2010)

• List of Contributors xviiIntroduction xxi1 The present state of agricultural statistics in developed countries: situation and challenges 11.1 Introduction 11.2 Current state and political and methodological context 41.2.1 General 41.2.2 Specific agricultural statistics in the UNECE region 61.3 Governance and horizontal issues 151.3.1 The governance of agricultural statistics 151.3.2 Horizontal issues in the methodology of agricultural statistics 161.4 Development in the demand for agricultural statistics 201.5 Conclusions 22Acknowledgements 23Reference 24Part I Census, Frames, Registers and Administrative Data 252 Using administrative registers for agricultural statistics 272.1 Introduction 272.2 Registers, register systems and methodological issues 282.3 Using registers for agricultural statistics 292.3.1 One source 292.3.2 Use in a farm register system 302.3.3 Use in a system for agricultural statistics linked with the business register 302.4 Creating a farm register: the population 342.5 Creating a farm register: the statistical units 382.6 Creating a farm register: the variables 422.7 Conclusions 44References 443 Alternative sampling frames and administrative data. What is the best data source for agricultural statistics? 453.1 Introduction 453.2 Administrative data 463.3 Administrative data versus sample surveys 463.4 Direct tabulation of administrative data 463.4.1 Disadvantages of direct tabulation of administrative data 473.5 Errors in administrative registers 483.5.1 Coverage of administrative registers 483.6 Errors in administrative data 493.6.1 Quality control of the IACS data 493.6.2 An estimate of errors of commission and omission in the IACS data 503.7 Alternatives to direct tabulation 513.7.1 Matching different registers 513.7.2 Integrating surveys and administrative data 523.7.3 Taking advantage of administrative data for censuses 523.7.4 Updating area or point sampling frames with administrative data 533.8 Calibration and small-area estimators 533.9 Combined use of different frames 543.9.1 Estimation of a total 553.9.2 Accuracy of estimates 553.9.3 Complex sample designs 563.10 Area frames 573.10.1 Combining a list and an area frame 573.11 Conclusions 58Acknowledgements 59References 604 Statistical aspects of a census 634.1 Introduction 634.2 Frame 644.2.1 Coverage 644.2.2 Classification 644.2.3 Duplication 654.3 Sampling 654.4 Non-sampling error 664.4.1 Response error 664.4.2 Non-response 674.5 Post-collection processing 684.6 Weighting 684.7 Modelling 694.8 Disclosure avoidance 694.9 Dissemination 704.10 Conclusions 71References 715 Using administrative data for census coverage 735.1 Introduction 735.2 Statistics Canada’s agriculture statistics programme 745.3 1996 Census 755.4 Strategy to add farms to the farm register 755.4.1 Step 1: Match data from E to M 765.4.2 Step 2: Identify potential farm operations among the unmatched records from E 765.4.3 Step 3: Search for the potential farms from E on M 765.4.4 Step 4: Collect information on the potential farms 775.4.5 Step 5: Search for the potential farms with the updated key identifiers 775.5 2001 Census 775.5.1 2001 Farm Coverage Follow-up 775.5.2 2001 Coverage Evaluation Study 775.6 2006 Census 785.6.1 2006 Missing Farms Follow-up 795.6.2 2006 Coverage Evaluation Study 805.7 Towards the 2011 Census 815.8 Conclusions 81Acknowledgements 83References 83Part II Sample Design, Weighting and Estimation 856 Area sampling for small-scale economic units 876.1 Introduction 876.2 Similarities and differences from household survey design 886.2.1 Probability proportional to size selection of area units 886.2.2 Heterogeneity 906.2.3 Uneven distribution 906.2.4 Integrated versus separate sectoral surveys 906.2.5 Sampling different types of units in an integrated design 916.3 Description of the basic design 916.4 Evaluation criterion: the effect of weights on sampling precision 936.4.1 The effect of ‘random’ weights 936.4.2 Computation of D2 from the frame 946.4.3 Meeting sample size requirements 946.5 Constructing and using ‘strata of concentration’ 956.5.1 Concept and notation 956.5.2 Data by StrCon and sector (aggregated over areas) 956.5.3 Using StrCon for determining the sampling rates: a basic model 976.6 Numerical illustrations and more flexible models 976.6.1 Numerical illustrations 976.6.2 More flexible models: an empirical approach 1006.7 Conclusions 104Acknowledgements 105References 1057 On the use of auxiliary variables in agricultural survey design 1077.1 Introduction 1077.2 Stratification 1097.3 Probability proportional to size sampling 1137.4 Balanced sampling 1167.5 Calibration weighting 1187.6 Combining ex ante and ex post auxiliary information: a simulated approach 1247.7 Conclusions 128References 1298 Estimation with inadequate frames 1338.1 Introduction 1338.2 Estimation procedure 1338.2.1 Network sampling 1338.2.2 Adaptive sampling 135References 1389 Small-area estimation with applications to agriculture 1399.1 Introduction 1399.2 Design issues 1409.3 Synthetic and composite estimates 1409.3.1 Synthetic estimates 1419.3.2 Composite estimates 1419.4 Area-level models 1429.5 Unit-level models 1449.6 Conclusions 146References 147Part III GIS and Remote Sensing 14910 The European land use and cover area-frame statistical survey 15110.1 Introduction 15110.2 Integrating agricultural and environmental information with LUCAS 15410.3 LUCAS 2001–2003: Target region, sample design and results 15510.4 The transect survey in LUCAS 2001–2003 15610.5 LUCAS 2006: a two-phase sampling plan of unclustered points 15810.6 Stratified systematic sampling with a common pattern of replicates 15910.7 Ground work and check survey 15910.8 Variance estimation and some results in LUCAS 2006 16010.9 Relative efficiency of the LUCAS 2006 sampling plan 16110.10 Expected accuracy of area estimates with the LUCAS 2006 scheme 16310.11 Non-sampling errors in LUCAS 2006 16410.11.1 Identification errors 16410.11.2 Excluded areas 16410.12 Conclusions 165Acknowledgements 166References 16611 Area frame design for agricultural surveys 16911.1 Introduction 16911.1.1 Brief history 17011.1.2 Advantages of using an area frame 17111.1.3 Disadvantages of using an area frame 17111.1.4 How the NASS uses an area frame 17211.2 Pre-construction analysis 17311.3 Land-use stratification 17611.4 Sub-stratification 17811.5 Replicated sampling 18011.6 Sample allocation 18311.7 Selection probabilities 18511.7.1 Equal probability of selection 18611.7.2 Unequal probability of selection 18711.8 Sample selection 18811.8.1 Equal probability of selection 18811.8.2 Unequal probability of selection 18811.9 Sample rotation 18911.10 Sample estimation 19011.11 Conclusions 19212 Accuracy, objectivity and efficiency of remote sensing for agricultural statistics 19312.1 Introduction 19312.2 Satellites and sensors 19412.3 Accuracy, objectivity and cost-efficiency 19512.4 Main approaches to using EO for crop area estimation 19612.5 Bias and subjectivity in pixel counting 19712.6 Simple correction of bias with a confusion matrix 19712.7 Calibration and regression estimators 19712.8 Examples of crop area estimation with remote sensing in large regions 19912.8.1 US Department of Agriculture 19912.8.2 Monitoring agriculture with remote sensing 20012.8.3 India 20012.9 The GEOSS best practices document on EO for crop area estimation 20012.10 Sub-pixel analysis 20112.11 Accuracy assessment of classified images and land cover maps 20112.12 General data and methods for yield estimation 20312.13 Forecasting yields 20312.14 Satellite images and vegetation indices for yield monitoring 20412.15 Examples of crop yield estimation/forecasting with remote sensing 20512.15.1 USDA 20512.15.2 Global Information and Early Warning System 20612.15.3 Kansas Applied Remote Sensing 20712.15.4 MARS crop yield forecasting system 207References 20713 Estimation of land cover parameters when some covariates are missing 21313.1 Introduction 21313.2 The AGRIT survey 21413.2.1 Sampling strategy 21413.2.2 Ground and remote sensing data for land cover estimation in a small area 21613.3 Imputation of the missing auxiliary variables 21813.3.1 An overview of the missing data problem 21813.3.2 Multiple imputation 21913.3.3 Multiple imputation for missing data in satellite images 22113.4 Analysis of the 2006 AGRIT data 22213.5 Conclusions 227References 229Part IV Data Editing and Quality Assurance 23114 A generalized edit and analysis system for agricultural data 23314.1 Introduction 23314.2 System development 23614.2.1 Data capture 23614.2.2 Edit 23714.2.3 Imputation 23814.3 Analysis 23914.3.1 General description 23914.3.2 Micro-analysis 23914.3.3 Macro-analysis 24014.4 Development status 24014.5 Conclusions 241References 24215 Statistical data editing for agricultural surveys 24315.1 Introduction 24315.2 Edit rules 24515.3 The role of automatic editing in the editing process 24615.4 Selective editing 24715.4.1 Score functions for totals 24815.4.2 Score functions for changes 25015.4.3 Combining local scores 25115.4.4 Determining a threshold value 25215.5 An overview of automatic editing 25315.6 Automatic editing of systematic errors 25515.7 The Fellegi–Holt paradigm 25615.8 Algorithms for automatic localization of random errors 25715.8.1 The Fellegi–Holt method 25715.8.2 Using standard solvers for integer programming problems 25915.8.3 The vertex generation approach 25915.8.4 A branch-and-bound algorithm 26015.9 Conclusions 263References 26416 Quality in agricultural statistics 26716.1 Introduction 26716.2 Changing concepts of quality 26816.2.1 The American example 26816.2.2 The Swedish example 27116.3 Assuring quality 27416.3.1 Quality assurance as an agency undertaking 27416.3.2 Examples of quality assurance efforts 27516.4 Conclusions 276References 27617 Statistics Canada’s Quality Assurance Framework applied to agricultural statistics 27717.1 Introduction 27717.2 Evolution of agriculture industry structure and user needs 27817.3 Agriculture statistics: a centralized approach 27917.4 Quality Assurance Framework 28117.5 Managing quality 28317.5.1 Managing relevance 28317.5.2 Managing accuracy 28617.5.3 Managing timeliness 29317.5.4 Managing accessibility 29417.5.5 Managing interpretability 29617.5.6 Managing coherence 29717.6 Quality management assessment 29917.7 Conclusions 300Acknowledgements 300References 300Part V Data Dissemination and Survey Data Analysis 30318 The data warehouse: a modern system for managing data 30518.1 Introduction 30518.2 The data situation in the NASS 30618.3 What is a data warehouse? 30818.4 How does it work? 30818.5 What we learned 31018.6 What is in store for the future? 31218.7 Conclusions 31219 Data access and dissemination: some experiments during the First National Agricultural Census in China 31319.1 Introduction 31319.2 Data access and dissemination 31419.3 General characteristics of SDA 31619.4 A sample session using SDA 31819.5 Conclusions 320References 32220 Analysis of economic data collected in farm surveys 32320.1 Introduction 32320.2 Requirements of sample surveys for economic analysis 32520.3 Typical contents of a farm economic survey 32620.4 Issues in statistical analysis of farm survey data 32720.4.1 Multipurpose sample weighting 32720.4.2 Use of sample weights in modelling 32820.5 Issues in economic modelling using farm survey data 33020.5.1 Data and modelling issues 33020.5.2 Economic and econometric specification 33120.6 Case studies 33220.6.1 ABARE broadacre survey data 33220.6.2 Time series model of the growth in fodder use in the Australian cattle industry 33320.6.3 Cross-sectional model of land values in central New South Wales 335References 33821 Measuring household resilience to food insecurity: application to Palestinian households 34121.1 Introduction 34121.2 The concept of resilience and its relation to household food security 34321.2.1 Resilience 34321.2.2 Households as (sub) systems of a broader food system, and household resilience 34521.2.3 Vulnerability versus resilience 34521.3 From concept to measurement 34721.3.1 The resilience framework 34721.3.2 Methodological approaches 34821.4 Empirical strategy 35021.4.1 The Palestinian data set 35021.4.2 The estimation procedure 35121.5 Testing resilience measurement 35921.5.1 Model validation with CART 35921.5.2 The role of resilience in measuring vulnerability 36321.5.3 Forecasting resilience 36421.6 Conclusions 365References 36622 Spatial prediction of agricultural crop yield 36922.1 Introduction 36922.2 The proposed approach 37222.2.1 A simulated exercise 37422.3 Case study: the province of Foggia 37622.3.1 The AGRIT survey 37722.3.2 Durum wheat yield forecast 37822.4 Conclusions 384References 385Author Index 389Subject Index 395