Statistics, Testing, and Defense Acquisition

For every weapons system being developed, the U.S. Department of Defense (DOD) must make a critical decision: Should the system go forward to full-scale production? The answer to that question may involve not only tens of billions of dollars but also the nation's security and military capabilities. In the milestone process used by DOD to answer the basic acquisition question, one component near the end of the process is operational testing, to determine if a system meets the requirements for effectiveness and suitability in realistic battlefield settings. Problems discovered at this stage can cause significant production delays and can necessitate costly system redesign. This book examines the milestone process, as well as the DOD's entire approach to testing and evaluating defense systems. It brings to the topic of defense acquisition the application of scientific statistical principles and practices. Table of Contents Front Matter Executive Summary 1 Introduction 2 Operational Testing and System Acquisition 3 A New Paradigm for Testing and Evaluation in Defense Acquisition 4 Upgrading Statistical Methods for Testing and Evaluation 5 Improving Operational Test Planning and Design 6 Analyzing and Reporting Test Results 7 Assessing Operational Suitability 8 Testing Software-Intensive Systems 9 Using Modeling and Simulation in Test Design and Evaluation 10 Increasing Access to Statistical Expertise for Operational Testing Appendix A Case Studies and System Descriptions Appendix B Abstracts of Background Papers Appendix C A Potential Taxonomy for Operational Tests Appendix D Elements of ISO 9000 Appendix E Glossary and Acronyms References Biographical Sketches of Panel Members and Staff