Experimentation, Validation, and Uncertainty Analysis for Engineers

HUGH W. COLEMAN, PHD, PE, is a Professor Emeritus of Mechanical and Aerospace Engineering at the University of Alabama in Huntsville, USA.W. GLENN STEELE, PHD, PE, is a Professor Emeritus of Mechanical Engineering at Mississippi State University, USA.Coleman and Steele received the prestigious AIAA Ground Testing award for "pioneering efforts in experimental uncertainty analysis with significant methodology advances and effective dissemination of knowledge through a straightforward engineering approach in their text and short courses." Steele was also awarded the ASME Performance Test Codes Medal in 2014. They have served on technical committees and on experimental uncertainty and validation standards committees associated with ASME, AIAA, SAE, ISO, and NATO AGARD. They are both Fellows of ASME and Associate Fellows of AIAA.

• Preface xv1 Experimentation, Errors, and Uncertainty 11-1 Experimentation, 21-1.1 Why Is Experimentation Necessary?, 21-1.2 Degree of Goodness and Uncertainty Analysis, 31-1.3 Experimentation and Validation of Simulations, 51-2 Experimental Approach, 61-2.1 Questions to Be Considered, 71-2.2 Phases of Experimental Program, 81-3 Basic Concepts and Definitions, 81-3.1 Errors and Uncertainties, 91-3.2 Categorizing and Naming Errors and Uncertainties, 131-3.3 Estimating Standard Uncertainties, 151-3.4 Determining Combined Standard Uncertainties, 161-3.5 Elemental Systematic Errors and Effects of Calibration, 181-3.6 Expansion of Concept from “Measurement Uncertainty” to “Experimental Uncertainty”, 201-3.7 Repetition and Replication, 221-3.8 Associating a Percentage Coverage or Confidence with Uncertainty Estimates, 241-4 Experimental Results Determined from a Data Reduction Equation Combining Multiple Measured Variables, 251-5 Guides and Standards, 271-5.1 Experimental Uncertainty Analysis, 271-5.2 Validation of Simulations, 291-6 A Note on Nomenclature, 31References, 31Problems, 322 Coverage and Confidence Intervals for an Individual Measured Variable 332-1 Coverage Intervals from the Monte Carlo Method for a Single Measured Variable, 342-2 Confidence Intervals from the Taylor Series Method for a Single Measured Variable, Only Random Errors Considered, 352-2.1 Statistical Distributions, 352-2.2 The Gaussian Distribution, 362-2.3 Confidence Intervals in Gaussian Parent Populations, 422-2.4 Confidence Intervals in Samples from Gaussian Parent Populations, 432-2.5 Tolerance and Prediction Intervals in Samples from Gaussian Parent Populations, 482-2.6 Statistical Rejection of Outliers from a Sample Assumed from a Gaussian Parent Population, 512-3 Confidence Intervals from the Taylor Series Method for a Single Measured Variable: Random and Systematic Errors Considered, 552-3.1 The Central Limit Theorem, 552-3.2 Systematic Standard Uncertainty Estimation, 562-3.3 The TSM Expanded Uncertainty of a Measured Variable, 582-3.4 The TSM Large-Sample Expanded Uncertainty of a Measured Variable, 612-4 Uncertainty of Uncertainty Estimates and Confidence Interval Limits for a Measured Variable, 632-4.1 Uncertainty of Uncertainty Estimates, 632-4.2 Implications of the Uncertainty in Limits of High Confidence Uncertainty Intervals Used in Analysis and Design, 65References, 67Problems, 683 Uncertainty in a Result Determined from Multiple Variables 713-1 General Uncertainty Analysis vs. Detailed Uncertainty Analysis, 723-2 Monte Carlo Method for Propagation of Uncertainties, 733-2.1 Using the MCM in General Uncertainty Analysis, 733-2.2 Using the MCM in Detailed Uncertainty Analysis, 753-3 Taylor Series Method for Propagation of Uncertainties, 783-3.1 General Uncertainty Analysis Using the Taylor Series Method (TSM), 793-3.2 Detailed Uncertainty Analysis Using the Taylor Series Method (TSM), 803-4 Determining MCM Coverage Intervals and TSM Expanded Uncertainty, 823-4.1 MCM Coverage Intervals for a Result, 823-4.2 TSM Expanded Uncertainty of a Result, 853-5 General Uncertainty Analysis Using the TSM and MSM Approaches for a Rough-walled Pipe Flow Experiment, 873-5.1 TSM General Uncertainty Analysis, 883-5.2 MCM General Uncertainty Analysis, 893-5.3 Implementation Using a Spreadsheet, 893-5.4 Results of the Analysis, 923-6 Comments on Implementing Detailed Uncertainty Analysis Using a Spreadsheet, 95References, 96Problems, 974 General Uncertainty Analysis Using the Taylor Series Method (TSM) 994-1 TSM Application to Experiment Planning, 1004-2 TSM Application to Experiment Planning: Special Functional Form, 1034-3 Using TSM Uncertainty Analysis in Planning an Experiment, 1074-4 Example: Analysis of Proposed Particulate Measuring System, 1094-4.1 The Problem, 1094-4.2 Proposed Measurement Technique and System, 1094-4.3 Analysis of Proposed Experiment, 1104-4.4 Implications of Uncertainty Analysis Results, 1124-4.5 Design Changes Indicated by Uncertainty Analysis, 1134-5 Example: Analysis of Proposed Heat Transfer Experiment, 1144-5.1 The Problem, 1144-5.2 Two Proposed Experimental Techniques, 1154-5.3 General Uncertainty Analysis: Steady-State Technique, 1174-5.4 General Uncertainty Analysis: Transient Technique, 1214-5.5 Implications of Uncertainty Analysis Results, 1234-6 Examples of Presentation of Results from Actual Applications, 1244-6.1 Results from Analysis of a Turbine Test, 1244-6.2 Results from Analysis of a Solar Thermal Absorber/Thruster Test, 125References, 126Problems, 1275 Detailed Uncertainty Analysis: Overview and Determining Random Uncertainties in Results 1315-1 Using Detailed Uncertainty Analysis, 1315-2 Detailed Uncertainty Analysis: Overview of Complete Methodology, 1345-3 Determining Random Uncertainty of Experimental Result, 1375-3.1 Example: Random Uncertainty Determination in Compressible Flow Venturi Meter Calibration Facility, 1395-3.2 Example: Random Uncertainty Determination in Laboratory-Scale Ambient Temperature Flow Test Facility, 1415-3.3 Example: Random Uncertainty Determination in Full-Scale Rocket Engine Ground Test Facility, 1435-3.4 Summary, 146References, 1466 Detailed Uncertainty Analysis: Determining Systematic Uncertainties in Results 1476-1 Estimating Systematic Uncertainties, 1496-1.1 Example: Estimating Uncertainty in Property Values, 1526-1.2 Example: Estimating Systematic Uncertainties in the Turbulent Heat Transfer Test Facility (THTTF), 1536-1.3 Example: An “Optimum” Calibration Approach Used in a Test to Determine Turbine Efficiency, 1636-2 Determining Systematic Uncertainty of Experimental Result Including Correlated Systematic Error Effects, 1656-2.1 Example: Correlated Systematic Error Effects with “% of Full Scale” (%FS) Systematic Uncertainties, 1686-2.2 Example: Correlated Systematic Error Effects with “% of Reading” Systematic Uncertainties, 1706-2.3 Example: Correlated Systematic Error Effects with Systematic Uncertainties that Vary with Set Point, 1716-2.4 Example: Correlated Systematic Error Effects When Only Some Elemental Sources Are Correlated, 1726-2.5 Example: Correlated Systematic Error Effects When Determining Average Velocity of a Fluid Flow, 1766-3 Comparative Testing, 1776-3.1 Result Is a Difference of Test Results, 1786-3.2 Result Is a Ratio of Test Results, 1816-4 Some Additional Considerations in Experiment Execution, 1836-4.1 Choice of Test Points: Rectification, 1836-4.2 Choice of Test Sequence, 1886-4.3 Relationship to Statistical Design of Experiments, 1896-4.4 Use of a Jitter Program, 1916-4.5 Comments on Transient Testing, 1936-4.6 Comments on Digital Data Acquisition Errors, 193References, 194Problems, 1957 Detailed Uncertainty Analysis: Comprehensive Examples 1997-1 TSM Comprehensive Example: Sample-to-Sample Experiment, 1997-1.1 The Problem, 1997-1.2 Measurement System, 2007-1.3 Zeroth-Order Replication-Level Analysis, 2017-1.4 First-Order Replication-Level Analysis, 2057-1.5 Nth-Order Replication-Level Analysis, 2067-2 TSM Comprehensive Example: Use of Balance Checks, 2077-3 Comprehensive Example: Debugging and Qualification of a Timewise Experiment, 2107-3.1 Orders of Replication Level in Timewise Experiments, 2117-3.2 Example, 2127-4 Comprehensive Example: Heat Exchanger Test Facility for Single and Comparative Tests, 2167-4.1 Determination of the Uncertainty in q for a Single Core Design, 2197-4.2 Determination of the Uncertainty in Δq for Two Core Designs Tested Sequentially Using the Same Facility and Instrumentation, 2247-5 Case Study: Examples of Single and Comparative Tests of Nuclear Power Plant Residual Heat Removal Heat Exchanger, 2307-5.1 Single Test Results for an RHR Heat Exchanger (HX1), 2317-5.2 Comparative Test Approach for the Decrease in Fouling Resistance and Its Uncertainty, 234References, 235Problems, 2358 The Uncertainty Associated with the Use of Regressions 2398-1 Overview of Linear Regression Analysis and Its Uncertainty, 2408-1.1 Uncertainty in Coefficients, 2418-1.2 Uncertainty in Y from Regression Model, 2418-1.3 (Xi, Yi) Variables Are Functions, 2438-2 Determining and Reporting Regression Uncertainty, 2438-2.1 MCM Regression Uncertainty Determination, 2448-2.2 TSM Regression Uncertainty Determination, 2448-2.3 Reporting Regression Uncertainties, 2448-3 Method of Least Squares Regression, 2468-4 First-Order Regression Example: MCM Approach to Determine Regression Uncertainty, 2498-5 Regression Examples: TSM Approach to Determine Regression Uncertainty, 2528-5.1 Uncertainty in First-Order Coefficients, 2528-5.2 Uncertainty in Y from First-Order Regression, 2538-5.3 Uncertainty in Y from Higher-Order Regressions, 2558-5.4 Uncertainty in Y from Regressions in Which X and Y Are Functional Relations, 2558-5.5 Uncertainty Associated with Multivariate Linear Regression, 2578-6 Comprehensive TSM Example: Regressions and Their Uncertainties in a Flow Test, 2598-6.1 Experimental Apparatus, 2618-6.2 Pressure Transducer Calibration and Uncertainty, 2618-6.3 Venturi Discharge Coefficient and Its Uncertainty, 2658-6.4 Flow Rate and Its Uncertainty in a Test, 269References, 273Problems, 2739 Validation of Simulations 2779-1 Introduction to Validation Methodology, 2779-2 Errors and Uncertainties, 2789-3 Validation Nomenclature, 2799-4 Validation Approach, 2809-5 Code and Solution Verification, 2849-6 Interpretation of Validation Results Using E and uval, 2849-6.1 Interpretation with No Assumptions Made about Error Distributions, 2859-6.2 Interpretation with Assumptions Made about Error Distributions, 2859-7 Estimation of Validation Uncertainty uval, 2869-7.1 Case 1: Estimating uval When Experimental Value D of Validation Variable Is Directly Measured, 2879-7.2 Cases 2 and 3: Estimating uval When Experimental Value D of Validation Variable Is Determined from Data Reduction Equation, 2909-7.3 Case 4: Estimating uval When Experimental Value D of Validation Variable Is Determined from Data Reduction Equation That Itself Is a Model, 2959-8 Some Practical Points, 297References, 299Answers to Selected Problems 301Appendix A Useful Statistics 305Appendix B Taylor Series Method (TSM) for Uncertainty Propagation 311B-1 Derivation of Uncertainty Propagation Equation, 312B-2 Comparison with Previous Approaches, 316B-2.1 Abernethy et al. Approach, 316B-2.2 Coleman and Steele Approach, 317B-2.3 ISO Guide 1993 GUM Approach, 318B-2.4 AIAA Standard, AGARD, and ANSI/ASME Approach, 319B-2.5 NIST Approach, 319B-3 Additional Assumptions for Engineering Applications, 319B-3.1 Approximating the Coverage Factor, 320References, 322Appendix C Comparison of Models for Calculation of Uncertainty 325C-1 Monte Carlo Simulations, 325C-2 Simulation Results, 328References, 336Appendix D Shortest Coverage Interval for Monte Carlo Method 337Reference, 338Appendix E Asymmetric Systematic Uncertainties 339E-1 Procedure for Asymmetric Systematic Uncertainties Using TSM Propagation, 340E-2 Procedure for Asymmetric Systematic Uncertainties Using MCM Propagation, 344E-3 Example: Biases in a Gas Temperature Measurement System, 344References, 351Appendix F Dynamic Response of Instrument Systems 353F-1 General Instrument Response, 353F-2 Response of Zero-Order Instruments, 355F-3 Response of First-Order Instruments, 356F-4 Response of Second-Order Instruments, 359F-5 Summary, 362References, 362Index 363