Experimentation, validation, and uncertainty analysis for engineers

CONTENTS
PREFACE
1. EXPERIMENTATION, ERRORS,  AND UNCERTAINTY
	1-1 EXPERIMENTATION
		1-1.1 Why Is Experimentation Necessary?
		1-1.2 Degree of Goodness and Uncertainty Analysis
		1-1.3 Experimentation and Validation of Simulations
	1-2 EXPERIMENTAL APPROACH
		1-2.1 Questions to Be Considere
		1-2.2 Phases of Experimental Program
	1-3 BASIC CONCEPTS AND DEFINITIONS
		1-3.1 Errors and Uncertainties
		1-3.2 Categorizing and Naming Errors and Uncertainties
		1-3.3 Estimating Standard Uncertaintie
		1-3.4 Determining Combined Standard Uncertainties
		1-3.5 Elemental Systematic Errors and Effects of Calibration
		1-3.6 Expansion of Concept from “Measurement Uncertainty”to “Experimental Uncertainty”
		1-3.7 Repetition and Replication
		1-3.8 Associating a Percentage Coverage or Confidence
with Uncertainty Estimates
	1-4 EXPERIMENTAL RESULTS DETERMINED
FROM A DATA REDUCTION EQUATION COMBINING
MULTIPLE MEASURED VARIABLES
	1-5 GUIDES AND STANDARDS
		1-5.1 Experimental Uncertainty Analysis
		1-5.2 Validation of Simulations
	1-6 A NOTE ON NOMENCLATURE
	PROBLEMS
	REFERENCES
2. COVERAGE AND CONFIDENCE  INTERVALS FOR AN INDIVIDUAL  MEASURED VARIABLE
	2-1 COVERAGE INTERVALS FROM THE MONTE CARLO METHOD
FOR A SINGLE MEASURED VARIABLE
	2-2 CONFIDENCE INTERVALS FROM THE TAYLOR SERIES
METHOD FOR A SINGLE MEASURED VARIABLE, ONLY RANDOM
ERRORS CONSIDERED
		2-2.1 Statistical Distributions
		2-2.2 The Gaussian Distribution
		2-2.3 Confidence Intervals in Gaussian Parent Population
		2-2.4 Confidence Intervals in Samples from Gaussian Parent Populations
		2-2.5 Tolerance and Prediction Intervals in Samples from Gaussian Parent
Populations
		2-2.6 Statistical Rejection of Outliers from a Sample Assumed from a
Gaussian Parent Population
	2-3 CONFIDENCE INTERVALS FROM THE TAYLOR SERIES
METHOD FOR A SINGLE MEASURED VARIABLE: RANDOM AND
SYSTEMATIC ERRORS CONSIDERED
		2-3.1 The Central Limit Theorem
		2-3.2 Systematic Standard Uncertainty Estimation
		2-3.3 The TSM Expanded Uncertainty of a Measured Variable
		2-3.4 The TSM Large-Sample Expanded Uncertainty of a Measured
Variable
	2-4 UNCERTAINTY OF UNCERTAINTY ESTIMATES AND
CONFIDENCE INTERVAL LIMITS FOR A MEASURED VARIABLE
		2-4.1 Uncertainty of Uncertainty Estimates
		2-4.2 Implications of the Uncertainty in Limits of High Confidence
Uncertainty Intervals Used in Analysis and Design2
	REFERENCES
	PROBLEMS
3. UNCERTAINTY IN A RESULT  DETERMINED FROM MULTIPLE  VARIABLES
	3-1 GENERAL UNCERTAINTY ANALYSIS VS. DETAILED
UNCERTAINTY ANALYSIS
	3-2 MONTE CARLO METHOD FOR PROPAGATION
OF UNCERTAINTIES
		3-2.1 Using the MCM in General Uncertainty Analysis
		3-2.2 Using the MCM in Detailed Uncertainty Analysis
			3-2.2.1 Detailed MCM Uncertainty Analysis: Propagation Using Individual
Variable Random Standard Uncertainties
			3-2.2.2 Detailed MCM Uncertainty Analysis: Propagation Using Directly
Determined Random Standard Uncertainty of the Result
	3-3 TAYLOR SERIES METHOD FOR PROPAGATION
OF UNCERTAINTIES
		3-3.1 General Uncertainty Analysis Using the Taylor Series Method (TSM)
		3-3.2 Detailed Uncertainty Analysis Using the Taylor Series Method (TSM)
			3-3.2.1 Detailed (TSM) Uncertainty Analysis: Using Propagation to Determine
sr
			3-3.2.2 Detailed (TSM) Uncertainty Analysis: Direct Determination of
sr
	3-4 DETERMINING MCM COVERAGE INTERVALS AND TSM
EXPANDED UNCERTAINTY
		3-4.1 MCM Coverage Intervals for a Result
		3-4.2 TSM Expanded Uncertainty of a Result
	3-5 GENERAL UNCERTAINTY ANALYSIS USING THE TSM
AND MSM APPROACHES FOR A ROUGH-WALLED PIPE FLOW
EXPERIMENT
		3-5.1 TSM General Uncertainty Analysis
		3-5.2 MCM General Uncertainty Analysis
		3-5.3 Implementation Using a Spreadsheet
		3-5.4 Results of the Analysis
	3-6 COMMENTS ON IMPLEMENTING DETAILED UNCERTAINTY
ANALYSIS USING A SPREADSHEET
	REFERENCES
	PROBLEMS
4. GENERAL UNCERTAINTY  ANALYSIS USING THE TAYLOR  SERIES METHOD (TSM)
	4-1 TSM APPLICATION TO EXPERIMENT PLANNING
	4-2 TSM APPLICATION TO EXPERIMENT PLANNING: SPECIAL
FUNCTIONAL FORM
	4-3 USING TSM UNCERTAINTY ANALYSIS IN PLANNING
AN EXPERIMENT
	4-4 EXAMPLE: ANALYSIS OF PROPOSED PARTICULATE
MEASURING SYSTEM
		4-4.1 The Problem
		4-4.2 Proposed Measurement Technique and System
		4-4.3 Analysis of Proposed Experiment
		4-4.4 Implications of Uncertainty Analysis Results
		4-4.5 Design Changes Indicated by Uncertainty Analysis
	4-5 EXAMPLE: ANALYSIS OF PROPOSED HEAT TRANSFER
EXPERIMENT
		4-5.1 The Problem
		4-5.2 Two Proposed Experimental Techniques
		4-5.3 General Uncertainty Analysis: Steady-State Technique
		4-5.4 General Uncertainty Analysis: Transient Technique
		4-5.5 Implications of Uncertainty Analysis Results
	4-6 EXAMPLES OF PRESENTATION OF RESULTS FROM ACTUAL
APPLICATIONS
		4-6.1 Results from Analysis of a Turbine Test
		4-6.2 Results from Analysis of a Solar Thermal Absorber/Thruster Test
	REFERENCES
	PROBLEMS
5. DETAILED UNCERTAINTY  ANALYSIS: OVERVIEW AND  DETERMINING RANDOM  UNCERTAINTIES IN RESULTS
	5-1 USING DETAILED UNCERTAINTY ANALYSIS
	5-2 DETAILED UNCERTAINTY ANALYSIS: OVERVIEW
OF COMPLETE METHODOLOGY
	5-3 DETERMINING RANDOM UNCERTAINTY
OF EXPERIMENTAL RESULT
		5-3.1 Example: Random Uncertainty Determination in Compressible Flow
Venturi Meter Calibration Facility
		5-3.2 Example: Random Uncertainty Determination in Laboratory-Scale
Ambient Temperature Flow Test Facility
		5-3.3 Example: Random Uncertainty Determination in Full-Scale Rocket
Engine Ground Test Facility
		5-3.4 Summary
	REFERENCES
6. DETAILED UNCERTAINTY  ANALYSIS: DETERMINING  SYSTEMATIC UNCERTAINTIES  IN RESULTS
	6-1 ESTIMATING SYSTEMATIC UNCERTAINTIES
		6-1.1 Example: Estimating Uncertainty in Property Values
		6-1.2 Example: Estimating Systematic Uncertainties in the Turbulent Heat
Transfer Test Facility (THTTF)
			6-1.2.1 Zero-centering Asymmetric Systematic Error Effects
			6-1.2.2 Systematic Uncertainty in Test Plate Surface Temperature
Tw
			6-1.2.3 Systematic Uncertainty in Power into Test Plate
		6-1.3 Example: An “Optimum” Calibration Approach Used in a Test
to Determine Turbine Efficiency
	6-2 DETERMINING SYSTEMATIC UNCERTAINTY OF
EXPERIMENTAL RESULT INCLUDING CORRELATED
SYSTEMATIC ERROR EFFECTS
		6-2.1 Example: Correlated Systematic Error Effects with “% of Full Scale”(%FS) Systematic Uncertainties
		6-2.2 Example: Correlated Systematic Error Effects with “% of Reading”Systematic Uncertainties
		6-2.3 Example: Correlated Systematic Error Effects with Systematic
Uncertainties that Vary with Set Point
		6-2.4 Example: Correlated Systematic Error Effects When Only Some
Elemental Sources Are Correlated
		6-2.5 Example: Correlated Systematic Error Effects When Determining
Average Velocity of a Fluid Flow
	6-3 COMPARATIVE TESTING
		6-3.1 Result Is a Difference of Test Results
		6-3.2 Result Is a Ratio of Test Results
	6-4 SOME ADDITIONAL CONSIDERATIONS IN EXPERIMENT
EXECUTION
		6-4.1 Choice of Test Points: Rectification
		6-4.2 Choice of Test Sequence
		6-4.3 Relationship to Statistical Design of Experiments
		6-4.4 Use of a Jitter Program
		6-4.5 Comments on Transient Testing
		6-4.6 Comments on Digital Data Acquisition Errors
	REFERENCES
	PROBLEMS
7. DETAILED UNCERTAINTY  ANALYSIS: COMPREHENSIVE  EXAMPLES
	7-1 TSM COMPREHENSIVE EXAMPLE: SAMPLE-TO-SAMPLE
EXPERIMENT
		7-1.1 The Problem
		7-1.2 Measurement System
		7-1.3 Zeroth-Order Replication-Level Analysis
		7-1.4 First-Order Replication-Level Analysis
		7-1.5 Nth-Order Replication-Level Analysis
	7-2 TSM COMPREHENSIVE EXAMPLE: USE OF BALANCE CHECKS
	7-3 COMPREHENSIVE EXAMPLE: DEBUGGING
AND QUALIFICATION OF A TIMEWISE EXPERIMENT
		7-3.1 Orders of Replication Level in Timewise Experiments
		7-3.2 Example
	7-4 COMPREHENSIVE EXAMPLE: HEAT EXCHANGER TEST
FACILITY FOR SINGLE AND COMPARATIVE TESTS
		7-4.1 Determination of the Uncertainty in q for a Single Core Design
			7-4.1.1 Case 1: No Shared Error Sources in Any Measurements
			7-4.1.2 Case 2: Possible Shared Error Sources in Temperature Measurements
		7-4.2 Determination of the Uncertainty in ????q for Two Core Designs Tested
Sequentially Using the Same Facility and Instrumentation
	7-5 CASE STUDY: EXAMPLES OF SINGLE AND COMPARATIVE
TESTS OF NUCLEAR POWER PLANT RESIDUAL HEAT REMOVAL
HEAT EXCHANGER
		7-5.1 Single Test Results for an RHR Heat Exchanger (HX1)
		7-5.2 Comparative Test Approach for the Decrease in Fouling Resistance
and Its Uncertainty
	REFERENCES
	PROBLEMS
8. THE UNCERTAINTY ASSOCIATED  WITH THE USE OF REGRESSIONS
	8-1 OVERVIEW OF LINEAR REGRESSION ANALYSIS
AND ITS UNCERTAINTY1
		8-1.1 Uncertainty in Coefficients
		8-1.2 Uncertainty in Y from Regression Model
		8-1.3 (Xi, Yi)
Variables Are Functions
	8-2 DETERMINING AND REPORTING REGRESSION UNCERTAINTY
		8-2.1 MCM Regression Uncertainty Determination
		8-2.2 TSM Regression Uncertainty Determination
		8-2.3 Reporting Regression Uncertainties
	8-3 METHOD OF LEAST SQUARES REGRESSION
	8-4 FIRST-ORDER REGRESSION EXAMPLE: MCM APPROACH TO
DETERMINE REGRESSION UNCERTAINTY
	8-5 REGRESSION EXAMPLES: TSM APPROACH TO DETERMINE
REGRESSION UNCERTAINTY
		8-5.1 Uncertainty in First-Order Coefficients
		8-5.2 Uncertainty in Y from First-Order Regression
		8-5.3 Uncertainty in Y from Higher-Order Regressions
		8-5.4 Uncertainty in Y from Regressions in Which X and Y
Are Functional Relations
		8-5.5 Uncertainty Associated with Multivariate Linear Regression
	8-6 COMPREHENSIVE TSM EXAMPLE: REGRESSIONS AND THEIR
UNCERTAINTIES IN A FLOW TEST
		8-6.1 Experimental Apparatus
		8-6.2 Pressure Transducer Calibration and Uncertainty
		8-6.3 Venturi Discharge Coefficient and Its Uncertainty
		8-6.4 Flow Rate and Its Uncertainty in a Test
	REFERENCES
	PROBLEMS
9. VALIDATION OF SIMULATIONS
	9-1 INTRODUCTION TO VALIDATION METHODOLOGY
	9-2 ERRORS AND UNCERTAINTIES
	9-3 VALIDATION NOMENCLATURE
	9-4 VALIDATION APPROACH
	9-5 CODE AND SOLUTION VERIFICATION
	9-6 INTERPRETATION OF VALIDATION RESULTS USING
E AND
uval
		9-6.1 Interpretation with No Assumptions Made about Error Distributions
		9-6.2 Interpretation with Assumptions Made about Error Distributions
	9-7 ESTIMATION OF VALIDATION UNCERTAINTY
uval
		9-7.1 Case 1: Estimating
uval
When Experimental Value D of Validation
Variable Is Directly Measured
		9-7.2 Cases 2 and 3: Estimating
uval
When Experimental Value D
of Validation Variable Is Determined from Data Reduction Equation
			9-7.2.1 Case 2: No Measured Variables Share Identical Error Sources
			9-7.2.2 Case 3: Measured Variables Share Identical Error Sources
		9-7.3 Case 4: Estimating
uval
When Experimental Value D of Validation
Variable Is Determined from Data Reduction Equation That Itself Is a
Model
	9-8 SOME PRACTICAL POINTS
	REFERENCES
ANSWERS TO SELECTED PROBLEMS
APPENDIX A.  USEFUL STATISTICS
APPENDIX B.  TAYLOR SERIES METHOD (TSM)  FOR UNCERTAINTY PROPAGATION
	B-1 DERIVATION OF UNCERTAINTY PROPAGATION EQUATION
	B-2 COMPARISON WITH PREVIOUS APPROACHES
		B-2.1 Abernethy et al. Approach
		B-2.2 Coleman and Steele Approach
		B-2.3 ISO Guide 1993 GUM Approach
		B-2.4 AIAA Standard, AGARD, and ANSI/ASME Approach
		B-2.5 NIST Approach
	B-3 ADDITIONAL ASSUMPTIONS FOR ENGINEERING
APPLICATIONS
		B-3.1 Approximating the Coverage Factor
	REFERENCES
APPENDIX C.  COMPARISON OF MODELS FOR  CALCULATION OF UNCERTAINTY
	C-1 MONTE CARLO SIMULATIONS
	C-2 SIMULATION RESULTS
	REFERENCES
APPENDIX D.  SHORTEST COVERAGE INTERVAL  FOR MONTE CARLO METHOD
	REFERENCE
APPENDIX E.  ASYMMETRIC SYSTEMATIC  UNCERTAINTIES
	E-1 PROCEDURE FOR ASYMMETRIC SYSTEMATIC
UNCERTAINTIES USING TSM PROPAGATION
	E-2 PROCEDURE FOR ASYMMETRIC SYSTEMATIC
UNCERTAINTIES USING MCM PROPAGATION
	E-3 EXAMPLE: BIASES IN A GAS TEMPERATURE
MEASUREMENT SYSTEM
	REFERENCES
APPENDIX F.  DYNAMIC RESPONSE OF  INSTRUMENT SYSTEMS
	F-1 GENERAL INSTRUMENT RESPONSE
	F-2 RESPONSE OF ZERO-ORDER INSTRUMENTS
	F-3 RESPONSE OF FIRST-ORDER INSTRUMENTS
	F-4 RESPONSE OF SECOND-ORDER INSTRUMENTS
	F-5 SUMMARY
	REFERENCES
INDEX