Verification and Validation in Computational Science and Engineering. by Patrick J. Roache. hermosa

Transcription

1 Verification and Validation in Computational Science and Engineering by Patrick J. Roache hermosa PO Box 9110 Albuquerque, New Mexico USA

2 TABLE OF CONTENTS Preface Acknowledgments Hi vii Part I Overview 1 Chapter 1 Introduction Sketch of Historical Development of CFD The New Impetus Toward Higher Quality Solutions A Personal Anecdote Illustrating the Importance of Systematic Code Verification 11 References for Chapter 1 14 Chapter 2 Semantics: Terminology, Taxonomies, and Definitions Introduction Semantics Code Verification and Validation: Numerical vs. Conceptual Modeling Code Confirmation Benchmarks and Inter-Code Comparisons Code Certification, Quality Assurance, and Accreditation Verification of Calculations Quantification of Uncertainty Grid Convergence vs. Iterative Convergence Error Taxonomies 36 IX

3 x Verification and Validation in Computational Science and Engineering 2.11 Truncation Error vs. Discretization Error Calibration and Tuning Quality assurance (QA) vs. Quality Work Customer Illusions vs. Customer Care Other Distinctions: Authors, Users, Modelers, Code and Software Sensitivity, Uncertainty, and Risk Etymology and Near Synonyms Accuracy vs. Reliability Additional Remarks on Verification Conclusion: Limitations of Semantic Distinctions 51 References for Chapter 2 52 Part II Verification 63 Chapter 3 A Methodology for Accuracy Verification of Codes: the Method of Manufactured Solutions Introduction Warnings: the Division of Labor in Code Development and Use Order of Convergence A General Procedure for Generating an Analytical Solution for Code Accuracy Verification: the Method of Manufactured Solutions Example: Verification of a 3-D Poisson Equation Code in Nonorthogonal Grid and a 3-D Grid Generation Code Another Path to Manufactured Solutions Code Verification Including Shock Waves Need for a Theorem Specific Analytical Solutions Manufactured Solutions vs. Infinite Series Solutions The Sensitivity of Grid Convergence Testing Examples of Unanticipated Convergence Rates Determined by Systematic Grid Convergence Tests Special Considerations for Problems with Multiple Scales: Turbulence Modeling Warnings: What the Method Does Not "Verify" Robustness and Confidence 89 References for Chapter 3 90 Chapter 4 Error Estimation for Quantification of Uncertainty; Verification of Calculations Introduction Error Estimation for Grid Adaptation vs. Quantification of Uncertainty Taxonomy for Additional Information for Error Estimates Grid Refining and Coarsening Levels of Simulation Use Verification of Computer Round-off Errors Effect of Differing formulations 103 References for Chapter Chapter 5 Systematic Grid Convergence Studies and the Grid Convergence Index (GCI) Introduction Background on Grid Convergence Reporting 108

4 Table of Contents xi 5.3 Richardson Extrapolation A Generalization of Richardson Extrapolation Richardson's Extrapolation for n Grid Convergence Index for the Fine Grid Solution Grid Convergence Index for the Coarse Grid Solution Example GCI Calculation Should the Coefficient Be "1" Or "3" Or "1.25"? Additional Features of Grid Convergence Studies for Verification of Codes and Calculations Conclusion 135 References for Chapter Chapter 6 Applications of Systematic Grid Convergence Studies and the Grid Convergence Index Introduction..' Two Further Examples of (Partial) Code Verification in Groundwater Flow Issues in Calculation Verification An Example of the Effective Grid Refinement Ratio A Benchmark Problem for Driven Cavity Flow A Benchmark Problem for Free Convection Laminar Plane Jet Impinging on a Heated Flat Plate A k-e Model of a Free Shear Layer Transonic Airfoil Calculations Ordered Estimation of Far-Field Boundary Error Artificial Dissipation Effects Single and Dual Porosity Contaminant Transport: Source Term Location Convergence Behaviors for Mixed-Order Methods Grid Convergence of Zero Drag Coefficient Anomalous Result Possibly Due to Grid Stretching Non-Smooth Property Variation: Global Error Norms." Discrete Vortex Methods Observed Convergence Rates for Euler Equations with Shocks Completed Richardson Extrapolation Truncation Error in Elliptic Grid Generation One Dimensional Moving Adaptive Grid Problems GCI Application in Solution Adaptive Grids with Non-Integer Grid Refinement High Quality Grid Resolution Studies Leading to a Safety Factor of Transport Code Verifications Using the GCI: Partitioning the Option Matrix Turbulent Separated Flow: the Error Estimator of Celik and Karatekin Level of Accuracy Estimates from Grid Convergence Studies Other Examples of Careful Use of Richardson Extrapolation Parameter Convergences of a Compressible Flow Code Near the Incompressible Limit Justification of the Dupuit Approximation Concluding Comment on Parameter Uncertainty vs. Numerical Uncertainty 206 References for Chapter Chapter 7 Single Grid Error Estimators Error Estimation from Higher or Lower Order Accuracy Solutions on the Same Grid (Category B) Auxiliary PDE Solutions on the Same Grid (Category C) Auxiliary Algebraic Evaluations on the Same Grid: Surrogate Estimators (Category D) Time Accuracy Estimation Concluding Remarks on Single Grid Error Estimators 230 References for Chapter 7 231

5 xii Verification and Validation in Computational Science and Engineering Chapter 8 Hard Stories Factors Influencing Convergence Rates Behavior of Quasi-Higher-Order Methods Some Good News for Turbulence Modeling The Myth of the Converged "Solution" Esoteric Coding Mistakes 'A False Verification Test of a Particle Tracker Hard-Wired Data vs. User Input Data Degraded Rate of Convergence Due to User Modeling Errors Lessons from Nonlinear Dynamics Adaptive and Local Time Stepping, and Steady State Other Questions Related to the Steady State 263 References for Chapter Part III Validation 271«Chapter 9 Difficulties With Experiments and Validation Credulousness Historical Methods of Validating Scientific Theories The Theory Laden Experiment Random and Systematic Errors in Experiments Experimental Errors in Physical Properties Boundary Conditions, Continuum and Numerical Trends, Computational and Experimental False Negatives and False Positives "Nearby" Problems Difficulty of the Option Tree Data Sparsity and Lack of Synchronicity: Groundwater, Ocean/Lake, and Meteorology Modeling The Effect of Parameter Resolution on Grid Convergence Scale of Unsteadiness Spatial Scales, Scaling Up, and Dimensionality Assumptions of Periodicity Other Difficulties of Validation in Aerospace Universal Turbulence Models vs. Zonal Modeling Strong and Weak Model Definitions and Model Validation The Myth of the Totally Validated Code 292 References for Chapter Chapter 10 Methodologies and Examples of Validations, Calibrations, and Certifications Sources of Physical Modeling Errors in Aerodynamics CFD Accuracy Level for Validation Generic Models vs. Realistic Models for Validation and Calibration: Phases of Validation CFD and Experimental Facility Corrections Verification Must Be Independent of Validation: Airfoil Calculations Synergism Between Computation and Validation Experiments The Difficulty of Defining a "Nearby" Problem Missing Experimental Details Onset of 3 Dimensionality in Backstep Flow Gray Area: "Validation" from a Calculated Benchmark Gray Area: "Validation" of an Experimental Technique by a Computation 310

6 Table of Contents xiii The MADE-2 Experience: Can Groundwater Flow Models Be Validated? Dynamic Stall Wind Tunnel Data: Who Does the Tweaking? Consortium Effort at CFD Code Certification Simulation Team Responsibilities in Validation Shifting Responsibilities and Gray Areas WUA Benchmarks in 1994 and CFD Triathlons Canadian CFD Society Test Case Workshops AGARD 1988 Validation of Computational Fluid Dynamics A Case Study for CFD Code Validation Methodology Joint Consideration of Experimental and Simulation Uncertainties Dynamic Databases for Validation 335 References for Chapter Part IV Broader Issues 349 ^ Chapter 11 Code Quality Assurance and Certification Introduction Quality Assurance (QA) vs. Quality Work QA vs. Creativity QA and Temperament Types The Prevalence of Errors in Scientific Software: Use of Static Analyzers Factors of Code Quality Assurance Some Components of Project Code Certification Engineering Teams and the Division of Labor Personnel Roles, Code Levels, and Code Sources Desirable (But Not Required) Code Characteristics Code Documentation Code Module Communication Structure Code Updates Built in Automatic Error Detection Tests Designing for Code Maintenance Commercial Codes and their Users Code to Code Comparisons General Software Certification and 1SO-9000 Standards QA for Large Public Policy Projects QA of Analyses QA / Certification of Users and Regulators Assessment of Codes? Or Users?.' QA Procedures A Template for a QA System Concluding Remarks On QA 380 References for Chapter Chapter 12 Conclusions 385 v 12.1 The Overall Process for Quantification of Uncertainty Internet Archive Fulfilling the Promise of Computational Power 387

7 xiv Verification and Validation in Computational Science and Engineering Appendix A. Need.for Control of Numerical Accuracy 389 I. Introduction 390 II. Resistance and Objections 390 III. Difficulties in Applications 395 IV. Examples of What Can Be Done 396 V. Conclusions and Recommendations 396 Appendix: Editorial Policy Statement on the Control of Numerical Accuracy 398 Acknowledgments 399 References 399 Appendix B. Other Journal Policy Statements on Control of Numerical Accuracy 403 Journal of Heat Transfer Editorial Policy Statement on Numerical Accuracy 403 International Journal for Numerical Methods in Fluids-Editorial 406 AIAA Editorial Policy Statement on Numerical Accuracy and Experimental Uncertainty 407 Journal of Fluids Engineering-Editorial and Policy Statement 408 Policy Statement on the Control of Numerical Accuracy 410 Appendix C. Comment on Oreskes et al 413 Comment on "Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences" 413 References 417 Appendix D. CFD Parody: Will the Wind Tunnel Replace the Computer? 419 Original 419 Response 420 Appendix E. A Biographical Sketch of Lewis Fry Richardson 423 References 427 Index 429