Dimensions of Credibility in Models and Simulations
|
|
- Dustin Brown
- 6 years ago
- Views:
Transcription
1 Dimensions of Credibility in Models and Simulations Martin J. Steele, Ph.D. National Aeronautics and Space Administration (NASA) IT-C1 / Kennedy Space Center, FL Martin.J.Steele@nasa.gov June 2008
2 Overview Recent Historical Perspective The Idea of Credibility in M&S Dimensions of Credibility Concerns Findings Summary Final Thoughts 2
3 M&S Standard Development Mgt Decision Maker Interviews Pilot Studies CAIB Report Diaz Report NASA OCE Direction M&S Literature Interim Nov 06 Final Submitted Nov 07 2 Credibility Scales 1 New Credibility Scale External Efforts NASA-wide Formal Review 3
4 Recent Historical Perspective Outside NASA DoD M&S Validation (VPMM) M&S Quality DoE Predictive Capability Maturity Model (PCMM) ASME M&S V&V in Computational Solid Dynamics AIAA M&S V&V in Computational Fluid Dynamics Inside NASA The NASA Standards for Models & Simulations 2007 Summer Computer Simulation Conference Interim Version Development of NASA s Models and Simulations Standard 2008 Spring Simulation Interoperability Workshop Final Version 4
5 Goals for Excellence in M&S Goals of the M&S Standard Ensure that the credibility of M&S results is properly conveyed to those making critical decisions Assure that the credibility of M&S meets the project requirements Additionally, Form a strong foundation for disciplined development, validation and use of M&S Include a standard method to assess the credibility of the M&S presented to the decision maker 5
6 The Idea of Credibility in M&S
7 Development Foundation Accreditation Development Sargent, R. G. (c. 1980). Verification Validation 7
8 Operational Foundation Input Pedigree Results Operations Uncertainty Quant. Robustness Obtaining Credible Results Requires: Good Input Understanding of Uncertainty Sensitivity Analysis Accreditation Development Ver Val 8
9 Supporting Evidence Use History Supporting Evidence Model Mgt People Qual. Adding to Credibility: Past Use M&S Mgt Developer / User / Analyst Qualifications Accreditation Results Development Operations Ver Val Input Pedigree Uncertainty Quant. Robustness 9
10 Breadth to Consider Problem Statement System Understanding Modeling Choice Conceptual Model Validation Computational Model Verification System Analysis Validation Coding the Model Real World System System Understanding Detailed Understanding Validation Conceptual Model Verification Implementable Conceptual Model Accreditation Results Development Operations Supporting Evidence Ver Val Input Pedigree Uncertainty Quant. Robustness Use History Model Mgt People Qual. 10
11
12 Verification & Validation Verification Structure Flow Fidelity Validation: determining the degree to which a model or a simulation is an accurate representation of the real world How: Comparing to Conceptual Model Code Tracing Primitive Tests Min/Max Value Tests 12
13 Input: Source Notional Subject Matter Expert Applicability to current problem Referent Quality relative to current problem Referent System Referent Environment Input Pedigree Input Form: What s the character of your analysis? Average Uniform Triangular Quantity Curvilinear Authoritative Data 13
14 Accuracy & Uncertainty Accuracy: True Value Modeled Value Uncertainty in Modeled Value Uncertainty in True Value Uncertainty: Sources Size How Confident Epistemic Aleatory Reducible Irreducible Subjective Variability Model Form Inherent Incomplete Information Stochastic 14
15 Robustness Robustness of Results, i.e., Sensitivity of: The Real World System (RWS) The M&S Insensitive To Changes M&S Worst Situation M&S shows a Robustness not present in the RWS - Validation Issue - M&S not so useful Best Situation RWS is robust (Insensitive to Changes) & the M&S matches the RWS Better Sensitive To Changes OK Situation RWS is sensitive to change & the M&S matches the RWS Not a Good Situation M&S is not robust, but RWS is - Validation Issue - Results will be overly conservative Sensitive To Changes RWS Insensitive To Changes 15
16 Use History & Management Use History: Similarity of Uses Analogous Systems Exact Systems Length of Time in Use Just Developed Just Updated Long-Term Successful Use M&S Management: Models & Data under Configuration Control Models are Maintained Sustained 16
17 People Qualifications & Tech Review People Qualifications: Education Training Experience In M&S With the Modeled (Real World) System Technical Review: When accomplished During M&S Development During M&S Operations Ver Development Val Input Pedigree Operations Uncertainty Quant. Qualifications & Independence of the Peer Review Group: Self Internal Organization External Non-Expert to Expert Robustness 17 Level of Formalism Planning Documentation
18 When to Apply the Standard?
19 Scope of the M&S Standard Standard covers the use of M&S affecting: Critical Decisions Human Safety Mission Success As defined by each Program 19 The focus is on M&S for flight and ground support projects Operations Test & Evaluation Manufacturing and Assembly Design and Analysis M&S Results Influence 4: Controlling 3: Significant 2: Moderate 1: Minor 0: None IV: Negligible III: Marginal II: Critical I: Catastrophic Project Consequence Sample Risk Matrix
20 Basic Ideas Documentation of M&S Activities (Sections ) Credibility Assessment (Section 4.7 & Appendix A) Reporting to Decision Makers (Section 4.8) M&S Analysis Results A statement on the uncertainty in the results Credibility of M&S Results Identify Unfavorable outcomes Violation of assumptions Unfavorable Use Assessment Difference Between V&V & Use Assessment 20
21 Use Assessment Expected Output Range Validated Output Range Validated Input Domain Intended Input Domain 21 Note this is a 2-dimensional example of a potentially multidimensional input domain & multi-dimensional output range
22 Sommaire & Future Directions
23 Future Directions Track 1 Internal Deploy Use of Standard / Collection of Data on Use Assess Revise Track 2 M&S Discipline Specific Develop Guides (RPGs, Handbooks) For each M&S type in relation to the NASA M&S Standard Track 3 External Collaboration Other M&S Standards/Guides Professional/Academic Organizations 23
24 Broadening the Perspective of M&S V&V and Simplifying It!
25 Network Layered Protocol Approach Like the Layered Network Protocol Model 25
26 Layered M&S View (Influences in M&S Results) M&S V&V and Credibility Assessment User Input including Run Setup M/S Input M/S Output Model / Simulation Application Software Analyzing Output including Post-Processing of Output Data Industry Standards and Broad Use Operating System Software Computer Hardware Need for a Clearinghouse for Commercial & Open Source M&S Languages & Application Software 26
27 Final Thoughts
28 Concerns & Findings General Concerns Broad Applicability Subjectivity Software Similarity The Analysis vs. The People If we don t have this Standard Many varieties of Credibility Assessment will be are being presented Findings Yet Another Standard Terminology (Denotations & Connotations Types Purposes Education, Experience, & Training 28
29 Why doesn t/can t the Software Engineering Req ts do the job? Models and simulations may be implemented in software, but they encompass much more than software engineering The SW Engineering NPR (7150.2) does not address many issues critical for M&S, e.g., development of models validation against experimental or flight data uncertainty quantification operations and maintenance of M&S Models are sometimes developed in hardware Physical Models (Scaled) Analog Models Software Engineering Models & Simulations 29
30 Comparison Software Performs a task within a system Purpose performance of tasks/functions for a system Requirements are functions the s/w shall perform More discrete functions in the code (function-centric) Software shall do this S/W shall do that Etc. Traditional s/w functions either work or don t work Typically, there is no uncertainty M&S Provides a representation of a system Purpose Analysis or Representation of a system or of system functions for insight Requirements are behaviors the simulation model shall exhibit System behavior-centric Includes more assumptions how different parts of the simulation model work Typically includes uncertainty in the behavior of the simulation Totally focused on Software Development Must include requirements (& reporting) on M&S Use 30
31 Software & M&S You can follow software standards correctly in model development BUT STILL produce inaccurate or missleading results Software Standards do not deal with Software (or Model) use! 31
32 Final Statements Communicating Results Framework for Discussion of Results Credibility Credibility Factors Beyond VV&A Accreditation Results Development Operations Supporting Evidence Ver Val Input Pedigree Uncertainty Quant. Robustness Use History Model Mgt People Qual. Technical Review With dimensions composing each factor of M&S Credibility 32 It is: High Level Broadly Applicable (To All M&S Types!)
33 Thank You
Independent Verification and Validation (IV&V)
Independent Verification and Validation (IV&V) 12 th Annual NDIA CMMI Conference November 2012 - Denver, CO The MITRE Corporation The author s affiliation with The MITRE Corporation is provided for identification
More informationBest Practices for the Architecture, Design, and Modernization of Defense Models and Simulations
1 Best Practices for the Architecture, Design, and Modernization of Defense Models and Simulations Dr. Katherine L. Morse, JHU/APL Brian Miller, US Army CERDEC NVESD Michael Heaphy, OSD(AT&L)/DMSCO Outline
More informationVerifying and Validating Software in a Regulated Environment
Verifying and Validating Software in a Regulated Environment William J. Bryan Corporate Quality Manager ANSYS, Inc. KEYWORDS QUALITY, VERIFICATION, VALIDATION, REGULATED ABSTRACT Founded in 1970, ANSYS,
More informationOPTIMIZING THE SELECTION OF VV&A ACTIVITIES A RISK / BENEFIT APPROACH
OPTIMIZING THE SELECTION OF VV&A ACTIVITIES A RISK / BENEFIT APPROACH Paul R. Muessig Naval Air Warfare Center, Weapons Division Code 418000D China Lake, California 93555, U.S.A. Dennis R. Laack John W.
More informationNASA Systems Engineering Processes and Requirements
NASA NPR 7123.1B Procedural Effective Date: April 18, 2013 Requirements Expiration Date: April 18, 2018 RESPONSIBLE OFFICE: Office of the Chief Engineer COMPLIANCE IS MANDATORY NASA Systems Engineering
More informationCertification of Safety-Critical Software Under DO-178C and DO-278A
Certification of Safety-Critical Software Under DO-178C and DO-278A Stephen A. Jacklin 1 NASA Ames Research Center, Moffett Field, CA, 94035 The RTCA has recently released DO-178C and DO-278A as new certification
More informationCMMI-DEV V1.3 CMMI for Development Version 1.3 Quick Reference Guide
processlabs CMMI-DEV V1.3 CMMI for Development Version 1.3 Quick Reference Guide CMMI-DEV V1.3 Process Areas Alphabetically by Process Area Acronym processlabs CAR - Causal Analysis and Resolution...
More informationPrinciples of Verification, Validation, Quality Assurance, and Certification of M&S Applications
Introduction to Modeling and Simulation Principles of Verification, Validation, Quality Assurance, and Certification of M&S Applications OSMAN BALCI Professor Copyright Osman Balci Department of Computer
More informationStatistically-based. Principal Engineering Fellow Raytheon Integrated Defense Systems
Statistically-based Test Optimization Neal Mackertich, Ph.D. Principal Engineering Fellow Raytheon Integrated Defense Systems 16th Annual A lp Practical ti l Software S ft & Systems Measurement Users Group
More informationUnclassified NAVAIR Public Release Approved for public release: distribution is unlimited
DETERMINING M&S CREDIBILITY: What The Accreditor Needs To Know Presented To The National Defense Industrial Association Modeling and Simulation Committee, 19 April 2016 James N. Elele, PhD Department of
More informationENEA, Italian National Agency for New Technologies, Energy and Sustainable Economic Development Bologna, Italy
Open issues associated with passive safety systems reliability assessment L. Burgazzi ENEA, Italian National Agency for New Technologies, Energy and Sustainable Economic Development Bologna, Italy Abstract.
More informationV&V and QA throughout the M&S Life Cycle
Introduction to Modeling and Simulation V&V and QA throughout the M&S Life Cycle OSMAN BALCI Professor Department of Computer Science Virginia Polytechnic Institute and State University (Virginia Tech)
More informationSimulation Governance:
Simulation Governance: Managing Simulation as a Strategic Capability Dr. Keith Meintjes CIMdata As companies drive towards virtual product development and complete digital descriptions of their products
More informationPallet Thermal Model (PTM) Validation
Pallet Thermal Model (PTM) Validation ASME 2017 V&V Symposium Eugen Nisipeanu, PhD Carlos Corrales, MS May, 2017 Disclaimer The information shared in this meeting is not for redistribution or reuse and
More informationThe Verification Company. Software Development and Verification compliance to DO-178C/ED-12C
The Verification Company Software Development and Verification compliance to DO-178C/ED-12C DO-178C/ED-12C in Context Airworthiness Requirements Federal Aviation Regulation (FAR) 25 Airworthiness Standards:
More informationStatus and Near-Future Activities for NE-KAMS Development to Support Verification and Validation of Advanced Modeling and Simulation
Status and Near-Future Activities for NE-KAMS Development to Support Verification and Validation of Advanced Modeling and Simulation Weiju Ren, Ph.D. Oak Ridge National Laboratory ASME 2015 Verification
More informationEvolving Lockheed Martin s Engineering Practices Through the Creation of a Model-centric Digital Tapestry
Evolving Lockheed Martin s Engineering Practices Through the Creation of a Model-centric Digital Tapestry 2011 Frontiers in MBSE Workshop Christopher Oster MBSD Rollout Manager Lockheed Martin Corporation
More informationPart I FEA of Metallic Stents (F2514)
Part I FEA of Metallic Stents (F2514) PERU Workshop on Medical Device Regulation: Policy and Technical Aspects Kenneth E. Perry, Ph.D. ECHOBIO LLC Bainbridge Island, WA ASTM F2514 1.1 Purpose This guide
More informationAvailable online at ScienceDirect. Procedia CIRP 28 (2015 ) rd CIRP Global Web Conference
Available online at www.sciencedirect.com ScienceDirect Procedia CIRP 28 (2015 ) 179 184 3rd CIRP Global Web Conference Quantifying risk mitigation strategies for manufacturing and service delivery J.
More informationInteractive evaluative model trading for resilient systems decisions
2016 Conference on Systems Engineering Research Interactive evaluative model trading for resilient systems decisions Adam M. Ross a *, Matthew E. Fitzgerald a, and Donna H. Rhodes a a Massachusetts Institute
More informationModeling Trust in Critical Systems with Möbius KEN KEEFE SENIOR SOFTWARE ENGINEER LEAD MOBIUS DEVELOPER
Modeling Trust in Critical Systems with Möbius KEN KEEFE SENIOR SOFTWARE ENGINEER LEAD MOBIUS DEVELOPER Course Overview Objective: Give course attendees an intense introduction to modeling systems using
More informationUnderstanding Return On Investment (ROI) and Total Cost of Ownership (TCO) for Machine Translation
Understanding Return On Investment (ROI) and Total Cost of Ownership (TCO) for Machine Translation Dion Wiggins Chief Executive Officer dion.wiggins@asiaonline.net There will always be someone who says
More informationThe new ISO/IEC 17025:2017
The new ISO/IEC 17025:2017 By Dr. George Anastasopoulos Director, Conformity assessment, IAS Email: ganastasopoulos@iasonline.org Introduction - Background information ISO/IEC 17025 was first issued in
More informationNASA Systems Engineering: Status and Initiatives
NASA Systems Engineering: Status and Initiatives presentation to SEDC 17 September 2013 Joe Smith NASA HQ Program Executive for Systems Engineering 202-358-0192 joseph.s.smith@nasa.gov Page 1 Purpose Current
More informationIntegrated Modeling and Analysis to Support MBSE. Dr. Peter Menegay, Research Manager Dr. Hongman Kim, Sr. Research Engineer
Integrated Modeling and Analysis to Support MBSE Dr. Peter Menegay, Research Manager Dr. Hongman Kim, Sr. Research Engineer 1 Another Phoenix Related MBSE Effort Using MBSE Tools to Perform Business Case
More informationSRR and PDR Charter & Review Team. Linda Pacini (GSFC) Review Chair
SRR and PDR Charter & Review Team Linda Pacini (GSFC) Review Chair Review Requirements Review requirements are taken from the following documents: GSFC STD 1001 Criteria for Flight Project Critical Milestone
More informationImplementing Open Architecture. Dr. Tom Huynh Naval Postgraduate School
Implementing Open Architecture Dr. Tom Huynh Naval Postgraduate School 1 Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated
More informationIT Service-Based Costing: Standardizing Provisioning and Servicing IT Resources
IT Service-Based Costing: Standardizing Provisioning and Servicing IT Resources Paul Brown Emily Jessen SCEA/ISPA National Conference, Orlando, Florida June 2012 2011 TASC, Inc. Agenda Background What
More informationApplying PSM to Process Measurement
Applying PSM to Process Measurement Name: Bill Golaz Title: LM Fellow & LM Aero Measurement and Analysis Lead PSM All rights reserved. Copyright 2010 1 Topics Why Measure Why Process Measurement Using
More informationReport of the Reliability Improvement Working Group (RIWG) Volume II - Appendices
Report of the Reliability Improvement Working Group (RIWG) Volume II - Appendices Appendix 1 Formulate Programs with a RAM Growth Program II-1 1.1 Reliability Improvement Policy II-3 1.2 Sample Reliability
More informationRisk Mitigation in a Core Banking Conversion
Risk Mitigation in a Core Banking Conversion Greg Meidt, chief information officer Chemical Bank Dick Smies, vice president IBS Conversion Services 1 800 822 6758 Introduction A core banking migration
More informationModel Risk Management
Model Risk Management Brian Nappi, Crowe Horwath 2017 Crowe Horwath LLP Agenda Regulatory Perspectives on Model Risk Management Model Basics MRM Audit Considerations MRM Best Practices FAQ s 2017 Crowe
More informationModified Cooper Harper Scales (MCH)
Modified Cooper Harper Scales (MCH) Wierwille and Casali 1986 Background and applications The modified Cooper Harper scale is a uni- dimensional measure that uses a decision tree to elicit operator mental
More informationArchitecture Centric Evolution
Architecture Centric Evolution A Personal Perspective Dr. Charles ( Bud ) Hammons Software Engineering Institute Acquisition Support Program cbh@sei.cmu.edu Ground Systems Architecture Workshop 2005 1
More informationREGULATORY GUIDE An Approach for Using Probabilistic Risk Assessment In Risk-Informed Decisions On Plant-Specific Changes to the Licensing Basis
REGULATORY GUIDE 1.174 An Approach for Using... Page 1 of 38 July 1998 REGULATORY GUIDE 1.174 An Approach for Using Probabilistic Risk Assessment In Risk-Informed Decisions On Plant-Specific Changes to
More informationWorkshop Information IAEA Workshop
IAEA Training Course on Safety Assessment of NPPs to Assist Decision Making Overview of Deterministic Safety Analysis: Input Data, Verification & Validation, Conservative/BE Approaches (Part. 2) Lecturer
More informationCOST EFFECTIVE VV&A: FIVE PREREQUISITES
COST EFFECTIVE VV&A: FIVE PREREQUISITES P. R. Muessig & D. R. Laack Naval Air Warfare Center, Weapons Division (Code 418100D) China Lake, California 93555 ABSTRACT Recent requirements to establish the
More informationAnalytics: The Widening Divide
Neil Beckley, FSS Leader, IBM Growth Markets Analytics: The Widening Divide How companies are achieving competitive advantage through analytics What you will take away from this session 1 Understand Why
More informationIIBA Global Business Analysis Core Standard. A Companion to A Guide to the Business Analysis Body of Knowledge (BABOK Guide) Version 3
IIBA Global Business Analysis Core Standard A Companion to A Guide to the Business Analysis Body of Knowledge (BABOK Guide) Version 3 International Institute of Business Analysis, Toronto, Ontario, Canada.
More informationNAS-Wide Performance: Impact of Select Uncertainty Factors and Implications for Experimental Design
Approved for Public Release: 12-0370. Distribution Unlimited. NAS-Wide Performance: Impact of Select Uncertainty Factors and Implications for Experimental Design Gareth O. Coville, MITRE Corporation Billy
More informationUsing JCIDS DoDAF Architecture Primitives to Assemble a Repository for Enterprise-wide Analysis and Decision-Making
12TH ICCRTS Adapting C2 to the 21st Century Using JCIDS DoDAF Architecture Primitives to Assemble a Repository for Enterprise-wide Analysis and Decision-Making Topics: Track 1: C2 Concepts, Theory, and
More informationIntroduction and Key Concepts Study Group Session 1
Introduction and Key Concepts Study Group Session 1 PD hours/cdu: CH71563-01-2018 (3 hours each session) 2015, International Institute of Business Analysis (IIBA ). Permission is granted to IIBA Chapters
More informationInspection Qualification and Implementation of ENIQ in Sweden
4th International CANDU In-service Inspection Workshop and NDT in Canada 2012 Conference, 2012 June 18-21, Toronto, Ontario ABSTRACT Inspection Qualification and Implementation of ENIQ in Sweden Tommy
More information9. Verification, Validation, Testing
9. Verification, Validation, Testing (a) Basic Notions (b) Dynamic testing. (c) Static analysis. (d) Modelling. (e) Environmental Simulation. (f) Test Strategies. (g) Tool support. (h) Independent Verification
More informationPertemuan 2. Software Engineering: The Process
Pertemuan 2 Software Engineering: The Process Collect Your Project Topic What is Software Engineering? Software engineering is the establishment and sound engineering principles in order to obtain economically
More informationCERT Resilience Management Model Capability Appraisal Method (CAM) Version 1.1
CERT Resilience Management Model Capability Appraisal Method (CAM) Version 1.1 Resilient Enterprise Management Team October 2011 TECHNICAL REPORT CMU/SEI-2011-TR-020 ESC-TR-2011-020 CERT Program http://www.sei.cmu.edu
More informationAnnex IV: Competency Framework
Making the railway system work better for society. NSA Monitoring 120 Rue Marc Lefrancq BP 20392 FR-59307 Valenciennes Cedex 1 / 11 Contents 1. Introduction... 3 2. Roles and responsibilities... 3 2.1.
More informationStudent Guide Course: NISP Self Inspection
Course: NISP Self Inspection Lesson 1: Course Introduction 1. Course Information Purpose Provide a thorough understanding of the NISP self inspection process Audience POC Pass/Fail % Estimated completion
More informationPSM. Practical Software and Systems Measurement A foundation for objective project management. PSM TWG ISO/IEC and IEEE Standards.
Practical Software and Systems Measurement A foundation for objective project management PSM PSM TWG ISO/IEC and IEEE Standards March 29, 2006 PSM 1 29 Mar 06 Current Standard Activities ISO/IEC 15939,
More informationCONCERT. technologies. Collaborative Partnerships for Nationwide Rollouts & Global Technology Deployments
CONCERT technologies Collaborative Partnerships for Nationwide Rollouts & Global Technology Deployments 4th of 4 in the Series A Guide to Selecting the Right Company for Your Project A Concert Technologies
More informationGAIA. GAIA Software Product Assurance Requirements for Subcontractors. Name and Function Date Signature 15/09/05 15/09/05 15/09/05 15/09/05 15/09/05
Title Page : i Software Product Assurance Requirements for Subcontractors Name and Function Date Signature Prepared by D.MUNCH Prime Contractor SPA Manager 15/09/05 Verified by D.PERKINS E-SVM PA Manager
More informationWhat Functional Groups Are Included in the CMMI -SE/SW/IPPD/SS Model?
What Functional Groups Are Included in the CMMI -SE/SW/IPPD/SS Model? Software Engineering Process Group Conference March 9, 2004 Roland G. Weiss Joan Weszka Lockheed Martin Systems & Software Resource
More informationResults of the IEC Functional Safety Assessment
Results of the IEC 61508 Functional Safety Assessment Project: 3051S Electronic Remote Sensors (ERS ) System Customer: Emerson Automation Solutions (Rosemount, Inc.) Shakopee, MN USA Contract No.: Q16/12-041
More informationAuditing Standards and Practices Council
Auditing Standards and Practices Council PHILIPPINE STANDARD ON AUDITING 315 UNDERSTANDING THE ENTITY AND ITS ENVIRONMENT AND ASSESSING THE RISKS OF MATERIAL MISSTATEMENT PHILIPPINE STANDARD ON AUDITING
More informationIntroduction and Key Concepts Study Group Session 1
Introduction and Key Concepts Study Group Session 1 PDU: CH71563-04-2017 (3 hours) 2015, International Institute of Business Analysis (IIBA ). Permission is granted to IIBA Chapters to use and modify this
More informationReview Plan U.S. Army Corps of Engineers Northwest Division Kansas City District. Kanopolis Dam Emergency Gate Replacement
Review Plan U.S. Army Corps of Engineers Northwest Division Kansas City District Kanopolis Dam Emergency Gate Replacement April 2015 1. PURPOSE AND REQUIREMENTS 1.1 PURPOSE This Review Plan is intended
More informationWhat is the Systems Engineering Capability Model (SECM)?
What is the Systems Engineering Model (SECM)? EIA Interim Standard 731: SECM S E C A T 1 Can a Maturity Model Help You? Is your company successful at learning from past mistakes? Are you confident in your
More informationRole of prototyping in the design process Selecting the correct prototyping strategy Linkage with design strategy: early or late concept lock
Lecture 10: Prototypes Prototyping Role of prototyping in the design process Selecting the correct prototyping strategy Linkage with design strategy: early or late concept lock Assignment You need to develop
More informationNATIONAL NUCLEAR REGULATOR
NATIONAL NUCLEAR REGULATOR For the protection of persons, property and the environment against nuclear damage REGULATORY GUIDE Guidance on the Verification and Validation of Evaluation and Calculation
More informationOPERATING SYSTEMS. Systems and Models. CS 3502 Spring Chapter 03
OPERATING SYSTEMS CS 3502 Spring 2018 Systems and Models Chapter 03 Systems and Models A system is the part of the real world under study. It is composed of a set of entities interacting among themselves
More informationAS9102 Frequently Asked Questions
AS9102 Frequently Asked Questions General Who developed this standard? This International Standard for First Article Inspection (FAI) is developed by International Aerospace Quality Group (IAQG). The IAQG
More informationIAEA SAFETY STANDARDS Seismic Hazards in Site Evaluation of Nuclear Installations
IAEA SAFETY STANDARDS Seismic Hazards in Site Evaluation of Nuclear Installations Regional Workshop on Volcanic, Seismic, and Tsunami Hazard Assessment Related to NPP Siting Activities and Requirements
More informationOSADL SIL2LinuxMP - Linux Safety Qualification
- Safety rg> October 11, 2014 - Safety Justifying GNU/ A bit on s Why for Automotive Safety Why for safety? Satisfy Demands: General security demands Performance demands in cognitive systems Functional
More informationSYSTEMS ENGINEERING REQUIREMENTS AND PRODUCTS
SMC Standard SMC-S-001 1 July 2013 ------------------------ Supersedes: SMC-S-001 (2010) Air Force Space Command SPACE AND MISSILE SYSTEMS CENTER STANDARD SYSTEMS ENGINEERING REQUIREMENTS AND PRODUCTS
More informationHighlights of CMMI and SCAMPI 1.2 Changes
Highlights of CMMI and SCAMPI 1.2 Changes Presented By: Sandra Cepeda March 2007 Material adapted from CMMI Version 1.2 and Beyond by Mike Phillips, SEI and from Sampling Update to the CMMI Steering Group
More informationChanges to the SCAMPI Methodology and How to Prepare for a SCAMPI Appraisal
Changes to the SCAMPI Methodology and How to Prepare for a SCAMPI Appraisal Presented by: Lemis O. Altan SEI-Certified SCAMPI V1.3 Lead Appraiser for Development Process Edge International, Inc. Copyright
More informationMidterm Test Department: Computer Science Instructor: Steve Easterbrook Date and Time: 10:10am, Thursday Nov 9, 2006
CSC340 Requirements Engineering page 1/8 Faculty of Arts and Science University of Toronto Midterm Test Department: Computer Science Instructor: Steve Easterbrook Date and Time: 10:10am, Thursday Nov 9,
More informationIntroduction to software testing and quality process
Introduction to software testing and quality process Automated testing and verification J.P. Galeotti - Alessandra Gorla Engineering processes Engineering disciplines pair construction activities activities
More informationAgenda. Enterprise Risk Management Defined. The Intersection of Enterprise-wide Risk Management (ERM) and Business Continuity Management (BCM)
The Intersection of Enterprise-wide Risk (ERM) and Business Continuity (BCM) Marc Dominus 2005 Protiviti Inc. EOE Agenda Terminology and Process Introductions ERM Process Overview BCM Process Overview
More informationMODEL EVALUATION DOES YOUR MODEL MAKE SENSE TO SOMEONE ELSE?
1 MODEL EVALUATION DOES YOUR MODEL MAKE SENSE TO SOMEONE ELSE? 2 OVERVIEW What is evaluation? Why is it useful? How do you evaluate environmental models? Dealing with uncertainty. Practical Evaluation
More informationSystems Engineering Concept
Systems Engineering Concept WHITE PAPER February 2017 The Systems Engineering Concept provides practical hands-on methods and tools, that enable companies to meet today s global business challenges through
More informationQuestion 2: Requirements Engineering. Part a. Answer: Requirements Engineering Process
Question 2: Requirements Engineering Part a. Answer: Requirements Engineering Process The requirements engineering process varies from domain to domain. But the general activities involved are: Elicitation
More informationProcurement of Project Management Training Services-
Statement of Work Table of Contents 2. Purpose and Scope... 3 3. Background Information... 3 4. Objectives... 3 5. Statement of Work... 3 6. Requirement Specifications for Services... 4 7. Requirement
More informationBuy:
Chapter-1.0 CONTENTS OF CMMI MATURITY LEVEL 3 DOCUMENT KIT (More than 160 document files) A. The Total Editable Document kit has 8 main directories as below in word/excel. CMMI-V1.3 Maturity Level 3 (Dev)
More informationIgloo. Financial modeling software for managing risk
Igloo Financial modeling software for managing risk Property & casualty (P&C) insurers and reinsurers are facing growing market and regulatory pressures to improve risk management. Igloo is a financial
More informationNaval Set-Based Design
Naval Set-Based Design March 1, 2017 Dr. Norbert Doerry SEA 05TD norbert.doerry@navy.mil (202) 781-2520 See http://doerry.org/norbert/papers/papers.htm for more papers and presentations 1 Point Based vs
More informationPAI Inspections, Observations and Data Integrity
PAI Inspections, Observations and Data Integrity Krishna Ghosh, Ph.D. Office of Pharmaceutical Quality Office of Process and Facilities Center for Drug Evaluation and Research November, 2017 20 November
More informationRequirements Gathering using Object- Oriented Models
Requirements Gathering using Object- Oriented Models Software Quality Assurance What is software? According to the IEEE (Institute of Electrical and Electronics Engineers) A software is: Programs, procedures,
More informationin touch Greetings from the Chairman: Greetings from the Chairman New ISO and new 9004 to come Page 1
Greetings from the Chairman: Dear IPC InTouch Followers, Greetings This year Aegina Island, Greece hosted to the 20th General Assembly Meeting of IPC. We have again successfully completed an intensive
More informationReport. Quality Assessment of Internal Audit at <Organisation> Draft Report / Final Report
Report Quality Assessment of Internal Audit at Draft Report / Final Report Quality Self-Assessment by Independent Validation by Table of Contents 1.
More informationUse of the Capability Maturity Model Integration (CMMI ) in software engineering management on NASA missions
Use of the Capability Maturity Model Integration (CMMI ) in software engineering management on NASA missions Tim Crumbley NASA Deputy Manager for Software Engineering Technical Discipline Team Business
More informationMETHOD VALIDATION: AN ESSENTIAL COMPONENT OF THE MEASUREMENT PROCESS. Presented by : Jerry Parr, The NELAC Institute
METHOD VALIDATION: AN ESSENTIAL COMPONENT OF THE MEASUREMENT PROCESS Presented by : Jerry Parr, The NELAC Institute Source Material The NELAC Institute, Volume 1, Module 2, Quality Systems General Requirements,
More informationJoel Castaneda. January Tecolote Research, Inc.
Exploiting ACE to Create Build-Up Estimates Joel Castaneda ACEIT Users Conference, Santa Barbara, CA January 2009 Tecolote Research, Inc. Copyright Tecolote Research, Inc. January 2009 Abstract Bottom-up
More informationCertification Specifications. and. Guidance Material. for Simulator Data CS-SIMD
Annex to ED Decision 2014/033/R European Aviation Safety Agency Certification Specifications and Guidance Material for Simulator Data CS-SIMD Initial Issue 2 December 2014 1 1 For the date of entry into
More informationSurviving the Top Ten Challenges of Software Testing
Surviving the Top Ten Challenges of Software Testing: A Closer Look at Understanding Software Testing Randy Rice, CQA, CSTE Rice Consulting Services, Inc. 405-692-7331 http://www.riceconsulting.com rcs@telepath.com
More informationSAI Performance Measurement Framework Implementation strategy
SAI Performance Measurement Framework Implementation strategy 2017-19 24 September 2016 Draft SAI PMF strategy, 2017-19 Page 1 SAI PMF Strategy 2017-2019 1. Introduction 1.1 What is the SAI PMF? The SAI
More informationAugust 24, Jean-Philippe Mathevet IAQG Performance Stream Leader SAFRAN Paris, France. Re: 2014 Supplier Performance Data Summary
Page 1 of 27 August 24, 2105 Jean-Philippe Mathevet IAQG Performance Stream Leader SAFRAN Paris, France Re: 2014 Supplier Performance Data Summary Dear Jean-Philippe: The following report is a summary
More informationLife Cycle Assessment A product-oriented method for sustainability analysis. UNEP LCA Training Kit Module f Interpretation 1
Life Cycle Assessment A product-oriented method for sustainability analysis UNEP LCA Training Kit Module f Interpretation 1 ISO 14040 framework Life cycle assessment framework Goal and scope definition
More informationSeite 1. KUGLER MAAG CIE GmbH
Requirements Engineering and Management with ISO 26262 and Automotive SPICE October 25, 2012 Milan 10th Workshop on Automotive Software & Systems Fabio Bella Kugler Maag Cie KUGLER MAAG CIE GmbH Seite
More informationVoluntary Pilot Meeting Preview: How will CDRH apply assessments in the voluntary program?
Voluntary Pilot Meeting Preview: How will CDRH apply assessments in the voluntary program? Cisco Vicenty Case for Quality Program Manager Center for Devices and Radiological Health U.S. Food and Drug Administration,
More informationChapter 1 INTRODUCTION TO SIMULATION
Chapter 1 INTRODUCTION TO SIMULATION Many problems addressed by current analysts have such a broad scope or are so complicated that they resist a purely analytical model and solution. One technique for
More informationPepsiCo s Sustainable Farming Initiative Scheme Rules
PepsiCo s Sustainable Farming Initiative Scheme Rules 1 Table of Contents 1. Introduction... 3 1.1 Setting the Stage... 3 1.2 Purpose of the Scheme Rules... 4 1.3 SFI Continuous Improvement Process...
More informationINTERNATIONAL STANDARD ON AUDITING 315 UNDERSTANDING THE ENTITY AND ITS ENVIRONMENT AND ASSESSING THE RISKS OF MATERIAL MISSTATEMENT CONTENTS
INTERNATIONAL STANDARD ON AUDITING 315 UNDERSTANDING THE ENTITY AND ITS ENVIRONMENT AND ASSESSING THE RISKS OF MATERIAL MISSTATEMENT (Effective for audits of financial statements for periods beginning
More informationMBSE, PLM and the Digital Thread: Where do we go from here?
MBSE, PLM and the Digital Thread: Where do we go from here? Don Tolle Director, Simulation-Driven Systems Development Practice Email: d.tolle@cimdata.com Tel: +1.513.295.3641 Global Leaders in PLM Consulting
More informationTest Maturity Model Integration (TMMi) Process Improvement for the Present and the Future
WWW.QUALTECHCONFERENCES.COM Europe s Premier Software Testing Event World Forum Convention Centre, The Hague, Netherlands The Future of Software Testing Test Maturity Model Integration (TMMi) Process Improvement
More informationLaboratory Quality Assurance Manager & Laboratory Assessor RULES & HANDBOOK
EOQ Personnel Registration Scheme Laboratory Quality Assurance Manager & RULES & HANDBOOK Prepared by: Dr. Eugenia Soboleva, Quality Austria In accordance with the working group on EOQ product development
More informationFuture Oriented Job Analysis
Future Oriented Job Analysis If your computer does not have speakers, please dial in at: 1-866-901-6455 Access Code: 408-634-854 CPS HR Webinar Series Geoff Burcaw, M.S., SPHR Senior Consultant CPS HR
More informationThe Auditor-Client Relationship
Manasa L. Cooper June 22, 2017 We will examine and discuss issues that are opportunities to add value to the auditor-client relationship 2 1 Fixed Rate vs. Variable Rate in the Annual Audit Contract What
More informationBUDGET ANALYST COMPETENCY PROFILE
Description of Work: The Budget Analyst provides leadership, oversight, and support in the execution of the budget to agency head and/or division management. Analysts examine budgets and expenditures for
More informationSoftware Safety Testing Based on STPA
Available online at www.sciencedirect.com ScienceDirect Procedia Engineering 80 (2014 ) 399 406 3 rd International Symposium on Aircraft Airworthiness, ISAA 2013 Software Safety Testing Based on STPA Changyong
More information