An Overview of Software Reliability
|
|
- Cathleen Underwood
- 6 years ago
- Views:
Transcription
1 Software Design For Reliability (DfR) Seminar An Overview of Software Reliability Bob Mueller
2 Software Quality and Software Reliability Related Disciplines, Yet Very Different
3 Definition of Software Quality FACTORS CRITERIA Functionality suitability accuracy interoperability security Usability understandability learnability operability attractiveness Software Quality *ISO9126 Quality Model Reliability Efficiency maturity fault tolerance recoverability time behavior resource utilization Software Quality The level to which the software characteristics conform to all the specifications. Portability analysability changeability stability testability adaptability installability Maintainability co-existence replaceability (v0.5) Ops A La Carte 3
4 Most Common Misconception FACTORS Functionality Usability CRITERIA suitability accuracy interoperability security understandability learnability operability attractiveness What organizations believe they are doing We have a strong SW quality program. We don t need to add SW reliability practices. Software Quality *ISO9126 Quality Model Reliability Efficiency maturity fault tolerance recoverability time behavior resource utilization What is missing Implementing sufficient SW reliability practices to satisfy customer expectations Portability analysability changeability stability testability What the organizations are really doing Implementing only a sparse set of SW quality adaptability installability Maintainability co-existence replaceability practices (v0.5) Ops A La Carte 4
5 Software Design For Reliability (DfR) Seminar Background on Software Reliability
6 Software Reliability Can Be Measured Software Reliability is 20 years behind HW reliability Ramifications of failure Education on the consumer side Many consumers just expect unreliable s/w Education on the manufacturer s side Mfgs don t know new innovative methods Mfgs don t figure out how users will use product Software engineers are more free-spirited than HW Entry cost for a SW devel. team less than for HW (v0.5) Ops A La Carte 6
7 Reliability vs. Cost OPTIMUM COST POINT TOTAL COST CURVE RELIABILITY PROGRAM COSTS COST HW WARRANTY COSTS RELIABILITY The SW impact on HW warranty costs is minimal at best (v0.5) Ops A La Carte 7
8 Reliability vs. Cost, continued SW has no associated manufacturing costs, so warranty costs and saving are almost entirely allocated to HW If there are no cost savings associated with improving software reliability, why not leave it as is and focus on improving HW reliability to save money? One study found that the root causes of typical embedded system failures were SW, not HW, by a ratio of 10:1. Customers buy systems, not just HW. The benefits for a SW Reliability Program are not in direct cost savings, rather in: Increased SW/FW staff availability with reduced operational schedules resulting from fewer corrective maintenance content. Increased customer goodwill based on improved customer satisfaction. (v0.5) Ops A La Carte 8
9 Defining Software Reliability
10 Software Reliability Definitions The customer perception of the software s ability to deliver the expected functionality in the target environment without failing. Examine the key points Practical rewording of the definition Software reliability is a measure of the software failures that are visible to a customer and prevent a system from delivering essential functionality. (v0.5) Ops A La Carte 10
11 Software Reliability Can Be Measured Measurements are a required foundation Differs from quality which is not defined by measurements All measurements and metrics are based on run-time failures Only customer-visible failures are targeted Only defects that produce customer-visible failures affect reliability Corollaries Defects that do not trigger run-time failures do NOT affect reliability badly formatted or commented code defects in dead code Not all defects that are triggered at run-time produce customer-visible failures corruption of any unused region of memory SW Reliability evolved from HW Reliability SW Reliability focuses only on design reliability HW Reliability has no counterpart to this (v0.5) Ops A La Carte 11
12 Software Reliability Is Based On Usage SW failure characteristics are derived from the usage profile of a particular customer or set of customers Each usage profile triggers a different set of run-time SW faults and failures Example Examine product usage by 2 different customers Customer A s usage profile only exercises the sections of SW that produce very few failures. Customer B s usage profile overlaps with Customer A s usage profile, but additionally exercises other sections of SW that produce many, frequent failures. Customer assessment of the product s software reliability Customer A s assessment - the SW reliability is high Customer B s assessment - the SW reliability is low (v0.5) Ops A La Carte 12
13 Reliability Correctness Correctness is a measure of the degree of intended functionality implemented by the SW Correctness measures the completeness of requirements and the accuracy of defining a SW model based on these requirements Reliability is a measure of the behavior (i.e., failures) that prevents the software from delivering the implemented functionality (v0.5) Ops A La Carte 13
14 Defects, Faults, and Failures
15 Terminology Defect A flaw in the requirements, design or source code that produces implementation logic that will trigger a fault Defect Defects of omission Not all requirements were used in creating a design model The design satisfies all requirements but is incomplete The source code did not implement all the design The source code has missing or incomplete logic Defects of commission Incorrect requirements are specified Requirements are incorrectly translated into a design model The design is incorrectly translated into source code The source code logic is flawed Defects are static and can be detected and removed without executing the source code Defects that cannot trigger a SW failure are not tracked or measured Ex: quality defects, such as test case and soft maintenance defects, and defects in dead code (v0.5) Ops A La Carte 15
16 Terminology (continued) Fault The result of triggering a SW defect by executing the associated implementation logic Defect Faults are NOT always visible to the customer A fault can be the transitional state that results in a failure Trivially simple defects (e.g., display spelling errors) do not have intermediate fault states Fault Failure A customer (or operational system) observation or detection that is perceived as an unacceptable departure of operation from the designed SW behavior Failures MUST be observable by the customer or an operational system Failures are the visible, run-time symptoms of faults Not all failures result in system outages Defect Fault Failure (v0.5) Ops A La Carte 16
17 Basic Failure Classification High-level SW failure classification based on complexity and time-sensitivity of triggering the associated defect: Bohr Bugs Heisen Bugs Aging Bugs Bohr Bugs Named after the Bohr atom Connotation: Deterministic failures that are straight-forward to isolate Failures are easily reproducible, even after a system restart/reboot Most frequent failure category detected during development, testing and early deployment These are considered trivial defects since every execution of the associated logic results in a failure (v0.5) Ops A La Carte 17
18 Basic Failure Classification (continued) Heisen Bugs Named after the Heisenberg uncertainty principle Connotation: Failures that are difficult to isolate to a root cause Intermittent failures that are rarely triggered and difficult to reproducible. Unlikely to reoccur following a system restart/reboot Common root causes: Synchronization boundaries between SW components Improper or insufficient exception handling Interdependent timing of multiple events Rarely detected when the SW is not mature (i.e., during early development and testing phases) The best methods to deal with these tough defects are by Identification using SW failure analysis Impact mitigation using fault tolerant code (v0.5) Ops A La Carte 18
19 Basic Failure Classification (continued) Aging Bugs Attributed to the results of continuous, long-term operations or use Connotation: Failures resulting from accumulation of erroneous conditions Transient failures occur after extended run-time or functional cycles where the contributing faults have occurred numerous times Preceding faults may lead to system performance degradation before a failure occurs Extremely unlikely to reoccur following a system restart/reboot due to the longevity requirement Common root causes: Deterioration in the availability of OS resources (e.g., depletion of device handles, memory leaks, heap fragmentation) Data corruption Application race conditions Accumulation of numerical round-off errors Gradual data accumulation for sampling or queue build-up The best methods to deal with these tough defects are by Identification using SW failure analysis Impact mitigation using fault tolerant code (v0.5) Ops A La Carte 19
20 What Is Reliable Software??
21 Reliable Software Characteristics Operates within the reliability specification that satisfies customer expectations Measured in terms of failure rate and availability level The goal is rarely defect free or ultra-high reliability Gracefully handles erroneous inputs from users, other systems, and transient hardware faults Attempts to prevent state or output data corruption from erroneous inputs Quickly detects, reports and recovers from SW and transient HW faults SW provides system behave as continuously monitoring, self-diagnosing and self-healing Prevents as many run-time faults as possible from becoming system-level failures (v0.5) Ops A La Carte 21
22 Common Paths to Software Reliability Traditional SW Reliability Programs - Predictions Program directed by a separate team of reliability engineers Development process viewed as a SW-generating, black box Develop prediction models to estimate the number of faults in the SW Reliability techniques used to identify defects and produce SW reliability metrics Traditional HW failure analysis techniques, e.g., FMEAs or FTAs Defect estimation and tracking SW Process Control Based on the assumption of a correlation between development process maturity and latent defect density in the final SW Ex: CMM Level 3 organizations can develop SW with 3.5 defects/ksloc If the current process level does not yield the desired SW reliability, audits and stricter process controls are implemented Quality Through SW Testing Most prevalent approach for implementing SW reliability Assumes reliability is increased by expanding the types of system tests (e.g., integration, performance and loading) and increasing the duration of testing Measured by counting and classifying defects (v0.5) Ops A La Carte 22
23 Common Paths to Software Reliability (continued) These approaches generally do not provide a complete solution Reliability prediction models are not well-understood SW engineers find it difficult to apply HW failure analysis techniques to detailed SW designs Only 20% of the SW defects identified by quality processes during development (e.g., code inspections) affect reliability System testing is an inefficient mechanism for finding run-time failures Generally identifies no more than 50% of run-time failures Quality processes for tracking defects do not produce SW reliability information such as defect density and failure rates Net Effect: SW engineers still end up spending more than 50% of their time debugging, instead of focusing on designing or implementing source code (v0.5) Ops A La Carte 23
24 Design for Reliability (DfR)
25 Software Defect Distributions Average distribution of SW defects by lifecycle phase: 20% 30% 35% 10% 5% Requirements Design Coding Bad Defect Fixes (introduction of secondary defects) Customer Documentation Average distribution of SW defects at the time of field deployment: (based on 1 st year field defect report data) 1% 20% 35% 44% Severity 1 (catastrophic) Severity 2 (major) Severity 3 (minor) Severity 4 (annoyance) (v0.5) Ops A La Carte 25
26 Typical Defect Tracking (System Test) System Test Build Severity #1 Defects Found Severity #2 Defects Found Severity #3 Defects Found Severity #4 Defects Found Total Defects Found SysBuild SysBuild SysBuild SysBuild (v0.5) Ops A La Carte 26
27 Defect Origin and Discovery Typical Behavior Defect Origin Requirements Design Coding Testing Maintenance Defect Discovery Requirements Design Coding Testing Maintenance Surprise! Goal of Best Practices on Defect Discovery Defect Origin Requirements Design Coding Testing Maintenance Defect Discovery Requirements Design Coding Testing Maintenance (v0.5) Ops A La Carte 27
28 Defect Removal Efficiencies Defect removal efficiency is a key reliability measure Defects found Removal efficiency = Defects present Defects present is the critical parameter that is based on inspections, testing and field data Requirements Design Coding Unit Testing System & Subsystem Testing Stages Field Deployment Inspection Efficiency Testing Efficiency Overall Efficiency Example: Origin Defects Found Metric Removal Efficiency Inspections 90 Unit Testing 25 Inspection Efficiency 43% = (90 / 210) Testing Efficiency 38% = (80 / 210) System & Subsystem Testing 55 Overall Efficiency 81% = (170 / 210) Field Deployment 40 TOTAL 210 (v0.5) Ops A La Carte 28
29 Reliability Defect Tracking (All Phases) Activity Total Failures Found Total Critical Failures Found Defect Density Reqmts Critical Defects Found Design Critical Defects Found Code Critical Defects Found Unit Test Critical Failures Found System Test Critical Failures Found Reqmts % 12 Design % 4 41 Code % Unit Test % Development Totals DRE (development) % 80% 78% 100% System Test % DRE (after system testing) 55% 66% 56% 25% 100% (v0.5) Ops A La Carte 29
30 Defect Removal Technique Impact Design Inspections / Reviews n n n n Y n Y Y Code Inspections / Reviews n n n Y n n Y Y Formal SQA Processes n Y n n n Y n Y Formal Testing n n Y n n Y n Y Median Defect Efficiency 40% 45% 53% 57% 60% 65% 85% 99% (v0.5) Ops A La Carte 30
31 Typical Defect Reduction Goals SysBld SysBld SysBld SysBld SysBld #1 #2 #3 #4 #5 System Test (v0.5) Ops A La Carte 31
32 Design for Reliability Goal is to Predict Defect Totals for Next Phase Req Design Code Unit SysBld SysBld SysBld SysBld SysBld Field Test #1 #2 #3 #4 #5 Failures Development System Test Deployment (v0.5) Ops A La Carte 32
33 Software Reliability Practices
34 Goals of Reliability Practices Reliability Practices split the development lifecycle into 2 opposing phases: Pre-deployment Focus on Fault Intolerance Post-deployment Focus on Fault Tolerance Fault Avoidance Techniques Prevents defects from being introduced Fault Removal Techniques Detects and repairs faults Fault Tolerance Techniques Allow a system to operate predictably in the presence of faults System Restoration Techniques Quickly restore the operational state of a system in the simplest manner possible Goal: Increase reliability by eliminating critical defects that reduce the failure rate Goal: Increase availability by reducing or avoiding the effects of faults (v0.5) Ops A La Carte 34
35 Software Reliability Practices Analysis Formal Scenario/Checklist Analysis FRACAS FMECA FTA Petri Nets Change Impact Analysis Common Cause Failure Analysis Sneak Analysis Design Formal Interface Specification Defensive Programming Fault Tolerance Modular Design Error Detection and Correction Critical Functionality Isolation Design by Contract Reliability Allocation Design Diversity Verification Boundary Value Analysis Equivalence Class Partitioning Reliability Growth Testing Fault Injection Testing Static/Dynamic Code Analysis Coverage Testing Usage Profile Testing Cleanroom (v0.5) Ops A La Carte 35
36 Design and Code Inspections The original rationale for inspections (current payback): Inspections require less time and resources to detect and repair defects than traditional testing and debugging Work done at Nortel Technologies in 1991 demonstrated that 65% to 90% of operational defects were detected by inspections at 1/4 to 2/3 the cost of testing Soft maintenance rationale (future payback): Data collected on 130 inspection sessions findings on the long-term, software maintenance benefits of inspections as follows: True Defects - The code behavior was wrong and an execution affecting change was made to resolve it. False Positives Any issue not requiring a code or document change. Soft Maintenance Changes Any other issue that resulted in a code or document change, e.g., code restructuring or addition of code comments. (v0.5) Ops A La Carte 36
37 Spectrum of Inspection Methodologies Method / Originator Team Size # of Sessions Detection Method Collection Meeting Post Process Feedback Fagan Large 1 Ad hoc Bisant Small 1 Ad hoc Gilb Large 1 Checklist Yes Group oriented Yes Group oriented Yes Group oriented None None Root Cause Analysis Meetingless Inspection Large 1 Unspecified No Individual Oriented None ADR Small >1 Scenario Yes Group oriented None Britcher Unspecified 4 Parallel Scenario Yes Group oriented None Phased Inspection Small >1 Sequential Checklist (comp) No (Mtg only to reconcile data) None N-fold Small >1 Parallel Ad hoc Yes Group oriented None Code Reading Small 1 Ad hoc No Mtg Optional None WOW! No wonder inspections are not well-understood, there s too many methodologies. AND, THERE ARE MORE OPTIONS (v0.5) Ops A La Carte 37
38 Spectrum of Technical Review Methodologies Inspections are just one of the many classes of Technical Review Methodologies. Informal Individual initiative Small time commitment General Feedback Defect Detection Formal Team-oriented Multiple meetings and premeeting preparation Compliance with Standards Satisfies Specifications Adhoc Review Pairs Programming Team Review Peer Desk Check (Passaround Check) Walkthrough Inspection (v0.5) Ops A La Carte 38
39 Why Isn t Software Reliability Prevalent?? Those are very good ideas. We would like to implement them and we know we should try. However, there just isn t enough time. The erroneous arguments all assume testing is the most effective defect detection methodology Results from inspections/reviews are generally poor Engineers believe that testers will do a more thorough and efficient job of testing than any effort they implement (inspections and unit testing) Managers believe progress can be demonstrated faster and better once the SW is in the system test phase Remember, just like the story of the lumberjack and his ax, If you don t have time to do it correctly the first time, then you must have time to do it over later! (v0.5) Ops A La Carte 39
40 Software DfR Tools by Phase Phase Activities Tools Concept Define SW reliability requirements Benchmarking Internal Goal Setting Gap Analysis Design Architecture & High Level Design Low Level Design Modeling & Predictions Identify core, critical and vulnerable sections of the design Static detection of design defects SW Failure Analysis SW Fault Tolerance Human Factors Analysis Human Factors Analysis Derating Analysis Worst Case Analysis Coding Static detection of coding defects FRACAS RCA Unit Testing Dynamic detection of design and coding defects FRACAS RCA Integration and System Testing SW Statistical Testing FRACAS RCA SW Reliability Testing Operations and Maintenance Continuous assessment of product reliability FRACAS RCA (v0.5) Ops A La Carte 40
41 Questions? (v0.5) Ops A La Carte 41
Measuring and Assessing Software Quality
Measuring and Assessing Software Quality Issues, Challenges and Practical Approaches Kostas Kontogiannis Associate Professor, NTUA kkontog@softlab.ntua.gr The Software Life Cycle Maintenance Requirements
More informationIntroduction To Software Testing. Brian Nielsen. Center of Embedded Software Systems Aalborg University, Denmark CSS
Introduction To Software Testing Brian Nielsen bnielsen@cs.auc.dk Center of Embedded Software Systems Aalborg University, Denmark CSS 1010111011010101 1011010101110111 Software development cycle 1. Programmer
More informationR.POONKODI, ASSISTANT PROFESSOR, COMPUTER SCIENCE AND ENGINEERING, SRI ESHWAR COLLEGE OF ENGINEERING, COIMBATORE.
R.POONKODI, ASSISTANT PROFESSOR, COMPUTER SCIENCE AND ENGINEERING, SRI ESHWAR COLLEGE OF ENGINEERING, COIMBATORE. UNIT I INTRODUCTION Testing as an Engineering Activity Testing as a Process Testing axioms
More informationISTQB Certified Tester. Foundation Level. Sample Exam 1
ISTQB Certified Tester Foundation Level Version 2015 American Software Testing Qualifications Board Copyright Notice This document may be copied in its entirety, or extracts made, if the source is acknowledged.
More informationTesting 2. Testing: Agenda. for Systems Validation. Testing for Systems Validation CONCEPT HEIDELBERG
CONCEPT HEIDELBERG GMP Compliance for January 16-17, 2003 at Istanbul, Turkey Testing for Systems Validation Dr.-Ing. Guenter Generlich guenter@generlich.de Testing 1 Testing: Agenda Techniques Principles
More informationBCS THE CHARTERED INSTITUTE FOR IT. BCS HIGHER EDUCATION QUALIFICATIONS BCS Level 6 Professional Graduate Diploma in IT SOFTWARE ENGINEERING 2
BCS THE CHARTERED INSTITUTE FOR IT BCS HIGHER EDUCATION QUALIFICATIONS BCS Level 6 Professional Graduate Diploma in IT SOFTWARE ENGINEERING 2 Friday 30 th September 2016 - Morning Answer any THREE questions
More informationAdvantages and Disadvantages of. Independent Tests. Advantages. Disadvantages
8.0 Test Management Outline 8.1 Test organisation 8.2 Test planning and estimation 8.3 Test program monitoring and control 8.4 Configuration management 8.5 Risk and testing 8.6 Summary Independent Testing
More informationInternational Standard ISO/IEC 9126
International Standard ISO/IEC 9126 Software Engineering Product quality Part 1: Quality model ISO 9126 - Content Product quality and the lifecycle Quality models for: Internal Quality, External Quality
More informationBabu Madhav Institute Of Information Technology,UTU
UNIT: 1 Introduction Short Questions: 1. List the two types of software analyst? 2. What is the role of software test analyst? 3. Define Software Testing 4. Define verification. 5. Define validation. 6.
More informationBugs are costly... Kinds of Quality Assurance
Bugs are costly... 1. Types of bugs (What type of bugs have you had in the past?) a. Race conditions and deadlocks b. Library misuse c. Logical errors (off by one, null, buffer overflow) d. Usability e.
More informationTesting throughout the software life cycle. Software Testing: INF3121 / INF4121
Testing throughout the software life cycle Software Testing: INF3121 / INF4121 Summary: Week 2 Software development models Sequential / Iterative-Incremental / Testing within a life cycle Test levels Component
More informationDisciplined Software Testing Practices
isciplined oftware Testing Practices r. Magdy Hanna Chairman International Institute for oftware Testing ponsored by: International Institute for oftware Testing International Institute for oftware Testing,
More informationChapter 6. Software Quality Management & Estimation
Chapter 6 Software Quality Management & Estimation What is Quality Management Also called software quality assurance (SQA) s/w quality:- It is defined as the degree to which a system, components, or process
More information9. Verification, Validation, Testing
9. Verification, Validation, Testing (a) Basic Notions (b) Dynamic testing. (c) Static analysis. (d) Modelling. (e) Environmental Simulation. (f) Test Strategies. (g) Tool support. (h) Independent Verification
More informationRequirements Gathering using Object- Oriented Models
Requirements Gathering using Object- Oriented Models Software Quality Assurance What is software? According to the IEEE (Institute of Electrical and Electronics Engineers) A software is: Programs, procedures,
More informationSoftware Quality. Lecture 4 CISC 323. Winter 2006
Software Quality Lecture 4 CISC 323 Winter 2006 Prof. Lamb malamb@cs.queensu.ca Prof. Kelly kelly-d@rmc.ca Required Reading Barbara Kitchenam, Sheri Lawrence Pfleeger; The Elusive Target, IEEE Software
More informationIntroduction to Software Testing
Introduction to Software Testing Introduction Chapter 1 introduces software testing by : describing the activities of a test engineer defining a number of key terms explaining the central notion of test
More informationIntegration and Testing
Integration and Testing 1 Today Software Quality Assurance Integration Test planning Types of testing Test metrics Test tools 2 Deliverables by Phase Possible Deliverables by Phase Concept Document Statement
More informationIntroduction to software testing and quality process
Introduction to software testing and quality process Automated testing and verification J.P. Galeotti - Alessandra Gorla Engineering processes Engineering disciplines pair construction activities activities
More informationChapter 26. Quality Management
Chapter 26 Quality Management - Quality concepts - Software quality assurance - Software reviews - Statistical software quality assurance - Software reliability, availability, and safety - SQA plan (Source:
More informationCERTIFIED RELIABILITY ENGINEER (CRE) BODY OF KNOWLEDGE MAP 2018
CERTIFIED RELIABILITY ENGINEER (CRE) BODY OF KNOWLEDGE MAP 2018 The Certified Reliability Engineer (CRE) Body of Knowledge () has been updated to ensure that the most current state of reliability practice
More informationThis document describes the overall software development process of microcontroller software during all phases of the Company Name product life cycle.
Maturity Process Owner Check Release Description Valid Name / Department Name / Department Name / Department Detailed procedure for software development Title: Software Development Procedure Purpose: This
More information1. Can you explain the PDCA cycle and where testing fits in?
1. Can you explain the PDCA cycle and where testing fits in? Software testing is an important part of the software development process. In normal software development there are four important steps, also
More informationDeveloping Software Quality Plans a Ten Step Process. Phil Robinson Lonsdale Systems. Software Quality Plans. We all agree that you need one
ing Quality Plans a Ten Step Process Phil Robinson Lonsdale Systems lonsdale@iinet.net.au www.iinet.net.au/~lonsdale/ Quality Plans We all agree that you need one but What do you put in them? How do you
More informationISTQB Sample Question Paper Dump #11
ISTQB Sample Question Paper Dump #11 1. Which of the following is true a. Testing is the same as quality assurance b. Testing is a part of quality assurance c. Testing is not a part of quality assurance
More informationRisk Based Testing Pragmatic Risk Analysis and Management
Risk Based Testing Pragmatic Risk Analysis and Management Risk Based Testing Pragmatic Risk Analysis and Management What is Risk Based Testing? What is Risk Based Testing? Risk: the possibility of a negative
More informationActivity Metrics. (book ch 4.3, 10, 11, 12) RIT Software Engineering
Activity Metrics (book ch 4.3, 10, 11, 12) Overview Metrics that indicate how well we are performing various activities Requirements, Design, Coding, Testing, Maintenance, Configuration Management, Quality
More informationISTQB CTFL BH QuestionsAnswers with Explanation
ISTQB CTFL BH0-10 - QuestionsAnswers with Explanation For Software Testing Articles Visit @ http://softwaretestinghelp.com Join the Best Software Testing Training Course @ http://softwaretestinghelp.org
More informationContractual Aspects of Testing Some Basic Guidelines CONTENTS
CONTENTS 1 Introduction... 1 1.1 Background... 1 1.2 Structure... 1 1.3 Some Conventions... 1 1.4 Feedback... 1 2 Test Schedule List of Contents... 2 3 Testing Deliverables... 3 4 Coverage Guidance...
More informationB.H. Far
SENG 521 Software Reliability & Software Quality Chapter 14: SRE Deployment Department t of Electrical l & Computer Engineering, i University it of Calgary B.H. Far (far@ucalgary.ca) http://www.enel.ucalgary.ca/people/far/lectures/seng521
More informationGENERAL PRINCIPLES OF SOFTWARE VALIDATION
GUIDANCE FOR INDUSTRY GENERAL PRINCIPLES OF SOFTWARE VALIDATION DRAFT GUIDANCE Version 1.1 This guidance is being distributed for comment purposes only. Draft released for comment on: June 9, 1997 Comments
More informationTesting. Testing is the most important component of software development that must be performed throughout the life cycle
Testing Testing is the most important component of software development that must be performed throughout the life cycle Testing must be carried out by developers continuously More methodical testing must
More informationCompliance driven Integrated circuit development based on ISO26262
Compliance driven Integrated circuit development based on ISO26262 Haridas Vilakathara Manikantan panchapakesan NXP Semiconductors, Bangalore Accellera Systems Initiative 1 Outline Functional safety basic
More informationBuilding quality into the software from the. Keeping and. the software. software life cycle
SENG 521 Software Reliability & Software Quality Chapter 14: SRE Deployment Department t of Electrical l & Computer Engineering, i University it of Calgary B.H. Far (far@ucalgary.ca) http://www.enel.ucalgary.ca/people/far/lectures/seng521
More informationAlso we will try to recap what we studied in the last class. (Refer Slide Time: 00:35)
Embedded Software Testing Unit 3: Static analysis and code reviews & Metrics Lecture 5 Seer Akademi-NPTEL MOU Hi all welcome to the next session of our embedded software testing that unit 3 series and
More informationSoftware Quality Management
2004-2005 Marco Scotto (Marco.Scotto@unibz.it) Contents Definitions Quality of the software product Special features of software Early software quality models Boehm model McCall model Standard ISO 9126
More informationSoftware Engineering
Software Engineering (CS550) Software Testing - I Jongmoon Baik Objectives To define and understand what software testing is To understand software testing strategies To describe software testing processes
More informationA Practical Guide to Implementing Levels 4 and 5
A Practical Guide to Implementing Levels 4 and 5 CMMI Technology Conference & User Group 14-17 November 2005 Rick Hefner, Ph.D. Northrop Grumman Corporation Agenda An Overview of Levels 4 and 5 New Behaviors
More informationSkill Category 7. Quality Control Practices
Skill Category 7 Quality Control Practices Testing Concepts Developing Testing Methodologies Verification and Validation Methods Software Change Control Defect Management Process Management Processes CSQA
More informationIntroduction to Software Engineering
Introduction to Software Engineering (CS350) Lecture 16 Jongmoon Baik Software Testing Strategy 2 What is Software Testing? Testing is the process of exercising a program with the specific intent of finding
More informationSoftware Safety and Certification
Software Safety and Certification presented to IEEE Spring Switchgear Committee Luncheon Seminar 4 May, 2004 by Howard Cox Laboratories 1 What we will cover... Functional Safety Concepts from IEC 61508
More informationCapability Maturity Model for Software (SW-CMM )
PHASE-IV: SYSTEMS IMPLEMENTATION Software Quality Assurance Application Development Installation and Support Software Quality Assurance Capability Maturity Model for Software (SW-CMM ) The Capability Maturity
More informationCLASS/YEAR: II MCA SUB.CODE&NAME: MC7303, SOFTWARE ENGINEERING. 1. Define Software Engineering. Software Engineering: 2. What is a process Framework? Process Framework: UNIT-I 2MARKS QUESTIONS AND ANSWERS
More informationAn Application of Causal Analysis to the Software Modification Process
SOFTWARE PRACTICE AND EXPERIENCE, VOL. 23(10), 1095 1105 (OCTOBER 1993) An Application of Causal Analysis to the Software Modification Process james s. collofello Computer Science Department, Arizona State
More informationBetter Defect Analysis and Defect Prevention for Software Process Quality Improvement
International Journal of Innovative Research in Computer Science & Technology (IJIRCST) ISSN: 2347-5552, Volume-3, Issue-3, May- 2015 Better Defect Analysis and Defect Prevention for Software Process Quality
More informationKNOWLEDGE AREA: SOFTWARE QUALITY
KNOWLEDGE AREA: SOFTWARE QUALITY 1. Introduction Software Quality Assurance (SQA) and Verification and Validation (V&V) are the processes of the Knowledge Area on Software Quality. The scope of this Knowledge
More informationSoftware Quality Engineering Courses Offered by The Westfall Team
Building Skills is a 3-day course that is a subset of our course. The course is designed to provide a fundamental knowledge base and practical skills for anyone interested in implementing or improving
More informationIT6004/ Software Testing
1) Define Software Engineering. Software Engineering is a discipline that produces error free software with in a time and budget. 2) Define software Testing. Testing can be described as a process used
More informationSoftware Quality Engineering Courses Offered by The Westfall Team
Courses is a 2-day course that is a subset of our course. The course is designed to provide an overview of techniques and practices. This course starts with an overview of software quality engineering
More informationSoftware Quality Management
Software Quality Management Minsoo Ryu Hanyang University msryu@hanyang.ac.kr Outline Software Quality Model Software Quality Management Process and Quality Quality Metrics 2 2 What is Quality? Quality,
More informationAIRBORNE SOFTWARE VERIFICATION FRAMEWORK AIMED AT AIRWORTHINESS
27 TH INTERNATIONAL CONGRESS OF THE AERONAUTICAL SCIENCES AIRBORNE SOFTWARE VERIFICATION FRAMEWORK AIMED AT AIRWORTHINESS Yumei Wu*, Bin Liu* *Beihang University Keywords: software airworthiness, software
More information0 Introduction Test strategy A Test Strategy for single high-level test B Combined testing strategy for high-level tests...
TPI Automotive Test Process Improvement Version: 1.01 Author: Sogeti Deutschland GmbH Datum: 29.12.2004 Sogeti Deutschland GmbH. Version 1.01 29.12.04-1 - 0 Introduction... 5 1 Test strategy...10 1.A Test
More informationWork Plan and IV&V Methodology
Work Plan and IV&V Methodology Technology initiatives and programs should engage with an IV&V process at the project planning phase in order to receive an unbiased, impartial view into the project planning,
More informationIntroduction to Software Project Management. CITS3220 Software Requirements & Project Management
Introduction to Software Project Management CITS3220 Software Requirements & Project Management "A project gets a year late one day at a time." "Anything that can be changed will be changed until there
More informationCapability Maturity Model the most extensively used model in the software establishments
International Journal of Scientific and Research Publications, Volume 6, Issue 5, May 2016 710 Capability Maturity Model the most extensively used model in the software establishments Ajith Sundaram Assistant
More informationT52-Software Engineering
T52-Software Engineering Unit - V Implementation and Integration: Implementation Phase Integration Phase - System testing Maintenance Phase. Software Quality Assurance: Quality concepts - cost of quality
More informationSoftware Metrics & Software Metrology. Alain Abran. Chapter 10 Analysis of Quality Models and Measures in ISO 9126
Software Metrics & Software Metrology Alain Abran Chapter 10 Analysis of Quality Models and Measures in ISO 9126 1 Agenda This chapter covers: Introduction to ISO 9126 The analysis models in ISO 9126 as
More informationRequirements Verification and Validation
SEG3101 (Fall 2010) Requirements Verification and Validation SE502: Software Requirements Engineering 1 Table of Contents Introduction to Requirements Verification and Validation Requirements Verification
More informationSoftware Quality Factors
Software Quality Factors The need for a comprehensive software quality requirements There are some characteristic common : All the software projects satisfactory fulfilled the basic requirements for correct
More informationVendor: ISEB. Exam Code: BH Exam Name: ITIL V3 Foundation Certificate in IT Service Management. Version: Demo
Vendor: ISEB Exam Code: BH0-006 Exam Name: ITIL V3 Foundation Certificate in IT Service Management Version: Demo QUESTION 1 Which of the following is NOT an example of Self-Help capabilities? A. Requirement
More informationIT6004/ Software Testing Unit-I
Question Bank: IT6004/ Software Testing Unit-I 1) Define Software Engineering. Software Engineering is a discipline that produces error free software with in a time and budget. 2) Define software Testing.
More informationDHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING IT6004 SOFTWARE ESTING UNIT I : INTRODUCTION
DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING IT6004 SOFTWARE ESTING UNIT I : INTRODUCTION PART A (2 Marks) 1) Define Software Engineering. Software Engineering
More informationSOFTWARE QUALITY IN 2005 A SURVEY OF THE STATE OF THE ART SOURCES OF SPR S QUALITY DATA. SPR clients from 1984 through 2005 BASIC DEFINITIONS
Software Productivity Research LLC SOFTWARE QUALITY IN 2005 A SURVEY OF THE STATE OF THE ART Capers Jones, Founder and Chief Scientist http://www.spr.com cjones@spr.com May 2, 2005 SOURCES OF SPR S QUALITY
More informationWHAT DO YOU NEED TO KNOW ABOUT SOFTWARE MAINTENANCE
WHAT DO YOU NEED TO KNOW ABOUT SOFTWARE MAINTENANCE Alain April, A. Abran and R. Dumke Software accounts now for a increasing share of the content of modern equipments and tools, and must similarly be
More informationISTQB CTFL BH0-010 Exam Practice Question Paper
ISTQ TFL H0-010 Exam Practice Question Paper For Software Testing rticlesvisit @ http://softwaretestinghelp.com Join the est Software Testing Training ourse @ http://softwaretestinghelp.org QUESTION 1:
More informationTest Management: Part I. Software Testing: INF3121 / INF4121
Test Management: Part I Software Testing: INF3121 / INF4121 Summary: Week 6 Test organisation Independence Tasks of the test leader and testers Test planning and estimation Activities Entry and exit criteria
More informationAnalysis of Software Artifacts
Analysis of Software Artifacts Inspection Inspection Jonathan Aldrich Analysis of Software Artifacts Portions 2007 by William L Scherlis. Used by permission. 1 The Computer s Perspective http://www.xkcd.com/371/
More informationMTAT : Software Testing
MTAT.03.159: Software Testing Lecture 01: Introduction to Software Testing (Textbook Ch. 1-3) Spring 2016 Dietmar Pfahl email: dietmar.pfahl@ut.ee Structure of Lecture 1 Introduction and Motivation Course
More informationT Software Testing and Quality Assurance Test Planning
T-76.5613 Software Testing and Quality Assurance 10.10.2007 Test Planning Juha Itkonen Outline Test planning, purpose and usage of a test plan Topics of test planning Exercise References: IEEE Std 829-1998,
More informationSOFTWARE QUALITY IN 2002: A SURVEY OF THE STATE OF THE ART
Software Productivity Research an Artemis company SOURCES OF SPR S QUALITY DATA SPR clients from 1984 through 2002 SOFTWARE QUALITY IN 2002: A SURVEY OF THE STATE OF THE ART Capers Jones, Chief Scientist
More informationRequirements Engineering: Part I. Software Requirements & Project Management CITS3220
Requirements Engineering: Part I Software Requirements & Project Management CITS3220 The Problems of Requirements What goal(s) are we trying to satisfy? How do we identify the scope and properties of the
More informationWhat is SQA? Software Quality Assurance. Quality Concepts. Quality Concept (cont.)
What is SQA? Software Quality Assurance Š SQA includes a quality management approach effective software engineering methodology (methods and tools) formal technical reviews a multitiered testing strategy
More informationSurviving the Top Ten Challenges of Software Testing
Surviving the Top Ten Challenges of Software Testing: A Closer Look at Understanding Software Testing Randy Rice, CQA, CSTE Rice Consulting Services, Inc. 405-692-7331 http://www.riceconsulting.com rcs@telepath.com
More informationChapter 4 Software Process and Project Metrics
Chapter 4 Software Process and Project Metrics 1 Measurement & Metrics... collecting metrics is too hard... it's too time-consuming... it's too political... it won't prove anything... Anything that you
More informationMeasuring software product quality during testing
Measuring software product quality during testing Rob Hendriks, Robert van Vonderen and Erik van Veenendaal Quality requirements of software products are often described in vague and broad terms. As a
More informationTest Management is Risk management. Risk Based Testing
Test Management is Risk management Risk Based Testing Hans Schaefer Software Test Consulting Reigstad N-5281 Valestrandsfossen NORWAY Phone +47 56 394880 Fax +47 56 394570 e-mail hans.schaefer@ieee.org
More informationIEC Functional Safety Assessment
IEC 61508 Functional Safety Assessment Project: 3051S HART Advanced Diagnostics Pressure Transmitter, option code DA2 Customer: Rosemount Inc. (an Emerson Process Management company) Chanhassen, MN USA
More informationSuccessfully Integrating Test Automation and Agile Projects 10/7/2009. Presented to Annex Consulting Group CIO Breakfast October 7, 2009
Presented to Annex Consulting Group CIO Breakfast October 7, 2009 Successfully Integrating Test Automation and Agile Projects Silverpath Technologies Inc. Trevor.Atkins@silverpath.com Thinking Through
More informationValidation, Verification and MER Case Study
Validation, Verification and MER Case Study Prof. Chris Johnson, School of Computing Science, University of Glasgow. johnson@dcs.gla.ac.uk http://www.dcs.gla.ac.uk/~johnson Introduction. Definitions and
More informationSOFTWARE QUALITY IN 2002: A SURVEY OF THE STATE OF THE ART
Software Productivity Research an Artemis company SOFTWARE QUALITY IN 2002: A SURVEY OF THE STATE OF THE ART Capers Jones, Chief Scientist Emeritus Six Lincoln Knoll Lane Burlington, Massachusetts 01803
More informationSOFTWARE QUALITY IN 2002: A SURVEY OF THE STATE OF THE ART
Software Productivity Research an Artemis company SOURCES OF SPR S QUALITY DATA SPR clients from 1984 through 2002 SOFTWARE QUALITY IN 2002: A SURVEY OF THE STATE OF THE ART Capers Jones, Chief Scientist
More informationBASICS OF SOFTWARE TESTING AND QUALITY ASSURANCE. Yvonne Enselman, CTAL
BASICS OF SOFTWARE TESTING AND QUALITY ASSURANCE Yvonne Enselman, CTAL Information alines with ISTQB Sylabus and Glossary THE TEST PYRAMID Why Testing is necessary What is Testing Seven Testing principles
More informationSoftware verification and validation. Introduction
Software verification and validation. Introduction Marius Minea September 27, 2017 Topics be discussed Black-box testing (no source access) Glass-box/white-box testing (with source access) Generating unit
More informationQuality 24 Process Improvement 26 Real processes. Product Quality. Quality Management. Quality Management. Quality Plan
Quality 24 Process Improvement 26 Real processes Cleanroom RUP XP Software Engineering 55 Software Engineering 56 Product Quality Acceptable: usable, learnable, compatible Efficient: response time, memory
More informationGAIA. GAIA Software Product Assurance Requirements for Subcontractors. Name and Function Date Signature 15/09/05 15/09/05 15/09/05 15/09/05 15/09/05
Title Page : i Software Product Assurance Requirements for Subcontractors Name and Function Date Signature Prepared by D.MUNCH Prime Contractor SPA Manager 15/09/05 Verified by D.PERKINS E-SVM PA Manager
More informationSample Exam ISTQB Agile Foundation Questions. Exam Prepared By
Sample Exam ISTQB Agile Foundation Questions Exam Prepared By November 2016 1 #1 Which of the following is the correct pairing according to the Agile Manifesto statement of values? a. Individuals and Interactions
More informationData and Information. Work session for Non-Practitioners
Data and Information Work session for Non-Practitioners Eight Dimensions of an Organisation Strategy Governance Investment Policy Standards Performance Business Data and Information Application and Software
More informationAMERICAN SOCIETY FOR QUALITY CERTIFIED RELIABILITY ENGINEER (CRE) BODY OF KNOWLEDGE
AMERICAN SOCIETY FOR QUALITY CERTIFIED RELIABILITY ENGINEER (CRE) BODY OF KNOWLEDGE The topics in this Body of Knowledge include additional detail in the form of subtext explanations and the cognitive
More informationmywbut.com Software Reliability and Quality Management
Software Reliability and Quality Management 1 Software Reliability Issues 2 Specific Instructional Objectives At the end of this lesson the student would be able to: Differentiate between a repeatable
More informationQuestion Bank Unit 4 & Unit 5
VELAMMAL COLLEGE OF ENGINEERING & TECHNOLOGY Course Name Branch/Semester Subject Code &Name Faculty incharge with Designation Question Bank : B. Tech : IT/VIII : IT6013/Software Quality Assurance : Dr.
More informationContents of the Failure Mode Effects Analysis the Plant Wellness Way Distance Education Course FMEA Training Online
Contents of the Failure Mode Effects Analysis the Plant Wellness Way Distance Education Course FMEA Training Online Use the FMEA process in your plant and equipment design, and for root cause analysis
More informationThe Components of the SW Quality Assurance System - Overview. 08/09/2006 SE7161 Software Quality Assurance Slide 1
The Components of the SW Quality Assurance System - Overview SE7161 Software Quality Assurance Slide 1 The SQA System An SQA architecture An SQA system always combine a wide range of SQA components, all
More informationTesting and Inspections (3C05/D22) Unit 11: Testing and Inspection. What is Testing?
Testing and Inspections (3C05/D22) Unit 11: Testing and Inspection Objectives To introduce software testing and to develop its role within the software development process. To introduce the use of formal
More informationBuilding High Assurance Systems with SAFe 4.0
Building High Assurance Systems with SAFe 4.0 Agile 2016 By Dean Leffingwell 2016 Scaled Agile, Inc. All Rights Reserved. 2016 Scaled Agile, Inc. All Rights Reserved. V4.0.0 1 What is a high assurance
More informationCHAPTER 2. Slide 2.1 THE SOFTWARE PROCESS
CHAPTER 2 Slide 2.1 THE SOFTWARE PROCESS Overview Slide 2.2 Client, Developer, and User Requirements Phase Specification Phase Design Phase Implementation Phase Integration Phase Maintenance Phase Retirement
More information7. Model based software architecture
UNIT - III Model based software architectures: A Management perspective and technical perspective. Work Flows of the process: Software process workflows, Iteration workflows. Check Points of The process
More informationSoftware Quality. A Definition of Quality. Definition of Software Quality. Definition of Implicit Requirements
Definition of Software Quality Software Quality The Ultimate Goal of Software Engineering Software must conformance to explicit and implicit requirements if it is to be considered to be of good quality.
More informationDO-178B 김영승 이선아
DO-178B 201372235 김영승 201372237 이선아 Introduction Standard Contents SECTION 1 INTRODUCTION SECTION 2 SYSTEM ASPECTS RELATING TO SOFTWARE DEVELOPMENT SECTION 3 SOFTWARE LIFE CYCLE SECTION 4 SOFTWARE PLANNING
More informationTest Workflow. Michael Fourman Cs2 Software Engineering
Test Workflow Michael Fourman Introduction Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests for every build within the iteration System tests
More informationSOFTWARE FAILURE MODES EFFECTS ANALYSIS OVERVIEW
SOFTWARE FAILURE MODES EFFECTS ANALYSIS OVERVIEW Copyright, Ann Marie Neufelder, SoftRel, LLC, 2010 amneufelder@softrel.com www.softrel.com This presentation may not be copied in part or whole without
More information