SE420 Software Quality Assurance Lecture 2 Software Specification Part-1 January 16, 2017 Sam Siewert
SQA LO s (Learning Objectives) Theory and Principles 1. Coverage of Current SQA Theory and Practice 2. Concurrent SWE and SQA Design and Development 3. Execution of SQA Tests for Compliance Throughout Life-cycle 4. Integration of SQA with SWE Life-cycle for External or Internal SQA 5. Software Project Failure and Success Root-Causes 6. When are We Done Testing? (Exit Criteria) 7. Why Can t Software (Realistically) be Zero Defect? If not, How Do We Build Mission Critical Software Systems? Practice 1. Use of Key TOOLS for SQA Automation and Metrics 2. Taking Someone s Code [Good, Bad, Great, ] and Improve it 3. Unit Testing Code as You Go, Debugging Skills 4. Integration and Test 5. Acceptance Testing and Delivery 6. Holding Effective Walk-throughs and Inspections 7. Scaffold Assignments to Produce Improved Application Using SQA Process (Specification, Design, Implementation, Compliance) Sam Siewert 2
Reading This Week SQA Text, Chapter 2 as Planned Boehm on Guidelines for Software V&V of Requirements and Design Specifications Skim SWEBOK, Chapters 1 & 2 [Also on Canvas] Are These Guidelines Consistent (E.g. Boehm and SWEBOK) and Do they fit Lewis SQA Overview? Think About Lifecycle and SQA Role in it Sam Siewert 3
4 Dimensions of SW Quality 1. Specification (What) What will be built 2. Design (How) How it will be Built 3. Development (How) Software Construction 4. Conformance Was the right thing built (validation) and does it work (verification) Which Interrogatives are Left Out? Who? Stakeholders End Users, Management (Marketing, Finance, Sales, Project, Functional), Customer, etc. Where? Organization (Distributed Team?) Why? Marketing Study (MRD -> SRD), Product Research, Customer Request When? Schedules Enemy of Quality? Time to Market (Relevance of a product, need for it) Estimation with Defined Process? Sam Siewert 4
Requirements Baseline Validated Requirements (Specification) and form Baseline Plan of Record (for Design and Development Team) Perform Validation to Prior to Design and Code Construction Long Before Test Re-validate during Design and Testing, Again in Acceptance Test Most Costly Error is Shipping Invalid Software Sam Siewert http://csse.usc.edu/csse/techrpts/1979/usccse79-501/usccse79-501.pdf 5
Modern V model Careful Distinction of When V&V is Designed and When it is Used Missing flow? Yes No reason for Architecture not to cross validate with I&T Sam Siewert www.ndia.org/divisions/divisions/systemsengineering/documents 6
Waterfall Model Specification is First Phase of Waterfall Model Validation of Requirements Validation of Requirements Can be Done as Late as Testing, but Why Not Start Earlier? See Barry Boehm Guidelines and Cost Estimation of Requirements Errors Highest Cost Error Found in Testing Phase E.g. Missing Feature Inconsistency Requirements (Analysis, Specification, Validation) Design Construction Testing Maintenance Sam Siewert 7
Evolutionary or Spiral Model Phases - Radial Distance and Area Represents Effort (Time / Cost) Plan to Re-visit Phases Goal is to Control Cost/Effort and Impact of Mistakes Design (High Level, Detailed, Validation) Detailed, Validated Specification (Requirements, Design, Validation Tests) Specification (Analysis, Requirements, Validation) Detailed Design Project HLD Finalize Code Modules and Unit Tests (Code inspections) Finalize Design Define Services And interfaces Concept Code (walk-through) I & T Readiness Software Construction (Interfaces, Objects/Modules, Algorithms, Coding) Refine specification and re-validate Project Proposal (Concept Validation) Write/review Reference Project Unit Testing System Integration Test Evolutionary Prototype System Demonstration (Acceptance) Compliance (Testing: Unit, Integrated, Regression, Acceptance) All Modules Complete & Unit Tested Sam Siewert 8
Requirements Validation - Boehm Complete? Review Customer sign off Consistent? Source Traceability Where is Requirement met in Design? Where is it Tested? Feasible? Proof-of-Concept Needed? Risk, Impact, Cost Testable? E.g. Requires Fault Injection Sam Siewert 9
Overall SQA Process IEEE Computer SWEBOK version 3 Organizations for SQA - http://www.sei.cmu.edu/ Process and Validation of Process - http://whatis.cmmiinstitute.com/ Process Must Be Defined, Repeatable, and Improved Over Time What Does SWEBOK say about Requirements Specification? What Does Lewis SQA book say about Validation (see pages 15-16)? Semantic issue of Designing Validation Tests vs. Executing or Using Validation Tests to Perform Validation of Requirements, Design, Code SQA Design should be Concurrent with SWE Design We ll take a closer look at this in the V-model for concurrent SQA/SWE Do Both Agree with Boehm? Sam Siewert 10
Test Specification -> Execution Conducting a Test? Not Really Specify Test Plan Design Test(s) Implement Test Drivers and Scripts Test the Test(s) Execute or Run Tests Unit Test (Developer) Nightly Build & Smoke Test Regression Test Stress Test Negative Testing http://bibliolore.org/2013/09/03/zappa-and-classical-music/ http://uploadvr.com/samsung-provides-20-gear-vrs-for-a-virtual-reality-programming-class-in-southern-california/ code has Sam Siewert 11 https://www.cartoonstock.com
SWEBOK Definition and Comments The Software Requirements knowledge area (KA) is concerned with the elicitation, analysis, specification, and validation of software requirements as well as the management of requirements during the whole life cycle of the software product. software projects are critically vulnerable when the requirements-related activities are poorly performed. requirements express the needs and constraints placed on a software product that contribute to the solution of some real-world problem. Sam Siewert 12
Scaffold Assignment Strategy for LO s Use Working Example? [E.g. Secure messaging, File RAID, Image Processing] Assignment #1 Familiarity by Walk-through of Reference (Example) Walkthrough and Debug Skills (Compared to Inspection) Assignment #2 - Specify Secure SMS (Text) Messaging System? Other? Leverage Reference Example, or Adapt PGP Open Source, others? Requirements Validation (Review) Requirements Traceability HLD High Level Design Assignment #3 - Design Secure SMS Detailed Design Design Inspection (V&V) Assignment #4 - Construct Secure SMS Code Units, Code Inspection, Unit Test Development Assignment #5 - Test Secure SMS (Unit, Negative, Integrated, Regression) Unit Test (V&V) Fault Injection / Negative Testing (E.g. Cryptanalysis) Assignment #6 - Deliver Full Lifecycle Example with Acceptance Test to User/Customer Integrate (Integrated V&V), Bug Fix, Regression Test, Deliver Sam Siewert 13