SE420 Software Quality Assurance Lecture 1 Introduction Part-2 January 16, 2017 Sam Siewert
Course Learning Objectives Theory of Overall SQA Process Process Models (Waterfall, Spiral, XP) using Agile Strategy Terminology (IEEE SWEBOK Chapter 4), Prepare for ISTQB Foundation Certification Principles and Methods for SQA Organization for SQA Practice Design Methods, Software Construction and Testing (Exam-1) 1. Requirements and Use of Basic Design Models (SA/SD from SE300, OOA/D from SE310) 2. Code Walk-throughs 3. Debugging 4. Feature Addition and Performance Tuning 5. Unit Testing, Exit Criteria (CSUs) 6. Code Inspection 7. Validation and Verification (Code Implements Design, Design is Correct) Practice, and Scaffold for Final Requirements and Design (Exam-2) 1. Integration and Test (CSUs to Build and Test CSCI) 2. System Testing (CSC) 3. Acceptance Testing 4. Regression and Test Automation 5. Overall Metrics, Process Improvement (SEI CMM) 6. Delivery and Release Notes 7. SQA Inspection Sam Siewert 2
Linux Skills Introduction Session Part-2 January 12, 2016 Sam Siewert
C Code Unit Development and Test Assignment #1 Questions? Examples-Crypto.zip Hands on Sessions to: 1. Download and unzip code 2. Build code 3. Modify Makefiles (for debug with g) 4. Re-build 5. Debug with DDD 6. Inspect variable values during debug 7. Set breakpoints 8. Step Over 9. Step Into 10. 11. If code SEGFAULTS, dump core and debug the core Sam Siewert 4
Powerful Features of Visual Debug Code Walkthroughs Static Analysis Dynamic too Display Data Structures Evolution Recursive Link List Trees Graphs Multiple Indirections CS317 B-tree Visualized Sam Siewert 5
Discussion SQA History (read SQA Text Chapter #1) Come to Class Prepared to Discuss Gurus (e.g. Edward Deming, Grady Booch, Barry Boehm, Edward Yourdon, Tom DeMarco, ) Organizations for SQA - http://www.sei.cmu.edu/ Process and Validation of Process - http://whatis.cmmiinstitute.com/ Assignment #1 Discussion I will Post Every Other Tuesday, We ll Discuss, Due Following Week on Friday Late Assignments 10% Penalty for 3-day late Turn-in [Sunday], After Monday, only with Instructor Permission Sam Siewert 6
Wisdom from QA, SQA & SWE Gurus W. Edwards Deming "You can expect what you inspect." Edsger Dijkstra - program testing can be used to show the presence of bugs, but never to show their absence E.W. Dijkstra, Notes on Structured Programming, T.H.-Report 70-WSE-03, Technological University, Eindhoven, 1970; http://www.cs.utexas.edu/users/ewd/ewd02xx/ewd249.pdf Barry Boehm Are we building the right product? [validation]; Are we building the product right? [verification] Sam Siewert 7
Overall SQA Process IEEE Computer SWEBOK version 3 Organizations for SQA - http://www.sei.cmu.edu/ Process and Validation of Process - http://whatis.cmmiinstitute.com/ Process Must Be Defined, Repeatable, and Improved Over Time Cost of Mistakes in Early Phases is Higher (E.g. Requirements or Design mistakes compared to Implementation) Organizational and Individual Engineer SQA More than One Acceptable Process (E.g. Agile strategy hosting Spiral process, using OO & Structured analysis and design) Phases Do Not Have to Be Fully Completed? Waterfall? Sam Siewert 8
Dimensions of SW Quality 1. Specification (What) What will be built 2. Design (How) How it will be Built (interface, function, protocol) 3. Development (How) Software Construction 4. Conformance Was the right thing built (validation) and does it work (verification) Basic Interrogatives (Start of Any New Project) Who? Team Where? Organization (Distributed Team?) Why? Marketing Study, Product Research, Customer Request What? Capabilities and Requirements Specification When? Schedules Enemy of Quality? Estimation with Defined Process? When is Testing Done? (Exit Criteria) How? Design Details and Software Construction Sam Siewert 9
Dimensions form Outline for SQA Process Concept, Statement of Work, Marketing Study, Team Building 1. SW Requirements Analysis, Specification, Validation 2. SW Design High Level Interfaces, Data flow, Concurrency, Objects/Modules Detailed State Machines, Features and Function, Algorithms Design Validation (E.g. Simulation with MATLAB) 3. SW Construction 4. SW Testing Unit Testing Integrated Testing Regression Testing Acceptance Testing 5. Maintenance Field Support, Sustaining Engineering Sam Siewert 10
Waterfall Model Complete One Phase and Do Not Return Requirements Waterfall with Feedback Modification (Go Back One or More Phases) Defined and Disciplined, but Criticized as Impractical Too Much Up Front Effort Difficult to Collaborate Rigid Design Construction Testing Maintenance Sam Siewert 11
Evolutionary or Spiral Model Phases, But Radial Distance and Area of Phase Represents Effort (Time / Cost) Plan to Re-visit Phases Goal is to Control Cost/Effort and Impact of Mistakes Design (High Level, Detailed, Validation) Detailed, Validated Specification Specification (Analysis, requirements, Validation) Detailed Design Project HLD Finalize Code Modules and Unit Tests (Code inspections) Finalize Design Define Services And interfaces Concept Code (walk-through) I & T Readiness Software Construction (Interfaces, Objects/Modules, Algorithms, Coding) Refine specification and re-validate Project Proposal Write/review Reference Project Unit Testing System Integration Test Evolutionary Prototype System Demonstration (Acceptance) Compliance (Testing: Unit, Integrated, Regression, Acceptance) All Modules Complete & Unit Tested Sam Siewert 12
SQA Many Activities to Coordinate SQA Management is Not Simple (Parallel with SWE Development) Activities Need to Be Scheduled at Appropriate Phase Can t have Code Inspection at a Walk-through Can t Do Integrated Testing Prior to Unit Tests Defined, Repeatable, Measureable, Repeatable Basic Requirements for Process May Tailor for Market Mission Critical Avionics Software In-Flight Infotainment System Mobile App for Android Storage Data Path Digital Media System Sam Siewert 13
Stretch Goals and Objectives Focus on Applying Requirements and Design Methods for SQA Use Case Studies (SQA Pitfalls) What Can Go Wrong? Why it Did? Cost to Fix? Requirements V&V (System and Acceptance Test) Design V&V (Unit and I&T) Agile Strategy for Evolutionary Process in SQA Sam Siewert 14
SQA Knowledge Survey Unit test drivers should attempt to modify the code being tested as little as possible: Edsger Dijkstra observed that Program testing can be used to show the presence of bugs, but never to show their absence!, so we most often define exit criteria for testing modules in terms of metrics such as path coverage rather than a zero defect claim: Path coverage criteria can be used only for White-box testing: Sam Siewert 15
SQA Knowledge Survey Black-box testing requires a mixed-mode source/assembly debugger, disassembly of source code into instructions and careful attention to compiler features such as short-circuit logic and conditional execution of instructions: Regression testing includes tests from all other phases of test development with the purpose to make sure that when a defect is fixed, that somehow new bugs are not introduced : Integration testing focuses on testing module-to-module interfaces and communication protocols to verify and validate module interfaces before they are integrated to compose an architecture design: Sam Siewert 16
SQA Knowledge Survey Module designs and code should be tested only for correct function and not stress, performance, or soak tested: Stress and soak time testing of a module with a unit test driver might reveal a memory leak in the module: Regression testing should only be run before a software product is shipped: If SQA Design is Concurrent with Software Design, Acceptance Testing would best be developed during development of: A_REQTS B_DESIGN C_CODING D_DON T KNOW Sam Siewert 17
SQA Knowledge Survey Ideally an acceptance test should be provided by the customer, but this rarely happens, so the software engineering team often must help define the acceptance test: System testing involves configuration of sub-system(s) from multiple software modules and a driver or external stimulus to exercise this sub-system: System test involves integration of sub-systems into a system tested in an end-to-end test environment, similar to how the system will be used: Sam Siewert 18