Building High Assurance Systems with SAFe 4.0

Transcription

1 Building High Assurance Systems with SAFe 4.0 Agile 2016 By Dean Leffingwell 2016 Scaled Agile, Inc. All Rights Reserved Scaled Agile, Inc. All Rights Reserved. V

2 What is a high assurance system? High assurance systems are systems that have an unacceptable social or economic cost of failure $ Most high assurance systems are subject to regularity or industry compliance requirements, including Verification and Validation. 2

3 Compliance meets Agile development Regulatory and compliance 4 Quality, safety, security, efficacy 4 Specifications 4Verification and validation 4 Inspections, audits, sign-off 4Documented quality management systems 4Metrics defects, requirements coverage, code coverage, traceability Agile Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan Welcome changing requirements. 3

4 Verification, validation, and compliance Verification is the confirmation through objective evidence that the specified requirements have been fulfilled. Verification demonstrates that the design and implementation correctly and completely embody the requirements. You built the solution right Validation is the confirmation via objective evidence that the system performs its intended function. The intended functions and how well the system performs those functions are determined by the customer. You built the right solution Compliance is an an organizations adherence to relevant laws, specifications and guidelines. Compliance typically requires documented, objective evidence of solution V&V through tests, inspections, analysis, or demonstrations. And you have evidence that you have done so 4

5 Example: US FDA Regulations for Medical Devices A documented software requirements specification (SRS) provides a baseline for both validation and verification. The software validation process cannot be completed without an established software requirements specification (Ref: 21 CFR 820.3(z) and (aa) and (f) and (g)). From the guidance, such a document typically contains: 4All software system inputs, outputs, and functions 4All performance requirements 5

6 So what s to be done? 4Build the solution incrementally 4Organize around value 4Build quality in 4Apply continuous verification and validation 6

7 SAFe Version 4.0 has the hooks we need 7

8 Build the Solution Incrementally 8

9 Apply fast learning cycles Accelerate knowledge with fast feedback 4 Improves learning efficiency by decreasing the time between action and effect Plan Do 4 Reduces the cost of risk-taking by truncating unsuccessful paths quickly PDCA 4 Facilitated by small batch sizes 4 Requires increased investment in development environment Adjust Check The shorter the cycles, the faster the learning Principles of Product Development Flow, Don Reinertsen Plan. Do. Check. Act. (Adjust), W. Edwards Deming, Walter Shewhart, et al. 9

10 10 Build Incrementally with fast, integrated learning cycles Documents Documents Unverified System System One PDA cycle P D C A P D C A P D C A P D C A P D C A P D C A P D C A P D C A P D C A

11 Iterate within the product lifecycle Exploration Feasibility Product Committed Solution Ready Limited Distribution General Distribution A P A P A P C D C D C D 11

12 The problem of phase-gate internal milestones There was in fact no correlation between exiting phase gates on time and project success the data suggested the inverse might be true. Lean Machine 4 Force too early design decisions; encourages false positive feasibility 4 Assume a point solution exists and can be built right the first time 4 Create huge batches, long queues; centralizes requirements and design in program management Requirements complete Design complete Planned deployment Point solution Not enough time to adjust Actual solution 12

13 Apply objective Milestones PI Demos routinely deliver objective progress, product, and process metrics Progress Product Process Objectives Performance Customer Feedback Improvement stories PI Demo PI PI PI 13

14 Organize around value 14

15 Why organize around value? Align the organization around projects and product lines. Allen C. Ward 1. Fewer handoffs, faster value delivery 2. Easier to build in quality 3. Built-in alignment between the business and software development 4. Optimizing the system as a whole Result: Faster delivery, higher quality, higher customer satisfaction 15

16 Understand the full Value Stream A fundamental thinking construct in Lean Trigger Lead time $ REPEAT 4Each Value Stream is the sequence of steps used to deliver value to the Customer 4It includes the whole sequence, from concept or customer order to delivery of value and/or receipt of cash 4It contains the people who do the work, as well as the flow of information and materials 16

17 Value streams are cross-functional 17

18 Implement value with cross-functional Agile Release Trains Business Product Mgt/ Sys. Engineering Hardware Software Quality Testing V&V AGILE RELEASE TRAIN Agile Teams 18

19 Synchronize with PI Planning Future product development tasks can t be pre-determined. Distribute planning and control to those who can understand and react to the end results. Michael Kennedy, Product Development for the Lean Enterprise 4 All stakeholders face-to-face (but typically multiple locations) 4 Management sets the mission, with minimum possible constraints 4 Requirements and design emerge 4 Important stakeholder decisions are accelerated 4 Teams create and take responsibility for plans For a short video PI planning example, see: 19

20 Build quality in 20

21 SAFe quality practices Assure every increment of the solution reflects quality standards Software 4Continuous integration 4Test-First 4Refactoring 4Pair-work 4Collective ownership Hardware 4 Exploratory early iterations 4 Model Based Systems Engineering (MBSE) 4 Set-Based Design 4 Frequent, system-level integration 4 Design verification 21

22 Architectural Runway Architectural Runway existing code, hardware components, etc. that technically enable near-term business features Implemented now Enabler Feature Feature Feature to support future features 4 Enablers build up the runway 4 Features consume it 4 Architectural Runway must be continuously maintained 4 Use Capacity Allocation (a percentage of train s overall capacity in a PI) for Enablers that extend the runway Architectural Runway 22

23 Test first and test automation Test automation enables incremental development via regression testing 4Automated tests are implemented in the same iteration as the functionality 4The team that builds functionality also automates the tests ü Test 1 Test 2 ü Test 3 Test 4 û Test 5 ü Test 1 ü Test 2 ü Test 3 ü Test 4 ü Test 5 4Actively maintain test data under version control Progress Done Building functionality Test automation Iteration 23

24 MBSE facilitates emergent specifications 4Generate specifications from models in Solution Intent 4Everyone contributes; everyone takes a systems view 4Ensures single source of truth, both as-is and to-be Subsystem Spec Fuel Feature Spec Performance Component Spec Ignition ECM 24

25 Continuous software integration 4Integrate every vertical slice of a user story 4Avoid physical branching for software 4Use development by intention in case of inter-team dependencies - Define interfaces and integrate first; then add functionality Agile Team 1 Check out most functionality Mainline Check in first slice Check newest changes back in Agile Team 2 Always current mainline increases program velocity 25

26 Frequent system-level integration and testing 4Frequent integration and testing is the only objective evidence of solution integrity 4Trade-offs are inevitable in terms of: Frequency of integration Depth of integration Fidelity of feedback 26

27 Solution Demo drives frequent solution integration 4Frequent solution integration and testing provides the best objective evidence 4A joint responsibility of ARTs and VS s system teams 4Provides early validation and regular risk reduction 4Increases actual velocity AGILE RELEASE AGILE RELEASE AGILE RELEASE System Teams TRAIN TRAIN TRAIN Program Increment Full or partial integration during the PI Full solution integration Program Increment Value Steam Program 27

28 Inspect and Adapt Every PI, teams systematically address the larger impediments that are limiting quality, compliance and velocity 28

29 Apply continuous verification and validation 29

30 Solution Intent supports verification Requirements Model Functional and use cases NFRs safety, security Customer, regulatory Domain Model Business process Business rules Traceability Solution Model Analysis Architecture/interfaces Test Model Test plans, test cases, execution results Compliance docs Realization Code, components Documentation Deployment 30

31 Agile testing quadrants support V&V Automated and Manual SUPPORTING DEVELOPMENT Automated Functional tests Feature/Capability acceptance tests Enabler acceptance tests Story acceptance tests Unit tests Component tests BUSINESS-FACING Q2 Q1 Q3 Q4 TECHNOLOGY-FACING Acceptance tests Scenario tests Exploratory tests User acceptance tests Alpha and beta tests System qualities tests Performance and load Security Other NFRs Enabler tests Adapted from Brian Marick, Crispin and Gregory Manual and Automated CRITIQUING THE SOLUTION Tools 31

32 SAFe requirements model supports continuous verification Functional behaviors Nonfunctional behaviors Feature realized by (one to many) Story Nonfunctional Requirement tested by tested by tested by Feature acceptance test Story acceptance test Unit test System qualities test 32

33 Make V&V activities part of regular flow 4Include compliance concerns in Definition of Done (DoD) 4Development continuously verifies 4System demos include validation status towards compliance Iteration verification Iteration validation PI validation Release validation System demo Solution demo Feature Story Story NFR Feature 33

34 Iteration verification Continuous verification 4Peer review of stories Agile Team Feature 4Implement stories and tests associated with committed features story story 4Test Solution with story acceptance tests 4Accept story into baseline 4Update Stories, tests and relationships in Solution Intent Primarily development team responsibilities 34

35 Iteration validation Feature NFR System Demo Product Owner Evaluate full system increment Regression test all functional stories and feature acceptance tests User/Product Owner validation Run NFR tests Update compliance documentation Development team, system team and program shared V&V responsibilities 35

36 PI validation Consider compliance in planning I P Solution Demo Program Increment Product Mgt Customer /user Evaluate full PI system increment 4Regression test all functional stories and feature acceptance tests 4User, PO, and PM validation 4NFR tests 4Update compliance documentation Program-based V&V responsibilities (May require handoff to Independent V&V) 36

37 Release validation Fitness for purpose 4 Final validation 4 Qualification including installation and operation 4 User acceptance testing Supporting documentation 4 Installation, user, and other 4 Compliance evidence of V&V Releasable Solution Solution Increment System Increment Team Increment 37

38 Questions? 38 38