Business Value and Customer Benefits Derived from High Maturity

Transcription

1 CMMI sm Technology Conference and User Group November 2002 Business Value and Customer Benefits Derived from High Maturity Alan Pflugrad Northrop Grumman Information Technology Defense Enterprise Solutions (DES) Chair, DES Engineering Process Group Executive Manager, Systems and Process Engineering Phone: (703)

2 Discussion Purpose and Agenda Purpose: Communicate business value and customer benefits derived from an application of high maturity system/software engineering processes, and How an integrated process framework helps Discussion Agenda Business Value/Customer Benefits & Process Highlights Quality and Process Goals Quality and Process Performance Process Highlights Integrated Process Improvement (CMMI) Limit 40 minutes including questions 2

3 Organizational and Project Quantitative Management Process Overview DES Business Objectives 1 DES management selects quality and process goals & measurements DES management checks org and project data against DES goals (process capability baseline). 6 2 Projects select related goals & measurements for each life cycle phase. 5 Projects check performance against project goals and business objectives. 3 Projects track process performance over time. 4 Projects improve performance by removing root causes for out-of-bound conditions. 3

4 DES Process and Quality Measures Acronym Measurement Process CPIm Cost Performance Index monthly Earned Value System SPIm Schedule Performance Index monthly Earned Value System EPVPm ETC Performance Variance Percentage monthly Earned Value System or other financial process DDr Defect Density from Peer Review Peer Review (all Life Cycle Stages) DDt Defect Density from Test & Operations Test 4

5 Process/Quality Improvements support Organizational Business Objectives DES Business Objectives Annual Operating Plan Achieve revenue and margin objectives Improve customer satisfaction rating DES Process & Quality Performance Goals Collective across participating projects 1. Achieve Cost Perf. Index = 1 ± 5% 2. Achieve Schedule Perf. Index = 1 ± 5% 3. Achieve Est-To-Complete-Var = 0 ± 5%. 4. Achieve 5% improvement in Defect Density for each life cycle phase. 1. Achieve Cost Perf. Index = 1 ± 5%. 2. Achieve Schedule Perf. Index = 1 ± 5%. 3. Achieve Est-To-Complete-Var = 0 ± 5%. 4. Achieve 5% improvement in Defect Density for each life cycle phase. 5

6 Optimizing Process Strategy Overview Test PR or Peer Review PR PR PR PR PR Defect Data Defect Analysis Before Pareto Analysis Common Causes Defect Categories Project Tailored Process Organization Defined Process Technology Innovations Root Cause Analysis Process Change Remove Root Causes From Process to Prevent Defects 6

7 SATS/SIGS Program and QM Indicators S Technical Goal: 20 +/- 5 defects/kloc40 Actual: 22.9 defects/kloc Action: Implementing DDt Defect Density at Review (all defects) Technical Highlights: Only 2% of all defects are found in the fielded system Cost Performance Index (Monthly) S Financial Goal: 1.0 +/ Actual: 0.98 Action: DP cycle for SCoV in April; Countermeasures improve estimation; change EV tracking Technical Highlights: CPI is still on target Mar-00 May-00 Jul-00 Sep-00 Dec-00 Feb-01 Apr-01 Jun-01 Aug-01 Oct-01 Nov-01 Jan-02 Mar-02 May-02 7

8 SATS/SIGS Program and QM Indicators M Schedule Goal: 1.0 +/- 0.1 Actual: Mar-00 May-00 Jul-00 Sep-00 Schedule Performance Index (Monthly) Action: Watching closely, DP cycle for SCoV in April; Countermeasures improve estimation; change EV tracking Technical Highlights: will be Satisfactory by 7/02 Dec-00 Feb-01 Apr-01 Jun-01 Aug-01 Oct-01 Nov-01 Jan-02 Mar-02 May-02 E Customer Satisfaction Goal: >= 95% Actual: 98.8% 110% 105% 100% 95% Award Fee Scores 90% Action: Continue to deliver Technical Highlights: Customer is very flexible due to track record M-00 M-00 J-00 S-00 N-00 J-01 M-01 M-01 J-01 S-01 N-01 J-02 8

9 Controlling Quality Performance Build AWIPS CM Build Defect Density AWIPS Release Errors per File B3-50 PS2 50 PS3-PS6 50 PS7-PS10 50 PS11- PS14 50 PS15-S1 50 S2-S5 50 S6-S9 50 S10-A4 50 A5-A8 Release 50 A9-B3 50 B4-511 PS1 511 PS2c- PS6 511 PS7- PS S1-S4 511 S5-S8 511 A1-A4 511 A5-A8 511 B2-512 PS3 Average UCL Mean LCL Statistical process control identifies build issues that can impact the development schedule. 9

10 Predicting Quality - Example 400 AWIPS Rel 4.2 DRs DRs 100 DRs Rayleigh Model 0 6/ / / Date 10

11 Controlling Process Performance AWIPS Release 5.1 Monthly CPIm X Chart X Dec-2000 Jan-2001 Feb-2001 Mar-2001 Apr-2001 May-2001 Jun-2001 Jul-2001 Aug-2001 Sep-2001 Oct-2001 Nov-2001 Dec-2001 CPIm CPIm Mean UNPLX LNPLX U2S L2S U1.5S L1.5S AWIPS Release 5.2 Monthly SPIm X Chart Cost and schedule can be managed with statistical process control Improves predictions of future performance Results: Build 4, 2% underrun R5.0, 4% underrun R5.1, 5% underrun Build 5 variance in last 12 months, 10% X Aug-2001 Sep-2001 Oct-2001 Nov-2001 Dec-2001 SPIm SPIm Mean UNPLX LNPLX U2S L2S U1.5S L1.5S Statistical process control improves cost & schedule performance. 11

12 JEDMICS Defect Density & Customer Satisfaction Survey Averaged Customer Satisfaction Defect Density (Test) Defect Density (Operations) 12

13 Quality Improvement Realized Defects/[Page KSLOC] Mean Defect Density by Phase Good Analysis PDesign CDesign Code Test Ops LifeCycle Phase PRC 9909 PRC 9912 A B D.05 defects/ KSLOC 13

14 Process Implementation Support Best Practice Links to other KPAs/PAs Links to SIM Links to other processes Templates Samples Policy Statements Quality Assurance Audits Process Integration Output Work Products Policy & Verifications Rollout Proposals Plans Pilot Projects Startups Ongoing Projects Process Product Checklists Metrics Corporate Customer Internal Support Processes Tools Process Champion Internal Consultant Subject Matter Experts Tailoring Guidance Roles & Responsibilities Requirements Training COTS & Glue Support Compatible Formats Corporate OJT One Per Domain Shows Variations in each Process/ Asset Expert Knowledge Per Process, Asset SWCMM SECMM ISO Customer Stds 14

15 Core Processes Common to Multiple Disciplines Information Technology Products & Services in Constant Change IT Consulting Sys Arch, Engin & Delivery Enterprise Integration Data Center Operation IT Infrastructure Management Applications Management SW Discipline Core Processes SE Discipline CMMI SW CMM SE CM IPPD-CMM Core Core SE Core SW IPD SETA Functional Process Outsourcing IPD Discipline Other CMM s 15

16 Context: Acquisition/Development Space Acquirer Mismatch Mature buyer must mentor low maturity developer Outcome not predictable Matched Team Match of maturity Team risk approach Execution to Plan Measurable performance Predictable results increasing Process Maturity Disaster No discipline No process Adhoc Crisis Management Outcome not predictable increasing Mismatch Customer is always right Customer encourages shorts cuts Developer 16

17 Why the CMMI fit s Mission Shortfalls Mission Area Planning Budgeting Priority Decision Analysis and Resolution Requirements Definition Requirements Development Directives, Constraints Contracting Supplier Agreement Management Integrated Project Management Risk Management Project Monitoring and Control Program Management Product Control Requirements Management Configuration Management Quality Assurance Causal Analysis and Resolution Activity Planning Project Planning Technical Solution Product Integration Technical Execution Concurrent Front-End Activities Deficiencies Outcome & Feedback Product Verification Assessment & Certification Measurement and Analysis Validation Organizational Process Management Products System Product Deliveries Process Focus Process Definition Innovation and Deployment Training Process Performance Quantitative Mgmt Courtesy: Mitre/Mike Bloom Process Maturation 17