Risk Mitigated SCAMPI SM Process

Transcription

1 Risk Mitigated SCAMPI SM Process Gary Natwick, Debra Perry Harris Corporation November 19, 2008 NDIA CMMI Conference - 1

2 Providing Value To Our Customers Communications and information networks Aviation electronics Intelligence, surveillance, and reconnaissance Space and ground satellite communications systems Operations and support services People Innovation Process Risk Mitigated SCAMPISM Process NDIA CMMI Conference - 2

3 Agenda Process background Risk mitigation approach Results Conclusions NDIA CMMI Conference - 3

4 Process Background Appraised CMMI -SE/SW v1.1 Level 3 in November 2005 Updated organizational integrated processes for CMMI - DEV+IPPD v1.2 compliance Identified expected artifacts for process compliance Validated approach using SCAMPI SM Class C appraisal Achieve CMMI -DEV+IPPD Level 3 NLT November 2008 NDIA CMMI Conference - 4

5 HARRIS CORPORATION Government Communications Systems Division P.O. Box 37 Melbourne, FL USA Integrated Compliance Approach CMMI Model Division Programs Pittsburgh, PA XYZ Program Management Plan 99 MONTH 9999 < TEMPLATE FOR PMP THAT IS DELIVERED TO MEET A PMP CDRL REQT > CMMI for Development, Version 1.2 CDRL XXXX Document Control Number: Contract Number: XXXXXXXXXXXXXXXXX CMMI-DEV, V1.2 CMU/SEI-2006-TR-008 ESC-TR Prepared for: CUSTOMER ADDRESS CITY-STATE-ZIP Improving processes for better products CMMI Product Team Prepared by: August 2006 Unlimited distribution subject to the copyright. Tailoring Program s Compliance Artifacts Improve Reuse Submit Command Media Organizational Learning Historical Data Best Practices Example Assets Improvement Requests Program s Compliance Metric NDIA CMMI Conference - 5

6 Integrated Process Manual IPM Program Management Processes Program Life-Cycle Processes Program Support Processes Organizational Processes Program Planning Estimation Program Monitoring and Control Supplier Acquisition & Management Change Management Requirements Analysis System Architecting/Design Design Code and Unit Test Fabrication and Assembly Product Integration Verification Validation Production Field Support Requirements Management Risk Management Configuration and Data Management Program Metrics Decision Analysis and Resolution Peer s Design Quality Assurance Integrated Logistics Support Process Improvement Training Division Metrics NDIA CMMI Conference - 6

7 Where are artifacts required? Overview A brief description of the process intent Entry Criteria State, Prerequisites, Criteria Inputs Needed work products, resources Exit Criteria State, Criteria Outputs Resulting work products Required Activities Mandatory tasks to implement the process Measures Process performance against plans Organizational Improvement Information Metrics, reusable work products Verification Process compliance oversight Tailoring Approved tailoring, process specific Implementation Guidance Common implementation descriptions Program artifacts needed to demonstrate IPM process compliance Supporting Documentation and Assets Applicable organizational references NDIA CMMI Conference - 7

8 Compliance to Industry Standards CMMI Pittsburgh, PA CMMI for Development, Version 1.2 CMMI Process Areas CMMI Specific/ Generic Practices CMMI Typical Work Products SCAMPI SM CMMI-DEV, V1.2 CMU/SEI-2006-TR-008 ESC-TR Improving processes for better products CMMI Product Team August 2006 Unlimited distribution subject to the copyright. SCAMPI SM Artifacts (Direct/Indirect) IPM IPM Processes IPM Process Statements IPM Expected Artifacts (Direct/Indirect) Process Compliance Monitor (PCM) Functional command media Best practices NDIA CMMI Conference - 8

9 Validation SCAMPI SM Class C CMMI -DEV+IPPD v1.2 model Independent assessment Planning Preparation ed expected artifact for process compliance SCAMPI SM * Process Definition Characterizations Implementation Risk * Using Appraisal Wizard NDIA CMMI Conference - 9

10 Process Definition Characterizations Fully Defined (FD) Largely Defined (LD) Partially Defined (PD) Not Defined (ND) One or more direct artifacts are present and judged to be adequate; and At least one indirect artifact exists; and No weaknesses are noted. One or more direct artifacts are present and judged to be adequate; and At least one indirect artifact exists and; One or more weaknesses are noted. Direct artifacts are absent or are judged to be inadequate; and One or more indirect artifacts suggest that some aspects of the practice are defined; and One or more weaknesses are noted; -OR - One or more direct artifacts are present and judged to be adequate; and No other evidence (indirect artifacts) supports the direct artifact(s); and One or more weaknesses are noted. Direct artifacts are absent or judged to be inadequate; and No other evidence (indirect artifacts) supports the practice implementation; and One or more weaknesses are noted. NDIA CMMI Conference - 10

11 SCAMPI SM Definition 95% of CMMI Practices have Fully Defined processes and artifacts NDIA CMMI Conference - 11

12 Practice Implementation Rating Red Yellow Green The intent of the model practice is judged to be absent or poorly addressed in the set of artifacts identified gaps or issues that will prevent goal achievement, if the deployment occurred in this way across the organizational unit, were identified. The intent of the model practice is judged to be partially addressed in the set of artifacts some gaps or issues were identified, which might threaten goal achievement if the deployment occurred in this way across the organizational unit. The intent of the model practice is judged to be adequately addressed in the set of artifacts identified in a manner that would support goal achievement, if the practice were deployed across the organizational unit. NDIA CMMI Conference - 12

13 SCAMPI SM Risk NDIA CMMI Conference - 13

14 Risk Mitigation Approach Develop SCAMPI SM success roadmap Assess risk level after each SCAMPI SM Evidence completeness Organizational confidence Adjust SCAMPI SM plan NDIA CMMI Conference - 14

15 Original SCAMPI SM Approach FY2007 (July 2006 June 2007) Resolve organizational process corrective actions from SCAMPI SM Obtain organizational sponsorship Select independent SCAMPI SM Lead Appraiser Select Focus/Non-Focus programs FY2008 (July 2007 June 2008) Develop SCAMPI SM Plan Class B for risk reduction Class C for gap awareness Class A for maturity compliance Complete SCAMPI SM Class B and C appraisals Perform risk assessment and update SCAMPI SM plan Resolve appraisal corrective actions FY2009 (July 2008 June 2009) Resolve appraisal corrective actions Complete SCAMPI SM -A before November 2008 NDIA CMMI Conference - 15

16 Roadmap to Next Appraisal CMMI-SE/SW L3 EXPIRES SCAMPI PROJS ID ED FY07 Q1 FY07 Q2 FY07 Q3 FY07 Q4 FY08 Q1 FY08 Q2 FY08 Q3 FY08 Q4 FY09 Q1 FY09 Q2 CMMI DEV Release MEDIA UPDATES IPM/PCM Release SCAMPI-B FEB SCAMPI-C GAPS-JUN CMMI L3 SCAMPI-A NOV INSTITUTIONALIZATION & TRAINING SCAMPI-B SCAMPI-B Jan Jan Readiness Readiness Feb Feb On-Site On-Site Overview, Overview, Evidence Evidence s s & & Feb Feb Final Final SCAMPI-C SCAMPI-C (B+4 (B+4 mo mo to to on-site) on-site) May May Readiness Readiness Jun Jun On-Site On-Site Gap Gap s s Jun Jun Final Final SCAMPI-A SCAMPI-A (C+4 (C+4 mo mo to to on-site) on-site) Sep Sep Readiness Readiness Oct Oct 29-Nov 29-Nov 7 7 On-Site On-Site Overview, Overview, Evidence Evidence s s & & Nov Nov 8 8 Final Final NDIA CMMI Conference - 16

17 SCAMPI SM Readiness Scorecard Preliminary review of artifacts before the SCAMPI SM Readiness NDIA CMMI Conference - 17

18 SCAMPI SM Plan January Update Assessed evidence preparation risk Not enough relevant artifacts available (70%) SCAMPI SM interviews a low risk from prior experience FY2008 (July 2007 June 2008) Updated SCAMPI SM Plan Class C for gap awareness Class B for risk reduction Class A for maturity compliance Complete SCAMPI SM Class C and B appraisals Perform risk assessment and update SCAMPI SM plan Resolve appraisal corrective actions FY2009 (July 2008 June 2009) Resolve appraisal corrective actions Complete SCAMPI SM -A before November 2008 NDIA CMMI Conference - 18

19 Roadmap to Next Appraisal CMMI-SE/SW L3 EXPIRES SCAMPI PROJS ID ED FY07 Q1 FY07 Q2 FY07 Q3 FY07 Q4 FY08 Q1 FY08 Q2 FY08 Q3 FY08 Q4 FY09 Q1 FY09 Q2 CMMI DEV Release MEDIA UPDATES IPM/PCM Release SCAMPI-B SCAMPI-C FEB SCAMPI-C SCAMPI-B GAPS-JUN MAY CMMI L3 SCAMPI-A NOV INSTITUTIONALIZATION & TRAINING REVISED SCAMPI-C SCAMPI-C SCAMPI-B SCAMPI-B Jan Jan Readiness Readiness Feb Feb On-Site On-Site Evidence Evidence s s & & Feb Feb Final Final SCAMPI-B SCAMPI-C SCAMPI-C SCAMPI-B (B+4 (C+4 (B+4 (C+4 mo mo to to on-site) on-site) May Apr May Apr Readiness Readiness Jun May Jun May On-Site On-Site Gap Evidence Evidence Gap s s Jun Jun s Final s Final & & May May Final Final SCAMPI-A SCAMPI-A (C+4 (B+4 (C+4 (B+4 mo mo to to on-site) on-site) Sep Sep Readiness Readiness Oct Oct 29-Nov 29-Nov 7 7 On-Site On-Site Evidence Evidence s s & & Nov Nov 8 8 Final Final NDIA CMMI Conference - 19

20 Evidence State Ratings* Established as a risk mitigation to focus on the number of artifact issues identified prior to the SCAMPI SM Readiness Complete Ample Direct and Indirect artifacts present Sufficient - Direct and Indirect present but need verbal confirmation Partial - Direct or Indirect missing or outstanding issue/need None - Direct and Indirect missing * Using Appraisal Wizard NDIA CMMI Conference - 20

21 SCAMPI SM Evidence February 2008 NDIA CMMI Conference - 21

22 SCAMPI SM Plan February Update Assessed evidence preparation risk Add appraisal team member SCAMPI SM interviews a low risk from prior experience FY2008 (July 2007 June 2008) SCAMPI SM Plan Class C completed Class B for risk reduction Class A for maturity compliance Complete SCAMPI SM Class B appraisal Perform risk assessment and update SCAMPI SM plan Resolve appraisal corrective actions FY2009 (July 2008 June 2009) Resolve appraisal corrective actions Complete SCAMPI SM -A before November 2008 NDIA CMMI Conference - 22

23 Roadmap to Next Appraisal CMMI-SE/SW L3 EXPIRES SCAMPI PROJS ID ED FY07 Q1 FY07 Q2 FY07 Q3 FY07 Q4 FY08 Q1 FY08 Q2 FY08 Q3 FY08 Q4 FY09 Q1 FY09 Q2 CMMI DEV Release MEDIA UPDATES IPM/PCM Release SCAMPI-B SCAMPI-C FEB SCAMPI-C SCAMPI-B GAPS-JUN MAY CMMI L3 SCAMPI-A NOV INSTITUTIONALIZATION & TRAINING REVISED SCAMPI-C SCAMPI-C SCAMPI-B SCAMPI-B Jan Jan Readiness Readiness Feb Feb On-Site On-Site Evidence Evidence s s & & Feb Feb Final Final SCAMPI-B SCAMPI-C SCAMPI-C SCAMPI-B (B+4 (C+4 (B+4 (C+4 mo mo to to on-site) on-site) May Apr May Apr Readiness Readiness Jun May Jun May On-Site On-Site Gap Evidence Evidence Gap s s Jun Jun s Final s Final & & May May Final Final SCAMPI-A SCAMPI-A (C+4 (B+4 (C+4 (B+4 mo mo to to on-site) on-site) Sep Sep Readiness Readiness Oct Oct 29-Nov 29-Nov 7 7 On-Site On-Site Evidence Evidence s s & & Nov Nov 8 8 Final Final NDIA CMMI Conference - 23

24 SCAMPI SM Evidence February 2008 April 2008 NDIA CMMI Conference - 24

25 SCAMPI SM Plan April Update Assessed evidence preparation risk Valid artifacts improved significantly (98%) SCAMPI SM interviews a low risk from prior experience FY2008 (July 2007 June 2008) Develop SCAMPI SM Plan Class C completed Class B On-Site no longer needed Class A for maturity compliance Resolve appraisal corrective actions FY2009 (July 2008 June 2009) Resolve appraisal corrective actions Complete SCAMPI SM -A in July 2008 NDIA CMMI Conference - 25

26 Roadmap to Next Appraisal SCAMPI PROJS ID ED CMMI L3 SCAMPI-A JUL-AUG CMMI-SE/SW L3 EXPIRES FY07 Q1 FY07 Q2 FY07 Q3 FY07 Q4 FY08 Q1 FY08 Q2 FY08 Q3 FY08 Q4 FY09 Q1 FY09 Q2 CMMI DEV Release MEDIA UPDATES IPM/PCM Release SCAMPI-C FEB SCAMPI-B MAY INSTITUTIONALIZATION & TRAINING ACCELERATED SCAMPI-C SCAMPI-C Jan Jan Readiness Readiness Feb Feb On-Site On-Site Evidence Evidence s s Feb Feb Final Final SCAMPI-B SCAMPI-B April April Readiness Readiness May May On-Site On-Site Evidence Evidence s s & & May May Final Final SCAMPI-A SCAMPI-A (B+4 (B+4 mo mo to to on-site) on-site) Sep Jul Sep Jul Readiness Readiness Oct Jul Oct Jul Nov Nov 7 7 On-Site On-Site Evidence Evidence s s & & Nov Jul Nov Jul Final Final NDIA CMMI Conference - 26

27 SCAMPI SM Evidence February April July NDIA CMMI Conference - 27

28 SCAMPI SM Results Objective Evidence (OE) 21,123 artifacts available for review 11,937 artifacts reviewed (56.5% of what was available) 1 Additional OE request Face to face interactions 34 Meetings Site briefings (1) Interview sessions (20) Preliminary findings briefings (2) Project briefings (12) 1 Follow-up interview Harris Proprietary Information NDIA CMMI Conference -28

29 Practice Characterizations* Fully Implemented (FI) Largely Implemented (LI) Partially Implemented (PI) Not Implemented (NI) Not Yet (NY) Direct artifacts present and adequate Supported by indirect artifact and/or affirmation No weaknesses noted Direct artifacts present and adequate Supported by indirect artifact and/or affirmation One or more weaknesses noted Direct artifacts absent or judged inadequate Artifacts or affirmations indicate some aspects of the practice are implemented One or more weaknesses noted OR Direct artifacts present and adequate No other evidence supports direct artifacts One or more weaknesses noted Direct artifacts absent or judged inadequate No other evidence supports the practice One or more weaknesses noted Project has not yet reached the stage in life cycle * Using Appraisal Wizard Harris Proprietary Information NDIA CMMI Conference -29

30 SCAMPI SM On-Site Results Harris Proprietary Information NDIA CMMI Conference -30

31 Goal and Process Area Ratings HARRIS (GCSD) SCAMPI SM A 31 July 2008 Satisfied Not Satisfied NDIA CMMI Conference - 31

32 Conclusions Risk mitigation approach can benefit cost and schedule Eliminated a SCAMPI SM On-site 3 ½ months ahead of schedule Risk assessment techniques Process definition characterizations Evidence state ratings Practice characterizations Lessons learned Plan early and develop a roadmap Assess risk level after each SCAMPI SM Evidence completeness Organizational confidence Be flexible Schedule Management Lead Appraiser and appraisal team Intangible benefit Saved stress on the programs NDIA CMMI Conference - 32

33 Contact Information Harris Corporation P.O. Box 37 Melbourne, Florida SEI Partner Gary Natwick SEI-Authorized Introduction to CMMI Instructor SEI-Authorized SCAMPI SM Class A Lead Appraiser (former) SEI-Authorized SCAMPI SM Class B&C Team Leader (former) Harris SEI Partner Business & Technical Point of Contact Debra Perry dperry@harris.com CMMI is registered with the U.S. Patent and Trademark Office by Carnegie Mellon University. SM SCAMPI is a service mark of Carnegie Mellon University. NDIA CMMI Conference - 33