Using PSM at All Levels in an Organization 6 June 2002 Garry Roedler Lockheed Martin Management and Data Systems Systems Integration GJR - 1 6 June 02
Agenda Enterprise level - Guidance development - Application Business unit level - Guidance development - Application Project level - Guidance development - Application GJR - 2 6 June 02
Multi-Level Application Enterprise Management Organizational Management Project Management Performance Technical and Business Policy Enterprise Decision Analysis Strategic Oversight Process Improvement Insight Project Planning Guidelines Performance Based Guidelines Organizational Baselines / Thresholds Organizational Decision Analysis Project Estimation & Planning Project Performance Tracking Project Decision Analysis Resource Management Project Baselines/Thresholds Risk & Quality Management Processes Issue-Driven Process GJR - 3 6 June 02
at the Enterprise Level GJR - 4 6 June 02
Enterprise Level Process Asset Hierarchy Top Level Lockheed Martin Integrated Engineering Process Standard integrated throughout -Activities integral to the whole set of processes rather than a separate process Standards Working Group Detail Lockheed Martin Integrated Guidebook Has a broader scope than PSM Includes candidate measures that have been successfully applied in Lockheed Martin GJR - 5 6 June 02
Integrated Guidebook Purpose The purpose of the Integrated Guidebook is to define a common, integrated measurement methodology that: Provides pragmatic and proven guidance and candidate measures to effectively implement and use measurement Fundamental practices and provide references to more detailed information Is consistent with standards and leverages leading guidance Provides candidate measures that have been applied successfully within LMC for insight into the LM-IEP processes and products Supports all capability levels and enables quantitative mgt Provides traceability and relationship to LM-IEP and CMMI SM Can be applied to any life cycle, phase, or project type Can be applied to subcontracts to improve consistency in addressing information needs SM - CMMI is a service mark of Carnegie Mellon University GJR - 6 6 June 02
LM Integrated Activities Process Inputs Assess and Control Performance Identify and Prioritize Information Needs Tailoring Improvement Actions Risks and Issues Prioritize Goals, Risks and Issues Common Management Processes Information Needs Select and Specify Measures Candidate Measures Collect and Process Data Specified Measures Applying Evaluate Process Project/ Organization Processes Integrate Into Standard Processes Processed Data Goals and Objectives Analyze Measures Planning Quantitative Management/ Plan Qualitative Information (Risks, Issues, Information Status, etc.) Make Decisions Can be applied at project or organization al level Analysis Results & Performance Measures User Feedback Manage Strategy Action GJR - 7 6 June 02 Process Outputs
Candidate Measures Common Issue A reas/ Inform ation N eeds A ddressed Schedule and P rogress R esources and C ost Product Size and S ta b ility Product Quality Action Item s P r o file R equirem ents V erification Risk Profile Schedule P erform ance Cost P erform ance Effort P rofile Risk Profile Staffing N eeds P rofile Staff T urnover P rofile B aseline G row th and S ta b ility End P roduct S ize P rofile R equirem ents T raceability TBD/TBR Profile Cost of Q uality In d e x Defect Profile R equirem ents T raceability R equirem ents V erification R ew ork Effort S ave-escape C ost P rofile T echnical P erform ance M easures (TPM s) GJR - 8 6 June 02 M easure N am e Each measure is chosen based on the Common Issue Area focus and specific information needs.
Candidate Measures (cont d) Common Is s u e A reas/ In fo r m a tio n N eeds A ddressed P rocess P erform ance Technology Effectiveness C ustom er S a tis fa c tio n M easure N am e A w ard Fee P ercent C ycle T im e P r o file P eer R eview P erform ance P rocess C haracteristics P r o file P rocess C o m p lia n c e P r o file P rocess Evaluation Findings P rodu ctivity Rework Effort S ave-escape C ost P r o file T raining C ourse P r o file T raining P lan P r o file P rodu ctivity T echnical P erform ance M easures (T P M s ) A w ard Fee C omments A w ard Fee P ercent Plus all PSM candidate measures are referenced GJR - 9 6 June 02
Other Guidance Quantitative Management - Provides both project and organizational level guidance - Statistical approaches (SPC and other) for thresholds/control limits - Use in decision making Technical Performance - Provides detailed selection and analysis information for these measures Relationship of to Risk Management and Quality Management GJR - 10 6 June 02
Other Guidance Enterprise - Provides guidance on addressing the enterprise perspectives Obtain information to better enable strategic decision making for business, systems and software development, and process Improvement Balanced Scorecard instruction Each candidate measure has enterprise perspective and business focus identified, as applicable Enterprise Perspective Project O versight Perspective B u sin e ss P e rsp e ctive O rg a n iz a tio n P e rsp e c tiv e Business Focus Project Perform ance Financial Perform ance C u sto m e r S a tisfa c tio n O rganizational Process Perform ance O rganizational Health, Innovation, and Learning GJR - 11 6 June 02
at the Business Unit Level GJR - 12 6 June 02
Aligning to Meet Both Model & Business Requirements Model Requirements Business Requirements Process Guidance (Corporate, PSM, INCOSE, etc.) Business Realities Tailored SE Model Implementation Trade-offs, interpretations, etc. Organization Process Assess GJR - 13 6 June 02
Implementation and Tailoring Infrastructure and Commitment - Commitment from the top down - Collection capability spanning organization and projects - team with members from the org & projects - Stakeholder involvement at all levels helps buy-in, consistency, and improvement - Automation is a key factor for maximizing the utility, but can start without it (Level 5 is achievable with just an office suite) Tailoring - Key to achieving a manageable set of measures - Based on issues/objectives/business needs at appropriate level Company/Line of Business/Functional Org. Measures Project Specific Measures - Quantity of measures and data availability are major considerations - Specify and document in Plans GJR - 14 6 June 02
Business Unit Standard Measures Issue Category Measure Customer Satisfaction Customer Feedback Award Fee Percent *# Process Performance Award Fee Comments *# Process Compliance Program/Process Tailoring # Self-Audit Findings *# Process Effectiveness Rework Effort Percent *# Process Efficiency Cycle Time Variance *# System Engineering # Productivity *# Product Quality Functional Correctness Approval Rates # Product Size and Stability Functional Size and Stability TBD/TBR (Percent Overdue) # Resources and Cost Financial Performance Cost Variance *# Schedule and Progress Personnel Work Unit Progress Effort *# Staffing *# Requirements Verification (Percent Overdue) # Self-Audit Progress # Milestone Performance * * Tailored variation of measure in PSM # Tailored variation of measure in LM Integrated Gdbk GJR - 15 6 June 02
Plans Applying Tools PMP SIPS Processes Execute PMP Processes Data Collection (in-process) Data Processing SI Meas. POCs Evaluate Utility & Incorporate Feedback Decisions and Actions (documented & tracked) Stake Holders Senior Management Root Cause Analysis SIPRB Program Management Reports 100 50 Establish, use and refine product/process thresholds & goals 0
at the Project Level GJR - 17 6 June 02
Project Tailoring Identify information needs/issues Start from Organization Standard measures as candidates - Deviation from standard measures requires rationale based on information needs and SOW - Documented in measurement compliance matrix (part of measurement plan) - Project-specific measures identified and defined Document measure definitions and project implementation in measurement plan Approval by project manager and concurrence by process review board GJR - 18 6 June 02
Code SI KWP Initial Code SI KWP - Rework 9 Other A Top-Level System Y Top-Level System Requirements Requirements C Study Report D Study Report Code SI Process 2 Program Planning Practical Software and Systems Data Collection Task Plan Task Implementation Entry of effort Cycle Time Planned Actual KWP ID Start End Start Date TBD/TBRs Approval Rates KWP ID Open # Approved Data Collection Templates Process Audits Effort DB Program POC Self-Audit DB data collection and reporting on monthly basis Meas. Data SI Org Lead GJR - 19 6 June 02
Analysis Considerations Most measures are analyzed monthly results are based on current month, except as specified otherwise: - Rework Effort Percent is based on rolling quarter - System Engineering Productivity and Award Fee Percent are based on rolling annual - Award Fee Percent and Award Fee Comments are based on individual program Award Fee schedules on a semiannual basis results are summarized as red, yellow or green based on thresholds and goals GJR - 20 6 June 02
Process Thresholds Management alert limits that indicate the range of expected results Thresholds based on historical data or business needs - Historical data thresholds establish upper and lower thresholds around 80% - 90% of the data points There should be at least twelve data points Thresholds depend on the stability of the process - Business needs thresholds are based on business goals Thresholds should fall within established historical data thresholds If not within historical data thresholds, improve process so that process capability and business needs fall within same range The range between the upper and lower thresholds is the process capability Additional performance goals may be added GJR - 21 6 June 02
Data Analysis and Reporting Effort Database Project Meas. Data Project Meas. Data Project Meas. Data Self- Audit Database Effort Rework Effort Approval Rates Award Fee Comments Award Fee Percent Cost Variance Cycle Time Process Tailoring Milestone Performance Staffing Self-Audit Findings Database Report Requests via the SI Org Lead Reports Monthly Organizational Quarterly Baseline Ad-hoc Analysis Proposal Operations Program Manager SIPRB GJR - 22 6 June 02
Summary PSM has successfully been used as the foundation of the measurement process at all levels - Provides a pragmatic approach - Extensible to all disciplines - Tailorable to address organization and enterprise needs Supplemented PSM to provide measurement guidance to meet all needs - Quantitative management - Enterprise measurement - Technical performance GJR - 23 6 June 02