A Case Study in Predictive Program Management

Transcription

1 Mission Critical Enterprise Systems Symposium A Case Study in Predictive Program Management John Welsh, Bipin Chadha, Zach Horiatis Lockheed Martin Advanced Technology Laboratories Oct. 2004

2 Major programs often encounter major cost and schedule problems despite detailed and comprehensive plans Despite their superiority, weapon systems routinely take much longer to field, cost more to buy, and require more support than investment plans provide for. Defense Acquisitions: Assessments of Major Weapon Programs, GAO Report, GAO , May 2003 Problem Complex interrelationships cause control of program cost and schedule to be difficult Program managers need a small set of key levers (vs. thousands of tasks, schedules, budgets, risks, action items, ) U.S. Navy and Air Force plans to stiffen penalties for cost overruns Bloomberg 2/5/2004 MCES 2004 October 26 28, 2004 Bipin Chadha Page 2

3 What If You Could Know that a software project would be delayed before it was reported See management data in real-time and play what-ifs Recognize cross IPT shortfalls and impacts immediately See third and fourth level effects of change rather than those immediately apparent Do multi-variant cost sensitivity analysis on the fly Recognize complex impacts of change at the speed of information rather than speed of status report delivery MCES 2004 October 26 28, 2004 Bipin Chadha Page 3

4 Levels of Understanding Events (Reactive) This event happened therefore we must take this action Q3 earnings results announced leading to drop in stock price Patterns of behavior (Responsive) Interpretation of current events in the context of long-term historical change Understand a pattern of behavior Product lifecycles follow the S curve Systemic structure (Generative) Addresses the question - what causes the pattern of behavior Attempt to address the underlying causes of behavior -- Senge, P., The Leader s New Work: Building Learning Organizations, SMR, Fall, 90 MCES 2004 October 26 28, 2004 Bipin Chadha Page 4 Systemic Methods Statistical Methods

5 Thesis Prediction on large complex programs is feasible because Most program surprises result from a small number of key elements Dependencies Systemic behavior patterns These are normally not considered in planning Hence root causes of program surprise go undetected Information for effective analysis is straightforward / easy to obtain Analysis methodology tolerates parameter variation and imprecision MCES 2004 October 26 28, 2004 Bipin Chadha Page 5

6 Comprehensive Program Management... Sched Tech Risk Cost 2 nd Integrated Management Framework Linked data elements Configuration management Workflow Access control Provides views across disciplines, supports change management and traceability 1 st Generation Program Management IPT s assessments: Predictive but subjective Provides drilldown capability Cost EVMS IMS/IMP TPM WBS Requirements Risk SOW Communicate Knowledge Decide Sense Production IMS SOW Joint Analysis LSI IMP Cost Act WBS GFX IPT 1 SE&I Program Management IPT 2 Management Risk IPT 3 Logistics IPT 4 XBR IPT 5 IPT 6 Requirements TPMs 3 rd Generation Predictive Program Management System dynamic model for program management Causal relationships are modeled Agent assessments: Predictive and objective Provides look ahead capability and what if levers MCES 2004 October 26 28, 2004 Bipin Chadha Page 6

7 Predictive Program Management Environment Statistical Models Prediction Models Design Structure Matrices System Dynamics Models Coupling and Dynamics Management Tools (EVMS, Schedule, etc.) Collaborative Program Management Environment Intelligent Mobile Agents Information Quality Internet Backbone MCES 2004 October 26 28, 2004 Bipin Chadha Page 7

8 Predicted Today Past History Alerting Agents Sched Tech Risk Cost Sched Tech Risk Cost Sched Tech Risk Cost Sched Tech Risk Cost Sched Tech Risk Cost Sched Tech Risk Cost Architecture Concept Sched Tech Risk Cost Sched Tech Risk Cost Recommender Agents Feedback Task Monitoring Agents Communicate IPT 1 SE&I Decide IPT 2 Management Management Team Knowledge Sense Act Production SOW IMS WBS Joint Analysis LSI Program Management IMP What-Ifs Project Control Dashboard Cost GFX Risk Requirements System IPT 3 Logistics IPT 4 Dynamics IPT 6 IPT5 XBR Models TPMs Data Gathering/ Monitoring Agents Program Plan EV Schedule Risk Program Information MCES 2004 October 26 28, 2004 Bipin Chadha Page 8

9 Key Coupling Factors Program Element Design Structure Matrix Program Element Dependency Type Coupling / Interdependencies Program Elements Activities Systems Organizations Dependency Types Information Physical Energy Resource Common dependency patterns inherent across programs enable a small number of models to represent the dominant behaviors. MCES 2004 October 26 28, 2004 Bipin Chadha Page 9

10 DSM Dependency Analysis For each dependency type, matrices are analyzed to find task or system groupings that are most important to watch Supports automated analysis Easy to interpret, visualize, and analyze Rows and columns: Program elements Matrix entries: Coupling strengths Principle eigenvalues: Key work / design modes; determine the rate and nature of convergence in the presence of change Principle eigenvector: Ranks tasks/subsystems by their relative impact on overall schedule / cost Task 1 Task 2 Task 3 Task 4 A B C D A B C D A x B x x C x x D A affects D Task 1 Task 4 Task 3 Task 2 A D C B A D C B A x x D x C x x B Modules Dependency analysis highlights the subsets programs which require closer attention / monitoring, or need to be decoupled MCES 2004 October 26 28, 2004 Bipin Chadha Page 10

11 DSM Analysis Design mode: Group of tasks that are closely related and cause rework. Characterized by eigenvalues and corresponding eigenvectors Eigenvalues and Eigenvectors determine the rate and nature of convergence of process Magnitude of eigenvalues identifies the geometric rate of convergence Large positive eigenvalues have greater impact Ignore complex eigenvalues with negative real component Negative and complex eigenvalues describe damped oscillations Eigenvectors characterize relative contribution of tasks to rework MCES 2004 October 26 28, 2004 Bipin Chadha Page 11

12 Selected Design Modes can be Analyzed for Systemic Behavior Patterns Interdependencies / Coupling Program Elements Activities Systems Organizations Dependency Types Informational Physical Energy Resource Dynamic Patterns Balancing Loop Reinforcing Loop Hidden Rework Fixes that Backfire Limits to Growth Shifting the Burden Tragedy of Commons Accidental Adversaries Success to the Successful Escalation Drifting Goals The Attractiveness Principle Growth and Underinvestment (core patterns) Common systemic patterns inherent across programs enable a small number of models to represent most of the dominant behaviors (80% solution). MCES 2004 October 26 28, 2004 Bipin Chadha Page 12

13 Systemic Behavior Behavior of a system that emerges due to dynamic interactions among components System characteristics System structure drives system behavior (especially the feedback loops) System exhibits properties and behaviors SYSTEM different from the parts Comp1 Everything is connected to everything Comp2 There are no independent variables Comp3 Many effects are indirect and delayed Comp1 Many dependencies are non-linear Behavior is dynamic Comp2 Comp3 Systemic Behavior Can be a Significant Source of Risk and Surprise MCES 2004 October 26 28, 2004 Bipin Chadha Page 13

14 System Dynamics Modeling (SDM) Symptomatic Solution B Problem Symptom B Fundamental Solution R Shifting the Burden Pattern Side Effect Observation Problem arises over and over again You apply the fix, there is some relief, but the problem surfaces again What is going on You are applying a symptomatic solution that alleviates symptoms immediately but exacerbates the problem over time The problem causes symptoms to reappear after some delay Since the fundamental solution has a delay built into it (before you see results) it forces you to take the symptomatic solution in the short term, thus trapping you in the vicious cycle Source: The Fifth Discipline Fieldbook by Senge, et al MCES 2004 October 26 28, 2004 Bipin Chadha Page 14

15 Where Have These Techniques DSM Been applied Before? Developed at MIT Applied in Automotive industry Applied on some DoD programs MIT runs regular courses on it for program managers Significant body of literature exists that supports this technique SDM Developed at MIT Applied in a wide range of industries to analyze systemic problems Applied on some DoD programs Significant body of literature exists that supports this technique Several patterns have been well documented Individually both techniques have been proven useful, but haven t been tied into a comprehensive solution MCES 2004 October 26 28, 2004 Bipin Chadha Page 15

16 Intelligent Mobile Agents for PM Agent Tasks Information gathering Data analysis resulting in R/Y/G labeling Determine each WBS s associated R/Y/G cost / schedule / risk / etc. Trend analysis Historical data analysis Action Item monitoring Critical path monitoring Generate alerts Leveraging Lockheed Martin Earned value analysis agent technology to manage a SDM; user sees Predictive analysis Information Recommendations Not agents Not volumes of data MCES 2004 October 26 28, 2004 Bipin Chadha Page 16

17 Case Study: DSM Based Rework Analysis DSM analysis shows one dominant design mode (eigenvalue) and relative impact of tasks (eigenvector) on the overall schedule Analysis is robust in the presence of parameter variation / uncertainty Analysis predicts approximate percent rework for the tasks due to dependencies / coupling Program manager can use this analysis to verify estimated rework Analysis can be used to evaluate the impact of task restructuring on expected rework (which is often the cause of surprise towards the end) MCES 2004 October 26 28, 2004 Bipin Chadha Page 17

18 Case Study: DSM used to Create System Dynamics Model (SDM) DSM can identify a small subset of dependencies that require detailed modeling via SDM Dependency Dependency With Delay MCES 2004 October 26 28, 2004 Bipin Chadha Page 18

19 Case Study: Model Results (Actual vs.. Predicted) Level 1 Problem Reports Linear Plan Non-Linear Results Planned Predicted Level 2 Problem Reports Actual Crossover Point Model is successfully predicting the observed data. MCES 2004 October 26 28, 2004 Bipin Chadha Page 19

20 Conclusions Systemic model demonstrates the behavior in the presence of rework and feedback The results of the model are broadly applicable All iterative processes will demonstrate this behavior to a certain extent (rework pattern) Model is successfully predicting the crossover effect This is where the status goes from green to red overnight Early analysis can predict this effect and program managers can plan accordingly to prevent surprise Model can be run anytime with latest data to detect possibility of crossovers (ideal task for an agent) Planning to deploy capability on programs MCES 2004 October 26 28, 2004 Bipin Chadha Page 20