Dynamics of Project Performance

Transcription

1 ESD.36J System & Project Management + Dynamics of Project Performance System Dynamics and Project Management Class Three (9/30/03) James M. Lyneis

2 Topics Review Practice for 9/30 The dynamics of project performance knockon effects Modeling the feedback effects Model Calibration Practice for 10/2 9/30/03 ESD.36J SPM James M. Lyneis 2

3 Topics Review Practice for 9/30 The dynamics of project performance knockon effects Modeling the feedback effects Model Calibration Practice for 10/2 9/30/03 ESD.36J SPM James M. Lyneis 3

4

5 What did you conclude re. the effect of productivity, quality, and rework discovery time on time required to finish the project? 9/30/03 ESD.36J SPM James M. Lyneis 5

6 Sensitivity of Completion Date to +/33% Variation in Parameters Quality Completion Date Productivity Completion Date Rew ork Discovery Delay Completion Date Low Q Base High Q 0 Low Pdy Base High Pdy 0 Low Rw dd Base High Rw dd 9/30/03 ESD.36J SPM James M. Lyneis 6

7 Sensitivity of Total Work Done to +/33% Variation in Parameters Quality Total Work Done Productivity Total Work Done Rework Discovery Delay Total Work Done Low Q Base High Q 0 Low Pdy Base High Pdy 0 Low Rwdd Base High Rwdd 9/30/03 ESD.36J SPM James M. Lyneis 7

8 Sensitivity of Completion Date Variations in Quality have biggest impact Nonlinear effect Determines total amount of work Rework discovery time has the least impact (but the simple model does not have a quality on quality feedback). 9/30/03 ESD.36J SPM James M. Lyneis 8

9 8 6 + Quality =.5 Graph for Undiscovered Rework 4 Quality = Quality = Time (Month) Undiscovered Rework : Class2 Step1 Undiscovered Rework : Class2 Step1 Q100 Undiscovered Rework : Class2 Step1 Q50 Task Task Task 9/30/03 ESD.36J SPM James M. Lyneis 9

10 Quality = 1.0 Graph for Work Done Quality =.75 Quality = Time (Month) Work Done : Class2 Step1 Work Done : Class2 Step1 Q100 Work Done : Class2 Step1 Q50 Task Task Task 9/30/03 ESD.36J SPM James M. Lyneis 10

11 Rework Shows Up As Revised Work Products Accomplishment of Revisions to Design Products (e.g., Drawings) REV 0 REV 1 REV 2 REV 3... TIME Virtually all complex projects have revision cycles. 9/30/03 ESD.36J SPM James M. Lyneis 11

12 Managing the rework cycle Do common project metrics handle (undiscovered) rework? Do management incentives encourage the discovery of rework? 9/30/03 ESD.36J SPM James M. Lyneis 12

13 Additional Metrics (Used in Model) Fraction Perceived to be Complete = (Work Done + Undiscovered Rework) / Initial Work to Do Fraction of Work Really Complete = (Work Done) / Initial Work to Do 9/30/03 ESD.36J SPM James M. Lyneis 13

14 Perceived vs. Really Complete in Simple Rework Cycle Model 1 Progress Time (Months) Fraction Perceived to be Complete : Step2 Fraction Really Complete : Step2 Fraction Fraction 9/30/03 ESD.36J SPM James M. Lyneis 14

15 On an actual project 100. Perceived vs. Actual Progress: SRS Development Perceived Actual Disguised results from actual aerospace project Year 1/93 1 Year2 1/94 Year 1/953 Year 1/964 Year 1/975 Year 1/986 Year 1/997 Year 1/008 Year 1/019 1/02 Year 10 TIME c:\mydocu~1\jml\hughes\hisjml.dyn 8/19/97 9:29 9/30/03 ESD.36J SPM James M. Lyneis 15

16 Topics Review Practice for 9/30 The dynamics of project performance knockon effects Modeling the feedback effects Model Calibration Practice for 10/2 9/30/03 ESD.36J SPM James M. Lyneis 16

17 Examples of Project Behavior Modes Project Staffing Typical Plan Time 9/30/03 ESD.36J SPM James M. Lyneis 17

18 Examples (continued) Productivity (Normalised) 2 Typical Plan 1 Time 9/30/03 ESD.36J SPM James M. Lyneis 18

19 Dynamics of Project Performance The rework cycle Quality Undiscovered rework Feedback effects on productivity and work quality Positive, reenforcing, often vicious circles Negative, controlling Knockon effects between work phases 9/30/03 ESD.36J SPM James M. Lyneis 19

20 A system dynamics model usually represents several phases of work, but is more aggregate than a CPM model Software Design Software Code and Test System Engineering Integrate and Test Hardware Design Hardware Build and Test 9/30/03 ESD.36J SPM James M. Lyneis 20

21 Software System Engineering Peace Shield Air Defence System Model Architecture Software Development Software Support CONUS Op. & Main/ ILS Mgt. & Admin. HW Installation & ICO Test SRS Development Top Level & Detailed Design Code & Unit Test Integration & Type II Test Type I Test KOSA SW & Testing Program Management Office Downstream Progress, Availability, & Quality Effects Upstream Rework Discovery Effects Support Effects Customer Hardware Logistics Subcontractors Other HASI Programs 9/30/03 ESD.36J SPM James M. Lyneis 21

22 Interphase Effects Downstream &/or Parallel Availability of work products Quality of work products Upstream Rework discovery Above apply internally and to suppliers General Sharing of staff 9/30/03 ESD.36J SPM James M. Lyneis 22

23 Interphase Connections: UpstreamDownstream Design Progress; Quality LABOR APPLIED Productivity Quality LABOR APPLIED Productivity Quality WORK TO BE DONE WORK BEING DONE WORK REALLY DONE WORK TO BE DONE WORK BEING DONE WORK REALLY DONE KNOWN REWORK UNDISCOVERED REWORK REWORK DISCOVERY KNOWN REWORK UNDISCOVERED REWORK REWORK DISCOVERY Progress Construction 9/30/03 ESD.36J SPM James M. Lyneis 23

24 Software Design Interphase Connections: Parallel WORK TO BE DONE LABOR APPLIED WORK BEING DONE Productivity Quality WORK REALLY DONE Progress: Quality KNOWN REWORK UNDISCOVERED REWORK REWORK DISCOVERY Progress; Quality LABOR APPLIED Productivity Quality WORK TO BE DONE KNOWN REWORK WORK BEING DONE UNDISCOVERED REWORK REWORK DISCOVERY WORK REALLY DONE Hardware Design 9/30/03 ESD.36J SPM James M. Lyneis 24

25 Dynamics of Project Performance The rework cycle Quality Undiscovered rework Feedback effects on productivity and work quality Positive, reenforcing, often vicious circles Negative, controlling Knockon effects between work phases Availability and quality of work products Progress to discover upstream rework 9/30/03 ESD.36J SPM James M. Lyneis 25

26 How might we represent suppliers? platforms?... 9/30/03 ESD.36J SPM James M. Lyneis 26

27 Topics Review Practice for 9/30 The dynamics of project performance knockon effects Modeling the feedback effects Model Calibration Practice for 10/2 9/30/03 ESD.36J SPM James M. Lyneis 27

28 If the world consists only of stocks and flows, what are those other variables indicated on the diagrams?? Auxillaries Constants (e.g., factors which may be stocks or flows, but which do not change over the time span of the simulation) External inputs 9/30/03 ESD.36J SPM James M. Lyneis 28

29 Auxillaries or Converters Serve Several Purposes... Break up rates into meaningful components Provide alternative measures for stocks or flows Reduce diagram clutter 9/30/03 ESD.36J SPM James M. Lyneis 29

30 Potential Work Rate Potential Work Rate = Staff * Productivity Tasks/Month People * Tasks/Month/Person 9/30/03 ESD.36J SPM James M. Lyneis 30

31 Productivity & Quality Productivity =?? Quality =?? 9/30/03 ESD.36J SPM James M. Lyneis 31

32 Productivity PRODUCTIVITY = NORMAL PRODUCTIVITY * EFFECT OF STAFF EXPERIENCE * EFFECT OF QUALITY OF PRIOR WORK *... Dimensions: Productivity Tasks/Month/Person Normal Productivity Tasks/Month/Person Effects Dimensionless 9/30/03 ESD.36J SPM James M. Lyneis 32

33 Quality QUALITY = NORMAL QUALITY * EFFECT OF STAFF EXPERIENCE * EFFECT OF QUALITY OF PRIOR WORK *... Dimensions: Quality Fraction Normal Productivity Fraction Effects Dimensionless 9/30/03 ESD.36J SPM James M. Lyneis 33

34 Effect of Staff Experience Effect of Experience = Function ( Years on Project) 9/30/03 ESD.36J SPM James M. Lyneis 34

35 Effect of Staff Experience Effect of Experience = Function (Years on Project) ? Years on Project 9/30/03 ESD.36J SPM James M. Lyneis 35

36 How do we determine these effects? The effects are first estimated based on common sense, later to be verified during model calibration. It helps to specify likely values at extreme points, and draw a smooth curve in between. 9/30/03 ESD.36J SPM James M. Lyneis 36

37 Effect of Staff Experience Extreme points 1.0 X X? Years on Project 9/30/03 ESD.36J SPM James M. Lyneis 37

38 Connecting the extreme points X X Years on Project 9/30/03 ESD.36J SPM James M. Lyneis 38

39 Some other connections are plausible Years on Project 9/30/03 ESD.36J SPM James M. Lyneis 39

40 Others are not Years on Project 9/30/03 ESD.36J SPM James M. Lyneis 40

41 Relationships Similar to the Experience Effect Would Be Specified For Other Effects on Productivity and Quality Experience Adequacy of supervision Availability of resources Morale Schedule pressure Fatigue Quality of prior work Skills available vs. needs Tools Processes Availability of supplier information &/or materials... 9/30/03 ESD.36J SPM James M. Lyneis 41

42 Note difference with statistical estimation Productivity = a + b*experience+c*prior quality 9/30/03 ESD.36J SPM James M. Lyneis 42

43 Rules of Thumb and Checks 1. Variables and constants should correspond to real system variables 2. Remember to represent delays in flow channels, in perceiving information, and in making or implementing decisions 3. Distinguish desires from realizations 4. Make sure that equations balance dimensionally 5. Check the validity of equations at extreme points, e.g., zero or very high values of the inputs 6. Avoid the use of imbedded functions they make the equations harder to understand and can lead to errors 9/30/03 ESD.36J SPM James M. Lyneis 43

44 Connecting Morale & Overtime... from a student s On a couple of occasions in lecture yesterday you discussed "Overtime" as it relates to "Morale". In doing causal loops, I find that the relationship between the 2 can be different and dependent on time. For example, initially the morale of employees may increase or support the increase of overtime which provides a positive relation between the 2. However, as the overtime continues, I believe the morale will decrease as employees will start to desire a life outside of work which supports a negative relation between the 2. I guess that in the actual SD modeling the developer would create a curve/equation for morale over time, but what about for causal loops? How do you handle the dynamics with causal loops? Maybe the loops are just an initial starting point for the development? 9/30/03 ESD.36J SPM James M. Lyneis 44

45 Representing morale 9/30/03 ESD.36J SPM James M. Lyneis 45

46 Topics Review Practice for 9/30 The dynamics of project performance knockon effects Modeling the feedback effects Model Calibration Practice for 10/2 9/30/03 ESD.36J SPM James M. Lyneis 46

47 Calibration (Parameter Estimation) 9/30/03 ESD.36J SPM James M. Lyneis 47

48 Parameter Estimation in System Dynamics A priori estimates based on direct observation educated guesses by managers similar parameters from other applications Refine initial estimates in the process of calibration 9/30/03 ESD.36J SPM James M. Lyneis 48

49 Calibration is part of the scientific method Calibration is not curvefitting via exogenous variables, but systematic comparison of model output to all data and resolution of any discrepancies changes in structure changes in parameters decision to ignore (or sometimes use exogenous effect) 9/30/03 ESD.36J SPM James M. Lyneis 49

50 Calibration Should Be a Part of the Standard Method Data is a real source of information Calibration often uncovers errors/omissions in insightbased models Cost is small relative to the stakes in most large organizations 9/30/03 ESD.36J SPM James M. Lyneis 50

51 Why do we calibrate a model? Assure model contains all of the structure necessary to create problem Accurately determine leverage points Accurately price costbenefits of alternatives Sell results to those not on project team Generate a benchmarking data base for future use and learning 9/30/03 ESD.36J SPM James M. Lyneis 51

52 Sources of information for calibration Hard, numerical time series (e.g., for staffing, work accomplished, overtime levels, etc.) Soft, management observations of what happened (e.g., how morale, experience, and other factors might have caused productivity and quality to change) such observations must be tied to what was happening on the project 9/30/03 ESD.36J SPM James M. Lyneis 52

53 Project Dynamics Time Remaining Scheduled Completion Time Expected Completion Time Morale Schedule Pressure OutofSequence Work Availability of Prerequisites Work Quality to Date Added Work Skill & Experience Organisational Size Changes Productivity Quality Work To Be Done Known Rework Perceived Progress Progress Turnover Overtime Undiscovered Rework Rework Discovery Equivalent Staff on Project Hiring Hours Expended to Date Staffing Requested Expected Hours at Completion 9/30/03 ESD.36J SPM James M. Lyneis 53 Staff Work Really Done Obsoleted Work Data likely to be available

54 Once calibration is completed... We have a model that is useful for: Learning what really happened on prior projects what could have been done better Dispute resolution 9/30/03 ESD.36J SPM James M. Lyneis 54

55 What happens if you can t calibrate? Use logical parameter estimates Simulation model still better than mental models 9/30/03 ESD.36J SPM James M. Lyneis 55

56 Topics Review Practice for 9/30 The dynamics of project performance knockon effects Modeling the feedback effects Model Calibration Practice for 10/2 9/30/03 ESD.36J SPM James M. Lyneis 56

57 Lessons Rework (Errors) Can Linger for Long Periods of Time as Undiscovered Rework Reducing Rework, and Detecting it Earlier, is the Best Way to Cut Cycle Times, Finish Earlier, and increase Delivered Quality Common metrics & reward systems do not encourage high quality and rework discovery Interactions between phases of work, different projects, suppliers, create opportunities for knockon effects Calibration will improve the accuracy of our parameter estimates, and ultimately policy decisions 9/30/03 ESD.36J SPM James M. Lyneis 57

58 Practice for 10/2 1. Download the Class2 Step2 model. This is the rework cycle model with a variable time to discover rework. Time to discovery rework, as illustrated in the following figure, depends on a maximum time and a minimum time, and an effect of work progress. Time to discover rework falls from the maximum to the minimum time as fraction complete increases. 9/30/03 ESD.36J SPM James M. Lyneis 58

59 Rework cycle with variable time to discover rework Maximum Time to Discover Rework Minimum Time to Perform a Task Work to Do Quality Maximum Work Rate Rework Generation Rework Discovery Time to Discover Rework Productivity Feasible Work Rate Work Accomplishment Undiscovered Rework Potential Work Rate Staff Work Done Project Finished Switch <Rework Generation> Rate of Doing Work Cumulative Work Done <Work Done> Work Believed to Be Done Minimum Time to Discover Rework <Work Accomplishment> Fraction Really Complete Fraction Perceived to be Complete <Initial Work to Do> Effect of Work Progress Table for Effect of Work Progress 9/30/03 ESD.36J SPM James M. Lyneis 59

60 Practice for 10/2 2. Add the effect of prior work quality on quality as discussed in class and illustrated in the following figure. 3. Simulate the model and compare to the original Class2 Step 2 results. How does the addition of quality on quality feedback affect project performance? 4. If we essentially eliminate undiscovered rework by setting the maximum time to discover rework to 0.25 months, what happens to project performance? 9/30/03 ESD.36J SPM James M. Lyneis 60

61 Productivity Normal Quality Quality Minimum Time to Perform a Task Maximum Work Rate Work Accomplishment Potential Work Rate Feasible Work Rate Staff Level Table for Effect of Prior Work Quality on Quality Effect of Prior Work Quality on Quality <Initial Work to Do> Project Finished Switch Work to Do Rework Generation Undiscovered Rework Work Done <Work Done> Average Work Quality Maximum Time to Discover Rework Rework Discovery Time to Discover Rework <Rework Generation> Rate of Doing Work Cumulative Work Done Minimum Time to Discover Rework <Work Accomplishment> Fraction Really Complete Work Believed to Be Done Fraction Perceived to be Complete Effect of Work Progress <Staff Level> Effort Expended Cumulative Effort Expended <Project Finished Switch> <Initial Work to Do> Table for Effect of Work Progress 9/30/03 ESD.36J SPM James M. Lyneis 61

62 Hints 1 Effect of Prior Work Quality on Quality: kup Table for Effect of Prior Work Qua Note: The effect of prior quality on current quality is assumed to be proportional and error in past work creates an error in current work. Given that in this simple model quality represents several effects of work errors, this strong relationship may be reasonable. 9/30/03 ESD.36J SPM James M. Lyneis 62

63 Hints 2 Average Work Quality: Max(1e006,Work Done)/Max(1e 006,Work Believed to Be Done) [Note: the max in the denominator prevents division by 0, and in the numerator allows the equation to return a quality of 1.0 when no work is yet completed.] 9/30/03 ESD.36J SPM James M. Lyneis 63

64 Hints 3 Normal Quality: Set this to 0.85 (vs in Class2 Step2 model) to reflect that we are now explicitly representing other factors affecting project quality. 9/30/03 ESD.36J SPM James M. Lyneis 64

65 Graphical Functions in Vensim 9/30/03 ESD.36J SPM James M. Lyneis 65

66 Step 1 Set Up Model As Shown Below Normal Quality Quality Effect of Prior Work Quality Table for Effect of Prior Work Quality Average Work Quality 9/30/03 ESD.36J SPM James M. Lyneis 66

67 Step 2 Equations... Quality = Normal Quality*Effect of Prior Work Quality Effect of Prior Work Quality = Table for Effect of Prior Work Quality(Average Work Quality) 9/30/03 ESD.36J SPM James M. Lyneis 67

68 Step 3 Define the Table Click to define equation for Table for Effect of Prior Work Quality Select the Type as Lookup Click on the As Graph button to see the function You can then type in values or create a line on the blank graph You can edit the values directly if desired 9/30/03 ESD.36J SPM James M. Lyneis 68

69 Voila! kup Table for Effect of Prior Work Qua /30/03 ESD.36J SPM James M. Lyneis 69

70 In the shortterm, overtime increases morale and productivity + Productivity Morale + Income + Overtime 9/30/03 ESD.36J SPM James M. Lyneis 70

71 But in the longerterm, sustained overtime reduces morale and productivity... + Productivity Morale + Average Overtime + Income + Overtime 9/30/03 ESD.36J SPM James M. Lyneis 71

72 Connecting the feedbacks through progress... + Productivity Average Overtime + Morale + Income + + Progress + Overtime + Staff Needed 9/30/03 ESD.36J SPM James M. Lyneis 72