Developing and Delivering Complex Projects using Quantitative Risk Analysis Colin Cropley BE(Chem), PMP, Cert PRINCE2 Practitioner Managing Director, Risk Integration Management Pty Ltd
Presentation Outline Monte Carlo & Australia s poor project performance QRA: An emerging project management tool What keep SRA & CRA apart Scepticism toward MCM Importance of Schedule to QRA True Integrated Cost & Schedule Risk Analysis (IRA) Comparison with Serial SRA to IRA Examples of use of IRA www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 2
Addressing Australia s Poor Project Record IPA (2009): 74% of completed large projects they assessed had been failures, mainly due to poor front-end planning PwC (2012): ~90% of project failures due to management Focus here on use of MCM to improve project development and delivery: Review history & use of MCM Identifies IRA as best available tool, compares with SRA to CRA Examples given of use of IRA Challenges to overcome for best practice QRA to prevail www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 3
History & use of Monte Carlo Method (MCM) Developed from the Manhattan Project in WWII First applied to schedules (PERT) by RAND Corp in 1963 MCM applied to Capex Discounted Cash Flows in 1964 World Bank applied MCM to 3 projects in 1970 Spreadsheet add-on tools @Risk & Crystal Ball released in 1987; used for Cost Risk Analyses (CRAs) Schedule Risk Analysis (SRA) tools mostly spreadsheet types linked to MS Project or built into CPM tools such as Pertmaster, now known as Oracle s Primavera Risk Analysis (PRA) Wide (not universal) acceptance of necessity for project SRAs &/or CRAs Quantitative Risk Analyses (QRAs) before and during project execution www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 4
PROJECT QRA: EMERGING TOOL Increasing Use of Monte Carlo Method for Projects without Consensus on Methodology and with Scepticism from Some www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 5
Separation of Cost & Schedule Risk Analyses Despite the acceptance that time is money in projects, there is resistance to integrating cost and schedule risk analyses. Here s why: CRAs done by estimators and cost controllers (quantity surveyors & cost engineers) using @Risk or Crystal Ball SRAs done by planners & schedulers (tend to be design & construction engineers) using PRA or @Risk for Project or other tools SRA tools that can handle costs are not as easy to use for costs as for schedule Even where same practitioners perform both SRA & CRA, they tend to use different simulation tools due to evolved specialisations www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 6
Scepticism about Monte Carlo Method Significant doubt about QRA value due to hearsay or experience of poor / misleading use in projects Believed to be due to multiple factors: Lack of published evidence project QRA is helpful Immaturity of Project QRA discipline resulting in wide range of offered capabilities and disillusionment Lack of understanding and inability to specify QRA Misuse of QRA to produce the desired results www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 7
Importance of Schedule to Rigour of Analysis SRA modelling of key milestones depends on how well the schedule model represents the project: Represents project scope and strategy Allows for inherent uncertainty of task durations (e.g., the daily trip to work) Allows for complexity of project schedule logic: Merge Bias Effect (next slide) Allows for risk events Allows for weather uncertainty www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 8
Merge Bias Effect If we have two identical strings of activities and resources to do them, each with a 20% probability of being finished by the target finish date, what is the probability of both being finished by that date? Start Activity A 20% Prob Activity B 20% Prob P A x P B = 4% Finish Date Probability This is known as Merge Bias Effect (MBE) and it is reason why it is so hard to finish a project on time when many strings of activities converge into finish and delays have occurred. Probability falls geometrically. It is also why it is important to model complex projects with detailed schedules. Summary schedules reduce MBE, falsely increase optimism. www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 9
Success using SRA on detailed schedules Australian LNG Train Completion to RFSU 5 months from planned startup (RFSU), schedule slipping each week 2,700 tasks converted to hourly schedule Duration ranges from superintendents Several SRA iterations, correcting startup logic 3 rd produced P50 within 3 days of actual Middle East LNG Train Completion 4 months from planned RFSU, slipping sched Construction not connected to commissioning, used pressure testing systems dates to link 1,000 task sched modified with SRA iterations Commissioning logic changes resulted in P50 within 1 day of actual RFSU Oil Refinery Enhancement Project 4 mths into 18 mths EPC project, owner required SRA of key milestones First milestone was 95% Engg Complete After ranging of ~700 task sched by EPC Contractor Proj Mgr & Chief Planner, Planned & SRA dates for 95% Engg were: Plnd: 16Sep; P50: 26Oct; P90: 5Nov Proj Director rejected results stating he was negotiating with EPC Contractor to bring 95% Engg Complete MS forward by 2 weeks Subsequently actual 95% Engg was 5Nov 2 further engagements on project resulted, during procurement and construction www.riskinteg.com Australian Lessons for Developing Complex Projects MCPC November 2014 10
INTEGRATED COST & SCHEDULE RISK ANALYSIS What is Integrated Cost & Schedule Risk Analysis (IRA) and How does it compare with Serial SRA to CRA (SRA2CRA)? www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 11
The IRA Process - 1 Schedule representing project scope & strategy and detailed enough to show various alternate critical paths Estimate overlaid on schedule in form of a series of hammock tasks loaded with summations of cost items with similar risk profiles, carefully split into time-dependent ( variable ) and time-independent ( fixed ) costs Task durations and line item costs are ranged with project team Cost & time impact risk events from Project Risk Register and risk factors from ranging workshops mapped into IRA model Probabilistic weather calendars added from weather records www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 12
The IRA Process - 2 Essential schedule and cost correlation added to the model Initial analysis performed, producing for key milestones and cost summaries: Probabilistic Date Distributions Simultaneous Probabilistic Cost Distributions Sensitivity Drivers for milestones and costs Results workshopped with project team, model changed Re-analyse, confirm results and drivers with team or re-revise Quantitative Exclusion Analysis (QEA) to produce accurate drivers Prepare Final Report www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 13
The Benefits of IRA - 1 Simultaneous analysis of schedule & cost distributions 70.0 Yandimoomba Schedule Risks A2000 - First Product : Finish Date 766 100% 29 Jun 16 95% 27 Dec 15 Yandimoomba Schedule Risks Entire Plan : Cost $8,051,127,617 100% $25,955,604,746 95% $18,354,892,326 90% 31 Oct 15 160 90% $16,385,327,473 85% 10 Sep 15 85% $15,016,401,365 60.0 80% 10 Aug 15 140 80% $13,930,930,162 75% 06 Jul 15 75% $12,925,428,558 50.0 70% 08 Jun 15 65% 14 May 15 120 70% $11,964,671,858 65% $11,257,058,900 Hits 40.0 30.0 60% 18 Apr 15 55% 25 Mar 15 50% 28 Feb 15 45% 05 Feb 15 40% 10 Jan 15 35% 16 Dec 14 Cumulative Frequency Hits 100 80 60 60% $10,396,676,291 55% $9,671,103,792 50% $8,807,273,046 45% $8,075,476,734 40% $7,399,934,946 35% $6,669,723,686 Cumulative Frequency 20.0 30% 17 Nov 14 25% 14 Oct 14 20% 14 Sep 14 40 30% $5,761,145,946 25% $4,964,691,030 20% $3,882,577,365 10.0 15% 10 Aug 14 10% 21 Jun 14 20 15% $2,640,383,242 10% $1,212,038,813 5% 07 Apr 14 5% ($881,768,079) 0.0 18 Jun 14 31 Oct 15 0% 27 Oct 13 0 0% ($9,364,585,128) $0 $20,000,000,000 www.riskinteg.com Distribution (start of Developing interval) Complex Projects using Integrated Cost & Distribution Schedule (start Risk of interval) Analysis MCPC Nov 2014 14
The Benefits of IRA - 2 Combined rankings of delay and cost drivers: www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 15
The Challenges of IRA IRA has pitfalls that must be avoided to produce reliable results: Schedule & estimate must be aligned (based on same assumptions) Splits of Fixed & Variable costs must be accurate No double-dipping between duration & cost ranges and risk events Costs must be spread over right groups of tasks to model variable costs Estimate must match schedule for level of breakdown detail Correlation through risk factors, durations & costs must be thorough Megaprojects exacerbate difficulty of getting IRA right: Size and complexity of model increases and need for correlation with it QEA becomes more complex, time-consuming and important www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 16
What is wrong with Serial SRA to CRA? Assumed cost/unit time takes no account of when and where schedule changes occur When before, during or after peak expenditure rate? Where delay costs may be real but not delay the finish What, why and how assumptions between schedule and estimate must be aligned to produce valid cost consequences from duration changes Serial process cuts off delay cause from cost effect and prevents integrated analysis of drivers of cost IRA More rigorous if done correctly Harder to do correctly Can provide key driver information, unifying schedule and cost drivers in one ranking as influences on project cost SRA to CRA Less rigorous Easier to perform Cannot reveal cost consequences of schedule delays schedule drivers are separate from cost drivers www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 17
Examples of Use of IRA All of Santos GLNG Project scope was analysed pre-fid from Sep09 to Nov10. SRA in Jan/Feb 2010 had following results: RIMPL worked with Talisman Energy to produce generic IRAs for O&G Exploration. Rig Moves (unrisked) contrast with Drilling (risked) KUPIO-1 WELL MILESTONE PLANNED P50 P90 ACT/ CURRENT FORECAST FID FIRST LNG CARGO 2SEP10 31JUL14 21DEC10 20MAY15 15APR11 29SEP15 JAN11 2015 RIG MOVE (UN-RISKED) ACT/PLAN DRILLING (RISKED) ACT/PLAN ACTUAL PLAN ACTUAL % PLAN ACTUAL % CF FORECAST TOTAL DAYS 21D 40D 190% 51D 53.8D 105% P87 TOTAL COST $5.79M $7.90M 136% $16.2M $15.4M 95% P45 MANTA-1 WELL RIG MOVE (UN-RISKED) ACT/PLAN DRILLING (RISKED) ACT/PLAN PLAN ACTUAL % PLAN ACTUAL % TOTAL DAYS 35D 45D 129% 31D 29.5D 95% TOTAL COST $8.23M $9.63M 117% $10.2M $9.7M 95% www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 18
Conclusions and Recommendations Development & Delivery of complex projects need major improvement Improving quality of key assessment tool, QRA would be big step forward Most realistic version of QRA shown by this paper to be IRA IRA challenging but useful where: SMEs can reach consensus on ranges & risks Sufficient detail defined, eg, PFS or FS/FEED Quality of results depends on: Capable experienced practitioners, good tools SMEs/Project team capable & experienced Inputs to analysis technically good & aligned IRA has rigorous foundation in Integrated Master Control Schedule All IRA inputs viewable and auditable Assumptions & exclusions in report Use of IRA is scalable: successfully used on small (<$5m) to very large (>$15b) projects IRA can materially improve probability of success through identification & optimisation of major risk drivers Project owners, managers & contractors need to improve their understanding of best practice QRA so can specify use & understand its value www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 19
QUESTIONS? Anyone who wants an electronic copy of the presentation should give me their business card or email me at: colin.cropley@riskinteg.com www.riskinteg.com Developing Complex Projects using Integrated Cost & Schedule Risk Analysis MCPC Nov 2014 20