DRASTICALLY REDUCE TIME AND CREATE ACCURATE FORECASTS. Estimation addiction. New Product Investment. Reality Waste in Complexity

Transcription

1 DRASTICALLY REDUCE TIME AND CREATE ACCURATE FORECASTS 2018 Semen Cirit A case study on real software projects New Product Investment Estimation addiction Customers come to us with an idea for a new product and People who find value on estimates are just addicted to a they always ask the questions how long will it take how much will it cost us to deliver? They need a delivery date and a budget estimate. practice (estimation) to get something they find valuable and feel comfortable: uncertainty reduction, budget projections, sales proposals, investment decisions etc. But, these are not customer value!!! Dream Customer Value Reality Waste in Complexity We are seeking for a certain plan in an uncertain Software systems are complex environment Deterministic planning masks the uncertainty instead of highlighting it Complex dependencies in software projects hide really big risks Accidental complications vs. Essential complication 1

2 Estimates Dysfunctions Detailed estimation Internal politics Project management paradigm believes that uncertainty Estimate bargaining Blame shifting Late changes could be eliminated by a more detailed planning Buffers are not very usefull, mostly not applicable due to budget constraints Way Estimation vs. Forecasting Planners could not know everything they needed to Our proposal is forecasting!!! know and every number is a random variable. So the idea is reducing the estimation effort and focus on stabilizing the team throughput and using past information to drive forecasts Forecasting promises more accuracy in plans by taking an "outside view" on the project being forecasted based on knowledge about actual performance in a reference class of comparable projects. Reference Class Forecasting is based on the work of the Princeton s psychologist Daniel Kahneman who won the Nobel Prize in economics in Choosing the Right Reference Class Reference class distribution Team structures A reference system is needed to: take an outside view on the development system Technologies used Development processes used and the method of capturing the requirements Client types Business domains that worked on the project forecast delivery time convince customer for investment 2

3 Project as a Kanban System Every Project can be seen as a batch of work items Each batch should have a customer value and start and end date for the successfull delivery Inter-Departure Time (IDT) is the time between two successive deliveries Delivery time (T) = = 30 days IDT = 0 IDT = 5 days IDT = 0 IDT= 10 Days IDT = 8 days IDT= 7 Days Takt Time In manufacturing they measure Takt Time in hours, minutes even in seconds for the mass production In software development, we are calculating takt time in days Delivery time Total time is needed for delivery of the whole product!" = $"" = $%&'()*+,-*).(/ "" (Takt Time) :;<3 3456< = 100 :;<3!!(!#$%!&'() =,-. / 01!, #67!! = 10 7%9:&(7 = 3 5#67/7%9:6 NTT assumes linear delivery rate Most projects have non-linear one 3

4 Z-curve Each leg in Z-curve First leg - learning climbing the learning curve conducting experiments to cover the riskiest work items Innovation! setting up environments adapting to client s culture and procedures understanding new business domain mastering new technology Third leg - Cleaning Clean up the battlefield Fix some outstanding defects Support the transition of the project deliverable into operation Different work type Different level of variation Different staffing in terms of headcount and level of expertise Second leg - productivity If the project is scheduled properly the system should be like a clockwork sustainable pace, no stress, no surprises Probability distribution of Takt Time To get better results, we need to bootstrap the distribution of the Takt Times which are taken by outside view Monte Carlo simulation can be used in engineering for quantitative probabilistic analysis instead of using deterministic inside view for planning 4

5 Bootstrapping It falls in the broader class of resampling methods. Bootstrap is also an appropriate way to control and check the stability of the results. Bootstrapping is consistent for the same reference class We will use it for resampling of the Takt Times of our sample Monte Carlo Simulation To model phenomena with significant uncertainty in inputs, such as the calculation of risk in business. With Monte Carlo Simulation, predictions of failures, cost overruns and schedule overruns are better than human intuition. Preparing distribution Need to capture «Inter Departure Time (IDT)» of each item by looking start and end date of the items in the sample Need to get «Count of Done (N)» and «In Parallel» numbers from «Done Date» to calculate takt time of each item Bootstrap Takt times of the sample Calculate «Delivery Date» of each resample Generate Average Takt Time (AvgTT) distribution using Monte Carlo simulation When a new project to estimate, generate Delivery time probability distribution based on AvgTT Use 85 percentile as the delivery time of the project Project delivery time T! = # $%!! $% + # $'!! $' + # $(!! $( Where:!! $% is the Takt Time for the 1 st leg of the Z-curve!! $' is the Takt Time for the 2 nd leg of the Z-curve!! $( is the Takt Time for the 3 rd leg of the Z-curve # $% is the number of items delivered during the 1 st leg of the Z-curve # $' is the number of items delivered during the 2 nd leg of the Z-curve # $( is the number of items delivered during the 3 rd leg of the Z-curve Case study on Jira We have a past project: 72 Stories and 512 product defects Dev environment was Java and Angular.js Remote customer Web domain Software methodolgy was Scaling Remote Scrum 5 team members 2 senior, 3 junior We will try to predict a new project: 50 Stories and 100 Product defect Same dev environment Same customer structure Same domain Same methodology 5 team members 2 senior, 3 junior Steps to create a distribution Need to capture «Inter Departure Time (IDT)» of each item by looking start and end date of the items in the sample Need to get «Count of Done (N)» and «In Parallel» numbers from «Done Date» to calculate takt time of each item Bootstrap Takt times of the sample Calculate «Delivery Date» of each resample Generate Average Takt Time (AvgTT) distribution using Monte Carlo simulation When a new project to estimate, generate Delivery time probability distribution based on AvgTT Use 85 percentile as the delivery time of the project 5

6 Steps 1 Need to capture «Inter Departure Time (IDT)» of each item by looking start and end date of the items in the sample Step 3 Step 2 Need to get «Count of Done (N)» and «In Parallel» numbers (N) from «Done Date» to calculate takt time of each item Step 4 Z curve Generate Average Takt Time (AvgTT) distribution using Monte Carlo simulation of the sample High level planning High level planning Z curve When a new project to estimate, generate Delivery time probability distribution based on AvgTT 50 stories and 100 product defects 6

7 Step 5 - Results More Results 72 stories and 512 Defects 245 days Real result 261,47 days forecast with 85% percentile Has 89% accuracy on average FIX VERSION Real JIRA data CREATED DATE COMPLETE D DATE TIME CYLE AVERAGE- T NO-ESTIMATION data PERC 85 PERC 95 # of Product Defect # of Story Accuracy X ,32 261,47 277, ,86% Y ,59 131,84 141, ,48% z ,35 168,24 214, % 50 stories and 100 defects probably will finish in: 88,88 days forecast with 85% percentile 7