Metrics for Management and VDC Methods to Predict and Manage them

Transcription

1 Metrics for Management and VDC Methods to Predict and Manage them John Kunz CIFE, Stanford University

2 Reflections W2 Tuesday

3 +/ Tuesday Share assumptions, Monitor, Challenge

4 4

5 5

6 Reflections W3 Tuesday

7 7

8 Simulation to assess meaning of input variables for project Build good model Create >= 3 cases with different assumptions about an input variable, e.g., Project failure rate Simulate each case; evaluate wrt target If all meet or fail to meet target, factor is not a risk If assumed value changes whether target is met then factor is a risk For assumed values that are risks, ask team best value to assume most reasonable Methods to mitigate risk that factor gets worse 8

9 Notices Please do Assessment-2 as individual assignment Submission-3: Business plan: due midnight July 26

10 Submission-2 grading rubric Metrics Item Task 1 Plans made before ICE sessions; 2 Project sketches for the project product, organization and process (POP); 3 Functional intent for the POP models; 4 Models of the project POP; P, O, P 5 Metrics - List of important features, performance objectives and predicted performance of these models; 6 Annotations to show relationships between sketches and project templates; Examples of prediction charts for the organization and process behavior of baseline and best cases, including SimVision project Gantt chart and project work 7 breakdown chart. Clearly indicate your evaluation of how well each predicted behavior conforms with functional process objectives; Checklist for process management reliability with 8 annotation to explain support for related product, organization and process functional goals; Project Design Versions template that lists, for each 9 case you created, the changes, rationale and summary of effects. One-slide summary of your business plan that includes 10 statement of who has a problem, the problem, solution with certain features and benefits ; 11 ORID analysis on what you learned this week: General 10

11 S2: Index Helpful! 11

12 S2: Pre-session plans Great example of narrative and detail as a template Cross reference would help 12

13 S2: Pre-session plans Simple, helpful, detailed 13

14 S2: Pre-session plans Simple, helpful Much more detailed; suggests that it may be substantive Best: both ICE pre-plan Problems Outcomes Participants Agenda Resources Member of Member of postsession space, Meeting Problems Outcome intent Intended Participant Pre-session pre-plan Role in ICE for session Desired outcomes Agenda items met? Participants discipline assignments team session wrapup technologies, to focus on (Yes/Partial/No) (yes/no) team models, tools (yes/no) Specify architectural spaces first floor Size building systems Spaces to model are specified Systems in first floor are sized and specified Identify building Components (first floor) construction that take > 1 hour to components install are listed and for first floor added to spec of components to model Mary Architect Owner share project goals and objectives Joe MEP Engineer Define and share team charter Hamid PM PM assure availability of BIM authoring and review tools for team members Sonya Owner Owner share project goals and objectives Yes Discipline expert Yes Review project goals and objectives No Discipline expert No Specify spaces, systems, components to model Yes Facilitator Yes assign modeling tasks to individuals No Recorder Yes Plan coordination activities for each modeling task Partial Yes No Smart Boards Excel Meeting space with tables, chairs for team of 10 14

15 S2: Pre-session plans Simple, helpful Much more detailed; suggests that it may be substantive. Note Outcome monitored results Best: both ICE pre-plan Problems Outcomes Participants Agenda Resources Member of Member of postsession space, Meeting Problems Outcome intent Intended Participant Pre-session pre-plan Role in ICE for session Desired outcomes Agenda items met? Participants discipline assignments team session wrapup technologies, to focus on (Yes/Partial/No) (yes/no) team models, tools (yes/no) Specify architectural spaces first floor Size building systems Spaces to model are specified Systems in first floor are sized and specified Identify building Components (first floor) construction that take > 1 hour to components install are listed and for first floor added to spec of components to model Mary Architect Owner share project goals and objectives Joe MEP Engineer Define and share team charter Hamid PM PM assure availability of BIM authoring and review tools for team members Sonya Owner Owner share project goals and objectives Yes Discipline expert Yes Review project goals and objectives No Discipline expert No Specify spaces, systems, components to model Yes Facilitator Yes assign modeling tasks to individuals No Recorder Yes Plan coordination activities for each modeling task Partial Yes No Smart Boards Excel Meeting space with tables, chairs for team of 10 15

16 S2: Pre-session plans Simple, helpful Much more detailed; suggests that it may be substantive. Problems Outcomes Participants Agenda Resources Problems for session to focus on Identify which features the app needs Specify layout in the application How will the algoritm work How is the market? What potential customers and competitors are there Desired outcomes A list of all necessary features A scetch of the application in the different views A proposal for how the main algoritm can be built A market research for what potential customers demand and what competitors there are Intended Participants Sofia Katrin Johanna Sofia Participant discipline PM Other Other PM Pre-session assignments Owner share project goals and objectives Owner share project goals and objectives Owner share project goals and objectives Owner share project goals and objectives ICE pre-plan Member of pre-plan team (yes/no) Role in ICE session Member of postsession wrap-up team (yes/no) Agenda items Outcome intent met? (Yes/Partial/ No) Yes Leader Yes Powerpoint No No Yes Discipline expert Discipline expert Discipline expert Yes Yes No Meeting space, technologies, models, tools Smartboard Smartboard Powerpoint 16

17 S2: Functional intent Simple, helpful Nice use of organization model and team charter together with annotations to link features of each Implication? 17

18 S2: Functional intent Simple, helpful Much more detailed; suggests that it may be substantive M Name Quality: POE satisfaction wrt program (%) Comment Metrics Intent Target How to use in Tolerance: +- Source of data value management Guide Client commissioning, assessment next job Prediction Evaluation Predicted/ Stakeholders measured Type [P, Collection Assesed who saw data Objective Weight value (how O] frequency value last week you are doing) Turnover time O Owner only months E Cost conformance to plan (item actual - predicted/predicted) Plan next job Client assessment O PM only Turnover time 25 1 T Schedule conformance to plan (%) Plan next job Client assessment O All on team Turnover time R Predicted Cost conformance to plan (item actual - predicted/predicted) Attention management Periodic project progress report P Subteam only Weekly I Production schedule conformance to plan (%) Attention management Periodic project progress report P Owner only Weekly C Assessed Quality conformance to plan (% of items with rating >=4 on scale 1:5) Attention management Periodic project progress report P Subteam only Weekly S Stakeholder participation that is timely and meaningful (%) Adjust plan by stakeholder review and assessments Periodic project stakeholder survey P All on team Weekly Latency: % responses received <= 1 day Attention management Periodic project stakeholder survey P All on team Weekly Field RFIs (count) 0 3 Set weekly BIM specifications Periodic project progress report P All on team Weekly

19 S2: Functional intent Note: Structural elements: Target values Target cost? An app to use different kinds of credit- /transport-/club-/etc cards 3 months VDC: development process To visualize and and predict the outcome of the development process of I-wallet We are now in the design phase Estimated cost: $100,000 19

20 S2: Functional intent Simple, helpful link to product model Measurable target values? Much more detailed; suggests that it may be substantive M E Metrics Intent Prediction Evaluation Predicted/ Stakeholders measured Target How to use in Type [P, Collection Assesed Name Comment Tolerance: +- Source of data who saw data Objective Weight value (how value management O] frequency value last week you are doing) Guide Quality: POE satisfaction wrt Client Turnover time commissioning, O Owner only program (%) assessment months next job Cost conformance to plan (item Client Plan next job O PM only Turnover time 25 1 actual - predicted/predicted) assessment Client T Schedule conformance to plan (%) Plan next job O All on team Turnover time assessment R I Predicted Cost conformance to plan Attention Periodic project P Subteam only Weekly (item actual - predicted/predicted) management progress report Production schedule conformance Attention Periodic project P Owner only Weekly to plan (%) management progress report C Assessed Quality conformance to plan (% of items with rating >=4 on scale 1:5) Attention management Periodic project progress report P Subteam only Weekly S Stakeholder participation that is timely and meaningful (%) Adjust plan by stakeholder review and assessments Periodic project stakeholder survey P All on team Weekly Latency: % responses received <= 1 day Attention management Periodic project stakeholder survey P All on team Weekly Field RFIs (count) 0 3 Set weekly BIM specifications Periodic project progress report P All on team Weekly

21 S2: Functional intent Simple, helpful link of function to model Measurable functional description of target values? Relationships w/functions? 21

22 Product S2: Functional intent Metrics Function describes target; Behavior describes predicted behaviors: Number of parking spots that weren t empty (predicted): Target: OK to show here if identified as assumed. Function Features: An app to find empty parking spots, navigate to them with the help of the smartphone s GPS and pay for the parking Performance objectives: - Finds parking spots that really are empty in 95 % of the cases - Navigates to the closest empty parking spots in 95 % of the cases - Reliable app that does not crash more than 5 times per month and user Design Four different interfaces: - Main menu (Navigation menu) - A list of the closest empty parking spots - Navigator to the chosen parking spot - Payment interface The app should rely on stable and secure code. Behavior: Predictions - Number of parking spots that weren t empty, predicted: 9 out of Number of times the app didn t chose the closest parking spot: 3 out of Number of times the application crashed: 14 out of

23 S2: Functional intent Metrics Great example of function and model. Legend to explain color would help explain the feature that color indicates. Cross-reference arrows would help 23

24 Organization S2: Functional intent Metrics Function describes target; Behavior describes predicted behaviors: Good Add target/predicted SV behaviors to make story more compelling! Function Intent: The persons in the organization should be capable of deliver the app to the market by the time of the decided product launch. Performance objectives: - Deliver the application by July 6 th, Budget: SEK Elements: Management, Financials, Customer/Sales function, UI design, App development, Testing Design Roles: - Manager - Finance team - Customer/ Sales team - UI designer - App developer (programmer ) - Tester Behavior - Complete work before 6 th of July, 2014: Predicting to deliver on time - Spent money: predicting SEK Process predictions relate to outcome targets 24

25 S2: Functional intent What does this chart show? Measurable target values? Features (in model)? Relationships? 25

26 S2: Functional intent BIM (VDC project model) Content Specification BIM content template with P, O, P content What does this chart show? Value to narrative? How reference it in proposal narrative? Model LOD A A+ B All functions Share your workout sessions Create events BIM element function - primary Type of BIM content BIM elements BIM Element Level of detail (LOD) - AIA Level of detail LOD) - comments Due date of next BIM version BIM content conforms to spec [Yes/No] Follow your friends App features 200 (Approximate geometry) No Share your workout sessions App features 200 (Approximate geometry) No Create compititions Find routes App features 200 (Approximate geometry) No Find routes Get inspiriation App features 200 (Approximate geometry) No Rate routes Get feedback on your workouts App features 200 (Approximate geometry) No Collect bonus points Create events App features 200 (Approximate geometry) No Get inspiriation Collect bonus points App features 200 (Approximate geometry) No Follow your friends Rate routes App features 200 (Approximate geometry) No Other Divide repsonsibilities Actor (team) Manager 300 (Precise geometry) Makes sure everything gets donechecklists Makes sure everything gets done- checklists Product Organization Stay on budget Keeps good quality Actor (team) Manager 300 (Precise geometry) Stay on schedule Stay on schedule Actor (team) Manager 300 (Precise geometry) A Keeps good quality Divide repsonsibilities Actor (team) Manager 300 (Precise geometry) B A A+ Actor (team) Manager 300 (Precise geometry) Process Develope Task group 100 (Conceptual) Build Task group 100 (Conceptual) Design Task group 100 (Conceptual) alfa testing Milestone group 100 (Conceptual) Marketing Task group 100 (Conceptual) Sales strategy Task group 100 (Conceptual) beta testing Milestone group 100 (Conceptual) accoutant Task group 100 (Conceptual) establish partnerships Milestone group 100 (Conceptual) Comment Type of BIM content: Features/ Due date 7/19/2013 Type of BIM content: Features/ Due date 7/19/2013 Type of BIM content: Features/ Due date 7/19/2013 Type of BIM content: Features/ Due date 7/19/2013 Type of BIM content: Features/ Due date Type of BIM content: Features/ Due date 7/19/2013 Type of BIM content: Features/ Due date 7/19/2013 Type of BIM content: Features/ Due date 7/19/2013 BIM element function - primary: Stay on budget malfunction in bim element function B partnerships Task group 100 (Conceptual) 26

27 S2: Functional intent Organization Note model, intent from template and crossreferences What is a good title? 27

28 S2: POP summary Nice example of text, sketches 28

29 S2: POP summary Suggestion: annotate features and implications Function has two parts: Content > Form Performance (target) Function Form Behaviour Product Organization Process Delayed Texts -Reliable texts > 6sigma -No crashes < 10/month -Intuitive Interfaces, layout, design -Mockup Usability -Ease of texting Crashes -#Crashes Project finished in 1 month Quality Backlog < 2 days Crossfunctional team. PM, designer, programmer, user Finished when? Quality? Backlog -Spec. -Predesign -Design -Develop -Test Specification Pre-study Program Test Quality risk Rework time? In the 3x3 matrix above, we have constructed a POP model. We would classify it as an AB-LOD due to the number of entries. We based this matrix from the sketch on the previous page, and it has helped us focus as a team. 29

30 S2: POP summary Suggestions: annotate features and implications Nice example of text, sketches Specific Functional Target values Belong in Function 30

31 S2: POP summary Suggestions: annotate features and implications Nice example of text, sketches, annotation of issue 31

32 S2: Sketch Suggestion: annotate features and implications 32

33 S2: Sketch Note: simply annotated features Implications? 33

34 S2: Controllable factors template Target value goes in Function of POP Good Factor Type Factor Range of options Product Battery consumption <= similar applications Product Accurate calculations >99% Product Functions implemented from the specification >99% Product Accurate currency rate >99% Organizaton Number of members in core group 3-7 members Organizaton Working time on the project 8h/dag Process Project delivery on time 6-12 months 34

35 S2: Team charter template Implications for investor? Team name Core Team Design team Developer team Testing team Team Purpose Design and build application Design application Develop application algorithm and code Test functionality of application Team goals, Objectives Build application tht appeals to a lot of people Design interface to meet cutomers' expectations Create accurate and easy-to-use aplication Comprehensiv e testing of all functinality Team Vision of Success Finished, high quality application Appealing design High quality and effective code Bug-free application Team Charter Key Deliverables Application Application design Code for functionality Comprehensiv ely tested application Interfaces with other teams Consists of design team, developer team and testing team Connects to the core team Connects to the core team Connects to the core team Methods to coordinate / manage interfaces PM overlooks the overall process. Automated communication through e.g. Template files. checking things off the backlog. Automated communication through e.g. Template files. checking things off the backlog. Automated communication through e.g. Template files. checking things off the backlog. Automated communication through e.g. Template files. checking things off the backlog. Marketing team Communicate the product to the customers Obtain as many customers as possible High initial market share and fast increase of market share High number of customers Connects to the core team and Project manager Communication through meetings, possibility to view template files and backlog from developers, designers and testers. 35

36 S2: Sketch and model Suggestion: annotate features and implications 36

37 S2: Model Nice example of model and Legend to annotate important features Implications? FEATURES Meetings Actors Tasks Milestones RELATIONSHIPS Primary Assignment Re-work Successor Secondary Assignme Meeting Participants 37

38 S2: Model Nice example of model and annotation of important features Implications? Note: model 38

39 S2: SimVision model Suggestions: annotate features and implications Note that teams report to Manager Position2 Meeting1 Start Task1 Finish Task2 39

40 S2: SimVision model Nice example of sketch model with annotation 40

41 S2: SimVision model What does this figure show? Features and implications? Gantt Model 41

42 S2: Metrics: List of important features, performance objectives and predicted performance of these models; Features and implications? Relationship to rest of proposal? How summarize for reader? Metrics Intent Prediction Evaluation Name Comment Target value Tolerance: +- How to use in management Source of data Type [P, O] Stakeholders who saw data last week Collection frequency Objective Weight Predicted/ measured value (how you are doing) Assesed status Cost conformance to plan (item actual - predicted/predicted) Quality conformance Plan next job Attention management Client assessment O All on team Major milestones Data sources P All on team Weekly Schedule conformance to plan (%) Plan next job Client assessment O All on team Major milestones Predicted Cost conformance to plan (item actual - predicted/predicted) Production schedule conformance to plan (%) Backtrack Stakeholder participation that is timely and Attention management Attention management 2 1 Managing time Attention management Periodic project progress report Periodic project progress report Periodic project progress report Periodic project progress report P Subteam only Weekly P Subteam only Weekly P PM only Weekly P Subteam only Weekly Overall performance Conformance to plan Plan next job Client assessment O All on team Major milestones

43 S2: Model predictions Nice example of implications of breakdown chart Gantt? Features? 43

44 S2: Model predictions Suggestion: annotate features and implications Evaluation of performance re functional intent? 44

45 S2: Model predictions Suggestion: annotate features and implications Annotations helpful Evaluation of performance re functional intent! 45

46 S2: Model predictions Questions: Annotations help Method to mitigate this risk? 46

47 S2: Model predictions Notice easy summary of versions, predicted performance Proposed risk mitigation strategy given that no option is perfect. 47

48 S2: Model predictions Notice prediction vs. baseline and comments, which identify risks. Pls comment on your proposed risk mitigation strategy 48

49 S2: Versions Great use of the template! Note reference to other templates Features, implications Design version 0 (baseline) Design change from previous version Design process summary: version descriptions Rationale for design change Predicted impact of design change in comparison with previous version(s) Reference to Metrics template for this version Made the first base model creation of model - MetricsGraph!A1 1 Added the task market research before the designphase which is done by a marketer 2 Added probability to the model We have to work with other projects at the same time so we can not work full time on this app. A later finish date MetricsGraph!A1 The world is uncertan and thats why the model is more truthful with uncertanity A higher risk MetricsGraph!A1 49

50 S2: Versions Great use of the template! Features, implications Design version 0 (baseline) 1 2 Design change from previous version Tried to make the model as likely as possible a "real life" situtation Changed the probability that things might go wrong Added communication between parties Design process summary: version descriptions Rationale for design change We chose to do a pre-design because we think that would save time in the end. Make the model more "real". Predicted impact of design change in comparison with previous version(s) Worse Make the model more "real". Slightly improved times Reference to Metrics template for this version 3 Added supervision from PM Make the model more "real". More or less the same results. 4 Improved designer experience 5 Add design meeting 6 Add users 7 Set development priority to high. The design-part was a big part of the project, so it seemed likely that the designer should be well experienced. Big impact on result, greatly reduced times Seemed likely that the designer and developer would work in unison, and for that they would need Worse times, due to time meetings. spent doing other things. We wanted more people to test the app. Big impact on test time. The development of the app is a big part of total time, so the priority should be high. Not as big as we thought. 50

51 S2: Versions analysis of implications Good example of analysis of meaning of predictions Relationship of predicted to target values? Reference Design versions (7) 51

52 S2: Checklist for process management reliability Question: How to make it stronger and more actionable? 52

53 S2: Checklist for process management reliability Relationship to rest of specification, model, analysis and proposal? 53

54 S2: Checklist for process management reliability Note opportunity to show green, yellow, red status 54

55 S2: Checklist for process management reliability Note P, O, P content 55

56 S2: Deliverables (template) Relevant How to reference in narrative? Deliverables Planned deliverables Responsible team, individuals Receiving team Due date Due date met (Yes/No)? Business Model Team class 24/7/2013 B Versions of POP model in SimVision Team class 24/7/2013 B Clear project objectives Team class 24/7/2013 A Final Presentation Tem class 25/7/2013 A+ Expected LOD Comments Overview of what we want to achieve with our project, not too detailed to prevent confusion within and outside the team Presenting an overview and highlight and go into more LOD in specific features of our model 56

57 S2: Annotation Suggestion: annotate features and implications Clear that assignments are important. Anything else? Use of template? 57

58 S2: Risks Suggestion: annotate features and implications Implications for team and investor Identified risk Potential impact of risk ($, time, effort) Severity: Low, Medium, High Risks - Example Parties affected by risk Individuals responsible to mitigate design, give approval Last responsib le moment mitigaati Mitigation activity Risk of work overload $, time and effort Medium Operations Manager 7/11/2013 Allocating resources $ and Competitive Medium Risk of project not completing on time disadvantages All Manager 7/12/2013 Regular quality check-ups Risk of project not being on budget $ Medium Manager 7/14/2013 Allocating resources Resolution date met (yes/no)? 58

59 Assessment-1 Q2 59

60 Assessment-1 Q3 60

61 Assessment-1 Q4 61

62 Assessment-1 Q5 62

63 Assessment-1 Q8 63

64 Assessment-1 Range of scores 64

65 Overview Metrics Session Goals Metrics [Lecture/ discussion; Demo; interactive planning session] Understand and experience: Types of metrics, including project outcome objectives, process performance, and controllable factors How to track them Methods to use them in management Define metrics and represent them in a template. 65

66 Performance Metrics in AEC The big idea: Progress in AEC effectiveness and efficiency comes from achievement of measurable goals, which include controllable, process and outcome metrics. You may get what you ask for 66

67 Performance Metrics in AEC Count your work Make your work count Martin Fischer,

68 Management by Objectives (MBO) requires metrics A process of agreeing on goals and objectives within an organization so that management and employees agree to and understand what they are. The term "management by objectives" was first popularized by Peter Drucker in his 1954 book The Practice of Management Source: Wikipedia 69

69 Performance Metrics in AEC Agenda Motivations for metrics VDC Objectives Methods Call to action 70

70 CIFE Classic Plus-Delta of Civil Engineering Metrics Provides fixed physical assets and wealth High global demand for infrastructure and housing Opportunity to impact global climate challenge significantly 71

71 CIFE Classic Plus-Delta of Civil Engineering Metrics Provides fixed physical assets and wealth High global demand for infrastructure and housing Opportunity to impact global climate challenge significantly Low productivity compete with other ways to spend $ High energy use and rising energy costs US Department of Commerce, compiled by P. Teicholz Persson, Sustainable City of Tomorrow: B01 Experiences of a Swedish Housing exposition Structural reliability << societal need (Chile) 72 Guilllermo Gomez, PUChile

72 Issue: How to respond to the issues of productivity, energy, structural reliability? Incremental improvement incremental change Decades of evidence Breakthrough improvement significant change Business objective: Significantly improve project delivery performance for the client To achieve breakthrough: select & align Outcome, Process and controllable performance metrics 73

73 Managers can do only a few things. Opportunity Owner: increase value of facility investment Contractors: increase efficiency and profit Method: Set controllable factors Measure process performance Observe outcomes 74

74 Objectives: theoretical framework Metrics Control (Management): What we want; what we can control e.g., Product, Organization, Process (POP) functions, scope, behaviors to predict, measure and manage Process What we measure day by day, week by week Basis on which we manage and intervene e.g., Design, schedule & cost conformance Outcome What we report to client, senior management e.g., project safety, schedule, cost, quality 75

75 Objectives: theoretical framework Goals: purpose Example: Aesthetically pleasing intangible abstract cannot be validated general broad relate to outcome Objectives: target Example: Schedule conformance >= 80% tangible: touch them concrete can be validated precise marrow relate to outcome 76

76 Public metrics in AEC practice Mace retrofit project at Heathrow Obayashi construction, Tokyo Schuff Steel Plant, Phoenix, AZ Sutter Health, CA 77

77 Virtual Design and Construction (VDC) Use of multi-disciplinary performance models of design-construction projects, including Product (i.e., facilities) Organization of the design-construction-operation team Work Processes Economic Impact (i.e., model of both cost and value of capital investments) to support (explicit, public) business objectives. 78

78 Schedule Practice Breakthrough Performance Practice: 2002 Objective: y Design ~1.5 y Construct Variance 5-100% 1 y Design <.5 y Construct Variance < 5% Cost Variance 5-30% Variance < 5% Delivered quality Large Variance Good? Productivity impact? Safety Good Better 0 variance, by POE Great, by POE ++ productivity Sustainability Poor 25% better than 2002 Globalization Some >= 50% of supply and sales 79

79 (Multiple) Controllable factors Metrics VDC strategy and plan Control P, O, P elements to design, visualize and track; to integrate; to design automatically or prefabricate Model and manage 100% of POP items with > 10 (1)% of time, cost, effort or energy Target project design objectives: Set by target design process VDC scope: P, O, P elements to model and analyze; to integrate; to design automatically or prefabricate, to schedule and track Model and manage 100% of POP items with > 10 (1)% of time, cost, effort or energy Maturity phase: Visualization, Integration, Automation Number of trained VDC engineers: 4/project Public process performance metrics: Weekly report safety + ~10 process performance metrics Project goals that can vary by project 80

80 (Multiple) Controllable objectives Stakeholder engagement: public and explicit definition of who, when, for what purpose Coordination activities (requests + responses) 90% of all coordination activity planned (weekly), explicit, planned and publicly reported 90% of all planned coordination activity is reported (weekly) by intended recipients to have been timely and suitable Prediction basis: > 80% of all predictions by founded, automated methods Design versions: 2 or more >= 80% of all decisions that affect more than 10% (2%) of cost, effort or schedule Globalization strategy and plan: >= 50% of components and services from global suppliers Lifecycle cost factors considered: money; natural resources consumed; emissions 81

81 Example Controllable Factors template Suggestion: clearly identify some things to control from the many things that can be controlled. Factor Type Factor Range of options Constraints Action(s) for this week Product Detail of water line in 3D Features of size from 1 mm to > 1m None important Model features size > 20 mm Product Location of in-water equipment Adjust both equipment location, water width/depth profile Size of equipment model equipment located in water this week Organization Number of BIM authors <1 many FTEs Adjust author count up and down slowly Budget BIM author work Organization Number of BIM reviewers Author team hundreds Size of BIM review facility Budget BIM reviewer work Process Construction duration 6 months to 2 years Need to plan prefabrication early to shorten construction period Get owner preference Process Size of weekly pre-con tasks One task for whole team/week to Availability of staff to plan, Build short-interval production plan and all tasks with > 0.5 FTE-day manage schedule schedule at feasible LOD Process Tool used for BIM review BIM authoring, BIM review, Training of review team to Provide and train in best available tool Navisworks understand content supported within company Process Invitation list next meeting do not do it to invite core team + Need to do it at least 3 days all who might help before meeting Invite estimator, others x 10 by COB today Process Develop agenda, purpose, intended outcomes of next meeting Controllable Factors Agenda items: subheadings; purpose set by create agenda; coordinate with AV; test current project status; outcomes meeting date; attendee availability technology none to schedule and commitment clarity of assigned tasks x 5-10 Action Taken? (Yes, Partial, No) 82

82 Use controllable factors Whatever your role: A, E, C or O Collaborating with your other stakeholders At least every two weeks Identity factors you can and want to control for your Overall project Next project phase Next 2 weeks Controllable Factors VDC Strategy: POP elements to design, integrate, automate Target values: project behaviors to explore broadly and deeply VDC Scope: models to make; LOD Coordination activity to plan, track, review and manage Number of trained VDC engineers Public process performance metrics to track Design versions Focus of attention of available staff 83

83 Controllable factors include: VDC implementation tasks Metrics Tasks: Choose Models to build, analyses to make Modeling, analysis, data management tools Software Hardware Data storage and network data sharing Display Meeting space: Room size(s) and location(s) Technology, e.g., SmartBoards, white boards, tables, chairs, 84

84 Controllable factors include: Operating agreements for VDC meetings Intended meeting participation Acceptable diversions while meeting Expected preparation of participants Start/end times Frequency Agenda and management strategy Reviews of quality, cost and schedule conformance Expected use of models, analyses, metrics 85

85 Controllable factors include: Operating agreements example template ICE Sessions Meetings Decision Making Process Ground Rules Item Agreement Agreement details Frequency: 1-2/week Resources Needed Operations Agreement Time: 8:30-11 am Agenda (Attached?): Format: (Physical + Video Conference,..) Location: Room 203 Frequency: Fridays Time: 1-2 pm Agenda (Attached?): Format: Physical Location: Room 107 Decision process: Goal is entire team consensus; if team cannot reach one, core team majority rules; Conflict resolution process: if team cannot reach consensus core team majority rules; Veto possibility: client has veto rights. Commitments: PM can ask for commitment on any deliverable with effort > 1 FTE-week or high importance deliverable. Commitment is to a deliverable in a specified formatw with specified content at specied LOD. Responsible team must accept commitment for it to be valid. Responsible team can ask for enabling coordination and delivery commitments as a condition for accepting a commitment proposal. Times: start and end on time; All participants areee to focus on session, not cell phone or or other distractions All stakeholders share work calendars Pre-session preparation: Pre-meeting agenda sent to all invitees 3 days in advance. Invitation includes meeting goals and objectives, homework and relevant information to enable meaningful session participation. Post-session: session notes, relevant models and commitments on project web site by COB of meeting day Measurable process objectives: Cost, schedule, quality conformance all >= 80%, latency < 2 days -- all weekly BIM, project commitment log; meeting room for 15 w/smart boards x 3; Video conference tool, 3 flip charts with 4 colored pens each 86

86 Controllable factors include: methods to use metrics data Collect performance data: existing reporting systems (e.g., cost), data-acquisition systems (e.g., energy) online surveys (e.g., quality conformance), post-it notes (e.g., latency), Display: bar charts; spider diagrams; time plots Include measured data, objective (range), traffic light status Report: on the wall; meetings; web Interpret: responsible party, selected groups 87

87 Controllable factors include: methods, tools, and modeling, processes to implement Methods: specific kinds of models, analyses and metrics Tools: specific 3D, 4D, organization, scheduling, product modeling and analysis tools Processes: modeling, data exchange, data collection, reporting, management Assessment methods: public display adds many eyes; ICE sessions discuss and address Interventions when metrics suggest a problem: Root cause analysis Small or large ICE session to plan interventions 88

88 VDC impacts traditional management practices Changes Management risk identification and attention allocation become model-based Stakeholders need a shared vocabulary and methods Stakeholders need skill to make, analyze and engage with models VDC becomes a new process to manage Issues Physical and social environment that fosters multi-stakeholder interaction Use irooms for design and construction management! Believability, timeliness of project definition Model project early and often! Train; give slack to learn Consider Certificate Program Stakeholder(s) to model and analyze Adapt organization and process! 90

89 Method to Manage Using VDC in Practice Method: objectives Controllable factors: choose them! Process performance: measured ~ daily, weekly Safety Schedule, cost, quality conformance Response, decision latency Risk Field RFIs Rework volume Field material delivery Project outcome: seek breakthrough 91

90 (Multiple) Measurable Process objectives Metrics Safety: 0 lost-work incidents Conformance to schedule (PPC), cost, quality, target value, delivery, stakeholder participation objectives: [>=90-99% within 2% of plan] Schedule conformance = percent of promises/plan complete (PPC) Stakeholder participation: reported meaningful and timely participation of intended stakeholders in design/construction reviews Latency Response (decision-making no earlier than necessary): [minutes in design sessions; mean <= 1 working days; 95% < 2 days] Decision (decision-making promptness): [minutes in design sessions; mean <= 1 working days; 95% < 2 days] Project objectives that can vary by project 92

91 (Multiple) Measurable Process objectives Risk: 100% concurrence by all intended review stakeholders that sub-tasks were completed per specification for all tasks that affect life or corporate safety Field performance: Material delivery: 98% within 24 hours of use Field-generated Requests for Information: 0 (for questions related to issues that could have been identified at the award of the construction contract) Rework volume: 0 Timely, meaningful and relevant meeting participation: 90% as reported by all intended stakeholders Project objectives that can vary by project 93

92 Target value Example Metrics template Metrics Predicted value Intent Metrics Process metrics Target Tolerance: How to use in Name Source of data value +- management Guide Quality: POE satisfaction wrt M commissioning, next program (%) job Cost conformance to plan (item E actual - predicted/predicted) Type [P, O] Stakeholders who saw data last week Client assessment O Owner only Collection frequency Turnover time months Objective Weight Prediction Predicted/ measured value (how you are doing) Evaluation Assesed value Plan next job Client assessment O PM only Turnover time T Schedule conformance to plan (%) Plan next job Client assessment O Alll on team Turnover time 35? NA R Predicted Cost conformance to plan (item actual - predicted/predicted) Attention management Periodic project progress report P Subteam only Weekly I C S Production schedule conformance to plan (%) Assessed Quality conformance to plan (% of items with rating >=4 on scale 1:5) Stakeholder participation that is timely and meaningful (%) Latency: % responses received <= 1 day Field RFIs (count) 0 3 Fraction of prefab assemblies that take <=1 day to install (%) Personal preparation: Faction of participants that say they understood modeled design content and rationale (%) Team preparation: Fraction of participants saying that other team members came prepared to meeting (%) Attention management Attention management Adjust plan by stakeholder review and assessments Attention management Set weekly BIM specifications Set BIM weekly specifications, procurement Allows meeting to focus on work, not explanation Allows meeting to focus on work, not explanation Periodic project progress report Periodic project progress report Periodic project stakeholder survey Periodic project stakeholder survey Periodic project progress report Project team design review Periodic project stakeholder survey Project team design review P Alll on team Weekly P Subteam only Weekly P Alll on team Weekly P Alll on team Weekly P Alll on team Weekly P Alll on team Weekly P Alll on team Every session P Alll on team Every session

93 Range of targets from your submissions Note Min ~ Max 95

94 Range of targets from your submissions Note Min = Max Industry practice 96

95 Use process metrics Whatever your role: A, E, C or O Collaborating with your other stakeholders Daily or weekly, measure process performance compare predicted and measured process metric values with objectives Based on variances, update product, organization and process scope to improve performance Safety Process Conformance to schedule (PPC), cost, quality, delivery, stakeholder participation objectives Latency (promptness) Risk Field material delivery, Requests for Information, rework Meeting participation 97

96 One company - current practice on metrics Note that most serve reporting, not management NAME TYPE UNITS COLLECT- ION cost - hours spent per "activity"/ "project" WHO MEAS- URES P hours daily Accounting ROLE overhead tracking schedule - PPC P % weekly project team weekly, senior mgt monthly Cost - Budget compliance P man hours/$ ($ or % deviation) bi-weekly, PM profitability measure 98

97 Control specification/ Conformance: Model product, process breakdown structures, LOD and scope VDC/Lean process steps Commitment schedule level of detail Metrics matrix for one project Note that most serve reporting, not management Communication Behavior: Meeting Effectiveness Visualization use Field Interest in model and metrics content Metrics Schedule: Cost: Quality: Latency Conformance (PPC) Field Material Delivery Detailed Cost Conformanc e Cost: Rework Volume Field- Generated RFIs Client Satisfaction Safety 99

98 Frequent metrics surveys assess client satisfaction continuous tracking frequent feedback Multi-perspective metrics

99 Stakeholder latency over time for most critical issue of the week: Data suggest some stakeholders do not respond promptly Color code: Red = bad Yellow = risk Green = OK Many responses come slowly. Why? Legend for Latency Levels 1: 1-2 days (objective) 2: 3-4 days 3: 5-6 days 4: 7-8 days 5: >8 days Bubble size 1 response 2-3 responses Latency an issue Many responses come promptly. Who provides them? 101

100 Client satisfaction over time: Data suggest client concern over GC performance Some survey results indicate satisfaction Objective: 4-5 Latency an issue Client clearly developed an issue. Caused by emergence of latency as an issue? 102

101 Example metric chart: Notice clear target; detailed data sources 103

102 Example metric charts: Notice summary and detailed analysis Groups with greatest challenge 104

103 Process performance metrics for your project Metrics Process (~weekly) performance objective Field safety Schedule conformance (%) Cost conformance (%) Quality conformance (%) Latency (days) Timely & meaningful meeting participation (%) Risk Other Typical performance today Your project objective 105

104 Process performance metrics for your project Metrics Process (~weekly) performance objective Typical performance today Field safety Some number 0 incidents Schedule conformance (%) Cost conformance (%) Quality conformance (%) Unknown <= 30% outcome contingency Unknown Your project objective > 80% on 1-2 week lookaheads ± 5% variance of weekly cost estimate to budget ± 5% variance of weekly design quality review to explicit goals Latency (days) Unknown <= 1 day >= 95% all inquiries Timely & meaningful meeting participation (%) Unknown > 90% of intended meeting attendees 106

105 (Multiple) Project Outcome objectives Safety: 0 lost hours Schedule: Design within 1 year (SD, DD, CD), (90% conformance) Construct within 6 months (95% conformance) Cost: Construct within 2% of budget (95% conformance) LifeCycle predicted cost 25% better than baseline evaluated at turnover LifeCycle predicted cost 25% better than baseline evaluated at 5-years Quality: 100% satisfaction by POE Sustainability: 20% better than previous recent jobs Delivered Scope: 100% satisfaction by POE evaluation (all jobs) Globalization: >= 50% of supply chain from global suppliers Project objectives that can vary by project 107

106 Outcome performance metrics for your project Outcome performance goal Field safety Schedule Cost Quality Risk Energy Other Typical performance today Your project objective 108

107 Outcome performance metrics for your project Outcome performance goal Typical performance today Your project objective Field safety Some number 0 incidents Schedule 90% of jobs finish on time Cost Use 10-20% contingency Quality Punch list items all fixed On time finish Use < 2% contingency 100% scope OK by POE Risk Unknown 100% of risk checklist items OK Energy/employee/ year Other Unknown Some number 109

108 Target value Example Metrics template Metrics Measured value M E Name Quality: POE satisfaction wrt program (%) Cost conformance to plan (item actual - predicted/predicted) Target value Tolerance: How to use in management Guide commissioning, next job Intent Metrics Source of data Type [P, O] Stakeholders who saw data last week Client assessment O Owner only Collection frequency Turnover time months Objective Weight Prediction Predicted/ measured value (how you are doing) Evaluation Assesed value Plan next job Client assessment O PM only Turnover time T Schedule conformance to plan (%) Plan next job Client assessment O Alll on team Turnover time 35? NA R Predicted Cost conformance to plan (item actual - predicted/predicted) Attention management Periodic project progress report P Subteam only Weekly I C S Production schedule conformance to plan (%) Attention management Periodic project progress report Assessed Quality conformance to Attention Periodic project plan (% of items with rating >=4 on Outcome metrics management progress report scale 1:5) Adjust plan by Stakeholder participation that is Periodic project stakeholder review timely and meaningful (%) stakeholder survey and assessments Latency: % responses received <= Attention Periodic project day management stakeholder survey Field RFIs (count) 0 3 Fraction of prefab assemblies that take <=1 day to install (%) Personal preparation: Faction of participants that say they understood modeled design content and rationale (%) Team preparation: Fraction of participants saying that other team members came prepared to meeting (%) Set weekly BIM specifications Set BIM weekly specifications, procurement Allows meeting to focus on work, not explanation Allows meeting to focus on work, not explanation Periodic project progress report Project team design review Periodic project stakeholder survey Project team design review P Alll on team Weekly P Subteam only Weekly P Alll on team Weekly P Alll on team Weekly P Alll on team Weekly P Alll on team Weekly P Alll on team Every session P Alll on team Every session

109 Metrics template Note: Overall design goodness = assessed Value * Objective Weight (current version on web) Assessed performance template shows all metrics Assesed value - taken from Metrics Example M E Name Quality: POE satisfaction wrt program (%) Cost conformance to plan (item actual - predicted/predicted) Comment 2Target value Tolerance: How to use in management Assesed value Guide commissioning, next job 100 Assessed Quality conformance 5 Plan next job T Schedule conformance Home to plan (%) timely and meaningful (%) Plan next job Other (please enter) Fraction of prefab assemblies that take <=1 day to install (%) Team preparation: Fraction of participants saying that other team members came prepared to meeting (%) Personal preparation: Faction of participants that say they understood modeled design content and rationale (%) Field RFIs (count) Latency: % responses received <= 1 day Overall goodness Quality: POE satisfaction wrt program (%) Stakeholder participation that is Cost conformance to plan (item actual - predicted/predicted) 18, Intent Schedule conformance to plan (%) Predicted Cost conformance to plan (item actual - predicted/predicted) Production schedule conformance to plan (%) to plan (% of items with rating >=4 on scale 1:5) Compare option vs. others Comment: Graph shows assessed status of the predicted/measured performance of each metric defined in Metrics template Metrics Managerial significance of this assessed metrics status graph: Most metrics have very good predicted performance for this design version. Team should focus attention on high risk areas of cost and production schedule conformance. Source of data Client assessment Client assessment Client assessment Project team design review Type [P, O] O Stakeholders who saw data last week Owner only Collection frequency Turnover time months Prediction Objective Weight Predicted/ measured value (how you are doing) Evaluation Assesed value O PM only Turnover time 25 1 O All on team Turnover time P All on team Every session 14,

110 VDC impacts traditional management practices Changes Model-based predictions become relevant, believable and inexpensive deliverables Daily/weekly performance measurement becomes a new process to manage Stakeholder interactions reference VDC models Issues Model and model-based analysis timeliness and believability Deliver at least weekly! Culture that welcomes (or tolerates) failure reports Make performance public and explicit! Model Level of Detail (LOD) Work carefully to build, integrate and manage multi-discipline models! Use separated but consistent models Help project use corporate methods! 112

111 Use outcome performance Whatever your role: A, E, C or O Collaborating with your other stakeholders At every major milestone Compare measured outcome values with objectives Based on variances, update objectives for controllable factors process and outcome objectives Outcomes Safety Schedule Cost Delivered Scope Sustainability Globalization 113

112 VDC impacts traditional management practices Changes Ability to model and make model-based analyses becomes requirements for success Responsibility for success and failure become public, often early Cross-project improvement in modeling and analysis methods becomes a crucial business competence Issues Ability to put VDC competence on all projects Support professional development of staff and stakeholder partners Early visibility of risks requires flexible management and resources to mitigate them Consider consulting swat teams! Corporate focus required to integrate models, automate processes, help as needed Empower a CIO team! 114

113 (Multiple) Predictable performance objectives: We recommend projects collect all process metrics Controllable Product, organization, process designs Coordination activity: planned, explicit, public, informed > 90% Facility managed Scope: 100% of items with > 2% of value, time, cost or energy Prediction basis: > 80% of predictions founded Design versions: 2 or more >= 80% Staff trained in VDC: >= 4/project Process [Conformance to plans] Organization Latency: mean <= 1; 95% within 2 working days Process Field-generated Requests for Information: 0 Process Rework volume: 0 (for field construction work); objective = 10-20% (virtual work) Product/Organization/Process Function (quality) conformance (%): >= 99% Process schedule conformance (%): >= 80% Product/Organization Cost conformance (%): >= 95% Outcome [Performance] Safety: 0 lost hours Schedule: completion on time; construction <= 6 months Cost: >= 95% of budgeted items within 2% of budgeted cost Delivered Scope: 100% satisfaction *Sustainability: >75% better energy, water, materials, than 2002, profitably

114 VDC models support metrics Product, organization, process analyses support process and outcome performance: Product Costs Daylight Energy Interferences (3/4D) Quantities Rentable space Structure Organization Backlog Commitments Process Risk Schedule, cost, quality Meeting participation Safety Process performance Conformance of schedule, cost, quality, delivery, stakeholder participation to objectives Latency (promptness) Field-generated Requests for Information Rework volume Timely & meaningful meeting participation Outcomes Safety Schedule Cost Delivered Scope Sustainability Globalization 116

115 SMART metrics Letter Major Term Minor Terms S Specific Significant, Stretching, Simple M Measurable Motivational, Manageable, Meaningful A R T Attainable Relevant Timely Appropriate, Achievable, Agreed, Assignable, Actionable, Adjustable, Ambitious, Aligned, Aspirational, Acceptable, Action-focused Result-Based, Results-oriented, Resourced, Resonant, Realistic Time-oriented, Time framed, Timed, Timebased, Time-bound, Time-Specific, Timetabled, Time limited, Trackable, Tangible Source: wikipedia; originally in Doran, G. T. (1981). There's a S.M.A.R.T. way to write management's goals and objectives. Management Review, Volume 70, Issue 11(AMA FORUM), pp

116 Suggestions on process metrics Use conformance to keep number of metrics small, e.g., Quality: 100% Conformance of designed spaces to stated architectural functional objectives Schedule: 90% of tasks start and finish within 1 day of operating schedule Cost: 98% of budgeted items have actual cost within 2% of estimates Stakeholder participation: 98% of all intended stakeholders report timely and meaningful participation in meetings Track performance Root cause problems reduce variance improve mean Show performance and objective on graphs w/evaluation 118

117 Suggestions on process and outcome metrics Public & explicit Display objective & data graphically Show performance over time Show multiple relevant metrics Display and discuss at least weekly Annotate evaluation Metrics 120

118 Suggestion - choose units carefully: cost vs. listcycle cost vs. value Cost [KwH / year - m 2 ] HKDL Performs better by cost WDW 2006/7 2006/7 Data Courtesy, WDI 121

119 Suggestion - choose units carefully: cost vs. listcycle cost vs. value Cost [KwH / year - m 2 ] Value [KwH / year - transaction - m 2 ] HKDL Performs better by value WDW 2006/7 2006/7 Data Courtesy, WDI 122

120 Metrics Suggestion - choose units carefully: cost vs. listcycle cost vs. value First cost (initial investment, design and construction) Lifecycle cost (LCC)/Period -- 20, 30, or whatever # of years Lifecycle cost per value earned (e.g., per person-hour, $ of profit, patient treated, ) Cost ($ or hours) First cost/sf LCC/sf LCC/value metric Energy (kbtu) Kwh construction/sf Kwh/sf Kwh/value metric CO2 (tons of CO2 equivalent) tco2 embodied (production + transportation) tco2/sf tco2/value metric Quality (% conformance to explicitly stated design intent) Cost of design and construction services plus interest Productivity cost of workers who must compensate for quality deficit Productivity cost of workers who must compensate for quality deficit/value earned 123

121 On a m2 basis the blue building performs poorly, i.e., it uses about 50% more energy than the green building. Figure Courtesy, WDI

122 On a transaction per m2 basis the green building performs poorly, i.e., it uses about 80% more energy than the green building. Figure Courtesy, WDI

123 Metrics Production Reliability: Notice variance reduction; improved mean 126

124 Suggestions on process and outcome metrics Focus on a small manageable set (>= 10) Display and interpret them publicly and explicitly Show objective and data Add traffic light to show status Add comments to explain why yellow or red status Continually reassess how metrics can and do support management 127

125 Comments from VDC Value Survey: Projects often do not track and manage using quantitative performance metrics today Quantitative impacts remain largely invisible VDC seems to sell itself Quantitative data response low 90% do not quantitatively track impacts Most respondents do not know cost performance!! Half of participants do not know performance!! 128

126 How to use business objectives Metrics Understand the options Select ~5 each for focusistrategy Controllable: (VDC Scope, stakeholders) Process: (schedule, cost, quality, delivery, stakeholder participation ) Project: (scope, cost, quality) Report status explicitly, publicly Value(s), trend line, variance Objective Status wrt objective (traffic light) Root causes Set explicit, public objectives: 1. Reduce variance 2. Improve mean Manage process, project to improve selected metrics: Make controllable change Plan, track, manage plan vs. actuals VDC strategy and plan Coordination activity Design versions: Risk management strategy Globalization strategy and plan Cost estimates Schedule Cost Field material delivery timeliness latency Field-generated RFIs Rework volume: Meetings Safety Schedule Cost, Scope Sustainability Globalization 129

127 Meeting new Performance Objectives Integrated business approach Culture: public commitment; public performance reporting Technology: predict, track Strategy: unique value People: skills, incentives Organization: structure Process: planning, execution, objectives Technology Culture Strategy Process People Organization 130

128 Metrics improve Management practices Management Practice & Productivity: Why they matter, Nick Bloom et al., McKinsie 131

129 Management: Company performance follows management quality in 4,000 companies [Bloom] Nick Bloom, Management Practice & Productivity: Why they matter. 132