Increasing Performance through Extremely Cool Prioritization and Project Selection

Transcription

1 Increasing Performance through Extremely Cool Prioritization and Project Selection June 26, 2017 Presented by: Tamara Haas, PE Director, Asset Management & Planning New Mexico DOT Mara Campbell, CQM Chief Technologist, Performance/Asset Management CH2M

2 Cross-Asset Allocation to Improve DOT Return on Investment MODA Project Prioritization and Selection Performance-Based Programming Develop a ranking for a set of projects based on DOT value for $ spent Test system performance with different levels of program funding Long-Range Planning Project Right-Sizing 2 Plan and schedule projects to meet multi-year budgetconstraints Conduct business case evaluation of project concepts to maximize value for $ spent

3 3 Cross-Asset Resource Allocation and the Impact on System Performance

4 Benefits Why do it? Accountability and transparency Tracks performance and investments Matches with outputs from internal management systems 4

5 Our Approach to Project Prioritization and Selection 5

6 Multi-Objective Decision Analysis (MODA) Prioritization & Selection Approach Select Projects that Provide the Most Value for the Dollars Spent Goals Safety Maintain the System Mobility Economic Vitality Other Considerations 1. Establish Criteria Reduce Fatalities Reduce Injuries 2. Develop Projects 3. Develop Performance Measures Project 1 4. Weighting W Fatalities W Injuries Project 2 5. Normalize, Calculate Value Scores, Prioritize using Value/Cost Calculate Project Optimize Overall measure of performance Optimize

7 Goals and Criteria What are your agency goals? Start with LRTP and STIP Confirm goals with executive leadership What programs would a prioritization and project selection framework apply to? What measurable criteria support those goals? Develop crosswalks from all relevant documents to those goals and criteria Vet criteria widely within organization 7

8 Prioritization Performance Measures Performance metrics a starting point for performance measures Could be different indicators for organizational performance versus project prioritization Establish a library of possible criteria and measures Assess available data Develop a starting point list of criteria and performance measures Use quantitative and qualitative measures Quantitative = direct measurement such as AADT, IRI Qualitative = a constructed scale (e.g., 1-10) with definitions 8

9 Example Criteria and Performance Measures 9

10 Two Main Methods of Establishing Weights Pairwise Comparison Analytic Hierarchy Process (AHP) Swing Weighting Multi-Objective Decision Analysis (MODA) Weights are personal, and subjective 10

11 MODA Prioritization & Selection Approach Select Projects that Provide the Most Value for the Dollars Spent Goals Safety Maintain the System Mobility Economic Vitality Other Considerations 1. Establish Criteria Reduce Fatalities Reduce Injuries 2. Develop Projects 3. Develop Performance Measures Project 1 4. Weighting W Fatalities W Injuries Project 2 5. Normalize, Calculate Value Scores, Prioritize using Value/Cost Calculate Project Optimize Overall measure of performance Optimize

12 How it worked at New Mexico DOT

13 NMDOT Highway Safety Improvement Program -- Pilot Process Used a group of projects from NMDOT HSIP to pilot the Multi Objective Decision Making (MODA) process How it happened 3 Key Workshops Kickoff - Framing Criteria and Prioritization Performance Measures Weighting and Evaluation Workshop Final Optimization and Report Here s how it works.. 13

14 Setting the Tool Up STEP 1: Setup Project Name: NMDOT HSIP Project Prioritization Protected Project Purpose: (One or two sentences: 50 word maximum) Prioritization excercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects Define Criteria Type Check this box if model includes criteria only (i.e., no sub-criteria) Preferred Weighting Method Swing Weighting Pairwise Comparison (AHP) Hierarchical Non-Hierarchical Basis for Prioritization Value / Cost Cost/Non-Cost Weight 14

15 STEP 2: Enter Criteria and Sub-Criteria STEP NMDOT 2: Enter HSIP Project Criteria Prioritization and Sub-Criteria Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects NMDOT HSIP Project Prioritization Prioritization excercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects ID# Safety 1.1 1A. Fatality and Serious Injury Crash Reduction 1.2 1B. Bike/Ped Crash Reduction 1.3 Sub3 1.4 Sub4 1.5 Sub5 1.6 Sub System Preservation 2.1 2A. Pavement Lane-Miles Preserved or Improved 2.2 2B. Bridge Functional Obsolescence Improvement 2.3 2C. Dollars Spent on Improving Condition of Other Assets Mobility 3.1 3A. Multimodal Improvement 3.2 3B. Reliability 3.3 3C. ITS Architecture Inclusion Economic Development 4.1 4A. Location-Based Factors Other Considerations 5.1 5A. Project Readiness B. Consistency with Plans 5.3 5C. Freight considerations Criteria Name

16 STEP 3: Define Prioritization Performance Measures and Scales STEP NMDOT 3: Define HSIP Project Prioritization Performance Measures and Scales Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects NMDOT HSIP Project Prioritization Prioritization excercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects ID# Prioritization Performance Measure Scale Endpoints (from Data) Best Worst Feasible Feasible Outcome Outcome Evaluation Criteria Direction Safety 1.1 1A. Fatality and Serious Injury Crash Reduction (1-CMF) x 100 Low to High B. Bike/Ped Crash Reduction Neg/No/Yes Low to High (1) System Preservation Low to High 2.1 2A. Pavement Lane-Miles Preserved or Improved Lane-Miles Low to High B. Bridge Functional Obsolescence Improvement Yes/No Low to High C. Dollars Spent on Improving Condition of Other Assets Dollars Low to High $0 $3,500, Mobility Low to High 3.1 3A. Multimodal Improvement -1-5 Scale Low to High (1) B. Reliability 0-5 Scale Low to High C. ITS Architecture Inclusion Yes/No Low to High Economic Development 0-7 Scale Low to High Other Considerations Low to High 5.1 5A. Project Readiness 0-5 Scale Low to High B. Consistency with Plans 0-5 Scale Low to High C. Freight considerations 0-5 Scale Low to High 0 5

17 STEP 4: Value Functions NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 17

18 STEP 5: Participant Weighting Form NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 18

19 STEP 6: Criteria Weights NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 19

20 STEP 7: Individual Providing Weights NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 20

21 Percent Weights NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 21

22 Step 8: Project Inputs NMDOT HSIP Project Prioritization 22

23 Project Analysis NMDOT HSIP Project Prioritization 23

24 Scenario Summary NMDOT HSIP Project Prioritization 24

25 MODA Value vs. Cost 25

26 Cumulative Value Efficient Frontier Cumulative Cost 26

27 MODA Score Construction Cost ($million) Component Value and Cost $7,000,000 $6,000,000 $5,000,000 $4,000,000 $3,000,000 $2,000,000 $1,000,000 $ Safety 2. System Preservation 3. Mobility 4. Economic Development 5. Other Considerations ProjectCost

28 Prioritization and Project Selection Example Sensitivity to Changes in Weights

29 Optimizing a Ranked Project List Ranked project list An end and a beginning Check constraints and opportunities Schedule Interdependencies (partners) Project sizing Equity Portfolio analysis Optimize against constraints

30 Helpful and Useful! This approach improved linkage between spending and agency goals Adjusted it for New Mexico DOT Project Timing Color of Money Politics will NOT go away; this approach gives a basis for departure Seeking insight NOT black box solutions Iterate, learn, and improve Provides good documentation about basis for funding Got staff talking and created more collaboration

31 Thank YOU! Tammy Haas, PE Mara Campbell, CQM 31