Increasing Performance through Extremely Cool Prioritization and Project Selection June 26, 2017 Presented by: Tamara Haas, PE Director, Asset Management & Planning New Mexico DOT Mara Campbell, CQM Chief Technologist, Performance/Asset Management CH2M
Cross-Asset Allocation to Improve DOT Return on Investment MODA Project Prioritization and Selection Performance-Based Programming Develop a ranking for a set of projects based on DOT value for $ spent Test system performance with different levels of program funding Long-Range Planning Project Right-Sizing 2 Plan and schedule projects to meet multi-year budgetconstraints Conduct business case evaluation of project concepts to maximize value for $ spent
3 Cross-Asset Resource Allocation and the Impact on System Performance
Benefits Why do it? Accountability and transparency Tracks performance and investments Matches with outputs from internal management systems 4
Our Approach to Project Prioritization and Selection 5
Multi-Objective Decision Analysis (MODA) Prioritization & Selection Approach Select Projects that Provide the Most Value for the Dollars Spent Goals Safety Maintain the System Mobility Economic Vitality Other Considerations 1. Establish Criteria Reduce Fatalities Reduce Injuries 2. Develop Projects 3. Develop Performance Measures Project 1 4. Weighting W Fatalities W Injuries Project 2 5. Normalize, Calculate Value Scores, Prioritize using Value/Cost Calculate Project 3 6 6. Optimize Overall measure of performance Optimize
Goals and Criteria What are your agency goals? Start with LRTP and STIP Confirm goals with executive leadership What programs would a prioritization and project selection framework apply to? What measurable criteria support those goals? Develop crosswalks from all relevant documents to those goals and criteria Vet criteria widely within organization 7
Prioritization Performance Measures Performance metrics a starting point for performance measures Could be different indicators for organizational performance versus project prioritization Establish a library of possible criteria and measures Assess available data Develop a starting point list of criteria and performance measures Use quantitative and qualitative measures Quantitative = direct measurement such as AADT, IRI Qualitative = a constructed scale (e.g., 1-10) with definitions 8
Example Criteria and Performance Measures 9
Two Main Methods of Establishing Weights Pairwise Comparison Analytic Hierarchy Process (AHP) Swing Weighting Multi-Objective Decision Analysis (MODA) Weights are personal, and subjective 10
MODA Prioritization & Selection Approach Select Projects that Provide the Most Value for the Dollars Spent Goals Safety Maintain the System Mobility Economic Vitality Other Considerations 1. Establish Criteria Reduce Fatalities Reduce Injuries 2. Develop Projects 3. Develop Performance Measures Project 1 4. Weighting W Fatalities W Injuries Project 2 5. Normalize, Calculate Value Scores, Prioritize using Value/Cost Calculate Project 3 11 6. Optimize Overall measure of performance Optimize
How it worked at New Mexico DOT
NMDOT Highway Safety Improvement Program -- Pilot Process Used a group of projects from NMDOT HSIP to pilot the Multi Objective Decision Making (MODA) process How it happened 3 Key Workshops Kickoff - Framing Criteria and Prioritization Performance Measures Weighting and Evaluation Workshop Final Optimization and Report Here s how it works.. 13
Setting the Tool Up STEP 1: Setup Project Name: NMDOT HSIP Project Prioritization Protected Project Purpose: (One or two sentences: 50 word maximum) Prioritization excercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects Define Criteria Type Check this box if model includes criteria only (i.e., no sub-criteria) Preferred Weighting Method Swing Weighting Pairwise Comparison (AHP) Hierarchical Non-Hierarchical Basis for Prioritization Value / Cost Cost/Non-Cost Weight 14
STEP 2: Enter Criteria and Sub-Criteria STEP NMDOT 2: Enter HSIP Project Criteria Prioritization and Sub-Criteria Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects NMDOT HSIP Project Prioritization Prioritization excercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects ID# 1.0 1. Safety 1.1 1A. Fatality and Serious Injury Crash Reduction 1.2 1B. Bike/Ped Crash Reduction 1.3 Sub3 1.4 Sub4 1.5 Sub5 1.6 Sub6 2.0 2. System Preservation 2.1 2A. Pavement Lane-Miles Preserved or Improved 2.2 2B. Bridge Functional Obsolescence Improvement 2.3 2C. Dollars Spent on Improving Condition of Other Assets 3.0 3. Mobility 3.1 3A. Multimodal Improvement 3.2 3B. Reliability 3.3 3C. ITS Architecture Inclusion 4.0 4. Economic Development 4.1 4A. Location-Based Factors 5.0 5. Other Considerations 5.1 5A. Project Readiness 15 5.2 5B. Consistency with Plans 5.3 5C. Freight considerations Criteria Name
STEP 3: Define Prioritization Performance Measures and Scales STEP NMDOT 3: Define HSIP Project Prioritization Performance Measures and Scales Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects NMDOT HSIP Project Prioritization Prioritization excercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects ID# Prioritization Performance Measure Scale Endpoints (from Data) Best Worst Feasible Feasible Outcome Outcome Evaluation Criteria Direction 1.0 1. Safety 1.1 1A. Fatality and Serious Injury Crash Reduction (1-CMF) x 100 Low to High 0 100 1.2 1B. Bike/Ped Crash Reduction Neg/No/Yes Low to High (1) 1 2.0 2. System Preservation Low to High 2.1 2A. Pavement Lane-Miles Preserved or Improved Lane-Miles Low to High 0 8 2.2 2B. Bridge Functional Obsolescence Improvement Yes/No Low to High 0 0 2.3 2C. Dollars Spent on Improving Condition of Other Assets Dollars Low to High $0 $3,500,000 3.0 3. Mobility Low to High 3.1 3A. Multimodal Improvement -1-5 Scale Low to High (1) 5 3.2 3B. Reliability 0-5 Scale Low to High 0 5 3.3 3C. ITS Architecture Inclusion Yes/No Low to High 0 1 4.0 4. Economic Development 0-7 Scale Low to High 1 5 5.0 5. Other Considerations Low to High 5.1 5A. Project Readiness 0-5 Scale Low to High 1 5 16 5.2 5B. Consistency with Plans 0-5 Scale Low to High 0 5 5.3 5C. Freight considerations 0-5 Scale Low to High 0 5
STEP 4: Value Functions NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 17
STEP 5: Participant Weighting Form NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 18
STEP 6: Criteria Weights NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 19
STEP 7: Individual Providing Weights NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 20
Percent Weights NMDOT HSIP Project Prioritization Prioritization exercise for NMDOT Highway Safety Improvement Program (HSIP) 2017 and 2018 Projects 21
Step 8: Project Inputs NMDOT HSIP Project Prioritization 22
Project Analysis NMDOT HSIP Project Prioritization 23
Scenario Summary NMDOT HSIP Project Prioritization 24
MODA Value vs. Cost 25
Cumulative Value Efficient Frontier Cumulative Cost 26
MODA Score Construction Cost ($million) Component Value and Cost 90 80 70 60 50 40 30 20 10 0 $7,000,000 $6,000,000 $5,000,000 $4,000,000 $3,000,000 $2,000,000 $1,000,000 $- 27 1. Safety 2. System Preservation 3. Mobility 4. Economic Development 5. Other Considerations ProjectCost
Prioritization and Project Selection Example Sensitivity to Changes in Weights
Optimizing a Ranked Project List Ranked project list An end and a beginning Check constraints and opportunities Schedule Interdependencies (partners) Project sizing Equity Portfolio analysis Optimize against constraints
Helpful and Useful! This approach improved linkage between spending and agency goals Adjusted it for New Mexico DOT Project Timing Color of Money Politics will NOT go away; this approach gives a basis for departure Seeking insight NOT black box solutions Iterate, learn, and improve Provides good documentation about basis for funding Got staff talking and created more collaboration
Thank YOU! Tammy Haas, PE TamaraP.Haas@state.nm.us Mara Campbell, CQM Mara.Campbell@ch2m.com 31