NCHRP Report 666 (NCHRP 8-70) For: TRB Webinar February 23, 2011
GoToWebinar Interface Click to edit Master title style 1. Viewer Window 2. Webinar Control Panel Our webinar presentation slides appear here on your computer screen TYPE HERE
Commonly asked question: Can I receive Click a copy to edit of the Master presentation title styleslides? Please view your webinar reminder email for a link to the presenter s slides. After the webinar, you will receive a link to the recording of today s session.
Click PDH Certificate to edit Master Information title style This webinar is valued at 1 Professional Development Hours. You must register and attend the webinar as an individual. For groups, only the person that registers and attends the session is eligible to receive a certificate. Email Reggie Gillum: RGillum@nas.edu
The Transportation Research Board has met the standards and requirements of the Registered Continuing Education Providers Program. Credit earned on completion of this program will be reported to RCEPP. A certificate of completion will be issued to each participant. As such, it does not include content that may be deemed or construed to be an approval or endorsement by NCEES or RCEPP.
Click Cambridge to edit Systematics Master title Presentation style Team Randy Halvorson: 952-920-8141, rhalvorson@camsys.com Erik Cempel: 312-665-0223, ecempel@camsys.com Anita Vandervalk: 850-219-6388, avandervalk@camsys.com Kim Hajek: 512-691-8509, khajek@camsys.com Kelsey Ahern: 512-691-8506, kahern@camsys.com 6
Click Agenda to edit Master title style 2:00 2:10 pm EST Purpose Randy Halvorson Research Approach Randy Halvorson 2:10 2:45 pm EST Guide to Target Setting Randy Halvorson/Erik Cempel 2:45 3:25 pm EST Guide to Data Management Anita Vandervalk/Kim Hajek 3:25 3:30 pm EST Concluding Remarks and Additional Research Underway Randy Halvorson *Question and Answer Moderator Kelsey Ahern 7
Click Purpose to edit of Research Master title (Targets) style Describe Framework for Performance-Based Resource Allocation (PBRA) Identify the role of performance measure targets within PBRA Outline the steps for setting performance targets 8
Click Purpose to edit of Research Master title (Data) style To provide a comprehensive description of the following components, which are needed to support (PBRA) decisionmaking Data management Information systems Institutional arrangements 9
Click Project to Approach edit Master title style Interviewed 25+ public and private sector agencies on the use of performance measures and targets and their supporting data systems Results compiled and documented in case studies Lessons learned and best practices published in Guide for Target-Setting and Data Management 10
Quality Data Click Performance to edit Master Management title style Framework PBRA takes place within an overall Management Framework Goals/Objectives Performance Measures Target Setting Evaluate Programs and Projects Allocate Resources Budget and Staff Measure and Report Results Actual Performance Achieved 11
Interrelationship Among Measures, Targets, Click Data to edit Master title style Performance measures and targets are integral elements in PBRA Measures and targets are supported by data and information systems Data management programs are used to organize data support functions 12
Click Performance to edit Master Measure title Target style Definition: A specific value for a measure that an agency would like to achieve. These values set a performance level for each organization measure. 14
Click Performance to edit Master Measure title Target style Example: Goal Area System Preservation Performance Measure International Roughness Index (IRI) Target 75% pavement good or better condition 15
Click Steps to For edit Target Master Setting title style Establish a Performance Management Framework Providing the context Evaluate the Factors Influencing Target Setting Making the choice Part 1 Select the Appropriate Approach to Making the choice Part 2 Target Setting 16
Click Steps to For edit Target Master Setting title style Establish Methods for Achieving Targets Meeting the targets/making trade-offs Track Progress Towards Targets Tracking investment impacts Adjust Targets Over Time Refining the process 17
Quality Data Click Steps to For edit Target Master Setting title style Establish a Performance Management Framework Providing the context Goals/Objectives Performance Measures Target Setting Evaluate Programs and Projects Allocate Resources Budget and Staff Measure and Report Results Actual Performance Achieved 18
Click Steps to For edit Target Master Setting title style Evaluate the Factors Influencing Target Setting Making the choice Part 1 Agency experience Data availability Financial resources Stakeholder and public involvement Political/legislative influence Timeframe 19
Click Steps to For edit Target Master Setting title style Select the Appropriate Approach to Target Setting Making the choice Part 2 Edict Expert Opinion Benchmarking Modeling Hybrid/Collaboration Preferred Approach 20
Click Steps to For edit Target Master Setting title style Select the Appropriate Approach to Target Setting Cont d Hybrid/Collaboration - Preferred approach for final selection Target definition specific value for a measure that an agency would like to achieve Data- based classification of performance condition develop a performance threshold table Link performance condition to investment levels Select specific target levels in collaboration with others 21
Example Application of Target Setting to Click Pavement to edit Master title style Goal Area System Preservation Performance Measure International Roughness Index (IRI) Target 75% pavement good or better condition 22
Click Pavement to edit target Master selection title style process (Step 3A) Establish a data-based classification of performance condition (Performance Threshold Table) 23
Pavement condition target selection process Click (Step to 3B) edit Master title style Establish a relationship between performance measure and annual funding levels Source: Highway Economic Requirements System- 2008 FHWA C & P 24
System Highway Miles (%) Click Pavement to edit condition Master title target style selection (step 3C) Develop a chart showing annual pavement condition 100 90 80 70 60 50 40 30 20 10 0 Target % "Good 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Year 25
Click Pavement to edit condition Master title target style selection (Step 3D) Select specific target using collaborative process Senior Agency Staff (i.e. Commissioner/Secretary and selected Division Heads) Agency performance area experts (i.e. state and district pavement engineers) Stakeholders (i.e. selected state and local elected officials, federal officials, freight haulers and the driving public) 26
Click Steps to For edit Target Master Setting title style Establish Methods for Achieving Targets Meeting the targets/making trade-offs Integration into daily routines Incentives Personnel performance appraisals Quality control and support Trade-off analysis 27
Click Steps to For edit Target Master Setting title style Establish Methods for Achieving Targets Cont d Trade-off analysis Define measures of effectiveness Assess current performance Develop funding scenarios Format results for comparison 28
Click Steps to For edit Target Master Setting title style Track Progress Towards Targets Tracking investment impacts Annual Attainment Reports Annual Snapshot 29
Click Steps to For edit Target Master Setting title style Adjust Targets Over Time Refining the process Funding adjustments Policy changes Trade-offs over time 30
Mn/DOT as Process Example Click for Implementing to edit Master Targets title style within PBRA Highway Investment Plan Development Process Step 1 Step 2 Step 3 Step 4 Step 5 Identify Investment Needs Project Future Revenues Set Investment Goals Develop Investment Plan Prioritize Unfunded Investment Needs Investments to Meet Performance Targets Regional and Community Improvement Priorites Revenue Outlook Investment Goals: Balanced Program 20-year Highway Investment Plan STIP 2009-2012 Mid-Range HIP 2013-2018 Long-Range 2019-2028 Total Unfunded Statewide High Priority Legislative Direction (Chapter 152) System Performance Stakeholder Input 31
Target Setting to Support Performance-Based Click Resource to edit Allocation Master title style 32
Click Data to Management edit Master Programs title style Data Management Programs are used to: Strengthen ability of data programs to support core business functions Improve data quality throughout the organization Protect data as an asset of the agency Limit risks associated with loss of data and information 34
Click Data to Management edit Master Programs title style Strengthen ability of data programs to support business functions -Alaska 35
Click Data to Management edit Master Programs title style Link Business Functions to Data Programs - Virginia 36
Steps For Implementing/Strengthening Data Click Management to edit Master Programs title style Establish Need for Data Management/ Data Governance Establish Goals for Data Management Assess Current State of Data Programs 37
Steps For Implementing/Strengthening Data Click Management to edit Master Programs title style Establish Data Governance Programs Use Technology to Support Data Management Link Data to Planning, Performance Measures, & Target-Setting Process 38
Step 1: Establish Need for Data Click Management/Data to edit Master Governance title style Establish need for data management/data governance Demonstrate Return on Investment (ROI) Support Performance Measurement and Target Setting Governance supports data management from policy, practical, technical perspectives Data Management Maturity Model assesses progression from un-governed to a governed state A strong Data Management program improves data quality and limits potential risks to the agency regarding the loss of critical data and information 39
Step 1: Establish Need for Data Click Management/Data to edit Master Governance title style Relationship of Data Management, Governance, Stewardship Source: Modified from Figure 1 Data Governance Team, The Data Governance Maturity Model, White Paper, RCG Information Technology, 2008. 40
Step 1: Establish Need for Data Click to edit Master title style Management/Data Governance Figure 1.2 Data Governance Data Governance Maturity Model Maturity Model High Think Locally, Act Locally Think Globally, Act Locally Think Globally, Act Collectively Think Globally, Act Globally Low Reward Risk Low High Undisciplined Reactive Proactive Governed People, Policies, Technology Adoption Source: Data Governance Maturity Model, DataFlux Corporation, 2007. 41
Step 1: Establish Need for Data Click Management/Data to edit Master Governance title style Maturity Model Matrix 42
Step 1: Establish Need for Data Click Management/Data to edit Master Governance title style Example Washington State DOT Data Council - set standards for data architecture, data modeling, etc. Data Stewardship Council address data issues across the department Two types of data stewardship Business stewardship executives, managers, business operations Technical stewardship system architects, database administrators Partnership of business and technical stewards strengthens data governance 43
Step 2: Establish Goals for Data Management Click to edit Master title style Establish goals for data management Use brainstorming sessions with data program stakeholders Identify business objectives of the agency Identify business functions/services that support business objectives Identify which business functions are supported by which data programs Establish data collection and use policies Establish data action plans to address needs and gaps in data and information Establish risk management program to protect valuable data program assets 44
Step 2: Establish Goals for Data Management Click to edit Master title style Success factors Start small with achievable goals Use a data business plan Manage expectations Use Business-Use Case Models Identify champions 45
Step 2: Establish Goals for Data Management Click Example to edit - MNDOT Master title style Mn/DOT Data Business Plan Framework MISSION Provide the highest quality, dependable, multi-modal transportation system through ingenuity, integrity, alliance, and accountability. STRATEGIC DIRECTIONS POLICIES SELECTED EMPHASIS AREAS & OBJECTIVES 1. Traveler Safety 2. Infrastructure Preservation 6. & 7. Mobility BUSINESS PROCESSES Plan Produce Operate / Maintain Support 46
Step 3: Assess Current State of Data Programs Click to edit Master title style Assess current state of data programs Health Assessment Highlight critical deficiencies and develop strategies to address the deficiencies Surveys Focus Groups Workshops Research Studies Risk Management identifies and prioritizes the risks associated with current data programs Risk prioritization matrix 47
Step 3: Assess Current State of Data Programs Click to edit Master title style Risk prioritization matrix Data Program Program A Program B Program C Value Ranking (Essential, Helpful, Not Needed) Addresses Key Performance Measures Used To Meet Federal Mandate Used To Meet State Mandate Used to Support One or More Defined Business Emphasis Areas Risk Level Associated With Data Program Program Z 48
Step 3: Assess Current State of Data Programs Click to edit Master title style Evaluating Data Programs Accuracy Timeliness Completeness Validity Coverage Accessibility 49
Step 3: Assess Current State of Data Programs Click Example to edit - Master title style Minnesota DOT (MNDOT) Surveys - Used to assess strengths/weaknesses of current data programs Focus groups Used to prioritize needs/gaps identified in surveys Data Program Workshops Used to propose solutions to address needs/gaps in data programs that support investments and resource allocation 50
Step 4: Establish Data Governance Programs Click to edit Master title style Develop a Data Governance Model Data Governance Framework Data Governance Participants 51
Step 4: Establish Data Governance Programs - Click General to edit Data Master Governance title style Framework Data Governance Framework Strategic Vision, Mission, Goals For Data Data Governance Board Division(s) Mission(s) and Goals Data Steward and Custodians Agency Data Programs Data Users and Stakeholders
Step 4: Establish Data Governance Programs Click to edit Master title style Define roles and responsibilities of Data Governance participants: Stewards Business Owners Stakeholders Communities of Interest Custodians (IT) 53
Step 4: Establish Data Governance Programs Click to edit Master title style Develop Data Governance Handbook or Manual Data Governance Charter Data management policy Data governance framework Roles of data governance participants Glossary 54
Step 4: Establish Data Governance Programs Click to edit Master title style Develop Data Catalog List of data programs List of business data owners List of data stewards Access to data standards and definitions 55
Step 4: Establish Data Governance Programs Click to edit Master title style Develop Business Terms Glossary Standard definitions of commonly used business terms 56
Step 4: Establish Data Governance Programs Click Alaska to Data edit Governance Master title style Framework Alaska DOT&PF Data Governance Framework, Overview IT SERVICES BUSINESS UNITS State Enterprise Technology Services (ETS) State Data Standards State Data Governance Structure Agency Goals (Performance Measures and Targets) Program Development Division Goals ADOT&PF 1. ASD/IS Data Custodians 2. Program Dev. Div. Programmers Data Governance Council (Division Director(s), Section Directors) Business Data Owners (Section Directors, staff) Core Business Programs Core Data Systems COIs and Working Groups 57
Step 5: Use Technology to Support Data Click Management to edit Master title style Use of Business Intelligence tools support executive decision-making and data sharing within the agency Geographic Information System (GIS) Dashboards and Scorecards Knowledge Management Systems Bidding, Auctions, and Cost Management Solutions Invest in staff through training opportunities to increase ROI 58
Step 5: Use Technology to Support Data Click Management to edit Master - Example title style Hennepin County, Minnesota Strong executive support from County Administrator Dashboards Provide visual display of performance for county programs based on targets and goals Balanced scorecards Track Key Performance Indicators (KPIs) for county programs Align staff around common goals and strategies Serve as essential decision-making tool 59
Step 5: Use Technology to Support Data Click Management to edit Master Balanced title style Scorecard 60
Step 6: Link Data to Planning, Performance Click Measures, to edit and Master Target-Setting title style Processes Fully integrate data management processes with agency performance measures and target-setting processes by doing the following: Do not use one size fits all approach Link performance measures and targets to budget allocations Include objectives regarding resource allocation in the agency Business Plan Use external data sharing agreements to obtain data for performance measures that the agency does not collect internally 61
Step 6: Link Data to Planning, Performance Click Measures, to edit and Master Target-Setting title style Processes Revise or stop using targets if data not available to support the target(s) Arrange performance measures (PMs) in a hierarchical order (i.e., federal, state, local PMs) Incorporate customer satisfaction as a measure in setting performance targets Reward business areas that consistently meet targets and goals 62
Click Step 6: to Example edit Master title style Alaska Department of Transportation and Public Facilities Planning Support Planning mission and goals through use of a data business plan Performance Measures Develop performance measures using data systems identified in the data business plan Target-setting Identify data needed to develop targets and performance measures by 2020 to reduce the 3-year moving average of traffic fatalities per 100 million vehicle miles traveled (VMT) by 3% per year Link Planning, Performance Measures, Target-setting to support budget and resource allocation decision-making at ADOT&PF 63
Data Management to Support Performance- Click Based to Resource edit Master Allocation title style 64
. Click to edit Master title style Additional Research Underway Risk Management Information Technology 65
Additional Research Underway Click for NCHRP to edit 8-70 Master Risk title Management style Literature review and case studies Identify elements of risk management Describe application to PBRA Produce a primer which recommends best approaches for use of risk management in PBRA 66
Additional Research Underway Click for NCHRP to edit 8-70 Master Data title Management style Literature review and case studies Identify Information Technology (IT) issues affecting data sharing Discuss how these issues negatively or positively affect data sharing Produce a primer which recommends best IT approaches for data sharing and access to support Performance-based resource allocation in transportation agencies 67
. Click to edit Master title style Concluding Remarks and Final Questions 68
Click Questions to edit Master and title Answers style Please type your questions into your webinar control panel We will ask as many of your questions as time allows.