Workshop Advice From Your Strategic Advisors

Transcription

1 Workshop Advice From Your Strategic Advisors Presented by Follow the Action #ResultsGC

2 First Presenter Greg Richards, MBA, Ph.D., FCMC Director, Centre for Business Analytics and Performance Director, MBA Program Telfer School of Management, University of Ottawa Copyright Interis BDO

3 Deliverology: Does it Deliver? Summary of research conducted by: Telfer School of Management University of Ottawa September 2016 Gregory Richards, MBA, Ph.D., FCMC Research sponsored by:

4 1 Deliverology in a nutshell Genesis Started in the UK with the Blair government (2001) Referred to as the Prime Minister s Delivery Unit Why? Difficulties encountered in turning policy declarations into measurable outcomes Well-staffed team focused on delivery Gather and analyze performance data Design routines to apply lessons learned Deliver on policy commitments

5 2 Implementations UK N.Lands 2007 UK Wales 2011 Albania 2015 Romania 2011 Ontario 2004 LA 2006 MD 2008 Chile 2010 Brazil 2011 UK 2001 Indonesia 2009 Malaysia 2009 Tanzania 2013 Australia 2003 Queensland 2004 Liberia 2009 Sierra Leone 2008 Rwanda 2008 South Africa 2010 Pakistan 2013 Kenya 2015 Adapted from London School of Economics Boston Consulting Group, 2015

6 3 Lessons learned: what works? 1. Sustained interest and involvement from the political level 2. Resources 3. Data: clean, credible, promotes collaboration 4. Routines that build a discipline around performance management 5. A permanent integrated structure focussed on performance 6. Adapt the methods to the context

7 4 Lessons learned: Challenges 1. Working towards the performance indicator instead of the objective; can lead to gaming behaviour 2. Silo effect 3. Cost 4. The philosophy of management

8 5 Conclusions-Does it Deliver? Criteria Evaluation Comments Coordination, focus, accountability Meet policy commitments Long term impact on the organization Yes Yes.but Can be positive Be careful of managing to the measure instead of to the outcome; monitor silo effects. Performance is at the departmental level, difficult to define the impact of the Delivery Unit. How are policy outcomes defined? Anticipate the management process changes required.

9 6 Selected References Ball, S., Maguire, M., Braun, A., Perryman, J., & Hoskins, K. (2012). Assessment technologies in schools: Deliverology and the play of dominations. Research Papers in Education, 27(5), Etheridge, Z., Thomas, P. (2015). Adapting the PMDU Model. The creation of a delivery unit by Haringey Council, London: A case study. Institute for Government, London, U.K. Mullin, C. (2014). Making things happen at the Implementation Unit Civil Service Quarterly. Retrieved August 20, 2016, from Poister, T. H., Pasha, O. Q., & Edwards, L. H. (2013). Does performance management lead to better outcomes? Evidence from the US public transit industry. Public Administration Review, 73(4), Reevely, D. (2016). David Reevely: Ontario s experience shows serious flaws in deliverology governance National Post. Retrieved August 20, 2016, from Seddon, J. (2009). In Defense of Higher Education: Barber's Deliverology. Retrieved August 20, 2016, from The New Deliverology: 14 years of Evolution. (2015). London School of Economics, Boston Consulting Group. Watkins, K. (2013). Jim Kim s science of delivery : what role for politics? Overseas Development Institute (ODI). Retrieved August 20, 2016, from World Bank. (2013). Transcript: Delivering Results - A Conversation with Jim Yong Kim, Tony Blair, and Michael Barber. Retrieved August 20, 2016, from

10 Second Presenters Murray Kronick, CMC, FCMC Senior Manager, National Service Lead, Planning and Performance, Interis BDO Kasia Polanska, MA Manager, Planning and Performance, Interis BDO Copyright Interis BDO

11 Deliverology: What Does it Mean? Deliverology is a concept originating in the UK and holds the adoptive organization accountable for delivering on planned goals and promises. Key considerations include: What are our priorities, goals, targets, expected results? How will we deliver upon these priorities, goals and targets? How will we know whether we are on track to succeed? What will we do when / if we are not on track? How will we demonstrate that intended outcomes and results have been achieved? Copyright Interis BDO

12 Policy Overview: Policy and Directive on Results Policy on Results Replaces the Policy on Management, Resources, Results and Structures (MRRS) and the Program Alignment Architecture (PAA) Focus is on the delivery of programs with the intent of attaining expected results across government Directive on Results Includes 5 appendices: Mandatory Procedures for Departmental Results Frameworks (DRF), Program Inventories (PI), Performance Information Profiles (PIPs) Mandatory Procedures for Evaluation Standard on Evaluation Standard on Mandatory Outcomes and Performance Indicators Standard on Tagging Copyright Interis BDO 2016 Policy on Results: Directive on Results: 12

13 Policy Overview: Government-wide Change Old MRRS Policy New Policy / Directive on Results RPP / DPR / PAA in need of a refresh Departmental Results Framework setting the stage for reporting to Parliament Inconsistent detail / information in submissions to central agencies Simplified templates for submissions to central agencies with a focus on results 5-year cycle for evaluations for all programs Targeted, risk-based evaluations Additional Ad Hoc reviews (SOR, DRAP, etc.) Periodic full reviews of departmental spending Lack of comparability of Internal Services efficiency Copyright Interis BDO 2016 Standard Internal Services Performance Indicators 13

14 Policy Overview: Key Players Central Agencies Departments / Agencies TBS Executive Director, Program Performance and Evaluation PCO Deputy Secretary to the Cabinet (Results and Delivery) Data Integration Group Central Innovation Hub Operations Cabinet Committee on Agenda, Results and Communications (ARC) chaired by PM Chair, Performance Measurement and Evaluation Committee (PMEC) at DM level Head of Performance Measurement (HoPM) at DG level Head of Evaluation (HoE) at DG level Results and Delivery Units (RDU) Copyright Interis BDO

15 Policy Overview: Key Outcomes Parliamentarians and Canadians can actually understand what departments are doing and what they are achieving Resources are allocated to areas where there is the most significant impact for Canadians Results frameworks that drive continuous improvement of policy, programs and service delivery (short- and long-term outcomes) All new spending supported by evidence (data traceability) and linked to results Greater alignment and coordination (horizontality) across departments and agencies More visible progress towards the government s political agenda Copyright Interis BDO

16 Policy Overview: How does it work? Stakeholders Tools and Measures Canadians Parliament Minister Senior Management Strategic Departmental Departmental Results Framework Program Inventory Core Responsibility Program Program Departmental Results and Performance Indicators Area of influence Description how programs influence Results Tagging DATA MANAGEMENT Manager Staff Programbased Performance Information Profile Performance Information Profile Program Outcomes Evaluations, external studies Copyright Interis BDO 2016 Area of control 16

17 while getting the policy right is hard, it is only 10 percent of the challenge. The other 90 percent is the blood, sweat, and tears of relentless implementation. Sir Michael Barber, The Origins and Practice of Delivery: Copyright Interis BDO Consulting

18 Implementation: Timeline Phase 1 Departments Phase 2 Departments 10 Departments identified as priority Fall 2016 Winter 2016 Spring 2017 Summer 2017 Fall 2017 Deadline of November 2017 Final Draft Here we are Phase 2 Departments creating Departmental Results Frameworks, Program Inventories, and Performance Information Profiles for all Programs Copyright Interis BDO

19 Transitioning to the New Policy: A Structured Approach Transition Planning Ensure the structures are in place to support the new Policy and allow for a seamless transition Frameworks Create new frameworks and deliverables that meet the requirements of the new Policy Transitioning to the new Policy on Results Change Management Increase awareness, understanding and acceptance with the right engagement strategy Governance Establish the right accountability and governance structure for the frameworks Copyright Interis BDO

20 Approach to Transitioning to the new Policy on Results Transition Planning Conduct organizational readiness assessment Determine the transition approach Define milestones Define resource and funding requirements Assign roles & responsibilities (RACI) Manage transition Establish and manage approvals process Conduct Lessons Learned Frameworks Create common glossary of terms Review and integrate evaluation requirements Create new Departmental Results Framework (DRF) Establish use of Logic Models & Outcomes Maps Establish Program Inventory (PI) Integrate Programs and Internal Services reporting Establish Performance Information Profiles (PIP) & tags Ensure linkage to Parliamentary estimates & Public Accounts Governance Launch Performance Measurement & Evaluation Committee (PMEC) Establish Results and Delivery Unit Establish Data Management governance structure Identify Head of Performance Adjust accountabilities in existing performance agreements Identify Head of Evaluation Consult with Program officials to ensure framework alignment Establish horizontal linkages to other departments Change Management Conduct stakeholder analysis Establish competency requirements & identify gaps Develop awareness and training materials Foster a culture of evidence-based decision-making Help programs to tell the performance story Support changes in Memoranda to Cabinets, Treasury Board Submissions Launch the new framework and tools Ongoing support and mentoring Copyright Interis BDO

21 Exercise: The No-Texting Campaign The Scenario: The province, as part of the Highway Safety Act, would like to reduce the growing trend of texting while driving. They are considering drafting legislation, setting up a campaign to increase driver and passenger awareness, and working with industry to find solutions. Instructions: Use the Post-it notes and the blank Outcomes Map in Envelope # 1. Work at your table to create the Outcomes Map / Logic Model: 1. Identify each symbol as an Output / Outcome / Manageable or Uncontrollable Risk and draw the appropriate shape on it. Use the Legend at the bottom of the sheet as reference. 2. Lay out the logical outcomes flow on the Outcomes Map sheet 3. Connect the symbols with arrows (contributions) 4. Select 2 outcomes for measurement in PIP and discuss their attributes Copyright Interis BDO

22 If You Text - You re Next Campaign Sample Outcomes Map for PIP Activities Outputs Immediate / Intermediate Outcomes Departmental Result Core Responsibilty A-6 Draft new No-Texting Legislation A-6 Manage the No-Texting campaign for drivers A-3 Work with Industry to add safety warnings / interlocks Decreased Rate of Road Collisions Road Safety Copyright Interis BDO Legend: Activity Immediate / Intermediate Outcome Societal Benefits Manageable Risk Risk Outside our Control Output Contribution Blue Border = PIP entry Mitigating Action

23 If You Text - You re Next Campaign Sample Outcomes Map for PIP Activities Outputs Immediate / Intermediate Outcomes Departmental Result Core Responsibilty A-6 Draft new No-Texting Legislation O-1 Passed new legislation / regulations R-4 Unable to reach all target audiences A-6 Manage the No-Texting campaign for drivers O-2 Created awareness materials targeting drivers, driving schools and passengers O-19 Increased awareness of texting risks / consequences passengers O-5 Decreased passenger acceptance of drivers texting while driving drivers A-3 Work with Industry to add safety warnings / interlocks O-3 Created warnings / interlocks for vehicles and devices O-6 Decreased incidence of texting while driving O-7 Decreased texting-related collisions Decreased Rate of Road Collisions Road Safety R-3 Industry is not amenable to adding warnings / interlocks R-6a New devices / technologies exacerbate the R-6b problem O-19 O-11 Noncompliance consequences are insufficient M-6b Ensure penalties and social stigma are severe Decreased rates of personal injury and death / Safer Canadians Decreased vehicle and property damage / insurance costs Copyright Interis BDO Legend: Activity Immediate / Intermediate Outcome Societal Benefits Manageable Risk Risk Outside our Control Output Contribution Blue Border = PIP entry Mitigating Action

24 Sample Performance Information Profile Additional information: Outcome ID (O-xx) Outcome Name Frequency / Indicator Data Type Data Source Methodology Target End Date Date to Achieve Target Thresholds Data Owner Description Comments Directional Profile Estimated Cost of Measurement O-4 Increased awareness of texting risks / consequences O-7 Decreased texting-related collisions Copyright Interis BDO

25 1 1 Deliverology in Practice Sample Performance Information Profile (PIP) Additional information: Outcome ID (O-xx) Outcome Name Frequency / Indicator Data Type Data Source Methodology Target End Date Date to Achieve Target Thresholds Data Owner Description Comments Directional Profile Estimated Cost of Measurement O-4 Increased awareness of texting risks / consequences Level of awareness as indicated through a questionnaire provided through several channels. Specific questions will be aggregated into one awareness index. Percentage Questionnaire Question to a random sample of drivers in the province Ongoing Questions will include questionnaire - perceived reaction times, summarized responsibility to others, monthly / scenarios to respond to, 3 years etc. Baseline Value: 40% of awareness index 85% awareness index By the end of Year 3 of the campaign Green - 80% and over Yellow % Red - 60% Project Lead, No-Texting campaign A driver's and passenger's awareness measure of the risks and consequences of texting while driving. These include impacts to other road users such as cyclists and pedestrians, as well as to other motor vehicles, passengers and the driver themselves. Need to break down awareness levels by demographic slices, as well as by the number of years driving. Medium - effort required in setting up questionnaire through various channels, consolidating and aggregating the results. O-7 Decreased texting-related collisions Number of textingrelated collisions relative to the total number of collisions causing injury or death. Number of collisions per million kilometers driven Ministry of Transportation and municipalities' databases Monthly / 3 years Query the Ministry of Transportation and municipalities' databases, looking for the types of collisions on accident reports. Baseline Value 30 per million kilometers driven 5 per million kilometers driven By the end of Year 3 of the campaign Green - 6 and less Yellow - more than 6 but less than 9 Red - 9 and over Project Lead, No-Texting campaign Reduction in the rate of collisions related to reported texting Based upon the collision report completed by the on-scene police officer or at the collision centre. Low - the overall queries are a known, simple calculation. The data is already captured through collision reports. Copyright Interis BDO

26 An Alternative Approach Copyright Interis BDO

27 Transitioning to the New Policy: Critical Success Factors Decide on evolutionary or clean-sheet approach Take a scalable approach for the organization s size and complexity Involve a multi-disciplinary set of people Ensure clear accountabilities for results Set results and outcomes first, then SMART performance indicators Set realistic targets and tolerances Structure the data for traceability and repeatability Critical Success Factors Ensure senior management commitment Allocate dedicated resources and funding Engage staff in the transition Leverage Best Practices and Wave 1 adopters Copyright Interis BDO

28 Workshop Advice From Your Strategic Advisors Presented by Follow the Action #ResultsGC