Assurance Review Process - Lessons Learned

Transcription

1 Assurance Review Process - Lessons Learned Benefits Realisation Management July 2012 Lessons learned and better practice The purpose of this publication is to present the lessons learned on project management since the implementation of the Australian Government s Assurance Review Process. Lessons learned are defined as observations arising from reviews which highlight opportunities for improvement. This publication is intended to assist agencies to better identify opportunities for improving their Benefits Realisation Management practices, and to promulgate the lessons learned and evidence of better practices observed from Assurance Reviews. The publication s intended audience is for project and program managers who may not necessarily be well versed or have the necessary expertise or capacity to effectively utilise Benefits Realisation. The intention is to convey to agencies the principles of Benefits Realisation Management rather than to provide detailed guidelines on any methodology. What have we learnt that indicates the need for a benefits lead approach? Benefits Management provides agencies with a way to track and maximise the achievement of benefits from the investment. Experience shows that agencies are more likely to maximise the realisation of benefits if: Benefits are managed strategically and dynamically throughout the program and beyond; Benefits are used in the decision making process allowing the prioritisation of investment decisions, and indicating when a program may be off track; Benefits are a key focus of the Business Case; The New Policy Proposal (NPP) and supporting Implementation Plan sets out the benefits expected from the program; Benefits are measured routinely and are part of normal planning and reporting functions not regarded as an optional and stand alone exercise; and Benefits are agreed and championed by all stakeholders. What are benefits? Page 1 of 9

2 The Government seeks to achieve benefits 1 for the Australian community primarily through programs delivered by agencies. The improvements that these programs introduce can be quantified as benefits. Benefits can be thought of as falling into one of three broad categories: Improved delivery and effectiveness of government services (e.g. via policy design); Reduced cost those benefits that reduce cost or avoid additional cost into the future (e.g. through efficient program design and agency productivity and efficiency); and Increased revenue those benefits that bring in revenue to an agency by collecting a levy or a tax. Ultimately it is the achievement of benefits that indicates success How do benefits relate to government processes? Changes to the way Government policy and services are delivered to the community are normally initiated through NPPs and Cabinet Submissions. Therefore, in the context of the new policy life cycle, benefits are the yardstick by which decisions to invest are made, the management of the program and the achievement of outcomes can be optimised, and the effectiveness of the program can be assessed once implemented. A benefits approach can help an agency maintain the achievement of outcomes by ensuring the capability continues to meet measurable targets. Programs that have a benefits focus are able to align to agency and government strategic objectives with clear and measurable targets. Aligning programs and Key Performance Indicators (KPIs) not only assists the agency to understand where the program fits with its own strategic priorities, but also provides external stakeholders with evidence the program can deliver an effective and justifiable impact. In some cases, programs can align to existing agency KPIs to derive key benefits, thereby increasing the link between program implementation and organisational objectives. A benefits approach is like having a compass to guide a journey Government projects and programs Program delivery is often seen as the creation, establishment or purchase of something new. This is in part true, but in essence, the extent to which this new asset or capability is used will determine the amount to which benefits are achieved. For example, a new ICT system can be considered a key project output. Once the system is built, the project might be considered complete. However the effective use of the ICT system is based on more than the delivery of the system itself. Change management processes are required to implement the new ways of working, in this case new ways of data input, analysis and use. The extent to which the new ICT system is being effectively used and the efficiencies gained (improved data accuracy), can be measured as a series of benefits. 1 A benefit is described as the measurable improvement resulting from an outcome which is perceived as an advantage by one or more stakeholders, which contributes towards one or more organisational objectives. Cabinet Office of HM Government, UK, Managing Successful Programs, UK, 2011, p.75. Page 2 of 9

3 The best and most effective benefits are tangible, so that they can be measured over time. A benefits profile is an excellent way to collect and manage benefits through the life of a program so they become incorporated into business as usual practices. Benefits that are envisaged before a program commences, for example at the NPP or conceptual development phase, are known as expected benefits. For example, a new software system is proposed to improve the way an agency manages transactions with the community. The justification (expected benefit) for the investment is that it will save the public time to interact with government via an online portal and will reduce the cost of processing by the agency. After the new software system has been delivered and a training and awareness campaign has taken place, the operational benefits are measured and compared to the expected benefits before the new system was introduced, tangibly showing the improvements achieved. Benefits and the New Policy Proposal life cycle Understanding the benefits of a proposal is critical to effective decision-making and to the achievement of the outcome desired from the investment Policy Conception Be clear about what you want to achieve Being explicit about the expected benefits to the decision making process can help achieve an agreed position on the potential new program. It also assists internal and external stakeholders to more easily understand and agree upon clearly stated outcomes, benefits and measures of success. There are many practical ways to achieve this via, for example, consultation during drafting through to facilitated and objective workshops that generate visual benefit logic maps. Achieving benefits is what makes the investment worthwhile Expected benefits should be linked to the strategic objectives of the agency; At this conceptual stage, policy teams should look beyond their immediate KPIs; and Collaborative and inclusive approaches with stakeholders in the identification of benefits can improve benefits specification. Policy Proposal Consider the best way to achieve benefits At this stage the NPP and a high level/initial Business Case are developed. Applying a benefits approach assists the justification of the initiative and informs the Business Case by clearly showing where the expected improvements are to be found. Information about benefits can be embedded into the NPP showing where offsets may be achieved and how outcomes/success will be measured. Page 3 of 9

4 Undertaking benefits analysis will also assist in: Providing evidence that the proposal will be effective and represents value for money; Demonstrating how the proposal is the simplest and most cost-effective solution or how it is superior to other simpler or more cost effective solutions; and Providing evidence of when citizens will see benefits from the proposal and how these benefits will be measured. Measurement of benefits may not always be easy, as some benefits cannot be easily quantified. However, there are ways of taking intangibles and turning them into measurable benefits, for example: client satisfaction can be mapped by taking surveys at regular intervals. That which is measured over time, improves The Benefits Realisation Plan should clearly articulate the nature of each benefit expected from the program, linked, where necessary, to the program objectives, the timeframe for measuring progress and the program owner of each benefit; and Responsibility for managing and delivering the benefits should be assigned to appropriate individuals/areas who are accountable for successful delivery and who are able to affect the outcome. Business Case From here to where why make this investment? A Business Case describes an investment, potential benefits and options to achieve these benefits. Apart from justifying why a particular NPP is a priority for an agency, understanding and articulating expected benefits provides guidance to options analysis and potential delivery approaches. This approach helps increase confidence in decision makers, it also allows a risk assessment to be undertaken against options and their likelihood to achieve the expected benefits. Benefits argue the case for which the proposal is a good investment Describe how each option within the Business Case is self standing and describe the differences in expected benefits and the associated risks for each; and To facilitate the effective prioritisation of programs and Benefits Realisation, financially evaluate the tangible benefits of all program elements. Project/Program Delivery Know when you are on the road to success (or not) During the delivery stage of a program, a clear understanding of the expected benefits helps focus on the delivery of the important things the activities that will achieve the expected benefits. Some interim benefits may be achieved during this stage, which provide an excellent indicator for the ongoing viability of the program. In addition, if changes occur that affect the scope of the program, any impact can be assessed against the existing benefits and the extent to which they are still likely to be achieved. This information can be used to inform an updated Business Case. This approach provides governance bodies with an excellent tool to manage and monitor programs, particularly those programs that are in the delivery stage for a number of years. Page 4 of 9

5 Benefits can t deliver themselves they don t happen by accident Suitable, measureable benefits should be identified at the outset; measurement needs to be integrated into program planning; and a culture of reporting realised benefits needs to be introduced into the governance of the initiative; and Develop a benefit realisation methodology that continuously identifies and measures actual and forecast benefits to guide the completion of the current implementation. KPI measurements being monitored should be capable of being linked to claimable business and government benefits that demonstrates that the program is progressively delivering the expected business and government benefits. Control over a program can be maintained by mapping delivery of program outputs, to expected benefits. Remembering that not everything goes according to plan, look out for events and circumstances that can affect programs and impact benefits. Sometimes, new opportunities occur that can be leveraged, particularly in long running programs. Using a benefits approach can help determine whether to make changes to an existing program. Benefits can also be used as a recovery action, or to refocus programs to deliver what is important and/or essential. Regularly updating the Benefits Realisation Plan will facilitate benefits quantification and can improve program control, thereby contributing to improved benefits realisation. Transition into the agency On time, budget and scope isn t enough When the newly developed program outputs are ready to be embedded into the agency, business change effort ramps up to ensure that the outputs can be effectively implemented, with as little impact on current operations as possible. A benefits focus ensures that business change, i.e. training, awareness campaigns, recruitment, restructuring etc, can be linked directly to achieving stated benefits. Not only does it help drive the change effort, measuring benefits before and after business change also shows the effectiveness of the change. Maintaining the new capability Protect the investment Benefits management lasts well after a program is finished. Once the outputs have been embedded in the agency, program implementation teams are usually closed down. At this stage, responsibility for achieving benefits from the program can be transferred to operational management. It is not unusual to measure benefits years after a program has been completed. Page 5 of 9

6 Closing a project/program is not the end for benefits Establish an agency centralised benefits management methodology/framework, outlining the transfer of responsibility for benefits realisation from the program to BAU (operational management). To ensure benefits are harvested in a way that increases the achievement of objectives and outcomes, it is important to create enduring teams and management structures to take care of the capability after programs are finished and program teams disbanded. In assessing the success of benefits realisation, aim to outline the benefits delivered to external stakeholders and the public, refine the metrics to capture user uptake and satisfaction, and measure the extent and quality of the enabling capability delivered; In order to maintain impetus in the future development and broader implementation (benefits) a strong BAU champion is required; and Communicate the success of the program within the agency in a manner that the capability can be further leveraged to maximise benefits realisation. Mapping benefits realisation A Benefit Map illustrates the relationships between outputs produced from programs, the business changes required to take on the new capability and the achieved outcomes, the intermediate and end benefits anticipated to be realised and lastly, the agency strategic objectives achieved. The diagram below is an outline of a benefit map, for further guidance refer to the Benefits Management Strategy Template. Page 6 of 9

7 How are benefits useful to Assurance Reviews? Review teams look at benefits throughout the Assurance Review Process. For the purposes of benefits management, reviews can be summarised into the following groups: Conception Pre-decision Implementation Post Implementation Review Process At the beginning of a potential program, the Review Team will look to see that benefits have been identified and measures have been considered. The likelihood of achieving these benefits can assist a risk assessment for the program. Prior to reviewing a decision on a proposal, the Review Team will look for evidence that a benefits management approach has been applied, the agency is clear about what it wants to achieve and can quantify the improvements to be sought. Benefits that are able to be measured before delivery begins can be used as a baseline to compare improvements throughout the program. Benefits can be used in the business case to compare options. During the implementation phases, the Review Team will look for evidence that the benefits agreed upon in previous stages have been incorporated and aligned to the delivery effort. Program success is more than meeting schedule, cost and scope. Programs should be able to show how their efforts and outputs contribute to benefits for the initiative. This can be achieved via a Benefits Realisation Plan. Some interim benefits may be achieved and reflected at these stages, increasing confidence of overall success. At this point in the life of the program, the Review Team seeks to ensure that the expected benefits that were envisaged earlier are being compared to measures taken following implementation of the new capabilities. In addition, the Review Team will also look to see how the management of benefits will be transferred into the agency for long term harvesting, bearing in mind that key benefits are often not realised at the time a program is closed. Expected Key Benefits Documents Potential Benefits described NPP Submission Business Case Benefits Management Strategy Benefits Profiles Benefit Map Implementation Plan Business Case Benefits Management Strategy Benefits Profiles Benefits Realisation Plan Updated Benefits Realisation Plan Transition Plan Program Closure Report Page 7 of 9

8 Better Practice Guidance Department of the Prime Minister and Cabinet, Guide to Implementation Planning, August Cabinet Office of HM Government, UK, Managing Successful Programs, UK, Australian National Audit Office, Better Practice Guide: Planning and Approving Projects an Executive Perspective, June Cabinet Office of HM Government, UK, Office of Government Commerce Best Practice Portfolio: Fundamentals of Benefit Realization, UK, Link Benefits Management Strategy Template Page 8 of 9

9 Further Information: Website: Assurance Reviews Branch Implementation and Performance Improvement Division Financial Management Group Department of Finance and Deregulation John Gorton Building, King Edward Terrace PARKES ACT 2600 ISSN X Copyright Notice Copyright (and any other intellectual property rights, if any) in this publication is owned by the Commonwealth of Australia. With the exception of the Commonwealth Coat of Arms and the Department of Finance and Deregulation s logo, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence (CC BY 3.0). A summary of the licence terms is available from The full licence terms are available from Attribution This publication and any material obtained from it is to be attributed as: Commonwealth of Australia 2012, Department of Finance and Deregulation, Assurance Review Process Lessons Learned Use of the Coat of Arms The terms under which the Coat of Arms can be used are detailed on the following website: Acknowledgements Australian Government Assurance Reviews are a combination of Implementation Readiness Assessment and Gateway Reviews. Gateway refers to the Gateway Review Process developed by the United Kingdom s Office of Government Commerce. Page 9 of 9