Delivering Integrated Care Masterclass workshop. Measuring Success. Lisa Larsen, PPL. February & March The Better Care Fund

Transcription

1 Delivering Integrated Care Masterclass workshop Measuring Success Lisa Larsen, PPL February & March 2017 The Better Care Fund

2 Summary Measuring success is key to performance management, service improvement and accountability. Health and care managers who have a considered, systematic and proportionate approach to monitoring and evaluation are able to: manage services more effectively focus efforts of staff and stakeholders across the system to work together to deliver impact build ownership, buy-in and support for greater integration and person centred care. There are different definitions of what success looks like. Effective programme leaders engage with stakeholders across the system commissioners, providers, regulators, patients, carers and community groups to establish the outcomes they want to achieve and the measures they want to use to track progress and impact. Competing priorities and trade-offs should be acknowledged and agreed outcomes and measures should align with local, regional and national priorities. 1

3 Summary (cont) From the outset, you need a clear understanding of how your initiatives will deliver benefits. A logic model is a tool which helps describe how a programme s various components inputs, interventions, outputs, outcomes and impact - are expected to fit together to achieve intended results. It enables programme leaders and stakeholders build a shared understanding of how a programme will likely deliver results, identify cause-effect relationships and test the feasibility and validity of the underlying assumptions. Further information For more information, and useful tools and resources please see How to understand and measure impact (This BCF guide will be updated shortly). 2

4 What we will cover today Defining success, outcomes and measures Linking activity to impact Different ways of monitoring and measuring success Close

5 What is success? Different perspectives across the system Local Authorities Community groups CCGs People Regulators & other National Bodies Carers Providers Patients STRATEGIC OPERATIONAL SOCIETAL - PERSONAL

6 Why is measuring success important? When done well, measuring success: Show that outcomes for citizens are being achieved across the system Demonstrate value for money and success against national metrics and local measures across many organisations Focus plans and inform priorities through clearly articulated key performance indicators DH Integration Standard measures National Outcomes Framework measures Statutory performance measures Locally specific measures Support organisations to work as one health and social care system to deliver impact and continually improve Dr Nick Goodwin, CEO, International Foundation for Integrated Care For health and care systems it is important to adopt and use a set of measures that align with the main elements of a national, regional or local strategy for person centred coordinated care. The complexity and the necessary variety in how integrated care strategies need to be developed, means that outcomes and measures need to be chosen to suit local and national priorities Better Care Fund: How to understand and measure impact (1)

7 Challenges of measuring success Aligning approaches across the whole system Defining and measuring Outcomes not just inputs, processes and outputs Attributing results to the intervention Setting smarter targets Devising fit-for-purpose monitoring and evaluation frameworks

8 What does good look like? Three elements of how to effectively measure success at a system level Defining outcomes and measures - Setting clear end goals of the intervention. This involves understanding what an outcome is and who it impacts people / the system? Linking initiatives to impact Mapping activity to change in outcomes and indicators. This involves understanding how interventions really work to impact the outcome and how much. Tailored, flexible approach Using a mix of ongoing monitoring, rapid assessment, deep-dives and longitudinal evaluations The most serious mistakes are not being made as a result of wrong answers. The truly dangerous thing is asking the wrong question Peter F. Drucker Men, Ideas and Politics (3)

9 Defining outcomes and measures What are outcomes? Outcomes are the results people care about most when seeking treatment, including functional improvement and the ability to live normal, productive lives International Consortium for Health Outcome Measurement, 2013 Why do we need whole system outcomes? x Failure to plan and co-ordinate services with and around people s needs, leads to fragmentations in care and suboptimal outcomes x It is unlikely that any provider can deliver an outcome in isolation, there must be joint accountability across stakeholders What does an outcome look like? Outcomes can range from broad lifestyle goals (e.g. no. of days individuals spend at home / in their communities) to specific quality of care outcomes (e.g. permanent admissions to nursing homes).

10 Defining outcomes and measures How can you deliver what you can t define? How do you identify impact if you don t measure? To choose the right measures, there needs to be a clear understanding of: Core aims of integrated care Range of desired outcomes Timeframe over which outcomes can reasonably be achieved How impact and attribution can be measured Robustness of measures Simplicity and ease of measurement Time series analysis What is a good indicator? 10 key questions: 1. What is being measured? 2. Why is it being measured? 3. How is this indicator actually defined? 4. Who does it measure? 5. When does it measure it? 6. Will it measure absolute numbers or proportions? 7. Where does the data come from? 8. How accurate and complete is the data? 9. Are there any caveats/ problems? 10. Are particular data tests needed?

11 Defining outcomes and measures A useful checklist Specific Measurable Achievable Relevant Time-bound Evaluated & Reviewed Measures can be clearly articulated to people with a basic knowledge of the system Criteria for measuring progress towards the attainment of the goal are concrete Measures are practical, achievable and realistic within operational constraints Measures offer insights that matter Clear timeframes that have been set and are evident to stakeholders Evaluation is happening on a consistent basis and regular review cycles are planned for

12 Linking initiatives to impact Logic modelling INPUTS INTERVENTIONS OUTPUTS WHOLE SYSTEM OUTCOMES IMPACT A logic model diagrammatically shows key components of a programme and the way that actions are intended to lead to outcomes. This helps to: Assess the strength of assumptions on how the programme will achieve change Identify cause-effect relationships Build an in-depth understanding of how results will be delivered Raise awareness and build common understanding amongst stakeholders

13 Practice example: Stockton on Tees strong links between schemes and intended outcomes A number of useful schemes Multi-disciplinary service Improving pathways and care for dementia Intensive Community Liaison Service Delayed transfers of care services and pathways Integrated digital care records (MIG) ispa service for Single Point of Access - triage Business cases to mainstream services What links them? A set of desired outcomes Improved health and wellbeing of patients Individuals maintaining independence in their own homes Increased integration of health and social care services Put Trust on a more sustainable financial footing Freeing up A&E capacity 12

14 Practice example: Stockton s Logic Model Inputs Activities Outputs Outcomes Multi-disciplinary service, including Intensive Community Liaison Service Improving pathways and care for dementia including: 1Increase awareness 2Dementia volunteers 3Information & early support 4Live well for dementia and support for carers 5Workforce development 7 days a week wellbeing facilitators providing wellbeing support assessments and plans Support services from health, social and voluntary services 8-8, 365 days a year, prompt comprehensive assessments and interventions for individuals living in the community and care homes Raising awareness through leaflets,, information for GP lists, dementia champions Information, advice, and support for diagnosed or undiagnosed dementia sufferers 22 weeks of maintenance cognitive stimulation therapy Carer home support service for 100 people Improve competence of staff who are working with older people Reduction in Non-Elective Admissions Reducing inappropriate admissions of over 65s into residential care % of over 65s still at home 91 days after discharge from hospital Reduction in delayed transfers of care from hospital per 1,000 population Increase in estimated diagnosis rate for dementia Improved patient experience of services Improved communication and integrated working Earlier intervention and prevention Increased integration of health and social care services Freeing up A&E capacity Put Trust on more sustainable financial footing Individuals maintaining independence in their own homes Improved health and wellbeing of patients

15 Practice example: Logic model for Integrated care for the over 65s in Ambridge

16 Systematic, tailored and flexible approach Singular Multiple Reviews/ assessments Point-in-time Case studies Regular Reviews Programme Evaluations Project Evaluations Performance Monitoring and reporting On-going

17 Practice example: Hounslow Better Care Fund Review Qualitative review of the schemes followed the approach outlined in the Better Care Fund: reflecting on 2015/16, planning for 2016/17 (examples below). Schemes have been rated on a scale of 1 10 where 1 is not at all and 10 is to a great extent. The schemes were then been evaluated based on the scores to assess whether they are worth continuing for 16/17 and how we can maximise the impact of schemes going forward to ensure that they have maximum impact.

18 Practice Example: Hounslow BCF review, cont. The quantitative evaluation criteria focused on assessing the value for money of each scheme based on the following criteria: Stage stage of the project: Funding in place (F), Project Initiation (I), Benefits Realised (B) Cost of scheme - (High cost > than 1,000k, Medium cost 400k-1000k, Low cost < 400k) KPIs what KPIs is the scheme being monitored on Meeting objectives is the scheme meeting its KPI target? Impact Is the scheme having an impact on BCF objectives? (Low no or very low impact, Medium having an impact but not meeting objectives, High meeting/above) Value for money is the scheme delivering value for money this will be determined based on the level of impact for the level of investment

19 Practice example: Decision making tool in the Hounslow review Based on the outcome of both the qualitative and quantitative evaluation the scheme will then be assessed as to whether it is worth continuing next year using the following approach: 18

20 Why it matters A well thought out monitoring and evaluation plan assists greatly with thinking through programme strategies, objectives, planned activities, and whether they are most appropriate to implement. Information collected and presented will support key decision making processes and create a narrative for BCF work. Key to this is when and how this is planned to happen. Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted Albert Einstein

21 Using results Establish a learning culture across the system: Leaders, operational managers, front line staff and people and patients Engage, share results and engage again! So what? Implications for the programme going forward Demonstrating that impact will help to further strengthen the case for change nationally and locally, and also assist us to understand and learn what works in different contexts. How to understand and measure impact

22 Useful links 1. How to understand and measure impact 2. Integrated care and support Pioneers: Indicators for measuring the quality of integrated care 3. Peter F. Drucker: Men, ideas and politics 4. International Consortium for Health Outcome Measurement 5. North West London: Whole Systems Integrated Care 6. PIRU: Integrated care and support Pioneers: Indicators for measuring the quality of integrated care 7. LGA: How do you know if STPs are making a positive impact?

23 The Better Care Fund Measuring success Collated notes from workshops at all masterclasses Local initiatives to consider Warwickshire uses the logic model for STP and finds it very useful. They find do however, find it challenging to add the costs. West Sussex The biggest challenge is financial, in particular evidencing value for money/return on investment. There are some perverse incentives hospitals paid for activity but community gets block funding people are territorial about their pots of money. You can measure anything but you need to get the incentive right. Important to recognise that certain schemes might not be funded through BCF but that does not mean that they are not good schemes. For example, if they are not able to demonstrate that they are about transformation and integration across health and social care, they are asked to demonstrate what are the benefits, measures, feedback loops etc. Need to be able to demonstrate the so what you ve done X number of assessments what was the impact? Organisations need to take ownership of this process. West Sussex has taken the LGA self-assessment framework and turned it into two questionnaires: one to look at stakeholders/leadership/engagement, one to look at the operational side. Regarding cross-referencing of social care and health data, different lenses are being used. For example, with DTOC data, West Sussex breaks down the information so they can see the reasons for different outcomes, including what is going on in the acute trusts. People are still doing their own thing and using their own systems - where systems are shared it is much easier to share a clear and compelling story Haringey The challenge has been to identify killer KPIs that need to be tracked and how they connect to the schemes in place. Otherwise it can be difficult to articulate the benefit of social care to the health economy. In Haringey they are the system so that they just have one narrative. Logic models Measuring success workshop notes 1 3

24 One area is using logic model in care homes very complex as multiple programmes. Moves away from who s paying etc more objective to review services and programmes. Peer review One area brought people together to review workstreams and projects including bringing strangers in to the room to ask questions. Intention was to understand what was stopping it from being as successful as would have liked and what/who was needed to unblock and accelerate progress. Identified challenges Attributing change to a particular scheme The logic model can be very challenging when there is the need to share data. So when defining KPIs have to be very pragmatic on what is measurable. Outcomes whose outcomes? I want to go home? What does home mean? Outcomes are not mutually exclusive or conflictive, our systems make them exclusive, so we may need to look at our systems and leadership is key for that. Data sharing Subjective definitions by people inputting data difference between clinical groups KPIs are targeted at over 65s, but in some areas under 65s are more costly this is a growing trend year olds who were being looked after by their parents but their parents have died or are too old to look after them any more. Having robust evidence and demonstrating causality Areas or types of further support needed Legislative framework that enables data sharing. Coproduction of KPIs with patients and service users Patient determined outcome measures/what was your goal (co-designed) and was it met? Agreeing what is a real outcome (e.g. avoided NEA vs. rapid response instead of GP) Consider Individual level > project/service level > system level Needing sufficient analytical capacity Need to be able to measure unintended consequences Tools to measure attribution demonstrate value from stories > metrics What does success look like? Achieving outcomes for individuals that they want or need, and proving success Everyone understanding the importance of respecting KPIs of other organisations Not losing focus on what is happening on the ground Using technology to make sure people are confident and independent. Also, making sure that family members are content. Making sure we are not just trying to avoid admissions. Is this flowing correctly? Delayed Transfers of Care is measuring failure however we need the morale boost of noting the successes Aligning outcomes and high level performance measures across partners Demonstrating value for money Measuring success workshop notes 2 3

25 Having a common workforce that has no boundaries care without walls Reducing DTOC meeting targets Ensure patients are risk stratified Ensure patients only give their story once Change or difference impact Measuring broader system impact including role of communities and voluntary sector People able to navigate complex systems Moving away from public sector funding using community assets Streamlined health and care across all age groups Effectiveness of integration at the point of delivery Common outcomes and how we will achieve shared responsibility People on the ground and patients value and quality outcomes Culture of trust and transparency patients at the heart Retention and recruitment of staff Find a measure that captures primary care/community/social care etc. Robust data sharing Measurement options Staff engagement/wellbeing/view of whether having impact GP survey patients who feel supported to manage their long term condition Patient activation measure Interoperability between systems Consistent measures across organisations Under 65 admissions and transitions/coming into care this is very costly and growing strain on council s budgets Readmissions or deaths within 4 weeks of discharge start of service/enablement Reablement service measures: o Mobility o Washing/personal care o Nutrition Qualitative clinical reviews and PROMS Length of stay in any service Reduction in hand offs Measuring success workshop notes 3 3