On Track: The evaluation of influence in system wide change

Transcription

1 On Track: The evaluation of influence in system wide change

2 Content Introduction to On Track The evaluation The findings Theory of change Organisational & community readiness Potential next steps Reflections

3 Influence The capacity or power of persons or things to produce effects on others by intangible or indirect means. Kirkhart, K. (2000) Reconceptualising evaluation use: An integrated theory of influence. New Directions for Evaluation. No 88 (pp 5-24)

4

5 Images of the future - 5 archetypes 1. Evolution and progress - more technology. 2. Collapse - we have reached our limits. 3. Gaia the world is a garden or a forest. 4. Globalism breaking down borders. 5. Back to the future returning to simpler times.

6

7 The eco-cycle of change Rigidity trap Source: Mark Cabaj (2013) The eco-cycle of change. The Collective Impact Institute.

8 The MH&A sector over time

9 Change pressures in the present Source: Platform Trust & Te Pou o Te Whakaaro Nui (2015, p30) On Track. Knowing where we are going.

10 On Track 7 priority action areas

11 The evaluation

12 Purpose of the evaluation On Track was published in April This was a developmental evaluation that was designed to give the project team a sense of whether or not we were on track. The information was also intended to help inform what we need to do in order to continue to influence change.

13 Methodology Literature review of relevant behaviour change theories. A review of On Track documentation. Key informant interviews (Te Pou & Platform Trust). Survey of key informants from 13 District Health Boards. Follow-up telephone interviews. Workshop with staff from a selected DHB success case.

14 Districts engaged through On Track workshops

15 On Track workshops: Estimated numbers of participants from May Sept 2016.

16 The findings

17 Charting significant positive change with On Track being identified as an important cause (n=9)

18 Examples of significant change On Track District Health Board responses Working in an inter-sectoral way. More emphasis on delivering services closer to where people live. Increased consumer & family participation through a consumer group. Shared leadership model (DHB & NGO). A work plan that is influenced by On Track and Rising to the Challenge. Shared training opportunities across all mental health and addiction services. New co-designed and/or co-ordinated services. NGO provider responses Increased emphasis on people determining their own health pathways. Increased inter-sectoral approaches. Innovative models of service delivery. Connecting with other organisations to improve service integration and to develop peer support, including suitable staff training.

19 WHAT HELPS & HINDERS POSITIVE CHANGE WHAT TO FOCUS ON? Enablers Those positive factors that appear to collectively enable innovation and the implementation of better and more integrated health & social policies and services. Barriers Those negative factors that halt progress or undermine and slow change efforts. Predisposing factors those that create the conditions for change. Perpetuating factors those that amplify and normalise a change state.

20 WHAT HELPS & HINDERS POSITIVE CHANGE Enablers Policy responses to systemic failures. Policies that set a new national direction. Workforce capability-building. External change agencies that seek to influence system change. Strategic leadership at the national level. Internal change agents local advocates for systems and service change. Disposition to collaboration/community engagement at the district level. Barriers Systemic shock (eg, the impact of sentinel events). Systemic stress no slack in the system, scare resources, workload issues and endemic lack of optimism. The workforce operating in survival mode. Challenges associated with rural areas eg, physical isolation and stretched infrastructure.

21 WHAT HELPS & HINDERS POSITIVE CHANGE Enablers (contd? Barriers Strategic leadership. Holding a vision and having a mandate for change at the local or district level. Lack of executive team buy-in. On Track as an intervention promoting a common agenda for change (ie, actionable). Having local On Track champions. Change leadership & adaptive management, including learning by doing and the ability and willingness to change tack. Network formation & teamwork. A sense of organisational and community self-efficacy. Integrative technology serving multiple purposes, including rapid system feedback. Tendency to independence vs, partnering. Underdeveloped feedback systems. Limited mechanisms for deliberative, responsive and progressive change. Lack of persistent external support. Losing champions - people familiar with and committed to sector development (eg, shared vision/common agenda). NB: This may be a function of a negative milieu or low selfefficacy.

22 Why & how On Track has been influential Qualities of On Track as an innovation Enacts government policy. Compatibility expresses a shared vision/common agenda. Provides an unambiguous advantage over the status quo. Adaptable to organisational & community needs not prescriptive. Easy to use certain components can be taken up opportunistically, reducing complexity and risk.

23 Theory of change

24 DRAFT Theory of Change...

25 DRAFT model.

26 Agenda setting. Leadership Reliant on the personal attributes of leaders and managers understanding the dynamic and multidimensional nature of the MH&A system. Strategic leadership Owning and articulating a vision for change. Tactical-level leadership Managers engaging critics, enabling innovators and sharing leadership (ie, distributed leadership). Adaptive management Managers driving learning and change, based on the evidence and a process of systematic inquiry.

27 Behaviour Change Frameworks Source:

28 Fostering a collaborative culture Checklist of behaviours for collaborative leaders: Encourage the process of goal, role and task clarification. Promote open communication in order to achieve team goals. Develop collaborative team norms operate with fewer rules but set high standards. Take personal responsibility for the team s success. Employ energising strategies. Source: Weiss, et al. (2014) The Interprofessional Health Care Team.

29 Organisational & community readiness for change

30 WHAT HELPS & HINDERS POSITIVE CHANGE Organisational and community readiness The importance of organisational and community readiness with regard to the combination of enablers and barriers that are in play. Note: There is considerable variation between New Zealand District Health Board areas in their overall readiness for change.

31 Organisational and community readiness model DHBs could consider their readiness from the following three angles: 1. Contextual readiness - their capacity for change. 2. The level of engagement with the vision outlined in On Track. 3. Subsequent developments in the direction of On Track (either caused by On Track and/or by other contributing factors). Source:

32 Potential next steps

33 Potential next steps Support districts to use theoretical frameworks for diagnosing system-related problems and appropriate change strategies. Support districts to implement a baseline and benchmarking process. Undertake outcomes harvesting across designated districts. Support local change agents to undertake evaluative monitoring activity (both qualitative and quantitative). Compile stories of success and significant change.

34 Reflections

35 Some reflections Evaluating influence is more difficult than evaluating a thing. Thinking systemically boundaries, perspectives & interrelationships. The interrelational aspect of the MH&A system has, to some extent, evaded easy capture. Self-organising complex systems - paying attention to emerging patterns and then responding to them.

36 We are now in a transition from an object-oriented to a system-oriented culture. Here change emanates, not from things, but from the way things are done. Jack Burnham Systems Aesthetics (1968)

37

38