Introduction to Implementation research. Yodi Mahendradhata

Transcription

1 Introduction to Implementation research Yodi Mahendradhata

2 Key questions What do we mean by implementation research? Why do we need implementation research? How to formulate implementation research objectives and questions? What constructs/variables are relevant for implementation research? What are common methods for implementation research? What are the key references for implementation research?

3 What do we mean by. Health system research? Implementation research? Operational research? Health policy research? Health services research?

4 Implementation research is that subset of HSR that focuses on how to promote the uptake and successful implementation of evidence-based interventions and policies. (Sanders & Haines 2006)

5 ..usefully defined as scientific inquiry into questions concerning implementation - the act of fulfilling or carrying out an intention (Peters et al 2013)

6 The multiple definitions of operational research, implementation research, and health systems research creates confusion and negatively affects the credibility and progress of the research (Remme et al 2010)

7 Defining research to improve health systems (Remme et al 2010) Research Domain Operational Implementation Health system Primary characteristics Focus Users Utility Operational issues of specific health programmes Implementation strategies for specific products or services Issues affecting some or all of the building blocks of a health system Health care providers, programme managers Programme managers, R&D managers Health system managers, policy makers Local Local/Broad Broad

8 Defining research to improve health systems (Remme et al 2010)

9 Examples of research questions for the three research domains (Remme et al 2010) Domain Operational Implementation Health system Question Which locations should be targeted for delivering HIV prevention services in Kawempe district, Uganda? Which of the current ART payment strategies in use in Nairobi should be retained for the new integrated programme? How to improve access to vaccination among children who are currently not reached by immunisation services? How to deliver ivermectin for onchocerciasis control and ensure sustained high treatment coverage in isolated rural communities? How effective are different policies for attracting nurses to rural areas? What has been the impact of the rapid scale-up of HIV programmes on fragile health systems?

10 Common interpretations of IR* Systematic approach to understanding and addressing barriers to effective and quality implementation of health interventions, strategies and policies. IR is demand-driven and the research questions are framed based on needs identified together with relevant stakeholders/implementers in the health system *WHO (2014)

11 Implementation research * addresses implementation bottlenecks, identifies optimal approaches for a particular setting, promotes the uptake of research findings leads to improved health care and its delivery. *WHO (2014)

12 Characteristics of Implementation Research* Systematic Multidisciplinary Contextual Complex *WHO (2014)

13 Characteristics of Implementation Research (WHO 2013) Systematic The systematic study of how evidence-based public health interventions are integrated and provided in specific settings, and how resulting health outcomes vary across communities. Balances relevance to real life situations with rigor, strictly adhering to norms of scientific inquiry. Multidisciplinary Analysis of biological, social, economic, political, system and environmental factors that impact implementation of specific health interventions. Interdisciplinary collaborations between behavioural and social scientists, clinicians, epidemiologists, statisticians, engineers, business analysts, policy makers, and key stakeholders. Contextual Demand driven. Framing of research questions is based on needs identified by implementers in the health system. Research is relevant to local specifics and needs, and aims to improve health care delivery in a given context. Generates generalizable knowledge and insights that can be applied across various settings. Mindful of cultural and community-based influences. Complex Dynamic and adaptive. Multi-scale: occurs at multiple levels of health systems and communities. Analyses multi-component programmes and policies. Non-linear, iterative, evolving process.

14 Why do we need implementation research?

15 Even when interventions are designed in similar ways, implementation occurs differently in different contexts, and with many different effects.. (WHO 2014)

16 The importance of implementation (WHO 2014) Efficacious tool Provider compliance Accessibility Patient adherence Effectiveness

17 The Prevention of Mother-To-Child-Transmission (PMTCT) cascade in Zambia ( )* *Ref: Mandala et al., 2009

18 There is growing funding for intervention research, particularly for drugs, vaccines.. However, there is still little funding for, and, therefore, a relative dearth of implementation research. (Sanders & Haines 2006)

19 Society s huge investment in technological innovations that only modestly improve efficacy, by consuming resources needed for improved delivery of care, may cost more lives than it saves. Health, economic, and moral arguments make the case for spending less on technological advances and more on improving systems for delivering care. (Woolf & Johnson 2005)

20 Neglecting implementation challenges costs lives and money. (WHO 2014)

21 Implementation research objectives and questions?

22 Implementation research objectives and questions* Research objective Explore Describe Influence Explain Predict Illustration of research question What are the possible factors and agents responsible for good implementation of a health intervention? For enhancing or expanding a health intervention What describes the context in which implementation occurs? What describes the main factors influencing implementation in a given context? Is coverage of a health intervention changing among beneficiaries of the intervention? How and why does implementation of the intervention lead to effects on health behaviour, services, or status in all its variations? What is the likely course of future implementation? *Peters et al 2013

23 IR Traditions, Research Questions, and Initial Core Disciplines (Peters et al 2013) Implementation Research Tradition Dissemination and implementation of evidencebased Medicine Participatory action research Management improvement Operational research Typical research questions What promotes the integration of research findings and evidence on interventions into healthcare policy and practice? How can we (community members as research participants) learn and be empowered to take action? How are the right services delivered to the right clients while meeting the right standards for quality? Which solution provides the most rational basis for a decision concerning the optimal performance of a system? Policy implementation How was a policy implemented, and what contributed to reaching the objectives or other effects? Which actors are involved in programme delivery in specific locations, how do they understand the problem of implementation, and what influences their behaviour? Core disciplines at origin Epidemiology, behavior change (psychology, sociology, education) Social psychology, education, and anthropology Management, engineering Management, engineering, math Political science, public policy, public administration

24 Constructs/variables for implementation research?

25 (proliferating) theoretical frameworks for IR? Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice: models for dissemination and implementation research. American Journal of Preventive Medicine, 43(3), doi: /j.amepre Flottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M.,... Eccles, M. P. (2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci, 8, 35. doi: / Meyers, D. C., Durlak, J. A., & Wandersman, A. (2012). The quality implementation framework: a synthesis of critical steps in the implementation process. American Journal of Community Psychology, 50(3-4), doi: /s x Cane, J., O'Connor, D., & Michie, S. (2012). Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci, 7(1), 37. doi: / Hendriks, A.-M., Jansen, M., Gubbels, J., De Vries, N., Paulussen, T., & Kremers, S. (2013). Proposing a conceptual framework for integrated local public health policy, applied to childhood obesity - the behavior change ball. Implementation Science, 8(1), 46. Chaudoir S.R., et al. Measuring factors affecting implementation of health innovations: A systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science 2013; 8:22 And more..

26 Consolidated Framework for Implementation Research (CFIR)* Inner setting Outer setting Intervention Individual Process *Source:

27 Consolidated Framework for Implementation Research (CFIR)* Intervention Outer setting Inner setting Individuals Process Intervention source Evidence strength & quality Patient needs & resources Cosmopolitanism Structural characteristics Networks & communications Knowledge & beliefs Self-efficacy Relative advantage Peer pressure Culture Individual stage of change Adaptability Trialability Complexity Design quality & packaging Cost External policy & incentive *Source: Implementation climate Readiness for implementation Identification with organization Other personal attributes Planning Engaging Executing Reflecting & evaluating

28

29

30 Implementation outcome variables (Peters et al 2013; WHO 2014) Variables Acceptability Adoption Appropriateness Feasibility Fidelity Implementation cost Coverage Sustainability Working definition The perception among stakeholders (for example, consumers, providers, managers, policy makers) that an intervention is agreeable The intention, initial decision, or action to try to employ a new intervention The perceived fit or relevance of the intervention in a particular setting or for a particular target audience (for example, provider or consumer) or problem The extent to which an intervention can be carried out in a particular setting or organization The degree to which an intervention was implemented as it was designed in an original protocol, plan, or policy The incremental cost of the implementation strategy (for example, how the services are delivered in a particular setting). The total cost of implementation would also include the cost of the intervention itself The degree to which the population that is eligible to benefit from an intervention actually receives it. The extent to which an intervention is maintained or institutionalized in a given setting

31 Common methods for Implementation research?

32 Methods particularly suitable for IR (Peters et al 2013) Mixed methods Participatory action research Quality improvement studies Effectivenessimplementation hybrid trials Pragmatic trials

33 Differentiating Implementation research?

34 Spectrum of implementation research relevance (Peters et al 2013) Implementation not relevant Implementation relevant but not considered Implementation relevant but effects reduced Context Controlled Largely controlled Real world & partial control IR Variables Not relevant Assumed to be controlled or not relevant Examples Basic science, Phase I and II clinical trials Efficacy study, Phase III RCT Controlled Pragmatic trials, Quasi experimental study with comparison Implementation studied as contributing factors Real world & population Independent variables Participatory research, Observational studies assessing implementation Implementation as primary focus Real world & population Primary outcome/ determinants Mixed method to determine acceptability

35 Key questions to assess research designs or reports on implementation research (Peters et al 2013) Does the research clearly aim to answer a question concerning implementation? Does the research clearly identify the primary audiences for the research and how they would use the research? Is there a clear description of what is being implemented (for example, details of the practice, programme, or policy)? Does the research involve an implementation strategy? If so, is it described and examined in its fullness? Is the research conducted in a real world setting? If so, is the context and sample population described in sufficient detail? Does the research appropriately consider implementation outcome variables? Does the research appropriately consider context and other factors that influence implementation? Does the research appropriately consider changes over time and the level of complexity of the system, including unintended consequences?

36 Key references?

37

38

39 Discussion time!

40 On-going developments Postgraduate scholarships for Implementation research Implementation research workshops Implementation research Massive Open Online Course Ethics in implementation research Reporting guidelines for implementation research Implementation research grants Updating of implementation research toolkit National strategy for implementation research

41