Independent Evaluation (IE) of Phase 1 of the Affordable Medicines Facility - malaria

Transcription

1 Presentation to Institute of Medicine Workshop on Evaluation Methods for Large-Scale, Complex, Multi-National Global Health Initiatives January 7 th, 2014, London Independent Evaluation (IE) of Phase 1 of the Affordable Medicines Facility - malaria

2 Antimalarial access the problems Artemisinin-based combination therapies (ACTs) highly effective, recommended treatment for malaria Despite free or highly subsidised public sector provision, access remained very poor, reflecting: Frequent public sector ACT stockouts 40-60% seek treatment from private sector retailers ACT retail prices are many times higher than those of common alternative antimalarials Most retail customers purchase less effective antimalarials (e.g. chloroquine, SP), artemisinin monotherapies, or just painkillers Concern that use of oral artemisinin monotherapies could exacerbate development of artemisinin resistance

3 The Affordable Medicines Facility malaria (AMFm) Two goals: 1. Contribute to malaria mortality reduction 2. Delay resistance to artemisinin Four objectives: 1. Increasing availability of quality-assured ACTs 2. Increasing affordability of quality-assured ACTs 3. Increasing market share of quality-assured ACTs 4. Increasing use of quality-assured ACTs

4 AMFm comprises three elements 1. Price negotiations with ACT manufacturers 2. Buyer subsidy (co-payments) at top of global supply chain 3. Supporting interventions to ensure effective ACT scale-up Public awareness and education Use of AMFm logo on all subsidised ACTs Recommended retail prices (RRPs) Provider training & supervision Strengthening regulation Enhanced pharmacovigilance

5 Location of AMFm Phase 1 Pilots ZANZIBAR Hosted by the Global Fund to Fight HIV/AIDS, Malaria and Tuberculosis, Geneva

6 Evaluation design Pre-and Post-test Design Baseline Assessment Intervention ( Financing platform in place and functional) Endpoint Assessment

7 Evaluation design Pre-and Post-test Design Baseline Assessment Intervention ( Financing platform in place and functional) Endpoint Assessment Outlet surveys on ACT availability, price & market share Outlet surveys on ACT availability, price & market share

8 Outlet Survey Data Collection Nationally representative outlet surveys at baseline and endline (6-15 months after first copaid drugs arrived in country) All categories of outlets with potential to sell antimalarials in private and public sector targeted

9 Examples of Antimalarial Outlets IMPACT2 team

10 Outlet Survey Data Collection Nationally representative outlet surveys at baseline and endline (6-15 months after first copaid drugs arrived in country) All categories of outlets with potential to sell antimalarials in private and public sector targeted Cluster sampling approach, stratified by urban and rural areas Sample size calculation based on detecting 20 percentage point change in QAACT availability

11 Evaluation design Pre-and Post-test Design Baseline Assessment Intervention ( Financing platform in place and functional) Endpoint Assessment Outlet surveys on ACT availability, price & market share Secondary household survey data on ACT use Outlet surveys on ACT availability, price & market share Secondary household survey data on ACT use

12 Household Survey Data Global Fund decided not to fund specific household surveys for AMFm evaluation, largely due to cost considerations Instead used other sources such as DHS, MIS and ACTWatch Appropriately timed baseline and endline household surveys were available for 5 of the 8 pilots Endlines took place months after first copaid drugs arrived in country

13 Evaluation design Pre-and Post-test Design Baseline Assessment Intervention ( Financing platform in place and functional) Endpoint Assessment Outlet surveys on ACT availability, price & market share Secondary household survey data on ACT use Documentation of key contextual factors Outlet surveys on ACT availability, price & market share Secondary household survey data on ACT use Documentation of AMFm implementation process & key contextual factors

14 Evaluation design Pre-and Post-test Design Baseline Assessment Intervention ( Financing platform in place and functional) Endpoint Assessment Outlet surveys on ACT availability, price & market share Secondary household survey data on ACT use Documentation of key contextual factors Outlet surveys on ACT availability, price & market share Secondary household survey data on ACT use Documentation of AMFm implementation process & key contextual factors Remote area study (Ghana, Kenya) AMFm logo study (Ghana, Kenya, Madagascar, Nigeria)

15 Success Metrics Determine how success would be assessed in relation to the AMFm outcomes E2Pi developed recommendations for success metrics for 1 year after the effective start date of AMFm Operationalized by the IE team

16 Objective Success benchmark Success Metrics Availability percentage point increase from baseline to endline in the percentage of outlets stocking ALL quality assure ACTs (QAACTs) (both with and without the AMFm logo) among outlets stocking antimalarials Price 2. Median price of one adult equivalent treatment dose (AETD )of AMFm QAACTs (with the logo) is less than 3 times the median price of one AETD of the most popular antimalarial in tablet form which is not a QAACT (in private for-profit outlets) 3. Median price of one AETD of AMFm QAACTs (with the logo) is less than the median price of one AETD of oral artemisinin monotherapy (AMT) tablets (in private forprofit outlets) Use 4. 5 percentage point increase from baseline in percentage of children under age 5 years with fever in the last 2 weeks who received ACT treatment Market share 5. Increase in market share of ALL QAACTS of 10 percentage points from baseline to endline 6. Decrease in market share of AMTs (all oral dosage forms) from baseline to endline

17 Strengths Limitations Eight operational pilots representing wide range of contexts Nationally representative outlet surveys, drawing on ACTwatch methods Standardized data collection and analysis across pilots Plausibility assessed using carefully documented process and context Evaluation by a team independent from those funding and implementing AMFm Short duration of AMFm implementation before endline outlet survey in some countries Secondary household survey data not available for all pilots Extrapolate with caution to countries with different antimalarial markets Areas beyond the IE scope include targeting by parasitemia status, patient adherence & prevalence of counterfeits Not possible to create comparison areas within pilot settings, and challenging to compare with non- AMFm countries

18 Potential comparators? Concern about comparability of others countries given variations in context and in implementation of other malaria control strategies Note selection bias given that AMFm countries had to meet a set of inclusion criteria Concern about potential contamination Nationally representative baseline and endline outlet surveys only available for 3 non-amfm countries (Zambia, Benin and Cambodia) which were highly imperfect comparators, so cost of collecting sufficient comparator data would have been high

19 Independent Evaluation Partners Institution Independent Evaluation ICF International LSHTM Outlet Surveys APHRC CRDH/CIERPA DNDi/KATH IHI - IMPACT 2 Project PSI - ACTwatch Project Country case studies Abdinasir Amin Catherine Adegoke Diadier Diallo Elizabeth Juma Sergio Torres Rueda IE members Country covered Kenya Niger Ghana Tanzania mainland Kenya, Madagascar, Nigeria, Uganda, Zanzibar Kenya Nigeria Niger Ghana Madagascar Tanzania mainland, Uganda, Zanzibar Institution Remote Area Surveys APHRC KATH AMFm logo study AIHD INSTAT TNS RMS Ghana TNS RMS Nigeria Limited Country covered Kenya Ghana Kenya Madagascar Ghana Nigeria

20 Acknowledgements Funding: The Independent Evaluation was funded by the Global Fund to Fight AIDS, Tuberculosis and Malaria. Additional funding for outlet surveys in three countries was provided by the Bill and Melinda Gates Foundation through its support of the ACTwatch project. We acknowledge and thank the following: The ACTwatch team for sharing their materials and experience Clinton Health Access Initiative for invaluable information about country implementation processes Global Fund Secretariat Research staff at ICF International and LSHTM Research staff in all 8 pilots Respondents in AMFm pilot countries Full report and supplement on ACT use available at: And outlet survey results published in Tougher et al, Lancet 2012

21 Success metrics: Summary Ghana Kenya Madagascar Niger Nigeria Tanzania mainland Uganda Zanzibar Benchmark percentage point increase in QAACT availability (p=0.99) 10 (p=0.99) 26 (p=0.14) Median price of QAACTs with AMFm logo is < 3 times the median price of the most popular antimalarial in tablet form that is not a QAACT (ratio) 3.0 (p=0.81) (p=0.99) (p=0.99) Median price of QAACTs with AMFm logo < median price of AMT tablets (difference, QAACT AMT) percentage point increase in percentage of children with fever who received ACT treatment Not available Not available 5.0 (p=0.50) Not available 6.7 (p=0.09) percentage point increase in market share of QAACTs (p=0.01) 8.6 (p=0.61) -8.8 (p=0.99) (p=0.23) 17 (p=0.08) Decrease in market share of oral AMTs (percentage point change) -3.9 (p=0.03) -12 Note that the outlet survey was not powered to detect a 10% change in market share in Madagascar, Tanzania mainland and Uganda, and the household survey was not powered to detect a 5% change in Madagascar. Data are not yet available to calculate p values for change in use in Tanzania mainland and Zanzibar.