TERMS OF REFERENCE FOR THE FINAL EVALUATION OF HAKI MKONONI RIGHTS IN OUR HANDS January 2017 Programme/project title /affiliate identification code Haki Mkononi Rights in our hands / R05168 PAFA35 Geographical coverage: Continental regional partner (SOAWR), Nigeria, Uganda, Tanzania, Liberia, Rwanda and South Africa Programme/project budget GBP 1,914,601 (including GBP 174,055 Oxfam s contribution) Evaluation commissioning manager Evaluation manager 1. Background, rationale and purpose of the evaluation Pan Africa Gender Justice Lead Pan Africa Programme Quality Lead These are the Terms of Reference for the final evaluation of the Haki Mkononi (HM) programme coordinated by Oxfam Pan Africa Programme. HM is a three year programme ending on 31 st March 2017. It is principally funded by the Swedish International Development Cooperation (SIDA) to the total amount of GBP 1,740,546 and Oxfam s contribution of GBP 174,055. Oxfam and partners are implementing a project entitled A regional empowerment programme on women rights. The overall goal of HM is to ensure African women realize their rights as guaranteed in the African Women s Protocol. Managed by the Oxfam Pan Africa Programme, the project seeks to accelerate the realization of women s rights by developing the capacity of Women Rights Organizations, legal mechanisms, national and regional structures using the Protocol to address women issues and ensure compliance towards its implementation. Central to this overall goal are key outcomes as follows; 1. 43 women s rights organizations have improved capacity to use the African Women s Protocol to address women s rights violations through regional mechanisms 2. Regional institutions (the African Union and its organs, RECs) call on state parties to adhere to obligations under the African Women s Protocol 3. The Protocol and other regional declarations on women s rights are increasingly applied by national governments and national justice mechanisms The project focus countries are Nigeria, Liberia, Uganda, Tanzania, Rwanda and South Africa with interventions at the continental level informed by country level work and experiences. Implementations and linkages of focus countries to the regional platforms are mainly ensured by the Equality Now, the secretariat to over 46 Women Rights Organizations championing compliance to African Women s Protocol and improving women rights in the continent. Sustainable changes in policies, practices and beliefs will often need longer to translate into measurable effects and even more so at national and regional levels. Therefore, the main aim of the evaluation will be to systematically analyse the actual outcomes of the programme and its underlying working mechanisms 1
and processes against the proposed outcomes and Theory of Change rather than the impact level. Where available, impact level data from existing sources (including related secondary source findings) should inform the evaluation. 2. Specific object and objectives of the evaluation The purpose of this final evaluation is two-fold: i. Building institutional knowledge: The evaluation will inform the development of current and future gender justice (and other multi-country) programmes and projects. To maximize learning for all those involved, the methodology for the final evaluation will ensure the active involvement from Oxfam staff, partners and women. ii. Accountability: The evaluation will ensure Oxfam s practice of accountability to the different stakeholders in this project including but not limited to Oxfam in-country teams, CSOs, Sida, national and regional administrative and policy making teams etc. The specific objectives of the evaluation are: a. Stimulate reflection and learning among country offices, partners and women s rights organisations, including learning from failures and challenges. b. Review and validate the achievements reached under each outcome as presented in the HM annual reports (and underlying documents like bi-annual monitoring reports). c. Validation of the Theory of Change of this project and its underlying assumptions. This means describing the process of how the changes in policies, practices and beliefs have been achieved, and analysing this against the Theory of Change of the project, including the underlying assumptions. d. Developing concrete recommendations for future programme and project development on gender justice and for multi-country programmes in general. 3. Key questions of the evaluation The below evaluation questions are in line with the reporting requirements of Sida (account and analysis of outcomes, analysis of factors that affected implementation and sustainability). A. What are the principal outcomes/contributions of the HM programme as a whole? (Impact) B. Did the approach suit the priorities and needs of the diverse groups of women and girls and their communities? Was it inclusive enough for harder-to-reach women and girls? (Relevance) C. Based on the available evidence, did the achieved outcomes lead to long-term improvements and are the exit strategies adequate? What is the likelihood that the benefits of the programme are sustained upon its completion? (Sustainability) D. Was the chosen approach and its practical implementation carried out in an economical and efficient way? (Efficiency) E. Was there an advantage of conceiving this intervention as a multi-country programme? How did the regional layer contribute to its success? (Effectiveness) NB Note that evaluation questions will be further fine-tuned throughout the process to ensure they respond to the specific information needs of our key stakeholders and purpose of evaluation. In answering the above questions, the evaluators will also consider whether and how the use of new technologies (online platforms, radio, TV, mobile, social media) contributed to the outreach and mobilisation of women and girls. 4. Scope of the evaluation and approach and methods, establishing the basic methodological requirements (if applicable) 2
The evaluation will be undertaken in Nigeria, Tanzania, Liberia, Rwanda, Uganda and South Africa targeting the communities and power holders. It is anticipated that a mix of qualitative and quantitative data collection methodologies will be used to gather data for the evaluation. Additional primary data for triangulation and verification will be collected during the country visits. The consultant is expected to establish a representative sample for the exercise using a scientific method in the 6 countries. He/she is expected to develop effective evaluation tools for evaluation but with modification to respond to the evaluation questions. Scope The evaluation will encompass the whole HM programme from 1 st August 2014 up to 31 st March 2017. The evaluation will include a validation of the results of the programme as presented in the annual reports of 2015, 2016 and 2017. The following basic principles are expected to guide the further development of the methodology: i. Participation: Allow for the meaningful participation of project stakeholders (i.e. Oxfam staff, partners and women etc.) in the evaluation process. ii. Data triangulation: The evaluators will gather information through different complimentary sources (outcome harvesting, process monitoring, but also direct observations of HM activities, key informant interviews, small survey data collected with help from partners/ WRO s). The detailed methodology proposed by the consultants should at least contain the following components: (i) (ii) (iii) (iv) (v) (vi) (vii) Train participants (WRO s, partners, staff) in function of what is needed to actively take part in the evaluation Fine-tuning of evaluation questions to the national context, together with participants Identification and documentation of the achieved outcomes by participants The methodology should include verification mechanisms to increase reliability of the documented outcomes (peer checks, solicitation of additional information from key source) Summary and interpretation of achieved outcomes with participants Presentation and discussion of preliminary findings At the end of the in-country visits, the consultant will present the preliminary findings from the document review, participatory workshops and any other data collection methods to the Oxfam country staff and selected partners. 5. Evaluation team: qualifications and skills needed, plan for organizing the evaluation team We are looking for a team of two/ (or more) experienced evaluators who are able to work in parallel during the in-country visits. One principal consultant will take the overall lead and responsibility. The team should be gender-balanced and should possess the following qualifications: a) Demonstrated experience in conducting multi-country evaluations (Desirable) b) Experience with facilitating participatory processes related to Monitoring & Evaluation, especially Theory of Change and Outcome mapping/ harvesting or similar c) Experience with programmes in least one, and preferably several of the following domains: Policy change, Gender Based Violence, Conflict Resolution 3
d) Excellent communication, writing and presentation skills in English e) Demonstrated understanding of issues related to gender equality, specifically with regards to women s rights (Desirable) f) Knowledge of the work of Oxfam or other international NGO s (Desirable) 6. Schedule, budget, logistics and deliverables. Include outline of the evaluation report (see below) Proposed timing The evaluation process is expected to complete as at 24 th March 2017, beginning on 20 th February 2017. Consultants should provide detailed work plan towards the implementation of this initiative. Budget All costs should be included in the quoted price of the contractor. Logistics Oxfam Country Leads will assign a responsible point person to arrange accommodation, transport and translation services. They will receive support from the Evaluation Manager. The country offices will also assist with the practical organisation of workshops and scheduling of interviews Key deliverables Deadline Deliverable Short inception report with a summary of the final methodology and 3 Days after meeting detailed work plan as agreed during the inception meeting (to be signed off by Reference Group) A clear and concise draft evaluation report, including: 6 th March 2017 - Executive summary that can be used as stand-alone document - Explanation and justification of used methodologies, including the perceived limitations - Findings from the document review - Documentation of findings from in-country visits (without identification of specific individuals) - General conclusions, including a clear and well-reasoned answer to the evaluation questions and recommendations: What should we do differently in the future? - Appendices: o Signed ToR and inception report with final work plan o List of participants, interviewees and locations visited o Outcomes per country identified during workshops o Raw datasets (interview notes, data worksheets) Final Evaluation Report (in line with the above requirements) 21 st March, 2017 7. Evaluation responsibilities and management arrangements The PAP Gender Justice Lead and Programme Quality Lead will be the Evaluation Managers. The Programme Quality Lead will be the first point of contact for the evaluation team and ensure the team has access to documents, people and other information needed to answer the evaluation questions. 4
The evaluation manager will be part of a broader Reference Group ensuring the quality of the evaluation and adherence to Oxfam s and back donor s procedures and requirements. The Reference Group will take role in the selection of the team of consultants, give input to the methodology during the inception meeting and approve later changes to evaluation work plans and budget. The Oxfam Country Leads will provide support during country visits, including for the logistical aspect. They can introduce the evaluators to partners and other relevant stakeholders. 8. Dissemination strategy, plan and responsibilities for sharing and using the findings Before the end of the country visits, the evaluators will share and discuss the preliminary evaluation results and initial recommendations with national staff, women s rights organizations and partners. In close collaboration with the Pan Africa Communication Officer/ Global Programmes and Communications Advisor, a specific action plan will be developed to ensure the dissemination of the evaluation results to WRO s and other colleagues. The plan will also indicate how the findings will inform the development of new gender justice and other programmes, organisational learning and advocacy. The Executive Summary and final evaluation report will be in English according to a mutually agreed outline. A copy of the final evaluation report will be published on Oxfam s website and will be made available to programme stakeholders, including, but not limited to, Sida. This is consistent with Oxfam s commitment to transparency and accountability. 9. Process of the selection of the evaluator or evaluation team and expectations for evaluation proposal Interested consultants should send their applications to hecaconsultancy@oxfam.org.uk on or before 14 th February 2017. The proposal should include proposed methodology for undertaking the evaluation, interpretation of the ToR, proposed sampling methodology, detailed work plan, budget and qualifications for the evaluation team. The proposal should not be more than 15 pages in total and should indicate availability of the evaluation team. 5