ACORD (Agency for Cooperation and Research in Development) Enhancing Iddir s Engagement in Slum Upgrading in Dire Dawa, Ethiopia (2012-2015) End of Project Evaluation Terms of Reference 1. Introduction and Background ACORD (Agency for Cooperation and Research in Development) is an Africa-led organisation working for social justice and development in Africa. For the last three decades ACORD has been working with partners, grassroots organizations and communities across Africa and is currently present on the ground in 17 countries. ACORD has been working in Ethiopia since 1994 and its activities in the country are governed by a specific agreement signed with the Government of Ethiopia. ACORD Ethiopia has designed and implemented long term urban and rural programmes that support community s own development and poverty reduction strategies. ACORD Ethiopia works in partnership with more than 300 community based organizations such as Iddirs (traditional burial associations) in urban areas and self help groups such as community managed disaster risk reduction committees in pastoral and agro pastoral areas. It also involves the participation of relevant government institutions and local administrative structures. Today ACORD works in three regions in Ethiopia with agro-pastoral communities in Boranna region as well as urban and peri-urban communities in Addis Ababa and Dire Dawa. With funding from Comic Relief ACORD Ethiopia in partnership with two Iddirs councils Sabiyan Tesfa Chora Charity Association and Goro Yegera Gudayoch Ageleglot Maheber have recently completed a 3 year project Enhancing Iddir s Engagement in Slum Upgrading in Dire Dawa, Ethiopia with the aim of improving the livelihood, food security and living conditions of disadvantaged groups of urban slum dwellers. The project addresses the basic material needs of slum households in relation to nutrition and food security, living conditions, kitchen and sanitation services as well as livelihood development. The project also promotes greater engagement between slum dwellers and local government structures. This evaluation has been commissioned to evaluate achievement against the 5 outcomes outlined in the grant start up form, specifically that: 1. By the end of Year 3, 600 slum dwellers in disadvantaged areas of Dire Dawa 1, have increased their income and food security 2. By the end of year 3, Iddirs are better organized at the kebele and village levels in slum areas to provide low cost quality sanitation services and to organize community managed common kitchen services for vulnerable households 1 Goro-Sabian Sub City
3. By the end of year 3, two Iddirs councils organized at sub city level are providing low interest credit services to 300 slum households to improve their housing conditions 4. By the end of year 3, Iddirs and Iddirs councils are organized and networked at kebele and city state levels to reflect the voice, concerns and interest of their slum household members in slum upgrading and ensuring their active participation in urban planning processes 5. By the end of year three, Iddirs and the women empowerment committees have increased capacity to protect vulnerable slum dwellers from social ills such as drug abuse, domestic violence and HIV 2. Overall Evaluation Objective The general objective of the planned evaluation is to learn from the work and to understand both what difference the project has made and HOW it has made the difference i.e understanding the key success factors for achieving long term change for the particular target group of urban slum dwellers. During the evaluation, special attention will be paid to sustainability of the interventions and appropriateness of the strategies used, so as to make recommendations for future programming, strategies and activities which would eventually contribute to sustainable improvement in the livelihoods of target population. 2.1 Key Priority Areas of Evaluation - Coherence: Establish the extent to which the project was linked with other interventions aimed at addressing the needs of vulnerable groups supported by ACORD Ethiopia and also establish the contribution made towards ACORD Ethiopia strategic plan - Efficiency: Assess the timeline of outputs, their delivery within budget, overall management arrangements, as well as compliance with the terms and conditions of the grant. - Equity: Reflection on the fairness in which resources were allocated to implement various aspects of the project so as to achieve the project outcomes - Effectiveness: Assessment of performance in relation to targets set in the Grant start up Form and the rate and cost at which inputs resulted in outputs. - Relevance to the people s need: Investigate whether the most vulnerable slum dwellers have been targeted and whether the project outcomes have been relevant to their needs. Focus should be placed on the relevance of the project to the people s needs as well as context in the project area. - Long term Impact: Details of the broader economic, social, and political consequences of the project and how it has contributed to the overall objectives of Comic Relief. Focus should be placed on outlining the positive and negative impacts that have come about as a result of the activities of the project or progress towards the same. - Sustainability: Potential for the continuation of the impact achieved and of the delivery mechanisms beyond the funding period. The focus should be placed on mechanisms that have been put in place to ensure sustained community benefits without external funding. - Replicability: How replicable is the process that introduced the changes/had impact in a similar context and elsewhere in the country - Risk management: Identify key risks identified before and during the project implementation and to what extent they were handled and outline key issues that may inform future programme designs - Cross cutting issues: Identify cross cutting issues that were key in the process of implementation and worth considering in informing programme improvement and delivery of future programmes - Lessons Learnt and sharing of experiences: Key lessons learnt throughout the period of the project which can be utilised to guide future strategies
- Recommendations: Recommendations for future interventions to consolidate and sustain the achievements based on observations during the evaluation process and the interviews with partners working with slum dwellers 2.2 Area of learning to focus on WHAT difference has the project made to slum dwellers lives (what, who, where, when)? - THE OUTCOMES: to what degree have the project outcomes been achieved? Were there any unexpected outcomes - THE BENEFICIARIES: who has benefited (men, women, girls and boys) and in what ways - NEEDS: are those changes (outcomes) relevant to people s needs? - SUSTAINABILITY: are they likely to be sustainable in the long run and which exit strategies have been put in place to ensure community-level sustainability? - POLICY & PRACTICE CHANGE: have there been changes to policies, practice and attitudes of decision and policy makers to benefit the slum dwellers and community as well? Have any potential entry points been identified where Iddirs could engage in policy formulation, planning and implementation? - CONTRIBUTION TO BROADER CONTEXT: to what extent has the project contributed to the achievement of broader national and international policies, conventions, targets etc in Ethiopia? - EXTERNAL CONTEXT: to what extent has the achievement of the changes/outcomes been influenced by external context and other factors? To what extent has The Charities and Societies Proclamation law impacted on the work of ACORD and Iddirs engaged in the project? HOW has the project made a difference? Approaches used by the project and implementing organisation - METHODS &APPROACHES: what have been the most effective methodologies and approaches the organisation used to bring about changes in slum dwellers lives? What has worked and what has not? What lessons have been learned? Who have they been shared with and what is the implication? - ORGANISATION TO FUND: How has the type of organisation funded (International NGO) helped or hindered the delivery of lasting change based on the management process and delivery systems? - PARTNERSHIP: how have the relationships between partners and relevant stakeholders helped or hindered the delivery of change/outcomes? - SYSTEMS: how effective have the project s management, monitoring, learning and financial systems been? How have they helped or hindered the delivery of lasting change? - COST EFFECTIVENESS: has the project been cost effective and what is the evidence of the same? - EVIDENCE OF CHANGE: which building blocks were required to bring about the desired goal? How do project outcomes, results, accomplishments, preconditions and assumptions demonstrate the change process? Approaches used by Comic Relief - GRANT PROCESSES: how have CR s grant making and processes (definition of strategies, outcomes, and assessment of applications) helped or hindered the delivery of lasting change? - GRANT MANAGEMENT: how has CR s approach to grant management (individual work with grant holders, and learning activities with other funded organisations) helped or hindered the delivery of lasting change? - ORGANISATIONAL ASSETS: how has the way CR used its organisational assets helped or hindered the delivery of change (use of media, access to decision makers)? - OTHER SUPPORTIVE WAY: are there any other ways in which CR has helped or hindered the delivery of change?
2.3 Geographical Area to be covered by the evaluation and partners involved The evaluator together with support from ACORD and partners will gather information from the target villages of Kebele 02, Goro-Sabian Dire Dawa. The evaluator will visit and discuss with slum dwellers who received support from the project, primary Iddirs and Iddirs Councils supported by the project, members of local governments involved in the project, community members, ACORD and Sabiyan Tesfa Chora Charity Association and Goro Yegera Gudayoch Ageleglot Maheber staff, other institutions and stakeholders working with slum dwellers. Discussions with Comic Relief s Programme Manager for Urban Slums can be organised virtually. 3. Methodology The participatory methodology will start with an induction session/process which will be undertaken by project staff to familiarise and guide the consultant on the evaluation s expectations. The consultant(s) will then employ their knowledge and expertise in participatory evaluation methodologies to best capture the contributions of project partners and beneficiaries. A detailed external assessment methodology will be submitted by consultants in a process of tender. The evaluation process should include: - A desk review of project information including the key documents provided (Proposal documents, grant start up form, outcomes and learning form, annual and mid-term review reports, project feedback documents [from CR], research reports, etc) - Semi structured (FGDs and KIIs) and structured interviews with different stakeholders including beneficiaries, community members, ACORD staff and partners to generate both quantitative and qualitative information - Documentation of case studies, lessons learned and best practices - Data analysis (using SPSS or any other appropriate statistical package) - The Evaluator(s) should present a preliminary overview of their findings to ACORD and receive comments from stakeholders before preparing the draft evaluation report. - Evaluators should submit the draft report to ACORD for written comments before presenting the findings during dissemination workshop and thereafter finalising the report, to minimise the chance of inaccuracies and to maximise ownership of the findings. 4. The Evaluation s Expected outputs and Outcome Outputs - Power-point presentation of not more than 15 slides - Presentation of draft report during dissemination workshop with ACORD and its partner on the findings of the evaluation. - A clear and concise simply-written, free of jargon evaluation report (not more than 30 pages) excluding preliminary pages, executive summary and annexes/appendices. The final report will be submitted to ACORD in English (1 hard copy and a soft copy on CD). Outline of the evaluation report The main body of the evaluation report will not exceed 30 pages and contain the different elements mentioned below - Cover page - Table of contents - Abbreviations and acronyms page - An executive summary that can be used as a document in its own right. It should include the major findings of the evaluation and summarise conclusions and recommendations.
- A short introduction to the project, the objectives of the evaluation and methodology - A justification of the methods and techniques used (including relevant underlying values and assumptions, theories) with a justification of the selections made (of persons interviewed, villages or projects visited). - Eventual limitations of the evaluation. - A presentation of the findings and the analysis thereof (including unexpected, relevant findings). All key priority areas and learning points above should be addressed, paying attention to gender issues - Conclusions, which will analyse the various research questions. Conclusions have to be derived from findings and analysis thereof. - Recommendations should be clearly related to conclusions and include how they might be implemented - Recommendations should be practical and if necessary divided up for various actors or stakeholders. The report will be accompanied by an analysis of up to 2 pages giving views of ACORD on the quality of the evaluation, the relevance and usefulness of the recommendations and what actions they propose to take in response as well as the dissemination plan Report appendices that include: - The Terms of Reference. - The technique used for data collection (including list of informants and locations visited) - The list of questions used or interview guide or topic list (also for possible group discussions). - The programme adherence (data and main features of the activities carried out, organogram). - Concepts and list of abbreviations. - List of documents and bibliography. - Composition of evaluation team and their profiles Outcomes and use of the report ACORD will learn and reflect on the documentation drawn from the evaluation capturing the expected and achieved results, outcomes and impact, achievements, lessons learnt and challenges. This learning process will serve as an input resource for ACORD Ethiopia s strategic planning and proposal development. The learning process and documentation will also be used to strategically influence development practices at local and national level. The final report will be used internally and externally as a learning and experience sharing tool 5. Evaluation Duration and Reporting. A total of 25 calendar days will constitute the evaluation period; 5 days for desk review and to prepare the research tools, an estimated 10 days for field work, and 10 days for compiling the information gathered, analyse, draft and complete the evaluation report. The consultant will arrange for a feedback meeting to share a draft of the evaluation report with the programme staff. A final report in English will then be provided after including the comments. 6. Profile of Consultant The consultant will work with an ACORD resource person and will be charged with the responsibility of undertaking the overall reporting and conclusion of the evaluation. The consultant must possess the following minimum qualifications; - Masters or PhD in economics, international development, social sciences, or other relevant development field of study (Essential) - Demonstrable experience minimum 8 years in conducting programme evaluations, with proven analytical skills (Essential)
- Experience and knowledge of urban development and urban slum populations (Essential) - Experience of institutional capacity building and participatory development programmes. (Essential) - Experience of socio-economic advocacy tools and approaches. (Essential) - Experience of working with local communities and non-governmental organisations. (Essential) - Experience in the design and use of participatory methods for assessment and evaluation. (Essential) - Fluency in English and Amharic. (Essential) - Expertise on Ethiopia civil society and economic context. (Essential) - Desirable knowledge of INGO s work and Human Rights based approaches. (Desirable) 7. Evaluation fees and modalities of remuneration The consultant and ACORD will agree on the remuneration rate per person/day. Charges related to taxes, mission allowances for fieldwork and secretarial fees are to be born by the evaluator. ACORD will cover the transportation costs to and from the field assignments, and accommodation during the field work. Remuneration will be agreed with the selected consultant/firm and pays to be calculated on basis of days the Evaluator has worked. 8. Expression of Interest To apply, please send a technical and financial proposal to ACORD Ethiopia by email to acord.eth@ethionet.et and Kassanesh.Misganaw@acordinternational.org with a copy to recruit@acordinternational.org by the 28 th December 2014. The proposal should contain: 1) A letter of interest addressed to the ACORD Ethiopia Country Director 2) A Technical offer showing the interpretation of the ToRs and indicating a detailed proposed methodology for the work demanded 3) CV of the consultant or CVs of consultancy team in case of a firm, showing previous relevant experience 4) A detailed financial offer (expressed in Ethiopian Birr indicating the cost required for the undertaking of the work required, estimated to take 25 days). Please also include your renewed business licence, VAT and TIN registration licences. For delivery by hand or post please use the following Address: ACORD Ethiopia, Bole Road, Behind Bole Mini, PO BOX 12377, Addis Ababa, Ethiopia. Telephone +251 11 6181566 Please note only shortlisted consultants will be contacted. ACORD Reserves the right to cancel part or the entire consultancy service.