Enhancing Urban and Peri-urban Agriculture in Liberia. Welthungerhilfe. European Union

Transcription

1 Enhancing Urban and Peri-urban Agriculture in Liberia AF 1582 / LBR 1026 EU: DCI-FOOD / 2009/ Implemented by Welthungerhilfe Co-financed by European Union Evaluation Report On behalf of Welthungerhilfe Barbara Jilg Hinterwössen, April 2013

2 Table of contents List of tables and diagrams... iii List of abbreviations and acronyms... iv I. Summary... v 1 Brief description of the project and framework conditions... v 2 Relevance... v 3 Effectiveness... v 4 Efficiency... vi 5 Outcomes and impacts... vi 6 Sustainability... vii 7 Recommendations... vii 8 General conclusions and lessons learnt... viii II. Main text Prior remarks Reason and aim of the evaluation Evaluation process Evaluation methods Description of the project Brief project description Framework conditions Brief description of the target group Relevance Relevance to core problems of the target group Relevance to the objectives of Welthungerhilfe Relevance to the objectives of the partner organisation and the partner country Relevance to the objectives of the European Union Effectiveness Achievement of project purpose Project preparation Quality of target group, actor and situation analysis Existent self-help approaches and structures Project planning Quality of the project planning matrix Analysis of the results chain Realism of project purpose Appropriateness of staff, material and financial planning Quality of the analysis of project holder (partners and external structure of Welth.) Organizational structure and focal points of work Staff, equipment and logistics Project steering and financial administration Steering by the head office, regional office and cooperation Quality of project execution Presentation and evaluation of the project outputs and activities Involvement of the target group and project cooperations Overall evaluation of the execution Occurrence of assumptions and risks Efficiency Cost / benefit ratio for the overall project Cost / performance ratio for the individual measures Quality of the internal project M&E system Direct and indirect, short, medium and long-term outcomes and impacts Use of outputs / outcomes i

3 5.1.1 Economic outcomes Socio-cultural outcomes Organizational and institutional / political outcomes Environmental outcomes Impacts Economic impacts Socio-cultural impacts Organizational and institutional / political impacts Environmental impacts Contribution of the project to the millennium development goals Sustainability Economic financial / food and nutrition security Socio cultural sustainability Organizational and institutional / political sustainability Environmental sustainability Recommendations Recommendations to Welthungerhilfe Recommendations to the project management General recommendations Recommendations regarding result Recommendations regarding result Recommendations regarding result Recommendations regarding result 4 and General conclusions Important lessons learnt Positive examples from the project / good practice Welthungerhilfe s specific annual theme for cross-cutting evaluation Annexes Annex 1: Terms of reference Annex 2: Project planning matrix Annex 3: Travel and work schedule Annex 4: Results of trend analysis Annex 5: Questionnaire and results of the questionnaire used during evaluation Annex 6: Break down of the budget for efficiency calculation Annex 7: Minutes of the de-briefing workshop Annex 8: Map of the project region ii

4 List of tables and diagrams Page Diagram 1: Reached target group 6 Diagram 2: Planned results chain 11 Diagram 3: Organizational chart of UPA 1 Project 14 Diagram 4: Comparison of planned and realized activities 17 Diagram 5: Comparison of planned and achieved objectives and results 24 Diagram 6: Average of the results of the trend analysis of 5 groups 29 Diagram 7: Beneficiaries view of the household income vs. income of other families in the region 30 Diagram 8: Number of meals per day 30 Diagram 9: Beneficiaries assessment of the variety of food during the previous month 31 Diagram 10: Save and Loan groups as good practice 38 iii

5 List of abbreviations and acronyms ACC Agricultural Coordination Committee ACF Action contre la Faim CAC County Agricultural Commissioner CFT Community Farmer Trainer ECHO European Commission Humanitarian Aid & Civil Protection EU European Union FAO Food and Agriculture Organization FLUPFA Federation of Liberian Urban and Peri-Urban Farmers Associations FRC Farmers Resource Centre FSF Food Security Facilitator FUN Farmers Unions Network HDF Human Development Foundation HDI Human Development Index IFAD International Fund for Agricultural Development LRD Liberian Dollar (1 LRD = 0,010 ; ; LISGIS Liberian Institute for Statistics and Geographical Information Systems MCC Monrovia City Council M&EO Monitoring and Evaluation Officer MOA Ministry of Agriculture MO Marketing Officer MPAP Multi-Stakeholder Action Planning and Policies Formulation MSF Multi-Stakeholder Forum MSFO Multi-Stakeholder Forum Officer NGO Non-Governmental Organization ROM Result oriented monitoring RUAF Resource Centres on Urban Agriculture & Food Security Foundation TUPUFU Tubmanburg Urban and Peri-Urban Farmers Union UL (UOL) University of Liberia UPA Urban and Peri-Urban Agriculture UPA 1 Evaluated project Enhancing urban and peri-urban agriculture in Liberia UPA 2 Follower project of UPA 1 S&L groups Save and Loan groups VS&L groups Village Save and Loan groups WOCHIDO Liberian NGO iv

6 I. Summary 1. Brief description of the project and framework conditions The project Enhancing urban and semi-urban agriculture in Liberia (UPA 1) formulated five results to be reached during project implementation: Result 1: UPA is accepted and actively supported by relevant ministries and municipal authorities Result 2: Organization and formalization of UPA farmer groups is strengthened, UPA farmer groups have increased visibility and political influence Result 3: Agricultural production is enhanced and diversified Result 4: Beneficiaries process agricultural and other products and realize value adding activities and other income generating activities Result 5: Marketing by UPA farmers and other beneficiaries is improved. A follower project is already working, with an overlapping of nearly one year. The direct target group was about 1,008 families: 785 beneficiary families participating in production, processing and marketing groups and about 223 backyard gardeners in Monrovia and Tubmanburg, what correspond to 5,040 family members as indirect beneficiaries. Additionally members of 8 institutional groups (schools, prison, orphanage and students) have been benefited. The majority of the target group are women. The project is co-financed by the European Union and was implemented by Welthungerhilfe in partnership with the International Resource Centres on Urban Agriculture & Food Security Foundation - RUAF (result 1), HDF - Human Development Fund (result 3) and WOCHIDO (result 4). For two of the five planned results (result 1 and 5) Welthungerhilfe worked in cooperation with CARE. Framework conditions Since the end of 14 years of civil war in 2003, Liberia is experiencing a period of slow improvement, but it remains one of the world s poorest nations with estimated two-thirds of Liberians living below the poverty line. During the war the population concentrated in the relatively safe capital Monrovia, whose population grew to more than 1/3 of the total population of the country. Urban and periurban agriculture offers large potentials to improve the situation of urban habitants, but there are also serious challenges, such as unsecure land rights or litter and waste water. 2. Relevance The project is relevant for the target group (food shortage, increasing food prices and high unemployment), for Welthungerhilfe and their implementing partners and also for Liberia (Poverty Reduction Strategy) and for the European Union. Only the University of Liberia, one of the partners, did not give priority to the project measures. Two important difficulties of the target group, lack of waste management and lack of irrigation possibilities in the gardens have not been considered in the project proposal. 3 Effectiveness It can be summarized that in general the project purpose: Small scale urban and peri-urban producers improve food security and increase income and UPA is officially recognized was achieved. It was possible to realize most of the planned activities, the activities of result 4 (processing) and some activities of result 5 (farmers market and marketing stand in Tubmanburg) have been realized with delay and their implementation needs to be completed during the follower project UPA 2. In the processing activities the delay was caused through difficulties of the local partner responsible for processing activities in the beginning of the project. v

7 The process of organizing urban farmers in FLUPFA and TUPUFU has been an important step towards the representation of urban farmers in the Multi-Stakeholder Forums and towards advocacy, but these organisations are still new and need further support through project measures. Extension service for the groups and backyard gardeners suffered from the fact that the Community Farm Trainers (CFTs) refused to provide voluntary training and extension work. In many cases it was not possible to accompany the groups and backyard gardeners closely enough to support them in resolving their production problems. Examples are pests in the gardens and the lack of fodder availability which is limiting the success of the animal husbandry component. The establishment of Save and Loan groups has been a matter of particular interest of the target groups; they seem to be sustainable. The cross cutting themes gender and HIV / AIDS have not been considered sufficiently. 4 Efficiency The project costs per family (more than 1,000.- in three years) and per person are high, compared to their benefits. Even considering the indirect benefits of improved knowledge and awareness about nutrition and UPA in the society and strengthening of the local partner organisations, the project remains expensive. For most of the participating families the activities do not produce benefits comparable to the costs per family. Possible benefits in the future such as improvements through the weekly market, processing activities or results of the lobbying of the UPA farmer organisations FLUPFA and TUPUFU were not considered in this calculation and may improve the cost / benefit ratio towards the end of the follower project UPA 2. Quality of the internal project M&E system The monitoring is a weak point of the project; the indicators of the logframe have not been measured in a systematic way. In the monthly reports of the staff there is information about the realized activities, but no information showing an overview of realized vs. planned activities or reached results. 5 Outcomes and impacts The agricultural and animal husbandry production was diversified and improved to a more ecological and more market oriented production among a part of the target group. The concept of low external input agriculture caused some positive environmental impacts, such as the conservation of soil fertility and positive impacts on human health and the environment through a reduced use of chemical pesticides. It can be estimated that about 60% of the target group (up to 500 families) which continued in project activities until the end of the project and some of the persons which did not continue improved and diversified their production. Additionally most of the eight institutional groups improved their production in different degrees. At present, three processing groups which are not farming (135 families) and three processing groups composed of UPA farmers have initiated their work. The results are still in their initial stage but more improvements can be expected in the future. vi

8 Market links for many group members and backyard gardeners have been established and one market stand in Monrovia has been constructed where about three to four traders are selling fruits and vegetables. In Tubmanburg a second market stand started to be constructed. Two Multi-Stakeholder Forums have been established in Monrovia and Tubmanburg and the importance of urban and peri-urban agriculture (UPA) is well accepted by governmental institutions and municipal authorities and first steps of active public support to farmers have been realized. However at this moment the concrete support is still limited. Project trainings and other measures strengthened 45 UPA farmer groups. Processing and Save and Loan groups and two umbrella organizations of UPA farmer groups (FLUPFA and TUPUFU) have been founded. The farmers organizations TUPUFU and FLUPFA are representing farmers interests in the Multi-Stakeholder Forums and at other opportunities. This process is relatively new and did not yet produce many tangible impacts for the farmers. Another relevant outcome are the increased knowledge and skills of the majority of the target group (more than 1000 persons) about agricultural and animal husbandry production, management, UPA and other topics. Some beneficiaries also mentioned increased solidarity and togetherness among group members. Especially the Save and Loan groups increased the self-help potential of their members. It can be estimated that about 60% of the target group increased their income through project activities. Food security was improved clearly, in terms of stability or diversification, among up to 70% of the beneficiaries. 6 Sustainability The sustainability of the individual gardens and partly also of the group gardens seems to be high, other results of project activities, such as FLUPFA and TUPUFU, the processing activities, the weekly market, animal husbandry activities and maybe the MSF at this moment are not yet able to continue without external advisory or financial support. The project UPA 2 is working to improve the sustainability of these activities and results during the next two years. 7 Recommendations The most important recommendations for the follower project UPA 2 are: to use and actualize the monitoring system, including the indicators for results to improve the cost / benefit ratio of the project to elaborate exit strategies for the project elements which are not sustainable until now (processing groups, farmers market, FLUPFA and TUPUFU, Farmer Resource Centre, animal husbandry) to train staff and include some information about the cross cutting themes gender and HIV/AIDS in the trainings to intensify trainings and extension service for the target group to leave the decision to the group members if they want to carry out the activities as group activities or individually to search for possibilities how FLUPFA and TUPUFU could get into a more intensive contact with their members and how to look for income generating activities to develop solutions for locally available and cheap animal feed with sufficient protein to elaborate strategies how to avoid hygiene problems through litter and wastewater vii

9 8 General conclusions and lessons learnt Complex implementation structures with several partner organisations Working with many project partners as implementing organizations increases costs and the dependency of the project on the quality of the performance of implementing organisations, but it also is able to develop local partner organisations and to bring together specific skills and approaches. Multi-level approach The multi-level approach to work with political decision makers, UPA farmers and their organizations allows increasing the possibility to develop and find solutions in critical areas and maybe enhance also the sustainability of some activities. Litter and waste water A critical factor which has to be considered in urban agriculture right from the start of project planning is the problem of litter and waste water requiring specific approaches. Animal husbandry Animal husbandry without own fodder production is highly risky, especially in urban farming, because land to produce own fodder often does not exist. The necessity of affordable and available feed should be considered if support to animal husbandry in urban regions is planned. viii

10 II. Main text 0 Prior remarks 0.1 Reason and aim of the evaluation An external evaluation of the UPA 1 project Enhancing urban and peri-urban agriculture (UPA) in Liberia (LBR 1026) was already planned in the project proposal as an end of project evaluation. A follow up phase (UPA 2) started already in the end of 2011, with an overlapping of the two projects of nearly one year. The evaluation was carried out during the last month of the project which ended in February According to the terms of reference the primary aim of the independent evaluation is to use impact analyses and a target-actual comparison to determine the project outcomes and impacts (direct and indirect, short, medium and long-term, intended and unintended)....the evaluation is thus based on DAC criteria (efficiency, effectiveness, relevance, impact and sustainability) and the standards of the DAC and DeGEval. The terms of reference include additional questions about the influence of the process of decentralisation of the governmental structures on the project progress, expectations of groups towards umbrella farmer organisations, about the promoting factors for strong groups, female participation in leadership and the adaption of diversified low external input agriculture, about the role of Community Farmer Trainers (CFT) and about the influence of marketing on farmers planning of crop production. An additional objective of the evaluation was the short description of a family or group to describe how they made use of the project. 0.2 Evaluation process In the period between and the external evaluation was carried out by an international consultant. The evaluation was well supported by all involved actors, especially parts of the local staff and a German intern assisted the process, carrying out interviews during the field visits. The visited groups and families have been selected together by the project coordination and the evaluator, trying to consider all targeted regions (urban and peri-urban, Monrovia, Tubmanburg and the corridor between Monrovia and Tubmanburg), the most important project and beneficiaries activities (vegetable production, different types of animal husbandry, processing, marketing, support of governmental institutions to UPA), different kinds of benefited families and groups (backyard gardeners, groups, institutional groups and umbrella organizations of the target group), active and inactive groups, and groups supported by Welthungerhilfe staff as well as groups supported by HDF staff. Four groups have been selected by chance. A group without any support of the UPA project or other interventions, which would have been a comparison group, could not be found. During the stay in Liberia, the following steps have been included into the evaluation process: 1

11 Briefing with the Regional Director of Welthungerhilfe and also in a meeting with the Head of Project Preparatory meeting to organize the evaluation: agreement on evaluation procedures, logistics, etc. and selection of communities / groups to be visited Short participation at the National Conference of UPA which took place during the evaluation and was part of the activities of the visited project Initial workshop with project staff Analysis of project documents, systematisation of information Two meetings with the Community Field Trainers, one in Monrovia and one in Tubmanburg Field visits in Monrovia and Tubmanburg, including beneficiaries groups, institutional groups (schools, prison, orphanage), umbrella organisations and Farmers Resource Centre) Semi structured interviews with key persons Final meeting: presentation and discussion of the most important findings and recommendations. 0.3 Evaluation methods During the evaluation process, the evaluator aimed to consider the opinion of all involved actors, such as target groups (individual backyard gardeners, groups and umbrella organizations), staff of all implementing organizations and their partners (Welthungerhilfe, RUAF, HDF, WOCHIDO, CARE, MOA, MCC), representatives of Welthungerhilfe and different key persons. Before the beginning of the evaluation, a partial results chain (with outputs, outcomes/results and impacts) was formulated by the evaluator based on the logframe of the project proposal. During the evaluation this results chain was discussed and compared with the activities, outputs, outcomes and impacts achieved during project implementation. The comparison was complemented by the analysis of relevance, efficiency and sustainability of the project. The required information was obtained by a mix of different participatory methods and sources of information, considering the specific situation of women and men. Additionally before the beginning of the evaluation a questionnaire for the beneficiaries (group members, leadership and CFT) and a trend analysis have been prepared, as well as guidelines for semi-structured interviews with resource persons and other guidelines for groups. The questionnaire was discussed with the project management and adapted in some points, a few additional questions, most of them about food pattern, have been added. Some of the questions repeat what was asked already in the baseline study and during data analysis these responses have been compared to analyse changes between 2011 and Methods of data collection and sources of information: Project documents: o Project proposal of LBR 1026 and the follower project LBR 1034 (including budget and logframe) o Urban and Peri-Urban Agriculture Baseline Survey (2/2011) o Monitoring and reporting documents of Welthungerhilfe, RUAF and HDF o 1 st Interim narrative report (12/ /2010) 2

12 o 2 nd Interim narrative report (1/ /2011) o Midterm evaluation report o Result oriented monitoring (ROM) report of EU o Welthungerhilfe: Landeskonzept Liberia o Agricultural products market survey o Need assessment o Urban Agriculture in Liberia a policy narrative, 2012 o Report of the Learning WS (8/2011) Other relevant documents Initial workshop with staff (Welthungerhilfe, HDF, RUAF), didactic material, Field visits: out of 52 groups supported in the beginning of the project, representatives of 22 groups (17 vegetable, animal husbandry, medicinal and processing groups and five save and loan groups) have been visited; additionally two groups of backyard gardeners, seven individual backyard gardeners and three individual traders have been met. During field visits the following methods have been used: o Group discussions with representatives of eight groups (interview guidelines), including also representative of five save and loan groups o Conversation with representatives of nine groups o Workshop with 29 Community Farmer Trainers (CFT) o Individual interviews with 34 group members and families benefited by the save and loan groups o Five trend analyses with 75 participants (32 men and 43 women) to analyse the situation of the target group before the project started and at present and which organisations contributed to improve the situation o Observations during visits of fields, gardens, animals Semi structured interviews with key persons, representatives of government departments and NGO. 1. Description of the project 1.1 Brief project description The project Enhancing urban and semi-urban agriculture in Liberia (UPA 1) with the project number AF 1582 /LBR ; EU: DCI-FOOD/2009/ formulated as overall objective to contribute to poverty alleviation and sustainable urban development in Liberia. The specific project objective is: Small scale urban and peri-urban producers improve food security and increase income and UPA is officially recognized. This objective is planned to be reached by five results: Result 1: UPA is accepted and actively supported by relevant ministries and municipal authorities Result 2: Organization and formalization of UPA farmer groups is strengthened, UPA farmer groups have increased visibility and political influence Result 3: Agricultural production is enhanced and diversified Result 4: Beneficiaries process agricultural and other products and realize value adding activities and other income generating activities Result 5: Marketing by UPA farmers and other beneficiaries is improved. 3

13 The project was implemented by Welthungerhilfe in partnership with the international NGO Resource Centres on Urban Agriculture & Food Security Foundation - RUAF (result 1) and the national NGOs HDF - Human Development Fond (result 3) and WOCHIDO (result 4). For two of the five planned results (result 1 and 5) Welthungerhilfe worked in cooperation with CARE. The approved budget is 1,738,220.-, Welthungerhilfe was financing 188,220.- of this budget, the European Union (EU) 1,550, The project period started in 09/2009 and ended in 03/ Framework conditions Since the end of 14 years of civil war in 2003, Liberia is experiencing a period of political stability, supported by a contingent of UN troops to maintain security. As consequence the situation is improving slowly. But Liberia remains one of the world s poorest nations with estimated two-thirds of Liberians living below the poverty line, which is on less than one dollar per day (EU web-site). In 2011 Liberia was still one of the countries with the lowest ranks of the Human Development Index (HDI), it ranked 182th out of 187 countries in the world. During the war the population concentrated in the relatively safe capital Monrovia, whose population grew to more than 1 million habitants, more than 1/3 of the total population of the country. The major part of rice, the main staple food, is imported and therefore expensive. This makes the population very vulnerable to price fluctuations. More than 80 % of the population do not have employment ( and the majority is not food secure. This was confirmed in the questionnaire used during the evaluation: 61 % of the interviewed persons said, that they experienced one or more months with insufficient food during the last year. Urban and peri-urban agriculture offers large potentials to improve the situation of urban habitants, but there are also serious challenges: land availability is an increasing problem in Liberia and especially in the urban regions, litter and waste water make it difficult to guarantee a minimum of hygienic standards and the lack of animal feed, which is a national problem, affects urban agriculture more than rural areas because normally there is not sufficient land to produce local animal feed. 1.3 Brief description of the target group Planned target group The evaluated project aimed at different target groups: as main beneficiaries 52 groups have been planned (22 crop producer groups, eight small animal producer groups, nine small ornamental and fruit tree nurseries groups, two fishery groups, 10 small agro enterprises and two student groups) in the urban and peri-urban regions of Monrovia, Tubmanburg and the corridor between Monrovia and Tubmanburg. It was planned that each group has 20 members what results in a target of 1,040 group members. The direct beneficiaries should be selected among vulnerable urban and peri-urban farmers and farming communities in 4

14 Western Greater Monrovia (Monteserrado County), and Tubmanburg (Bomi County), with special emphasis on women and youth farming groups. Additionally to the 1,040 group members it was planned to support 70 food security facilitators (later called CFT) and 300 backyard gardeners, as well as 150 municipal authorities and representatives of governmental institutions. This results in 1,560 directly benefited persons or about 1,500 persons if it is considered that the food security facilitators normally are part of their groups. In the project proposal the total number of persons benefited directly by project activities is mentioned as 1,500 persons and indirectly (five family members in each family with exception of the 20 students and the 150 authorities) 6,200 persons. In this calculation it was not considered that normally in one group several members of the same family are participating. Reached target group Table 1 below gives an overview of the reached target group of the project under evaluation. The comparison with the planned target group (1,040 group members and 300 backyard gardeners) shows that the number of beneficiaries in the beginning of the project was higher (about 1,228 group members and 342 backyard gardeners) than planned. In the course of the project and after the distributions of seed, tools or animals the number of beneficiaries lowered to about 1,008 families: 785 beneficiary families participating in groups (the number of members is higher considering that in many groups more than one member per family participate) and about 223 backyard gardeners. 6 additional groups in Tubmanburg are not anymore active and some other groups are nearly inactive. The 1,008 families correspond to 5,040 family members as indirect beneficiaries. Additionally four processing groups have been accompanied by the local organization WOCHIDO during nearly one year, three of them are active at this moment, but not participating as target group of UPA2. 46 Community Farmer Trainers (CFT) have been trained (instead of 70 planned) and many persons participated in the process of the Multi- Stakeholder Action Planning and Policies Formulation (MPAP) process. In general it can be assessed that in the beginning of the project the numbers of target groups reached correspond to the planned numbers, even if the composition of the target groups is not in all details as planned (e.g. small ornamental and fruit tree nurseries groups, small agro enterprises and student groups are less than planned). But at the end of the project the target groups reduced clearly. Partly the target group is composed of very vulnerable persons, such as one group of disabled persons or woman headed families (24 % of the persons interviewed during the evaluation are members of a woman headed household). More than 50 % of the direct target group are women (55 % in the agricultural groups and the majority in most processing and in all S&L groups). However in groups leadership women are in the minority. It was an agreed opinion during the evaluation that leadership trainings for women would be the best way to increase their participation in the groups management. Due to difficulties to find existing youth groups, the youth was not targeted in a special form. 5

15 Diagram 1: Reached target group Monrovia Target group at the beginning of the project measure (number of persons) Actual target group (at the end of the project) 20 groups with 350 members 20 groups (HDF) 225 families Actual number of families 11 groups with 310 members 11 groups (Welth.)(incl. S&L) 150 families 117 additional families in the S&L groups 4 processing groups with 202 families as members (pepper sauce, peanut butter, cookies) 3 processing groups, not involved in agricultural activities 135 families 288 backyard gardeners (6 groups) Backyard gardeners 105 families (HDF) (6 groups) Backyard gardeners 75 Families (Welth.) 3 school-gardens 3 school-gardens (HDF) The schools have between 120 and 400 pupils each one 1 student group with 33 students 1 students group (Welth.) 33 students Tubman -burg 17 groups with 366 members 11 active groups (Welth.), and 6 which are not active 158 active families 93 are not active 54 backyard gardeners Backyard gardeners (Welth.) 43 families 2 schools 2 schools 450 pupils 1 prison 1 prison 38 prisoners 1 orphanage 1 orphanage 35 orphans Total 52 groups and individual 51 groups (including S&L, About 1008 active and 93 backyard gardeners, totally with processing groups and 6 non active families (in about 1570 members (persons) inactive groups) agriculture, animal Additional 8 institutional groups husbandry, processing and Additional: 8 institutional (Additional: 4 processing groups groups supported by WOCHIDO) S&L activities including backyard gardeners ) (117 of the 1008 families are only in S&L groups) 33 students Pupils of 4 schools 38 prisoners 35 orphans * The number of families was calculated as an average based on the information of staff and CFT. 2 Relevance 2.1 Relevance to core problems of the target group The project is relevant for the target group (food shortage, increasing food prices and high unemployment). This showed the base line study and the need assessment elaborated at the beginning of the project and also the meetings during the evaluation. 6

16 It seems that the planned project activities have not been of special importance for some of the responsible persons of the University of Liberia which was planned to be one of the partners of the project. One issue which was not sufficiently considered in the project planning is the problem of lack of waste management. Litter and polluted water are serious hygienic problems in some of the gardens. For these problems, typical for urban agriculture, no measures have been planned in the project proposal. Also for the irrigation difficulties in some gardens no activities have been planned. The introduction of treadle pumps has been added to the activities at a later stage. The follower project UPA 2 is considering the waste problem and also the improvement of irrigation. A comprehensive base line study provided information for the adaptation of project activities to the concrete needs and situation of the target group. Even if the general project is highly relevant for the beneficiaries, many of them expected more free distributions. These expectations sometime made project implementation difficult, especially in the beginning. 2.2 Relevance to the objectives of Welthungerhilfe The objectives of the project under evaluation are fully in line with the Welthungerhilfe country policy which mentions as purposes and specific contribution of Welthungerhilfe in Liberia: Rural communities, urban groups of poor and especially vulnerable groups undertake own efforts to improve their situation and perspectives, to strengthen social togetherness and to save peace. 2.3 Relevance to the objectives of the partner organisation and the partner country The project and its objectives fit into the strategy and experiences of the main partner organisations of Welthungerhilfe in project implementation: RUAF Foundation is an international network of Resource Centres on urban agriculture and food security and the local NGO Human Development Foundation (HDF) has been implementing projects in the areas of agriculture, health and education. The project is also relevant for the objectives of the Government of Liberia (Poverty Reduction Strategy and MDG 1 and 3). 2.4 Relevance to the objectives of the European Union According to the web-site of the European Union the key issues in EU-Liberia relations are: In line with the Liberia's development policy and the EU Strategy for Africa, EU Cooperation strategy under the 10 th EDF (European Development Fund ) is based on a flexible approach of Linking Relief Rehabilitation and Development (LRRD) in the context of Post- Conflict Fragile States. The overall amount of 150 M is being implemented through three main areas of intervention: (1) LRRD in a post-conflict fragile state, with funds allocated totalling 145 M within two main components: i) Rehabilitation of Priority/Basic Infrastructure (roads, electricity and 7

17 water) and the provision of basic social services (education and healthcare) M; ii) Governance, Institutional Support and Capacity-Building Cross-Cutting Programme ( 20 M). (2) general budget support, 20.2 M (3) non-focal sector, 5 M, comprising actions intended to strengthen the current and future capacity of the Ministry of Planning and Economic Affairs and non-state actors/civil society. The project fits into the 3 rd intervention area of the EU (strengthen capacity of ministries and civil society). The UPA approach was proposed by the EU, what shows additionally the relevance of this approach for the EU. 3. Effectiveness 3.1 Achievement of project purpose The specific objective of the project is: Small scale urban and peri-urban producers improve food security and increase income and urban agriculture is officially recognized. In the logframe the following indicators have been formulated for the project objective: Indicator 1: The importance of UPA is recognized in at least 50% of the targeted municipalities Indicator 2: At least 80% of direct beneficiaries have increased and diversified their food intake by 2012 Indicator 3: At least 60% of the beneficiaries, of which at least 35% are female farmers, state that they have increased their income from agriculture by 20% or more at the project s end. Due to the lack of monitoring data the assessment of the achievement of the project purpose is based on information collected during the evaluation and estimations made by the evaluator considering all information available. The achievement of the indicators of the project purpose has been estimated as follows: Indicator 1: The importance of UPA is well recognized and accepted by the counties and municipalities participating in project activities Indicator 2: 84 % of the persons interviewed during the evaluation (half of them leaders!) confirmed improvements of availability of food at household level, but 32 % only little (29%) or very little (3%) improvements Indicator 2: Only about 25% of the direct beneficiaries consider the variety of food better than in 2011 (baseline survey), 50% eat new food today Indicator 3: More than 35% of the beneficiaries are women (approximately between 50 and 60 %) Indicator 3: 44% of surveyed households answered that their household income has improved much or very much through project activities, 41% considered the improvement little or very little. In the case of surveyed farmers which are not CFT or group leaders the percentage of answers confirming a strong improvement of income is lower, this is also the result of observations realized during the evaluation. The results of the survey gave a higher percentage than 80% of beneficiaries with improved income (including little or very little improvement), however considering all 8

18 information gathered during the evaluation the number of beneficiaries which increased their income by at least 20% is the minority. It can be summarized that in general the project purpose was achieved, in some points what was achieved exceeded what was planned (e.g. percentage of women participating,) and in some points it remained slightly below the target (diversification of food, level of income increase). 3.2 Project preparation Quality of target group, actor and situation analysis The evaluated project was planned by a consultant with support of Welthungerhilfe staff of other projects. An ECHO financed project was implemented in the same regions (Monteserrado and Bomi County), but with the rural population. Therefore Welthungerhilfe had available information about the region and their population. Extensive analyses (baseline study, needs assessment of the target group, situation and actor analysis) have been realized after the project proposal has been accepted and an adjustment of the project planning was realized based on this information. The baseline study provided a good basis for the project evaluation Existent self-help approaches and structures During the project preparation existent groups have been considered. It was tried to identify (together with the respective Ministries) existing women and youth groups, but finally no youth group could be identified which could participate in the project activities. About 50 % of the supported groups have been previously existing groups. Also during the establishment and promotion of the umbrella organisations for urban and peri-urban agriculture FLUPFA (Federation of Liberian Urban and Peri-Urban Farmers Associations) and TUPUFU (Tubmanburg Urban and Peri-Urban Farmers Union) the already existing national Farmers Union FUN was included into the process. However it may be doubted if it was the best decision to create new organisations instead of integrating an UPA sub-group into the existing and already consolidated FUN which is better known and recognized among farmers. FUN also receives support from international donors such as EU and IFAD (International Fund for Agricultural Development). The difference between periurban and rural agriculture is not always clear and there is an overlapping of the target groups of FUN and the urban farmers organisations. Therefore one strong organization with sub-groups may have been the better option. 3.3 Project planning Quality of the project planning matrix In general the project planning matrix was well elaborated, including indicators for objectives and the five results. However in some points there are contradictions between the text of the project proposal and the project planning matrix and budget (e.g. in the text of the activity 2.2 social workers are mentioned which do not appear in the budget or logframe). 9

19 Some project measures have been planned too diversified and were not covered by project experts, such as promoting fish processing or ornamental plants nurseries. For the cross cutting issues gender and HIV / AIDS mentioned in the text of the project proposal no objectives or indicators have been formulated in the project planning matrix. Therefore these issues have not been covered very intensively during project implementation Analysis of the results chain The diagram 2 below shows the planned results chain elaborated by the evaluator according to the logframe. As the Save & Loan groups have been added later to the project activities they are not included in this results chain. It was discussed with Welthungerhilfe staff during the evaluation and adopted in some points. During the discussion it was criticized, that the five results are a mixture between outcomes, outputs and use of outputs. The evaluator did not reformulate these results/outcomes. The activities and use of outputs were not formulated during the elaboration of the results chain. The extension service planned with Food Security Facilitators was not adapted to the Liberian reality and did not work. As consequence, the necessary accompaniment of the farmers and groups was not sufficiently considered during planning. Otherwise the results chain is logical and consistent. 10

20 Diagram 2: Planned results chain Impact UPA officially recognized and Government at different levels works together with UPA producers as partners in development Improved food security Increased income Outcomes/ results R1: UPA is accepted and actively supported by relevant ministries and municipal authorities R2: Organization and formalization of UPA farmer groups is strengthened, UPA farmer groups have increased visibility and political influence R3: Agricultural production is enhanced and diversified R4: Beneficiaries process agric. and other products and realize value adding activities and other income generating R5: Marketing by UPA farmers and other beneficiaries is improved Market surveys Outputs MSF in Monrovia and Tubmanburg develop City Strategic Agendas and an enabling and regulatory environment for UPA MoA and other stakeholders are promoting UPA UPA is included in the curriculum of the Faculty of Agriculture of Liberia University Lessons learned and best practices are elaborated MPAP (Multi Stakeholder Action Planning and Policies Formulation) facilitation team is working Study tour to Freetown with local authorities 2 situation analysis of UPA UPA is integrated in and use mapping Farming system assessment Institutional network analysis Policy review UPA farmer groups organized and registered UPA farmer groups have internal organization and management capacity improved Multi-stakeholder fora of local authorities to promote UPA Stakeholders (esp. MPAP facilitating team members) with awareness and capacity in UPA 11 Nutrition education campaigns for farmer groups and urban consumers Community exchange visits Knowledge about improved and more diversified production exists Small scale animal production is promoted Demo plots for ornamental plants, fruit trees, indigenous vegetables, medicinal plants and others Extension service for UPA exists Knowledge about improved storage and post-harvest protection exists Farmers have improved access to agric. inputs Food processing groups formed and registered Non-food agricultural production is improved Processing equipmen t provided to groups Farmers trained in food processing Farmers with improved business management and marketing skills Producer groups, marketing associations and potential bulk purchaser are linked Establishmen t of farming and marketing cooperatives is promoted Farmer groups are assisted to create brands (labeling, logo, stickers, stamps etc.) Local marketing and establishment of market stands for farmer groups are encouraged

21 3.3.3 Realism of project purpose The specific objective of the project: Small scale urban and peri-urban producers improve food security and increase income and urban agriculture is officially recognized was realistic and also the respective indicators: The importance of UPA is recognized in at least 50 % of the targeted municipalities At least 80 % of direct beneficiaries have increased and diversified their food intake by 2012 At least 60 % of the beneficiaries, of which at least 35 % are female famers, state that they have increased their income from agriculture by 20 % or more at the project end. In the project proposal the total number of persons benefited directly by project activities is mentioned as 1,500 persons and indirectly (five family members in each family with exception of the 20 students and the 150 authorities) 6,200 persons. In this calculation it was not considered that often in one group there are several members of the same family. Therefore the number of indirectly benefited persons was not fully realistic Appropriateness of staff, material and financial planning Staff Four field officers (two with Welthungerhilfe and two with HDF) have not been sufficient to work with 50 farmers groups, additional to the difficulty that the CFT did not work voluntarily as trainers for their groups as it was planned in the project proposal. Another problem was that specialized personnel was missing for some of the planned activities, such as experts for ornamental plants, fish processing or medicinal plants. External consultants, foreseen in the project proposal, often could not be found for these activities. The list of planned and employed staff in April 2009 was as follows: 2.5 Coordinators: 2 Welthungerhilfe and 0.5 HDF 4 Agricultural Field Officers: 2 Welthungerhilfe and 2 HDF 4.5 other technical staff (Marketing Officer, MSF-Officer, M&E Officer, FRC staff): 4.5 Welth. 3.9 Administrative staff (Administrators, Admin. Assistants, Cashier): 2.4 Welthungerhilfe and 1.5 HDF 12.5 Support staff (Security, Driver/Mechanic, Asst. Mechanic, Caretaker): 11 Welthungerhilfe and 1.5 HDF. Compared to the total number of staff (more than 27), field officers working directly with the target groups are underrepresented. The financing of the Multi Stakeholder Forum (MSF) - Officer by two organizations (50% Welthungerhilfe, 50% CARE) was not a good solution, due to unclear work places, reporting structures, etc. Equipment Equipment and vehicles have been sufficient, considering that vehicles from the previous ECHO (European Commission Humanitarian Aid & Civil Protection) funded project have been transferred to the UPA project. With funds from the project budget one vehicle and eight motorbikes have been purchased. Additional funds were available for the rent of trucks and of a vehicle for HDF. 12

22 Financial planning In the project proposal the project costs of 1,538,877.- (excluding 5 % reserve and 7 % administrative costs) are distributed among the following main expenditures: Human Resources: 620,337.- Travel: 39,250.- Equipment and supplies: 126,300.- Local offices: 221,400.- Publications, studies, Resource, auditing, evaluation, financial services, costs of conferences, seminars: 120,310.- Project implementation (5 results) 411,280.- The organisational setup resulted in a relatively high budget. The project considered a complex structure for implementation with three external partners (RUAF, HDF and WOCHIDO) and cooperation with CARE. This led to higher costs in terms of staff (not at field level) and offices. Additionally it should be considered that Liberia is very expensive and most equipment and material has to be imported, an additional fact of rise in costs. 3.4 Quality of the analysis of project holder (partners and external structure of Welthungerhilfe) Organizational structure and focal points of work In Liberia, among governmental and non governmental institutions Welthungerhilfe is a well known and appreciated organization. At this moment Welthungerhilfe is implementing seven projects in Liberia with a total financial volume of more than 30 millions, financed mainly by EU and German Development Cooperation. Welthungerhilfe has a regional office in Monrovia, responsible also for Sierra Leone, and several local offices in Liberia, one of them in Tubmanburg. Since the end of the civil war in 2003, Welthungerhilfe has been active in emergency and refugee aid and school feeding; actually its focus is turning towards development aid. The actual focal points of Welthungerhilfe s activities in Liberia are rehabilitation of infrastructure, water and sanitation, agriculture and food security, basic education and promotion of women (for more details see: Welthungerhilfe, Landeskonzept Liberia ). The organizational structure in relation to the evaluated project is presented in the diagram below. 13

23 Diagram 3: Organizational chart of UPA 1 Project Other project Other project Regional Director Other project M & E officer Project Leader Head of Logistics Project Administrator Executive Director HDF* RUAF Administrative asst. Accountant Coordinator HDF Coordinator Procurement officer MSFO Driver mechanic 2 Cleaners Agricultural team leader Warehouse administrator Cashier 2 Agricult. officers HDF FRC support Administrator HDF Agricultural officers (2) Security HDF FRC assistant WOCHIDO (only 1 year) * Not paid by project 14

24 3.4.2 Staff, equipment and logistics In general the different projects have sufficient staff, equipment and logistics. In the compound of Welthungerhilfe in Monrovia there is an own garage for the large car pool. This compound includes also two buildings with several offices, a meeting room and a guest house. The expatriate Project Manager for the project under evaluation is also responsible for two additional projects, an IFAD funded project for rural farmers and the follower project UPA 2. During the last year of implementation of UPA 1 the Project Manager was supported by an expatriate specialist for agriculture and monitoring whose contract is finishing now Project steering and financial administration The Project Manager is the person responsible for the implementation of the project UPA 1. He knows the project activities from regular field visits. The financial administration is realized by the administrative staff in Liberia and seems to function well, no critical voices have been heard in this regard. In May 2011 a budget revision with several smaller adaptations was realized. 3.5 Steering by the head office, regional office and cooperation The regional director intervenes only in the case of major difficulties but participates in certain cases in project activities as it was the case during the National Conference of Urban Agriculture, taking place during the evaluation. Also the responsible persons for the project in the head office maintain contact with the project, visiting the project at specific opportunities. During the National Conference of Urban Agriculture, also head office staff was participating and took the chance to visit the project. Major interventions have not been necessary until now. 3.6 Quality of project execution Presentation and evaluation of the project outputs and activities The diagram below compares the planned and realized activities. It can be summarized that most of the planned activities could be realized or at least efforts were made to realize them (e.g. support to University). Two Multi-Stakeholder Forums have been established, UPA farmer groups have been trained and two umbrella organizations for urban farmers have been founded. The agricultural approach promoted by Welthungerhilfe and HDF tries to make farmers more independent from external inputs (like mineral fertilizer or chemical pesticides); it is well adapted and ecologically sustainable. Project measures promoted agricultural production systems (especially vegetable and small scale animals). The Save and Loan groups have been established and apparently they are working very well and sustainably. Additionally some activities have been realized to improve processing and other activities to improve non-food agricultural production and marketing for a part of the target group. 15

25 In general the activities have been carried out in good quality. The following points show the limitations of the realized outputs: The activities of result 4 (processing) and some of result 5 (marketing) have been realized only at the end of the project what makes it necessary to continue some of the activities in the follower project UPA 2. It was not possible to offer a regular extension service with voluntary facilitators (CFTs), as it was planned, because the target group did not agree with non paid extension work. Nevertheless, during the evaluation no bigger change of the role of CFTs was mentioned. In some cases the seed distributions did not occur in time and some of the seed did not germinate. Support to appropriate animal feeding (especially chicken and pigs) is a weak point of the project, especially since the flour plant which sold wheat bran, stopped their production. Some farmers feed their pigs with food leftovers of restaurants or similar, but others do not have access to such food resources. Lack or shortage of feed threatens to destroy the success of the animal husbandry component. Animal husbandry and horticultural/agricultural activities were normally realized with different target groups, hindering synergies such as the use of manure or the production of fodder. The umbrella organizations for UPA, FLUPFA and TUPUFU, are not really known among many of their members. There is very little information and communication between leadership and members. The expectations of the members regarding their umbrella organisations are quite varied (see results of the questionnaire in annex 5); many expect the distribution of seed and tools and some also expect trainings. Others mentioned that they should work as a national NGO, implementing project measures as a partner of Welthungerhilfe. Also support in the land issue, to receive animal feed or technical service was mentioned. Some persons wished that the umbrella organisations would care for the farmers, empower the groups, link them to the Government and other institutions and make advocacy work for UPA farmers. Many farmers pointed out that they would like to see better information and communication between members and FLUPFA and TUPUFU. The cross-cutting issues gender and HIV-AIDS have not been worked intensively by the project. On HIV-AIDS there was only one initial workshop for the project staff. 16

26 Diagram 4: Comparison of planned and realized activities Planned activities (A 1.1) Preparations and establishment of a local MPAP facilitating team (A 1.2) Organise a workshop to raise awareness on the importance of urban agriculture and prompt local authorities to create Multi-stakeholder fora to promote urban agriculture (A 1.3) Capacity building of local MPAP facilitating team members and other stakeholders (A 1.4) Situation analysis of urban and peri-urban agriculture in Monrovia and Tubmanburg, including land use mapping, farming system assessment and institutional network analysis and policy review (A 1.5) Organise study tour to Freetown with local authorities from Monrovia (6) and Tubmanburg (2) (A 1.6) Facilitation of MPAPs in Monrovia and Tubmanburg to develop City Strategic Agenda's and the development of an enabling and regulatory environment for urban agriculture (A 1.7) Promotion of urban and peri-urban agriculture during coordination meetings organised by the MOA (A 1.8) Organise two workshops on lessons learned and best practices (A 1.9) Include urban and peri-urban agriculture in the curriculum of students at the Faculty of Agriculture of Liberia University Realized activities Two facilitating teams (smaller than the Multi-Stakeholder Forums) were established and trained with the objective to prepare the MSFs Yes, the WS was organized Capacity training / coaching of the facilitating teams was realized In the beginning of 2011 information about UPA and institutional networks was collected, but not as detailed as planned (e.g. no land use mapping and no information about the amount of harvested products of the farmer families) The study tour was organized Five meetings took place in Monrovia and five in Tubmanburg. A policy narrative was elaborated, the two MSFs agreed to it, but no official UPA policy has been adopted until now. The Ministry of Agriculture has a budget for UPA, from which an officer for UPA is financed. During the coordination meetings, presentations about UPA were realized; umbrella organizations for UPA participate and promote UPA One workshop in August 2011 Not realized due to internal changes and low capacity in the University; project staff offered lessons for students at the University 17

27 (A 2.1) Identify, organize and register UPA farmer groups (A 2.2) Provide training to improve internal organization and management capacity of farmer groups (A 2.3) Increase Visibility and political influence of UPA Farmer Groups (A.2.4) Train and equip 7 VS&L groups (A 3.1) Set up demonstrations plots for ornamental plants, fruit trees, indigenous vegetables, medicinal plants, and other species (A 3.2) Purchase and provide improved agricultural inputs (A 3.3) Promote small scale animal production (poultry, ducks, ground hogs, small ruminants, pigs etc.) 41 vegetable and 10 livestock groups have been identified Different trainings have been realized for all groups Different measures increased the visibility (conference, sign boards, videos, shirts, newspaper articles). UPA farmer organizations TUPUFU and FLUPFA are members of the MSFs and try to influence politics; FLUPFA organized two WSs on land issues for about 100 farmers and is responsible for the newly installed farmers market Seven groups have been trained Demo plots in the University of Liberia were installed, but collapsed (no ownership of the University). Good demonstrations of different horticultural, agricultural and livestock practices are existing at the FRC 42 farmer groups and 342 backyard farmers received twice different kinds of seed and tools (according to project information the following tools have been distributed: 6,480 cutlasses, 126 shovels, 498 regular hoes, 498 small hoes, 414 watering cans, 288 diggers, 372 roll twine rope, 424 rakes, 42 pick axe, 414 files and 42 wheel barrows). The third time the seeds were given to the CFTs for onward distribution. 19 treadle pumps have been distributed to 19 groups 400 hens, 55 exotic roosters, 140 ducks, 14 goat, 16 sheep and 27 pigs have been distributed to the 14 animal husbandry groups. Animal husbandry trainings were realized (A 3.4) Organize training to improve and diversify agricultural production (food crop, small scale animals, flower and fruits etc.). (A 3.5) Disseminate crop and harvest loss reduction techniques (low-cost storage and post-harvest protection) (A 3.6) Provide regular extension service to farmer groups and individual farmers Different trainings have been organized Some trainings about the reduction of post-harvest losses were realized and four solar dryers were distributed Extension service was provided, but more is needed In 2012, one exchange visit with six groups was realized 18

28 (A 3.7) Organize community exchange visits to share knowledge, problems and experiences (A 3.8) Carry out nutrition education campaigns for farmer groups as well as urban consumers (A 4.1) Provide training on food processing and value adding techniques (processing, drying, cooking, food preparation, packaging) (A 4.2) Purchase and provide processing equipment to groups (solar dryer, rice mills, cassava grater etc) (A 4.3) Identify and improve use of added value to non-food agricultural products (e.g. medicinal plants) (A 4.4) Identify, form and register food processing groups (A 5.1) Carry out market surveys (A 5.2) Provide business management and marketing training to farmer groups (A 5.3) Establish direct links between producer groups marketing associations and potential bulk purchaser (A 5.4) Promote the establishment of farm and marketing cooperatives (A 5.5) Assist farmer groups to create brands (labelling, logo, stickers, stamps etc.) Two series of nutrition workshops have been realized, one for six schools (three times) and one for 20 farmer groups. Newspaper articles and a poster about Liberian food Seven groups have been trained; after difficulties with the national partner WOCHIDO (insufficient experience and capacity) Welthungerhilfe implemented the activities by itself): one cassava processing group, one medicinal plant group, two fish processing groups, one peanut butter group, one pepper sauce group, one juice group (non-functional); all seven groups received equipment. One group was supported to produce Moringa and other medicinal plant products. So far two groups are officially registered, three of seven groups are continuing in UPA 2 A short market survey was elaborated; market data are collected and price information is distributed Several trainings about management and marketing have been carried out Different groups have been linked to traders; the link to a bulk purchaser was difficult due to the demand of quantity and quality, one group delivers vegetable to UNMILL No cooperative has been established, but the cassava processing group is in the process to be a processing and marketing cooperative Three groups (cassava processing, medicinal plants and pepper sauce) have been supported to receive imported vases and sticks (A 5.6) Encourage local marketing and the establishment of market stands for farmer groups 19 One stand in Monrovia was build, another in Tubmanburg has started to be constructed; a weekly farmers market with 12 stalls was opened in 2/2013

29 3.6.2 Involvement of the target group and project cooperations Welthungerhilfe is trying to strengthen the ownership of the beneficiaries of project measures and expect contributions from them (e.g. 50% of the processing equipment or distributions of the offspring of received animals). This is not always easy, because the expectation of free distributions is still widespread among the target group. Through the organisations FLUPFA and TUPUFU the urban farmers participate in different levels of project implementation, such as in the MSFs or in the marketing activities. Unfortunately many group members do not see these organisations as their representatives. In general the cooperation and coordination between the different partners of implementation, Welthungerhilfe, CARE, RUAF and HDF have been good and successful. CARE and Welthungerhilfe complemented one another and learned from each other. Only the partnership with WOCHIDO and with the University of Liberia was not or only partly successful. WOCHIDO could not realize the expected activities and the University of Liberia did not cooperate as planned Overall evaluation of the execution In spite of a relatively complex system of implementation partners it was possible to realize most of the planned activities. The activities of result 4 (processing) and some activities of result 5 (farmers market and marketing stand in Tubmanburg) have been realized with delay and their implementation needs to be finished during the follower project UPA 2. It was an important step to support organization processes for urban farmers but FLUPFA and TUPUFU are too much oriented to Welthungerhilfe instead of being representatives of their members. Similar is the situation of the CFT (what is tried to modify in UPA 2). The planned extension service suffered from the fact that CFTs refused to provide voluntary training and extension work. In many cases it was not possible to accompany the groups and backyard gardeners closely enough to support them in resolving their production problems. Examples are distributed treadle pumps which are not working or the lack of fodder availability which is limiting the success of the animal husbandry component. Some seed distributions have not been on time and some seed did not germinate well. Other constraints are partly extreme hygienic situations in the gardens and difficulties to access irrigation water. For both problems no measures have been planned in the project proposal. The project management acted flexibly and in some cases adapted the project implementation to reality. The water problem was faced during project implementation by including treadle pumps and the waste problem was considered in the follower project UPA2. Save and Loan groups has been a matter of particular interest of the target groups, praised during the evaluation by their members. 20

30 The multi-level approach to work with political decision makers, UPA farmers and their organizations allows increasing the possibility to develop and find solutions in critical areas and maybe enhance also the sustainability of some activities. The cross cutting themes gender and HIV / AIDS have not been considered sufficiently Occurrence of assumptions and risks The following assumptions have been mentioned in the logframe of the project: 1 Political situation keeps stable, peace building process continues, sudden prize shocks do not happen 2 Official government and municipal bodies are willing to cooperate 3 Land owners are ready to cooperate and offer long term lease agreements to interests 4 Willingness of stakeholders to participate (result 1) 5 Access to land 6 Seeds are available or can be purchased, willingness to adopt new technologies and new varieties 7 The institutional framework (standardisation/control/registration) is in favour of small scale enterprises 8 Smallholders/groups see the opportunities of generating extra income by value adding. (Willingness to engage in value adding) 9 Willingness of possible market partners existent 10 Backyard gardens are big enough for market production 11 The timelines of the UPA activities are in sync with processes at the University of Liberia and other relevant Universities to revise curricula and engage in international cooperation 12 Willingness of communities to participate 13 Willingness to undertake training 14 Specialist inputs and machinery are available 15 Farmer groups are willing to cooperate 16 Access to land guaranteed 17 Cooperation by the Fac. of Agriculture: MoU is obtained for running of demonstration plots 18 Needed specialists are available 19 Package materials are available on the local market 20 Equipment available and affordable The underlined assumptions number 3, 5, 6, 10, 11, 13, 14, 16, 17, 18 and 19 did not occur or only partly. Some of these assumptions which not occurred affected the project only in specific measures, such as the cooperation with the University or promotion of ornamental plants. However the assumptions regarding the access to land affected a relatively large number of beneficiaries, for example in Fiamah about 80 families. As consequence, the sustainability of many family economies is affected (e.g. due to land problems or dependency on imported packaging material) and the need to import most of the materials is one of the facts which makes the project expensive. Assumption 13, the willingness to undertake training, did not exist among the CFT what made the implementation of the planned project approach difficult. 21

31 4 Efficiency 4.1 Cost / benefit ratio for the overall project The project costs per family and per person are high, compared to their benefits. Due to the complexity of the project, it is difficult to calculate the cost / benefit ratio. Therefore the following calculation should be seen as estimation to have an approximate idea of project costs and benefits per family. Subtracting 200, as costs for result 1 (policy level) we have about 1,500, project costs for the direct work with beneficiaries. Again subtracting 18 % (270, ) as costs for the 8 institutional groups (schools, etc.) the remaining project costs for the 45 active groups are 1,230,000.-, considering 1,000 families (all active groups and backyard gardeners). This means costs of 1, per family in three years and 410, - per year. Even considering the indirect benefits of improved knowledge and awareness about nutrition and UPA in the society and strengthening the local partner organisation HDF, the project remains expensive. For most of the participating families the activities do not produce benefits comparable to the costs per family. The majority of them improved their food security and their income, but to very different extents. There is no general information about the extent of income improvement but in the survey realized during the evaluation 85% of the households reported improvements of the income and some gave examples of the amount of increase between 15 and per year. However, in some cases the income seems to be not household, but group income and the information from different sources gathered during the evaluation led to the impression that many group members or backyard gardeners have only small income improvements. Possible benefits in the future such as improvements through the weekly market, processing activities or results of the lobbying of the UPA farmer organisations FLUPFA and TUPUFU were not considered in this calculation and may improve the cost / benefit ratio after UPA Cost / performance ratio for the individual measures Analyzing the costs of the project budget (after the last approval), only 27% ( ,- ) of the total costs of the project ( ,- ) could be attributed to one of the expected results. The general costs, such as costs for different offices (Monrovia, Tubmanburg, HDF), vehicle costs or costs of staff not working directly with the target groups seems relatively high. The following costs, attributed directly to the five expected results including staff working directly to reach the respective result - have been calculated: Result 1: ,- Result 2: ,- Result 3: ,- Result 4: ,- Result 5: ,- 22

32 The costs for the Farmer s Resource Centre have been calculated at ,-. They are part of the costs for result 3. Below the project performance is assessed per result. Result 1: The activities of result 1 caused interest and awareness in the members of the Multi-Stakeholder Forums (different organizations, institutions and local authorities) for the need and difficulties of urban agriculture. Concrete benefits for the urban and peri-urban farmers can be expected in the future, but until now it is not clear up to which extent. Result 2: The outputs of the activities of result 2 are better organized groups and especially the two umbrella organizations for urban and peri-urban agriculture FLUPFA and TUPUFU. Also in this case concrete monetary benefits can be expected only in the future. The save and loan groups are highly appreciated by the beneficiaries and seem to bring them important improvements and a certain security system, such as loans available in different critical situations. Result 3: Through the measures of result 3 more than 1,000 families have been trained and received seed, animals and / or tools. Many of them improved their farming systems. Only in the case of the distribution of animals the positive cost / performance ratio is not sure, due to losses and feeding problems. Results 4 and 5: The measures have started late and are still in their initial stages. The market stand in Monrovia, as it is used at the moment, seems relatively expensive, considering that it benefits only very few persons. The specific costs for the results 1, 2, and with - exception of the distribution of animals result 3 can be justified with the achieved results or the results expected in the future. The specific costs for the results 4 and 5 can be justified if the processing and marketing activities will be sustainable, an assumption that is not sure at this moment. 4.3 Quality of the internal project M&E system The logframe of the project includes indicators for the objectives and for the five results. However these indicators have not been measured in a systematic way. It was explained, that the monitoring officer (who already left the project) had difficulties to realize an effective monitoring. In the monthly reports of the staff there is information about the realized activities, but no information showing an overview of realized vs. planned activities or reached results. 5. Direct and indirect, short, medium and long-term outcomes and impacts 5.1 Use of outputs / outcomes The project planning matrix identified five results with their respective indicators. These indicators are mainly outcomes and uses of outputs of the project. Two indicators measure the achievement of the overall goal. The following table shows to which degree the targets have been achieved. 23

33 Diagram 5: Comparison of planned and achieved objectives and results Overall Objective: Expected results Intervention logic Contribute to poverty alleviation and sustainable urban development in Liberia (R 1) Relevant Ministries and municipal authorities accept the importance of urban agriculture and actively support farmers Indicator Income and food security have improved (e.g. increased intake of meals, increased agricultural production and increased GDP) The importance of urban agriculture is recognized by the Government and the municipal authorities A Multi-stakeholder forum to promote UPA is established after the first project year and meets at least on a quarterly basis What was achieved Income and food security of many beneficiaries improved, but not in all cases significantly The importance of urban agriculture is recognized at different levels of governmental institutions The importance of UPA is well accepted by Ministries and MCC and first steps of active support to farmers are realized; the concrete support to UPA farmers is still limited Two Multi-stakeholder forums are established, they meet quarterly (with short delays) (R 2) The formalization and organization of UPA farmer groups is strengthened (R 3) Agricultural production is enhanced and diversified UPA is sufficiently recognized in the ongoing planning process for a Monrovia Master Plan At least 12 awareness raising actions (Press reports, Serial Radio program on air etc.) on UPA are documented per year 42 UPA farmer and producer groups are formalized and organized after third project quarter At least six Training workshops have been carried out after the 1 semester of the second year UPA farmer representatives actively participate in the multistakeholder forum meetings UPA farmer representatives actively participate in MoA coordination meetings. Seven farmer groups have successfully completed one cycle of VS&L 24 A Monrovia Master Plan is not yet existing, but UPA was included in the national Poverty Reduction Strategy and the MOA has a budget of 7,000.- US$ available for UPA; in Tubmanburg UPA was included in the city ordinances Many events and activities (including radio programs) were realized with good press coverage; recently also the National Conference of UPA took place The organization and in some cases also the formalization of UPA farmer groups is strengthened, but most groups are still weak; Additionally the UPA umbrella organizations FLUPFA and TUPUFA have been founded, but they are still new and depend on external support (especially financially) and are not sufficiently known and recognized by their members 52 groups (including processing and save and loan and institutional groups) have been supported, six of them are inactive Several training workshops happened They participate, but they should be more active and should have more responsibility Yes Agricultural production was enhanced and diversified, but not always to the expected extent; animal husbandry activities suffered serious difficulties (chicken, pig feeding)

34 (R 4) Processing and value adding activities for agricultural produce are implemented and income generation activities are identified and promoted (R 5) Market links and marketing concepts for UPA products are established By 2012 at least five new production technologies are adapted and used by the target groups Yields are increased by at least 20% and post harvest losses reduced by at least 20% At least five improved or new seed and tree varieties are used and available 300 Households (the majority female headed and youth) have improved backyard gardening at project end Processing and value adding activities increase the income of the groups at least by 40% At least 5 newly processed products are available and sold regularly By the end of the project 10 small agro-enterprises are established 20 Project groups and 50 backyard gardeners are connected to market By the end of the project at least three new brands are labelled and sold regularly on at least three different markets At least two new urban farmer market stands are established by 2012 Some of the farmers use five or more new technologies (especially use of manure, composting, bio-pesticides, elevated seed beds with shading, crop rotation) others hardly changed the production technologies in their gardens No data available; it can be estimated that yields increased by 20% among a part of the target group, other beneficiaries reported limited increases of yields or none due to difficult atmospheric conditions Some of the beneficiaries have five or more new seed varieties, but this is not the majority; about 250 tree seedlings (orange, avocado, moringa, mango) have been distributed, partly they died because they came too late (at the beginning of the dry season). It can be estimated, that about 200 backyard gardeners, the majority female (but not all of them head of households) improved their gardens at different levels; only few of them are young people The processing activities are starting or in a very early stage, requiring more advisory support; six active processing groups (fish, cassava, pepper sauce, medicinal plants) No data available; until now very limited increase of income of the processing groups More than five products, but not regularly sold until now Project activities started with seven groups, six participated until the end of the project The weekly farmers market is just starting, if it will be successful, there is space for about 20 groups, other groups have been linked to markets; Many backyard gardeners have been connected to market (the evaluator did not receive concrete numbers of linked farmers and groups) New labels exist for moringa, pepper sauce and gari One in Monrovia is established and one in Tubmanburg is under construction Elaborated by project staff together with the evaluator Use of output The use of outputs suffered some constraints: problems in the implementation of the planned methodology (CFT), delay with the processing and marketing activities and 25

35 difficulties in the agricultural activities (pests and diseases, pig and chicken feeding, seed quality) and insufficient accompaniment of the introduction of some technologies (e.g. treadle pumps) affected the results of the realized activities. The University did not use the offered support in an adequate way and many members of FLUPFA and TUTUFU do not accept these organizations as their representatives. Results 5 results have been planned to be achieved during project implementation: Result 1: Relevant Ministries and municipal authorities accept importance of peri-urban agriculture (UPA) and actively support farmers Result 2: The formalization and organization of UPA farmer groups is strengthened Result 3: Agricultural production is enhanced and diversified Result 4: Processing and value adding activities are implemented and income generation activities are identified and promoted Result 5: Market links and marketing concepts for UPA are established. The different outcomes of the project are described in terms of the achievement of the five results in the following sub-chapters Economic outcomes Result 3: Planned: Agricultural production is enhanced and diversified. Achieved: The agricultural production was enhanced and diversified, but not to the aspired degree. Some difficulties hindered the realization of this result. The planned approach to work with 70 Facilitators was modified towards working with 46 Community Farmers Trainers (CFT). This approach did not work as planned because the CFT refused to train their groups without payment. It could not be observed during the evaluation that the role of CFT in their groups changed during project implementation. Difficulties as pests and diseases or animal feeding (pigs and chicken) and thefts could not be resolved. So the production of agriculture and animal husbandry was diversified and improved to a more ecological and more market oriented production, but only among a part of the target group, others were discouraged due to the problems which they could not resolve. This is confirmed by the survey realized during the evaluation (34 questionnaires). Its results showed that 56% of interviewed people consider that agricultural or horticultural production improved much or very much through the project, 29% consider that it improved little or very little, 15% see no improvement. Less than half of the interviewed person participated in animal husbandry activities. 54% of these consider that their animal husbandry improved much or very much, 6% confirmed little improvement, 40% see no improvement. 63% have diversified their production, growing new vegetables or crops they had not cultivated 3 years ago, especially cabbage, lettuce and pineapple. 87% of interviewed persons apply new techniques they did not apply three years ago, mostly environmentally friendly techniques such as using manure, compost and bio-pesticides. 26

36 Calculating with 756 families involved in agricultural and animal husbandry activities until the end of the project, it can be estimated that about 60% of them improved and diversified their production that means up to 500 families. It can be expected that additionally some of the persons participating in the beginning of project activities, but not continuing to cooperate with the project as well as most of the eight institutional groups (schools, prison orphanage) have improved and diversified their production to a certain degree. Result 4: Planned: Processing and value adding activities are implemented and income generation activities are identified and promoted. Achieved: The processing and value adding activities and also income generation activities suffered delay due to lack of experience of the local partner WOCHIDO. As a consequence, Welthungerhilfe itself promoted the mentioned activities. At present, three processing groups which are not farming (135 families) and three processing groups composed of UPA farmers have initiated their work. The expected result is still in its initial stage: although activities have been identified and promoted, people are just starting to apply the new activities. 38% of interviewed persons involved in project measures dealing with processing reported much or very much improvement of their processing activities, 24% little or very little improvement, 38% saw no improvement. Result 5 Planned: Market links and marketing concepts for UPA are established. Achieved: The result as formulated can be achieved fully during UPA 2. Up to now important steps towards that result have been achieved. Market links for many group members and backyard gardeners (no concrete numbers available) have been established and one market stand in Monrovia has been constructed where about three to four traders are selling fruits and vegetables. In Tubmanburg a second market stand started to be constructed, it will be managed by TUPUFU. During the Conference of UPA in February 2013 a weekly farmers market was opened, with space for about 20 sellers (which could represent 20 groups). It is too early to assess if this market will be successful or not. The results of the survey were quite positive: 64% of interviewed persons involved in project measures dealing with marketing reported much or very much improvement of their marketing activities, 26% little or very little improvement, 10% saw no improvement Socio-cultural outcomes Result 2: Planned: The formalization and organization of UPA farmer groups is strengthened. Achieved: Project trainings and other measures strengthened 45 UPA farmer groups. Processing and Save and Loan groups and two umbrella organizations of UPA farmer groups (FLUPFA and TUPUFU) have been founded. Only some of the groups as well as FLUPFA und TUPUFU are formally registered. Many groups and especially TUPUFU and FLUPFA still depend on training and partly also on financial support of the project. During the follower project UPA 2 it is planned to strengthen the groups further. During the evaluation the group members mentioned as main factor for strong groups the success of the group. If the members have tangible benefits from their group, they are willing to participate and to improve their group. Another factor observed was the transparency of group management and of the administration of funds. 27

37 Another relevant outcome is the increased knowledge of the majority of 1000 persons about agricultural and animal husbandry production, management and other topics. 70% of interviewed persons confirm that their knowledge has improved much or very much through project activities. However the knowledge about the cross-cutting issues gender and HIV- AIDS was less divulgated. 50% of interviewed persons say that project activities have not improved their knowledge about these topics, only 35% much or very much. Some beneficiaries also mentioned increased solidarity and togetherness among group members. Especially the Save and Loan groups increased their self-help potential, allowing 234 members to overcome situations of emergency and sending children to school. Another socio-cultural outcome of the project is the decreased dependency on external inputs, which are expensive and often not available on time and in good quality Organizational and institutional / political outcomes Results 1 and 2 are covering organizational and institutional changes, result 1 at the political level, result 2 at the level of beneficiaries and has already been described under sociocultural outcomes (5.1.2). Result 1: Planned: Relevant Ministries and municipal authorities accept importance of peri-urban agriculture (UPA) and actively support farmers. Achieved: The importance of UPA is well accepted by governmental institutions and municipal authorities and first steps of active public support to farmers have been realized. However at this moment the concrete support is still limited. In the case of Fiamah, the governmental support to UPA farmers is still under negotiation. Gardeners / farmers have to return the land to the town administration of Monrovia which needs the land for the reactivation of the city purification plant, but did not yet offer them another piece of land to continue their farming activities. At present, farmers with project support negotiate with the town administration about the access to alternative land. The fact that dialogue and negotiation takes place is a positive outcome; the successful access to land would prove that the town administration takes the support to UPA serious. Another positive outcome is a public budget of 7 million LRD for UPA, according to the MOA, including the salary of one officer for UPA, even if it is not totally clear what will be financed with this budget. In one of the visited groups the members mentioned as an important result of the project that the group was linked to governmental institutions During the evaluation interview partners did not confirm the preoccupation expressed by some experts in Liberia that the successes achieved with regard to result 1 could be damaged by the decentralisation process and by reorganisation of local government structures. On one hand the opinion was expressed that decentralisation and reorganisation are very slow processes which will not have many consequences in the near future, on the other hand it was mentioned that if the decentralisation process was successful, this would not mean necessarily that there is more personnel change than usual. 28

38 5.1.4 Environmental outcomes With the exception of the concept of low external input agriculture, environmental aspects were no specific concern of the project and no specific environmental measures were planned. Many of the members of the garden groups and backyard gardeners used the training and extension offered by the project to practice low external input technologies such as use of manure, composting and bio-pesticides (see 5.1.1). 5.2 Impacts The following diagram shows the average of the results of the trend analyses with the five groups which participated in this exercise, comprising a total number of 75 participants (32 men and 43 women). Two of the five groups were groups composed of CFTs, one in Monrovia and one in Tubmanburg. Diagram 6: Average of the results of the trend analyses of 5 groups Agricultural production Food security Marketing of agricultural products Income Support of Government to UPA /13 Source: own survey The group members assessed their situation before the project (2009) and today with numbers between 0 (= nothing) and 5 (= very much / very good) and analyzed the reasons for the change of the situation. The results show that the average of the marketing and income situation improved more than the agricultural production and food security. The support of the Government to urban and peri-urban agriculture was assesses as less during the last years. This is caused by the fact that one of the groups has to leave the land they had received from Government and by the fact that in 2009 one group had received boots from the MOA, but no other handouts after that year. In general the situation of the two CFT groups with exception of the support of governmental institutions to UPA - improved clearly more than the situation of other groups Economic impacts Regarding the income of beneficiaries, the comparison of the results of the survey realized during the evaluation and the baseline study carried out in February 2011 showed the following changes: 29

39 Diagram 7: Beneficiaries view of the household income vs. income of other families in the region Baseline study 2011 Survey of evaluation 2013 Very poor 1% 3% Poor 25% 9% Fair 27% 12% Good 34,5% 52% Very good 5,5% 24% The table indicates that the number of surveyed persons with good and very good income increased strongly from 2011 to On another question, 44% of households surveyed during the evaluation answered that their household income has improved much or very much through project activities, 41% considered the improvement little or very little, 15% considered that there was no improvement. Considering only the group members which are not CFT or members of the leadership the situation is less positive: 38% of surveyed households answered that their household income has improved much or very much through project activities, 45% considered the improvement little or very little and 17% said that there was no improvement. It can be estimated that about 60% of the target group increased their income through project activities. There are no data available about the concrete extent of income increase neither from the baseline study nor from the monitoring system. The survey 2013 showed that CFT improved their income more than ordinary farmers (see answers to question 18). Households reported improvements of the income of a few percent up to 100% or between 15 and 640 per year. However, in some cases these reported income increases seem to be not at household, but at group level Socio-cultural impacts The comparison of the results of the survey realized during the evaluation and the baseline study carried out in February 2011 showed the following changes regarding the number of meals per day: Diagram 8: Number of meals per day Baseline study 2011 Survey 2013 One time 60% 38% Two times 36% 56% Three times 3% 12% The results show a clear improvement of the number of meals per day, and therefore of food security. This impact is verified by the answers to the question 9 of the survey 2013, asking if the availability of food improved through project activities: 30% confirmed little improvement, 24% much improvement and 30% very much improvement. Also the trend analyses (see above) showed an improvement of food security between 2009 and 2012/2013, however to a lesser extent. The number of households suffering from insufficient food during the last 12 months was reduced from 87% in 2011 to 61% in 2013 (baseline study vs. survey 2013). 30

40 Also the variety of food improved. Comparing 2011 and 2013 the surveyed households gave the following answers: Diagram 9: Beneficiaries assessment of the variety of food during the previous month Baseline study 2011 Survey 2013 Very poor 1,5% 0% Poor 28% 6% Fair 19% 15% Good 50% 55% Very good 0,5% 21% According to the survey 50% of households eat new sorts of food today which they did not eat three years ago. The different information leads to the conclusion that food security was improved clearly, in terms of stability or diversification, among up to 70% of the beneficiaries Organizational and institutional / political impacts The farmers organizations TUPUFU and FLUPFA have been founded and are representing farmers interests in the Multi-Stakeholder Forums and at other opportunities. This process is relatively new and did not yet produce many tangible impacts for the farmers. Two Multi- Stakeholder Forums have been established in Monrovia and Tubmanburg, creating high levels of interest among concerned Ministries and municipal councils. They allow contact and cooperation between UPA producers and Government, one of the impacts formulated in the results chain. It is possible that in the future TUPUFU and FLUPFA will play a significant role and that political decisions will favour farmers as a result of common policy formulation and land use planning. The results of the evaluation confirm that at present farmers do not note much improvement of the support by governmental institutions. According to question 26 of the survey, 47% consider that governmental support for UPA, processing and marketing activities did not improve in the last years, only 30% consider that governmental support improved much or very much. The result of trend analyses (see above) showed a decrease of governmental support to UPA in the last years. This is due to the loss of land by a group (Fiamah) and the discontinuation of distributions to UPA farmers by government (see also 5.1.3). Two groups participating in trend analysis were affected by these developments, leading to negative results Environmental impacts The concept of low external input agriculture caused some positive environmental impacts, such as the conservation of soil fertility and positive impacts on human health and the environment through a reduced use of chemical pesticides. On the other hand the partly extreme environmental conditions in the gardens through litter and waste water have not been treated sufficiently during UPA 1. In some cases the project measures have caused the extension of gardens and marketing of the products, increasing the hygienic problems and the risks for human health. 31

41 5.3 Contribution of the project to the millennium development goals The project is contributing especially to the millennium development goal 1 (eradicate extreme poverty and hunger). Even if there is no gender policy of Welthungerhilfe in Liberia and no special measures have been implemented to improve gender equality in the evaluated project, the fact that a high percentage of the beneficiaries are women improved their situation in many cases. Therefore the project measures contributed to the empowerment of women, what is part of the millennium development goal 3 (promote gender equality and empower women). With the divulgation of low external input agriculture project measures influenced many gardeners to use more organic material to ensure soil fertility and to use less chemical pesticides. This contributed up to a certain degree to the millennium development goal 7 (ensure environmental sustainability). Indirectly the Save and Loan groups contributed to a lower extent also to the millennium development goal 2 (achieve universal primary education), enabling school attendance of the children also in situations of financial constraints. When the members of Save and Loan groups were asked for what they take loans, school fees was one of the answers given most frequently. 6 Sustainability The project UPA 2 is working to improve the sustainability of the activities and results of UPA 1 during the next two years. The survey realized during the evaluation produced the following answers on the question which activities or results initiated through the project will continue (in the opinion of the interviewed persons) if there is no more support by the project: Improvements of agriculture and horticulture through project activities: 25 (78.13%) Improved animal husbandry: 8 (25.00%) Processing activities initiated by the project: 10 (31.25%) Marketing activities initiated by the project: 24 (75.00%) Group activities (if you are member of a group): 18 (56.25%) Irrigation systems: 2 (6.25%) Activities of the Community Farmer Trainers (CFT): 19 (59.38%) Urban Farmers Associations (FLUPFA or TUPUFU): 17 (53.13%) Demonstration plots: 4 (12.50%) Others - please explain what kind of activity: 1 (3.13%) The observations and interviews realized by the evaluator did not always confirm these results if they refer to the end of UPA 1. In several cases they seem to be too optimistic, especially in the case of marketing, CFTs and TUPUFU /FLUPFA. 6.1 Economic financial / food and nutrition security It can be expected that the improvement of food and nutrition security will continue after the end of project support, because in many cases beneficiaries will continue to apply their enhanced knowledge in subsistence production. The sustainability of the individual gardens and partly also of the group gardens seems to be high. On the other hand, the income of beneficiaries is depending on the success and sustainability of various factors such as the processing activities, the weekly market and 32

42 animal husbandry activities which at this moment are not yet able to continue without external advisory or financial support. UPA 2 is challenged to work intensively in these areas. In the case of animal husbandry, the sustainability depends on the availability of and access to affordable local feed for poultry and pigs. A solution to this difficulty still has to be found by the project in cooperation with the farmers. In marketing activities, the market linkages supported by the project are more sustainable than the weekly market at this moment. The daily market stand in Tubmanburg is still under construction; therefore no assessment of sustainability is possible at present. Most of the processing activities are quite new and not yet consolidated. In conclusion, the perspectives of a sustainable income increase are not as good as in the case of food security. 6.2 Socio cultural sustainability Some of the socio-cultural improvements, such as mutual support of group members, the level of togetherness and the empowerment of many women which participated in project activities is likely to continue at least partly - without outside support. The enhanced knowledge of beneficiaries in production, management and other areas appears to be sustainable and can be used in the future in different ways. This concerns especially women which are more than half of the target group. The expected voluntary work of CFTs as trainers and extension agents did not work as planned and it cannot be expected that most of them continue as CFT, but they have received much knowledge during the project, so some of them will continue to play an important role as leaders in their groups. In UPA 2 project staff trys to realize a more sustainable approach, the groups are selecting their representatives for trainings and other measures and they are responsible to their groups and no longer to project staff. 6.3 Organizational and institutional / political sustainability A large percentage of groups supported during the actual phase are likely to continue, considering the continuous support during UPA 2. This is especially the case for the Save and Loan groups. At present FLUPFA and TUPUFU have neither the financial conditions nor the backing and support of their members to continue independently. The Multi - Stakeholder Forums are able to continue if the costs for meetings etc. will be financed by the UPA budget of the Ministry of Agriculture or another source. No decision has been taken in this regard up to now. 6.4 Environmental sustainability It can be expected that a part of the farmers will continue to practice low external input agriculture, keeping the level of negative environmental impacts low. 33

43 7 Recommendations 7.1 Recommendations to Welthungerhilfe Welthungerhilfe is recommended to continue its support to the follower project, ensuring an adequate monitoring system and analysing together with the project management if and how the cost / benefit ratio could be improved. 7.2 Recommendations to the project management As the project under evaluation UPA 1 already finished, all recommendations are supposed to be considered during the follower project UPA General recommendations 1. Monitoring It is necessary to use and actualize the monitoring system, including the indicators for results. Maybe simple record sheets for each target group could improve the overview of difficulties and successes of each group and enable a more individual accompaniment of the groups. 2. Cost / benefit ratio It should be tried to improve the cost / benefit ratio, improving the impacts of project activities for the families, to increase the number of families (what is already in process) and to reduce the costs for activities which do not have direct results for the target groups. 3. Sustainability The sustainability of different project elements (processing groups, farmers market, FLUPFA and TUPUFU, Farmers Resource Centre, animal husbandry) is still not ensured. Exit strategies should be elaborated for these elements during the project UPA Gender and HIV /AIDS Staff should be trained in the cross cutting themes gender and HIV/AIDS and some information about these issues should be included in the following trainings. 5. Trainings, accompaniment and coaching of the target group In general, different kinds of trainings with the group members should be continued and intensified (if funds are available). In spite of the difficulties to find good consultancies for calculation and management trainings, the efforts to train groups (especially processing and marketing groups) in these areas should be continued and intensified. Before the processing groups receive equipment, they should be able to make their own cost/benefit calculation. Special leadership trainings for women should be included if they are interested in these trainings. The accompaniment and supervision of groups should be done more systematically and more adapted to the individual cases. Together with the group members it should be analyzed what kind of advisory service each group needs. Some groups, especially the processing groups, need more knowledge in calculating and book keeping, not only for the leadership but also for interested members. Maybe one of the staff members could be responsible for a continuous accompaniment of the groups, supporting their accounting and 34

44 other administrative activities. The field officers should work together with group members to identify viable solutions to the special problems of each group. This refers also to the marketing (weekly market) and processing activities Recommendations regarding result 1 6. Cooperation with the Ministry of Agriculture (MOA) Representatives of the Ministry of Agriculture suggested in spite of a good cooperation in general - to improve the coordination between the project and MOA. It should be analyzed if and how this coordination could be improved. 7. Farmers Resource Centre (FRC) To reduce project costs and to improve the extension service for the farmers it should be analyzed if for the Farmers Resource Centre another solution can be identified without the need of financing through the project. In this case the field officer working at the FRC now could improve the project extension service and trainings could be organized at farmers fields or gardens. An additional reason for this recommendation is the fact that chances for the FRC s sustainability after project end is relatively low. 8. Land issues Some of the groups have serious land problems; in some cases project staff tried to support them in these difficulties, in others not. The access to land should still be more focussed by project staff. These issues should be addressed in the dialogue with municipal and other authorities which have to be held accountable for the access to land of their citizens Recommendations regarding result 2 9. Group approach In the future it should be decided by the group members if they want to carry out the activities as group activities or individually and also if they plan to become a formally registered group or even a cooperative or if they would like to remain an informal group. It seems that in animal husbandry activities and distributions an individual approach inside a group (e.g. to ensure the transfer of animals towards a larger group of beneficiaries) should be focused on. Artificial groups which are only built to receive inputs or animals should be avoided. Existing organizational structures should be considered and included. If new groups are selected, project staff should elaborate criteria for the selection. Also for the assessment if the groups are weak or strong, to discuss their development and to carry out self evaluations, criteria should be elaborated together with the group members. 10. FLUPFA and TUPUFU TUPUFU and FLUPFA leadership needs to inform their (future) members (not only CFTs) much more and should search for possibilities to get into a more intensive contact with their members. One possibility could be to organize plenary meetings to elaborate together with the members clear objectives and activities of the organizations and strategies for the financial sustainability. Some members of the leadership aim to develop TUPUFU towards a local NGO, but at the moment this seems too early and could cause conflicts of interests 35

45 (between the representation of interests of the UPA farmers and the interests of a local NGO). Also it should be analyzed if FLUPFA and TUPUFU as independent organisations are the best way of urban farmers representation or if a sub-group of the more consolidated national farmers organisation FUN was a better solution. Project staff should support this process without taking too much initiative. Income generating activities for the UPA farmers organisations are necessary to ensure their sustainability. FLUPFA and TUPUFU may consider complementing the operations of FUN, focusing on supporting services for economic activities of their members such as marketing, providing animal feed, seed and plants. FLUPFA and TUPUFU (and maybe also FUN) should play a stronger role in the Multi- Stakeholder Forum (MSF) and in the Multi-Stakeholder Action Planning and Policy Formulation (MPAP) process. If necessary and wanted, the project should offer supervision for these organizations, reflecting together the issues of discussion. Additionally it should be tried to include FLUPFA into the process to decide on the application of the MOAs budget for UPA. 11. Save and Loan (S&L) groups The target groups are very interested in Save and Loan groups. It should be tried to motivate people still more to invest money in agricultural, animal husbandry and other business activities Recommendations regarding result More extension service to agricultural and horticultural groups The agricultural low external input approach should be continued and further improved. The project staff should intensify the extension service in the case of problems and look together with the target groups for solutions to special problems (grasshopper, crabs, water, etc.). Additional bio-pesticides (e.g. neem products which are already developed) should be explored and included in trainings. 13. Feeding of animals Locally available and cheap animal feed with sufficient protein should be a central issue in the trainings (simple and easily understandable didactic material is needed) and during the follow-up extension visits of the field officers. If possible (land available) animal husbandry should be connected to agricultural or gardening activities (as planned in UPA 2) and feeding should be diversified to avoid the dependency of only one feed stuff (wheat bran). Production of cassava (tubers combined with the leaves which contain relatively much protein) and fish meal or fish and slaughter waste, rice bran, etc. could be components of animal feeding. 14. Linking of agricultural and animal husbandry activities In order to improve feeding of the animals, soil fertility, etc., agricultural / horticultural activities and animal husbandry should be linked whenever possible - within the farming groups or if this is not possible between groups (this is already done in UPA 2). 36

46 15. Distributions The project is working with a target group which is still expecting free distributions as was the common practice in relief projects. During UPA 1 the distributions have been reduced gradually. This should continue during UPA 2. In the future, distributions without payment should be avoided wherever possible. If seed is distributed, only seed of very good quality should be distributed. Distributions have to be on time, considering the agricultural calendar. 16. Waste, polluted water and hygiene In the UPA 2 project it is planned to analyze the hygienic and pollution situation of soils and water in the gardens. This problem could turn out to be a serious health hazard and should therefore be focused by the follower project. Respective measures (at least awareness trainings) should be realized Recommendations regarding result 4 and 5 See recommendation Focussing processing products The support to processing activities should focus only on the most promising products (this seems to be already realized in UPA 2). The recommendations number 1 (monitoring), 2 (cost / benefit ratio), 3 (sustainability), 4 (gender and HIV/AIDS), 5 (trainings and extension service), 9 (group approach), 10 (FLUPFA and TUPUFU), 13 (feeding of animals) and 16 (waste, polluted water and hygiene) are seen as priority. 8 General conclusions 8.1 Important lessons learnt Complex implementation structures with several partner organisations Working with many project partners as implementing organizations increases both costs and the dependency of the project on the quality of the performance of implementing organisations. On the other hand, this approach is able to develop local partner organisations, improving the knowledge and skills among their staff. This may contribute to the long-term development of partner countries. The cooperation with international organisations is able to bring together specific skills and approaches such as UPA in the case of RUAF and experiences about Save and Loan groups in the case of CARE. Multi-level approach The multi-level approach to work with political decision makers, UPA farmers and their organizations allows increasing the possibility to develop and find solutions in critical areas such as land, marketing and waste management and maybe enhance also the sustainability of some activities. 37

47 Litter and waste water A critical factor which has to be considered in urban agriculture right from the start of project planning is the problem of litter and waste water requiring specific approaches. Animal husbandry Animal husbandry without own fodder production is highly risky, especially in urban farming because land to produce own fodder often does not exist. The necessity of affordable and available feed should be considered if support to animal husbandry in urban regions is planned. Irrigation Also in very wet climate conditions, irrigation in vegetable has to be considered in the planning phase and measures have to be planned if there is lack of availability of irrigation water. Voluntary work It is risky to base the project methodology on voluntary work of the target group as it was done in the case of CFTs, especially in the situation of Liberia where other projects are paying representatives of the target group for similar activities. Alphabetization If project measures to improve processing or marketing are planned it should be analyzed if the target group needs training in basic knowledge, such as reading, writing or calculation. 38

48 8.2 Positive examples from the project / good practice The seven Save and Loan groups established with support of the project staff are highly appreciated by their members and seem to improve their situation with the introduction of self help instruments. One of the groups is described in chapter 9 as example for the annual topic The following table describes the good practice of Save and Loan groups. Diagram 10: Save and Loan groups as good practice Title Save and Loan groups in Liberia Context The population is often confronted with situations of financial shortage which sometimes lead to food shortage of the families, to the ejection of children from schools if the fees cannot be paid or other difficult situations. Save and Loan groups can provide loan and solidarity in these difficult situations. Loans can be used also to start small income generating activities (often trading) or to undertake investments in agriculture and animal husbandry. Resources Project staff / a save and loan agent is supporting the establishment of the groups with training and follow-up visits. Description The different steps to establish a Save and Loan groups are described in the VSLA (village save and loan agent) training manual which is sent together with the evaluation report. General A group needs about 30 seriously interested persons with a minimal conditions financial conditions (the number of meetings per year and the saved amount of money will be determined by the group members) and the instructions of an agent, experienced in the process to establish Save and Loan groups. Links Contact: Andre Stelder (head of project LBR 1026) in Monrovia Date March Welthungerhilfe s specific annual theme for cross-cutting evaluation One of the seven Save and Loan groups supported during the implementation of LBR 1026 was selected as an example to show how the intervention of the project improved the selfhelp potential of a group sustainably. Victoria Sirleaf from Monrovia is member of two groups supported by the Welthungerhilfe project: one is a vegetable group and the other is a Save and Loan group. The group started with 30 women and also Victoria s daughter is one of these members. 39

49 During one year the 30 group members met twice per month and each member paid to LRD (= ) in each meeting. The money saved in this way is used for loans. Everybody can take a loan for a period of up to three months and pays 10% interest for this period. If anybody takes a second loan the interest rates are lower. At the end of the year the interests are shared between the members. If a person does not return a loan, the contract suggests that the house of this person is locked by the other group members until the money was paid. Victoria Sirleaf is happy that this did not happen until now, everybody paid as foreseen. The group members take loans for critical financial situations, often to pay school fees, but also for small trading activities such as paddy trading. Victoria Sirleaf stated that the Save and Loan group is important for the group members because they support each other in the case of financial difficulties. The following photos show Victoria Sirleaf in the garden of her vegetable group. They demonstrate that the lack of waste management can be a serious problem to urban agriculture. 40

50 Annexes Annex 1: Terms of reference Standard Terms of Reference 1 for the evaluation of individual projects in the field of development cooperation Welthungerhilfe Evaluation Unit here: AF 1582 /LBR 1026 The primary aim of the independent evaluation is to use impact analyses and a target-actual comparison to determine the project outcomes and impacts (direct and indirect, short, medium and long-term, intended and unintended). The evaluation is based on information in Welthungerhilfe s internal project document (binding target system) and in case the contract with the co-financer, the project s own monitoring and the evaluation of the consultant. The foundations for the evaluation are provided in the concept paper Outcomes and Impact- Oriented Evaluation of the Work of Welthungerhilfe (previously: German Agro Action) 2. The evaluation is thus based on DAC criteria (efficiency, effectiveness, relevance, impact and sustainability) and the standards of the DAC and DeGEval. Welthungerhilfe s Standard Terms of Reference consist of two elements: the standard outline for the evaluation report (A) and the orientation questions (B). Notes on the evaluation (a) and a template (outline) for the brief report for the management of Welthungerhilfe (b) are attached in an annex. Country: Project title: Project No.: Project holder: Approved budget: Committed funds: Co-financer (line): Liberia Enhancing urban and semi-urban agriculture in Liberia AF 1582 /LBR ; EU: DCI-FOOD/2009/ Welthungerhilfe in partnership with Human Development Foundation (HDF) ,-Euro Welthungerhilfe: ,-Euro; EU: ,- Euro European Union Project period: 09/ / The Standard Terms of Reference of Welthungerhilfe were discussed with the evaluation departments of the Federal German Ministry for Economic Co-operation in Bonn and with the EU in Brussels. The quality of the evaluation of Welthungerhilfe is assessed by external agencies acting upon the commission of the Federal German Ministry for Economic Co-operation on a regular basis. 2 Welthungerhilfe concept paper: Wirkungsorientierte Evaluation der Arbeit der Welthungerhilfe, 2. überarbeitete und aktualisierte Auflage, April

51 Introduction: Overall objective of the project is to contribute to the poverty alleviation and sustainable urban development in Liberia. Specific objective: Small scale urban and peri-urban producers improve food security and increase income and UPA is officially recognized Partner is the Human Development Foundation (HDF) Target group: 1500 direct beneficiaries: Selected vulnerable urban and peri-urban Farmers and Farming Communities in Liberia, Western Greater Monrovia, and Tubmanburg (Bomi County), with special emphasis on women and youth farming groups and local and municipal authorities. Final beneficiaries: Vulnerable Liberian Farmers, urban and peri-urban communities, urban consumers (6200 indirect beneficiaries) Estimated results: Result 1: Relevant Ministries and municipal authorities accept importance of peri-urban agriculture (UPA) and actively support farmers. Result 2: The formalization and organization of UPA farmer groups is strengthened Result 3: Agricultural production is enhanced and diversified. Result 4: Processing and value adding activities are implemented and income generation activities are identified and promoted. Result 5: Market links and marketing concepts for UPA are established. Main activities are (a) to establish agricultural production systems that make efficient use of existing human and natural resources and improve food security and income (b) to establish a market orientation which produces the right product for the right buyer at the right time and price; (c) to develop appropriate business and marketing skills and organizational schemes which lead to economies of scale, and; (d) to improved links among market chain actors and flows of information and technologies. The evaluation is an end of project evaluation, a follow up phase is envisioned. A Outline: Table of contents List of abbreviations and acronyms List of tables, illustrations, boxes, etc. Acknowledgement and disclaimer I. Summary (reporting language) 1 Brief description of the project and framework conditions 2 Relevance 3 Effectiveness 4 Efficiency 42

52 5 Outcomes and impacts 6 Sustainability 7 Most important recommendations 8 General conclusions and lessons learnt II. Main text 0 Prior remarks 0.1 Reason and aim of the evaluation 0.2 Evaluation process 0.3 Evaluation methods 1 Description of the project 1.1 Brief project description 1.2 Framework conditions 1.3 Brief description of the target group 2 Relevance 2.1 Relevance to core problems of the target group 2.2 Relevance to the objectives of Welthungerhilfe 2.3 Relevance to the objectives of the partner organisation and the partner country 2.4 Other aspects (e.g., relevance in the sense of the technical notes of the EU) 3 Effectiveness 3.1 Achievement of project purpose 3.2. Project preparation Quality of target group, actor and situational analyses Extent self-help approaches and structures are taken into account 3.3 Project planning Quality of the project planning matrix Analysis of the results chain Realism of project purpose Appropriateness of staff, material and financial planning 3.4 Quality of the analysis of project holder (partner(s) or external structure of the Welthungerhilfe) Organisational structure and focal points of work Staff, equipment and logistics Project steering and financial administration 3.5 Steering by the head office, regional office and cooperation with the project holder 3.6 Quality of the project execution Presentation and evaluation of the project outputs and activities Involvement of the target group and project cooperations Overall evaluation of the execution Occurrence of assumptions and risks 4 Efficiency 4.1 Cost/benefit ratio for the overall project 4.2 Cost/performance ratio for the individual measures 4.3 Quality of the internal project M&E system 4.4 Other aspects 5. Direct and indirect, short, medium and long-term outcomes and impacts 5.1 Use of Outputs / Outcomes Economic outcomes Socio-cultural outcomes Organisational and institutional/political outcomes Environmental outcomes 5.2. Impacts Economic impacts Socio-cultural impacts Organisational and institutional/political impacts 43

53 5.2.4 Environmental impacts 5.3 Contribution of the project to the millennium development goals 5.4 Other aspects 6 Sustainability 6.1 Economic-financial / food and nutrition security 6.2 Socio-cultural 6.3 Organisational and institutional/political 6.4 Environmental 6.5 Other aspects 7 Conclusions and recommendations 7.1 Project specific conclusions 7.2 Recommendations 8 General conclusions 8.1 Important lessons learnt 8.2 Positive examples from the project (good practice) 9 Welthungerhilfe s specific annual theme for cross-cutting evaluation Annexes Terms of reference Project planning matrix Travel and work schedule References,sources (resource persons, documents, surveys, etc.) Case Studies, special narrative, Minutes of the final on-site meeting, de-briefing, de-briefing workshop Maps Pictures, photographs and comments, explanations Others B Orientational questions: The orientational questions listed below, which were developed in a participative manner, draw attention to issues which particularly need to be taken into account. These questions have been formulated by the project holder or partner, by the regional desk in charge or the Evaluation Unit. They are of special interest in this evaluation. The responses to the orientational questions are supposed to be presented under the corresponding chapters of the report in accordance with their content. If necessary, separate sub-chapters should be inserted. Orientational questions: Concerning Result 1, it would be interesting to know if the progress made with some townships can be sustained, if the process of decentralisation and the reorganisation of local government structures (now a political battle field ) will reshape the playing field. Concerning Result 2, there will be increased working with farmer organisations in the successor project. What do groups expect and need from farmer organisations? Also: why do some groups flourish and others perish (positive deviance?) What are the critical success factors? How can female participation in group leadership be improved? Also Result 2: did the role of the Community Farmer Trainers (CFT, lead farmers) improve during the project (first there were the usual expectations of money in the 44

54 pocket before they do something, later the coaching on the job and the support to CFT in the form of inputs for onward selling to group members)? Result 3: do the farmers see the benefits of the low external input technologies? Did they actually change their view on farming (more ecological and market oriented approaches)? Why do some farmers adapt and others not? (again positive deviance). Result 4: Why do some individual farmers in a group show good sales of many different crops whilst other group members just take the easy way out producing the fast growing fast selling (but very low value) indigenous crops. Do the farmers make any attempt to counter the seasonality of their sales (meaning to say: Do they actually try to grow crops for sales off season at higher prices). Annual topic : Since 2006, Welthungerhilfe has started to have questions of general and strategic interest analysed in a systematic manner also through individual project evaluations. To this end an annual topic is named which is supposed to be evaluated in all of the individual project evaluations for the respective year. Then a general assessment is performed. For no specific topic has been set, instead, focus should be on the socalled stakeholders and specifically the target group of the project or program, as well as criteria resp. sub-criteria (like e.g. vulnerability) of the target group. At this stage, in 2013, each evaluation should come up with one or two short authentic case studies, indicating on how a small group, few individuals or an individual him- /herself participated and made use of the project, has been reached and changed certain aspects of his/ her life or family. The evaluator will be free to choose and decide for a specific case, influence of the project on an individual participant or person could be positive or even (unintentionally) negative. Most important is, that the case should be somehow special The format here is free and should be considered as room for creativity, photographs with captions (name, location, activity shown) and /or graphs and maps are most welcomed to be added for illustration. 45

55 Annex: a) Notes: General notes on the evaluation: The consultant is to produce the following during the evaluation: o Documentation of the preliminary findings and recommendations along with the final discussion on site (lingua franca) o Main evaluation report (to be respectively specified) o Internet version of the report summary (language as in the main report) o Brief report of the evaluation for the Secretary General (German) In particular: o The most important findings and recommendations of the consultants are to be presented to the project partner/project team on site (in writing) and discussed with them. The most important results of this final discussion (consensus/disagreement) are to be recorded. This final protocol is to be signed by the consultant and the director of the partner organisation or the project manager. A representative of the regional office in charge should participate in this discussion if possible. o The main report should comprise about 35 pages (not including table of content and annexes). It should also be readily understandable to outsiders. o Chapter I. (summary) of the main report will be published together with the cover sheet of the main report of Welthungerhilfe in the Internet 3 (Internet version). o The brief report of the evaluation results for the Secretary General is to be based in terms of its content on the summary of the main report, but is to be drafted in German and should not exceed 3 pages. The headings for this summary are provided in the Annex. The evaluation should be outcomes and impact-oriented. This should be evident not only in the chapter on outcomes and impacts (5) attention should also be devoted in all the other chapters to making sure that aspects related to outcomes and impacts are taken into account. Welthungerhilfe is often called differently in the local working language. Only the German name Welthungerhilfe [ full name, no abbreviation or acronym please!] should be used in the report. The final version of the main report is to be submitted electronically both in PDF and in a format which can be edited whenever possible, and otherwise as a printout. The main report may contain up to a maximum of 5 pictures with direct explanatory notes, Otherwise a separate annex should be provided and even more than 5 pictures are most welcomed. Best would be to avoid any pictures in the report (in order to keep soft copy storage space for communication limited) and preferably put all pictures (with captions) and maps in a separate annex. 3 For data-protection reasons it should be taken into account that it is not allowed to state any personal data in this chapter. Personal data is individual information on personal or material conditions of natural persons who are either known or can be imputed (persons affected). 46

56 Special notes on the individual outline points in the report: I. Summary The summary should comprise a maximum of approximately 3 pages, it should be written in the reporting language and be in line with the short German version for the Secretary General of Welthungerhilfe. II. Main text; in particular: To chapter 0.: 0.2: The following aspects among others should be taken into account: time period of the evaluation, involvement of the holder in the drafting of the TOR and execution of the evaluation, extraordinary events, composition of the team. Also (0.3): scope of the analysis, what methods and if applicable participative questions and what sort of triangulation have been used (it is probably meaningful to use e.g. a triangulation from observations and data gathered on site by the consultants, participative survey of the perception of the target group as well as interviews with key persons and analysis of documents ). To chapter 1.: (1.1) Very brief description of the project; note whether project is being implemented by a partner or Welthungerhilfe itself; if relevant, where it is located in a programme (1.2) framework conditions relevant to the project should be presented at the national and at the project level. (1.3) The direct and indirect target groups are also to be quantified. To chapter 2.: (2.2) The relevance of the project to the objectives of Welthungerhilfe is oriented towards the respective goals set out in the statutes, concept papers, country/regional concept papers and programs, and the Welthungerhilfe orientation frameworks, (including the impacts guidelines 4 ) or towards the Venro documents which are binding on Welthungerhilfe, (which documents this is to be based on is to be coordinated with the consultant in the individual case). Has the project been designed in accordance with theses concept papers and orientation frameworks?) Under other aspects (2.4) the consultant has in the context of EU-cofinanced projects, for example, to evaluate whether and how the project is relevant to the Objectives of the programme and priority issues guideline for applicants mentioned in the EU technical note ) (referred to as promotional concept in the cases of projects co-financed by the Federal Ministry for Economic Cooperation. To chapter 3.: The chapter effectiveness includes the aspects of quality assessment of project preparation, planning, steering and execution as well as analysis of the holder. The chapter starts with an evaluation of the achievement of the project purpose. At outcome level, there could be some overlaps with chapter 5: In chapter 5, all identified outcomes are 4 Welthungerhilfe guideline Wirkungsorientierung in den Projekten und Programmen der Welthungerhilfe, 3 parts, October

57 to be presented. In chapter 3.1 only the outcome(s) described in the project purpose are to be presented and this in a brief manner (in order to avoid duplication). (if needed, a table showing how indicators have been met can be presented,). All following sub-chapters should concentrate on the conducive and hindering factors for the achievement of the purpose and overall goal. (3.2) Among other things, the problem and potential analysis, selection of target groups and coordination and complementarities with partners are to be analysed under project preparation. The involvement of target groups is a qualitative feature in evaluating project preparation. The quality of the project planning matrix (including the quality of the indicators and the gender aspect) is to be analysed under summary of project planning (3.3.1) (including the quality of indicators and the gender aspect) 5. In the analysis of the results framework (3.3.2), it should be determined what results hypotheses the project planning matrix is based on (or what results hypotheses can be derived from the project planning matrix) and to what extent these conform to the perspective of the target groups, partners and requirements on site. (3.4 and 3.5) In the case of projects implemented by Welthungerhilfe itself, the term holder applies to the Welthungerhilfe project team on site. (3.4) The evaluation of the holder should include gender aspects. (3.6.1) The presentation and evaluation of project outputs are to be oriented towards the project planning matrix and the indicators relating hereto (planned / executed). Possible subpoints in chapter 3.6 should be created in accordance with the output areas; the main activities conforming to these can be summed up in a table. (3.6.4) The occurrence of planned assumptions and risks, unexpected risks which developed and adjustments in the planning and steering required as a result should be discussed here. To chapter 4.: The statements made regarding efficiency should be to the extent possible assigned numbers. In the case of certain project types (e.g. building a social infrastructure through Food for Work ), this is only warranted to a limited extent. Here it suffices to make an estimate and if need be discuss whether there are more cost-effective alternatives available. The analysis of the cost/benefit ratio for the overall project (4.1) relates to the relationship between impact and input, the analysis of the cost/benefit ratio for the individual measures (4.2) applies to the relationship between input and outputs. In the analysis of the efficiency of the individual measures (4.2), calculations of efficiency of the measures themselves (if applicable including calculations of the contribution to the variable gross margin) should be considered. (4.3) In assessing the internal project M&E, the monitoring of outcomes and impacts particularly needs to be assessed. Here it should not only be stated whether the data has been systematically recorded, but also whether it is being used by the project management. (The following general monitoring indicator is identified in the Welthungerhilfe Strategy and it should be possible to answer this based on the evaluation reports: Outcomes and impacts are systematically recorded, analysed and translated into improvements in the project work with respect to project work abroad.) 5 VENRO, gewusst wie Gender in der Entwicklungsarbeit, December

58 To chapter 5.: of the report. The chapter on direct and indirect outcomes and impacts is the focus Note: A project of Welthungerhilfe concentrates on the first level of impact. This usually conforms to the project goal: How is the project output used by the target groups and what direct outcomes emanate from the project? This level must be evaluated by the internal project M&E system and used for steering. The more general development-policy impact may be, but must not be, covered by the project s own M&E system. This should then be examined and assessed by a consultant with suitable instruments. In the case of short-term emergency aid projects (less than 12 months), the emphasis is more on monitoring of outputs and not on impact monitoring. The chapter differentiates outcomes (5.1) and impacts (5.2) The most important aspect here is the use and the benefit of the project for the respective target group. The chapter starts with the economic (5.1.1), socio-cultural (5.1.2), structureforming (including self-help capacities which have been established) 6 (5.1.3) and environmental outcomes (5.1.4). Similarly, the impacts can be differentiated (5.2.1 to 5.2.4), however if due to whatsoever reason the evaluation will be less able to specify, the evaluator may feel free to change the outline of this chapter and elaborate respective details in a different way. The presentation on these dimensions should relate to intended and unintended as well as positive and negative impact. The impact is to be analysed in a two-fold approach: with respect to the target indicators in the project planning matrix and in accordance with the perception of the persons involved in the project, in particular the target groups (gender sensitive) 7. There may be some overlapping with chapter 3.1 in the area of direct outcomes. In chapter 5, all of the outcomes which are identified are to be recorded. In chapter 3.1 only those direct outputs are to be recorded which have been described in the project target. Suitable participatory methods should be used to analyse the impact from the perspective of the target groups (MAPP, PALSA etc.) (The following general impact indicators are identified in the Welthungerhilfe Strategy ; it should be possible to use the evaluation reports to answer these: in 50% of the evaluations of relevance, a positive impact has been found to have taken place with respect to at least three livelihood factors.) Most of the projects of Welthungerhilfe relate directly to one or more of the millennium goals (5.3); the impact on these goals is also to be described; to avoid duplicate work, if applicable a reference to the relevant chapter merely needs to be inserted here. If additional dimensions are of relevance, these can be inserted as well (5.4). To chapter 6.: The chapter on sustainability must generally be broken down according to the same dimensions ( economic, socio-cultural, organisational/institutional, 6 The Welthungerhilfe working paper entitled Hilfe zur Selbsthilfe in der Programmarbeit should be used here. 7 The VENRO working paper Gewusst wie Gender in der Entwicklungszusammenarbeit should be used as background information for this. 49

59 environmental ) as the chapter on outcomes and impacts. In the case of ongoing projects, sustainability can usually only be examined in terms of potential sustainability. Sustainability can also be classified according to different criteria within the said dimensions (e.g. project-oriented, output-oriented, innovation-oriented or system-oriented). In other words, work aiming at providing support continues to be performed after the project is completed, other actors assume the measures or the target group is able to continue to develop the measures in the changed context. (6.3) Organisational and institutional/political may relate both to an improvement in community based organisations (water committees, etc.) and their institutions as well as the possible support for state structures and their institutions. To chapter 7.: Recommendations 7.1 To avoid duplication, the recommendations are only presented in chapter 7 (in addition to the summary).. The recommendations are also to be provided with a reference to the findings corresponding to them directly and containing the proper evidence. The following ordering criteria are recommended: To whom is the recommendation directed to (e.g. Welthungerhilfe or the project holder)? What priority does its implementation have? In what period of time is it supposed to be implemented? To chapter 8.: General Conclusions 8.1 Lessons learnt according to DAC are: Generalizations based on evaluation experiences with projects, programmes or policies that abstract from specific circumstances to broader situations. Frequently, lessons highlight strengths or weaknesses in preparation, design and implementation that affect outputs, outcomes and impact. Here also those lessons learnt should be stated, which the evaluator only considers to be of potential relevance and recommends to Welthungerhilfe to continue to monitor these aspects to determine their suitability as consolidated lessons learnt. To chapter 9.: The annual topic is to be explained in the orientation questions. Special notes on the use of important terms at WELTHUNGERHILFE Definitions and terms embedded in results chains are not used in a uniform manner by organisations working in the field of development cooperation. For this reason the most important terms for Welthungerhilfe in the results chain (diagram 1) are presented in the following diagram. (This diagram has been taken from Welthungerhilfe s 2007 guidelines Achieve Outcomes and Impacts.) 50

60 Diagram 1: Schematic overview of the results chain Diagram 2: Different terms in the result chain concept and project planning (based on the DAC Glossary) 51