Grenada Rural Enterprise Project (GREP) Project Completion Report Validation. September 2011 Independent Office of Evaluation of IFAD

Transcription

1 Grenada Rural Enterprise Project (GREP) Project Completion Report Validation September 2011 Independent Office of Evaluation of IFAD A. Basic Data A. Basic Project Data Approval (US$ m) Actual (US$ m) Region/Province Total project costs Country Grenada IFAD Loan and % % % of total Loan Number 559-GD Borrower (gov t) % % Type of project Rural Development Co-financier % % (sub-sector) CDB Financing Type F Co-financier 2 Lending Terms 1 I Co-financier 3 Date of Approval 26 Apr 2001 Co-financier 4 Date of Loan 25 May 2001 From Beneficiaries % NA Signature Date of Effectiveness 03 Oct 2002 From Other Sources: Loan Amendments 1 Number of beneficiaries (if appropriate, specify if direct or indirect) households beneficiaries (Women 5 030) (There is no estimate of indirect beneficiaries) Loan Closure Extensions 1 Cooperating Institution Country Programme Manager Regional Isabel Lavadenz Director(s) Josefina Stubbs PCR Reviewer Miguel Torralba IFAD Loan Disbursement at project completion (%) PCR Quality Fabrizio Felloni Control Panel Anne-Marie Lambert Please provide any comment if required: Caribbean Development Bank (CBD) Jaana Keitaanranta Loan Closing Date 30 June Dec 2009 Mid-Term Review Feb % (in SDR terms) Sources of this table: President s report, PCR, PPMS, LGS, MTR, Appraisal Report, Supervision reports 1 According to IFAD s Lending Policies and Criteria, there are four types of lending terms: (i) special loans on highly concessional terms, free of interest but bearing a service charge of three fourths of one per cent (0.75%) per annum and having a maturity period of 40 years, including a grace period of 10 years; (ii) loans on hardened terms bearing a service charge of three fourths of one per cent (0.75%) per annum and having a maturity period of 20 years, including a grace period of 10 years; (iii) loans on intermediate terms with a rate of interest per annum equivalent to 50% of the variable reference interest rate, and a maturity period of 20 years, including a grace period of 5 years; (iv) loans on ordinary terms with a rate of interest per annum equivalent to one hundred per cent (100%) of the variable reference interest rate, and a maturity period of 15 to eighteen 18 years, including a grace period of 3 years. 1

2 B. Project Outline 1. The Grenada Rural Enterprise Project (GREP) is the second project, out of three projects financed by IFAD in Grenada since It targets rural poor communities with annual income below the poverty line; unemployed young men and women; and marginal farmers and fishing households facing constraints to developing their potential. It covers the Parishes of St Patrick, St Andrew, St Mark, St John and St David and the islands of Carriacou and Petite Martinique. It aimed at reducing rural poverty in a sustainable manner in indigent households and poor households, improving the livelihoods of approximately individuals. 2. The causes of poverty in the country are complex and related to historical and economic factors, not least the vulnerability of the economy due to the country s small size and its exposure to natural disaster, i.e. tropical storms and hurricanes. At the rural household level, poverty is determined by a lack of: (i) access to productive resources including credit; (ii) social and/or productive organizations; (iii) literacy, technical and/or entrepreneurial skills; (iv) access to markets and/or information; and (v) access to technical and financial support services. 3. Rationale of the intervention. The projects intended to address rural poverty by promoting income earning activities and diversifying the risk by having a range of food and income sources. GREP was demand driven and promoted participatory approaches, supporting Community Based Management Organizations (CBMO) through NGOs. The project initial design emphasized enterprise development, with the intention of instilling a business oriented approach in rural communities and rural service providers. 4. The stated project objectives were: i) diversify, improve and sustain income earning activities; ii) strengthen the capacity and build confidence at the level of the rural communities; iii) strengthen rural service providers to meet community needs; iv) promote efficient and environmentally sustainable production and processing activities; and, v) foster sector and market linkages. 5. The project had three main components that would contribute to the overall goal of reducing rural poverty: a) Community Participation and Support (CPS), to mobilize and strengthen the capacity of the target communities to identify their needs and prepare Action Plans to address the identified needs; b) Enterprise Development (ED), to promote and facilitate viable business enterprises arising from the Community Action Plans (CAPs); and c) Rural Financial Services (RFS), to provide the financial resources necessary for the financing of business enterprises promoted under the previous component. A Project Coordination component would support the activities of all of these. 6. Major changes in context. Grenada had political stability throughout the implementation of GREP. However, it was affected by several shocks: the terrorist attack in the US in 2001 negatively affected Grenada s tourist driven economy 2 ; Hurricane Ivan in 2004 and Hurricane Emily in 2005 caused severe damages to physical infrastructure and to the agricultural sector; and the escalation on prices of energy and essential foods in late 2007 and early 2008 threatened food security. As a result, the poverty levels have risen in most of the parishes in the last decade Implementation. GREP has had two distinct phases in its implementation: the early period was characterized by a slow pace and low management capacity within the PCT. In this period the project lacked a clear strategy on how to promote enterprise development (see relevance section) and suffered from high staff turn over rates in the Project Coordination Team (PCT) and changes in the Project Coordinating Committee (PCC). In the second phase that started after the hurricanes (see context below), there were changes in GREP s strategies and approaches; i) the phased process that linked the three components did not work as expected and GREP focussed primarily on community development; ii) as from early 2007, the implementation strategy for community mobilization changed from delivering services to CBMOs through CIAs to a direct implementation approach, where GREP 2 3 2

3 hired and trained Community Officers (COs) for deployment in the targeted communities The management team was also strengthened improving the capacity of GREP to respond to the needs of the target group. 8. The project components and cost allocations are seen in the table below. Components Estimated Costs Actual USD 000 % USD 000 % 3, , , , , Community Participation and Support Rural Financial Services Enterprise Development Project Coordination Physical and Price contingency Interest during implementation Total 8, , Source: (i) CDB Appraisal Report; (ii) GREP 9. The project was implemented by the Ministry of Finance (MOF). The project established a coordinating Committee composed of representatives from the MOF, Ministry of Agriculture, NGOs, and beneficiaries- and the PCT. The Caribbean Development Bank, which co-financed GREP, was the CI. Project Performance C.1 Relevance C. Main Assessment Review of Findings by Criterion The project original design was relevant to the needs of the poor, addressing their main constraints i.e. poverty, unemployment, low productivity and limited number of crops and business types. Moreover, the project responds to main issues raised by the Country Poverty Assessment carried out in 2008 in the country which claims that transition of the economy to a diversified base as the only basis for sustained poverty reduction... for many of those trapped in structural poverty. Grenada is a vulnerable small island state where high proportion of the population live in poverty and cut off from the mainstream economy, with limited capacity to find new ways of earning income. 11. Despite the above, the project did not propose a clear and realistic strategy on how to promote enterprise development. The specific type of approaches and support needed to transform subsistence farmers and small businesspersons into successful micro-entrepreneurs was not clearly described at the design stage. It assumed that institutions chosen to implement the project (MOA, Grenada Marketing and National Importing Board (GMNIB) and selected NGOs) would overcome their deficiencies and adopt participatory methodologies and a business approach. The design did not recognize the weakness of the institutions which had limited capacity as well as neither the experience or the expertise required. This led to flaws in contracts, poor quality of the business plans and low level of innovation and technological development in the majority of economic activities proposed. The lack of a sound mechanism, and strategy within the project design to develop projects beyond the idea stage, both at beneficiary and agency levels has been the single most important bottleneck to the efficient implementation of the enterprise component of the project. 12. The revised strategy after the MTR in 2008 focused on a single area, community mobilization, as a result of difficulties encountered promoting enterprise development (see above) and also responding to the needs of the target population after the significant damage caused by the hurricanes. The new strategy (without falling into a post-disaster relief operation) concentrated on revitalizing 4 For definition of and guidance on the criteria, please refer to the Evaluation Manual: 3

4 organizations while contributing to improve community infrastructure. While this strategy contributed to results in human and social capital, also resulted in a limited scope in term of income generating activities intended to raise family income and alleviate rural poverty. IOE rating of the project relevance is moderately satisfactory (4). C.2 Effectiveness 13. The shift in project focus to community development combined with the limited scope of the enterprise development intervention means that results achieved by the project relate mostly to the strengthening of the capacity of the rural communities, even though targets were not fully met (while GREP was given a goal at Appraisal of reaching 41 poor communities, the MTR identified 29 active CBMOs in communities throughout the project area). According to the PCR, the projects may have contributed to prepare the population to better exploit job and income earning opportunities, but not contribute and expand such opportunities. 14. The other objectives in terms of income generation and diversification, promotion of production and fostering of market linkages made very limited progress. The MTR did not find a single business plan that was properly prepared and could be accepted for funding. This is not withstanding that a large number of project ideas had been generated by the target beneficiaries. As far as rural financial services (RFS), this component was not implemented in pract ical terms as projects actions were limited to engaging a preliminary way in identifying and negotiating with a possible Loan manager and two credit unions that could undertake the role of financial intermediaries. RFS component started very late in the project implementation influenced also by the very slow start of the business enterprise component. Overall project effectiveness was affected by the initial slow pace which has resulted in several of works not yet completed or lack some minor type of intervention to become fully operational. 15. The assessment of the project effectiveness is challenged by the limited availability of specific data or information in relation to the achievement of project objectives in the PCR as well as other documentation reviewed such as the MTR and project supervision reports. The PCR only provides information on indicators at the output level, and does not compare it to targets. 16. Project disbursement reached approximately 67% of resources committed to GREP, as a result to slow pace of implementation. During the initial four years of implementation, only approximately 33% of the amounts disbursed from the CDB and IFAD resources were used. This was somehow compensated in the last two years when over 50% of the total disbursements were made. 17. Implementation was negatively affected by weaknesses in the Project Coordination team, along with limited capacities in the implementing institutions. Delays are also associated to a challenging context related to overall destruction caused by the hurricanes and related consequences such as reducing employment opportunities in rural areas. 18. The redirection of project focus in 2008 prevented the completion of the three-stage process envisaged at design that would lead to a sustainable increase in production and income levels. While the empowerment of communities paves the way to future income generating activities, project delays did not allow enough time for this to happen. 19. In consideration of the above, while also taking in to account the challenging environment that surrounded implementation which influenced the shift in project focus, IOE rating for effectiveness is moderately unsatisfactory (3). C.3 Efficiency 20. The overall average actual cost per beneficiary of the project as a whole was USD 591, which falls within common ranges for similar interventions. Average cost per beneficiary for social projects (USD 187) was in the same range as similar interventions covering a large number of beneficiaries in 4

5 the region. On the other hand for economic projects such as e.g. irrigation, livestock development, cold storage facility, the average cost per beneficiary (USD 1 815) is higher than comparable interventions. 21. On the other hand a number of issues compromise overall project efficiency. First, even though the documentation reviewed does not provide information on a revised financial/economic analysis, the limited results of the project in terms of enterprise development -and consequently limited results compared to expected increases in production an income- suggest a quite possible deterioration in the in overall cost effectiveness as most results achieved by the project where intangible. Second the percentage of project coordination costs (23.4%) are higher than the estimated at design (20%), which might be explained by the sunk costs of the initial stage with low management capacity in the PCT causing slow initial pace. Third, while the time between Board approval and effectiveness falls within the average in the region (18 months), there were significant delays (three years) between the project identification and the delivery of works and items of equipment experienced by most groups also suggest that GREP was not efficient, particularly in the years prior to On the other hand over the last two years the delivery rates for contracts, disbursements to contractors and hand-over of projects improved significantly. 22. Based on the above analysis and despite recent improvement IOE rates efficiency as unsatisfactory (2). Rural Poverty Impact C.4 Impact Household Income and Net Assets 23. As the project ended-up focusing on strengthening community organizations (vs. income generating activities), the project s impact on income as a whole was limited and also restricted to a relatively smaller number of the beneficiaries of few pilot initiatives to support income generating activities (the total number of beneficiaries of economic projects was 460, compared to beneficiaries of social projects). GREP supported at least four income earning initiatives: the Belmont sea moss intensive production system, the Waltham cold storage facility, the Conference herbal garden and the irrigation projects in La Poterie and La Fortune/Levera. The documentation reviewed does not provide data on increased incomes associated to these projects. The PCR only provides anecdotal evidence of improved incomes levels and employment in two irrigation projects. There is no quantitative data either to assess the contribution of skills training to poverty reduction. One the other hand the PCR claims that each one of the income-earning initiatives constitute a valuable model for increasing assets of rural poor and supporting enterprise development. For example, the sea moss project appears to be an example of practical/hands-on type training and study tour leading to what appears to be one of the better small enterprise development type projects. 24. The negative effect of the hurricanes and the poor results at national level of diversifying agriculture and increasing productivity must also be taken into account as they posed a challenging environment for rural enterprises. Overall, the overall impact of the project on income appears very limited. Rating: Unsatisfactory (2). Human and Social Capital and Empowerment 25. There is no doubt that the most significant impact of the project relates to human and social capital and empowerment. The strengthening of Community Based Management Organizations (CBMO) improved access to social services, resulted in better identification of needs and laid the foundation for further development interventions. 5

6 26. The project also contributed to improve people s skills in productive activities such as cake baking, hotel management, hair braiding, through training. The PCR provides only anecdotal evidence (no quantitative data) of beneficiaries being able to find jobs related to the training received or become self-employed. 27. GREP support to the poorest communities (4 communities supported by GREP were among the 10 poorest in Grenada) resulted in marked improvements in these communities such as access to learning opportunities, to health care, to recreation and sports and led to improved bonds among community members. Rating: Moderately satisfactory (4). Food Security and Agricultural Productivity 28. Reports do not provide any data on changes in food security or agricultural productivity as a result of the project. Documentation reviewed only provide few anecdotal evidence of increases in productivity, for example a cold storage facility in the Waltham has reduced costs and increased quality of produce because they no longer have to go to the town of Victoria to freeze their catch, fish can be stored in a fresher state, improving at the same time productivity and profit margin. As a whole impact is quite limited. Rating: Unsatisfactory (2). Natural Resources and Environment (including climate change issues) 29. Attention to the environment was indirectly addressed by one of the project objectives: promote efficient and environmentally sustainable production and processing activities. Even though there was no environmental screening in the design (at the time it was not mandatory) the PCR reports minimal disturbance on the environment through infrastructure works such as irrigation facilities as they were mainly renovations of existing facilities. Farm projects supposedly used chemicals minimally (due to limited money or environmental awareness), even though limited evidence is provided.. Rating: Moderately satisfactory (4). Institutions and Policies 30. Strengthening co-implementing agencies (CIAs) (e.g. the Grenada Community Development Agency, the Agency for Rural Transformation, the Grenada Citizen Advice and Small Business), and other rural service providers to act in partnership with communities was a goal of the project. These organisations were expected to be instrumental in mobilizing communities and the project foresaw a strong effort in strengthening their capacity. Impact in this domain is however limited as contracts signed between MOF and CIAs were flawed (provided little basis for accountability and demanding better performance) which eventually led to the replacement of CIAs by Community Officers directly hired by the project in implementing community projects. Also some of the institutions that were supposed to have a key role in enterprise development, such as the MOA and the GMNIB, were not given clear roles and responsibilities and did not benefit from institutional strengthening. On the other hand, the contribution of the project to build capacity in the government to implement rural development projects through training and experience gained by the Community Officers needs to be acknowledged. Rating: Unsatisfactory (2). Other Performance Criteria C.5 Sustainability 31. A strong sense of ownership by the communities ensured by the use of participatory approaches in community mobilization represents an important supporting factor for sustainability of project results. However, most groups still need additional support and guidance to consolidate as functioning organizations capable of planning and implementing community projects. The CDB and the GOG financed a bridge project for the period between the completion of GREP and the new IFAD-funded 6

7 project (MAREP) in order to address this issue. The bridge project contributed to maintaining some of the staff as well as to maintaining community interest and involvement 32. As far as social infrastructure, the project made some attempts to ensure proper staffing as well as financial and minimal maintenance expense with the relevant line Ministry. However, the sustainability of social infrastructure is directly linked to the capacity of CBMOs to continue in operation. The PCR reports a limited number of business oriented projects as sustainable, but also significant risks to sustainability of income generating projects related to marketing issues, diversification, technology improvement and business orientation. Rating: Moderately unsatisfactory (3). C.6 Pro-Poor Innovation, Replication and Scaling-Up 33. The main innovation introduced by GERP was its participatory nature, giving project clients control over investment planning, implementation and monitoring. This goes a step further than other projects in the region and it was not standard practice in the institutions that were responsible for providing services to small poor farmers. 34. The institutional arrangements proposed by the project are also innovative but proved too complex to implement. The project initially involved NGOs and other service providers in the various support activities, giving these Co-Implementing Agencies (CIAs) a key role in the training and strengthening process of the Community Organizations. Other institutions, such as the Grenada Marketing and National Importing Board (GMNIB) would be involved in enterprise development. The expected collaboration of several institutions with different cultures (MOA, GMNIB, NGOs) did not take sufficiently into consideration limited experience in this type of interventions as well as existing weak implementation capacity. 35. Some sub-projects, e.g. the sea moss initiative have also introduced innovative approaches in organizing the delivery of technical assistance, promoting participatory approached that fosters empowerment and entrepreneurship. In particular the sea moss sub-project organized knowledge exchanges between beneficiaries in various project locations, having the opportunity to see business at work and identify both challenges and opportunities. 36. GREP participatory approach for community development has the potential to be replicated in other rural development projects in the country. The approaches for business development require further refinement in terms of identification of appropriate institutions to provide required services, introduction of microenterprise income-generating activities in Community Action Plans and identification of funding sources. Delays in effective implementation prevented scaling up during project life. However, the project has been used as the platform for a new IFAD project 5 approved in December 2010 which has managed to mobilize additional contribution from the government as well as co-financing from CDB. 37. At the end of the project IFAD carried out and facilitated with GREP staff and CBOs a systematization workshop. A total of 4 systematization studies were produced which: i) analyse and reflect on successful community participation experiences; and ii) document working methods that could facilitate the implementation of the new project (MAREP), as well as to be used at the wider Caribbean level as inputs for implementation. Overall the project has introduced noteworthy innovations in relation to its participatory methodologies and managed to ensure its replication and scaling up by the GOG and the CDB. On the other hand some of the innovations had limited success during implementation and require further refinement. IOE rates the performance for this criterion as moderately satisfactory (4), one point higher than PMD rating, which does not acknowledge achievements under replication and scaling up, nor the systematization efforts undertaken at the end. 5 Market Access and Enterprise Development Project (MAREP) 7

8 C.7 Gender Equality and Women s Empowerment 38. Gender issues were taken into consideration by the project and some results have been achieved, particularly in terms of the effort made to benefit women and include them in all project activities. More than half (5 030) of the total direct beneficiaries 9736 were women. Moreover, in almost every category more than 50% of the beneficiaries were women. For example skills training and capacity building courses were provided to women and 448 men. Women also exceeded men in the membership of the CBMOs and occupied seats in the steering bodies. 39. It is important to note that all of the Community officers hired by the project were women. This pilot experience - having female staff at the field level- became a practical example of women s involvement in non-traditional roles to the target population. The project has included a Gender Specialist, but only for the first two years of implementation, which unfortunately coincided with the initial stage of slow implementation. 40. Despite the emphasis on gender described above, the support provided and skills acquired resulted mostly in subsistence traditional activities with limited growth potential. Rating: moderately satisfactory (4). C.8 Performance of Partners IFAD s Performance 41. In the first place, IFAD failed to produce a solid project design that could serve as basis for successful implementation. The original design contained important flaws (see relevance section), including a lack of a clear strategy and was overambitious in terms of the estimated capacity of partners. Moreover, in the initial period, IFAD failed to take the necessary action to address the challenges being faced by the project, such as e.g. staffing levels, withdrawal rates, etc. 42. The initial period of indecision or inaction was markedly reversed after the MTR. IFAD provided technical assistance to assess project implementation (including M&E system) and to assist in capacity building. It cooperated closely with the CDB and the GOG and approved GOG request to extend the project completion. Unfortunately IFAD reaction was delayed for too long which limited its positive contribution. Rating: Moderately unsatisfactory (3). Government s Performance 43. Overall, the GOG has demonstrated a good level of ownership and commitment towards the project. The Ministry of Finance (implementing agency) played an important role in facilitating disbursements and collaborated with IFAD for extending the project closing date. It has also provided good support to missions by IFAD and the Cooperating Institutions. The MOA provide good technical services to specific sub projects, upon request The GOG initiatives approving programmes and policies in support to agricultural enterprises in Grenada must also be commended. 44. The Project Coordination Committee (PCC) has been well represented (including public sector, non-governmental organizations and beneficiary communities) and has met frequently (with exception of a period of inactivity in ). On the other hand decisions were sometimes focused on operational and administrative matters which led to sporadic conflicts in the roles of PCC and Project Coordination Committee (PCT) overall the relation between PXCC and PCT have been good. 45. On the other hand the PCT has suffered from high level of rotation, including the project coordinator, which was held by 5 different persons over the project implementation period. At no time in project implementation was the PCT constituted in the manner contemplated at appraisal. The project operation manual was not approved. Also the project lacked a management information system (designed but not implemented). Financial information was the data collected and maintained. Overall IOE rates the performance of the Government as moderately satisfactory (4). 8

9 Cooperating Institution 46. Project supervision was entrusted to the Caribbean Development bank (CBD). Its double role as cooperating institution and co-financer placed it a favourable position to do its job. Unfortunately, as in the case of IFAD CDB did not react adequately in addressing the challenges faced by GREP and failed to use resources committed for supplemental supervision of GREP to assist in addressing the challenges. After the MTR CDB engaged consultancy services in mid-2008 to substantially increase project supervision effort with active support from IFAD. Despite this marked improvement at the end, overall the supervision effort provided was insufficient in terms of time spent in the field and the level of technical support provided.. Rating: Moderately unsatisfactory (3). NGOs 47. Project designed identified NGOs present which had been serving the target communities as service providers. They were named co-implementing agencies (CIAs). NGOs performance has been challenged by a number of issues. In the first years of implementation NGOs operational relations with the PCT suffered due to high rotation in the PCT. There were also conflicts with the PCT in relation to failures to access capacity building from the project, ineffective oversight, and a perceived notion of disrespect for the collaboration with NGOs. Moreover, performance was affected by weak capacity, limited experience and political antagonism. C.9 Overall Assessment of Project Achievement 48. Following IFAD s initial investment on Artisanal Fisheries (closed in 1989) the Grenada Rural Enterprise Project (GREP) intended to offer rural households the chance to enhance their income by helping them recognize and realize economic opportunities. The project introduced a new participatory methodology and a new approach promoting business in order to generate alternative income sources. 49. The project original design was relevant to the needs of the poor, addressing their main constraints i.e. poverty, unemployment, low productivity and limited capacity to find new ways of earning income. However, the project did not propose a clear and realistic strategy on how to promote enterprise development and its design did not recognize the weakness of the institutions chosen to implement the project. The revised project strategy after the MTR in 2008 focused on a single area, community mobilization, and prevented the completion of the three-stage process envisaged at design (community development; enterprise development; access to rural finance from intermediaries) that would lead to a sustainable increase in production and income levels. 50. The most significant impact of the project relates to human and social capital and empowerment. The strengthening of Community Based Management Organizations (CBMO) improved access to social services, resulted in better identification of needs and laid the foundation for further development interventions. The project also contributed to improve people s skills in productive activities through training. On the other hand, the other objectives in terms of income generation and diversification, promotion of production and fostering of market linkages made very limited progress. The project s potential impact on income was restricted to the beneficiaries of few pilot initiatives to support income generating activities. Gender issues were taken into consideration by the project and some results have been achieved. However the support provided and skills acquired resulted mostly in subsistence traditional activities with limited growth potential. 51. Overall the project missed an opportunity to make more progress beyond community development. The challenging environment (hurricanes Ivan and Emily in 2004 and 2005) surely posed a significant challenge to implementation, but the project s shortcomings are found beyond this issue. A new project recently approved in Grenada Market Access and Rural Development Programme already incorporate important lessons learned from GREP. D. Assessment of the PCR Quality 9

10 (i) Scope 52. The PCR followed the annotated outline for PCR as indicated in the Guidelines of Project Completion (2006), and it covered the key aspects o f the programme design, implementation, management, and key criteria of performance and impact. In general, it provided an account of the implementation and achievements in relation to community development, financial services and enterprise development. On the other hand the assessment of effectiveness is not well organised around the extent to which agreed project objectives were achieved and does not make comparison with targets. The impact assessment is weak and does not report separately on various impact domains. The section of Performance of Partners is overall well prepared and provides useful information on government, IFAD, cooperating institution and collaborating NGOs. The scope of the PCR is rated as moderately satisfactory (4) by the validation, which is below the PMD rating of satisfactory (5) given the weak analysis of effectiveness and impact. (ii) Quality (methods, data, participatory process) 53. The assessment method is overall appropriate and consistent with the methodology used by evaluation. The organization of a stakeholder workshop as part of the PCR mission ensured a good participatory process. Generally, the report addresses concisely the main issues under the evaluation criteria, i.e. performance, impact, sustainability, etc. On the other hand the report acknowledges important constraints related to the incompleteness of the data and information provided by the M&E system and the fact that it was not geared to measure higher level objectives beyond outputs. The analysis of performance and impact is mainly qualitative and does not provide any quantitative evidence. In view of poor M&E data the PCR is based mainly on data gathered through field visits, interviews with the GREP staff and the stakeholders workshop. 54. IOE rates the quality of the report as moderately unsatisfactory ( 3) in view of the limited availability of quantitative evidence to support the assessment. (iii) Lessons 55. The PCR has made a good effort to extract lessons from the project. Lessons identified emanate directly form the project experience and appear relevant to future interventions in Grenada and IFAD as a whole. However, in view of the many challenges faced by this project, the report should have tried to analyse in more depth the key issues and underlying causes of underperformance. The lessons section would have also benefited from a clearer distinction between lessons directly related to future implementation and others of a more strategic nature. Rating: moderately satisfactory (4). (iv) Candour 56. The report is fair and impartial. It provides a balanced account of the strengths and achievements of he project as well of its shortcomings. The analysis provided appears transparent and frank. Rating: Satisfactory (5). 57. Based on the above analysis, the overall rating of the PCR quality is satisfactory (5). This rating is not linked to the assessments of the project itself. E.1 Lessons Learned E. Final Remarks 58. Innovation requires enhanced attention to capacity in project design. The introduction of a new participatory methodology, new implementation arrangements and a new approach promoting business in order to generate alternative income without adequate assessment of capacity resulted in significant implementation challenges. Future project designs need to be more simplified and adapted to the management capacity and experience of the local staff, avoiding weak assumptions on 10

11 methodological changes or adjustments that NGOs or public institutions will make to adapt to the specific needs or strategies of the project. Project design also needs to provide clear operational strategies on how to address each area and ensure flexibility to devise appropriate strategy changes during implementation. Supervision needs to be more intense and proactive to address the challenges along implementation of innovative projects. 59. Attention to sustainability from the start. This is a recurrent issue in IFAD projects and requires continued attention. Exit strategies to ensure sustainability of intervention need to be devised since the beginning of the project. In GREP community officers had an important role in supporting communities but future projects need to consider an exit strategy, including maintaining the works and equipment as well as partnerships with other institutions. Roles and responsibilities in the management and operation of infrastructure need to be clarified before project closes. Moreover, project implementation represents an opportunity to involve public and private sector institutions and could become a means to contribute to sustainability. E.2 Issues for IOE Follow-up (if any) 60. There are no issues to follow up by IOE. F. Rating Comparisons Project ratings Criterion PMD Rating 6 IOE Rating Net Rating Disconnect (IOE PCRV - PMD) Relevance Effectiveness Efficiency Project Performance 7-3 Rural Poverty Impact (a) HH Income and Net Assets (b) Human and Social Capital Empowerment (c) Food Security and Agricultural Productivity (d) Natural Resources, and NA 4 - Environment (including climate change issues) (e) Institutions and Policies Overall rural poverty impact Sustainability Pro-poor Innovation, Replication and Scaling Up Gender equality and women s empowerment Overall project achievement Performance of partners 6 Rating scale: 1 = highly unsatisfactory; 2 = unsatisfactory; 3 = moderately unsatisfactory; 4 = moderately satisfactory; 5 = satisfactory; 6 = highly satisfactory; n.p. = not provided; n.a. = not applicable. 7 Arithmetic average of ratings for relevance, effectiveness and efficiency 8 This is not an average of ratings of individual impact domains. 9 This is not an average of ratings of individual criteria. Moreover the rating for performance of partners is not a component of overall performance rating" 11

12 (a) IFAD (b) Government (c) Cooperating Institution Average Net Disconnect 0.08 Ratings of the PCR PMD rating IOE PCRV Net disconnect document quality rating (a) Scope (b) Quality (methods, data, participatory process) (c) Lessons (d) Candour Overall rating PCR document

13 G. List of Sources Used for PCR Validation IFAD Strategy and Policy Documents: IFAD (2007), IFAD Strategic Framework Project Documents: IFAD (2009), Project Completion Report IFAD (2001), Report and Recommendation of the President to the Executive Board IFAD (2001), Loan Agreement IFAD (2008), Mid-Term Review IFAD (2009, March, June, August)), Supervision Reports IFAD (2008), Supervision Report UNOPS (2007, Sept, Dec), Supervision Reports UNOPS (2006, May, Dec), Supervision Reports UNOPS (2005), Supervision Report UNOPS (2004), Supervision Report UNOPS (2003), Supervision Report 13