Abstract... X. Extrait... XI. 1 Foreword... 1

Transcription

1 Content Abstract... X Extrait... XI 1 Foreword Introduction to the study framework Groups of investment support measures and financial support during Intervention logic of the four groups of investment support Group A: Productive investment support to private beneficiaries to increase economic performance/business competitiveness Group D: Investment support to private beneficiaries for investments required to meet minimum standards under Axis 1 measures 121 and 123 other than those which improve the economic performance of the holding Group B: Investments in public infrastructures which are complementary to private investments intended to improve economic performance/business competitiveness Group C: Non-productive investments to private beneficiaries for environmental or non-market purposes Methodology and tools of the study Evaluation Questions, judgement criteria and indicators Approach to answering Evaluation Question Approach to answering Evaluation Question Approach to answering Evaluation Question Overview of evaluation methods for the assessment of investment support Selected methods tested in the fieldwork phase Input-Output Analysis (IO) Econometric counterfactual design Method for Impact Assessment of Programmes and Projects (MAPP) Programme-theory-based evaluation (TBE) Strategic Environmental Assessment (SEA) Cost-Effectiveness Analysis (CEA) Choice and justification of the selected RDP territories Criteria for the selection of case study territories Data availability for proposed methods in case study regions Combination of evaluation methods tested in case study regions page I

2 4 Spotlight on selected case study findings IO method in case study Austria Econometric counterfactual method in case study Austria MAPP Method in case study UK/Scotland Programme-theory-based method in case study Czech Republic Strategic Environmental Assessment / Cost-Effectiveness Analysis in case study Greece Answers to Evaluation Questions Answers to Evaluation Question Rationale Judgement criteria Answering EQ1 in a best case situation Overview of methods given a "best case situation The best case" reference situation compared with the observed situation Answer to Evaluation Question Rationale CEA and SEA results on EQ IO results on EQ MAPP results on EQ Results of PSM to answer EQ Comparing the efficiency of types of measures TBE results on EQ Contrasting results and complementarity between methods Answer to Evaluation Question Answers to evaluation sub-question 1-3 on achieving an RDP objective Answers to evaluation sub-question 4-6 on allocating funds to target groups exemplified on the case study M121 in AT Conclusions and recommendations Conclusions on the appropriateness of methods to evaluate investment support measures Recommendations on identifying appropriate methods to evaluate investment support measures Rationale and overview Capacities, resources, preferences and characteristics Making a decision on an evaluation method page II

3 6.3 Conclusions on the effectiveness, efficiency and impact of investment support Conclusions on targeting Further observations Bibliography International guidebooks Specific evaluation methodologies for rural development Specific sources for IO-Method Specific sources for Counterfactual design Specific sources for MAPP method Specific sources for Theory-Based Impact Evaluation Specific sources for SEA Specific sources for CEA Glossary of key terms page III

4 Tables and Figures Table 1. Investment groups and measures covered by the study... 4 Table 2. Financial inputs/expenditures of EAFRD for investment measures, Table 3. Sub-Evaluation Questions, judgement criteria and evidence for EQ Table 4. Overview of evaluation approaches Table 5. Pre-Selection of methods for fieldwork phase Table 6. Factsheet for Input-Output Analysis Table 7. Factsheet for an Econometric counterfactual design Table 8. Factsheet for the MAPP Method Table 9. Factsheet on Programme-theory-based evaluation Table 10. Factsheet on Strategic Environmental Assessment (SEA) Table 11. Factsheet on Cost Effectiveness Analysis (CEA) Table 12. Overview of data availability in case study regions Table 13. Combination of methods per case study and overview of targeting analysis Table 14. Capacity-Adjustment Analysis national Managing Authority data: Inputs to the Austrian IO Model (in EUR million) Table 15. Capacity-Adjustment Analysis econometric counterfactual analysis data: Inputs to the Austrian IO Model (in EUR million) Table 16. Impact Analysis, Austria based on national managing authority data (average annual effects compared to 2009) (in EUR million) Table 17. Impact Analysis based on data from the econometric counterfactual analysis, Austria (average annual effects compared to 2009) (in EUR million) Table 18. New Jobs Generated per million EUR of Investment, Austria (FTEs) Table 19. Austria: Effect of M121 on GVA (EUR) Table 20. Austria: Effect of M121 on total farm income (in EUR) Table 21. Austria: Effect of M121 on total farm employment (in AWU) Table 22. Austria: Effect of M121 on farm labour productivity (in EUR/fully employed) Table 23. Influence matrix from North-East Scotland case study Table 24. Conceptual model for the bundle of measures M121, M123, M Table 25. Implementation of planned activities for the bundle of measures M121, M123, M Table 26. Direct gross effects for beneficiaries of implemented operations under the bundle of measures M121, M123, M page IV

5 Table 27. Summary table on the contribution of measures M121, M123, M311 to the intended change Table 28. Completeness of TBE building blocks for M121, M123, M Table 29. M125 projects of the Greek RDP, comparison of CER Table 30. How appropriate are different evaluation methods to analyse efficiency, effectiveness and impact of investment support measures - assuming a best case situation Table 31. Ranking of resources used for the fieldwork of geographic experts (most resources: rank 1) Table 32. Subjective assessment of how close case studies in relation to the "best case" Table 33. Shocks of regional or country IO model (million EUR) Table 34. Efficiency, effectiveness and impact results based on the IO analysis Table 35. Aggregated results of MAPP Table 36. Overview of PSM results on efficiency and impact Table 37. Overview of PSM results on effectiveness Table 38. Aggregated TBE results on the effectiveness of investment support measures Table 39. Results on labour market related indicators of four methods applied in Czech Republic and measure 121 (M121) Table 40. Number of sub-measures analysed to identify approaches to targeting Table 41. Distinct statistically significant characteristics of the clusters Table 42. Correlation between clusters and efficiency criteria from EQ Table 43. Overview of essential capacities necessary to carry out an evaluation Table 44. Tool for decision makers to state his or her preferences Table 45. Characteristics of generic evaluation methods Figure 1. Intervention logic by type of beneficiary for measures 121, 122, Figure 2. Intervention logic by type of beneficiary for measures 311, 312, Figure 3. Intervention logic by type of beneficiary for measures 121, Figure 4. Intervention logic by type of beneficiary for measure Figure 5. Intervention logic by type of beneficiary for measures 216 and Figure 6. Workflow of Input-Output Method Figure 7. Quasi-experimental design with treatment and control groups Figure 8. Workflow of PSM Figure 9. Workflow of MAPP Method page V

6 Figure 10. Workflow of Programme-Theory-Based Impact Evaluation Figure 11. Workflow of Strategic Environmental Assessment Figure 12. Workflow of Cost-Effectiveness Analysis Figure 13. Comparison of IO results on efficiency (jobs created per million expenditure) for investment support measures on efficiency across regions Figure 14. Efficiency of investment support measures based on PSM estimates Figure 15. Efficiency indicator on employment effects based on PSM estimates Figure 16. Effectiveness indicator based on PSM estimates Figure 17. Impact indicator based on PSM estimates on a log-scale Figure 18. Efficiency for job creation from IO grouped by measure types Figure 19. Efficiency of GVA estimated with PSM indicator 1a Figure 20. Comparison of IO results on efficiency (jobs created per million expenditure) for investment support measures across regions with and without parameters obtained from PSM Figure 21. Dendrogram of the clusters generated from the approaches to targeting. 132 Figure 22. Decision tree page VI

7 List of abbreviations AEM AIR APR ATT AWU BGS CAP CAPRI CBA CEA CEQ CER CGE CIE CMEF DG AGRI DG EMPL DG REGIO DiD EAA EAFRD EIA ERDF ESF EDM ENRD EQ EU EUROSTAT FADN FBI FFI Agro-Environmental Measures Annual Implementation Report Annual Progress Report Average Treatment on Treated Annual work unit Biogas Common Agricultural Policy Common Agricultural Policy Regionalised Impact Cost-benefit analysis Cost-effectiveness analysis Common Evaluation Question Cost-effectiveness ratio Computable general equilibrium Counterfactual impact evaluation Common Monitoring and Evaluation Framework Directorate-General for Agriculture and Rural Development Directorate-General for Employment, Social Affairs and Equal Opportunities Directorate-General for Regional and Urban Policy Difference-in-differences method Economic Accounts of Agriculture European Agricultural Fund for Rural Development Environmental Impact Assessment European Regional Development Fund European Social Fund Equilibrium displacement model Evaluation Network for Rural Development Evaluation Question European Union Statistical Office of the European Union Farm Accountancy Data Network Farm birdland index Family Farm Income page VII

8 FFV FTE GDP GE GIS GPS GRIT GVA Ha HNV IO IV M NVZ LCA LFA MAPP MTE MS PA PASMA PRA PSM P-TBE RCT RD RDD RDP SEA SME SWOT TBE Fresh fruit and vegetables Full time equivalent Gross domestic product Geographic Expert Geographic information system Generalised propensity score matching Generation of Regional Input-Output Tables Gross value added Hectare High nature value Input-Output Instrument variable Measure Nitrate Vulnerable Zone Life-cycle analyses Less favoured areas Method for Impact Assessment of Programmes and Projects Mid-term evaluation Member State Paying Agency Positive Agricultural Sector Model Austria Participatory Rural Appraisal Propensity score matching Programme-theory-based impact evaluation Randomised controlled trial Rural Development Regression discontinuity design Rural Development Programme Strategic Environmental Assessment Small and medium-sized enterprises Strengths, weaknesses, opportunities and threats Theory-based evaluation page VIII

9 ToR UAA WIFO WWC Terms of Reference Utilised Agricultural Area Austrian Institute of Economic Research What Works Clearinghouse page IX