Evidence Based Agri-environmental Policies Can Institutionalized Evaluation Procedures Provide Useful Input? The Swedish Experiences

Transcription

1 Evidence Based Agri-environmental Policies Can Institutionalized Evaluation Procedures Provide Useful Input? The Swedish Experiences Sören Höjgård and Ewa Rabinowicz Department of Economics, Swedish University for Agricultural Sciences and AgriFood Economics Centre

2 Outline Introduction Objectives of the paper RDP/Agri-environmental schemes and the CAP Overview of agri-environmental schemes in Sweden The evaluation process Methodological problems Reflections on the evaluation of the Swedish programme Major problems with the evaluation procedure Suggested improvements

3 Introduction Considerable amount of money is spent in Europe on agri-environmental schemes: 44 per cent of the RDP spending + domestic funds Extensive debate on efficiency and value for money for European taxpayers The schemes are subject to an institutionalised evaluation procedure Evaluations have potential to provide broad coverage and contribute to the debate and to inform policy formation But are the evaluation procedures up to the task? - most of the evaluation procedures simply examine uptake patterns.. - common sense methodology widely used

4 Objectives of the paper Rather than providing an evaluation of the past evaluations, the paper focuses on the common evaluation procedure as such as it is presently designed and examines prospects or precondi tion for this procedure to generate science-based analysis. The paper builds on ex periences from work to evaluate the Swedish RDP where Axis 2 (agrienvironmental schemes) accounts for almost 70 per cent of the spending

5 Agri-environmetal schemes Make up Axis 2 of the Pillar 2 of the CAP Menu of optional measures, not less than 25% of total RDP spending Axis 2 aims at improving the environment and countryside by means of support to land man ag ement The most important measures are agri-environmental payments The payments are intended to com pensate for the addi tional costs, or income foregone and are usually paid on a per hectare basis Subject to compulsory institutionalised evaluation process

6 Budgetary allocation per axis for the Swedish programme period , M euro and corresponding shares in the EU Axis Swedish budget M euro Percent of Swedish budget Percent of budget in EU Axis 1 to improve the competitive ness of the agri cul ture and forestry sector Axis 2 to improve the environment and the country side Axis 3 to improve the quality of life in rural areas and encourage the diver sification of economic activ ities Axis 4: LEADER Technical assistance Total

7 Overview of the Axis 2 in Sweden Budgetary allocation to measures within Axis 2 Measure Total budget (m euro) Percentage of budget 211, 212 Natural handicap payments to farmers in mountain areas and to farmers in other areas with handicaps 214, 216 Agri-environment payments and support for nonproductive investments Support for non-productive investments forestry 34 1 Total, Axis

8 Overview of the Axis 2 in Sweden (2) Payments within measure 214, Agri-environment payments Payments for Indicative budget share % 1. biodiversity and cultural heritage in seminatural 32.9 grazing lands, mown meadowland and wetlands 2. valuable natural and cultural 5.8 environments in the agricultural landscape and reindeer herding areas 3. regional priorities traditional cultivated plants and livestock 0.4 breeds 5....reduced nutrient leaching from arable 6.1 land 6....environment protection measures organic forms of production extensive ley management for a better environment and an open landscape 29.1

9 Overview of the Axis 2 in Sweden (3) The architecture of Axis 2 is im mensely complicated because payments within measure 214 are divided in many categories and even subcategories. For instance: Payments for biodiversity in semi-natural grazing lands are differentiated according to land use, land type and environmental quality level. Payments for valuable cultural elements in the agricultural landscape are extended to point elements (12 different categories) and line elements (9 differ ent categories). Regional priorities, are subdivided in 22 categories, etc, etc, etc. Coexistence of very many small payments an large schemes Not uncommon that the same piece of land receives several payments with different conditions attached

10 The Evaluation Process Unique feature in the CAP-context Potential to contribute to a more efficient allocation of the EU-budget by benefitting from experiences in all member states Important that results from different member states are comparable Process therefore guided by pre-specified common questions to be addressed using pre-specified common indicators Recommended that indicators are generated by counterfactual analysis

11 Evaluation process: questions and indicators Input, output, result and impact indicators Impact indicators for Axis 2 measures: changes in biodi versity, high nature value areas, nutrient bal ances, and production of renew able energy Example of evaluation questions for measure 214: - To what extent have agri-environmental measures contributed to maintaining or promoting sustainable farming systems? -. contributed to maintaining or improving habitats and biodiversity? -. contributed to maintaining or improving water quality? -. contributed to maintaining or improving soil quality? -. contributed to mitigating climate change? -. contributed to maintaining and improving landscapes and its features? -. contributed to improving the environment? Distinguish between the contribution of agri-environmental measures implemented a demanding sitespecific measures and less demanding measures which are widely applied

12 Methodological problems Difficult to identify control groups and apply econometric tech niques to Axis 2 measures (often all farmers eligible, lack of data). Payments are interacting with each other because the same piece of land usually receives sev eral different pay ments. This implies that is often implausible to evaluate the programme measure by measure and then add the results. Payments interact also with the supports in Pillar 1 (price and the Single Payment Scheme) because of the jointness in production and because of the cross com pliance condi tions attached to the SPS.

13 Modelling approaches as an evaluation tool Agricultural sector modelling constitutes often the only option but several difficulties are involved 1. there are often no off-the-shelf-models available. Ex isting mod els have to be adapted, possible to a very limited degree 2. aggregated agricultural sector models do not lend themselves easily to a precise assessment of many, small heteroge neous schemes. 3. key parameters of many simula tion models, such as elasticities, are often derived from external sources and uncertain but may drive the results 4. agri cultural sector models provide an estimate of the behavioural changes and need to be supplemented with measures of the environmental impacts 5. Combined eco nomic and environmental models are preferable but difficult to achieve on a large (country) scale.

14 Evaluating Swedish agrienvironmental schemes General problems with using models were to a large extent encountered in the evaluation work in Sweden. Following methodology was used: Agricultural sector models (CAPRI and AgriPoliS) were used to analyse the behavioural changes due to the support and some of the environmental impacts. The impact on water quality was analysed using the models SOILNDB and ICECREAMDB. Spatial comparisons between areas receiving support and not re ceiving support was by GIS. In addition, expert judgement was relied upon

15 Reflections on the evaluation of the Swedish programme Result of the evaluation is more of an overview than an in depth assessment. The programme is very complicated and should in principle be evaluated at an appropriate level of complexity. Transfer the benefits or patterns of behaviour between the different schemes questionable The very justification of specialised small schemes is the fact that they are supposed to deliver unique benefits. An evaluation of the programme in all the details was not possible under the time and resource limitations.

16 Major problems with the evaluation procedure (1) Questions too big or too difficult to be answered within time and resource framework of the evaluation Unless suitable methodology and appropriate data are available to the evaluators, reliable answers cannot be produced. The evaluation may not produce much new information beyond what is already known through monitoring and previous scientific research There is a shortage of data for analyses of environmental effects FADN, EUOROSTAT and EUROFARM not sufficient. FADN is entirely based on economic data and less representative for farm population as a whole National registers in some cases contain additional information but often timeconsuming and costly to merge with data on RDP-support due to privacy considerations Questions asking for effects not targeted by indicators Contribution to a viable rural community instead of increase in biodiversity May induce evaluators to construct additional indicators and compromise comparability of results from different MS

17 Problems (2) Overlapping questions and vague concepts Difference between maintaining a viable rural community and maintaining the country side? Difference between promoting sustainable farming systems and improving the environment? Requires interpretation which may compromise comparability of results No questions about cost-effectiveness or cost-benefit ratios Evaluation unable to give information on efficiency of measures targeting nonmarketed goods

18 Suggestions Common evaluation questions should be precise The link between questions and indicators needs to be strengthened Broad, unanswerable questions should be dropped Gaps in basic knowledge suggest a separation of questions into short-term evaluation questions and long-term research issues preferably allocated to separate activities Lack of empirical data calls for a coordinated data generation process in the MS Simulation models need to be further developed Common methodological standards should be prescribed and enforced