Environmental impacts of farm scenarios according to five assessment methods

Transcription

1 Agriculture, Ecosystems and Environment 118 (2007) Environmental impacts of farm scenarios according to five assessment methods Hayo M.G. van der Werf a, *, John Tzilivakis b, Kathy Lewis b, Claudine Basset-Mens a a INRA, UMR Sol Agronomie Spatialisation de Rennes-Quimper, 65, rue de Saint Brieuc CS 84215, Rennes Cede, France b Agriculture and Environment Research Unit (AERU), Science and Technology Research Institute, University of Hertfordshire, College Lane Campus, Hatfield, Hertfordshire AL10 9AB, United Kingdom Received 6 April 2005; received in revised form 31 May 2006; accepted 6 June 2006 Available online 4 August 2006 Abstract It is not known to what etent the outcome of studies assessing the environmental impacts of agricultural systems depends on the characteristics of the evaluation method used. The study reported here investigated five well-documented evaluation methods (DIALECTE, Ecological Footprint, Environmental Management for Agriculture, FarmSmart, Life Cycle Assessment) by applying them to a case study of three pig farm scenarios. These methods differ with respect to their global objective (evaluation of impact versus evaluation of adherence to good practice), the number and type of environmental issues they consider, the way they define the system to be analysed, the mode of epression of results (for the farm as a whole, per unit area or per unit product) and the type of indicators used (pressure, state or impact indicators). The pig farm scenarios compared were conventional good agricultural practice (GAP), a quality label scenario called red label (RL) and organic agriculture (OA). We used the methods to rank the three scenarios according to their environmental impacts. The relative ranking of the three scenarios varied considerably depending on characteristics of the evaluation method used and on the mode of epression of results. We recommend the use of evaluation methods that epress results both per unit area and per unit product. Environmental evaluation methods should be used with great caution, users should carefully consider which method is most appropriate given their particular needs, taking into consideration the method s characteristics. # 2006 Elsevier B.V. All rights reserved. Keywords: DIALECTE; Ecological footprint; Environmental management for agriculture; FarmSmart; Life cycle assessment; Organic agriculture; Label rouge; Pig production 1. Introduction Avariety of methods have been proposed for the evaluation of the environmental impacts of farms (Von Wirén-Lehr, 2001; Van der Werf and Petit, 2002; Halberg et al., 2005). The development of such methods is essential, as they can serve as decision support tools for guiding the evolution towards more sustainable agricultural production systems (Hansen, 1996). * Corresponding author. Tel.: ; fa: address: Hayo.vanderWerf@rennes.inra.fr (H.M.G. van der Werf). These tools are increasingly used by farmers (Goodlass et al., 2003), researchers (De Koeijer et al., 2002) and political decision makers (Schröder et al., 2004). The authors of studies using such methods to assess the environmental impacts of agricultural systems rarely acknowledge that the results obtained depend not only on the characteristics of the systems compared, but also on those of the evaluation method used. A methodological reflection on the structure of methods for the environmental evaluation of farms seems appropriate. Such methods generally present five major stages (adapted from Petit and van der Werf, 2003): /$ see front matter # 2006 Elsevier B.V. All rights reserved. doi: /j.agee

2 328 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) Definition of the global objective of the method, e.g. The evaluation of environmental impact or the evaluation of adherence to good agricultural practice. This stage involves choices with respect to the intended user, the spatial scale for which the method is designed, and the consideration of economic and social dimensions, in addition to environmental impacts. 2. Definition of environmental objectives. Global objectives cannot be directly assessed or quantified, a set of more specific environmental objectives is required, which is at the heart of the evaluation method (Van der Werf and Petit, 2002). We define the term environmental objective as an environmental issue of concern and its associated desired trend. Other terms used for environmental issues (OECD, 1999; EEA, 2005) are environmental themes (Pointereau et al., 1999) and impact categories (Guinée et al., 2002). Some eamples of environmental objectives: reduction of energy use, reduction of emissions of nitrate or maintenance of soil quality. 3. Definition of the system to be analysed. Many methods are restricted to the evaluation of the direct impacts of a system, by considering only the impacts from operation of the system. Other methods also consider indirect impacts, resulting from the production of the inputs (fertilisers, feeds) to the system. 4. Construction or identification of indicators for each environmental objective. To quantify the degree to which the environmental objectives are attained, a set of indicators serving as evaluation criteria is required. The quality of an indicator will largely depend on the validity of its calculation algorithm. 5. Calculation of results. Indicator values are calculated for each of the production systems or scenarios to be compared. A partial or total aggregation of results may facilitate their interpretation. These stages involves choices, in particular with respect to the global objective of the method (stage 1), its environmental objectives (stage 2), the way in which the system is defined (stage 3), and concerning the indicators, since for each environmental objective one or several indicators are selected from many possible candidates (stage 4). Although the outcome of the assessment will obviously be affected by these choices, they are rarely discussed or justified by those proposing such methods. Methods show great variability with respect to the implementation of these choices. For instance, a review of 12 methods used for the evaluation of environmental impacts at the farm level revealed that the number of environmental objectives considered per method varied from 2 to 13 (VanderWerfandPetit,2002). Of the total of 26 objectives, some were considered in si or seven methods, whereas others were considered in a single method only. Although different methods for environmental evaluation have been compared on the basis of their published descriptions (Von Wirén-Lehr, 2001; Van der Werf and Petit, 2002; Braband et al., 2003; Halberg et al., 2005), we did not find any comparative studies based on the actual application of different methods to a set of farms or farm scenarios. The study reported here investigated five well-documented evaluation methods by applying them to a case study of three farm scenarios. The objectives of the study were to eamine to what etent the five methods produce different results and to investigate which characteristics of the methods affected the outcome. It should also allow us to propose recommendations for the selection of evaluation methods. 2. Materials and methods 2.1. Farm scenarios This study compared three contrasting scenarios of farms producing crops and pigs in Bretagne, western France, showing major differences with respect to crop production practices and input use, animal housing systems and crop and animal production levels (Table 1). Technical performance, input use and emissions to the environment used to construct the scenarios were based mainly on published data, complemented by real farm data supplied by a range of eperts (Basset-Mens and van der Werf, 2005). The good agricultural practice (GAP) scenario corresponds to a current intensive (or conventional ) pig production farm, optimised in particular with respect to fertilisation practices, as specified in the French Agriculture Raisonnée standards (Rosenberg and Gallot, 2002). In the GAP scenario, pigs are raised in a slatted-floor building. The organic agriculture (OA) farm scenario corresponds to organic agriculture according to the French version of the European rules for organic animal production (Ministère de l Agriculture et de la Pêche, 2000) and the European rules for organic crop production (CEE, 1991). The Red Label (RL) farm scenario corresponds to the Porc Fermier Label Rouge quality label (Groupement des Fermiers d Argoat, 2000). In the OA and RL scenarios, pigs are born and raised outdoors until weaning, and in an open-front straw-litter building at low animal density after weaning. An inventory of some of the main emissions for the three farm types is presented in Table 2. Details on crop and feed production practices and on animal production practices can be found in Basset-Mens and van der Werf (2005). For the three scenarios we assumed that farmers adopt good agricultural practice, and respect current regulations. Some of the evaluation methods applied here, in particular EMA, but also FarmSmart, require etensive information with respect to the farm conservation practices (e.g. concerning hedgerows, field margins, and

3 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) Table 1 Characteristics of the good agricultural practice (GAP), red label (RL) and organic agriculture (OA) farm scenarios GAP RL OA Crop production Farm size (ha) Annual crops Pea, winter triticale, winter wheat Grain maize, winter barley, winter triticale Horse bean, grain maize, spring barley, winter oats, winter wheat Production of cereal straw 261 t, sold off farm 67 t, used as animal bedding; an additional 62 t bought off-farm 96, 65 t of which used for animal bedding, no straw sold Perennial crops Ryegrass clover paddock (10 ha) Ryegrass clover paddock (4.4 ha), lucerne (4.8 ha) % weight of pig feed produced on-farm Animal production: piglet production Housing Slatted-floor Outdoor Outdoor Herd size (no. of sows) Weaned piglet/sow/year Weaning age (days) Feed/sow (boars incl.) (kg/year) Animal production: weaning to slaughtering Housing Slatted floor Straw litter Straw litter Surface per pig (m 2 ) Feed consumed (kg) Slaughter weight (kg) Pig live weight produced/ha of farm surface/year watercourses), in this respect the farm scenarios were assumed identical. For the three scenarios, all crops produced were used as components of the concentrate feed for the pigs. For GAP and RL, the crops produced on-farm contributed 40% to the weight of the concentrate feed used, for OA 70%. The three scenarios strongly differ in pig live weight production per ha of farm surface per year (Table 1): 4857 kg for GAP, 3530 kg for RL, and 1480 kg for OA. Input use (additional pig feed, fertiliser, electricity, diesel) per ha was largest for GAP and smallest for OA Evaluation methods In the recent past, farms had a single principal function: production of food and fibre for the market. Nowadays farms have a second main function, which becomes increasingly important: production of non-market goods (e.g. environmental services). In the evaluation of the environmental impacts of farms, both functions should be considered. In this study we epressed farm impacts, when methods allowed, by two functional units: on the one hand per unit area, reflecting the farm s function as a producer of nonmarket goods, and on the other hand per unit product, reflecting its function as a producer of market goods Life cycle assessment (LCA) LCA is a technique to evaluate the environmental burdens associated with a product, process, or activity. In the inventory analysis phase the resources consumed and the emissions to the environment, both on-farm and associated with the production and delivery of the inputs used on the farm, are listed. In the impact assessment phase, resources used and emissions are interpreted in terms of environmental Table 2 Inventory of some of the main emissions according to Basset-Mens and van der Werf (2005), epressed per hectare of land used (both on and off farm) and per kg of pig, for the three farm scenarios Emission Unit (kg) Per hectare of land used Per 1000 kg of pig GAP RL OA GAP RL OA Nitrate NO a Oides of nitrogen NO Ammonia NH Sulphur dioide SO Methane CH Nitrous oide N 2 O Carbon dioide CO Carbon monoide CO a For each substance, lowest value per ha of land used and per kg of pig in bold.

4 330 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) impacts (Guinée et al., 2002), by multiplying the aggregated resources used and the aggregated emissions of each individual substance with a characterisation factor for each impact category to which it may potentially contribute. The results presented here are based on a detailed LCA study of pig production (Basset-Mens and van der Werf, 2005), which epressed results per kg of pig live weight produced and per ha of land used (i.e. including land offfarm, used for the production of crop-based ingredients for concentrate feed) Ecological footprint (EF) The ecological footprint of a designated population is the area of productive land and water ecosystems required to produce the resources that the population consumes and to assimilate the wastes that the population produces, wherever on Earth the land and water is located (Rees, 2000). The area of productive land and water ecosystems available per capita is designed as a fair Earth share, and constitutes a threshold value which can be used as a benchmark in an EF analysis (Wackernagel and Rees, 1996). We calculated the farm EF as the sum of four components. The first component is the land surface of the farm being assessed; the second is the land surface required to produce the ingredients of the concentrate feed which were not produced on-farm. The latter surface was calculated using average yields for (FAO, 2002). The third component, energy land, corresponds to the land required to produce the non-renewable energy used on the farm and for the production and delivery of the farm inputs. This was calculated assuming a net productivity of 80 GJ/ha (Wackernagel and Rees, 1996). The fourth component, carbon-sink land, reflects the land area needed to sequester the CO 2 corresponding to the greenhouse gasses emitted on the farm and for the production and delivery of the farm inputs. Greenhouse gasses resulting from the use of non-renewable energy were not included here, as these were taken care of in the energy land component. CO 2 -equivalents were calculated according to the GWP 100 factors by IPCC (Houghton et al., 1996) inkg CO 2 equiv.: N 2 O: 310, CH 4 : 21. An annual CO 2 sequestration rate of 6.6 t ha 1 was assumed (Wackernagel and Rees, 1996) Environmental management for agriculture (EMA) Lewis and Bardon (1998) proposed EMA, a computerbased informal environmental management system for agriculture. The core of the system is the performance assessment (PA) mode, which compares actual farmer production practices and site-specific details with regulatory compliance and what is perceived to be best practice for that site. EMA has been designed on a modular basis, with each module producing a report and an environmental performance inde, known as an eco-rating, for a specific aspect of farming. Within each module, where appropriate, an estimate of emissions is made, which, when collated, forms an emissions inventory (EI) for the farm being assessed (Lewis et al., 1999). EMA seeks to encourage continuous improvement in environmental performance, tackling issues and problems in small steps, that are practically manageable and financially affordable. Its technical system is a support mode, which incorporates modules to eplore What-If scenarios, to identify site-specific solutions to environmental problems and so improve future eco-ratings. This mode helps the user identify solutions to problems spotted in the performance assessment mode. The second support mode is a hypertet advisory system. In this study the EMA 2004 version was used FarmSmart In 2000 the UK government launched a pilot set of agricultural sustainability indicators to provide a means of measuring the economic, social and environmental impacts of agriculture in Great Britain at national level (Tzilivakis and Lewis, 2004). It was hoped that stakeholders would find the indicators valuable for regional and local use. However, many of the indicators have been defined from the policy top-down perspective, some are not measurable directly on farm, and few have direct links with on-farm management decisions. Consequently, the key messages emerging from the indicators can easily be lost at farm level. In order to address these issues, FarmSmart, a simple tool for farmers, was developed (Tzilivakis and Lewis, 2004). It collates relevant information to identify appropriate indicator values for a specific farm and location and provide a management focus, such that farmers are provided with information to help them select indicators relevant to their situation, assess their performance, and take steps for improvements where required. The method yields 35 indicators referring to Economy, Management, Inputs, Resources and Conservation. In this study results are presented for those indicators showing differences for the farms compared. The FarmSmart Beta version was used DIALECTE Solagro (2000) proposed DIALECTE for the evaluation of the environment at farm level by means of a comprehensive, simple and rapid approach. This method is an improved version of the Solagro Diagnostic proposed by Pointereau et al. (1999). The method yields 16 agro-environmental indicators (AEI) allowing a rapid and global evaluation of the environmental risks of the farm. It further produces a whole farm approach (WFA) consisting of an Energy analysis, of performance levels for Farm diversity and Management of inputs, and of an assessment of the potential impacts of the farm on water, soil, biodiversity and resource use. The method can be applied to all agricultural production systems in France. In this study DIALECTE version 4.0 (January, 2004) was used. No

5 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) results were presented for three of the 16 Agro-Environmental indicators (livestock units/ha of forage crop, irrigated surface, and length of the grazing season), as they were not relevant for the farms compared here. 3. Results 3.1. Characterisation of the methods Global objective of the method LCA, EF, FarmSmart and DIALECTE share the same global objective: the evaluation of environmental impact, whereas EMA s global objective is the assessment of adherence to best practice Environmental objectives Environmental objectives were grouped in four classes (Van der Werf and Petit, 2002): farming practice-related, input-related, emission-related and system state-related. The methods compared differ with respect to the number and type of environmental objectives taken into account (Table 3). LCA, EF and FarmSmart consider input-related and emission-related objectives. EMA considers farming practice-related and emission-related objectives, whereas DIA- LECTE considers farming practice-related, input-related and system state-related objectives. EF is narrow in focus, as only three objectives are considered. EMA (12 objectives) and DIALECTE (19 objectives) are wide ranging, and are the only methods taking into account objectives related to farming practices. FarmSmart and LCA (both si objectives) Table 3 Characterisation of the evaluation methods with respect to their environmental objectives, which were grouped as: farming practice-related, input-related, emission-related, related to the state of the system Environmental objectives Methods a LCA EF EMA FarmSmart DIALECTE PA EI AEI WFA Farming practice related Fertiliser usage \ b Organic manure management \ Odour management \ Pesticide usage and general management \ Pesticide treatment frequency # Overall soil management \ Growing of legume crops and grass \ Crop diversity \ Soil cover by crops in winter " On-farm production of feed " Livestock husbandry \ Livestock diversity \ Energy and water efficiency \ Farmland conservation \ Input related Use of non-renewable energy # Land use # Water use # N fertiliser use # P fertiliser use # N balance (input output) # P balance (input output) # Pesticide use # Emission related Emission of greenhouse gases # Emission of acidifying gases # Emission of eutrophying substances # Emissions concerning terrestrial ecotoicity # System state related Landscape quality Agricultural biodiversity Water quality Soil quality An indicates that the objective is taken into account. a LCA, life cycle assessment; EF, ecological footprint, EMA, environmental management for agriculture; PA, performance assessment; EI, emissions inventory; AEI, agro-environmental indicators; WFA, whole farm approach. b #, objective to be minimised; \, objective to be optimised; ", objective to be maimised.

6 332 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) Table 4 Indicator types for the evaluation methods Indicator type Methods LCA EF EMA FarmSmart DIALECTE PA EI AEI WFA Pressure State Impact For method acronyms see Table 3. are of intermediate scope, with FarmSmart relying mainly on input-related objectives, and LCA more on emissionrelated objectives. The methods compared thus are quite distinct with respect to the number and the type of environmental objectives considered Definition of the system For EMA and DIALECTE, the system evaluated consists of the farm only, whereas for LCA, EF and FarmSmart the system includes the production of inputs Indicators used Indicators serve as criteria to quantify the degree to which environmental objectives are attained. The indicators used by the five methods were categorised according to the Driving forces Pressures State Impact Responses (DPSIR) framework proposed by the European Environmental Agency (EEA, 1999). According to this view, social and economic Driving forces eert Pressure on the environment, as a consequence the State of the environment changes, this leads to Impacts on human health and ecosystems, which may elicit a societal Response. None of the methods use Driving forces or Responses indicators (Table 4). LCA and EF use Impact indicators, EMA and FarmSmart use Pressure indicators, whereas DIALECTE uses both Pressure and State indicators. Thus, the methods compared here differ with respect to the types of indicators used Evaluation of the farm scenarios The results for each method will be presented in this section, and the three scenarios will be ranked. We use ranking here as a tool to condense the etensive and diverse output produced by the different methods in order to be able to compare their results for the three scenarios. This does not imply that we consider ranking of farming systems to be the primary objective of these methods. A ranking of the three scenarios is only uncontroversial in the case where all indicators within a given evaluation method separately rank the three scenarios in the same order. This was, however, never the case. Our ranking was mainly based on the number of best and worst scores the scenarios obtained, details are given below. We implicitly considered thus that all indicators are equally important, which is a subjective choice and will not necessarily be true in the real world. When best and worst scores were more or less evenly distributed among the scenarios, we attributed identical ranks. This procedure obviously is not 100% objective, but it is transparent (since the full information is available in Tables 5 11) and allows an Table 5 The environmental impacts calculated according to Basset-Mens and van der Werf (2005) using life cycle assessment (LCA), epressed per ha of land used and per kg of pig produced for the three farm scenarios Impact category Unit Per hectare of land used Per kg of pig GAP RL OA GAP RL OA Eutrophication kg PO 4 -equiv a Climate change kg CO 2 -equiv Acidification kg SO 2 -equiv Terrestrial toicity kg 1.4-DCB-equiv Non-renewable energy use MJ LHV Land use m 2 year a For each impact category lowest value per ha of land used and per kg of pig in bold. Table 6 The ecological footprint of the three farm scenarios, values in ha year per ha of farm surface and in ha year per 1000 kg of pig live weight Footprint component Per hectare of farm surface Per 1000 kg of pig GAP RL OA GAP RL OA Farm a Additional pig feed Non-renewable energy Carbon sink Total footprint a For each component lowest value per ha and per kg of pig in bold.

7 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) Table 7 The eco-ratings calculated using the environmental management for agriculture (EMA) performance assessment mode for the three farm scenarios Eco-rating Range GAP RL OA Fertiliser usage +100/ a Organic manure +100/ 100 S2 3 3 Odour management 0/ 100 S Crop pesticide usage 0/ S9 n.a. General pesticide management 0/ 100 S19 20 n.a. Overall soil management +100/ Indoor pigs +100/ Outdoor pigs +100/ 100 n.a. 5 5 Energy efficiency +100/ 100 S Water efficiency +100/ Farmland conservation +100/ Average eco-rating a For each eco-rating highest value in bold. overall assessment of the three scenarios by the five methods (Section 3.3, Table 12) Life cycle assessment When LCA results were epressed per ha of land used, the OA scenario did best: it had lowest impact values for eutrophication, climate change and energy use. GAP had highest values for all impacts ecept climate change (Table 5), so we rank scenarios OA > RL > GAP. When results were epressed per kg of pig produced, the picture was very different: OA did worst for all impacts with the eception of acidification, for which GAP had the highest impact. Overall GAP did best, as it had the lowest values for four impacts; RL had the lowest values for two impacts, here we rank scenarios GAP > RL > OA Ecological footprint Epressed per ha of farm surface, OA had the smallest Ecological Footprint, and GAP the largest (Table 6). Footprint components (ecluding farm surface, which by definition was identical across scenarios) were smallest for OA, RL had the largest value for carbon sink land and GAP for the other two components. Overall we rank OA > RL > GAP. Epressed per kg of pig, results were inverted, now GAP had the smallest footprint and OA the largest. GAP had the lowest values for all footprint components ecept land for additional pig feed, where OA had the smallest value. We rank GAP > RL > OA EMA The eco-ratings did not clearly differentiate the scenarios (Table 7). GAP did best for three eco-ratings, both RL and OA did best for two eco-ratings, differences were often minor. The average eco-rating was slightly better for OA than for the other two scenarios. RL and OA had worse eco- Table 8 The emissions inventory calculated using environmental management for agriculture (EMA), epressed per hectare of farm surface and per 1000 kg of pig produced, for the three farm scenarios Emission Unit (kg) Per hectare of farm surface Per 1000 kg of pig GAP RL OA GAP RL OA Minimum potential nitrate leaching NO a Oides of nitrogen NO Ammonia NH Sulphur dioide SO Methane CH Carbon dioide CO Carbon monoide CO a For each emission lowest value per ha and per 1000 kg of pig in bold. Table 9 Values of farm-level indicators, epressed per hectare of farm surface and per 1000 kg of pig produced, calculated using FarmSmart for the three farm scenarios Indicator Unit Per hectare of farm surface Per 1000 kg of pig GAP RL OA GAP RL OA Pesticide active ingredient used kg a Growth regulator active ingredient used kg N fertiliser use N (kg) P fertiliser use P 2 O 5 (kg) Ammonia emission NH 3 (kg) Methane emission CH 4 (kg) Nitrous oide emission N 2 O (kg) Carbon dioide emission CO 2 (kg) Direct energy consumption GJ Indirect energy consumption GJ a For each indicator lowest value per ha and per 1000 kg of pig in bold.

8 334 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) Table 10 Values of agro-environmental indicators calculated using DIALECTE for the three farm scenarios Indicator Unit Unfavourable a Favourable GAP RL OA Grass > 2 year % b 17 N from manure on surfaces receiving manure kg/ha N from manure/total N % Surface receiving manure % Length of hedges and woodland borders m/ha Direct energy use in diesel litre equivalents l/ha N balance (input output) kg/ha P 2 O 5 balance (input output) kg/ha K 2 O balance (input output) kg/ha S11 Number of species grown score Pesticide treatment frequency inde c ha/ha Surface without crop cover on December 31 % Legume crop surface % a To guide interpretation, values considered unfavourable and favourable are indicated. b For each indicator most favourable value in bold. c Average number of standard pesticide treatments used by area and year. Standard treatment is the approved dosage for a certain crop. ratings for odour management than GAP, which is surprising, as it is generally perceived that pig production on straw litter (as for OA and RL) causes less odour problems than pig production on slatted floors (as for GAP) (Paul Robin, pers. comm., 2004). We rank GAP RL OA. Epressed per ha of farm surface, the EMA emission inventory consistently yielded the lowest values for OA, whereas GAP had highest values for five of the seven substances considered (Table 8), we rank OA > RL > GAP. Per kg of pig produced, no scenario clearly stood out. OA showed the lowest values for nitrate loss and ammonia loss, for two other emissions it shared the lowest values with RL. RL had the lowest carbon dioide emissions, while GAP emitted least oides of nitrogen and methane. We rank GAP RL OA FarmSmart Epressed per ha of farm surface, OA showed the lowest values for all indicators, whereas GAP had the highest value for eight indicators out of ten (Table 9), we rank OA > RL > GAP. Per kg of pig, OA showed the lowest value for nine indicators out of ten, but here RL did rather worse than GAP, as it had the highest value for seven indicators, we therefore rank OA > GAP > RL DIALECTE For the agro-environmental indicators presented here, OA showed the most favourable value in eight out of thirteen cases, whereas both RL and GAP presented the most favourable value for two indicators each (Table 10). We rank OA > RL GAP. Table 11 Values calculated using the DIALECTE whole farm approach for energy efficiency, farm diversity, input management and potential farm impacts for the three farm scenarios Indicator Unit Maimum score GAP RL OA Energy efficiency, output (in meat)/input a Farm diversity Crop diversity and soil cover Score Livestock diversity, autonomy and fertility transfer Score Natural elements and space Score Input management Nitrogen Score Phosphorous Score Water Score Pesticides Score Energy Score Potential farm impact on: Water (quality and quantity) Score Soil (fertility, erosion control) Score Biodiversity (animal and plant) Score Resource use Score Total score a For each indicator highest score in bold.

9 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) Table 12 Ranking of the three farm scenarios by the five evaluation methods Mode of epression Scenario Methods a LCA EF EMA FarmSmart DIALECTE PA EI AEI WFA Farm as a whole GAP RL OA Epressed per ha b GAP RL OA Epressed per kg of pig live weight GAP RL OA a For EMA the performance assessment (PA) and emissions inventory (EI) modes are distinguished; for DIALECTE the agro-ecological indicators (AEI) and whole farm approach (WFA) are distinguished. For each mode of epression numbers within a column indicate ranks with 1, best; 3, worst. b Per ha of land used (including land off-farm) for LCA, per ha of farm surface for EF, EMA, and FarmSmart. Within the whole farm approach, OA presented the highest and GAP the lowest energy output/input ratio (Table 11). For Farm diversity, Input management and potential farm impacts, OA showed the highest (i.e. best) scores for all indicators (Table 11). For seven indicators GAP had lowest scores and for one indicator RL had the lowest score. We rank OA > RL > GAP Ranking of the farm scenarios In order to obtain a view of the overall assessment of the three scenarios by the five methods, the rankings proposed in Section 3.2 (Tables 5 11) have been summarised in Table 12. Depending on the method, results were epressed for the farm as a whole, per ha, and per kg of pig live weight. LCA and EF produced identical rankings: when results were epressed per ha, OA did best and GAP worst, epressed per kg of product GAP did best and OA worst (Table 12). At the farm level, EMA established similar environmental performance for the three scenarios. In the EMA emissions inventory however, OA did best and GAP worst. When emissions were epressed per kg of pig produced, no clear differentiation emerged (Table 12). According to FarmSmart, OA did best, no matter whether results were epressed per ha or per kg of pig produced. When results were epressed per ha GAP did worse than RL, when results were epressed per kg of pig, RL did worse than GAP (Table 12). At the whole farm level, DIALECTE ranked OA first, both through its agro-environmental indicators and its whole farm approach. The agro-environmental indicators ranked GAP and RL similarly, whereas, in the whole farm approach, RL had better scores than GAP Emissions inventories Three methods produced emissions inventories, which are summarised in Table 13. For GAP and LR, LCA and EMA produce values that are reasonably close, whereas FarmSmart values (when available) are consistently higher. For OA, EMA produces lower values than LCA and FarmSmart. These differences in levels of emissions per unit surface should be related to the way the three methods define the system under evaluation and to the etent to which they consider off-farm emissions. LCA epresses results per ha of land used (including land off the farm) and considers off-farm emissions associated with a wide range of inputs: the Table 13 Emission inventories according to LCA (per ha land used), EMA-EI and FarmSmart (both per ha of farm surface), for the three farm scenarios Emission Unit (kg) LCA EMA FarmSmart GAP RL OA GAP RL OA GAP RL OA Nitrate NO a Oides of nitrogen NO Ammonia NH Sulphur dioide SO Methane CH Nitrous oide N 2 O Carbon dioide CO Carbon monoide CO a For each substance, lowest value in bold.

10 336 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) construction of pig housing, the production and delivery of concentrate pig feed (including growing of crops), and of fertilisers, pesticides, agricultural machines and energy carriers, including all sea and road transport involved. FarmSmart also considers a wide range of inputs, including concentrate feed, but emitted substances considered are limited to CO 2 and N 2 O, while emissions are epressed per ha of farm surface. This means, for instance, that nitrate leaching associated with the crops for the concentrate pig feed was part of the system in the LCA approach, but not in FarmSmart. EMA finally does not include any off-farm emissions associated with inputs and epresses emissions per ha of farm surface. In addition to these differences in system definition, the use of different algorithms or emission factors has further contributed to differences among the inventories. 4. Discussion 4.1. Characterisation of scenarios and methods The three farm scenarios evaluated here differ strongly, both in input use (e.g. non renewable energy use was 33 GJ ha 1 for OA and 77 GJ/ha 1 for GAP (Table 5), purchase of concentrate feed was 1.7 t ha 1 for OA and 9.0 t ha for GAP, data not shown), and in output (pig production per ha of farm surface was 1480 kg for OA and 4857 kg for GAP, Table 1). Thus, in relative terms, OA can be characterised as low input low output, GAP as high input high output, with RL being intermediate. The methods consider different environmental objectives and use different types of indicators. LCA and EF both consider input-related and emission-related objectives (Table 3), quantified by impact indicators (Table 4). EF (three objectives), has a more narrow focus than LCA (si objectives). EMA-PA deals uniquely with farming practice-related objectives, it is wide-ranging (eight objectives), whereas EMA-EI deals with four emission-related objectives. Both EMA-PA and EMA-EI use pressure indicators. FarmSmart, as LCA and EF, relies on input-related and emission-related objectives, which are quantified through pressure indicators. DIALECTE-AEI (11 objectives) and DIALECTE-WFA (13 objectives) are the only methods based on system staterelated objectives, in addition to farming practice and input use-related objectives. These methods use both pressure and state indicators Ranking of the scenarios We used the methods to rank the three scenarios according to their environmental impacts. Depending on the method used and on the way results were epressed (for the farm as a whole, per ha or per kg product), ranking from best to worst was OA > RL > GAP, or its inverse: GAP > RL > OA, or ranking proved inconclusive. For three methods (EMA-PA, DIALECTE-AEI, DIA- LECTE-WFA) rankings were established at the scale of the farm as a whole (Table 12). EMA-PA did not differentiate among the three scenarios, whereas DIALECTE-AEI and DIALECTE-WFA both ranked OA best, with DIALECTE- WFA ranking GAP worst, while DIALECTE-AEI attributed similar ratings to GAP and RL. EMA s global objective (assessment of adherence to best practice) suffices to eplain its lack of differentiation of the three scenarios, since all three confess adherence to good practice. Four methods (LCA, EF, EMA-EI, FarmSmart) allowed epression of results both per unit area and per unit product (Table 12). These methods are based on a limited number (three to si) of input-related and emission-related environmental objectives, several of which are shared (Table 3). The methods use impact and pressure indicators (Table 4). With results epressed per ha, all four methods ranked OA (low input low output) best and GAP (high input high output) worst (Table 12). However, with results epressed per kg of pig, rankings were not identical. LCA and EF ranked GAP best and OA worst, whereas FarmSmart ranked OA best and RL worst, while EMA-EI did not differentiate the scenarios. These results deserve a closer analysis. Increased use of inputs (e.g. fertilisers, concentrate feed) per unit area allows higher output of desired products, but also inevitably leads to more undesired outputs, i.e. emissions to the environment (e.g. De Koeijer et al., 2002; Schröder et al., 2003; Lewis et al., 2003). As a result, on a per area basis, impacts will logically tend to increase with increasing level of input use of the farm. This was confirmed here by the four methods producing identical rankings (OA > RL > GAP) when results were epressed per unit area. However, when impacts are epressed per kg of product, the correlation between input use per ha and impacts will depend on the ratio of undesired outputs (emissions to the environment) over desired outputs (products), both of which increase with input use, but not necessarily at the same rate. In the case-study presented here, the four methods did not produce identical rankings when impacts were epressed per unit product, illustrating greater sensitivity of this mode of epression to differences among the methods for environmental objectives considered and calculation algorithms used for the indicators. The results of the methods compared here clearly vary. Two main sources of difference can be distinguished (Table 12): (a) differences between modes of epression (e.g. LCA per ha versus LCA per kg) and (b) differences between methods within a mode of epression (e.g. EMA- PA versus DIALECTE-AEI) Differences between modes of epression The mode of epression of results had a major effect on rankings of the scenarios. The question whether impacts of agricultural production systems should be epressed per unit area or per unit product has been subject of considerable

11 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) debate. From the LCA point of view (Guinée et al., 2002), impacts should be epressed per unit product when the function of the system is the production of commodities, and per unit area for a non-market function (e.g. environmental services). Haas et al. (2000) have argued that for local/ regional impacts, such as eutrophication, epression per unit area is most appropriate, whereas for global impacts (e.g. climate change) impacts should be epressed per unit product. De Koeijer et al. (2002) prefer epression of impacts per unit area to take into account the carrying capacity of the environment. We feel that there is a strong case for epressing impacts of agricultural production systems both per unit area and per unit product. The impact/unit area ratio combines an environmental criterion and an area, the latter supplying a contet, allowing the implementation of area-based threshold values founded on critical limits or ceilings, which can be derived from national or regional goals for emissions or impacts (IPCC, 2001; Erisman et al., 2003). The impact/unit product ratio combines an environmental criterion and a production criterion, and thus is a measure of environmental efficiency (Olsthoorn et al., 2001), rather than a measure of environmental impact. The two modes of epression are clearly complementary. Reliance on the sole impact/unit area ratio may well lead to a preference for low input low output systems, which may decrease impacts at regional level, but may create a need for additional land use elsewhere, giving rise to additional impacts. On the other hand, reliance on the impact/unit product ratio only may well lead to a preference for high input high output systems, which, when concentrated at regional scale, have been shown to cause major pollution problems (Tamminga, 2003) Differences within modes of epression Within two of the three modes of epression major differences in the rankings were observed (Table 12). For the methods epressing results for the farm as a whole, EMA-PA differs from DIALECTE for its global objective (assessment of adherence to best practice for EMA, evaluation of environmental impact for DIALECTE), for the environmental objectives considered (Table 3) and the indicators used (Table 4). EMA-PA considers eight environmental objectives, and DIALECTE nineteen, but the methods have only two environmental objectives in common (Table 3). Thus here both the different global objectives and the nearly total disagreement with respect to the set of environmental objectives may have caused the different rankings. LCA, EF, EMA-EI and FarmSmart, which epress results both per ha and per kg of pig produced, yielded identical rankings when results were epressed per ha. When results were epressed per kg of pig produced, contrasting rankings were obtained, or scenarios were not differentiated. These methods are based on a limited number of environmental objectives most of which are not shared (Table 3). So here differences in the set of environmental objectives will probably have contributed to the different rankings. However, differences in calculation algorithms for the indicators used and in the way the boundaries of the system to be analysed were defined will also have played a major role, as can bee seen from the contrasting results obtained by the emissions inventories produced by three of these four methods (Table 13). As the four methods differ both for their set of objectives, indicators used and system definition, it is not possible to assess the relative contribution of each of these factors to the contrasting rankings obtained. 5. Conclusions This work has clearly demonstrated that the outcome of studies using indicator-based environmental evaluation methods to compare farming systems depends not only on the characteristics of the systems compared, but also to a large etent on those of the evaluation methods used. Five methods for evaluation of the environmental impacts of farms were used to rank three farm scenarios. Depending on the method, rankings obtained were similar, somewhat different or completely inverse, or ranking proved inconclusive. Outcomes differed due to differences in the evaluation methods concerning: (i) the global objective of the method, (ii) the set of environmental objectives considered, (iii) the definition of the boundaries of the system to be analysed, (iv) the calculation algorithms of the indicators used as evaluation criteria. As the methods compared differed for more than one of these characteristics, it was not possible to assess the relative importance of the contribution of each of these four factors. This study furthermore revealed the mode of epression of results (for the whole farm, per unit area, or per unit product) as a fifth factor strongly affecting the rankings obtained. Epression of impacts per unit area is complementary to epression per unit product. Reliance on the sole impact/unit area ratio may well lead to a preference for low input-low output systems, which may decrease impacts at regional level, but may create a need for additional land use elsewhere, giving rise to additional impacts. On the other hand, reliance on the impact/unit product ratio only may well lead to a preference for high input-high output systems, which, when concentrated at regional scale, have been shown to cause major pollution problems. We therefore recommend the use of evaluation methods that epress their results both per unit area and per unit product. More generally we recommend that environmental evaluation methods be used with great caution. Users should carefully consider which method is most appropriate given their particular needs, taking into consideration the method s global objective, its system definition, its set of environmental objectives, the quality of the indicators used and its mode of epression of results.

12 338 H.M.G. van der Werf et al. / Agriculture, Ecosystems and Environment 118 (2007) Acknowledgements This research was funded by an OECD research fellowship within the OECD Co-operative Research Programme: Biological Resource Management for Sustainable Agriculture Systems. The authors are solely responsible for the data and opinion herein presented, that do not represent the opinion of OECD. This work is part of the research programme Porcherie Verte (Green Piggery) and was financially supported by ADEME (Agence de l Environnement et de la Maîtrise de l Energie) and OFIVAL (Office National Interprofessionnel des Viandes, de l Elevage et de l Aviculture). References Basset-Mens, C., van der Werf, H.M.G., Scenario-based environmental assessment of farming systems: the case of pig production in France. Agric. Ecosyst. Environ. 105, Braband, D., Geier, U., Köpke, U., Bio-resource evaluation within agri-environmental assessment tools in different European countries. Agric. Ecosyst. Environ. 98, CEE, Règlement du conseil (CEE) no. 2092/9122, 1991 concernant le mode de production biologique de produits agricoles et sa présentation sur les produits agricoles et les denrées alimentaires. Journal Officiel de la Communauté Européenne, L198. De Koeijer, T.J., Wossink, G.A.A., Struik, P.C., Renkema, J.A., Measuring agricultural sustainability in terms of efficiency: the case of Dutch sugar beet growers. J. Environ. Manage. 66, EEA, Environmental indicators: typology and overview. Technical Report No. 25. European Environmental Agency, Copenhagen, Denmark. EEA, EEA core set of indicators. Guide. EEA Technical Report. No. 1/2005. European Environmental Agency, Copenhagen, Denmark. Erisman, J.W., Grenfelt, P., Sutton, M., The European perspective on nitrogen emission and deposition. Environ. Int. 29, FAO, FAOSTAT Agriculture Data. Goodlass, G., Halberg, N., Verschuur, G., Input output accounting systems in the European community an appraisal of their usefulness in raising awareness of environmental problems. Eur. J. Agron. 20, Groupement des Fermiers d Argoat, Cahier des charges éleveur porcs fermiers élevés en plein air. Homologation no , Saint- Brieuc, France. Guinée, J.B., Gorrée, M., Heijungs, R., Huppes, G., Kleijn, R., de Koning, A., van Oers, L., Wegener Sleeswijk, A., Suh, S., Udo de Haes, H.A., de Bruijn, H., van Duin, R., Huijbregts, M.A.J., Life cycle assessment. In: An Operational Guide to the ISO Standards, Centre of Environmental Science, Leiden University, Leiden, The Netherlands. Haas, G., Wetterich, F., Geier, U., Life cycle assessment framework in agriculture on the farm level. Int. J. Life Cycle Assess. 5 (6), Halberg, N., Verschuur, G., Goodlass, G., Farm level environmental indicators; are they useful? An overview of green accounting systems for European farms. Agric. Ecosyst. Environ. 105, Hansen, J.W., Is agricultural sustainability a useful concept? Agric. Syst. 50, Houghton, J.T., Meira Filho, L.G., Callander, B.A., Harris, N., Kattenberg, A., Maskell, K., Climate Change 1995: The Science of Climate Change. Cambridge University Press, Cambridge, United Kingdom. IPCC, In: Mitigation, Metz, B. (Eds.), Climate Change IPCC Secretariat, Geneva, Switzerland. Lewis, K.A., Bardon, K.S., A computer-based informal environmental management system for agriculture. Environ. Modell. Software 13, Lewis, K.A., Newbold, M.J., Tzilivakis, J., Developing an emissions inventory from farm data. J. Environ. Manage. 55, Lewis, D.R., McGechan, M.B., McTaggart, I.P., Simulating fieldscale nitrogen management scenarios involving fertiliser and slurry applications. Agric. Syst. 76, Ministère de l Agriculture et de la Pêche, Cahier des charges concernant le mode de production et de préparation biologique des animau et des produits animau définissant les modalités d application du règlement CEE no. 2092/91 modifié du conseil et/ou complétant les dispositions du règlement CEE no. 2092/91. Homologué par l arrêté interministériel du 28 août Journal Officiel de la République Française (30 August 2000). Paris, France. OECD, Environmental indicators for Agriculture. Concepts and Framework. Organisation for Economic Co-operation and Development, Paris, France. Olsthoorn, X., Tyteca, D., Wehrmeyer, W., Wagner, M., Environmental indicators for business: a review of the literature and standardisation methods. J. Cleaner Production 9, Petit, J., van der Werf, H.M.G., Perception of the environmental impacts of current and alternative modes of pig production by stakeholder groups. J. Environ. Manage. 68, Pointereau, P., Bochu, J.L., Doublet, S., Meiffren, I., Dimkic, C., Schumacher, W., Backhausen, J., Mayrhofer, P., Le diagnostic agrienvironnemental pour une agriculture respectueuse de l environnement. Trois méthodes passées à la loupe. Travau et Innovations. Société Agricole et Rurale d Edition et de Communication, Paris, France. Rees, W.E., Eco-footprint analysis: merits and brickbats. Ecol. Econ. 32 (3), Rosenberg, P.E., Gallot, J., Référentiel de l agriculture raisonnée. Arrêté du 30 avril 2002 relatif au référentiel de l agriculture raisonnée. Journal Officiel de la République Française (Paris, France) 104. Schröder, J.J., Aarts, H.F.M., ten Berge, H.F.M., van Keulen, H., Neeteson, J.J., An evaluation of whole-farm nitrogen balances and related indices for efficient nitrogen use. Eur. J. Agron. 20, Schröder, J.J., Scholefield, D., Cabral, F., Hofman, The effects of nutrient losses from agriculture on ground and surface water quality: the position of science in developing indicators for regulation. Environ. Sci. Policy 7, Solagro, DIALECTE, Diagnostic Liant Environnement et Contrat Territorial d Eploitation. User Manual, First Version. Solagro, Toulouse, France. Tamminga, S., Pollution due to nutrient losses and its control in European animal production. Livestock Production Sci. 84, Tzilivakis, J., Lewis, K.A., The development and use of farm-level indicators in England. Sustain. Dev. 12, Van der Werf, H.M.G., Petit, J., Evaluation of the environmental impact of agriculture at the farm level: a comparison and analysis of 12 indicator-based methods. Agric. Ecosyst. Environ. 93, Von Wirén-Lehr, S., Sustainability in agriculture an evaluation of principal goal-oriented concepts to close the gap between theory and practice. Agric. Ecosyst. Environ. 84, Wackernagel, M., Rees, W.E., Our Ecological Footprint. Reducing Human Impact on the Earth. New Society Publishers, Gabriola Island, British Columbia, Canada.