Weighted Summation (WSum)

Transcription

1

2 Table of Contents Weighted summation...1/6 1 Introduction...1/6 2 Methodology...1/6 3 Process...1/6 3.1 Value functions and standardization methods...2/6 3.2 Weighting methods...2/6 4 Review...3/6 4.1 Evaluation of results...3/6 4.2 Experiences...4/6 4.3 Combinations...4/6 4.4 Strengths and weaknesses...5/6 4.5 Further work...6/6 4.6 References...6/6 i

3 Weighted summation Marjan van Herwijnen 1 Introduction Weighted summation can be used to address problems that involve a finite and discrete set of alternative policies that have to be evaluated on the basis of conflicting objectives[1]. For any given objective, one or more different attributes or criteria are used to measure the performance in relation to that objective. These aspects, the impacts of all alternative options for all attributes, are presented in a so called evaluation table. The attributes are usually measured on different measurement scales and therefore cannot be compared with each other directly. Weighted summation is a compensatory method, which means that bad criterion scores can be compensated by good ones. The method is a special form of Multi Attribute Value Theory (MAVT, Keeney and Raiffa 1976) and is also called linear additive model. 2 Methodology Weighted summation makes the incomparable attributes comparable, prioritises them by assigning weights and finally reduces the amount of information by aggregating the weighted standardized scores. This process provides not only a ranking of the alternatives, but also comprehensibility and the strengths and weaknesses of the policy alternatives. Note that weighted summation, as all MAVT methods, can only be applied when the mutual preferential independence axiom applies.[2] 3 Process The process to be followed to carry out weighted summation is simple. The following steps have to be followed: 1. Definition of alternatives: identify the policy alternatives which are to be compared with each other. 2. Selection and definition of criteria: identify the effects or indicators relevant for the decision. 3. Assessment of scores for each alternative: assign values to each effect or indicator for all alternatives. 4. Standardization of the scores in order to make the criteria comparable with each other. 5. Weighting of criteria, in order to assign priorities to them. 6. Ranking of the alternatives. A total score for each alternative is calculated by multiplying the standardized scores with its appropriate weight, followed by summing the weighted scores of all criteria The first three steps are the same as in most MCA methods. Step 4 to 6 are specific for weighted summation. The calculation of the total score for each alternative a j, score(a j ), can be calculated using the following equation: (1) With: N: number of criteria 1/6

4 v i : value function for criterion c i s ij : score from alternative a j for criterion c i w i : weight for criterion c i A main difficulty with weighted summation is choosing an adequate value function to standardize the criterion scores and the attribution of the weights. Various methods are developed to carry out this task: 3.1 Value functions and standardization methods Given the variety of scales on which attributes can be measured, multicriteria decision analysis requires that the scores of the various criteria are transformed to comparable units. Only if the scales of the criteria are the same, the scores of these criteria can be compared or combined. Making the scores of the criteria comparable is often called standardization or normalization. Through a value function or standardization procedure the measurement units are made uniform, and the scores loose their dimension along with their measurement unit. Various methods to standardize scores are available: linear scale transformation methods like maximum standardization, interval standardization and goal standardization and also non linear value function approach. In particular, in applying weighted summation for Environmental Impact Assessments (EIAs), the linear function that presents interval standardisation is an often used form for the function v i (see Equation 2). This function has two parameters: the lowest and highest score for criterion c i, i.e. min and max. The effect scores s ij (=c i (a j )) are then transformed according to their relative position on the interval between the lowest and highest score ([min, max]) and to their relative position on the interval [0, 1]. This can be formulated as: (2) 3.2 Weighting methods Applying weighted summation is only possible if information about priorities of criteria is available. The decision maker has to give his/her preferences with respect to the evaluation criteria incorporated into the decision model. These preferences are expressed in priorities or weights and indicate trade offs between the criteria. The method of swing weighting to elicit weights for the criteria is mostly used (see von Winterfeldt and Edwards, 1986). This method is based on comparisons of differences: how does the swing from 0 to 100 on one preference scale compare to the 0 to 100 swing on another scale? To make these comparisons, assessors are encouraged to take into account both the difference between the least and most preferred options, and how much they care about that difference. Thus, the weight on a criterion reflects both the range of difference of the options, and how much that difference matters. Another way to assess weights is direct estimation of their relative importance by assigning a value to each criterion. This method assigns weights to criteria using a scale such as 0 to 10 or 0 to 100. If used carefully, i.e. taking into account the ranges of each criterion, direct estimation can be an effective methodology. Both methods become more complex if the number of criteria is large. In this case, the criteria can be arranged into groups. This hierarchical arrangement appears to be a good way for human beings to cope with complexity. The weights can be assigned in steps, first within each group separately and then between the groups. The final weights for each criterion can be calculated by multiplying the group weights by the weights within the groups. In practice, the weights within groups are assigned by experts and the weights between groups (for example between economical, social and environmental criteria) are assigned by the policy makers. 2/6

5 Other methods have been developed to estimate the relative importance of the criteria. A number of these methods are implemented in Janssen et al Review 4.1 Evaluation of results Evaluation results Weighted summation is theoretically well established, can be easily explained, and it is easy to use. Therefore, weighted summation is very suitable to apply in participatory processes. Weighted summation has been used to support the evaluation of a large number of problems all over the world. Difficulties are found with assigning weights to the criteria because of the subjectivity. Also, standardization implies to loose some information that may be valuable in some stages of the evaluation and implementation of policy options. Policy processes Weighted summation supports structuring the problem and guiding the search for further information. Therefore, it could also be used in investigating the nature of a problem and identifying conflicting assumptions. Weighted summation is not suitable to give support in the remaining policy processes, e.g., such as the implementation of the selected policy option. In particular, weighted summation is especially developed to support the comparison of policy options, to give an overview of the strong and weak points of the alternative policy options and finally to support the selection of the best one. Since it is relatively easy to use, it can support participative process. The explicit statement of the criteria s weights can enhance public debate, as well as the choice of the indicators. Because weighted summation identifies the strong and weak points of alternative policy options, it could also be used in exploring possible solutions to alleviate, mitigate or resolve the problem. Sustainable development aspects Similarly to the other MCA methods, weighted summation can shed light on different aspects of sustainability, since it uses various criteria that measure the performance of criteria in aspects which belong to the economic, the social and the environmental dimensions. Therefore, the three pillars of sustainable development can be and is often used to structure the criteria and use this structure to show the different impacts of the alternative policy options. Weighted summation can incorporate the following SD aspects as separate criteria to compare alternative policies: (de )coupling aspects, adaptability and (ir )reversibility. Weighted summation is suitable to compare impacts independently of the gauge year. Weighted summation can incorporate the impacts on different groups/sectors/regions as separate categories and give a clear overview of the differences for these sectors. Weighted summation is suitable to compare impacts independently of the spatial dimension, as long as the spatial dimension of the separate criteria are comparable. And finally, spatial weighted summation methods exist that can be applied to spatial data (Sharifi and Herwijnen 2002, Herwijnen 1999). Weighted summation is a method that transforms and aggregates available data into useful information, but is not a method that aimed at collecting data. It is not a method to address specific aspects of sustainable development in a separate manner. But it can be used to combine information in such a way that it can clarify sustainable development aspects. 3/6

6 Operational aspects Again, it is very difficult to estimate the costs for applying weighted summation, but a rough figure now is about 7000 euro. This is measured without gathering data. The costs for gathering data totally depend on the type of problem and cannot be estimated here. About two weeks are needed to apply weighted summation based on a collection of data already gathered. One week to structure the problem and one week to apply the method itself. Because ideally gathering data can only be done after the problem is structured, the time needed for an assessment is much longer and totally depends on the time needed to gather the data (estimation: between 1 and 12 months). Nevertheless, a lot of data is needed to estimate the impacts. If quantitative data is not available, expert judgement can be used to estimate the impacts on a qualitative scale. Weighted summation can handle quantitative as well as qualitative data. The main results of weighted summation are a rank order of policy options and an overview of the weak and strong points of policy options. The first result is easy to explain and not much expert judgement is needed for that. The second result is more difficult to get out of the method and the amount of expert judgement needed is higher. Computer programs are available to perform the needed calculations. Spreadsheets could also be used to apply weighted summation, but then the amount of expert knowledge needed. Weighted summation is very easy to understand and explain and the transparency is high. The separate steps are easy to follow, especially if a computer program is used. Most computer programs are user friendly and the reliability is high. Most computer programs support the analysis of uncertainty of the scores and/or weights. The marginality of effects can be distinguished because weighted summation compares each policy alternative with other alternatives and/or the current situation. The impacts of the policy alternatives can be compared to overall scope for change by using goal standardization. Weighted summation has no mandatory use. The time the results become outdated depends totally on the input data and the context of application, so this cannot be established. Weighted summation looks at impacts of one selected year, which year is up to the user. Because the criteria are defined by the user and depend on the problem, weighted summation is independent of country, geographic level and level of aggregation. One condition however is that the criteria should be defined on comparable countries, geographic levels and levels of aggregation. 4.2 Experiences In the Netherlands, the commission of Environmental Impact Assessment (EIA) advised to apply weighted summation to evaluate alternative solutions. Weighted summation has been applied in a large number of EIAs in the Netherlands. The standardization process and the weights applied entails a high degree of subjectivity, so that they are usually difficult to explain. Incorporating stakeholder perspectives, explaining the strengths and weaknesses of the alternatives policies and presenting and explaining the final ranking has always been successful. Weighted summation has been applied in other countries too (ODPM 2004, Beinat and Nijkamp 1998). To apply an effective weighted summation good supporting software is essential (Belton and Stewart 2002, Janssen et al. 2001). 4.3 Combinations Weighted summation can be used to compare the impact of alternative policies generated by other tools like physical assessment tools, modelling tools and environmental appraisal tools. This means that weighted summation can and even has to be combined with modelling tools and indicator based assessment to estimate 4/6

7 specific impacts of alternative policies. Weighted summation can also be combined with Geographic Information Systems and spatial models to analyse specific spatial impacts of alternative policies (Herwijnen, 1999). Because weighted summation is a consultative process, it has also been used in participatory processes to present the strengths and weaknesses of the alternative policies. In particular, to find out the perspectives of various stakeholders weighted summation can be combined with stakeholder analyses. On the one hand, weighted summation can use the input from stakeholder groups to construct the criteria tree, to develop alternative options/solutions for the problem and to prioritize the criteria. Since it is very difficult to reach an agreement among the stakeholders on these issue, a sensitivity analysis can be used to check if the result is robust enough for these diverging perceptions. In this sense, transparency plays a crucial role. On the other hand, the stakeholders can use the method as a tool of communication to clarify the different points of views and show the results. Then the tool is used as a tool for reasoning in order to support and improve negotiations. Finally, weighted summation can be used to support the evaluation of alternative policies/plans/projects in SIA and SEA. 4.4 Strengths and weaknesses Strengths of weighted summation As all MCA methods, weighted summation helps in structuring a problem. By classifying the problem in various objectives, criteria to measure the objectives and alternative options weighted summation provides a structured approach of dealing with the problem. Weighted summation accommodates various types of information: quantitative as well as qualitative. Weighted summation provides transparency to the evaluation process, due to its simple, easy to explain, methodology. Therefore it is very suitable to be used in participatory processes. Weighted summation provides a means of communication for reasoning and negotiations by clarifying the strengths and weaknesses of the alternative policies and by the possibility to clearly visualize and communicate the intermediate and final results. If used in a participatory way, weighted summation can incorporate the diverse views of stakeholder groups to construct the criteria tree, to develop alternative options/solutions for the problem and to prioritize the criteria. On the other hand, the stakeholders can use the method a tool for reasoning in order to support and improve negotiations. To apply an effective weighted summation, a good supporting software is essential (Belton and Stewart 2002). Much software is available to apply weighted summation (see Appendix 1, Janssen et al and bosda.nl). An advantage of using software is the ease to test the robustness of the results using sensitivity analyses. Weaknesses of weighted summation In weighted summation s the similar weaknesses made for MAVT can be considered. These are mainly the assumption of compensability between criteria and the loss of information due to normalization. Also, assigning weights is a difficult task, especially if the number of criteria is large and the criteria are very different in character. To support the decision makers with this task, different methods to estimate the relative importance of the criteria have been developed. Weighted summation can only be applied if the attributes are additive. This assumption means that there should be no interaction between the attributes, meaning that the attributes should be independent of each other, which is in many cases an unrealistic assumption. 5/6

8 4.5 Further work Examples at the EU scale are not found yet. These applications are necessary to demonstrate the use of this method for the assessment of EU policies. Weighted summation, as it is the case for the other MCA method, compares different alternative options under different criteria at the same time, but this situation in reality, rarely occurs within the EU policy practice. Although now, a first step has been made in the impact assessment guidelines developed by the EU (SEC, 2005) who advice to condider alternative policy options instead of only one. 4.6 References. Beinat, E. & P. Nijkamp (1998). Multicriteria Analysis for Land Use Management. Kluwer Academic Publishers: Dordrecht. Belton, V., Stewart, T.J. (2002). Multiple Criteria Decision Analysis: An Integrated Approach. Kluwer Academic Publishers: Dordrecht. Herwijnen, M. van (1999). Spatial Decision Support for Environmental Management. Vrije Universiteit, Amsterdam. Janssen, R., M. van Herwijnen and E. Beinat (2001). DEFINITE for Windows. A system to support decisions on a finite set of alternatives (Software package and user manual). Institute for Environmental Studies (IVM), Vrije Universiteit: Amsterdam. Internet: bosda.nl Keeney, R. & Raiffa, H. (1976). Decisions with multiple objectives: preferences and value trade offs. Wiley: New York. Office of the Deputy Prime Minister (ODPM, Government UK, 2000). DTLR multi criteria analysis manual. Corporate Publication. Internet: SEC (2005). Impact Assessment Guidelines, European Commission, SEC (2005) 791, 15 June Sharifi, A. and M. van Herwijnen (2002). Spatial Decision Support Systems. International Institute for Geoinformation Science and Earth Observation (ITC) Enschede. Von Winterfeldt, D. and W. Edwards (1986). Decision Analysis and Behavioural Research. Cambridge University Press: Cambridge. 6/6