Project Risk Evaluation Using a Fuzzy Analytic Hierarchy Process: An Application to Information Technology Projects

Transcription

1 Project Risk Evaluation Using a Fuzzy Analytic Hierarchy Process: An Application to Information Technology Projects Fatih Tüysüz, 1, * Cengiz Kahraman 2, 1 Department of Mathematics, Istanbul Bilgi University, Dolapdere, Istanbul, Turkey 2 Department of Industrial Engineering, Istanbul Technical University, Macka, Istanbul, Turkey Projects are critical to the realization of performing organization s strategies. Each project contains some degree of risk and it is required to be aware of these risks and to develop the necessary responses to get the desired level of project success. Because projects risks are multidimensional, they must be evaluated by using multi-attribute decision-making methods. The aim of this article is to provide an analytic tool to evaluate the project risks under incomplete and vague information. The fuzzy analytic hierarchy process ~AHP! as a suitable and practical way of evaluating project risks based on the heuristic knowledge of experts is used to evaluate the riskiness of an information technology ~IT! project of a Turkish firm. The means of the triangular fuzzy numbers produced by the IT experts for each comparison are successfully used in the pairwise comparison matrices Wiley Periodicals, Inc. 1. INTRODUCTION Today s increasingly uncertain world yields a highly competitive environment for every business. For any organization to be successful in such an environment, it is required to adapt and respond to the changes by developing appropriate business strategies. One of the means of achieving this is undertaking projects that are critical to the realization of the performing organization s strategies. Projects may differ in size, duration, objectives, uncertainty, complexity, pace, and some other dimensions. It does not matter how different or unique a project is; there is no doubt that every project contains some degree of uncertainty and there is no risk-free project. Experience and some studies show that risk management undertaken in a project has an effect on the level of success of a project. Elkington and Smallman 1 find that there is a strong relationship between the amount of risk *Author to whom all correspondence should be addressed: fatiht@bilgi.edu.tr. kahramanc@itu.edu.tr. INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, VOL. 21, ~2006! 2006 Wiley Periodicals, Inc. Published online in Wiley InterScience ~

2 560 TÜYSÜZ AND KAHRAMAN management undertaken in a project and the level of success of the project; more successful projects use more risk management. Also in a more recent study, Raz et al. 2 observe that project risk management practices are correlated with success in achieving time and budget goals, which is one of the most common risks encountered in most projects. Because unanticipated risks or unmanaged risks are two of the main causes of project failure, organizations or project teams need to be prepared for project risks in order to attain the desired level of project success. This requires being aware of project risks and managing them. Risk management in projects has been practiced since the mid-1980s and is one of the nine main knowledge areas of the Project Management Institute s Project Management Body of Knowledge. Risk management is defined as the systematic process of identifying, analyzing, and responding to project risks. 3 Cooper and Chapman 4 and Chapman and Ward 5 identify risk management as a multiphase risk analysis that covers identification, evaluation, control, and management of risks. Chapman 6 also suggests that a formal risk management process should be applied at all stages in the project life cycle and it should be a project in itself. Although there are different project risk management approaches in the literature, the aim of project risk management is the same, which is to increase the project performance and also to achieve the project objectives by identifying and evaluating the possible risks and developing appropriate responses to them. This article mainly focuses on the evaluation phase of the project risk management process, which is a certain common element in all approaches. Chapman and Ward 5 consider the evaluation phase as central in the risk management process. Once the possible risks and their characteristics that may affect the project are identified, they must be evaluated. Risk evaluation is the process of assessing the impact and likelihood of identified risks. The aim of risk evaluation is determining the importance of risks and prioritizing them according to their effects on project objectives for further attention and action. Evaluation techniques can be mainly classified into two groups; these are qualitative methods and quantitative methods. Qualitative methods describe the characteristics of each risk in sufficient detail to allow them to be understood. Quantitative methods use mathematical models to simulate the effect of risks on project outcomes. 7 The most commonly used qualitative methods are the probability impact risk rating matrix, which is constructed to assign risk ratings to risks or conditions based on combining probability and impact scales, 3 and the use of a risk breakdown structure ~RBS! to group risks by source. 7 Quantitative methods include Monte Carlo simulation, decision trees, and sensitivity analysis. These two kinds of methods, qualitative and quantitative, can be used separately or together. As indicated before, the main aim of risk evaluation is to determine the relative significance of different sources of risk on the overall project. In other words, it is for determining which risk events warrant response. 3 This is because every project has different risks and, indeed, different levels of risk. 1 Chapman and Ward 5 suggest the approach of evaluating and assessing the risk as groups, and then determining the impact on the project in a cumulative manner. 1 There are different classifications of risk groups in the literature. Elkington and Smallman 1 classify project

3 PROJECT RISK EVALUATION 561 risks in four groups, which are business risks, procurement risks, management risks, and technical risks. Miller and Lessard 8 classify project risks in a more general way as market-related risks, technical risks, and institutional risks. Mustafa and Al-Bahar 9 classify different sources of risk in construction projects as acts-of- God risks, physical risks, financial and economic risks, and job-site-related risks. Kerzner 10 gives a more detailed classification of risks, which are cost, funding, schedule, contract relationship, political, technical, production, and support risks. These classifications are important and can especially be used in the identification phase of risk factors. Because every project is different from another, not every risk factor valid for a certain project will be valid for others. Risk factors for a project should be considered as specific to that project. Keeping this in mind, the role of risk evaluation can be summarized as determining the weights of predetermined risk factors and determining the ones that need to be handled. The need for this comes from the fact that because projects face a large number of risks, each having different effects on the project, it may be impractical or usually even impossible to manage them all because of time and resource constraints. Multicriteria decision-making methods are an important set of tools for addressing challenging business decisions because they allow the manager to better proceed in the face of uncertainty, complexity, and conflicting objectives. 11 Because risks are multidimensional, 8 they should be evaluated with respect to more than one criterion to get more accurate and reliable results. The analytic hierarchy process ~AHP! is one of the extensively used multicriteria decision-making methods. One of the main advantages of this method is the relative ease with which it handles multiple criteria. In addition to this, AHP is easier to understand and it can effectively handle both qualitative and quantitative data. Mustafa and Al-Bahar 9 introduce the approach of using AHP for project risk evaluation. They apply AHP in assessing the riskiness of a construction project in Bangladesh. The importance of their work is that it is the first on the utilization of AHP in risk evaluation. Dey 12 uses AHP and decision tree analysis as a quantitative approach to construction risk management. He uses the AHP for determining the probability of occurrence of various risk factors and displays the benefits of using it. Millet and Wedley 13 show how AHP can be used to model risk and uncertainty in a variety of ways by introducing prototypical case studies. The risk level evaluation of project risks is a complex subject including uncertainty. The imprecise and vague terms will exist, because most project managers find it more practical and easier to evaluate risk in linguistic terms. Fuzzy sets theory introduced by Zadeh 14 is especially powerful when there is a need to take into consideration the ideas and judgments of people because of complexity and lack of proper information. Fuzzy sets provide representation of the knowledge of project managers in a better and more natural way. Because AHP does not take into account the uncertainty associated with the mapping of one s judgment to a number and also the subjective judgments, selection, and preference of decision makers exert a strong influence in the AHP 15 ; fuzzy sets theory can be used to overcome these shortcomings of AHP. In this article, we propose the use of fuzzy AHP ~FAHP! as a suitable and practical way of evaluating project risks based on the heuristic knowledge of project managers. Although fuzzy logic and AHP have

4 G G 562 TÜYSÜZ AND KAHRAMAN been separately used in the evaluation of project risks, the significant contribution of this article, as the first, is the suggestion of the use of FAHP in project risk evaluation. The organization of the article is as follows. In Section 2, fuzzy sets and a literature review about risk measurement are given. In Section 3, multi-attribute evaluation under fuzziness and its literature review and the comparison of the fuzzy AHP methods are presented. In Section 4, the extent analysis method on fuzzy AHP is explained. In Section 5, an application of risk evaluation for an IT project using fuzzy AHP is given. Finally the conclusions are presented. 2. FUZZY SETS AND RISK MEASUREMENT To deal with vagueness of human thought, Zadeh 14 first introduced the fuzzy set theory, which was based on the rationality of uncertainty due to imprecision or vagueness. A major contribution of fuzzy set theory is its capability of representing vague knowledge. The theory also allows mathematical operators and programming to apply to the fuzzy domain. A fuzzy number is a normal and convex fuzzy set with membership function m A ~x!, which both satisfies normality, m A ~x! 1, for at least one x R and convexity, m A ~x '! m A ~x 1!Lm A ~x 2!, where m A and x 1, x 2 #. L stands for the minimization operator. A tilde will be placed above a symbol if the symbol represents a fuzzy set. A fuzzy number is a special fuzzy subset of the real numbers. The membership function of a triangular fuzzy number ~TFN!, M, is defined by m~x6m G! ~m 1, f 1 ~ y6m G!/m 2, m 2 /f 2 ~ y6m G!, m 3! ~1! where m 1 m 2 m 3, f 1 ~ y6m G! is a continuous monotone increasing function of y for 0 y 1 with f 1 ~06M G! m 1 and f 1 ~16M G! m 2 and f 2 ~ y6m G! is a continuous monotone decreasing function of y for 0 y 1 with f 2 ~16M G! m 2 and f 2 ~06M G! m 3. m~x6m G! is denoted simply as ~m 1 /m 2, m 2 /m 3!. Figure 1 presents a TFN, M. The extended operations of fuzzy numbers can be found in Ref. 16. Fuzzy numbers are used for risk evaluation in various ways in the literature. Kuchta 17 puts forward a fuzzy way of measuring the criticality of project activities and of the whole project. The criticality measure serves as a measure of risk and helps in making the decision whether to accept or to reject the project. In this approach, the decision maker expresses what he/she understands by very critical, a little critical, and so forth in the form of a fuzzy number. According to this approach, the criticality of the project ~CritPr! is obtained by Equation 2: N Crit Pr ( w i {Crit i ~2! i 1 where w i ~i 1,2,...,N! are weights such that (i 1 N w i 1. N is the number of project activities and Crit i is the criticality of the project activity i. Bonvicini et al. 18 provide an application of fuzzy logic to the risk assessment of the transport of hazardous materials by road and pipeline in order to evaluate

5 PROJECT RISK EVALUATION 563 Figure 1. A Triangular fuzzy number, M. G the uncertainties affecting both individual and societal risk estimates. In evaluating uncertainty by fuzzy logic, the uncertain input parameters are described by fuzzy numbers, and calculations are performed using fuzzy arithmetic. A connection between the degree of membership and the probability of occurrence is established by means of a bijective transformation that turns a probability measure into a degree of membership. Carr and Tah 19 present a fuzzy risk analysis model in which a hierarchical risk breakdown structure is described to represent a formal model for qualitative risk assessment. The relationship between risk factors, risks, and their consequences are represented on cause and effect diagrams. Risk descriptions and their consequences are defined using descriptive linguistic variables. They use fuzzy approximation and composition to identify and quantify the relationship between risk sources and the consequences on project performance measures. The main objective of their model is to evaluate the risk exposures considering consequences in terms of time, cost, quality, and safety performance measures of the entire project based on fuzzy estimates of the risk components. Cho et al. 20 propose a methodology for incorporating uncertainties using fuzzy concepts into conventional risk assessment frameworks. They introduce some forms of fuzzy membership curves that are designed to consider the uncertainty range that represents the degree of uncertainties involved in both probabilistic parameter estimates and subjective judgments. They use linguistic variables such as Close to any value or Higher/Lower than analyzed value and so forth that include some quantification with giving specific value or scale. Three types of membership functions proposed for the statements Close to, Lower than, and Higher than curves are defined. For example, the membership function for the linguistic variable close to is defined as 0.0 y x ' 0.5 y ~@~2 2x ' 1/y!# y! n 0.5 y x ' 1.0 y ~3! f A ~x! ~@2x ' 1/y# y! n

6 564 TÜYSÜZ AND KAHRAMAN where n is the coefficient of power according to linguistic variables, x ' is the estimated or assumed value of each risk event as a fuzzy number ~such as occurrence probability, etc.!, and y is calculated by using the value of the fuzzy number at the midpoint, so that 0.5 y x '. Huang 21 uses the interior-outer-set model to calculate the risk of crop flood and rank farming alternatives for Huarong County, China, where only a small sample of eight observations is available. The risk calculated by the suggested model is a particular case among imprecise probabilities, called possibility probability distribution. He discusses in detail how to order alternatives based on a possibility probability distribution and shows that the ordering based on a calculated fuzzy risk is better than one based on a histogram estimate. Huang and Moraga 22 develop a matrix algorithm for the same model because the model involves combination calculus that is very difficult to follow. This matrix algorithm consists of a moving subalgorithm and an index subalgorithm. A moving subalgorithm works out leaving and joining matrices and an index subalgorithm is a combination algorithm to get index sets. They also present an example of how to calculate a risk of a strong earthquake with the algorithm. Lee et al. 23 present a new and flexible algorithm to evaluate the rate of aggregative risk in fuzzy circumstances by fuzzy sets theory during any phase of the software development life cycle. In this algorithm, each individual risk item is ranked using two fuzzy sets with triangular membership functions, grade of risk and grade of importance. Then the rate of each individual risk item is evaluated by multiplication by the centroid method. According to this algorithm, the final rate of aggregative risk ~New_RIK! in software development is obtained by Equation 4: New_RIK n ( h 1 GW 2 ~h! * R2~h! n ( h 1 GW 2 ~h! 6 ( GW 2 ~h! * R2~h! ~4! h 1 where h 1,2,...,n is the number of the risk attributes, GW 2 ~h! is the grade of importance of the risk items for the hth attribute, and R2~h! is the rate of risk for n the hth attribute ~(h 1 GW 2 ~h! 1!. Chen and Chen 24 present a method for fuzzy risk analysis based on similarity measures of generalized fuzzy numbers. They propose a method named the simple center of gravity method ~SCGM! to calculate the center-of-gravity ~COG! points of generalized fuzzy numbers and then use the SCGM to measure the degree of similarity between generalized fuzzy numbers. Their method takes into consideration the degrees of confidence of decision makers opinions. 3. MULTI-ATTRIBUTE EVALUATION UNDER FUZZINESS: FUZZY AHP There are many fuzzy AHP methods proposed by various authors. These methods are systematic approaches to the alternative selection and justification

7 PROJECT RISK EVALUATION 565 problem by using the concepts of fuzzy set theory and hierarchical structure analysis. Decision makers usually find that it is more certain to give interval judgments than fixed value judgments. This is because usually he/she is unable to be explicit about his/her preferences due to the fuzzy nature of the comparison process. The earliest work in fuzzy AHP appeared in Ref. 25, which compared fuzzy ratios described by triangular membership functions. Buckley 26 determines fuzzy priorities of comparison ratios whose membership functions are trapezoidal. Stam et al. 27 explore how recently developed artificial intelligence techniques can be used to determine or approximate the preference ratings in AHP. They conclude that the feed-forward neural network formulation appears to be a powerful tool for analyzing discrete alternative multicriteria decision problems with imprecise or fuzzy ratio-scale preference judgments. Chang 28 introduces a new approach for handling fuzzy AHP, with the use of triangular fuzzy numbers for pairwise comparison scale of fuzzy AHP, and the use of the extent analysis method for the synthetic extent values of the pairwise comparisons. Cheng 29 proposes a new algorithm for evaluating naval tactical missile systems by the fuzzy analytical hierarchy process based on grade value of the membership function. Weck et al. 30 present a method to evaluate different production cycle alternatives adding the mathematics of fuzzy logic to the classical AHP. Any production cycle evaluated in this manner yields a fuzzy set. The outcome of the analysis can finally be defuzzified by forming the surface center of gravity of any fuzzy set, and the alternative production cycles investigated can be ranked in order in terms of the main objective set. Kahraman et al. 31 use a fuzzy objective and subjective method obtaining the weights from AHP and make a fuzzy weighted evaluation. Deng 32 presents a fuzzy approach for tackling qualitative multicriteria analysis problems in a simple and straightforward manner. Lee et al. 33 review the basic ideas behind the AHP. Based on these ideas, they introduce the concept of a comparison interval and propose a methodology based on stochastic optimization to achieve global consistency and to accommodate the fuzzy nature of the comparison process. Cheng et al. 34 propose a new method for evaluating weapons systems by an analytical hierarchy process based on linguistic variable weight. Zhu et al. 35 discuss the extent analysis method and applications of fuzzy AHP. Chan et al. 36 present a technology selection algorithm to quantify both tangible and intangible benefits in a fuzzy environment. They describe an application of the theory of fuzzy sets to hierarchical structural analysis and economic evaluations. By aggregating the hierarchy, the preferential weight of each alternative technology is found, which is called fuzzy appropriate index. The fuzzy appropriate indices of different technologies are then ranked and preferential ranking orders of technologies are found. From the economic evaluation perspective, a fuzzy cash flow analysis is employed. Chan et al. 37 report an integrated approach for the automatic design of flexible manufacturing systems ~FMS!, which uses simulation and multicriteria decision-making techniques. The design process consists of the construction and testing of alternative designs using simulation methods. The selection of the most suitable design ~based on AHP! is employed to analyze the output from the FMS simulation models. Intelligent tools ~such as expert systems, fuzzy systems, and neural networks! are developed for supporting the FMS

8 566 TÜYSÜZ AND KAHRAMAN design process. The active X technique is used for the actual integration of the FMS automatic design process and the intelligent decision support process. Leung and Cao 38 propose a fuzzy consistency definition with consideration of a tolerance deviation. Essentially, the fuzzy ratios of relative importance, allowing certain tolerance deviation, are formulated as constraints on the membership values of the local priorities. The fuzzy local and global weights are determined via the extension principle. The alternatives are ranked on the basis of the global weights by application of maximum minimum set ranking method. Kuo et al. 39 develop a decision support system for locating a new convenience store. The first component of the proposed system is the hierarchical structure development for the fuzzy analytic process. Wang and Lin 40 use a fuzzy multicriteria group decisionmaking approach to select configuration for software development. Bozdağ et al., 41 Kahraman et al., Buyukozkan et al., 45 and Kulak and Kahraman 46 use Chang s 28 fuzzy AHP for various decision-making problems. Sheu 47 presents a hybrid fuzzy-based method that integrates fuzzy-ahp and fuzzy multi-attribute decision making ~MADM! approaches for identifying global logistics strategies when corresponding supply and demand environments are complicated and uncertain. Table I gives a comparison of the fuzzy AHP methods in the literature that have important differences in their theoretical structures. The comparison includes the advantages and disadvantages of each method. Because the advantages of Chang s 28 extent analysis on fuzzy AHP are relatively superior to the others due to the reasons mentioned in Table I, this method will be used in project risk evaluation. In the literature, no publication dealing with project risk evaluation using fuzzy AHP has been encountered. In the following section, the extent analysis method on fuzzy AHP will be presented and then it will be applied to project risk evaluation. 4. EXTENT ANALYSIS METHOD ON FUZZY AHP Let X $x 1, x 2,...,x n % be an object set, and G $g 1, g 2,...,g m % be a goal set. According to the method of Chang s extent analysis model, each object is taken and extent analysis for each goal, g i, is performed, respectively. 28,50 Therefore, m extent analysis values for each object can be obtained, with the following signs: M 1 gi, M 2 gi,...,m m gi, i 1,2,...,n ~5! j where all the M gi ~ j 1,2,...,n! are triangular fuzzy numbers ~TFNs!. The value of fuzzy synthetic extent with respect to the ith object is defined as m S i ( j 1 j M gi n ( i 1 m j 1 ( j 1 M gi ~6! The degree of possibility of M1 M2 is defined as V~M 1 M 2! sup{min~m M1 ~x!, m M2 ~ y!!} ~7! x y

9 Table I. The comparison of different fuzzy AHP methods. Sources The main characteristics of the method Advantages ~! and disadvantages ~! Ref. 25 Ref. 26 Ref. 48 * Direct extension of Saaty s AHP method with triangular fuzzy numbers * Lootsma s logarithmic least square method is used to derive fuzzy weights and fuzzy performance scores * Extension of Saaty s AHP method with trapezoidal fuzzy numbers * Uses the geometric mean method to derive fuzzy weights and performance scores * Modifies van Laarhoven and Pedrycz s method * Presents a more robust approach to the normalization of the local priorities ~! The opinions of multiple decision makers can be modeled in the reciprocal matrix. ~! There is not always a solution to the linear equations. ~! The computational requirement is tremendous, even for a small problem. ~! It allows only triangular fuzzy numbers to be used. ~! It is easy to extend to the fuzzy case. ~! It guarantees a unique solution to the reciprocal comparison matrix. ~! The computational requirement is tremendous. ~! The opinions of multiple decision makers can be modeled. ~! The computational requirement is tremendous. Ref. 28 * Synthetical degree values ~! The computational requirement is relatively low. * Layer simple sequencing ~! It follows the steps of crisp AHP. It does not involve additional operations. * Composite total sequencing ~! It allows only triangular fuzzy numbers to be used. Ref. 49 * Builds fuzzy standards ~! The computational requirement is not tremendous. * Represents performance scores by membership functions * Uses entropy concepts to calculate aggregate weights PROJECT RISK EVALUATION 567 ~! Entropy is used when probability distribution is known. The method is based on both probability and possibility measures. When a pair ~x, y! exists such that x y and m M1 ~x! m M2 ~ y!, then we have V~M 1 M 2! 1. Because M1 and M2 are convex fuzzy numbers, we have that V~M 1 M 2! 1 iff m 1 m 2 ~8! V~M 2 M 1! hgt~m 1 M 2! if m2 m1 0, if l 1 u 2 m M2 ~d! 1, l 1 u 2 ~m 2 u 2! ~m 1 l 1!, otherwise ~9!

10 G 568 TÜYSÜZ AND KAHRAMAN where d is the ordinate of the highest intersection point D between m M1 and m M2 ~see Figure 2!. To compare M 1 and M 2, we need both the values of V~M 1 M 2! and V~M 2 M 1!. The degree possibility for a convex fuzzy number to be greater than k convex fuzzy numbers M i ~i 1,2,...k! can be defined by V~M M 1, M 2,...,M k! M 1! and ~M M 2! and... and ~M M k!# Assume that min V~M M i!, i 1,2,3,...,k ~10! d ' ~A i! min V~S i S k! ~11! for k 1,2,...,n; k i. Then the weight vector is given by W ' ~d ' ~A 1!, d ' ~A 2 '!,...,d ' ~A n!! T ~12! where A i ~i 1,2,...,n! are n elements. Via normalization, the normalized weight vectors are W ~d~a 1!, d~a 2!,...,d~A n!! T ~13! where W is not a fuzzy number. The issue of consistency in fuzzy AHP is another subject that needs to be examined. The consistency of a comparison matrix in crisp AHP is measured by the consistency ratio CR, which is equal to CR CI/RI ~14! Figure 2. The intersection between MG 1 and M 2.

11 where RI is a random index. If the CR 0.10, the decision maker has to make the pairwise judgments again. 51 A fuzzy comparison matrix is defined to be consistent within tolerance deviation d, ifthea-level cut feasible region S a ' is not empty 38 : S a ' w : ~1 d!l ija w i w j ~1 d!u ija, i j 1,...,n,w j 0, ( w j 1 ~15! j 1,...,n where w i and w j are the weights of the ith and jth elements, respectively. Here d represents deviations from the upper bound U ija and the lower bound L ija. A practical way to test the fuzzy comparison consistency within tolerance deviation d is to solve the following auxiliary linear program: min b b 1 b 2 PROJECT RISK EVALUATION 569 s.t. ln~1 d!l ija ln~w i! ln~w j! b 1ij b 2ij ln~1 d!u ij1, i j,1,...,n b 1 b 1ij, b 2 b 2ij, b 1ij, b 2ij 0 ~16! where ln~w i!, b 1ij, b 2ij, b 1, b 2 are decision variables. If b 0, the fuzzy comparison matrix is consistent within tolerance deviation d. Ifb 0, this means that there are no feasible weights ~S 1 ' f!, that the fuzzy comparison matrix is not consistent within d. In this case, the decision maker would make the judgments again. Mikhailov s 52 approach is based on extending the region by deviation parameters, but it does not require the existence of a nonempty extended region. The deviation parameters are mainly used to define appropriate membership functions, which measure the decision-maker satisfaction with different candidate solutions. 5. RISK EVALUATION OF AN INFORMATION TECHNOLOGY PROJECT USING FUZZY AHP In this part of the study, the fuzzy AHP method based on Chang s extent analysis will be applied to an information technology project to measure its risk level. The first step of the fuzzy AHP method requires constructing an appropriate hierarchy of the fuzzy AHP model, which consists of the goal, criteria, and the alternatives. The triangular fuzzy conversion scale used in the model is given in Table II. Our goal is to evaluate the riskiness of an IT project and it is at the top of the hierarchy as shown in Figure 3. Risk factors used in the model are selected from the studies of Barki et al. 53 and Schmidt et al. 54 They are divided into six different risk groups and shown on the second level. The third level consists of 28 sub-risk factors. The explanation of these risk factors is given in Table III.

12 570 TÜYSÜZ AND KAHRAMAN Table II. Triangular fuzzy conversion scale. Linguistic scale Triangular fuzzy scale Triangular fuzzy reciprocal scale Just equal ~1, 1, 1! ~1, 1, 1! Equally important ~1/2, 1, 3/2! ~2/3, 1, 2! Weakly ~1, 3/2, 2! ~1/2, 2/3, 1! Strongly ~3/2, 2, 5/2! ~2/5, 1/2, 2/3! Very strongly ~2, 5/2, 3! ~1/3, 2/5, 1/2! Absolutely ~5/2, 3, 7/2! ~2/7, 1/3, 2/5! A software development firm in Turkey, SOFTEK Co., considers investing in an IT project. The top management will make its decision depending on the opinions of the 11 IT experts of the firm. The top management categorizes the risk levels as high risk, moderate risk, and low risk. If the final result of the risk evaluation based on the experts opinions is revealed as high risk, the top management will reject the project; otherwise the IT project will be acceptable. The risk evaluation matrix with respect to the goal is given in Table IV. The experts now compare the sub-risk factors with respect to the main risk groups. First they compare the sub-risk factors of environment and ownership. The fuzzy comparison matrices for main risk groups and the weight vectors of each matrix are given in Tables V X. Table V gives the fuzzy comparison data of environment and ownership. Finally, the experts assess the risk level of sub-risk factors by again making pairwise comparisons between high risk, moderate risk, and low risk. One sample comparison matrix for risk levels for each sub-risk factor under main risk groups is given in Tables XI XVI. Sample questionnaire forms to receive the experts assessments are given in Appendix ~Tables AI and AII!. By using Formula 6, fuzzy synthetic values are obtained as follows: S EO ~0.08, 0.138, 0.228!, S RM ~0.084, 0.147, 0.25! S PM ~0.143, 0.248, 0.407!, S RP ~0.088, 0.15, 0.277! S PS ~0.117, 0.202, 0.351!, S T ~0.72, 0.117, 0.203! By using Formulas 10, 11, and 12, V~S EO S RM! 0.941, V~S EO S PM! 0.436, V~S EO S RP! V~S EO S PS! 0.634, V~S EO S T! 1, V~S RM S EO! 1 V~S RM S PM! 0.514, V~S RM S RP! 0.982, V~S RM S PS! V~S RM S T! 1, V~S PM S EO! 1, V~S PM S RM! 1 V~S PM S RP! 1, V~S PM S PS! 1, V~S PM S T! 1

13 PROJECT RISK EVALUATION 571 Figure 3. The hierarchy of the AHP risk evaluation model.

14 572 TÜYSÜZ AND KAHRAMAN Table III. IT project risk factors. Risk group Risk factor Explanation Environment and Business or corporate environment unstability ~U! A climate of change in the business and organizational environment that creates instability in the project. Ownership ~EO! Change in the ownership ~CO! New owners set new business direction that causes mismatch between corporate needs and project objectives. Lack of top management commitment and support ~MS! This includes oversight by executives and visibility of their commitment, committing required, resources, changing policies as needed. Failure to get project plan approval from all parties ~PA! Agreement of all parties on the project plan. Lack of sharing responsibility ~SR! Sharing of the responsibility and risk between the shareholders. Relationship Failure to manage end-user expectations ~EE! Expectations determine the actual success or failure of a project. Expectations mismatched with what is Management deliverable too high or too low cause problems. In order to avoid failure, expectations must be managed. ~RM! Lack of adequate user involvement ~UI! Active participation of functional users in the project team and their commitment of their deliverables responsibilities. Managing multiple relationships with stakeholders ~MR! Management of relationships between all parties in order to prevent confusion. Failure to meet stakeholders expectations ~SE! Not to be able satisfy all the parties. Project Lack of effective management skills ~EMS! Project teams are performed and the project manager does not have the power or skills to succeed. The administration Management of project must be properly addressed. ~PM! Lack of effective project management methodology ~PMM! The team employs no change control, no project planning, or other necessary skills or processes. Not managing change properly ~CHM! The process of managing change so that scope and budget are controlled. Extent of changes in the project ~ECH! The size and effect of changes on project objectives and deliverables. Unclear project scope and objectives ~SO! Not clearly identifying project scope and objectives. Resources and Resource shortage ~RS! Not having enough resources to perform the project. Planning ~RP! No planning or inadequate planning ~NOP! Attitude that planning is unimportant or impractical. Misunderstanding the requirements ~MIS! Not thoroughly defining the requirements of the new system before starting, consequently not understanding the true work effort, skill sets, and technology required to complete the project. Unrealistic deadlines ~UD! Presence of unrealistic deadlines or functionality expectations in given time period. Underfunding the development ~UF! Setting the budget for a development effort before the scope and requirements are defined or without regard to them. Personnel and Project team expertise ~EX! Lack of necessary experience and expertise. Staffing ~PS! Dependence on a few key people ~KP! Presence of few competent people vital for the project. Poor team relationships ~PTR! Strains existing in the team due to such things as burnout or conflicting egos and attitudes. Lack of available skilled personnel ~SP! People with the right skills are not available when you need them. Project manager s experience ~PME! Project manager s familarity with the such applications. Technology ~T! Technical complexity ~TC! The technological complexity or technical difficulty of the project. Newness of technology ~NT! Development of a brand new technology or the usage of new technology or method Need for new hardware and software ~HS! Requirement for the use of new hardware and software in project. Project size ~SZ! Size and scope of the project.

15 Table IV. PROJECT RISK EVALUATION 573 The fuzzy evaluation matrix with respect to the goal. EO RM PM EO ~1, 1, 1! ~0.852, 1.246, 1.657! ~0.386, 0.478, 0.629! RM ~0.603, 0.803, 1.173! ~1, 1, 1! ~0.57, 0.803, 1.108! PM ~1.589, 2.091, 2.593! ~0.903, 1.246, 1.755! ~1, 1, 1! RP ~0.833, 1.084, 1.507! ~0.740, 1, 2.036! ~0.467, 0.603, 0.803! PS ~1.431, 1.949, 2.460! ~1.084, 1.461, 2.371! ~0.437, 0.561, 0.784! T ~0.631, 0.850, 1.149! ~0.532, 0.715, 1.035! ~0.427, 0.549, 0.784! RP PS T EO ~0.663, 0.922, 1.201! ~0.407, 0.513, 0.699! ~0.786, 1.176, 1.585! RM ~0.64, 1, 1.351! ~0.484, 0.684, 0.922! ~0.967, 1.398, 1.879! PM ~1.246, 1.657, 2.141! ~1.275, 1.783, 2.287! ~1.275, 1.821, 2.341! RP ~1, 1, 1! ~0.66, 0.956, 1.303! ~0.786, 1.176, 1.585! PS ~0.768, 1.046, 1.516! ~1, 1, 1! ~1.246, 1.797, 2.322! T ~0.631, 0.85, 1.272! ~0.431, 0.556, 0.803! ~1, 1, 1! Table V. Evaluation of the subattributes with respect to environment and ownership. U CO MS PA SR U ~1, 1, 1! ~0.924, 1.32, 1.733! ~0.715, 1.013, 1.362! ~0.786, 1.176, 1.585! ~0.742, 1.149, 1.565! CO ~0.577, 0.758, 1.082! ~1, 1, 1! ~0.574, 0.813, 1.134! ~0.684, 1.108, 1.496! ~0.786, 1.021, 1.38! MS ~0.734, 0.987, 1.398! ~0.882, 1.046, 1.741! ~1, 1, 1! ~1.176, 1.585, 2.064! ~0.725, 1.134, 1.552! PA ~0.631, 0.85, 1.272! ~0.668, 1.084, 1.679! ~0.367, 0.631, 0.85! ~1, 1, 1! ~0.623, 1.059, 1.532! SR ~0.639, 0.871, 1.348! ~0.725, 0.979, 1.272! ~0.644, 0.882, 1.38! ~0.653, 0.944, 1.605! ~1, 1, 1! The weight vector from Table V is calculated as W EO ~0.22, 0.18, 0.22, 0.18, 0.20! T. Table VI. Evaluation of the subattributes with respect to relationship management. EE UI MR SE EE ~1, 1, 1! ~1.661, 2.169, 2.674! ~1.176, 1.552, 1.963! ~1.11, 1.516, 1.991! UI ~0.374, 0.53, 0.708! ~1, 1, 1! ~0.833, 1.068, 1.431! ~0.871, 1.227, 1.683! MR ~0.509, 0.74, 0.85! ~0.699, 0.936, 1.201! ~1, 1, 1! ~0.623, 1.149, 1.661! SE ~0.502, 0.66, 0.901! ~0.594, 0.815, 1.149! ~0.602, 0.871, 1.605! ~1, 1, 1! The weight vector from Table VI is calculated as W RM ~0.43, 0.20, 0.20, 0.17! T. Table VII. Evaluation of the subattributes with respect to project management. EMS PMM CHM ECH SO EMS ~1, 1, 1! ~0.66, 0.922, 1.285! ~0.752, 1.496, 1,922! ~0.786, 1.176, 1.585! ~0.561, 0.833, 1.11! PMM ~0.778, 1.084, 1.516! ~1, 1, 1! ~0.891, 1.431, 1.949! ~0.944, 1.351, 1.821! ~0.574, 0.956, 1.334! CHM ~0.52, 0.668, 1.33! ~0.513, 0.669, 1.122! ~1, 1, 1! ~0.549, 0.871, 1.246! ~0.574, 0.813, 1.134! ECH ~0.631, 0.85, 1.272! ~0.549, 0.74, 1.059! ~0.803, 1.149, 1.821! ~1, 1, 1! ~0.582, 1.149, 1.511! SO ~0.901, 1.201, 1.783! ~0.75, 1.046, 1.741! ~0.882, 1.23, 1.741! ~0.662, 0.871, 1.719! ~1, 1, 1! The weight vector from Table VII is calculated as W RM ~0.22, 0.23, 0.16, 0.19, 0.20! T.

16 574 TÜYSÜZ AND KAHRAMAN Table VIII. Evaluation of the subattributes with respect to resources and planning. RS NOP MRQ UD UF RS ~1, 1, 1! ~0.776, 1.035, 1.443! ~0.631, 0.873, 1.185! ~1.046, 1.38, 1.838! ~0.53, 0.903, 1.275! NOP ~0.693, 0.967, 1.289! ~1, 1, 1! ~0.693, 0.91, 1.217! ~1.351, 1.864, 2.371! ~0.631, 0.964, 1.351! MRQ ~0.844, 1.149, 1.585! ~0.822, 1.099, 1.443! ~1, 1, 1! ~1.797, 2.322, 2.837! ~1.23, 1.783, 2.309! UD ~0.625, 0.725, 0.956! ~0.422, 0.536, 0.74! ~0.352, 0.431, 0.556! ~1, 1, 1! ~0.506, 0.668, 0.922! UF ~0.784, 1.108, 1.888! ~0.74, 1.037, 1.585! ~0.433, 0.561, 0.813! ~1.084, 1.496, 1.974! ~1, 1, 1! The weight vector from Table VIII is calculated as W PM ~0.19, 0.23, 0.31, 0.06, 0.21! T. Table IX. Evaluation of the subattributes with respect to personnel and staffing. EX KP PTR SP PME EX ~1, 1, 1! ~1.589, 2.091, 2.593! ~0.758, 1.108, 1.552! ~0.596, 0.85, 1.201! ~0.549, 0.803, 1.134! KP ~0.386, 0.478, 0.629! ~1, 1, 1! ~0.776, 1.035, 1.443! ~0.631, 0.964, 1.351! ~0.441, 0.668, 0.944! PTR ~0.644, 0.903, 1.32! ~0.693, 0.967, 1.289! ~1, 1, 1! ~0.742, 0.977, 1.33! ~0.407, 0.693, 0.91! SP ~0.833, 1.176, 1.679! ~0.74, 1.037, 1.585! ~0.752, 1.024, 1.348! ~1, 1, 1! ~0.699, 0.922, 1.246! PME ~0.882, 1.201, 1.821! ~1.059, 1.496, 2.268! ~1.099, 1.443, 1.974! ~0.803, 1.084, 1.431! ~1, 1, 1! The weight vector from Table IX is calculated as W PS ~0.23, 0.15, 0.17, 0.20, 0.25! T. Table X. Evaluation of the subattributes with respect to technology. TC NT HS SZ TC ~1, 1, 1! ~0.684, 1.084, 1.496! ~0.457, 0.758, 1.084! ~0.596, 0.85, 1.201! NT ~0.668, 0.922, 1.461! ~1, 1, 1! ~0.684, 1.084, 1.496! ~0.822, 1.246, 1.719! HS ~0.992, 1.32, 2.187! ~0.668, 0.922, 1.461! ~1, 1, 1! ~0.776, 1.217, 1.697! SZ ~0.833, 1.176, 1.679! ~0.616, 0.871, 1.398! ~0.589, 0.822, 1.289! ~1, 1, 1! The weight vector from Table X is calculated as W T ~0.23, 0.26, 0.27, 0.24! T. Table XI. Evaluation of riskiness with respect to business or corporate environment unstability. High risk Moderate risk Low risk High risk ~1, 1, 1! ~0.596, 0.977, 1.33! ~0.944, 1.272, 1.719! Moderate risk ~0.693, 0.967, 1.289! ~1, 1, 1! ~0.944, 1.38, 1.864! Low risk ~0.582, 0.786, 1.059! ~0.536, 0.725, 1.059! ~1, 1, 1! The weight vector from Table XI is calculated as W U ~0.37, 0.38, 0.25! T, W CO ~0.34, 0.36, 0.30! T, W MS ~0.73, 0.27, 0.00! T, W PA ~0.38, 0.44, 0.18! T, W SR ~0.51, 0.39, 0.10! T.

17 PROJECT RISK EVALUATION 575 Table XII. Evaluation of riskiness with respect to failure to manage end-user expectations. High risk Moderate risk Low risk High risk ~1, 1, 1! ~1.783, 2.287, 2.789! ~2.287, 2.789, 3.291! Moderate risk ~0.359, 0.437, 0.561! ~1, 1, 1! ~1.334, 1.741, 2.226! Low risk ~0.304, 0.359, 0.437! ~0.449, 0.574, 0.75! ~1, 1, 1! The weight vector from Table XII is calculated as W EE ~0.73, 0.27, 0.00! T, W UI ~0.61, 0.39, 0.00! T, WMR ~0.49, 0.41, 0.10! T, W SE ~0.60, 0.40, 0.00! T. Table XIII. Evaluation of riskiness with respect to lack of effective management skills. High risk Moderate risk Low risk High risk ~1, 1, 1! ~1.532, 2.048, 2.557! ~1.864, 2.371, 2.876! Moderate risk ~0.391, 0.488, 0.653! ~1, 1, 1! ~1.477, 2.048, 2.582! Low risk ~0.348, 0.422, 0.536! ~0.387, 0.488, 0.677! ~1, 1, 1! The weight vector from Table XIII is calculated as W EMS ~0.70, 0.30, 0.00! T, W PMM ~0.69, 0.31, 0.00! T, W CHM ~0.51, 0.36, 0.13! T, W ECH ~0.63, 0.37, 0.00! T, W SO ~0.60, 0.40, 0.00! T. Table XIV. Evaluation of riskiness with respect to resource shortage. High risk Moderate risk Low risk High risk ~1, 1, 1! ~1.037, 1.413, 1.821! ~1.413, 1.821, 2.309! Moderate risk ~0.549, 0.708, 0.964! ~1, 1, 1! ~1.431, 1.949, 2.46! Low risk ~0.433, 0.549, 0.708! ~0.407, 0.513, 0.699! ~1, 1, 1! The weight vector from Table XIV is calculated as W RS ~0.55, 0.44, 0.01! T, W NOP ~0.73, 0.27, 0.00! T, W MIS ~0.75, 0.25, 0.00! T, W UD ~0.57, 0.36, 0.07! T, W UF ~0.43, 0.14, 0.43! T. Table XV. Evaluation of riskiness with respect to project team expertise. High risk Moderate risk Low risk High risk ~1, 1, 1! ~1.589, 2.091, 2.593! ~2.064, 2.572, 3.077! Moderate risk ~0.386, 0.478, 0.629! ~1, 1, 1! ~1.84, 2.352, 2.86! Low risk ~0.325, 0.389, 0.484! ~0.35, 0.425, 0.543! ~1, 1, 1! The weight vector from Table XV is calculated as W EX ~0.77, 0.23, 0.00! T, W KP ~0.57, 0.43, 0.00! T, W PTR ~0.60, 0.40, 0.00! T, W SP ~0.68, 0.32, 0.00! T, W PME ~0.87, 0.13, 0.00! T. Table XVI. Evaluation of riskiness with respect to technical complexity. High risk Moderate risk Low risk High risk ~1, 1, 1! ~1.176, 1.585, 2.064! ~1.351, 1.821, 2.309! Moderate risk ~0.484, 0.631, 0.85! ~1, 1, 1! ~1.864, 2.371, 2.876! Low risk ~0.433, 0.549, 0.708! ~0.348, 0.422, 0.536! ~1, 1, 1! The weight vector from Table XVI is calculated as W TC ~0.54, 0.46, 0.00! T, W NT ~0.54, 0.46, 0.00! T, W HS ~0.48, 0.48, 0.04! T, W SZ ~0.49, 0.51, 0.00! T.

18 576 TÜYSÜZ AND KAHRAMAN V~S RP S EO! 1, V~S RP S RM! 1, V~S RP S PM! V~S RP S PS! 0.755, V~S RP S T! 1, V~S PS S EO! 1 V~S PS S RM! 1, V~S PS S PM! 0.819, V~S PS S RP! 1 V~S PS S T! 1, V~S T S EO! 0.854, V~S T S RM! V~S T S PM! 0.314, V~S T S RP! 0.777, V~S T S PS! are obtained. By applying Formulas 10 and 11, d ' ~EO! V~S EO S RM, S PM, S RP, S PS, S T! min~0.941, 0.436, 0.921, 0.634, 1! d ' ~RM! V~S RM S EO, S PM, S RP, S PS, S T! min~1, 0.514, 0.982, 0.707, 1! d ' ~PM! V~S PM S EO, S RM, S RP, S PS, S T! min~1, 1, 1, 1, 1! 1 d ' ~RP! V~S RP S EO, S RM, S PM, S PS, S T! min~1, 1, 0.578, 0.755, 1! d ' ~PS! V~S PS S EO, S RM, S PM, S RP, S T! min~1, 1, 0.819, 1, 1! d ' ~T! V~S T S EO, S RM, S PM, S RP, S PS! min~0.854, 0.799, 0.314, 0.777, 0.503! Weight vector W ' ~0.436, 0.514, 1, 0.578, 0.819, 0.314! T. Via normalization the normalized weight vector with respect to goal W G is obtained as follows: W G ~0.12, 0.14, 0.27, 0.16, 0.22, 0.09! T. For the rest of the calculations one example of a matrix for each subattribute is given in Tables XI XVI. The weight vectors of the others are directly given. By applying Leung and Cao s 38 fuzzy consistency approach, the consistency of the pairwise judgment matrices were examined and it was determined that all the matrices were consistent. The result of the risk evaluation, which is given in Table XVII, for the IT project of Softek Co. is obtained as 0.62 for high risk, 0.33 for moderate risk, and 0.05 for low risk. The top management of the firm should reject investing in this project because it is a high risk project.

19 PROJECT RISK EVALUATION 577 Table XVII. Summary of combinations of priority weights. Subattributes of environment and ownership U CO MS PA SR Risk level s weight Weight Risk levels High risk Moderate risk Low risk Subattributes of relationship management EE UI MR SE Risk level s weight Weight Risk levels High risk Moderate risk Low risk Subattributes of project management EMS PMM CHM ECH SO Risk level s weight Weight Risk levels High risk Moderate risk Low risk Subattributes of resources and planning RS NOP MIS UD UF Risk level s weight Weight Risk levels High risk Moderate risk Low risk Subattributes of personnel and staffing EX KP PTR SP PME Risk level s weight Weight Risk levels High risk Moderate risk Low risk Subattributes of technology TC NT HS SZ Risk level s weight Weight Risk levels High risk Moderate risk Low risk ~continued!

20 578 TÜYSÜZ AND KAHRAMAN Table XVII. Continued. Main attributes of the goal EO RM PM RP PS T Risk level s weight Weight Risk levels High risk Moderate risk Low risk CONCLUSION Project risk evaluation should be performed by using one of the multi-attribute evaluation methods because of the multidimensional nature of risks. This requires a method that allows the use of decision makers vague judgments in the pairwise comparison of attributes. The fuzzy AHP method meets this requirement. There are many fuzzy AHP methods developed in the literature. In this study, Chang s 28 extent analysis method on fuzzy AHP is selected and applied to the risk evaluation of an information technology project. For further research, applying other fuzzy AHP methods to project risk evaluation and then comparing their results with the results of this study are recommended. In addition to this, some of the other multiattribute evaluation methods such as TOPSIS, DEA, multi-attribute utility analysis, outranking methods ~PROMETHEE, ELECTRE, ORESTE! of which fuzzy forms have been developed, can be used for comparing the results. References 1. Elkington P, Smallman C. Managing project risks: A case study from the utilities sector. Int J Proj Manag 2002;20: Raz T, Shenhar AJ, Dvir D. Risk management, project success, and technological uncertainty. R&D Manag 2002;32: Project Management Institute. A guide to project management body of knowledge. Newton Square, PA: Project Management Institute; Cooper D, Chapman C. Risk analysis for large projects. New York: John Wiley and Sons; Chapman C, Ward S. Project risk management: Processes, techniques and insights. New York: John Wiley and Sons; Chapman C. Project risk analysis and management PRAM, the generic process. Int J Proj Manag 1997;15: Hillson D. Using a risk breakdown structure in project management. J Facil Manag 2003;2: Miller R, Lessard D. Understanding and managing risks in large engineering projects. Int J Proj Manag 2001;19: Mustafa MA, Al-Bahar JF. Project risk assessment using the analytic hierarchy process. IEEE Trans Eng Manag 1991;38: Kerzner H. Project management: A systems approach to planning, scheduling, and controlling, 7th ed. New York: John Wiley and Sons; 2001.

21 PROJECT RISK EVALUATION Hahn ED. Decision making with uncertain judgments: A stochastic formulation of the analytic hierarchy process. Decis Sci 2003;34: Dey PK. Project risk management: A combined analytic hieararchy process and decision tree approach. Cost Eng 2002;44: Millet I, Wedley WC. Modelling risk and uncertainty with the analytic hieararchy process. J Multi-Criteria Decis Anal 2002;11: Zadeh L. Fuzzy sets. Inform Control 1965;8: Cheng CH, Mon DL. Evaluating weapon systems by AHP based on fuzzy scale. Fuzzy Set Syst 1994;63: Zimmermann H-J. Fuzzy set theory and its applications, 2nd revised ed. Boston, MA: Kluwer Academic Publishers; Kuchta D. Use of fuzzy numbers in project risk ~criticality! assessment. Int J Proj Manag 2001;19: Bonvicini S, Leonelli P, Spadoni G. Risk analysis of hazardous materials transportation: Evaluating uncertainty by means of fuzzy logic. J Hazard Mater 1998;62: Carr V, Tah JHV. A fuzzy approach to construction risk assessment and analysis: Construction project risk management system. Adv Eng Software 2001;32: Cho H-N, Choi H-H, Kim Y-B. A risk assessment methodology for incorporating uncertainties using fuzzy concepts. Reliab Eng Syst Saf 2002;78: Huang C. An application of calculated fuzzy risk. Inform Sci 2002;142: Huang C, Moraga C. A fuzzy risk model and its matrix algorithm. Int J Uncertainty Fuzziness Knowl Base Syst 2002;10: Lee HM, Lee SY, Lee TY, Chen JJ. A new algorithm for applying fuzzy set theory to evaluate the rate of aggregative risk in software development. Inform Sci 2003;153: Chen SJ, Chen SM. Fuzzy risk analysis based on similarity measures of generalized fuzzy numbers. IEEE Trans Fuzzy Syst 2003;11: van Laarhoven PJM, Pedrycz W. A fuzzy extension of Saaty s priority theory. Fuzzy Set Syst 1983;11: Buckley JJ. Fuzzy hierarchical analysis. Fuzzy Set Syst 1985;17: Stam A, Minghe S, Haines M. Artificial neural network representations for hierarchical preference structures. Comput Oper Res 1996;23: Chang D-Y. Applications of the extent analysis method on fuzzy AHP. Eur J Oper Res 1996;95: Cheng C-H. Evaluating naval tactical missile systems by fuzzy AHP based on the grade value of membership function. Eur J Oper Res 1997;96: Weck M, Klocke F, Schell HR, Rüenauver E. Evaluating alternative production cycles using the extended fuzzy AHP method. Eur J Oper Res 1997;100: Kahraman C, Ulukan Z, Tolga E. A fuzzy weighted evaluation method using objective and subjective measures. In: Proc Int ICSC Symp on Engineering of Intelligent Systems ~EIS 98!, Vol. 1, University of La Laguna Tenerife, Spain; pp Deng H. Multicriteria analysis with fuzzy pairwise comparison. Int J Approx Reason 1999;21: Lee M, Pham H, Zhang X. A methodology for priority setting with application to software development process. Eur J Oper Res 1999;118: Cheng C-H, Yang K-L, Hwang C-L. Evaluating attack helicopters by AHP based on linguistic variable weight. Eur J Oper Res 1999;116: Zhu K-J, Jing Y, Chang D-Y. A discussion of extent analysis method and applications of fuzzy AHP. Eur J Oper Res 1999;116: Chan FTS, Chan MH, Tang NKH. Evaluation methodologies for technology selection. J Mater Process Technol 2000;107: Chan FTS, Jiang B, Tang NKH. The development of intelligent decision support tools to aid the design of flexible manufacturing systems. Int J Prod Econ 2000;65: Leung LC, Cao D. On consistency and ranking of alternatives in fuzzy AHP. Eur J Oper Res 2000;124: