Application of AHP in Education Legislation Project. Ying-ying YAO and Yong-sheng GE *

Transcription

1 2017 3rd International Conference on Applied Mechanics and Mechanical Automation (AMMA 2017) ISBN: Application of AHP in Education Legislation Project Ying-ying YAO and Yong-sheng GE * 130 Meilong RD-Xuhui, Shanghai, PRC * Corresponding author Keywords: AHP, Performance evaluation of colleges, Comprehensive evaluation. Abstract. Authorized by Shanghai education committee, we evaluated the education legislation project comprehensively. By adopting AHP (Analytic Hierarchy Process) and consulting experts, we built the evaluating system, achieving the combination of quantitative and qualitative measurement. In the evaluation, we broke through the previous methods like input-output analysis and balanced scorecard, and adopted AHP to quantify the problem. Overcoming the difficulties to observe and measure some activities or achievements of colleges, AHP operates scientifically and effectively. It only requires the comparison between every two factors to get the quantitative result of the index. This bold attempt breaks the limits of the difficulty to judge the qualitative problem, and we hope to spread AHP to more areas where qualitative problems are hard to measure. Introduction To build a platform for Shanghai education legislation, Shanghai education committee decided to implement "base construction for education legislation and service research" project (hereinafter referred to as "education legislation project"). Five universities, FD, HZ, CJ, JT and SS (for universities privacy, we use code for its name) successfully declare this project. To test the periodic results, we graded the construction progress of every base in the middle of the project. Education legislation project focuses on whether the universities can be targeted to carry out the study and research about the related policy, explore the education reform and education legislation, and realize the research achievements. The assessment of the project, actually is the performance assessment of the colleges. So we categorize the assessment here as the performance evaluation of colleges. Literature Review Performance evaluation of colleges, in essence, is examining whether resources get the reasonable utilization and whether achievement of education construction is significant. It shows the effectiveness of the education systems and supervises the allocation of resources. The result of evaluation also can provide some guidance for educational appropriation. Since the evaluation is so valuable, the method applied in it becomes so important. Currently, the widely used methods are input-output analysis and the balanced scorecard. Input-output analysis uses essentially an economic philosophy, which measures the economic efficiency and benefit through input value and output value. Using the input-output analysis method in the performance evaluation of colleges, is discomposing the actual activities and achievements of university education, from various perspectives like goal, aim, and mission of education, and doing the quantitative calculation. Balanced scorecard is an effective tool that measures performance based on quantitative and qualitative analysis, combining strategic management with performance management. It is mainly applied to profit organizations like enterprises. Guided by enterprise strategy, balanced scorecard decomposes goal into specific measurable indicators from four angles: finance, customer, internal operation and learning and growing. Through balanced scorecard, we can combine firm s performance with its strategy organically. But two methods above are not suitable enough in the performance evaluation of colleges. 206

2 Input-output analysis method incorporates an economic concept, which isn t suitable here. 1. University is a non-profit institution and it doesn t pursue short-term, immediate economic benefits like enterprises do. They aim at long-term output, like reputation. 2. Some goals, like reputation and teaching quality, can t be measured precisely by money or other numerical indicators. So the evaluation result may have some bias. Balanced scorecard also has some questions. 1. There is no specific customer base, and won t be changes through learning and growing in the education legislation project. So two angles are useless. 2. As a non-profit organization, university doesn t have a clear, precise and the only objective like firm. 3. Universities have different plans and focuses in developing. We can t use one strategy to evaluate its performance. Considering the deficiencies of the above two methods in this project, we introduce a new method -- Analytic Hierarchy Process (hereinafter referred to as AHP) into the evaluation of education legislation project. AHP (Analytic Hierarchy Process) Proposed by A.L. Saaty, an American operations research scientist, Analytic Hierarchy Process is a decision analysis method combining qualitative and quantitative concept. It is a method of modeling and quantifying the complex decision-making process. In accordance with the general target, subtarget and evaluation criteria, it decomposes decision-making problem into different levels. Then according to different degrees of influence of factors on each level on last adjacent level, a value will be assigned, forming a straight reciprocal matrix. And we use the method of solving matrix characteristic vector to get the weighing factor of each level on last level and eventually obtain the decision results by summarizing the weights of each factor. AHP features in systematizing, mathematizing and modeling the thinking process; it doesn t require too many quantitative data, but the factors involved and their relations should be specific and clear; it applies to decision analysis for complex problems with multiple criteria and goals. This method actually reflects the process of thinking. By comparing every two factors, we can judge their different importance respectively within the same level and between adjacent layers. And reliable thinking can be ensured by consistency test. Currently, AHP is mainly used in supplier selection. Luo (2011) [1] combines AHP and TPPSIS to present an evaluation and selection model of suppliers in multi-level green chain. And in the research of enterprise management, it is quite common to use AHP to determine the specific index weights. Han (2009) [2] applies AHP to make up for pity of few combinations of qualitative and quantitative approaches in current internal control assessments, offering a convenient tool for enterprises and CPA to appraise internal control system. The method is also applied in risk warning system, natural science and so on. As for the education legislation project, applying AHP is a great choice. Based on the particularity of college activities, not all activities can be observed. Even if activities can be observed, their results may not necessarily form quantitative data. This is the problem of evaluation of colleges, which can be perfectly solved by the advantage to quantify metrics of AHP. We don t have to give the absolute value and only need to compare every two specific factors and judge their relative importance to the last adjacent factor, getting the final evaluation result. It s easy to operate and effective because we can ensure the internal logic by consistency test. As for that universities are diverse in developing, AHP can adjust proper indicators and their weights flexibly. In the next part, we use AHP to evaluate comprehensively the performance of the project. Finally, complete content and organizational editing before formatting. Please take note of the following items when proofreading spelling and grammar: The Application of AHP in Education Legislation Project Complying with the requirements of the AHP, we need to decompose the goal of the education legislation project into several factors, and the build the hierarchical structure model. 207

3 Since performance evaluation of public sector follows the "3E" principle, namely economy, efficiency and effect, we believe the indicators selected should also meet the requirement: using sparing resources and small costs to bring greater economic benefits and efficiency. Based on that not all variables in the implementation can be observed, we consider the availability of data. Following these principles, we build the index system in three levels, as shown in table 1. As shown in table I, indicator A reflects project decision. A1 and A2 reflects project decision, indicating the appropriateness to set up the project. Indicator B demonstrates project management. B1, B2, and B3 are the assessment during the implementation of the project, implying its effectiveness. Indicator C reflects the outcome of the project from four perspectives. Table 1. Index system. First-level indicator Second level indicator Third-level indicator A project decision B project management C project outcome A1 project establishment A2 project objective B1 investment management B2 financial management B3 project implementation C1 project output Table 2. Judgment matrix. A11 conforming with the policy A12 rational project approval A13 normative project approval A21 reasonable goal A22 specific indicators B11 implementation of budget B12 rationality of budget B21 usage of fund B22 healthy financial system B23 effective financial supervision B31 healthy management system B32 effective implementation of management system C11 completion rate of plan C12 effective implementation C13 effective base construction C14 update follow-up plan A1 A11 A12 A13 A11 b11 b12 b13 A12 b21 b22 b23 A13 b31 b32 b33 208

4 Aimed at elements in last adjacent level, we compare the relative importance of every two elements and give judgment, forming matrix, namely judgment matrix. Judgment matrix represents the relative importance of some related factors (A11, A12, A13) in this level to an element (A1) in last level, generally in the following form, as shown in table 2. In table 2, bij represents the relative importance of the element in row i to the element in column j, for A1. Such as, b21 indicates the importance of A12 to A11 for A1. If b21 > 1, then A12 is more important than A11 for A1. If b21 < 1, then A12 is less important. If b21 1, A12 is equally important as A11 for A1. In AHP, the key point to judge accurately is to quantify the relative superiority of two elements for element in last adjacent level. In comparison. AHP adopts the 1-9 scale method. If the unit is 1, it means two elements are equally importance. If it exceeds 1, the greater the unit, the element is more important. If the unit is smaller than 1, the element is less important than the other. Then we use AHP to evaluate the performance of five universities (FD, HZ, CJ, JT, SS), including the following three steps. The first step is to construct the judgment matrix of the six indicators that evaluate the performance of the five universities, and determine whether they meet the criteria for acceptable consistency. Overall performance of the project is reflected in six aspects: establishment, objective, investment, finance, implementation, and output. We appraise them from the specific indicators, as table 1 shows. After consulting experts, we think their importance in performance is: output > implementation finance > investment> objective > establishment. Based on the relative importance, we get the judgment matrix in table 3. To ensure the reliable logic of our subjective judgment, we will test consistency for the matrix in table 3. Table 3. Optimal judgment matrix of comprehensive performance. Performance Establishment Objective Investment Finance Implementation Output Establishment 1 1/2 1/3 1/4 1/4 1/5 Objective 2 1 1/2 1/3 1/3 1/4 Investment /2 1/2 1/3 Finance /2 1/3 Implementation /2 Output (1) The first step is to normalize all of the elements in columns in the matrix as formula 1. For each row in judgment matrix b, we calculate them together and make it divided by the number n, then we can get the weight of each index as formula 2. And we can get the consequence as follows. bij b (i,j1,2,3...n) (1) b ij W n j 1 n b ij (i,j1,2,3 n) (2) b W (2) We can obtain the approximate solution of the eigenvector by the multiplication to get AW. Then the maximum characteristic root λ for the judgment matrix can be calculated. Using λ, we can calculate the consistency value of the optimal matrix CI (consistency indicator) and CR 209

5 (consistency ratio). The smaller CI and CR in the calculation results, the better. Small value means the logic in the judgment matrix is close to complete consistency. Otherwise, matrix has great deviation from complete consistency. When we get the value for CI, with the help of RI from the mean random consistency index, we can get CR. When N6, RI1.24. AW X λ max / n max λ CI n CI CR < 0.10 RI 1.24 The CR we get is smaller than 0.10, indicating that this judgment matrix has acceptable consistency. If CR 0.10, appropriate adjustment of the matrix is required until CR < Therefore, the logic in optimal judgment matrix is consistent. The second step is to build judgment matrix of five universities in six specific evaluation indices and test whether they meet the acceptable consistency condition. The calculating procedure and theory is the same as what we introduced above. Since space is limited, we omit six judgment matrixes and consistency tests, only giving the consequence. CR value for each index is respectively less than 0.10, so we can accept the judgment. The third step is to summarize the previous data we calculated, and give the ranking values for five universities. The optimal selection is the university with the maximum value. Table 4 is where displays the result. The first line is the criterion layer. Below criteria layer, the first column is score for indicator establishment of five universities, and the second column is score for objective and so on. The final score for each university is listed in the last column. We can see that, SS gets the highest score, becoming the optimal selection in performance. Conclusion Facing the task entrusted by the Shanghai education committee, we break through previous methods like balanced scorecard and input and output analysis method, and adopt AHP to judge qualitative problem. It s a bold attempt. Based on the difficulty in observing and quantifying certain activities and results in universities, we build the evaluating system of specific indices with the guidance of experts and compare every two factors between universities, quantifying a qualitative problem. The method is feasible, scientific and effective. 210

6 Table 4. Sorting table. Establishment Objective Investment Finance Implementation Output Result Criterion layer FD HZ CJ JT SS AHP can also be applied to other areas where it s difficult to quantify indicators. Using this method, we don t need to judge the absolute value in each level. We just need to compare every two factors and score their importance for last adjacent level. In the operating process, it is easy to realize. And we can ensure the internal consistency and logic through the follow-up consistency test, guaranteeing the scientific and effective characteristics of subjective judgments. We hope this method can be applied more extensively. References [1] Luo Xinxing, Peng Suhua. Supplier judgment and selection based on AHP and TOPSIS in green supply chain. Soft Science, 2011, 25(2): pp [2] Han Chuanmo, Wang Shiguo. Comprehensive evaluation of internal control based on AHP. Accounting Research, 2009(4): pp