Benchmarking facility management: applying analytic hierarchy process

Transcription

1 Benchmarking facility management: applying analytic hierarchy process John D. Gilleard and Philip Wong Yat-lung The authors John D. Gilleard is Associate Professor and Philip Wong Yat-lung is Research Student, both in the Department of Building Services Engineering, The Hong Kong Polytechnic, Hung Hom, Kowloon, Hong Kong SAR, China. Keywords Analytic hierarchy process, Benchmarking,, Performance measure Abstract This paper illustrates the theoretical framework of applying the analytic hierarchy process (AHP) when benchmarking facility management service provider performance. A case study is presented to demonstrate the structure and organization of the model. The case study also illustrates how AHP is particularly effective for handling performance measures that involves multi-attribute multivariate qualitative and quantitative data. The paper concludes by identifying three critical features: ranking criteria, establishing consensual data input and applying sensitivity testing. Electronic access The Emerald Research Register for this journal is available at The current issue and full text archive of this journal is available at Volume 22. Number 1/ pp # Emerald Group Publishing Limited. ISSN DOI / Introduction Facility management (FM) benchmarking issues are typically driven by financial, organizational, change management, and customer-related needs. These may be either internally focussed or externally driven. When choosing the latter an organization either makes direct comparisons with industry competitors, or they benchmark other organizations outside of their immediate domain. Typical facilityoriented benchmarking indicators are flexibility, effective use of space, management of maintenance, provision for a safe environment and value for money. Note however that flexibility - a business driven imperative - implies accommodating change through shrinkage or growth, a natural consequence of today s dynamic marketplace. On the other hand, spatial considerations frequently focus on static issues such as space standards, occupancy density and churn - typical facility-related benchmarks. These issues may ignore business driven imperatives such as agility, the need to accommodate rapid reconfiguration, the desire to use of space for multiple purposes and, increasingly, simply to use less space. This, in turn, leads to a dichotomy, i.e. should FM benchmarking focus on business performance or facility-related issues? FM benchmarking may be broken down into a number of distinct elements. For example Anderson and Pettersen (1996) identify five distinct phases: (1) planning; (2) searching; (3) observation; (4) analysis; and (5) adaptation. Planning and analysis taken together account for ~70 percent of the time typically consumed during the whole benchmarking process. These two phases are normally judged to be critical to success. During the planning phase facility managers need to select what needs to be measured based on the client s expectations and suggestions from senior management. Facility managers also need to form a benchmarking The authors would like to acknowledge funding for the work from The Hong Kong Polytechnic University, research grant G

2 team within the constraints of time and resources available. Only then can they establish suitable performance measures. The planning phase is also important to the whole FM benchmarking process because this phase largely decides what data to collect. Atkin and Brooks (2000) similarly identify separate benchmarking steps: (1) identify the subject of the exercise; (2) decide what to measure; (3) identify who to benchmark both within your sector and outside; (4) collect information and data; (5) analyse findings and determine gap; (6) set goals for improvement; (7) implement new order; (8) monitor the process of improvement. Applied together, Anderson and Pettersen s (1996) five phases and Atkin and Brooks (2000) eight steps would appear to offer a straightforward, if slightly overlapping, set of benchmarking procedures. Both highlight what to benchmark as critical, echoing the truism if it can be measured it can be benchmarked. However, conventional facility management indicators tend to focus on relatively simple common denominators, e.g. cost per unit area/workstation/worker. These are benchmarks that are easily measurable. In addition, many organizations routinely collect benchmarking data on energy and water consumption, itemized cleaning and security costs as well as standardized maintenance cost information. These stand alone metrics are useful when making direct comparisons on a process-by-process basis. Unfortunately, simply applying cross referencing, seeking lowest cost solutions puts tremendous pressure on those FM teams that value customer driven issues such as the delivery of quality, timely services. It also fails to take into account how an organization performs at a strategic level, whether from the worker or workplace perspective. To overcome these difficulties Hinks and McNay (1999) suggest the application of a management by variance tool. The tool identifies business and facility key performance indicators (KPI), helping to create a rank order among the benchmarking criteria. This ability to rank issues would appear to be fundamental 20 if benchmarking is truly to deliver quality improvements. In addition, Hinks and McNay (1999) emphasize the need to measure performance gaps between service delivery and customer satisfaction. Hence Hinks and McNay (1999) stress the need to rank benchmark criterion, linking these to performance and service in such a way that their overall influence may be evaluated against business driven imperatives. However, in practice facility managers find great difficulty in trying to assess significance and relationships between two or more benchmarking metrics. In addition, benchmarking problems may be exacerbated by unorganized, non-prioritized information. Organizations can also suffer from having a surfeit of benchmarking data, magnified by continuing IT trends in remote access monitoring of premises that can quickly lead to data grid lock. In an attempt to overcome some of these intrinsic benchmarking problems a comprehensive yet flexible approach is suggested adopting the decision tool Analytic Hierarchy Process, or AHP. Applying AHP to FM benchmarking The AHP is a powerful and flexible decisionmaking process developed to set priorities and make the best decision when both qualitative and quantitative aspects of a decision need to be considered. By reducing complex decisions to a series of pair-wise comparisons, then synthesizing the results, AHP not only helps decision-makers arrive at the best decision, but also provides a clear rationale for the decision, (Expert Choice, 2002). AHP was designed in the 1970s by Saaty (1980) to help decision-makers structure complex problems into smaller parts, proceeding from the primary goal to objectives to sub-objectives through to alternative course of action. Decision-makers then make simple pair-wise comparison judgments throughout the hierarchy to arrive at overall priorities for the alternatives. The decision problem may involve social, political, technical, and economic factors. The AHP helps people cope with the intuitive, the rational and the irrational, and with risk and uncertainty in complex settings. It can be used to: predict

3 likely outcomes, plan projected and desired futures, facilitate group decision-making, exercise control over changes in the decision-making system, allocate resources, select alternatives, do cost/benefit comparisons, evaluate employees and allocate wage increases (Expert Choice, 2002). AHP has been applied in a wide range of applications within the construction sector, e.g. architecture (Saaty and Beltran, 1982); facility location decisions (Yang and Lee, 1997); construction management (Fong and Choi, 2000); construction procurement (Wong and Chan, 2003). Case study The following example illustrates how critical the analysis stage is to benchmarking success: Shengwei, Director of Facility Management Services at Guangxi Property Development Company, has a problem. The company owns, manages and leases superior grade office and retail developments in Hong Kong. Guangxi s in-house FM team is small, providing an intelligent client role within the firm. Once a development project has been completed Guangxi generally forms a total facility management (TFM) contract from among Hong Kong s growing number of FM service providers (SP). The company to date has shown no partiality in selecting its SP, nor has it attempted to consolidate these TFM contracts. A recently completed customer satisfaction survey among all tenants across the company s portfolio has indicated that the majority of tenants are well satisfied with their respective SP. However, the company is concerned about the survey results from one of its flagship projects. These results indicate a serious breakdown between tenant expectations and service level provision. Subsequently Shengwei has issued a warning to the SP, instructing them to improve. However, he has not issued any clear guidance on how this might be achieved. Shengwei s instruction might, at first glance, act as a good motivational directive. However, since no clear guidelines or suggestions on how the deficient SP team can get better have been given it seems unlikely that significant improvements will ensue. What, therefore, should Shengwei do? At a practical level Shengwei might undertake a comparative study between the weaker SP and one judged to be superior. Comparisons might include staff training procedures, organizational structure, financial performance, complaints, etc. But how might Shengwei prioritize this information, and 21 how is he best able to identify superior work methods and practices? Shengwei would also like to involve all parties - the in-house FM team, the SP staff and the tenants - when measuring performance but is unsure how to proceed. Lastly, given that the original customer satisfaction survey contained qualitative questions only, Shengwei would like to incorporate hard, quantitative data but again he is uncertain how best to build a suitable benchmarking model. Restructuring the problem As a first step Shengwei has decided to restructure the question to set the primary goal, establish benchmarking criteria/sub-criteria and solution alternatives (see Figure 1). Aggregating each criterion score or measure should identify which FM service provider team is superior. For example, a standard approach for aggregating data at a team level is to conduct a tournament-ranking scheme using the original customer survey data. The first step is to develop a matrix of wins, i.e. a matrix W = (w ij ) where each entry, wij, represents the performance outcome between participant (i) and participant (j). In our case this would be an assessment between the two or more SP. Winning implies superior service. (The technique can be applied to more than two SP and the number of wins can vary depending on the number of customer surveys undertaken.) If the participants have never competed against each other the entries are left blank and treated as missing data. Where participant (i) outperforms participant (j) for all of their common processes a zero is entered in cell w ij, and the number of common processes entered into w ji. The second step is to develop a Figure 1 The AHP structure

4 matrix of wins-to-losses where each entry is replaced by w ij /w ji. It is unlikely, though, that any one team would outperform in all aspects. The notion of winning between two SP teams, although superficially attractive, is not tenable. Customer service comparative data is generally qualitative is hence unsuited to expressions of winning and losing. However, if we adopt a bounded scale, to use Saaty s (1996) expression, each entry on the matrix may be represented by the expression e = ln( ) (w ij - w ji /w ij - w ji ), where e represents the bounds of how much better one organization is over another (Frei and Harker, 1998). To apply AHP, the general procedures require the facility manager initially to decompose the problem into a hierarchy, representing facility management key performance indicators (KPI). These have been identified in Shengwei s example as: (1) financial performance; (2) productivity; (3) project performance; (4) equipment availability; (5) compliance; (6) complaint and accident frequency; and lastly (7) customer satisfaction. Sub-goals have also been assigned to each major KPI. Applying Saaty s (1980) expert choice, four essential steps are undertaken. These are: (1) Determine the rank order for each criterion. (2) Establish pair-wise scale values between all major and sub-criteria. (3) Determine the eigenvector for the scaled values. (4) Check data for consistency. Table I identifies the standard scale for making these comparisons. Pair-wise comparisons are also made between each two pair-element and at Table I Standard scale of pair-wise comparison for AHP Rating Description 1 Equally preferred 3 Moderately preferred 5 Strongly preferred 7 Very strongly preferred 9 Extremely strongly preferred 22 each of the seven main levels previously identified (see Table II). Similarly pair-wise comparisons are made among the sub-criteria, e.g. actual operations cost vs budget, FM management staff per sq ft net, FM staff ad hoc orders, Security systems downtime total hours and Number of accidents recorded by helpdesk per occupant, etc. Finally, pair-wise comparisons are made among the two SP with respect to all individual criteria at the lowest level. Subsequently, we obtain one 7 7; four 3 3 and two 2 2 matrices among criteria and matrices among the SP with respect to each criterion. Each of these is normalized. The value in each matrix cell/row expresses the corresponding weight among the comparable criteria. The score for each benchmarked FMO is subsequently obtained by multiplying the weight of each criterion among themselves to the weight of each SP with respect to each criterion. Finally, summing up the products for each SP (see Figure 2). Figure 2 identifies the pair-wise values set by the facility manager for each of the main and sub-criteria. In this case study example, financial performance (L:0.320) is the most significant benchmark with complaint and accident frequency (L:0.054) the least. Typically the broad strategic goals are established through consensus among the FM team. Saaty (1999) proved that, in the application of AHP, two issues in group decision-making are paramount, i.e. they help to aggregate individual judgments, and second, provide a means by which consensual group decisions can be furnished. In our example SP, team A is judged overall to be superior to team B (see Figure 3). The overall numerical ratings for the two teams are and respectively. However, it should be noted that team A was not judged to provide superior service for all criteria. For example, team A project performance (second highest ranked criteria) is judged to be inferior to team B. Team A productivity is also marginally worse than that for team B. Generally team A has a higher score across the majority of criteria when compared to team B. Applying AHP it is possible to readjust the pair-wise values, particularly between the main goals, to test the sensitivity of the data (Figures 4 and 5). In this example, financial performance

5 Table II Pair-wise comparison of the seven criteria Criterion Criterion Financial performance Productivity Project performance Equipment availability Compliance Complaint and accident frequency 7 Customer satisfaction Figure 2 Final AHP ouput indicating FMO contractor team A is ``better than team B has been de-emphasized and is now ranked second behind project performance. This latter criterion was one in which team B provided a superior service. As a result of these changes the two SP teams are now judged to provide an equivalent level of service. The ability to dynamically change pair-wise data is a valuable feature of Saaty s (1999) Expert Choice AHP software tool. One criticism of AHP is the possibility that individual pair-wise criteria may exhibit non-linearity. However, the Expert Choice programme has a built-in function to adjust for non-linearity. Although the adjustment is an approximation, Saaty (1999) argues that homogeneity and clustering can enlarge the scale from 1 to 9, to 1 to infinity (Saaty, 1999). 23 Conclusions and future research This paper has demonstrated the advantage of applying the AHP to facility management benchmarking. AHP allows the facility manager to assimilate all the facts, weigh the pros and cons and communicate the performance benchmarks in a logical manner. Data is ranked according to a broad sense of priority, with each benchmarking criterion assessed against each other. It also allows for quantitative (absolute) and qualitative (relative) data comparisons. Once an initial decision has been made, the AHP benchmarking model may be easily adjusted to provide dynamic sensitivity testing. Given AHP s ability to deal with multiple attributes, the technique is well suited for facility managers who are often subject to pressures from various parties such as customers, employees and senior executive. Facility related objectives, thought to be unchanged under these influences, become less dominant where re-evaluation is required. Gradually, priorities are changed until reshaped decisions emerge. Without an adjustable decision model, auditable trails are lost and facility managers may find it impossible to systematically review the steps and decisions made during the benchmarking process. The difficulty of conducting a proper review increases exponentially as the number of objectives, benchmarking partners and alternatives increase. However, AHP may be said to overconstrain the problem by artificially restricting the decision process. To a certain extent this is true, but no less true than other decision tools that seek optimality by exerting restraint on the number of critical issues to be resolved. For

6 Figure 3 Performance sensitivity graph for the two buildings A and B Figure 4 Dynamic sensitivity analysis ± original scenario example, many facility managers when faced with benchmarking will limit the need to process information to ensure timely completion of the process. Critical KPI will naturally be given greater priority than those deemed to be less important. In the field these 24 decisions are often intuitive (and possibly arbitrary). AHP on the other hand, with its framework for setting (and changing) pair-wise comparisons, applying sensitivity testing and establishing rank order provides the user with considerable flexibility. Nevertheless it is

7 Figure 5 Dynamic sensitivity analysis ± with adjustment on the relative importance of project performance acknowledged that any final judgment will be a relative one rather than an absolute one. This is inevitable with any decision tool that deals with both objective and subjective data, as well as ranked disparate criteria. Nevertheless, within the framework of AHP group decisions are encouraged. Using the ratio scale approach of AHP, it can be shown that because now the individual preferences are cardinal rather than ordinal, it is possible to derive rational group choice satisfying the four conditions of decisiveness, unanimity, independence of irrelevant alternatives and no single individual preferences determining the group order (Saaty, 1999). Finally, it is acknowledge that the paper presents a hypothetical case study, not a complete live case. However, ongoing developmental work using industry data (and industry complexity) is currently being undertaken. The results of the study will be published in an upcoming paper. References Anderson, B. and Pettersen, P. (1996), The Benchmarking Handbook, Chapman and Hall, London, pp Atkin, B. and Brooks, A. (2000), Total Facility Management, Blackwell Science, Oxford, pp Expert Choice (2002), ``Analytic hierarchy process (AHP) decision-making software, available at: Fong, P. and Choi, S. (2000), ``Final contractor selection using the analytical hierarchy process, Construction Management and Economics, Vol. 18, pp Frei, F. and Harker, P. (1998), ``Measuring aggregate process performance using AHP, Financial Institution Center working paper, The Wharton School, University of Pennsylvania, Philadelphia, PA. Hinks, J. and McNay, P. (1999), ``The creation of a management-by-variance tool for facilities management performance assessment,, Vol. 17 No. 1/2, pp Saaty, T.L. (1980), The Analytic Hierarchy Process, McGraw-Hill, New York, NY. Saaty, T.L. (1996), Decision Making with Dependence and Feedback: The Analytic Network Process, RWS Publications, Pittsburgh, PA. Saaty, T.L. (1999), ``The seven pillars of AHP, Proceedings of the ISAHP Conference, Kobe. Saaty, T.L. and Beltran, M.H. (1982), ``The analytic hierarchy process: a new approach to deal with fuzziness in architecture, Architectural Science Review, Vol. 25, pp Wong, Y. and Chan, T.S. (2003), ``Application of analytic hierarchy process in construction procurement, Proceedings of the CIB Student Chapters International Symposium ± Innovation in Construction and Real Estate, The Hong Kong Polytechnic University, Hong Kong. Yang, J. and Lee, H. (1997), ``An AHP decision model for facility location selection,, Vol. 15 No. 9/10, pp