Building Commissioning Practices in New Construction and Existing Building Markets in the Pacific Northwest

Transcription

1 F i n a l R e p o r t Building Commissioning Practices in New Construction and Existing Building Markets in the Pacific Northwest prepared by SBW Consulting, Inc. report # October SW Third Avenue, Suite 600 Portland, Oregon telephone: fax:

2 SBW Consulting, Inc. Report No FINAL REPORT Building Commissioning Practices in New Construction and Existing Building Markets in the Pacific Northwest Submitted to NORTHWEST ENERGY EFFICIENCY ALLIANCE 522 SW Fifth Ave., Suite 410, Portland, Oregon Submitted by SBW CONSULTING, INC Northup Way, Suite 230 Bellevue, WA in conjunction with R.W. BECK and KAPLAN ENGINEERING October 1998

3 TABLE OF CONTENTS EXECUTIVE SUMMARY...Exec-1 1. INTRODUCTION SAMPLE DESIGN AND SELECTION SAMPLE OF BUILDING OWNERS SAMPLE OF COMMISSIONING SERVICE PROVIDERS DATA COLLECTION AND ANALYSIS TELEPHONE SURVEYS ASSIGNING A COMMISSIONING LEVEL SCORE RELIABILITY OF THE COMMISSIONING LEVEL SCORE ASSIGNING AN IMPORTANCE SCORE PRIVATE SECTOR NEW CONSTRUCTION MARKET ANALYSIS CHARACTERISTICS OF NEW CONSTRUCTION BUILDING OWNERS LEVEL OF COMMISSIONING IN RECENT NEW CONSTRUCTION ATTRIBUTES OF THE NEW CONSTRUCTION COMMISSIONING MARKETS EXPECTED GROWTH IN NEW CONSTRUCTION COMMISSIONING MARKETS BENEFITS OF NEW CONSTRUCTION COMMISSIONING BARRIERS TO NEW CONSTRUCTION COMMISSIONING POSSIBLE STRATEGIES FOR EXPANDING NEW CONSTRUCTION COMMISSIONING MARKETS PRIVATE SECTOR EXISTING BUILDING MARKET ANALYSIS CHARACTERISTICS OF EXISTING BUILDING OWNERS LEVEL OF COMMISSIONING IN EXISTING BUILDINGS ATTRIBUTES OF EXISTING BUILDING COMMISSIONING MARKETS EXPECTED GROWTH IN EXISTING BUILDING COMMISSIONING MARKETS BENEFITS OF EXISTING BUILDING COMMISSIONING BARRIERS TO EXISTING BUILDING COMMISSIONING POSSIBLE STRATEGIES FOR EXPANDING EXISTING BUILDING COMMISSIONING MARKETS SURVEY RESULTS FOR COMMISSIONING SERVICE PROVIDERS STRATEGIC PLAN FOR INTERVENING IN THE COMMISSIONING MARKETS SELECTING THE BEST STRATEGIES EDUCATIONAL PROGRAMS CASE STUDIES TAX CREDITS...52 APPENDICES Appendix A: Private Sector Building Owner Survey Results A1: New Construction Practices A2: Building Operation Practices SBW Consulting, Inc. Page i

4 Appendix B: Appendix C: Appendix D: Appendix E: Appendix F: Appendix G: Public Sector Building Owner Survey Results B1: New Construction Practices B2: Building Operation Practices Analysis of Benefits, Barriers, and Strategies for Public Sector Building Owners Commissioning Service Provider Survey Results Private Sector Building Owner Survey Part 2 - New Construction Practices Part 3 - Building Operations Practices Same Respondent Part 4 - Building Operations Practices Different Respondent Public Sector Building Owner Survey Part 2 - New Construction Practices Part 3 - Building Operations Practices Same Respondent Part 4 - Building Operations Practices Different Respondent Commissioning Service Provider Survey SBW Consulting, Inc. Page ii

5 TABLES Table 1: Quota and Completions for the Sample of Private Building Owners...3 Table 2: Completed Service Provider Surveys by Primary Type of Business...4 Table 3: Method for Calculating Commissioning Level Score for New Construction...7 Table 4: Continuation of New Construction Scoring...8 Table 5: Method for Calculating Commissioning Level Score for Existing Buildings...9 Table 6: Reliability of New Construction Commissioning Level Scores Table 7: Reliability of Existing Building Commissioning Level Scores Table 8: New Construction - Number of Owners, Projects, and Floor Area in Sample Table 9: Level of Commissioning in Recent New Construction Table 10: New Construction - Number of Owners, Projects, and Floor Area by Commissioning Level Table 11: New Construction - Familiarity with the Terms "Commissioning" and "Functional Performance Testing" Table 12: New Construction - Implemented Commissioning and FPT Table 13: New Construction - Separate Commissioning Specifications and Budget Table 14: New Construction - Design Review and Design Review Effectiveness Table 15: New Construction - Who Conducts FPT Table 16: New Construction - Future Role of FPT Table 17: New Construction - When Does HVAC Testing Staff Join Project Table 18: New Construction - Installation Checklists and Independent Spot Checks of Controls Table 19: New Construction - Trending with BAS and Documentation Beyond Pass/Fail Table 20: New Construction - Documentation of Problems, Testing Staff Hand-Off to Operators and Operators Involved in Design/Construction Reviews Table 21: New Construction - HVAC System Types Table 22: New Construction - Constant-Volume Package Rooftop Tests Table 23: New Construction - VAV Packaged Rooftop Tests Table 24: Continuation of VAV Packaged Rooftop Tests Table 25: New Construction - Central Chiller System Tests Table 26: Continuation of Central Chiller System Tests Table 27: Future Role of FPT and Selling Potential for New Construction Markets Table 28: Benefits of New Construction Commissioning (Rank) Table 29: Benefits of New Construction Commissioning (Raw Importance Score) Table 30: Barriers to Expanded Role for Commissioning in New Construction Markets (Rank) Table 31: Barriers to Expanded Role for Commissioning in New Construction Markets (Raw Importance Score) Table 32: Strategies for Expanding the Role of Commissioning in New Construction Markets (Rank) Table 33: Strategies for Expanding the Role of Commissioning in New Construction Markets (Raw Importance Score) Table 34: Existing Buildings - Number of Owners, Buildings, and Floor Area in the Sample Table 35: Level of Commissioning in Existing Buildings Table 36: Existing Buildings - Number of Owners, Projects, and Floor Area by Commissioning Level in Sample Table 37: Existing Buildings - Familiarity with the Terms "Commissioning" and "Functional Performance Testing" Table 38: Existing Buildings - Preventive Maintenance and Fraction of Buildings Receiving O&M Surveys Table 39: Existing Buildings - Motivations for O&M Surveys Table 40: Existing Buildings - Conducted Functional Performance Tests SBW Consulting, Inc. Page iii

6 Table 41: Existing Buildings - Fraction of Buildings Receiving FPT for HVAC, Lighting and Refrigeration Table 42: Existing Buildings - Motivations for FPT Table 43: Existing Buildings - Who Does FPT? Table 44: Existing Buildings - Future Role of FPT Table 45: Future Role of FPT and Selling Potential for Existing Building Markets Table 46: Benefits from Commissioning in Existing Building Markets (Rank) Table 47: Benefits from Commissioning in Existing Building Markets (Raw Importance Score) Table 48: Barriers to Expanded Role for Commissioning in Existing Building Markets (Rank) Table 49: Barriers to Expanded Role for Commissioning in Existing Building Markets (Raw Importance Score) Table 50: Strategies for Expanding the Role of Commissioning in Existing Building Markets (Rank) Table 51: Strategies for Expanding the Role of Commissioning in Existing Building Markets (Raw Importance Score) Table 52: Service Providers - Actively Market Commissioning and Market Growth Table 53: Service Providers - New and Existing Markets, Public and Private Markets and Last Year's Projects Table 54: Service Providers - Work by Market Segment Table 55: Service Providers - Services Provided for New Construction Table 56: Service Providers - New Construction Services (Continued) Table 57: Service Providers - New Construction Services Continued) Table 58: Service Providers - Existing Building Services Table 59: Service Providers - Existing Building Servic es (Continued) SBW Consulting, Inc. Page iv

7 Background and Purpose Executive Summary This market study, sponsored by the Northwest Energy Efficiency Alliance ("the Alliance"), is an investigation of the commercial building commissioning practices and related attitudes currently prevailing in the Pacific Northwest. It has been undertaken to provide the information needed to develop a strategic plan for making commissioning business as usual in the Pacific Northwest over the next five to ten years. The effort involved a telephone survey of public and private sector Building Owner s Representatives and Commissioning Service Providers in the Pacific Northwest (Washington, Oregon, Idaho, and Montana). Data from the surveys has been used to characterize baseline-commissioning practices and develop strategies for encouraging growth of the private sector commissioning market. Sample Design and Selection Two samples were developed for this research. The first sample is a group of building owners, i.e., the buyers of commissioning services (demand-side). The second sample was a group of firms who provide commissioning services, referred to as commissioning service providers (supply-side). Collecting information from both the demand-side and supply-side allows this study to fully characterize the commissioning markets. The research was limited to market segments that members of the Northwest Commissioning Collaborative believed were likely to be important future markets for commissioning services. Table Exec - 1 provides a list of the market segments surveyed and the quota established for each segment. The table also shows the number of owners for which surveys were completed. These owners were asked about new construction practices if they had projects that reached substantial completion during the last three years. We also sought respondents that could describe commissioning practices for buildings that had been occupied for more than five years (existing buildings market). Table Exec - 1: Quota and Completions for the Sample of Private Building Owners Sample of Building Owners Number of Responses for Sector/Market Segment Quota Complete New Construction Practices Existing Building Operation Practices All Segments Groceries Hospitals Hotels Nursing Homes/Assisted Living Offices (Commercial) Offices (High Tech Ind) Retail Stores Universities/Colleges SBW Consulting, Inc. Page Exec-1

8 A sample of 20 commissioning services providers was developed. As shown in Table Exec-2, this sample came equally from two different sources. One half was derived from various directories of commissioning agents and firms actively promoting their commissioning services in the Pacific Northwest. The other half was derived from the building owner surveys. Each building owner respondent was asked for the name of the person that most frequently conducts functional performance tests for their company. This sample design ensured a broader representation of firms that might play an important part in the commissioning markets, including those which primarily sell other services. Table Exec - 2: Completed Service Provider Surveys by Primary Type of Business Primary Type of Business Established Commissioning Providers Testing Firms Referred by Building Owners All Types HVAC Designer 2 1 HVAC Contractor 1 3 TAB Contractor 0 3 Controls Contractor 0 1 Commissioning 7 0 Other 0 2 Data Collection and Analysis Telephone Survey Data was collected for this market study through telephone interviews. Great effort was expended to find and recruit the best respondent for each building owner. For new construction, we sought a mid-level project manager, someone who had to be familiar with the owner's current and recent practices, but was close enough to the project front line to know about specific commissioning activities. For existing building information, we sought a supervisor in the building owner's facility operations staff. Commissioning Level Score One of the important goals of this study was to establish a method for measuring the level of commissioning in recent new construction and for the existing building market (buildings occupied for more than five years). This needed to be a repeatable methodology so that a similar study could be performed some years from now to measure the impact of the Alliance's interventions in the commissioning market. A multi-attribute scoring method was developed to measure the level of commissioning in both markets. Certain questions from the interviews were selected as being key indicators of the level of commissioning. The highest possible score in this system is 300. Importance Score The other major goal of this study was to gather information that could be used in formulating a strategic plan for expanding and enhancing the commissioning markets. A series of questions were asked of both building owner representatives and commissioning service providers about the benefits of commissioning, barriers to the expansion of commissioning activities, and strategies for overcoming those barriers. The respondents were asked to evaluate the importance of each benefit, barrier, or strategy SBW Consulting, Inc. Page Exec-2

9 in two ways. First, they were asked to rate the importance of each separately on a three-point scale. Once all on each list had been evaluated, the respondent was then asked to choose the most important benefit, barrier, or strategy. A joint importance score was computed for each benefit, barrier, or strategy for each respondent. Private Sector New Construction Market Analysis Table Exec-3 shows the number of owners, projects, and floor area associated with the responses to the new construction surveys. Across all market segments, the responses describe 569 projects with more than 28 million square feet of floor area that reached substantial completion in the last three years. Table Exec - 3: New Construction - Number of Owners, Projects, and Floor Area in Sample New Construction (Substantial Completion ) Number of Projects Total Floor Area Floor Area / Project Market Segment Number of Owners Total # Miss (000's Sq. Ft.) # Miss (000's Sq. Ft./Project) All Segments , Groceries , Hospitals , Hotels Nursing Homes/Assisted Living , Offices (Commercial) , Offices (High Tech Ind) , Retail Stores , Universities/Colleges # Miss - Number of responses missing, either Don't Know or Not Asked Responses from the surveys were used in assigning a "Level of Commissioning" score to each building owner participating in the market study. The mean scores for each market segment are shown in Table Exec - 4. SBW Consulting, Inc. Page Exec-3

10 Table Exec - 4: Level of Commissioning in Recent New Construction Sector/Market Segment Mean Commissioning Level Score (300 point scale) New Construction (Substantial Completion ) Percent of Floor Area with Low, Medium and High Commissioning Scores Low (0 to 100 Medium (100 to 200) High (200 to 300) All Segments (N=95) % 39% 19% Groceries (N=8) 88 75% 25% Hospitals (N=15) % 33% 53% Hotels (N=3) 76 67% 33% Nursing Homes/Assisted Living (N=16) % 31% 19% Offices (Commercial) (N=15) % 53% 20% Offices (High Tech Industrial) (N=8) % 13% 38% Retail Stores (N=19) % 53% Universities/Colleges (N=11) % 45% 9% Private Sector Existing Building Market Analysis Table Exec - 5 shows the number of owners, buildings, and floor area associated with the existing building surveys. Across all market segments, the responses described typical commissioning practices for 97 building owners, responsible for 1,650 buildings, containing more than 102 million square feet of floor area. Table Exec - 5: Existing Buildings - Number of Owners, Buildings, and Floor Area in the Sample Existing Buildings (Occupied for at Least Five Years) Number of Buildings Total Floor Area Floor Area / Building Market Segment Number of Owners Total # Miss (000's Sq. Ft.) # Miss (000's Sq. Ft./Building) All Segments , Groceries , Hospitals , Hotels , Nursing Homes/Assisted Living , Offices (Commercial) , Offices (High Tech Ind) , Retail Stores , Universities/Colleges , # Miss - Number of responses missing, either Don't Know or Not Asked SBW Consulting, Inc. Page Exec-4

11 As shown in Table Exec- 6, the existing building markets display somewhat higher commissioning scores than the new construction markets. A less complete set of factors was considered in forming these scores. Table Exec - 6: Level of Commissioning in Existing Buildings Sector/Market Segment Mean Commissioning Level Score (300 point scale) Existing Buildings (Occuppied for at Least Five Years) Percent of Floor Area with Low, Medium and High Commissioning Scores Low (0 to 100 Medium (100 to 200) High (200 to 300) All Segments (N=97) % 9% 51% Groceries (N=6) 94 50% 33% 17% Hospitals (N=12) % 8% 67% Hotels (N=7) % 57% Nursing Homes/Assisted Living (N=10) % 40% Offices (Commercial) (N=17) % 12% 65% Offices (High Tech Industrial) (N=16) % 6% 56% Retail Stores (N=16) % 19% 31% Universities/Colleges (N=13) % 54% Survey Results for Commissioning Service Providers Interviews were also conducted with a sample of 20 commissioning service providers. Ten of the respondents are representatives of firms found in various directories of commissioning agents or firms actively promoting their commissioning services in the Pacific Northwest. The other ten were cited by building owners as being the person that most frequently conducts functional performance tests for their company. These service providers were asked about the same topics as were the building owner representatives. Table Exec - 8 shows the type and size of the markets served by these firms. Table Exec - 7: Service Providers - New and Existing Markets, Public and Private Markets and Last Year's Projects Percent of Commissioning Work in New and Existing Markets Percent of Commissioning Work in Public and Private Markets Commissioning Related Projects in the Last Year Sample/Primary Business of the Firm New Construction Existing Buildings Other* Public Private Other* Number of Projects Floor Area All Providers HVAC Designer (N=3) 93% 7% 0% 43% 57% 0% 14 1,450,000 HVAC Contractor (N=4) 65% 35% 0% 64% 36% 0% 92 18,500,000 TAB Contractor (N=3) 73% 27% 0% 45% 55% 0% ,000 Controls Contractor (N=1) 50% 50% 0% 50% 50% 0% 75 Commissioning (N=7) 69% 31% 0% 21% 79% 0% 56 3,730,000 Other (N=2) 46% 55% 0% 95% 6% 0% ,000 Total (N=20) 69% 31% 0% 45% 55% 0% ,530,000 SBW Consulting, Inc. Page Exec-5

12 Strategic Plan for Intervening in the Commissioning Markets The purpose of this strategic plan is to identify specific market interventions that will achieve the goal of making commissioning "business as usual" over the next 5 to 10 year period. Our surveys of building owners and service providers produced considerable evidence that substantial portions of the new construction and existing building markets already practice aspects of commissioning. It is possible to conclude that commissioning is "business as usual," for at least a portion of these markets. However, this study also demonstrates that there is ample room for expanding these practices. Expansion could be achieved by bringing more market segments to higher levels of commissioning practice. In addition, expansion could come from improving the depth, scope, and quality of commissioning practice. The market study provides significant information about what interventions might be effective. This comes from the opinions of owners' representatives about what would help advance the practice of commissioning in their own organizations. These same respondents also provided substantial information about perceived benefits of commissioning and barriers to its expanded practice. These opinions were reinforced by commissioning service providers who were asked the same questions. Clearly, we can not know for sure what actions will be effective. However, the best place we have to start is with the people who are on the front line, either buying commissioning services (owners representatives) or selling those services (commissioning service providers). The market study provides a wealth of information from both of these perspectives. As detailed as the market study was, it still only provides broad directions for the strategic plan. Deciding, in detail, how to pursue these strategic directions will have to be left to organizations that broadly represent the market of commissioning service buyers and sellers, such as the Northwest Commissioning Collaborative and the Alliance. Three strategic directions have been identified. Education Participants in the new construction market clearly desire educational programs that will explain testing procedures and the benefits of commissioning. Two educational programs are needed. The first would focus on members of the design community. For all market segments this should include design engineers. For most market segments the program should also address architects. The second program would focus on the education of building owners. There are three types of educational programs needed for the existing building markets. For some market segments, the provision of technical assistance for first time users is an important strategy. Educational programs for building owners will be important for a number of market segments. Finally, building operator education is needed in many market segments Case Studies The lack of documented benefits was the 3 rd most important barrier to commissioning in the new construction market. Well-crafted case studies might be the solution to this barrier although this strategy was ranked 5 th overall. Certain market segments clearly see the value of case studies. Service providers also ranked this strategy 2 nd. Informal comments from respondents indicate that they would need case studies that are specific to the type of buildings built by their organization. Respondents may believe that case studies would be too generic to be of use and thus discount them as an important strategy. Some additional research should be done to determine where market segment specific case studies would be of SBW Consulting, Inc. Page Exec-6

13 value. We also have some indication from owners who have built large numbers of buildings that case studies would have to be done for one of their buildings, in order for the results to be useful. In the exiting building market, case studies were ranked 3 rd. Some market segments also gave a high rank to technical assistance for first time users of testing procedures. These two strategies can be effectively combined. Case studies for first time users will clearly demonstrate the benefits of commissioning. Clearly, service providers are keen on the combination of these two strategies, as they ranked them 1 st and 2 nd. Tax Incentives Tax credits were the 2 nd rank strategy for existing building markets and the 4 th rank strategy for new construction markets. This is a complex topic and it is not clear what actions would be effective. More research should be conducted with market segment representatives to identify the specific tax credits they feel would be effective. This research could also identify and examine existing tax credits that have been used for similar purposes, such as the Oregon Business Energy Tax Credit. Further specification of the action plan in this area will have to be accomplished by the experts that conduct this additional research. What Not to Do Sometimes what is not to be done is just as important what is to be done. The surveys for both new construction and existing buildings clearly identified two actions that should not be taken. 1. Professional Certification. The certification of commissioning service providers has been much debated over recent years. The markets do not place a high value on certification. 2. Non-Financial Awards. Some commissioning programs have relied on non-financial awards, e.g., certificates of merit, to expand the practice of commissioning. Again, the markets place little value on these awards. The Alliance has limited resources for intervening in the commissioning markets. It is vital that these resources be narrowly focused on the best possible strategies and not be diverted to common but ineffective actions. SBW Consulting, Inc. Page Exec-7

14 1. Introduction This market study, sponsored by the Northwest Energy Efficiency Alliance ("the Alliance"), is an investigation of the commercial building commissioning practices and related attitudes currently prevailing in the Pacific Northwest. It has been undertaken to provide the information needed to develop a strategic plan for making commissioning business as usual in the Pacific Northwest over the next five to ten years. The effort involved a telephone survey of public and private sector Building Owner s Representatives and Commissioning Service Providers in the Pacific Northwest (Washington, Oregon, Idaho, and Montana). Data from the surveys has been used to characterize baseline-commissioning practices and develop strategies for encouraging growth of the commissioning market. Specific objectives of this research are to: Determine the most appropriate market segments for commissioning. Identify current barriers to commissioning service delivery and broader adoption of building commissioning practices. Determine what opportunities exist for overcoming barriers to commissioning. Establish a measurement technique for commissioning practices that will allow quick assessment of market transformation effects. One hundred and thirty surveys have been conducted with representatives of private sector building owners. The detailed results from the private sector surveys are tabulated in Appendix A. Strategic analyses of the private-sector survey results, along with results from twenty surveys completed with commissioning service providers, appear in various sections of the main body of this document. The detailed results of the service provider surveys are tabulated in Appendix D. The survey instruments used with the private sector building owners appears in Appendix E and the instrument used with service providers appears in Appendix G. In addition, fifty surveys were completed, with the help of government agency staff from each of the four participating states, with public building owners. A detailed tabulation of these public sector survey results appears in Appendix B and a set of summary analyses of responses to questions concerning commissioning benefits, barriers and strategies for enhancing the commissioning market appear in Appendix C. The survey instrument used with public sector building owners appears in Appendix F. SBW Consulting, Inc. Page 1

15 2. Sample Design and Selection Two samples were developed for this research. The first sample is a group of building owners, i.e., the buyers of commissioning services (demand-side). The second sample was a group of firms who provide commissioning services, referred to as commissioning service providers (supply-side). Collecting information from both the demand-side and supply-side allows this study to fully characterize the commissioning markets. The following sections describe how each of these samples was developed. 2.1 Sample of Building Owners The primary data source for the private sector building owners sample was the American Business Information (ABI) businessusa database. This database contains 10 million business/organization listings compiled from yellow page listings for the entire country and supplemented by information from other sources and ABI s own surveys. It provides the business name, address, telephone number, contact name, employment (by ranges), sales (by ranges), primary and three auxiliary SIC codes to describe the line of business, identification of headquarters or branch offices, number of years listed in the telephone book, and the size of the telephone book advertisement. For a number of market segments, we supplemented the ABI listing with listings from Puget Sound Business Journal Office Leasing Guide, Portland Business Journal, and the BOMA Portland Office Leasing Guide. In addition, certain parts of the sample were uncovered by "networking" with respondents. For example, once we understood how difficult it would be to identify the firms that build new hotels, we started asking each respondent in that segment to suggest other firms that they knew had recently built hotels. Other kinds of networking were also necessary. For example, many retail listings in ABI were for firms that only lease space. We would ask these firms for the name of the firm that owns the building they lease. The research was limited to market segments that members of the Northwest Commissioning Collaborative 1 believed were likely to be important future markets for commissioning services. Table 1 provides a list of the market segments surveyed and the quota established for each segment. A sample list containing at least three times the targeted number of completed surveys for each market segment was developed. The table also shows the number of owners for which surveys were completed. These owners were asked about new construction practices if they had projects that reached substantial completion 2 during the last three years. We also sought respondents that could describe commissioning practices for buildings that had been occupied for more than five years (existing buildings market). The count of owners providing information about new and existing practices is also shown in Table 1. 1 The Northwest Commissioning Collaborative provided invaluable guidance, throughout the course of this study, on the importance of various market segments and numerous other research design issues. 2 For new buildings, the date when the work or designated portion of the work is sufficiently complete in accordance with the contract documents so the owner can occupy or utilize the space for its intended use. SBW Consulting, Inc. Page 2

16 Table 1: Quota and Completions for the Sample of Private Building Owners Sample of Building Owners Number of Responses for Sector/Market Segment Quota Complete New Construction Practices Existing Building Operation Practices All Segments Groceries Hospitals Hotels Nursing Homes/Assisted Living Offices (Commercial) Offices (High Tech Ind) Retail Stores Universities/Colleges The goal of this sample was to include owners that would account for a large fraction of the regional floor space in each market segment. The floor area covered by the completed sample of owners is shown in Table 8 (New Construction) and Table 34 (Existing Buildings). 2.2 Sample of Commissioning Service Providers A sample of 20 commissioning services providers was developed. This sample came equally from two different sources. One half was derived from various directories of commissioning agents and firms actively promoting their commissioning services in the Pacific Northwest. The other half was derived from the building owner surveys. Each building owner respondent was asked for the name of the person that most frequently conducts functional performance tests 3 for their company. This sample design ensured a broader representation of firms that might play an important part in the commissioning markets, including those which primarily sell other services. Table 2 provides a breakdown of the two parts of the service provider sample, showing the number of firms by their primary type of business. 3 Defined as the process of determining the ability of a building system to perform in accordance with the design intent, through all of the intended modes of operation. SBW Consulting, Inc. Page 3

17 Table 2: Completed Service Provider Surveys by Primary Type of Business Primary Type of Business Established Commissioning Providers Testing Firms Referred by Building Owners All Types HVAC Designer 2 1 HVAC Contractor 1 3 TAB Contractor 0 3 Controls Contractor 0 1 Commissioning 7 0 Other 0 2 SBW Consulting, Inc. Page 4

18 3. Data Collection and Analysis 3.1 Telephone Surveys Data was collected for this market study through telephone interviews. The interview instruments used in this study are provided in Appendices E, F, and G. Appendix E contains a copy of the instrument used to collect new construction and existing building data from representatives of private sector building owners. Appendix F contains a very similar instrument used to collect this same information from representatives of public sector building owners. Appendix G contains the instrument used to collect information from commissioning service providers. Great effort was expended to find and recruit the best respondent for each building owner. For new construction, we sought a mid-level project manager, someone who had to be familiar with the owner's current and recent practices, but was close enough to the project front line to know about specific commissioning activities. Some owners outsource construction project management. In those cases, we spoke with the outsource firm and found a project manager who was familiar with the building owner's (their client) current and recent practices. For existing building information, we sought a supervisor in the building owner's facility operations staff. A recruitment specialist on our team was responsible for obtaining agreement from each owner's organization to participate in the survey and identifying the best respondent for new construction and existing building information. The contact was passed to one of our mechanical engineers who was responsible for completing the interview. The interview involves a sufficiently technical dialogue that it had to be conducted by an engineer who had considerable background in building systems, particularly HVAC systems. At the start of each interview, our engineer would ask a number of questions intended to confirm that we had the best respondent. In a number of cases, other better respondents were identified and contacted before we completed the survey. Responses recorded from the telephone interview were entered in an MS ACCESS database. The database field names assigned to each question in this database are shown on the interview instruments in Appendices E, F, and G. 3.2 Assigning a Commissioning Level Score One of the important goals of this study was to establish a method for measuring the level of commissioning in recent new construction and for the existing building market (buildings occupied for more than five years). This needed to be a repeatable methodology so that a similar study could be performed some years from now to measure the impact of the Alliance's interventions in the commissioning market. A multi-attribute scoring method was developed to measure the level of commissioning in both markets. Certain questions from the interviews were selected as being key indicators of the level of commissioning. Each of these questions were given a weight based on our study team's assessment of their relative importance (weight ranged from 1 = least importance to 5 = most importance) in measuring the level of commissioning. Similarly, a response score (1 = lowest level of commissioning and 5 = highest level of commissioning) was assigned to each possible response to these questions. Tables 3 and 4 list the questions that were included in the new construction scoring system along with the question weights and the response scores. The highest possible score in this system is 300. An additional rule was applied to the portion of the score associated with the HVAC system testing questions (15a, 15b, and 15c). Some of the respondents claimed that the tests described in these questions SBW Consulting, Inc. Page 5

19 were typically performed, although they did not always receive documentation of the test results (question 15d). The study team felt that this might be an indication that the tests were not performed. To correct this problem, the score for questions 15a, 15b and 15c were reduced if the answer to question 15d was not "always." If the respondent reported that documentation was received "sometimes," the score was reduced by 50%. If the response was "never," the score was reduced by 90%. Table 5 lists the weights and response scores that defined the existing building scoring method. A significantly less detailed discussion was completed with existing building respondents and thus there are fewer questions involved in this scoring method. The highest possible score for this method was 66. However, these scores were scaled-up to a maximum of 300 so that the results for both markets could be presented in a comparable format. The precision of both new construction and existing building scores is expected to be low. Thus, a score of 100 is probably not significantly different from a score of 125. To emphasize this fact, we have placed all respondents in one of three scoring categories: Scores from 0 to 100 Scores from 100 to 200 Scores from 200 to 300 Many of the survey results presented in Section 4, 5, and 6 show the fraction of respondents in each of these scoring categories. SBW Consulting, Inc. Page 6

20 Table 3: Method for Calculating Commissioning Level Score for New Construction Question Number Question Question Weight Response 3 Have you ever implemented building 2 Yes 5 commissioning on any of the buildings you ve constructed (for )? Response Score No 0 Don't know 0 In your project specs, do you include a 4 3 Yes 5 separate section that deals with Sometimes 3 commissioning? No 0 Don't know 0 5 Do you have a separate line item in your 4 Yes 5 project budgets for commissioning? Sometimes 3 No 0 Don't know 0 7 Have you ever conducted functional 2 Yes 5 performance testing on equipment and No 0 systems in the buildings you construct [for Don't know 0 8 Who usually conducts the tests to assure 4 Member of the company's staff 4 that the systems are working properly? Prime contractor for the project 2 HVAC or controls system designer or engineer 2 HVAC or controls system contractor 2 Test and Balance (TAB) subcontractor 2 Test and Balance (TAB) independent 3 Independent contractor 4 Independent commissioning agent 5 Don't Know 0 10 When are people involved with HVAC 4 Schematic design 5 system testing first brought into the Design development 5 project? Construction documents 4 1st 75% 3 Last 25% 2 Acceptance 2 Post-acceptance 1 Don't know 0 11 Do you usually have the person 2 Yes 5 responsible for the HVAC system testing Sometimes 3 complete a design review before No 0 construction begins? Don't know 0 12 How effective are these design reviews in 3 Very 5 finding design issues that need to be Somewhat 3 resolved? Not 0 Don't know 0 13a Are checklists provided which document 3 Always 5 that all components are installed according Sometimes 3 to design documents? Never 0 Don't know 0 13b Are controls and test and balance results 3 Always 5 independently spot checked? Sometimes 3 Never 0 Don't know 0 13c Is trending conducted with the building 4 Always 5 automation system or separate monitoring equipment and docu-mentation provided (within the first year)? Sometimes 3 Never 0 Don't know 0 SBW Consulting, Inc. Page 7

21 Table 4: Continuation of New Construction Scoring Question Number Question Question Weight Response Response Score 13d Are test results, besides the basic TAB 4 Always 5 report, documented beyond a pass/fail Sometimes 3 indication? Never 0 Don't know 0 13e Is documentation provided on the problems 5 Always 5 found by the testing and how to correct Sometimes 3 them? Never 0 Don't know 0 13f Do those conducting the tests participate in 3 Always 5 handing-off of the building to operations Sometimes 3 staff? Never 0 Don't know 0 14 How often do building operations staff 2 Always 5 attend design and construction progress Sometimes 3 review meetings? Never 0 Don't know 0 15a Most basic testing (on/off). Specific 1 Always 5 wording for each of three HVAC system Sometimes 3 types. Never 0 Don't know 0 15b Mid-level testing. Specific wording for each 4 Always 5 of HVAC system types. Sometimes 3 Never 0 Don't know 0 15c Trending / monitoring. Specific wording for 3 Always 5 each of three HVAC system types. Sometimes 3 Never 0 Don't know 0 15d Documentation received. Specific wording 4 Always 5 for each of three HVAC system types. Sometimes 3 Never 0 Don't know 0 SBW Consulting, Inc. Page 8

22 Table 5: Method for Calculating Commissioning Level Score for Existing Buildings Question Number Question Question Weight Response 3 Does your organization have a preventative 1 Yes 1 Response Score maintenance program for its buildings in the No 0 Pacific Northwest? Don't know 0 Has your company conducted operations and 4 2 Yes 1 maintenance surveys in buildings that have been occupied for more than five years? These could be whole building surveys or surveys of specific No 0 systems such as HVAC. Don't know 0 4 Approximately what fraction of these buildings 3 none 0 have been surveyed? less than 1/4 1 1/4 to 1/2 2 1/2 to 3/4 3 More than 3/4 4 5 Has your company conducted functional 2 Yes 5 performance tests on the existing systems and No 0 equipment in these same buildings? Don't know 0 5 Approximately what fraction of these buildings 4 none 0 have been tested? less than 1/4 1 1/4 to 1/2 2 1/2 to 3/4 3 More than 3/4 4 6 Who usually conducts the functional performance 5 Company staff 4 tests on older buildings? Mech./Elec. contractor Reliability of the Commissioning Level Score Independent agent 5 Don't know 0 There are many sources of error in the commissioning level scores. Much of the information used in the scoring is qualitative and subject to interpretation. It is difficult to quantitatively measure the reliability of this scoring method. One technique is to independently derive a second score. This was done by asking the mechanical engineer conducting the interview to place each of the respondent's organizations on a five point commissioning level scale, based on their overall impression of the scope and depth of the owner's commissioning activities. The interviewer was asked to consider the entire dialogue with each respondent, including the informal commentary that was not recorded in the survey database. This fivepoint scale was scaled-up to a 300-point maximum so that it could be compared to commissioning level score computed by the methods shown in Tables 3, 4, and 5. Separate assessments were made by the interviewer for new construction and existing building commissioning levels. Table 6 compares the interviewer judgement score to the more formal algorithm for each of the new construction market segments. In general, given the expected precision of these scales, the two scoring methods are highly consistent, which provides some assurance as to the reliability of the scores. Table 7 shows this comparison for the existing building market segments, and the result is similar. SBW Consulting, Inc. Page 9

23 One of the primary sources of error in the scoring method is that respondents may not be fully knowledgeable about their firm's new construction or existing building commissioning practices. Our recruitment staff and the interviewing engineer went to great lengths to find the best respondent for each building owner. However, a final judgement about the quality of the respondent was not possible until each interview was complete. At that time the interviewer was asked to assess their level of confidence that the respondent was the best available spokesperson for that building owner. Tables 6 and 7 show the distribution among three confidence levels (low, medium and high) for each new construction and existing building market segment. For new construction, our interviewer gave a medium or high confidence of having spoken to the best respondent for 82 percent of the sample. For existing buildings, this level of confidence was achieved for 96 percent of the sample. Table 6: Reliability of New Construction Commissioning Level Scores New Construction (Substantial Completion ) Mean Commissioning Level Score (300 point scale) Level of Interviewer Confidence that Best Respondent was Interviewed Percent Market Segment Interviewer Judgement Scoring Algorithm Low Medium High All Segments (N=95) % 27% 56% Groceries (N=8) % 13% 63% Hospitals (N=15) % 27% 60% Hotels (N=3) % 0% 100% Nursing Homes/Assisted Living (N=16) % 31% 31% Offices (Commercial) (N=15) % 33% 60% Offices (High Tech Ind) (N=8) % 13% 88% Retail Stores (N=19) % 37% 42% Universities/Colleges (N=11) % 20% 70% SBW Consulting, Inc. Page 10

24 Table 7: Reliability of Existing Building Commissioning Level Scores Existing Buildings (Occupied for at Least Five Years) Mean Commissioning Level Score (300 point scale) Level of Interviewer Confidence that Best Respondent was Interviewed Percent Market Segment Interviewer Judgement Scoring Algorithm Low Medium High All Segments (N=97) % 20% 76% Groceries (N=6) % 17% 67% Hospitals (N=12) % 25% 67% Hotels (N=7) % 29% 57% Nursing Homes/Assisted Living (N=10) % 30% 70% Offices (Commercial) (N=17) % 29% 71% Offices (High Tech Ind) (N=16) % 6% 94% Retail Stores (N=16) % 19% 75% Universities/Colleges (N=13) % 8% 92% 3.4 Assigning an Importance Score The other major goal of this study was to gather information that could be used in formulating a strategic plan for expanding and enhancing the commissioning markets. A series of questions were asked of both building owner representatives and commissioning service providers about the benefits of commissioning, barriers to the expansion of commissioning activities, and strategies for overcoming those barriers. In each case, a long list of benefits, barriers, or strategies was discussed with the respondents. The respondents were asked to evaluate the importance of each benefit, barrier, or strategy in two ways. First, they were asked to rate the importance of each separately on a three-point scale. Once all on each list had been evaluated, the respondent was then asked to choose the most important benefit, barrier, or strategy. They were allowed to select more than one "most" important if they couldn't decide on just one. A joint importance score was computed for each benefit, barrier, or strategy for each respondent. The score was equal to 1 if the respondent assigned an importance of "very" on the three point scale. The score was equal to 3 if the respondent selected the item as the most important in the list. The score was 4 if the respondent rated the benefit, barrier or strategy as both "very" important and most important in the list. These scores were summed by market segment and for the sample as a whole and were used to rank the lists of benefits, barriers, and strategies. The results of these rankings were important considerations in formulating the strategic plan that is described in the last section of this document. SBW Consulting, Inc. Page 11

25 4. Private Sector New Construction Market Analysis This section provides selected results from the interviews with building owner representatives concerning their new construction commissioning practices and related factors. Included are all of the responses that were used in formulating the commissioning level scores. In addition, information is presented that describes other features of the commissioning activities that are relevant to the formulation of the strategic plan. All of the results summarized in this section are for the private sector sample. 4.1 Characteristics of New Construction Building Owners Table 8 shows the number of owners, projects, and floor area associated with the responses to the new construction surveys. Across all market segments, the responses describe 569 projects with more than 28 million square feet of floor area that reached substantial completion in the last three years. Table 8: New Construction - Number of Owners, Projects, and Floor Area in Sample New Construction (Substantial Completion ) Number of Projects Total Floor Area Floor Area / Project Market Segment Number of Owners Total # Miss (000's Sq. Ft.) # Miss (000's Sq. Ft./Project) All Segments , Groceries , Hospitals , Hotels Nursing Homes/Assisted Living , Offices (Commercial) , Offices (High Tech Ind) , Retail Stores , Universities/Colleges # Miss - Number of responses missing, either Don't Know or Not Asked 4.2 Level of Commissioning In Recent New Construction As described in Section 3, responses from the surveys were used in assigning a "Level of Commissioning" score to each building owner participating in the market study. The mean scores for each market segment are shown in Table 9. SBW Consulting, Inc. Page 12

26 Table 9: Level of Commissioning in Recent New Construction Sector/Market Segment Mean Commissioning Level Score (300 point scale) New Construction (Substantial Completion ) Percent of Floor Area with Low, Medium and High Commissioning Scores Low (0 to 100 Medium (100 to 200) High (200 to 300) All Segments (N=95) % 39% 19% Groceries (N=8) 88 75% 25% Hospitals (N=15) % 33% 53% Hotels (N=3) 76 67% 33% Nursing Homes/Assisted Living (N=16) % 31% 19% Offices (Commercial) (N=15) % 53% 20% Offices (High Tech Industrial) (N=8) % 13% 38% Retail Stores (N=19) % 53% Universities/Colleges (N=11) % 45% 9% One of the most striking conclusions of this study is that elements of commissioning are "business as usual" in the Pacific Northwest. This is particularly true, as shown in Table 9, for certain market segments like Hospitals, where more than half of recent new construction has a high commissioning score. More than a third of the High Tech Office segment also falls in this category. These high scores do not indicate that all possible commissioning practices have been applied. But they do demonstrate that significant commissioning activity is present in the market. Both high and low scores are of strategic interest. It may be easy to continue sales of commissioning services to those organizations that already consider it an important part of standard construction practice, i.e., those with high score. This can provide a stable base for an expanding commissioning industry. Markets with low score should either be ignored or special programs should be developed that can overcome the resistance to commissioning in those markets. 4.3 Attributes of the New Construction Commissioning Markets The following tables show a number of important attributes of the commissioning market as expressed by the new construction respondents. Many of these attributes were used in constructing the level of commissioning scores. Table 10 shows how the 95 new construction respondents are distributed across the commissioning level scale. The smallest portion of the owners, projects and floor has the highest level of commissioning. SBW Consulting, Inc. Page 13