FINAL EVALUATION REPORT High Performance New Construction and Residential New Construction Programs

Transcription

1 FINAL EVALUATION REPORT High Performance New Construction and Residential New Construction Programs November 2013

2 [This page intentionally left blank.] Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 2

3 COPYRIGHT STATEMENT Ontario Power Authority All Rights Reserved. No part of this report or any of its contents may be reproduced, copied, modified or adapted, without the prior written consent of the Ontario Power Authority, unless otherwise indicated for stand-alone materials. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 3

4 PREPARED BY: Frontier Associates LLC PREPARED FOR: The Ontario Power Authority Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 4

5 AKNOWLEDGEMENTS Frontier Associates would like to thank Kausar Ashraf, Senior Evaluation Specialist at the Ontario Power Authority (OPA), for her assistance in coordinating this evaluation effort. We would also like to thank OPA staff for their willingness to provide information and assistance in a timely and organized manner. With this support, we were able to complete this evaluation report successfully and efficiently. Frontier Associates would also like to thank the staff at the Local Distribution Companies (LDCs) for providing their insight on the HPNC and RNC programs. Finally, we thank the participants who participated in our telephone surveys. The data provided in these surveys helped our effort to provide quality feedback on the OPA s new construction initiatives that will help improve program delivery and administration for the coming years. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 5

6 [This page intentionally left blank.] Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 6

7 Table of Contents TABLE OF CONTENTS EXECUTIVE SUMMARY... 8 SUMMARY OF EVALUATION GOALS AND OBJECTIVES... 8 SUMMARY OF IMPACT EVALUATION RESULTS... 9 SUMMARY OF PROCESS EVALUATION RESULTS CONCLUSION AND RECOMMENDATIONS INTRODUCTION EVALUATION GOALS AND OBJECTIVES PROGRAM DESCRIPTIONS PROGRAM PURPOSE AND GOALS REPORT OVERVIEW IMPACT EVALUATION METHODOLOGY RESULTS DISCUSSION BY PROGRAM: HIGH PERFORMANCE NEW CONSTRUCTION DISCUSSION BY PROGRAM: RESIDENTIAL NEW CONSTRUCTION RECOMMENDATIONS PROCESS EVALUATION HIGH PERFORMANCE NEW CONSTRUCTION RESIDENTIAL NEW CONSTRUCTION AUDIT OF THE OPA S PROGRAM TRACKING SYSTEMS (BOTH HPNC AND RNC) ASSESSMENT OF DIRECT EMPLOYMENT EFFECTS (BOTH HPNC AND RNC) PROGRAM DESIGN & OTHER RECOMMENDATIONS (BOTH HPNC AND RNC) CONCLUSION AND RECOMMENDATIONS APPENDIX A: GLOSSARY APPENDIX B: HPNC PARTICIPANT SURVEY APPENDIX C: RNC PARTICIPANT SURVEY APPENDIX D: UPDATED PRESCRIPTIVE ASSUMPTIONS Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 7

8 Executive Summary EXECUTIVE SUMMARY Ontario Power Authority (OPA) hired Frontier Associates to conduct this impact and process evaluation of the OPA s High Performance New Construction (HPNC) and Residential New Construction (RNC) programs. The HPNC program is designed to encourage the construction of energy efficient buildings exceeding the requirements of the Ontario Building Code. Participants are facilities managers and owners. The RNC program is designed to encourage residential homebuilders to construct energy efficient homes. Participants in the RNC are homebuilders. Summary of Evaluation Goals and Objectives Frontier s goals and objectives of the Final Evaluation are as specified in Frontier s contract with the OPA: Impact Evaluation o Assess the gross and net energy and demand savings attributed to the programs through verification of gross savings. Process Evaluation o Determine the overall effectiveness and comprehensiveness of the programs by assessment of the effectiveness of the programs market delivery, audit of OPA s program tracking systems, review of customer motivations, and assessment of direct employment effects. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 8

9 Executive Summary Summary of Impact Evaluation Results Frontier was charged with reviewing HPNC prescriptive and custom savings for projects completed in 2011 and Table 1 provides the summary of the impact evaluation results found by the Evaluation Team. Table 2 breaks out the realization rates by program track within the HPNC and RNC. Table 1: Summary of Impact Evaluation Results Program Metric HPNC RNC Number of Participants Program Realization Rate (% kwh) 98% 586% Program Realization Rate (% kw) 96% 803% Gross Verified Demand Savings (MW) Gross Verified Annual Energy Savings (GWh) Gross Verified Lifetime Energy Savings (GWh) Net to Gross Ratio Net Peak Demand Savings (MW) Net Annual Energy Savings (GWh) Net Lifetime Energy Savings (GWh) Unique participants, not projects. For the HPNC program, participants are the facility owners or managers, some of whom implemented more than one project. For the RNC program, the participants are home builders or developers (not the home owners). Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 9

10 Executive Summary Table 2: Realization Rates by Program Track Realization Rates by Program Track % kwh % kw High Performance New Construction 98% 96% Prescriptive 93% 90% Engineered No projects No projects Custom 100% 100% Residential New Construction 586% 803% Prescriptive 84% 91% Performance 610% 837% Custom No projects No projects HPNC Impact Estimates HPNC program realization rates were 98 percent for kwh and 96 percent for kw savings. This high realization rate was due to a 100% realization rate for custom projects, which accounted for approximately 66 percent of the overall kwh savings and 62 percent of the overall kw savings for the HPNC program. Realization rates were less than 100 percent for the HPNC prescriptive program due to minor baseline adjustments and the exclusion of all pin-based compact fluorescent measures, which were removed from the HPNC program in The prescriptive project assessment included a review of assumed baselines and change cases, savings calculation methodologies, and associated assumptions. The custom project assessment was based on a review of computer modelling software, including modelling baselines, inputs, and simulation reports. RNC Impact Estimates The Evaluation Team s high realization rates for the RNC program stem from the performance track projects, which made up 91 percent of the RNC projects. Using energy consumption estimates from HOT 2000 reports as a baseline and using EnerGuide scores to establish energy and demand savings, the Evaluation Team found that verified gross savings estimates were higher than the claimed gross savings. Frontier has reservations concerning the validity of the program realization rates due to the quality of EnerGuide reporting paperwork provided to Frontier for the RNC program. In the future, better data may help the team refine verified savings and program realizations rate. The team suggests that the OPA provide baseline and change case energy consumptions and housing characteristics (square footage, Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 10

11 Executive Summary number of floors, heating type and capacity, HVAC type and capacity, insulation levels, water heating type and capacity). Summary of Process Evaluation Results Frontier evaluated the satisfaction of participating homebuilders and facility managers and LDCs with the HPNC and RNC programs. Participating homebuilders and facility managers in the programs report high levels of satisfaction with the programs. However, LDCs are dissatisfied with the programs. The Evaluation Team believes that program design changes made incrementally and with stakeholder input could transform the HPNC and RNC programs into successful and valuable programs. Participation in both the HPNC and RNC programs is low. A geographic analysis of both the HPNC and RNC projects shows that the programs are not penetrating Ontario s new construction market. Most HPNC and RNC projects occur outside of the Census Metropolitan Areas (CMAs) that show the most growth in commercial or residential new construction. Program participation seems disconnected from Ontario s significant new construction market. Although recent changes in the HPNC and RNC program have improved its effectiveness, concerns continue in the following areas, according to LDCs and participants: Twenty-one percent of HPNC and RNC participants surveyed found the application process unreasonable and difficult to understand. Minimum project size requirements eliminate many small commercial project opportunities. The application process is overly burdensome relative to the size of the incentives. The size of the incentives being offered is insufficient to draw participation. Eighty percent of HPNC and RNC participants surveyed would have included the energy efficient measures if incentives had not been available, indicating that the incentive levels are not motivating participants who had no original intention to install the energy efficiency measures. Many participants commented on how the application is difficult to amend to reflect changes implemented by the builder, tenant, or owner during the construction phase. LDCs have concerns about the transition to the next generation of this program. The LDCs cannot guarantee an incentive if the new building or project may not be completed after December 31, 2014 and the planning and construction phases of larger projects can last multiple years. 2 LDCs note that the procedures for requesting program requirement waivers from the OPA are lengthy and cumbersome. 2 In September 2013, the OPA announced that it intended to allow projects initiated before December 31, 2014 but completed in 2015 to be eligible for financial incentives without needing a waiver from the OPA. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 11

12 Executive Summary Conclusion and Recommendations The HPNC and RNC programs have great potential to reduce energy consumption and on-peak demand in Ontario. Frontier believes that OPA should implement incremental changes to the design of both programs based on stakeholder input and undertake an increased marketing effort focusing on areas of higher intensity of new construction activity to stimulate participation in the programs. Impact Evaluation Frontier verified the energy and demand savings claimed for the HPNC and RNC programs. Savings for the new construction initiatives are dominated by the HPNC program, which contributed more than 99 percent of total savings. Roughly 66 percent of savings come from the HPNC Custom track, and 33 percent of savings from the HPNC Prescriptive track, as indicated below. Table 3: Gross Energy and Demand Savings by Initiative Initiative Track Gross Demand Savings (kw) Gross Energy Savings (kwh) HPNC Prescriptive 1,195 3,580,613 HPNC Custom 2,306 8,148,078 RNC Prescriptive RNC Performance ,312 A document review indicated that the most of the calculation methods and assumptions listed in the 2011 Quasi-Prescriptive Measures and Assumptions List (MAL) Version 1.0 were reasonable. Where applicable, Frontier made adjustments to baselines and change cases. Additionally, Frontier recommends a savings estimation strategy that makes use of building-specific operating hours. These hours are already available in the MAL and will improve the accuracy of the savings estimates. A review of the available documents for other custom projects indicated that the modellers used good practice in modelling architectural and mechanical, electrical and plumbing systems for energy consumption, yielding reasonable energy and demand savings. There were no recommended changes to the custom savings estimates. Only three projects were submitted to the RNC Prescriptive track, with minimal impact on overall program savings. Some measure assumptions were not utilized because the assumptions were deemed to be invalid based on such a small project sample size. In these cases, project-specific information was used instead. RNC Performance projects submitted ranged in EnerGuide score from 83 to 87, with the majority falling at a score of either 83 or 85, the minimum threshold score to receive an incentive. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 12

13 Executive Summary Overall, the HPNC initiative is driving the energy and demand savings for OPA s new construction initiatives. Specifically, one custom project contributed roughly 60 percent of total new construction initiative savings. Agribusiness ventilation is also a primary driver of savings for the prescriptive projects. Process Evaluation Program Design Recommendations The OPA should begin planning the next generation of new construction energy efficiency programs for Ontario as soon as possible. Among the parties involved in the existing new construction programs, there is a general consensus that the programs should be improved; however, there is no consensus over what the future structure should be. 1. Some parties would favor a structure employing a limited number of province-wide program implementers, similar to the design of the first generation of the HPNC. This approach would make the program easier for national or regional chain accounts to participate, and would result in a more uniform marketing message. 2. Many LDCs would prefer a more decentralized program approach, which would offer the LDCs greater flexibility in program design and management. This approach would enable LDCs to better tailor their programs to local needs and the local customer base and alter some of the program features that have proved an impediment in their service areas (e.g., minimum project sizes). 3. Other proposals from the LDCs include merging the new construction programs with other programs that have enjoyed greater success (e.g., a merger of the RNC with the Residential HVAC program or combining the HPNC with the Commercial Retrofit program), or converting these programs into simple prescriptive rebate programs for the key stand-alone measures. Some parties expressed concerns about any potential dramatic changes in program structures, recalling the confusion and resulting drop in participation following the transition in early 2011 to initiatives requiring much greater involvement by the LDCs. While these parties agree that program refinements should continue, they emphasize that changes should be gradual, so that entirely new program infrastructure and new educational programs need not be developed all in one year. A hybrid approach may also prove viable. OPA could consider a strategy whereby one or more provincewide program implementers market the program to national or province-wide chain accounts and/or larger builders, while the LDCs leverage their local presence and customer relationships to foster participation among their on potential participants with a more local presence. Based on Frontier s experiences outside of the province and understanding of the success of the earlier new construction programs in Ontario, any of these directions could prove successful. Stakeholders, Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 13

14 Executive Summary including the electric LDCs, the natural gas distribution utilities, the OPA, the building community, and others must be actively involved the design of any new or revised initiatives. General Recommendations While the future direction of new construction energy efficiency programs is debated, a number of nearterm steps are possible to improve the present HPNC and RNC programs. Incentive Levels Consider raising the incentive levels, especially for the RNC. Eighty percent of HPNC and RNC participants surveyed would have included the energy efficient measures if incentives had not been available, indicating that the incentive levels are not motivating participants who had no original intention to install the energy efficiency measures. Allow certain efficiency measures presently eligible for a prescriptive incentive in the HPNC program to be eligible for an incentive through the retrofit program. For these new construction measures, current code baselines would need to be used as the basis for calculating savings and incentive levels for certain measures within the expanded retrofit program. Project Sizes Consider lowering minimum project size requirements, so that smaller projects (and smaller commercial or business energy consumers) can receive an incentive through the HPNC. Program Design Consider alternative approaches to promoting the RNC. For example, providing payments to home energy raters, rather than home builders, has proven effective in some other markets. Explore how requests for waivers from program requirements could be addressed in a more expeditious manner. Informational Materials A program manual or more informative website is necessary to explain the programs and clear up lingering confusion on key issues such as what constitutes the completion of a building construction project and whether building permits are a program requirement. Provide information up front to the participant that fully explains the timeline of the project and the potential data requests that the participant may need to complete. A well-written program manual may provide this necessary information. Tracking System Frontier s process evaluation revealed a clear need to address discrepancies between data available to OPA and the data the LDCs maintain about the projects they have underway. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 14

15 Executive Summary Consider investing in a tracking system for both programs that is streamlined with a clearly outlined process that both LDCs and the OPA can use to track applications. OPA should also consider requiring periodic reporting from the LDCs on projects underway to assist OPA in anticipating program activity levels. Assign Project IDs and Site IDs to each project. Document Requirements Frontier recommends the following additional required document submittals for HPNC custom projects - (1) the final commissioning report and (2) the as-built control diagrams from the building automation contractor. These documents would be submitted when the project is complete and would allow reviewers to confirm that the building automation contractor had furnished controls implementing the specified sequence of operations and that the facility operated as designed when placed in service. For the RNC program, the team suggests that the OPA provide baseline and change case energy consumptions and housing characteristics (square footage, number of floors, heating type and capacity, HVAC type and capacity, insulation levels, water heating type and capacity) in order to provide additional assistance with verifying energy and demand savings. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 15

16 Introduction INTRODUCTION Evaluation Goals and Objectives Frontier Associates was hired to assess the overall impacts and delivery effectiveness of the OPA s HPNC and RNC initiatives. The goals and objectives of this evaluation are as follows: Impact Evaluation o Assess the gross and net energy and demand savings attributed to the Programs through verification of gross savings. Process Evaluation o Determine the overall effectiveness and comprehensiveness of the Programs by assessment of the effectiveness of the programs market delivery, audit of OPA s program tracking systems, review of customer motivations, and assessment of direct employment effects. Program Descriptions The following program descriptions section will provide an overview and a program process diagram for each program. High Performance New Construction The HPNC program is designed to encourage the construction of energy efficient buildings above the Ontario Building Code by allowing commercial customers to participate in three incentive tracks. Participants in this program are facility managers and owners. Table 4 provides an overview of the HPNC program. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 16

17 Introduction Table 4: HPNC Overview HPNC Track Prescriptive Description Incentives Eligible Measures Set incentive for preapproved technologies Lighting: up to $400/kW Non-lighting: up to $800/kW Appliances: $75/unit -Lighting -Unitary AC Equipment -Hot Water Alternative Energy Measures -Motors -Multi-Residential In-Suite Appliances -Synchronous Belts -Variable Frequency Drives -Agribusiness specific measures Engineered Preset calculation worksheets for a variety of measures Lighting: up to $400/kW or $0.05/kWh Non-lighting: up to $800/kW or $0.10/kWh -Directional Lamps -Exterior Lighting -High Bay Lighting -Interior Lighting -Unitary AC Equipment Custom (Design) Provides incentives to participants who develop energy and demand savings from modelling software < 25% above Code: $50/kW or $ /kWh > 25% above Code: $100/kW or $0.0125/kWh > 50% above Code: $150/kW or $ /kWh Mix of measures; savings determined via modelling software Custom (Modelling) Provides incentives to participants who develop energy and demand savings from modelling software Less of $10,000 or 100% of third costs to prepare the Simulation Summary Report Mix of measures; savings determined via modelling software Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 17

18 Introduction Figure 1 provides a diagram of the process for participating in the HPNC program. Figure 1: HPNC Program Process Diagram Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 18

19 Introduction Residential New Construction The RNC program is designed to encourage residential homebuilders to construct energy efficient buildings by participating in one of three tracks. Participants in this program are homebuilders. The three tracks are prescriptive, performance, and custom. Prescriptive Measures Table 5: Residential New Construction Program Incentive Tracks All Off Switch ENERGY STAR Qualified Central Air Conditioner (CAC) High efficiency furnace with a fully variable speed electronically commutated motor (ECM) Lighting Control Products ENERGY STAR Qualified Niche Lighting ENERGY STAR Qualified indoor light fixtures (hardwired) Prescriptive Track Description Master switch that controls multiple electrical sockets in multiple locations in the home (hard wired) Minimum 15 SEER and 12.5 EER as identified by the OPA Prescriptive Incentive $50 $30 High efficiency as identified by the OPA $50 Hard-wired indoor and outdoor timers and motion sensors Hard-wired dimmer switches ENERGY STAR Qualified recessed lighting must have GU24 replacement ENERGY STAR Qualified under the counter lighting ENERGY STAR Qualified LED lighting 1 or 2 sockets 3 or more sockets Performance Track $3 $15 $3 $10 Performance Measures Description Incentive Performance EnerGuide 83 or 84 rating $500 Performance EnerGuide 85 or better $1,000 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 19

20 Introduction Custom Track Custom Measures Description Incentive Projects not eligible for Prescriptive or Performance incentives. Measures must meet following criteria: Target space heating, space cooling, building envelope or water heating end uses; lighting and appliance projects are excluded Must be more efficient than Ontario Building Code Custom All machinery, equipment, parts, fixtures, and any other accessories or items associated with the custom measure are commercially reasonable equipment and the costs thereof are commercially reasonable Greater of $800/kW or $0.10/kWh, not-toexceed 50% of total project costs Have a positive Total Resource Cost (TRC) test Cannot be a pilot or demonstration project of unproven results. Cannot be part of OPA s Feed-In Tariff Program. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 20

21 Introduction Figure 2 provides a diagram of the process for participating in the RNC program. Figure 2: RNC Program Process Diagram Program Purpose and Goals The program logic diagram below explains the how the HPNC and RNC programs should be operating to achieve the outcomes of long-term capacity and carbon emission reductions and market transformation. The HPNC and RNC programs are put in motion by the Ministry of Energy s goals for energy efficiency, funding from the Ministry of Energy, and Canadian laws that regulate utilities. The OPA and the LDCs then design, plan and implement the programs, attracting participants from the LDCs customer base. Participation results in cost-effective programs that provide energy and demand reduction and reduced electricity bills. However, external influences affect the success of these programs. These external Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 21

22 Introduction influences include Ontario building codes, new technology development, shifting technology baselines, the new construction market, and the economy. Figure 3: Program Logic Diagram for HPNC and RNC Programs Report Overview The Executive Summary provides a summary description of the program and its impacts, as well as the key recommendations from this evaluation. The report itself includes an Introduction, Impact Evaluation, Process Evaluation, Conclusion and Recommendations, and Appendix. The report is organized to focus on the four tasks as specified by the OPA. 1. Assessment of the Effectiveness of the Programs Market Delivery 2. Review of Customer Motivations 3. Audit of the OPA s Program Tracking Systems 4. Assessment of Direct Employment Effects Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 22

23 Introduction In addition, this report also includes sections that cover Incentives Analysis, the New Construction Market, and Program Design. The Appendix provides the surveys used to get feedback from participants and a glossary of terms. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 23

24 Impact Evaluation IMPACT EVALUATION This Impact Evaluation serves to describe the methodologies used to estimate the average per project reduction in electricity demand as well as the overall gross and net energy and demand savings delivered through the High Performance New Construction (HPNC) and Residential New Home Construction (RNC) Initiatives. Methodology Specific detail of Frontier s methodologies for each initiative and track (prescriptive, performance, or custom) is given in the sections that follow this executive summary. Generally, Frontier Associates ( Frontier or The Evaluation Team ) employed the following methodology: 1. The Evaluation Team gathered project information from the OPA, the local distribution companies (LDCs), and participants. 2. Project site meter data was not available, and Frontier used previously established load profiles provided by the OPA. 3. Frontier reviewed the project information and established appropriate baselines and per unit input assumptions for each prescriptive measure. 4. The Evaluation Team mapped energy consumption to EnerGuide scores for evaluation of the performance-based projects completed under the RNC program. 5. For the custom projects, Frontier reviewed primarily the computer modelling summary reports for reasonableness of the modelling rationale and the calculated reductions in demand and consumption for the reference design versus proposed design. 6. Frontier used the OPA s definition of average peak demand savings to estimate average peak demand savings The Evaluation Team summed the savings estimates for individual projects to obtain programlevel gross savings. 8. For net savings, a net-to-gross (NTG) ratio of 0.49 should be applied to the gross 2011 and 2012 results Prescriptive and Quasi Prescriptive Measures and Assumptions, Appendix A: Peak Demand Savings Methodology. 4 Cross-Cutting Evaluation of OPA s Commercial New Construction Activities 2008 to 2010, Navigant Consulting. December Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 24

25 Impact Evaluation Results Savings for the new construction initiatives are dominated by the HPNC program, contributing more than 99% of total savings. Roughly 65% of savings come from the HPNC Custom track, and 34% of savings from the HPNC Prescriptive track. Figure 4 shows this breakdown in savings contribution by initiative. Total annual energy and summer peak coincident demand savings for each initiative and track are summarized in Table 6. Figure 4: Impact Evaluation - Program Contribution to Energy and Demand Savings RNC, 1% HPNC Prescriptive Track, 65% HPNC, 99% HPNC Custom Track, 34% Table 6: Impact Evaluation - Gross Energy and Demand Savings by Initiative Initiative Track Gross Demand Savings (kw) Gross Energy Savings (kwh) HPNC Prescriptive 1,195 3,580,613 HPNC Custom 2,306 8,148,078 RNC Prescriptive RNC Performance ,312 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 25

26 Impact Evaluation: HPNC Program Discussion by Program: High Performance New Construction Frontier s evaluation of the HPNC Program consisted of 63 projects with 91 measures completed in the 2011 and 2012 program years. Of those projects, 56 submitted applications under the prescriptive track and 7 submitted applications under the custom track. Figure 5 shows the breakdown of demand savings for the HPNC initiative: Figure 5: Impact Evaluation - HPNC Gross kw Savings by Track and Measure HPNC Custom 65.86% HPNC Prescriptive 34.14% Agribusiness Ventilation 24.16% Unknown 5.94% Other 0.36% Lighting 2.81% Unitary AC 0.87% HPNC: Methodology Prescriptive Track Frontier was charged with reviewing prescriptive savings for projects completed in 2011 and 2012, including assumed baselines and change cases, savings calculation methodologies, and associated assumptions. Key changes to assumptions are outlined as follows: Air Cooled Unitary AC Equipment Savings were calculated as specified in the 2011 Quasi-Prescriptive Measures and Assumptions List (MAL) Version 1.0, except that baseline system efficiencies were determined by system size and configuration rather than using a single assumed baseline. Installed system efficiencies were used instead of the assumed change cases, and EFLH were determined by building type (if specified) rather than using a single value of 1,000 EFLH. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 26

27 Impact Evaluation: HPNC Program Ground Source Heat Pumps Savings were calculated as specified in the HPNC Prescriptive Input Assumptions Version 2.0. These savings were not verifiable using the methodology outlined in the MAL, as the listed methodology appeared to always result in negative savings. However, because installed systems meet the specified baseline and change case specifications from the prescriptive assumptions, specified savings will be accepted for projects installed in 2011 and The GSHP savings calculation methodology from the MAL will need to be reviewed in greater detail before evaluation of GSHP installations for subsequent years. In-Suite Temperature Controls Due to minimal participation, savings specified in the prescriptive assumptions are accepted. Savings calculation methodology may need to be reviewed in greater detail for subsequent years. Recirculation Ventilation HVLS Fans Savings and operating hours specified in the MAL and prescriptive assumptions are accepted. Deemed consumptions may need to be reviewed or validated through metering for subsequent years. Dual and Natural Exhaust Ventilation (Dairy Tie-Stalls) Savings and operating hours specified in the MAL and prescriptive assumptions are accepted. Deemed consumptions may need to be reviewed or validated through metering for subsequent years. T5/T8 Medium and High Bay Fixtures Energy savings were determined as specified in the MAL. While the specified baseline technology was appropriate, the baseline wattages were revised based on a review of available products. The baseline wattage for a 250 W Metal Halide was lowered from 295 W to 290 W. The baseline wattage for a 400 W Metal Halide was lowered from 460 W to 450 W. Luminosity correction factors were determined to be appropriate. When available, installed fixture system wattages were used in place of assumed system wattages from the MAL. If there was a discrepancy between the installed fixture system wattages from the MAL and those from the prescriptive assumptions, the value from the MAL was used. EFLH were determined by building type (if specified) rather than using the single assumed value. Demand savings determined using the methodology specified in the MAL were determined to be too low and did not seem to match the deemed demand savings from the prescriptive assumptions. Therefore, Frontier calculated demand savings using an assumed 0.77 coincidence factor for all measures. This coincidence factor is based on a weighted average of multiple building types. Site specific coincidence factors may be used for subsequent years. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 27

28 Impact Evaluation: HPNC Program The MAL specified that interactive effects were not accounted for in the prescriptive deemed savings. Frontier chose to include a conservative 10% demand savings multiplier and 5% energy savings multiplier to account for interactive effects. Greater contributions from interactive effects should be verified through site specific metering. Other T5/T8 Linear Fluorescent Fixtures Energy savings were determined as specified in the MAL. Baseline technologies, wattages, and luminosity correction factors were determined to be appropriate. When available, installed fixture system wattages were used in place of assumed system wattages from the MAL. If there was a discrepancy between the installed fixture system wattages from the MAL and those from the prescriptive assumptions, the value from the MAL was used. EFLH were determined by building type (if specified) rather than using the single assumed value. Demand savings calculated using the methodology specified in the MAL were determined to be too low and did not seem to match the deemed demand savings from the prescriptive assumptions. Therefore, Frontier calculated demand savings using an assumed 0.77 coincidence factor for all measures. This coincidence factor is based on a weighted average of multiple building types. Site specific coincidence factors may be used for subsequent years. The MAL specified that interactive effects were not accounted for in the prescriptive deemed savings. Frontier chose to include a conservative 10% demand savings multiplier and 5% energy savings multiplier to account for interactive effects. Greater contributions from interactive effects should be verified through site specific metering. Pin Socket CFLs/GU-24 Lamps Savings specified for this measure in the HPNC Prescriptive Input Assumptions Version 1.0 were later removed in version 2.0 of the prescriptive assumptions because CFLs were determined to be the accepted best practice. Frontier agrees that CFLs should be the assumed baseline for this measure, so no savings were claimed for this measure. LED Par Lamps Savings were determined as specified in the MAL. Baseline technologies, wattages, and luminosity correction factors were determined to be appropriate. When available, installed fixture system wattages were used in place of assumed system wattages from the MAL. If there was a discrepancy between the installed fixture system wattages from the MAL and those from the prescriptive assumptions, the value from the MAL was used. EFLH were determined by building type (if specified) rather than using the single assumed value. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 28

29 Impact Evaluation: HPNC Program Demand savings calculated using the methodology specified in the MAL were determined to be too low and did not seem to match the deemed demand savings from the prescriptive assumptions. Therefore, Frontier calculated demand savings using an assumed 0.77 coincidence factor for all measures. This coincidence factor is based on a weighted average of multiple building types. Site specific coincidence factors may be used for subsequent years. The MAL specified that interactive effects were not accounted for in the prescriptive deemed savings. Frontier chose to include a conservative 10% demand savings multiplier and 5% energy savings multiplier to account for interactive effects. Greater contributions from interactive effects should be verified through site specific metering. Frontier was unable to locate an OPA specified EUL for this measure. Therefore, an EUL of 9 years was used for integrated-ballast LED lamps. This EUL is based on a 30,000 hour manufacturer rated life and weighted average 2,360 annual operating hours from the 2003 U.S. DOE Lighting Market Characterization Study. ENERGY STAR Refrigerators Energy savings were determined using the calculation methodology and associated assumptions from the current ENERGY STAR appliance calculator. Demand Savings were determined by dividing the energy savings by 8,760 hours and applying a 1.22 demand factor. Pacific Northwest National Laboratory (PNNL) estimates that the peak monthly load index (ratio of peak monthly consumption to average monthly consumption) for this measure is The average load index for hours is The product of these values gives an average load index of 1.22 for peak hours during August. ENERGY STAR Clothes Washers Energy savings were determined using the calculation methodology and associated assumptions from the current ENERGY STAR appliance calculator. Demand savings were determined by dividing the energy savings by 312 hours (annual operating hours assumption from ENERGY STAR calculator) and applying a 0.05 coincidence factor. This coincidence factor was taken from a 2009 National Renewable Energy Laboratory (NREL) Building America Research Benchmark study based on a peak hour of 4 PM. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 29

30 Impact Evaluation: HPNC Program ENERGY STAR Dishwashers Energy savings were determined using the calculation methodology and associated assumptions from the current ENERGY STAR appliance calculator. Demand savings were determined by dividing the energy savings by hours (annual operating hours assumption from ENERGY STAR calculator) and applying a coincidence factor. This coincidence factor was taken from a 2009 National Renewable Energy Laboratory (NREL) Building America Research Benchmark study based on a peak hour of 4 PM. ENERGY STAR Ceiling Fans Energy savings were determined using the calculation methodology and associated assumptions from the current ENERGY STAR appliance calculator. The MAL specified baseline assumption of three (3) 60 W incandescent lamps was determined to be appropriate. The MAL specified change case of three (3) 20 W CFLs was changed to one (1) 32 W CFL based on ENERGY STAR calculator assumptions. The MAL specified 2.7 hours/day operation was changed to 3 hours/day operation based on ENERGY STAR calculator assumptions. Demand savings were determined as specified in the MAL. HPNC: Results Prescriptive Track The Evaluation Team s review of the available documents indicates that most of the calculation methodologies and assumptions listed in the 2011 Quasi-Prescriptive Measures and Assumptions List Version 1.0 were reasonable. Where applicable, Frontier made adjustments to baselines and changes cases, as specified in the Methodology section. The table of changes is available in Appendix D. The main correction that Frontier made to estimated savings was to apply building specific operating hours (if possible), fixture specific wattages for lighting measures (where available), and updated assumptions from ENERGY STAR calculators for appliance measures. Site specific metering may be used to validate various measures in future evaluations. There were 84 projects completed under the HPNC prescriptive track. Due to missing data in the information provided, Frontier was unable to identify which measures were installed and those savings are reported under the unknown measure type. Savings are largely driven by agribusiness ventilation measures which contributed roughly 70% of total savings under the prescriptive track. Savings per measure are presented in Table 7. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 30

31 Impact Evaluation: HPNC Program Table 7: Impact Evaluation - HPNC Prescriptive - Gross Energy and Demand Savings by Measure Measure Gross Demand Savings (kw) Gross Energy Savings (kwh) Appliances ,866 GSHP 4.7 4,709 Temperature Controls 6 5,451 Unitary AC 31 40,353 Lighting ,478 Ventilation 846 2,455,616 Unknown ,140 HPNC: Methodology Custom Track Frontier was charged with reviewing the computer modelling for reasonableness of the modelling rationale and the calculated reductions in demand and consumption for the reference design versus proposed design. These custom projects required computer simulation modelling utilizing approved software to demonstrate the difference in annual energy consumption between the OBC-compliant reference building design and the proposed design with energy efficiency improvements. The applicant is required to submit the following project documentation in electronic format, which Frontier would utilize in the evaluations: 1. Custom worksheet 2. Computer simulation report (with backup appendices) 3. Energy model files 4. Energy demand savings summary (hourly simulation output) 5. Drawings, specs, manufacturer s data, building control sequences 6. Post-project completion form For the completed custom projects, the first two items on the list above, less the simulation report appendices, were made available by OPA for the Evaluation Team s review. The other items were generally not available to us. Therefore, Frontier was only able to perform a high level review for reasonableness of modelling rationale and calculated results, considering the type and size of building, application and measures installed. The objective was to render a judgment on whether the modelling represented code-compliant reference designs, representative proposed designs, and resulting differential energy consumption and demand. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 31

32 Impact Evaluation: HPNC Program HPNC: Results Custom Track Frontier reviewed 7 projects under the HPNC custom track, with one project contributing 70% of the total demand savings and 85% of the total energy savings for the custom projects. For that project, the energy model files and hourly consumption output were not available for Frontier s review. Frontier s review of the available documents for these projects indicates that the modellers used good practice in modelling architectural and mechanical, electrical and plumbing systems for energy consumption, yielding reasonable energy and demand savings. Frontier does not recommend any changes in the savings figures, based on the information available to the Evaluation Team. If desired by OPA in ongoing reviews, and if given access to the building model files and entire simulation summary report with appendices, Frontier will open the building model files, review the modelling in detail and check the inputs and outputs against the other documents. Table 8: Impact Evaluation - HPNC Custom - Gross Energy and Demand Savings Track Gross Demand Savings (kw) Gross Energy Savings (kwh) Custom 2,306 8,148,078 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 32

33 Impact Evaluation: RNC Program Discussion by Program: Residential New Construction Frontier s evaluation of the RNC Program consisted of 44 projects completed in the 2011 and 2012 program years. Of those projects, 41 submitted applications under the performance path and 3 submitted applications under the prescriptive path. Figure 6 shows the contribution to RNC demand savings by track. Figure 6: Impact Evaluation - RNC Gross kw Savings by Track Prescriptive 6% Performance 94% RNC: Methodology Prescriptive Track Frontier was charged with reviewing prescriptive savings for projects completed in 2011 and 2012, including assumed baselines and change cases, savings calculation methodologies, and associated assumptions. Key changes to assumptions are outlined as follows: SEER 15 Central Air Conditioner Savings were calculated as specified in the 2011 Prescriptive Measures and Assumptions List (MAL) Version 1.0. Installed system efficiencies were used instead of the assumed 15 SEER change case. Gas Furnace with ECM In the 2011 Prescriptive MAL, deemed Gas Furnace with ECM savings are presented for continuous and non-continuous fan usage. Savings are also specified for space heating only or for space heating and cooling. OPA provided weighted energy and demand savings that were derived using 2011 retrofit participation values. However, due to the small number of measures completed in the Residential New Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 33

34 Impact Evaluation: RNC Program Construction program, these weighted savings are not applicable, as a representative sample size was not achieved. Demand savings were taken directly from the MAL. If the Gas Furnace with ECM measure was completed at the same time as the SEER 15 Central Air Conditioner, the measure is assumed to be for space heating and cooling. Otherwise, the measure was assumed to be for space heating only. Because fan usage was not specified anywhere in the project submittals, fan usage was assumed to be noncontinuous. Energy savings were estimated at 50% of fan energy during heating operation. Independent field studies quantifying electrical savings from variable speed furnace fans are few in number. Additionally, realized savings are subject to climate zone, manufacturer-specific controls design, system static pressure, and homeowner behavior. An exhaustive field study by the Energy Center of Wisconsin found that fan power savings from ECM were 50% during heating operation and 30% during cooling operation. Applying a 0.5 savings factor is a more credible predictor of actual savings than rigorous simulations of multi-stage furnace and variable speed fan central heating systems. Frontier calculated energy savings by applying the 0.5 factor against the AHRI specified EAE, or average annual auxiliary electrical energy consumption for gas furnaces in kwh/year. The 30% savings during cooling operation was ignored, as fan usage is already taken into account in determining the efficiency ratings of the cooling HVAC equipment. Increases in HVAC cooling efficiencies are already taken into account in determining savings for the 15 SEER Central Air Conditioner measure. RNC: Results Prescriptive Track There were 3 projects submitted under the RNC prescriptive track, with 2 of those also submitting applications under the performance track. Because the performance track savings are presented on a whole-home basis, the savings for the prescriptive measures are included only in the performance results and excluded from the prescriptive savings totals. Table 9 summarizes the total savings for projects completed under the RNC prescriptive track. Table 9: Impact Evaluation - RNC Prescriptive - Gross Energy and Demand Savings by Measure Measure Gross Demand Savings (kw) Gross Energy Savings (kwh) Gas Furnace with ECM Unitary AC Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 34

35 Impact Evaluation: RNC Program RNC: Methodology Performance Track The documentation provided for the performance projects generally consisted of the project application, the project invoice, and the EnerGuide score. There were 8 projects for which no EnerGuide score was given, but based on the incentive amount of $500 Frontier assumed an EnerGuide score of 83 for those projects. Frontier contacted LDCs and builders to request additional project documentation, and for 28 projects Frontier was provided with the EnerGuide Energy Efficiency Evaluation Report which gives the estimated energy consumption of the home by fuel type. Six months of hourly meter data was available for 2 of the project homes. For 17 projects, general information on the envelope, space heating, and water heating equipment characteristics were provided. However, Frontier lacked information on the measures contributing to the electric savings, such as appliances and lighting. Frontier calculated baseline energy consumption by applying the Energy Efficiency Rating Calculation Procedure from the EnerGuide New Homes Administrative and Technical Procedures Manual. 5 The value for benchmark total energy consumption was used as the total baseline energy consumption for each project. Because measure information was incomplete, in order to make a determination regarding how much of the energy savings is attributable to electric measures, Frontier multiplied the percent total savings by the percent estimated electric consumption. (1) Frontier assumes that the percent benchmark electric consumption is the same as the percent estimated electric consumption. The reported energy savings are the result of subtracting the estimated electric consumption from the benchmark electric consumption. For projects where neither the site meter data nor the estimated consumption were available, Frontier calculated benchmark total energy consumption by assuming an average floor area of 129 m 2 and 0.79 GJ/m 2 as the average energy use index. 6 According to the EnerGuide for Houses Database Analysis report, 7 approximately 12% of homes use electric heat, and Frontier applied this percentage to the project homes. For homes assumed to use non-electric heating, Frontier multiplied the benchmark total (2) 5 EnerGuide for New Houses: Administrative and Technical Procedures. January Appendix A: Energy Efficiency Rating Calculation Procedure. 6 Natural Resources Canada Report: Energy Efficiency Trends in Canada 1990 to December Canadian Residential Energy End-use Data and Analysis Centre: EnerGuide for Houses Database Analysis. April Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 35

36 Impact Evaluation: RNC Program energy consumption by 38%, which is the average percent estimated electric consumption of homes where estimated energy consumption was provided. The percent total savings was determined based on the project s EnerGuide score as shown in Table 10: Table 10: Impact Evaluation - RNC Performance - Savings by EnerGuide Score EnerGuide Score Percent Total Savings 83 15% 84 20% 85 25% 86 30% 87 35% For the projects where site metered data was available, Frontier extrapolated the available data to populate an annual consumption profile. Frontier adjusted the estimated electric consumption to match the profile, and applied the percent electric savings to produce the reported annual savings value. Average peak demand savings were calculated according to the 2011 Prescriptive Measures and Assumptions, Appendix A: Peak Demand Savings Methodology. For performance projects, where the measure information was not provided, Frontier averaged the seasonal energy savings patterns (SESP) and coincidence factors for electric appliances and lighting. Frontier excluded space cooling because the EnerGuide Energy Efficiency Evaluation Reports did not include cooling in the estimated end uses for the project homes. The resulting percent SESP used to determine average peak demand was 4.6%, and the coincidence factor was RNC: Results Performance Track Performance projects submitted ranged in EnerGuide score from 83 to 87, with the majority falling at a score of either 83 or 85 which are the minimum score thresholds to receive an incentive. Savings results for the RNC performance track are presented in Table 11. Table 11: Impact Evaluation - RNC Performance - Gross Energy and Demand Savings by EnerGuide Score EnerGuide Score Project Count Gross Demand Savings (kw) Gross Energy Savings (kwh) , , ,074 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 36

37 Impact Evaluation: RNC Program EnerGuide Score Project Count Gross Demand Savings (kw) Gross Energy Savings (kwh) , ,258 Recommendations Savings for the new construction initiatives are dominated by the HPNC program, contributing more than 99% of total savings. Roughly 66% of savings come from the HPNC Custom track, and 34% of savings from the HPNC Prescriptive track. Table 12: Gross Energy and Demand Savings by Initiative Initiative Track Gross Demand Savings (kw) Gross Energy Savings (kwh) HPNC Prescriptive 1,195 3,580,613 HPNC Custom 2,306 8,148,078 RNC Prescriptive RNC Performance ,312 A review of the available documents indicated that the most of the calculation methodologies and assumptions listed in the 2011 Quasi-Prescriptive Measures and Assumptions List (MAL) Version 1.0 were reasonable. Where applicable, Frontier made adjustments to baselines change cases, and assumptions to improve the accuracy of prescribed savings estimations. Frontier recommends the following changes to prescriptive savings estimation strategies: 1. All Measures: a. Utilize building specific operating hours rather than assuming a single value for operating hours for all projects. These hours are already available in the MAL. 2. HVAC Retrofit Measures: a. Utilize project specific (installed) system efficiencies rather than assumed change case efficiencies. 3. Lighting Retrofit Measures: a. Utilize project specific (installed) fixture wattages rather than assumed change case wattages. b. Decrease baseline wattage for high bay lighting fixtures as outlined in Appendix D. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 37

38 Impact Evaluation: RNC Program c. Do not claim savings for CFL measures in the HPNC program. OPA removed this measure from Version 2 of the prescriptive assumptions, which is used for projects applying for permits on or after January 1, ENERGY STAR Appliance Measures: a. Utilize updated kwh savings for ENERGY STAR Refrigerators, Clothes Washers, and Dishwashers as outlined in Appendix D. These savings estimations were calculated using an updated version of the ENERGY STAR appliance calculator. A single project contributed 70% of the total demand savings and 85% of the total energy savings for the HPNC custom track. For that project, the energy model files and hourly consumption output were not available for review. A review of the available documents for other custom projects indicates that the modellers used good practice in modelling architectural and mechanical, electrical and plumbing systems for energy consumption, yielding reasonable energy and demand savings. Based on available information, the Evaluation Team does not recommend any changes to the custom savings estimations. Frontier recommends the following additional required document submittals for custom projects - (1) the final commissioning report and (2) the as-built control diagrams from the building automation contractor. These documents would be submitted when the project is complete and would allow reviewers to confirm that the building automation contractor had furnished controls implementing the specified sequence of operations and that the facility operated as designed when placed in service. Only 3 projects were submitted to the RNC Prescriptive track, with a minimum impact on overall program savings. Some measure assumptions were not utilized because the assumptions were deemed to be invalid based on such a small project sample size. In these cases, project specific information was used instead. RNC Performance projects submitted ranged in EnerGuide score from 83 to 87, with the majority falling at a score of either 83 or 85, the minimum score thresholds to receive an incentive. While the OPA and LDCs were generally responsive to requests for information, there were some inconsistencies and missing project information as described above. Therefore, the results of the impact evaluation and cost-effectiveness tests should be reviewed with the understanding that specific project information may not be entirely aligned with actual projects completed in the 2011 and 2012 program years. Overall, the HPNC initiative is driving the energy and demand savings for OPA s new construction initiatives. Specifically, one custom project contributed roughly 60% of total new construction initiative savings. Agribusiness ventilation is also a primary driver of savings for the prescriptive projects. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 38

39 Process Evaluation: HPNC Program PROCESS EVALUATION Frontier Associates assessed the delivery effectiveness of the OPA s HPNC and RNC initiatives. The goals and objectives of this process evaluation are to determine the overall effectiveness and comprehensiveness of the Programs by assessment of the effectiveness of the programs market delivery, audit of OPA s program tracking systems, review of customer motivations, and assessment of direct employment effects. This section is divided into the following high-level sections: 1. High Performance New Construction a. Assessment of the Effectiveness of the Programs Market Delivery b. Review of Customer Motivations c. Incentive Levels Analysis d. New Construction Market 2. Residential New Construction a. Assessment of the Effectiveness of the Programs Market Delivery b. Review of Customer Motivations c. Incentive Levels Analysis d. New Construction Market 3. Audit of the OPA s Program Tracking Systems (Both HPNC and RNC) 4. Assessment of Direct Employment Effects (Both HPNC and RNC) 5. Program Design & Other Recommendations (Both HPNC and RNC) The two program-specific sections cover topics including Assessment of the Effectiveness of the Programs Market Delivery, Review of Customer Motivation, and Incentive Level Analysis. Those sections include methods, results, discussions, and recommendations as applicable. The three final sections address special topics with relevance to both the HPNC and the RNC. High Performance New Construction Frontier s process evaluation covers four dimensions of HPNC program performance: Program Market Delivery Effectiveness, Customer Motivation, Incentive Levels, and Penetration of the New Construction Market. HPNC: Assessment of the Effectiveness of the Program s Market Delivery The Evaluation Team conducted a comprehensive process review to provide feedback on the design, delivery, and quality of the HPNC initiative. This review included surveys of participants and modeling firms and interviews with LDCs. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 39

40 Process Evaluation: HPNC Program Methodology Frontier assessed the delivery of the HPNC program by interviewing LDCs, participants, and firms involved in the modelling of custom projects. Frontier interviewed representatives from 11 LDCs. Seven of the interviews were conducted in-person, while the remaining interviews were conducted via telephone. Frontier also conducted eleven surveys of participants and seven surveys of modellers or salespersons involved in the application process. The survey instruments are available in the Appendices. These surveys were administered over the phone by Frontier personnel. Table 13 shows the total number of participants and modellers or salespersons surveyed compared to the total number in the program. Table 13: Process Evaluation - HPNC Surveys Survey Recipients Number Surveyed Total Number in Program Percentage Surveyed Participants % Modellers or Salespersons % Results Survey participants rated the importance of various program design factors in encouraging program participation. Figure 7 shows the importance of contact from the OPA, contact from the LDC, OPA and LDC informational literature, and the availability of the incentive as factors in deciding to participate in the HPNC program. Survey participants were asked to rate these factors on a scale of 1 to 5 with a 5 representing a great impact and 1 representing a negligible impact. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 40

41 Process Evaluation: HPNC Program Figure 7: Process Evaluation HPNC Participants' Ratings of the Importance of Certain Factors Availability of Incentive 4.2 LDC Informational Literature and Advertising 3.3 Contact by LDC's Customer Representatives 3.3 OPA Informational Literature and Advertising 2.5 Contact by OPA's Customer Representatives While the incentive was the most important factor, survey responses also show that LDC communications, whether marketing or direct contact from LDC representatives, were more important than OPA outreach. When asked whether they found the application process to be reasonable and understandable, 75 percent of participants and modellers or salespersons assented. However, 25 percent found the application process to be difficult to understand. Figure 8 provides a breakdown of the responses. Figure 8: Process Evaluation HPNC Response When Asked Whether Application Process Was Reasonable and Understandable No, 3 Yes, 9 Survey recipients were asked to rate their satisfaction with the application process and the ease of completing the application on a scale of 1 to 5, with one representing dissatisfaction and five representing the highest level of satisfaction. Figure 9 shows the responses of participants and modellers or salespersons that have filled out the applications for the HPNC program. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 41

42 Process Evaluation: HPNC Program Figure 9: Process Evaluation HPNC Participant and Modeller or Salesperson Opinions on Application Process Satisfaction with Application Process 3.4 Ease of Application Completion While some respondents did complain about the complexity of the application process (as discussed below), overall the survey indicates moderate satisfaction with the application process. Recirculation ventilation fans made up the majority of measures installed in the HPNC program in program years 2011 and Figure 10 shows that these fans make up 31 percent of measures installed. Lighting measures make up the second greatest portion of measures installed, and HVAC measures are third. A breakdown of the specific measures installed is provided in Table 14. Figure 10: Process Evaluation - HPNC Measures Installed in PY Gound Source Heat Pump, Closed Loop, 2 Dual and Natural Exhaust Ventilation: Dairy Tie-Stall, 1 In-Suite Temperature Controls: Space Cooling, 1 ENERGY STAR Appliances, 6 Custom, 7 Recirculation Ventilation Fans: HVLS Fans, 28 HVAC, 12 Unknown, 13 Lighting Measures, 21 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 42

43 Process Evaluation: HPNC Program Figure 11: Process Evaluation - HPNC Measures by Business Sector for PY Industrial, 4 Storage, 4 Office, 4 House of Worship, 3 Data Center, 1, 0 Warehouse, 11 Agribusiness, 29 Multi Residential, 13 Retail, 14 8 Please note that these numbers may not add up to the number of projects. Participants often selected more than one building type. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 43

44 Process Evaluation: HPNC Program Table 14: Process Evaluation - HPNC Measures Installed PY Quantity Measure Air-Cooled Unitary AC Equipment, 1.5 ton 1 Air-Cooled Unitary AC Equipment, 17.5 ton 1 Air-Cooled Unitary AC Equipment, 20 ton 2 Air-Cooled Unitary AC Equipment, 3 ton 2 Air-Cooled Unitary AC Equipment, 4 ton 1 Air-Cooled Unitary AC Equipment, 6 ton 1 Air-Cooled Unitary AC Equipment, 6.5 ton 1 Air-Cooled Unitary AC Equipment, 7.5 ton 2 Air-Cooled Unitary AC Equipment, 8.5 ton 1 (Measure not Provided) 13 Custom 7 Dual and Natural Exhaust Ventilation: Dairy Tie-Stall 1 ENERGY STAR Ceiling Fan 1 ENERGY STAR Clothes Washer 1 ENERGY STAR Dishwasher 1 ENERGY STAR Refrigerator 3 Ground Source Heat Pump, Closed Loop 2 In-Suite Temperature Controls: Space Cooling 1 Lighting: High Performance Medium Bay T8 Fixtures 3 Lighting: High Performance T8 Fixtures 3 Lighting: LED PAR Lamps 1 Lighting: Occupancy Sensors 1 Lighting: Pin Socket CFL - 4-Pin Lamps 2 Lighting: Pin Socket CFL - GU-24 Lamps 4 Lighting: T5 Medium and High Bay Fixtures 7 Recirculation Ventilation Fans: HVLS Fans 28 Grand Total 91 9 Please note that these numbers may not add up to the number of projects. Participants often selected more than one building type. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 44

45 Process Evaluation: HPNC Program Discussion The LDCs and participants reported the following key concerns with the present design of the HPNC: Minimum Project Size The minimum project size requirements preclude a number of smaller projects and smaller businesses from participating in the program. Application Process Twenty-five percent of participants surveyed found the application forms and contracts to be difficult to understand and unreasonable. Participants commented that application forms are too long and complicated and contain too much legal language. The interviewed modelling firms commented that much of the information required for preapproval of an application may be costly to develop, and furthermore is likely to change as a project undergoes construction. LDCs find that the application is difficult to amend to reflect changes implemented by the builder, tenant, or owner once the application has been completed. Many participants found the amount of paperwork required to be burdensome. o Repeat participants commented about the steep learning curve when they first started filling out applications for the program. This steep learning curve may discourage first time participants from completing the application. Incentive Processing Participants cite widely varying lengths of time to receive their incentive check. Some participants report receiving checks in two to four weeks, while others cite up to six months to a year to receive an incentive check. Permit Requirements Many participants commented that the building and occupancy permit requirements are confusing. One participant had difficulties when he found that the city of the participant did not issue occupancy permits on his building type and that the LDC would not accept a substantially complete permit even though both permits are stated in the contract. One participant found the permitting requirements between LDCs to be inconsistent and that some LDCs require items that weren t clearly identified in the beginning, such as an architect s letter. Another participant cited the building permit process as the biggest hassle and that the necessity of the permit is unclear to him. Many participants agreed that the LDCs need to provide clearer explanations concerning the permitting process and associated timelines, and be more uniform in their permitting requirements across the LDCs in Ontario. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 45

46 Process Evaluation: HPNC Program Incentive Levels The incentives levels are too low, relative to the cost of completing the application process. Seventy-five percent of participants surveyed would have implemented the energy efficient measures if the incentives had not been available the incentive levels are not attracting many participants who had no original plans for installing energy efficient measures. Tracking System OPA staff finds that the absence of a working tracking system makes it difficult for the OPA to monitor the level of activity in the program. From an evaluator s perspective, the lack of unique site and project IDs made the applications and associated paperwork difficult to track down. Inconsistencies in the labeling of project paperwork added work to the process and often created challenges for the evaluators. Overall Program Satisfaction When asked about general satisfaction with the HPNC program, most participants rated their level of satisfaction as moderately to greatly satisfied with the program. Only one participant was highly dissatisfied. This participant was highly dissatisfied with how long the application process to him to complete. The participant stated that he spent about 6 hours working on the application. He also said that he possibly would have participated more if the incentives had been adequate to cover the expenses of participating in the program. Most other participants also cited concerns with the long application process and the complicated paperwork submission processes as problematic and a likely barrier preventing other potential participants from participating. Waivers LDC staff find the process of requesting waivers from program requirements for specific projects from the OPA is very slow (notably, projects which were started during the first generation of the HPNC but not completed until after the beginning of 2011). Some LDCs report that the waiver process has taken over a year, although the OPA reports that waivers typically take about four weeks to process. Program Staff Interaction Some participants cite occasional problems with LDC staff who do not follow up quickly to inquiries. Surveys revealed that many participants were unsure as to whether they were dealing with OPA, LDC, or Enbridge staff. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 46

47 Process Evaluation: HPNC Program Program Future LDCs are concerned about the transition to the next generation of this program. The LDC cannot guarantee an incentive if the new building or project may not be completed until While there is presently a very healthy level of dialogue with the OPA over program enhancements, initially little stakeholder input was included into the program design and processes. LDCs cited high level of staff turnover at the OPA as a concern. Recommendations Please see Program Design & Other Recommendations section for the Program Design, Application Revision, and Blanket Waiver Recommendations. The table below provides recommendations specific to the HPNC program: it also specifies where certain program aspects were more thoroughly explored and more detailed recommendations provided - in other sections of the report. Table 15: HPNC Specific Program Delivery Recommendations Program Aspect Minimum Project Size Application Process Permit Requirements Recommendations Consider lowering the minimum project size requirements to allow smaller projects and business to participate. Consider revising the application and the contract in a manner that clearly outlines the program requirements. o o Some of the information is repetitive and some of the requirements seem unnecessary. The OPA/LDCs should either revisit some of the requirements or make clear why certain items, such as building and occupancy permits, are required. Vital information concerning permitting and resubmission requirements is not presented clearly. The website materials should present clearer timelines and program requirements. See the Program Design & Other Recommendations (both HPNC & RNC) for more specific application recommendations Clarify the building and occupancy permit requirements from the very beginning of the application process. Make these requirements uniform across all LDC territories. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 47

48 Process Evaluation: HPNC Program Incentive Levels Consider raising the incentive levels. See the HPNC: Incentive Levels Analysis section for a benchmarking of incentive levels. Informational Materials Include a contact list for each LDC to assist confused participants. Tracking System Waivers Implement a streamlined tracking system that follows a clear process flow and allows LDCs and the OPA to easily track applications and highlights those projects that need attention. Consider adding project and site IDs. See the Audit of OPA s Program Tracking System (both HPNC & RNC) for more specific recommendations Explore how requests by the LDCs to the OPA for waivers from program requirements could be addressed in a more expeditious manner. See the Program Design & Other Recommendations (both HPNC & RNC) for specific recommendations concerning waivers. HPNC: Review of Customer Motivations Methodology The Evaluation Team analyzed survey results related to participants motivations in participating in the program. The Team also compared the incentive levels of HPNC with other similar programs to determine the adequacy of the incentive levels for motivating participation. Results Participants were asked to rate the importance of a number of factors in deciding to install energy efficient measures on a scale of 1 to 5 (with 5 being very important and 1 being not very important): Maintenance, Building Aesthetics, Rebates, Environmental or Emission Concerns, Initial Project Cost, and Electricity Costs and Expected Energy Savings. Figure 12 shows the importance of these decision making factors to participants when deciding on how to proceed with an energy efficiency project. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 48

49 Process Evaluation: HPNC Program Figure 12: Process Evaluation HPNC Importance of Decision Making Factors to Participants Electricty Costs and Expected Energy Savings 4.7 Maintenance 4.4 Rebates 4.3 Building Aesthetics 3.8 Environmental or Emission Concerns 3.6 Initial Project Cost Four of the surveyed participants reported having corporate policies regarding energy savings for new construction projects, as shown in Figure 13. Figure 13: Process Evaluation HPNC Participants Working Under Energy Efficiency Corporate Policies Yes, 4 No, 5 Participants were asked whether the desire to be perceived as energy efficient or green was an important factor in deciding to include energy efficient measures. Figure 14 shows the responses to this question. Broadly, participants are conscious of community perception: according to the results, 75 percent of participants reported that being perceived as energy efficient or green was a very or fairly important factor in deciding to include energy efficient measures in the new construction project. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 49

50 Process Evaluation: HPNC Program Figure 14: Process Evaluation HPNC Importance of "Being Green" to Participants Very unimportant, 1 Neutral, 1 Very, 3 Fairly, 3 Participants were asked whether they would have pursued the same level of efficiency in the absence of the program or incentive. According to survey results, free ridership may be a concern: 75 percent indicated that they would have pursued the same level of efficiency without the program or incentive, as shown in Figure 15. Figure 15: Process Evaluation HPNC Necessity of Incentive or Program to Participants No, 2 Yes, 6 Figure 16 shows how participants responded when asked whether they would have implemented their project differently had the incentives not been available. Responses to this question also indicate freeridership is a concern: seventy-five percent indicate they would have included the energy efficient measures in the same way if incentives had not been available. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 50

51 Process Evaluation: HPNC Program Figure 16: Process Evaluation HPNC Participant Responses to Scenarios with No Available Incentives 2 Would have done the project the same exact way 6 Would have done the project but would have scaled back certain costs Finally, participants were asked whether they would have had the funds to cover the cost of the energy efficiency measures had the incentives not been available. One hundred percent of the participants indicated that they would have been able to cover the entirety of the energy efficiency measures absent the incentive. HPNC: Incentive Level Analysis Survey results indicate that most participants in the program are already motivated to install energy efficient measures due to corporate energy policies or an interest in being perceived as green. All of the businesses stated that they would have had the funds to install these measures in the absence of the program. Seventy-five percent of survey respondents would have committed to the same project in absence of the program. These results indicate that the HPNC program is not motivating many customers that would not have installed measures without the program. Table 16 compares OPA s HPNC program to similar programs in other jurisdictions. As listed below, in some cases, OPA pays the HPNC incentives according to the kw saved per project. Because the other utility incentives are in dollars per kwh saved, Frontier Associates calculated what OPA would have paid as a $/kwh incentive per project, both prescriptive and custom. Frontier performed this calculation by dividing each project s total incentive paid by the project s kwh saved. This calculation provided an imputed $/kwh incentive to compare the incentive levels with other programs. The highest was a data center project, where the equivalent of $80.39 was paid per kwh saved. However, removing this outlier, the second highest was $3.24 per kwh saved. The lowest was $0.01 per kwh saved. The average was $0.30 per kwh saved. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 51

52 Process Evaluation: HPNC Program As detailed in Table 16, comparisons with the other three commercial new construction programs indicate that the HPNC program offers an incentive level on par with other programs. However, because the HPNC is focused on demand reduction as opposed to energy reduction, in contrast to these other programs that offer incentives based on energy reduction, the energy reduction per measure in the HPNC may be less than other programs, therefore driving up the average dollar per kwh paid by the OPA. The Evaluation Team recommends that the OPA consider raising the incentive levels and increasing outreach to encourage greater participation among participants that are not already considering energy efficient measures. Table 16: Process Evaluation HPNC Benchmarking of Incentive Levels Utility Description Incentives OPA 2011/2012 HPNC DTE Energy AEP Ohio MidAmerican Energy Four tracks based on a specified incentive level, % over code, or cost: (1) Prescriptive: $/kw, $/kwh, or $/unit payment (2) Engineered: $/kw or $/kwh (3) Custom Design: % above Code (4) Custom Modelling: Based on cost of Simulation Summary Report Three tracks based on a specified % over baseline energy savings: (1) 10-20% over baseline (2) 20-30% over baseline (3) > 30% over baseline Three tracks based on project type: (1) Whole Building Performance: Minimum 10% > ASHRAE (2) Prescriptive - Lighting: Power Density must be 10% lower than wattage in ASHRAE (3) Custom: Custom requirements Two Tracks based on specified % over Iowa energy code: (1) Custom: At least 15% > code (2) Custom Plus: At least 40% > code (1) Prescriptive: Lighting: up to $400/kW; Nonlighting: up to $800/kW; Appliances: $75/unit (2) Engineered: Lighting: up to $400/kW or $0.05/kWh; Non-lighting: up to $800/kW or $0.10/kWh (3) Custom Design: < 25% above Code: $50/kW or $ /kWh; > 25% above Code: $100/kW or $0.0125/kWh; > 50% above Code: $150/kW or $ /kWh (4) Custom Modelling: Lesser of $10,000 or 100% of third costs to prepare the Simulation Summary Report (1) 10% to 20% over baseline energy savings: $0.08 per kwh and $4.00 per Mcf (2) > 20%, up to 30%, energy savings: $0.10 per kwh and $6.00 per Mcf (3) > 30% energy savings: $0.12 per kwh and $8.00 per Mcf (1a) Whole Building - Design: Range $ $0.04/kWh (incentives increase as % over ASHRAE increases) (1b) Whole Building - Owners: Range $ $0.12/kWh (incentives increase as % over ASHRAE increases) (2) Prescriptive - Lighting: $0.40/watt below ASHRAE standard (3) Custom: $0.08/kWh Incentives range from $0.06 to $0.19/kWh and $0.60 to $1.90/therm Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 52

53 Process Evaluation: HPNC Program HPNC: Penetration of the New Construction Market The Evaluation Team conducted a geographical analysis by comparing the location of HPNC projects with estimates of investment in commercial new construction projects. Figure 17 shows where HPNC projects were implemented, while Figure 18 and Figure 19 highlight the density of non-residential construction activity in metropolitan areas in 2011 and 2012, respectively.. 10 Certain areas of Ontario show a high level of investment in commercial new construction, particularly in the Census Metropolitan Areas (CMAs) of Toronto, Ottawa Gatineau, London, and Hamilton. 11 Despite this activity, only 23 HPNC measures (of a total of 182) were implemented in these areas in program years Statistics Canada. Table Investment in non-residential building construction, by type of building, province and census metropolitan area (CMA), quarterly (dollars), CANSIM (database). (accessed: ). 1&p1=-1&p2=9 11 A Census Metropolitan Area, or CMA, is defined as an area of one or more neighbouring municipalities situated around a core. A CMA must have a total population of at least 100,000, of which 50,000 or more live in a core. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 53

54 Process Evaluation: HPNC Program Figure 17: Geographical Representation of HPNC Projects by CMA in Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 54

55 Process Evaluation: HPNC Program Figure 18: Investment in Non-Residential Construction in 2011 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 55

56 Process Evaluation: HPNC Program Figure 19: Investment in Non-Residential Construction in 2012 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 56

57 Process Evaluation: RNC Program Residential New Construction This section covers RNC performance relative to four topics: Program Market Delivery Effectiveness, Customer Motivation, Incentive Levels, and Penetration of the New Construction Market. RNC: Assessment of the Effectiveness of the Programs Market Delivery The Evaluation Team conducted a comprehensive process review to provide feedback on the design, delivery, and quality of the RNC initiative. This review included surveys of participants and modeling firms and interviews with LDCs. Methodology The Evaluation Team surveyed RNC participants and firms involved in the application process, as well as LDC staff, to assess the effectiveness of the RNC program. Frontier interviewed representatives from 11 LDCs. Seven of the interviews were conducted in-person, while the remainder were conducted via telephone. Eleven builders participated in the RNC program in program years 2011 to These eleven builders had 34 projects in the program. Two of the builders were responsible for21 projects (64 percent of all projects). The Evaluation Team interviewed the two largest builder participants in the program to gauge current effectiveness of program delivery. Figure 20 shows the breakdown in builder participation. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 57

58 Process Evaluation: RNC Program Figure 20: Process Evaluation - RNC - Builder Participation in PY Builder J, 1 Builder I, 1 Builder H, 1 Builder G, 1 Builder K, 2 Builder A, 1 Builder B, 11 Builder F, 10 Builder E, 3 Builder C, 1 Builder D, 2 Results Most RNC builders participated via the Performance track, as shown in Figure 19. In the Performance track, participants built houses to meet certain EnerGuide performance ratings. According to this chart, 91 percent of builders participated in the Performance track, while 9 percent followed the Prescriptive track. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 58

59 Process Evaluation: RNC Program Figure 21: Process Evaluation - RNC Project Participation in PY Gas Furnace with ECM, 3 SEER 15 Central Air Conditioner, 1 83 or 84 Performance Rating, or Greater Performance Rating, 21 Builders were asked to rate their satisfaction with the application process. Builder 1 rated the application process a 5, indicating they were greatly satisfied. Builder 2 rated the process a 3, and mentioned that the differing permitting requirements among the different LDCs was very problematic. See Figure 22. Figure 22: Process Evaluation RNC Builder Satisfaction with Application Process Builder 1 5 Builder Both builders indicated that they found the application process and the contract to be reasonable and understandable. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 59

60 Process Evaluation: RNC Program Discussion While the interviewed builders indicated relative satisfaction with the program, the LDCs reported the following key concerns with the present design of the RNC: The application process is cumbersome and costly for applicants. The application forms are too long and complicated and contain too much legal language. There are concerns about the transition to the next generation of this program. The LDCs cannot guarantee an incentive if the new home may not be completed until In contrast to the RNC, the Residential HVAC program is viewed as successful. The application process is much simpler. One LDC suggested simply establishing appropriate new home construction baselines within the Residential HVAC program and allowing new home construction projects to apply for HVACrelated incentives in the HVAC program. The Evaluation Team notes that this might discourage comprehensiveness in the promotion of energy efficiency in new home construction, but this is a concept that should be further considered. Application Process Builders find the process for resubmitting documents difficult. Frequently, a house s EnerGuide rating may change during the building process, but the process for resubmitting a house s new EnerGuide rating is quite difficult. Builders cite a steep learning curve required for participating, but once the builder has learned how to submit the forms, the process becomes much easier. After learning how to process the forms, one builder said it took her less than an hour per form, while another builder cited 3 to 4 hours per form. Permit Requirements Builders seem to have less of a problem with the building permits than in the HPNC program, although they dislike that the process seems to be different for each LDC. One builder said that submitting the building permits was straightforward and easy. Incentive Levels The incentives levels are too low relative to the cost of completing the application process, unless the builder/developer operates on a large scale and intends to apply for incentives for many new homes. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 60

61 Process Evaluation: RNC Program Tracking System The absence of a working tracking system makes it difficult for the OPA to monitor the level of activity in the program. From an evaluator s perspective, the lack of unique site and project IDs made the applications and associated paperwork somewhat confusing to track down. Inconsistencies in the labeling of project paperwork received from the OPA added work to the process and often lead to confusion for the evaluators. LDC Consistency Builders find a lack of consistency in requirements across the territory some LDCs require preliminary forms, or have their own form to fill out, and some do not use OPA forms at all. Program Informational Materials Builders stated that the website materials ranged from somewhat to very helpful, although the process could be made clearer from the beginning and some important information seems to be missing. Program Future There are concerns about the transition to the next generation of this program. The LDC cannot guarantee an incentive if the new home may not be completed until Recommendations Please see the Program Design & Other Recommendations section for the Program Design, Application Revision, and Blanket Waiver Recommendations. The table below provides recommendations specific to the RNC program, and specifies where certain program aspects are more thoroughly explored with more detailed recommendations in other sections of the report. Table 17: RNC Specific Program Delivery Recommendations Program Aspect Recommendations Application Process Streamline the application process and provide a clear process flow to potential Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 61 participants. Consider revising the application and the contract in a manner that clearly outlines the program requirements. o Some of the information is repetitive and some of the requirements seem unnecessary. The OPA/LDCs should either revisit some of the requirements or make clear why certain items, such

62 Process Evaluation: RNC Program Program Aspect Recommendations as building and occupancy permits, are required. See the Program Design & Other Recommendations (both HPNC & RNC) for more specific application recommendations Permit Requirements Clarify the building and occupancy permit requirements from the very beginning of the application process. Make these requirements uniform across all LDC territories. Vital information concerning permitting and resubmission requirements is not presented clearly. The website materials should present clearer timelines and program requirements. Incentive Levels Consider raising the incentive levels, especially for the Prescriptive track. See the RNC: Incentive Levels Analysis section for a benchmarking of incentive levels. Tracking System Implement a streamlined tracking system that follows a clear process flow, allows LDCs and the OPA to easily track applications, and highlights those projects that need attention. Consider adding Project and Site IDs. See the Audit of OPA s Program Tracking System (both HPNC & RNC) section for more specific recommendations Informational Materials Include a contact list for each LDC to assist confused participants. RNC: Review of Customer Motivations Method The Evaluation Team analyzed responses to survey questions about participants motivations in participating in the program. The team also compared the RNC program incentive levels with those of similar programs to assess the adequacy of the incentive levels for motivating participation. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 62

63 Process Evaluation: RNC Program Results Survey Results The two builders were asked to rate the importance of the factors listed in Figure 23 towards their decision in choosing the type of equipment to install in new construction programs. The results show that electricity costs and energy savings and maintenance requirements are very important to builders. Figure 23: Process Evaluation RNC Importance of Factors to Builders' Decision Making Electricity Costs and Expected Energy Savings Maintenance Requirements Rebates or Discounts Initial Project Cost Environmental or Emission Concerns Building Aesthetics Both builders surveyed participated in the Performance track of the program. Both were asked whether they would have become EnerGuide builders without the incentive of the program. Both builders responded affirmatively, that they would have been EnerGuide builders. One builder commented that they were already an EnerGuide builder prior to the program. Builders were asked whether they found the financial incentive satisfactory. Both builders stated they found the Performance track incentives satisfactory, but one builder mentioned that the Prescriptive track incentives were not worth [his] time. RNC: Incentive Levels Analysis The Evaluation Team repeatedly heard from the LDCs that the incentive levels for the RNC were too low to encourage participation by home builders. Undoubtedly, higher incentive levels would encourage greater participation, ceteris paribus. A review of similar programs around North America suggests that the incentives provided through the RNC are moderate. Utilities in Massachusetts and New Jersey offer higher incentives through their programs. Utilities in Texas, Missouri, and New Mexico generally offer lower incentive levels. It is important to note that both the efficiency thresholds (participation requirements) and baseline conditions (driven by differences in regional building codes, etc) may differ Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 63

64 Process Evaluation: RNC Program across programs, which makes apples-to-apples program comparisons difficult. However, some high level comparisons can provide good insight into how residential new construction programs are implemented in different jurisdictions. Table 18 provides an overview of similar programs across North America. Table 18: Benchmarking RNC Incentives Program Administrator Efficiency Standards Incentive Level OPA RNC Energuide $500 OPA RNC Energuide $1,000 Massachusetts (Single Family) Tier I improvement over baseline $750 Tier II improvement over baseline $1,250 Tier III 45% improvement over baseline MA home $8,000 New Jersey $2,250 - $4,000 Single Family HERS Rating IECC 2006 HERS Rating IECC 2009 $2,250 - $4,000 $2,250 - $3,501 Multifamily Single Level 75% of single family incentive 75% of single family incentive Multifamily Multi Level 50% of single family incentive 50% of single family incentive New York Energy Star Standards $1,250-$1,501 CPS Energy (Texas) Energy Star Standards or Utility up to $1,500 Standards Oncor (Texas) HERS Rating $945/home CenterPoint (Texas) HERS Rating $364/home Missouri HERS rating of 85 or less $1,000 Entergy Texas Energy Star Certified $300/home Public Service of New Mexico Energy Star Homes Version 2.5 $750/home Program success does not necessarily increase with higher incentive levels. Some of the utility programs offering relatively low incentives are very successful. For example, Oncor and CenterPoint Energy in Texas offer relatively-low incentives through their award-winning programs, 12 but serve markets with a lot of new home construction activity and large-volume home builders. In addition, some of North America s better new home construction programs focus on certification, rather than incentives, 12 Oncor and CenterPoint received ENERGY STAR Partner of the Year New Homes in 2003, 2004, 2005 and In 2007, 2008, 2009, 2010, and 2011, the EPA recognized Oncor s accomplishments in the ENERGY STAR Homes Program by awarding it the ENERGY STAR Sustained Excellence Award. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 64

65 Process Evaluation: RNC Program including Austin Energy s Green Building program and Wisconsin Focus on Energy. 13 Some programs involve incentives to raters, 14 while some provide incentives to both builders and raters, including programs in Connecticut. 15 Increases in program incentives must be balanced against the need to make the RNC cost-effective. Based on the results of Frontier s impact evaluation, OPA s performance incentives per home are very high compared to the prescriptive rebates. Some adjustments may be necessary to create optimal incentive levels that encourage participation and ensures cost-effectiveness for all tracks of the RNC program. RNC: New Construction Market The Evaluation Team conducted a geographical analysis by comparing the location of RNC projects with estimates of investment in new housing starts. See Figure 24, Figure 25, and Figure 26 for maps highlighting the geographic location of RNC program projects and new housing starts. 16 Certain areas of Ontario show a high level of investment in new housing starts, particularly Toronto, Hamilton, Barrie, Oshawa, London, and Ottawa Gatineau (otherwise known as the National Capital Region). However, the greatest amount of new construction projects with the RNC occurred in Norfolk or outside of these Census Metropolitan Areas (CMAs). 17 One would expect a residential new construction energy efficiency program s participation to be tied to areas of growth in housing starts however, the RNC finds that the bulk of its participation in PY occurred outside of these CMAs. The RNC has not been successful at engaging the residential construction market, as evidenced by the general lack in participation. In 2012, Ontario had 76,742 new housing starts. 18 Of those new housing starts, 23,382 were single-family homes. With only 44 projects, the RNC is not affecting the new home construction market in Ontario. 13 Residential New Homes. Focus on Energy. Wisconsin Utilities The rater incentives offered by Texas-New Mexico Power Company is $341/home. % pdf 15 Residential New Construction Program Project Application Connecticut Light and Power Company Canada Mortgage and Housing Corporation. Housing Market Outlook - Ontario Region Highlights. Date Released = Third Quarter 2013 Table: Ontario Region Housing Forecast - New Construction (p. 8) A Census Metropolitan Area, or CMA, is defined as an area of one or more neighbouring municipalities situated around a core. A CMA must have a total population of at least 100,000, of which 50,000 or more live in a core. 18 Canada Mortgage and Housing Corp., Starts and Completions Survey. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 65

66 Process Evaluation: RNC Program Figure 24: Geographical Representation of RNC Projects by CMA in Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 66

67 Process Evaluation: RNC Program Figure 25: Single Family Housing Starts in 2012 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 67

68 Process Evaluation: RNC Program Figure 26: Multifamily Housing Starts in 2012 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 68

69 Process Evaluation: Audit Tracking Systems (HPNC & RNC) Audit of the OPA s Program Tracking Systems (both HPNC and RNC) Methodology The OPA uses a Microsoft Dynamics Customer Relationship Management (CRM) software application for program tracking. The CRM is cloud-based, with the capability for multiple user access rights. The OPA and LDC staff have access rights to create projects and enter project information into the CRM. The system tracks customer contact information, general project information, and workflows. Frontier interviewed OPA staff members involved with processing applications using the audited tracking system. Discussion OPA staff report a great deal of frustration with the CRM tool. Although LDCs have the ability to submit applications, the process is too complicated, and LDCs have resorted to supplying the documentation to the OPA via , ftp site, CD, or USB drive. The LDCs submit applications via a template called the Preliminary Billing Report (PBR). The application itself is a hard copy of a PBR report. Participants fill out the hard copy of the application, and LDCs are re-entering the same information into the PBR, and delivering the PBR to the OPA via , ftp site, CD, or USB drive. Then, the PBR and supporting documents are uploaded into the tracking system by the OPA. According to system users, the disconnection between the participants, LDCs, and OPA staff within the tracking system creates a frustrating impression that many tasks are unnecessarily repetitive. A welldesigned tracking system would be able to integrate all of these steps, remove redundant tasks, and provide the parties involved a communication tool to interact regarding the status of applications. OPA staff report that the CRM is not intuitive and does not provide a clear process to follow. OPA staff find that the CRM s many idiosyncrasies are a hindrance when trying to evaluate the status of an application. For example, when clicking on various links in the system, the CRM will automatically open a new window. Staff also finds that the CRM operates very slowly. The CRM also does not clearly delineate the status of applications. As a result, the system provides users with no assistance in tracking applications. Recommendations The Evaluation Team makes the following recommendations: Open up the ability for LDCs to upload supporting documentation directly to the system, so that there s no ambiguity about the LDCs next steps. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 69

70 Process Evaluation: Audit Tracking Systems (HPNC & RNC) Make the process very clear by adding a process flow to the tracking system. Add functionalities to the tracking system to make abundantly clear which application needs attention, enabling users to truly track the application. Make mechanisms for communication between LDCs and the OPA clear so that the parties can effectively share information about the status of an application. Add a functionality to allow the OPA to be aware of applications that have been started by the LDCs/participants but are not ready for submittal to the OPA, in order to allow the OPA to anticipate potential activity levels. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 70

71 Process Evaluation: Employment Effects (HPNC & RNC) Assessment of Direct Employment Effects (both HPNC and RNC) Methodology Participants were asked whether they hired any additional employees as a result of participating in the program. Modellers and salespersons were also asked whether they have hired any additional employees. If the answer to the question was yes, the survey recipient was asked to quantify the number of employees hired. Results No net jobs were created as a result of the HPNC and RNC programs. According to survey results, no participants in the HPNC or RNC programs hired additional personnel as a result of participating in the HPNC program. Modellers and salespersons have hired employees, but this hiring is likely not only in response to only the HPNC and RNC programs, but all energy efficient programs that are available in the province, including retrofit. Six out of the seven modellers and salespersons said they hired additional personnel to handle projects related to utility energy efficient programs. One modelling firm stated they had hired 5 full time staff members to help work on energy efficient projects that are related to utility programs. Figure 27: Process Evaluation HPNC & RNC - Firms that Hired Additional Personnel in Response to Utility Energy Efficiency Programs No, 1 Yes, 6 Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 71

72 Recommendations (HPNC & RNC) Program Design & Other Recommendations (both HPNC and RNC) Future Program Design Options Through the Evaluation Team s interviews with the LDCs, program participants, and other stakeholders, the team received a wide variety of proposals for redesigning the HPNC and RNC programs. This section describes those proposals and highlights the advantages and disadvantages of each. Based on Frontier s experience implementing similar programs in the U.S. and knowledge of other successful new construction programs in North America, many of these proposals could indeed prove effective. Nonetheless, this report does not endorse any particular proposal. One of the concerns voiced by a number of stakeholders was that their input into the initial design of the present new construction programs was limited and greater input from stakeholders could have resulted in the avoidance of some later problems. A program design process involving a variety of stakeholders could indeed result in a superior program design and perhaps the analysis presented here could advance that dialogue. Clearly, a need exists to improve or re-design these underperforming programs, particularly the RNC. The key program design issue involves striking an appropriate balance between province-wide consistency and local control by the LDCs. National chain accounts, large-scale home builders, and commercial construction companies operating in numerous LDC service areas hope to see greater consistency in programs across the province. They would prefer to work with a single point of contact for the programs, rather than multiple contacts in multiple utility service areas. Dealing with multiple LDCs each with slightly different interpretations of the program rules and local practices discourages participation in the programs. Most though, not all of the LDCs would prefer much greater local control over the programs. They prefer to maintain direct contact with builders constructing new homes and commercial developments within their service areas. These LDCs have the best understanding of local energy efficiency opportunities. Many LDCs view inflexible province-wide program guidelines, application forms, and approval processes as impediments to meeting the needs of their customers. This perspective is further complicated by the resources and expertise of the LDCs. Some of the LDCs have the knowledge and resources to successfully implement energy efficiency programs with no outside help. Many of those LDCs that do not possess such expertise presently outsource many of their program marketing and implementation activities to Enbridge Gas and would not be opposed to the establishment of province-wide program implementers. A further consideration is the effect that any dramatic program design changes would have on program activity. The transition from the earlier new home construction program implemented by the Canadian Homebuilders Association and the earlier HPNC program implemented jointly by Enbridge Gas, Union Gas, and Toronto Better Buildings to the present program model with greater LDC control, led to great confusion and lost opportunities to promote energy efficiency. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 72

73 Recommendations (HPNC & RNC) These factors contribute to a wide spectrum of proposals to change the two programs, including the following: 1. Centralized programs with a province-wide third-party implementer selected through a competitive solicitation. The implementer would market the program, enroll participants, enforce program guidelines, and provide uniform program design and administration across LDC territories. Advocates of this proposal point to the success of the previous residential new construction under the Canadian Homebuilders Association s guidance, and the previous HPNC implemented through three organizations. 2. Allow LDCs to design and implement their own unique programs, provided they are consistent with a general program template, evaluation guidelines, cost-effectiveness requirements, and reporting guidelines approved by the OPA. Under this approach, each LDC could design its own program requirements, application forms, contracts with applicants, and incentive levels. This is the model in many U.S. states, including Texas. 3. Combine the HPNC and RNC programs with more-successful programs, such as the Commercial Retrofit program and the Residential HVAC program. This would require the expansion of existing programs to accommodate new construction baselines and savings calculations. 4. Hybrid approaches whereby one or more province-wide program implementers could market the program to national or province-wide chain accounts or larger builders, while the LDC s marketing and outreach could focus on potential participants with a local presence and focus. 5. Focus on gradual improvements to the existing HPNC and RNC initiatives. Some of the advantages and disadvantages of each of these proposals are presented in Table 19. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 73

74 Recommendations (HPNC & RNC) Table 19: Design Options Matrix Design Option 1. Centralized programs with a provincewide third-party implementer selected through a competitive solicitation. 2. Allow LDCs to design and implement their own unique programs, provided they are consistent with a general program template, evaluation guidelines, costeffectiveness requirements, and reporting guidelines approved by the OPA. 3. Combine the HPNC and RNC programs with more-successful programs, such as the Commercial Retrofit program and the Residential HVAC program. 4. Hybrid approaches whereby one or more province-wide program implementers could market the program to national or province-wide chain accounts or larger builders, while the LDC s marketing and outreach could focus on potential participants with a local presence and focus. 5. Focus on gradual improvements to the existing HPNC and RNC initiatives. Advantages This would facilitate greater participation by potential participants active in numerous service areas. Greater consistency in the program guidelines across the province would be achieved. More-effective province-wide marketing could be conducted. Greater flexibility would lead to greater participation, at least in the service areas of the LDCs with the higher levels of expertise and resources. This would take advantage of the some of the other programs' superior infrastructure. This could facilitate participation by chain accounts and larger builders, while meeting the needs of the LDCs. This would reduce confusion in the market place and perhaps avoid many of the transitional problems that adversely affect efficiency initiatives during previous transitions. Disadvantages Smaller projects and smaller potential participants may receive less attention. Another organization may alter the direct relationship between an LDC and its retail customers. This may lead to less consistency across the province in project eligibility, incentive levels, timelines, and application procedures. Greater differences among the LDCs would further frustrate chain accounts and the national or province-wide potential participants. Some of the more-comprehensive features of the RNC (e.g., incentives based on EnerGuide ratings) might be lost. New baselines and savings calculations would need to be added to the other programs. Delineating the responsibilities of the provincewide implementer vis a vis the LDCs could be complicated. At least for the RNC, a major overhaul may be necessary. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 74

75 Recommendations (HPNC & RNC) Possible Application Revisions The Evaluation Team compared OPA s HPNC and RNC applications with other energy efficiency new construction program applications. Most other programs have their applications integrated with a tracking system that sits behind an Internet wall requiring a log in and password. However, the team found that ComEd s New Construction program requires a PDF application that provides a comparison to OPA s application forms. OPA staff should consider how ComEd s application clearly outlines the process of participating in the program by providing a process flow in a numerical order. OPA s application does not provide this numerical order, providing only a checklist that is written in the style of legalese commonly found in small print. OPA s checklist could be integrated into a similar numerical order that provides clarity to participants, and written in a style more accessible to participants. This process description could also include some descriptions of how and when the building and occupancy permits are required. Please note the following comparisons between the two examples: Figure 28: Process Outline from OPA's Application Form Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 75

76 Recommendations (HPNC & RNC) Figure 29: Process Outline from ComEd's Application Form Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 76

77 Recommendations (HPNC & RNC) Waivers and Blanket Waivers LDCs will request a waiver from the OPA if the applicant or project inadvertently did not meet one or more of the Eligibility Criteria as outlined in Exhibit A of the HPNC Schedule. The process requires LDC staff to fill out an OPA Request for Waiver Template and submit this request to OPA staff. The OPA may require additional information before approving a waiver. For example, the OPA may require LDCs to demonstrate that the customer had indicated their intent to apply to the program prior to initiating the project, if this issue is the nature of the waiver. According to OPA staff, the process of approval typically takes about four weeks. However, some LDCs report that it can take over one year to receive a waiver from the OPA. The wait time for waivers that have taken over one year to receive should be investigated. However, the Evaluation Team has the understanding that for some other energy efficiency programs the OPA is starting to issue blanket waivers for common categories of waiver requests to expedite their processing. The team encourages the OPA to continue and extend this practice. A more timely response to waiver requests will improve satisfaction with the programs among LDCs and participants. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 77

78 Conclusion & Recommendations CONCLUSION AND RECOMMENDATIONS The HPNC and RNC programs have great potential to reduce energy and demand savings in the province of Ontario. Frontier believes that incremental changes made to program design with stakeholder input as well as an increased marketing effort made to pierce the new construction market in Ontario will likely result in a very successful program. Impact Evaluation The impact evaluation looked at verifying the energy and demand savings claimed for the HPNC and RNC programs. Savings for the new construction initiatives are dominated by the HPNC program, contributing more than 99% of total savings. Roughly 66% of savings come from the HPNC Custom track, and 33% of savings from the HPNC Prescriptive track. Table 20: Gross Energy and Demand Savings by Initiative Initiative Track Gross Demand Savings (kw) Gross Energy Savings (kwh) HPNC Prescriptive 1,195 3,580,613 HPNC Custom 2,306 8,148,078 RNC Prescriptive RNC Performance ,312 A document review indicated that the most of the calculation methods and assumptions listed in the 2011 Quasi-Prescriptive Measures and Assumptions List (MAL) Version 1.0 were reasonable. Where applicable, Frontier made adjustments to baselines and changes cases. Additionally, Frontier recommends a savings estimation strategy that makes use of building-specific operating hours. These hours are already available in the MAL and will improve the accuracy of the savings estimates. A review of the available documents for other custom projects indicated that the modellers used good practice in modelling architectural and mechanical, electrical and plumbing systems for energy consumption, yielding reasonable energy and demand savings. There were no recommended changes to the custom savings estimates. Only 3 projects were submitted to the RNC Prescriptive track, with a minimum impact on overall program savings. Some measure assumptions were not utilized because the assumptions were deemed to be invalid based on such a small project sample size. In these cases, project-specific information was used instead. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 78

79 Conclusion & Recommendations RNC Performance projects submitted ranged in EnerGuide score from 83 to 87, with the majority falling at a score of either 83 or 85, the minimum threshold score to receive an incentive. The team suggests that the OPA provide baseline and change case energy consumptions and housing characteristics (square footage, number of floors, heating type and capacity, HVAC type and capacity, insulation levels, water heating type and capacity). Overall, the HPNC initiative is driving the energy and demand savings for OPA s new construction initiatives. Specifically, one custom project contributed roughly 60% of total new construction initiative savings. Agribusiness ventilation is also a primary driver of savings for the prescriptive projects. Process Evaluation Program Design Recommendations The table below provides an overview of the recommendations made in this Final Evaluation Report. Table 21: Process Evaluation Recommendations Program Aspects Recommendations Incentive Levels Consider raising the incentive levels, especially for the RNC. Allow certain efficiency measures presently eligible for a prescriptive incentive in the HPNC program to apply for an incentive through the retrofit program. This change would require new construction or building code baselines to be used as the basis for calculating savings and incentive levels for certain measures within the expanded retrofit program. Project Sizes Consider lowering minimum project sizes, so that smaller projects (and smaller commercial or business energy consumers) can receive an incentive through the HPNC. Program Design Consider alternative approaches to promoting the RNC. For example, providing payments to home energy raters, rather than home builders, has proven effective in some other markets. With stakeholder input, consider the advantages and disadvantages of different program design options. These options include: (1) centralizing the programs with a provincewide third party implementer; (2) decentralizing and allowing the LDCs to design and implement their own unique programs; (3) combining the program with more successful OPA programs; (4) hybrid approaches that include both a province-wide implementer for chain accounts and local LDC marketing and outreach; or (5) focus on gradual improvements to the programs. Explore how requests by the LDCs to the OPA for waivers from program requirements could be addressed in a more expeditious manner. Informational Materials A program manual or more informative website is necessary Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 79

80 Conclusion & Recommendations to explain the programs and clear up some lingering confusion issues such as what constitutes the completion of a building construction project and whether building permits are a program requirement. Provide information upfront to the participant that fully explains the timeline of the project and the potential data requests that the participant may need to complete. A wellwritten program manual may provide this necessary information. Tracking System Consider investing in a tracking system for both programs that is streamlined with a clearly outlined process that both LDCs and the OPA can use to track applications. OPA should also consider implementing a periodic reporting requirement in order to determine what applications the LDCs may soon be submitting to the OPA. This requirement would assist OPA in anticipating program activity levels. Assign Project IDs and Site IDs to each project. Document Requirements Frontier recommends the following additional required document submittals for HPNC custom projects - (1) the final commissioning report and (2) the as-built control diagrams from the building automation contractor. These documents would be submitted when the project is complete and would allow reviewers to confirm that the building automation contractor had furnished controls implementing the specified sequence of operations and that the facility operated as designed when placed in service. For the RNC program, the team suggests that the OPA provide baseline and change case energy consumptions and housing characteristics for each home (square footage, number of floors, heating type and capacity, HVAC type and capacity, insulation levels, water heating type and capacity) in order to provide additional assistance with verifying energy and demand savings. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 80

81 Appendix A: Glossary Appendix A: Glossary Census Metropolitan Area (CMA): An area of one or more neighbouring municipalities situated around a core. A CMA must have a total population of at least 100,000, of which 50,000 or more live in a core. Energy Efficiency: The use of less energy to provide the same or an improved level of service to the energy consumer in an economically efficient way. Energy conversation is a term that has also been used, but it has the connotation of doing without in order to save energy rather than using less energy to perform the same or better function. Evaluation, measurement, and verification (EM&V): The process of determining and documenting the results, benefits, and lessons learned from an energy efficiency program. The term evaluation refers to any real time and/or retrospective assessment of the performance and implementation of a program. Measurement and verification is a subset of evaluation that includes activities undertaken in the calculation of energy and demand savings from individual sites or projects. Free rider: A program participant who would have implemented the program measure or practice in the absence of the program. Impact Evaluation: Used to determine the actual savings achieved by different programs and specific measures. Measures: Installation of equipment, installation of subsystems or systems, or modification of equipment, subsystems, systems, or operations on the customer side of the meter, in order to improve energy efficiency. Net-to-gross ratio: A key requirement for program-level evaluation, measurement, and verification. This ratio accounts for only those energy efficiency gains that are attributed to, and the direct result of, the energy efficiency program in question. It gives evaluators an estimate of savings that would have occurred even without program incentives. Participant: In the HPNC program, a participant is a building or facility owner or manager that receives an incentive for energy efficient measures in a new construction project. In the RNC program, a participant is a home builder that receives an incentive for energy efficient measures in a new construction project. Portfolio: Either (a) a collection of similar programs addressing the same market, technology, or mechanisms or (b) the set of all programs conducted by one organization. Process Evaluation: This form of evaluation assesses the extent to which a program is operating as it was intended. It typically assesses program activities conformance to statutory and regulatory requirements, program design, and professional standards or customer expectations. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 81

82 Appendix A: Glossary Project: A project is any one energy efficiency new construction plan that involves one application for an incentive to the OPA. Program: Any activity, project, function, or policy that has an identifiable purpose or set of objectives. Program Administrators: Typically procure various types of energy efficiency services from contractors (e.g., consultants, vendors, engineering firms, architects, academic institutions, community-based organizations), as part of managing, implementing, and evaluation their portfolio of energy efficiency programs. Program administrators in many states are the utilities; in some states they are state energy agencies or third parties. Sources: Understanding Cost-Effectiveness of Energy Efficiency Programs: Best Practices, Technical Methods, and Emerging Issues for Policy-Makers. EPA. November Glossary at Impact Evaluations website for the EPA. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 82

83 Appendix B: HPNC Participant Survey Appendix B: HPNC Participant Survey High Performance New Construction Survey Business: Call attempt 1 Customer Name: «Customer» Call attempt 2 Phone: «Customer_Phone» Call attempt 3 Phone2: Call attempt 4 «HOMEPHONE» LDC: Background/Experience of the Firm Module: Introduction and Qualifications Hello, my name is, from Frontier Associates. I am calling on behalf of the Ontario Power Authority as my firm has been contracted to conduct the High Performance New Construction Program evaluation. Your opinions are important to the OPA and it would only take around twenty minutes. Do you have time to talk today? No, Too Busy Right Now Yes (GO TO QA) (IF NO) Is there another time I can call you for a short interview? (SCHEDULE ALTERNATE INTERVIEW TIME: ) I. Our records indicate that you participated in OPA s High Performance New Construction Program in <Program Year>. Are you the best person to talk to about the participation in the program? No Yes (GO TO Q1) II. If NO: is there someone else I can talk to about the projects that were completed through the Program? No (THANK AND QUIT) Yes: NAME: PHONE: (ASK IF THAT PERSON IS AVAILABLE. THANK AND CONTINUE FROM BEGINNING OR QUIT) Marketing Module 1. What is your role as a participant in the HPNC program? Are you a facilities manager or facilities owner? Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 83

84 Appendix B: HPNC Participant Survey 2. How did you become aware of the Program? OPA Staff Enbridge Gas Distribution Bill Insert Mailing Advertising Word-of-mouth Location Distribution Company (LDC) Other: 3. What do you think is the best way to reach potential program participants? 4. How useful were the OPA s/ldcs website materials for supplying you with the necessary information? (Were they not useful, somewhat useful, very useful). a. If possible, provide justification/reason why they were useful, somewhat useful, or very useful? b. Any suggestions on how to make the program materials more useful? c. Did OPA or the LDC provide any additional materials aside from those available on the website? Participant Motivation Module/Measure-Specific Motivation 5. When your business is deciding what type of equipment to install in new construction or major retrofit projects, how important are the following factors to you, on a scale of 1 to 5, with 5 being extremely important. Electricity costs and expected energy savings Initial project cost Environmental or emission concerns Rebates or discounts Building aesthetics Maintenance requirements Any Other Factors? (SPECIFY AND RATE) Do you have corporate policies or goals pertaining to energy management and cost reduction? 7. How concerned is your company with operating costs? (scale of 1 to 5, with 5 being extremely important) Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 84

85 Appendix B: HPNC Participant Survey 8. With respect to the specific measures you chose for your project, can you rank the importance of the following factors on a scale of 1 to 5, with 5 being extremely important? Electricity costs and expected energy savings Initial measure cost Rebate level Aesthetics Maintenance requirements Difficulty of Installation Any Other Factors? (SPECIFY AND RATE) On a scale of 1 to 5, with 5 being Most Impact and 1 being the Least Impact, did any of the following factors motivate you to participate in the program? Contact by OPA s customer representatives OPA informational literature and advertising Contact from your LDC s customer representatives LDC informational literature and advertising Availability of an incentive Other: Were there other energy efficiency features you were interested in pursuing in your project(s), but you didn t pursue them because they wouldn t qualify under the program or the program s incentives were insufficient? Program Delivery Effectiveness Module For Prescriptive Track: 11. Can you please confirm that Prescriptive-based is the type of incentive track you participated in? 12. Did you consider the Engineered incentive track? Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 85

86 Appendix B: HPNC Participant Survey a. If so, why didn t you choose that track? 13. Did you consider the Custom incentive track? a. If so, why didn t you choose that track? 14. Did prescriptive incentive levels influence the types of measures you installed in your projects? (NTG) a. If yes, how so? b. If no, why not? 15. How easily were you able to complete the Prescriptive Worksheets? a. Can you rank its ease of use on a scale of 1 to 5. (where 5 means VERY EASY and 1 means VERY DIFFICULT) Ease of Working With Online Prescriptive Worksheets Comments: For Custom Track (only one participant): 16. I see you applied for and received a Design Decision Maker Incentive. a. Would you have been able to complete this project without the design decision maker incentive? b. Did you find the incentive to be adequate? c. Who was your design decision maker? (architect, engineer, consultant) 17. I see you applied for and received a Modeling Incentive grant. a. Would you have been able to complete this project without the modeling incentive grant? b. Did you find the incentive grant to be adequate? c. Did you work with a third party to model the project? i. Who did you work with to model the project? (architect, engineer, consultant) 18. How easily were you able to complete the Custom worksheets? a. Can you rank its ease of use on a scale of 1 to 5. (where 5 means VERY EASY and 1 means VERY DIFFICULT) Ease of Working With Online Custom Worksheets Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 86

87 Appendix B: HPNC Participant Survey Comments: All Tracks: 19. Are you satisfied with the amount of the financial incentive you received? (YES/NO) If no, why not? 20. After you submitted your application, how long did it take until you received your rebate check? 0 to 2 weeks 2 to 4 weeks 4 to 6 weeks 6 to 8 weeks 8 to 10 weeks 10 to 12 weeks more than 12 weeks Was that an acceptable amount of time? NO YES 21. Did you fill out the program application for the project? If so, what do you think of it? (If NO, skip to Q11). a. Please rank ease of completion of the application on a scale of 1 to 5. (where 5 means VERY SATISFIED and 1 means VERY DISSATISFIED) Ease of Application Completion Comments: 22. How did you find the application process? A. Please rank your satisfaction with this process on a scale of 1 to 5. Satisfaction with payment process Comments: 23. Was the application process reasonable and understandable? 24. Was your facility inspected by the LDC after completion? a. Please rank your satisfaction with this process on a scale of 1 to 5. Satisfaction with verification process Comments: Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 87

88 Appendix B: HPNC Participant Survey 25. Was the contract that you had to sign in order to participate in the program understandable? Were the contractual requirements reasonable? 26. Did you encounter any difficulties in meeting the program s requirements for applying and receiving an occupancy/building permit? Did you encounter any difficulties supplying the LDC with a copy of the occupancy/building permit? Could this process be improved upon? 27. Do you have any suggestions for improving the program? 28. How did you submit the application? a. Mail b. c. Web portal d. Other 29. How did you check on the status of your application? 30. Throughout your involvement with the program, regarding communication with program staff [DETERMINE WHETHER PARTICIPANT WORKED WITH OPA/LDC OR ANOTHER ENTITY BEFORE MAKING THE CALL] A. When you called or ed staff, did they get back with you quickly? (YES/NO) i. If no, how long did you have to wait? B. Were they able to effectively communicate with you? (YES/NO Participant NTG Module/Market Effects 31. Did you learn about the OPA s Program BEFORE or AFTER you decided to build or renovate? 32. If financial incentives through the High Performance New Construction Program had not been available, what is the likelihood that you would have pursued the energy efficiency features? (not likely, somewhat likely, very likely) Is the question whether the building would have been constructed? Or whether an energy efficient building would have been pursued in lieu of a lessefficient building? 33. If you had never learned you could get financial incentives from your LDC, which of the following best describes what your business would have done: a. put off the project for at least a year b. done the project but scaled back certain costs c. done the project the exact same way d. done something else. If so, what? Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 88

89 Appendix B: HPNC Participant Survey 34. If your business had not received incentives from your LDC, would it have had the funds, internal or otherwise, to cover the entire cost of the energy efficiency features? (YES/NO) The project? Or the energy efficiency features within the project? 35. Would you have pursued the same level of efficiency without the program/incentive? (YES/NO) 36. How important was the desire to be energy efficiency or green in your decision to include energy efficiency measures in your new construction project? (very/fairly/neutral/fairly unimportant/very unimportant) 37. We d like to get a sense of what influenced you to build an energy-efficient project. How influential was the cash incentive in your decision to include energy efficiency in your project? (very influential/somewhat/not at all) 38. Has participating in the program influenced how you feel about energy efficiency? How so? 39. Going forward, will you pursue projects that adhere to high standard of energy efficiency, even if no incentives are available? 40. Do you have anything else that you would like to share with me about the influence of the program on the energy efficiency of your project? 41. Do you participate in any other OPA SaveOnEnergy programs? Participant Spillover/Market Effects Module 42. Since participating in the High Performance New Construction program, have you included any additional energy efficient measures in project(s) that have gone through the program, but for which you did not receive an incentive? (YES, NO, don t know - If NO/don t know, go to Q2, If YES continue) 43. Did these other measures receive any rebates or incentives through a utility or government program? a. If so what were these measures and rebates? Can you tell me the efficiency and quantity? b. Did you learn about these measures through training or other information provided through the OPA new construction program? c. If you have not participated in the program, how likely is it that you would have included this measure in your project (not likely, somewhat likely, very likely) Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 89

90 Appendix B: HPNC Participant Survey 44. Since participating in the program, have you included any energy efficient measures in projects not associated with the program? (YES, NO, don t know - If NO/don t know, go to Q4, If YES continue) a. Did you receive a rebate through a government or utility program? b. What was the measure? Quantity and efficiency? c. Did you learn about these measures through training or other information provided through the OPA new construction program? d. If you had not participated in the program, how likely is it that you would have included this measure in your project (not likely, somewhat likely, very likely) 45. Do you think your experience participating in the program has changed the way you do business in any way? Employment Effects Module 46. Did you hire any additional employees to complete projects associated with this program? a. If not, how many of your current employees devoted time to work on these projects? b. Could you estimate how many hours or days full-time employees worked on these projects? Conclusion 47. How would you rate your overall satisfaction with the program, on a scale of 1 to 5? Overall program satisfaction Comments: 48. Is there anything else that you would like to let us know based on the topics we covered today, including any ways to improve the program if possible or how the program has affected your use of energy efficient measures or design in projects? Thank you for your time! Your feedback will be used to improve the program and provide new opportunities for builders to save money on energy efficient projects. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 90

91 Appendix C: RNC Participant Survey Appendix C: RNC Participant Survey Residential New Construction Participant Survey Business: Call attempt 1 Customer Name: «Customer» Call attempt 2 Phone: «Customer_Phone» Call attempt 3 Phone2: Call attempt 4 «HOMEPHONE» LDC: Background/Experience of the Firm Module: Introduction and Qualifications Hello, my name is, from Frontier Associates. I am calling on behalf of the Ontario Power Authority as my firm has been contracted to conduct the New Home Construction Program evaluation. Your opinions are important to the OPA and it would only take around twenty minutes. Do you have time to talk today? No, Too Busy Right Now Yes (GO TO QA) (IF NO) Is there another time I can call you for a short interview? (SCHEDULE ALTERNATE INTERVIEW TIME: ) III. Our records indicate that you participated in OPA s New Home Construction Program in <Program Year>. Are you the best person to talk to about the participation in the program? No Yes (GO TO Q1) IV. If NO: is there someone else I can talk to about the projects that were completed through the Program? No (THANK AND QUIT) Yes: NAME: PHONE: (ASK IF THAT PERSON IS AVAILABLE. THANK AND CONTINUE FROM BEGINNING OR QUIT) Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 91

92 Appendix C: RNC Participant Survey Marketing Module 1. How did you become aware of the Program? OPA Staff Enbridge Gas Distribution Bill Insert Mailing Advertising Word-of-mouth Location Distribution Company (LDC) Other: 2. What do you think is the best way to reach potential program participants? 3. How useful were the OPA s/ldcs website materials for supplying you with the necessary information? (Were they not useful, somewhat useful, very useful). A. If possible, provide justification/reason why they were useful, somewhat useful, or very useful? B. Any suggestions on how to make the program materials more useful? C. Did OPA or the LDC provide any additional materials aside from those available on the website? Participant Motivation Module/Measure-Specific Motivation 4. When your business is deciding what type of equipment to install in new construction or major retrofit projects, how important are the following factors to you, on a scale of 1 to 5, with 5 being extremely important. Electricity costs and expected energy savings Initial project cost Environmental or emission concerns Rebates or discounts Building aesthetics Maintenance requirements Any Other Factors? (SPECIFY AND RATE) Who is the decision-maker for deciding the energy efficient features of the home? The homeowner? The developer? Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 92

93 Appendix C: RNC Participant Survey 6. With respect to the specific measures chosen for your project, can you rank the important to the primary decision-maker (Homeowner? Developer?) to the following factors on a scale of 1 to 5, with 5 being extremely important? Electricity costs and expected energy savings Initial measure cost Rebate level Aesthetics Maintenance requirements Difficulty of Installation Any Other Factors? (SPECIFY AND RATE) On a scale of 1 to 5, with 5 being Most Impact and 1 being the Least Impact, did any of the following factors motivate you to participate in the program? Contact by OPA s customer representatives OPA informational literature and advertising Contact from your LDC s customer representatives LDC informational literature and advertising Availability of an incentive Other: Were there other energy efficiency features you were interested in pursuing in your project(s), but you didn t pursue them because they wouldn t qualify under the program or the program s incentives were insufficient? Program Delivery Effectiveness Module For Performance-based Track: 9. Can you please confirm that Performance-based is the type of incentive track you participated in? 10. Did you consider participating in the Prescriptive-based incentive track? Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 93

94 Appendix C: RNC Participant Survey A. If so, why didn t you pursue the Prescriptive-based incentive track? 11. Did you consider the Custom-based incentive track? A. If so, why didn t you pursue the Custom-based incentive track? 12. Would you have become an EnerGuide builder in absence of the program? How difficult is the process to become an EnerGuide builder? 13. I see your project achieved an (83 or 84 performance rating)/(85 or higher performance rating). Did the Performance-based incentive levels influence the types of measure you installed in your projects? (NTG) A. If yes, how so? B. If no, why not? For the Prescriptive-based Track: 14. Can you please confirm that Prescriptive is the type of incentive your received? 15. Did you consider the Performance-based incentive track? A. If so, why didn t you pursue the Performance-based incentive track? 16. Did you consider the Custom-based incentive track? A. If so, why didn t you pursue the Custom-based incentive track? 17. If furnace was a measure: Did you receive any additional incentive from Enbridge? All Tracks: 18. Are you satisfied with the amount of the financial incentive you received? (YES/NO) If no, why not? 19. After you submitted your application, how long did it take until you received your rebate check? 0 to 2 weeks 2 to 4 weeks 4 to 6 weeks 6 to 8 weeks 8 to 10 weeks 10 to 12 weeks more than 12 weeks Was that an acceptable amount of time? NO YES 20. Were the homes inspected by the LDC after completion? A. Please rank your satisfaction with this process on a scale of 1 to 5. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 94

95 Appendix C: RNC Participant Survey Satisfaction with inspection process Comments: 21. Did you or an employee fill out the preliminary and final applications for the incentive? A. If so, can you rank its ease of use on a scale of 1 to 5. (where 5 means VERY EASY and 1 means VERY DIFFICULT) Ease of Working With Online Application Worksheets Comments: 22. How did you find the application process? A. Please rank your satisfaction with this process on a scale of 1 to 5. Satisfaction with payment process Comments: 23. Was the application process reasonable and understandable? 24. Was the contract that you had to sign in order to participate in the program understandable? Were the contractual requirements reasonable? 25. Did you encounter any difficulties in meeting the program s requirements for applying and receiving an occupancy/building permit? Did you encounter any difficulties supplying the LDC with a copy of the occupancy/building permit? Could this process be improved upon? 26. Do you have any suggestions for improving the program? 27. How did you submit the application? A. Mail B. C. Web portal D. Other 28. How did you check on the status of your application? Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 95

96 Appendix C: RNC Participant Survey 29. Throughout your involvement with the program, regarding communication with program staff [DETERMINE WHETHER PARTICIPANT WORKED WITH OPA/LDC OR ANOTHER ENTITY BEFORE MAKING THE CALL] C. When you called or ed staff, did they get back with you quickly? (YES/NO) i. If no, how long did you have to wait? D. Were they able to effectively communicate with you? (YES/NO) Participant NTG Module/Market Effects 30. Did you learn about the OPA s Program BEFORE or AFTER you decided to build or renovate? 31. If financial incentives through the New Homes Construction Program had not been available, what is the likelihood that you would have pursued the energy efficiency features? (not likely, somewhat likely, very likely) Is the question whether the building would have been constructed? Or whether an energy efficient building would have been pursued in lieu of a lessefficient building? 32. If you had never learned you could get financial incentives from your LDC, which of the following best describes what your business would have done: A. put off the project for at least a year B. done the project but scaled back certain costs C. done the project the exact same way D. done something else. If so, what? 33. If your business had not received incentives from your LDC, would it have had the funds, internal or otherwise, to cover the entire cost of the energy efficiency features? (YES/NO) The project? Or the energy efficiency features within the project? 34. Would you have pursued the same level of efficiency without the program/incentive? (YES/NO) 35. How important was the desire to be energy efficiency or green in your decision to include energy efficiency measures in your new construction project? (very/fairly/neutral/fairly unimportant/very unimportant) 36. We d like to get a sense of what influenced you to build an energy-efficient project. How influential was the cash incentive in your decision to include energy efficiency in your project? (very influential/somewhat/not at all) 37. Has participating in the program influenced how you feel about energy efficiency? How so? Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 96

97 Appendix C: RNC Participant Survey 38. Going forward, will you pursue projects that adhere to high standard of energy efficiency, even if no incentives are available? 39. Do you have anything else that you would like to share with me about the influence of the program on the energy efficiency of your project? 40. Do you participate in any other OPA SaveOnEnergy programs? (Peaksaver, coupons etc) (relevant more for NHC customers) Participant Spillover/Market Effects Module 41. Since participating in the New Homes Construction program, have you included any additional energy efficient measures in project(s) that have gone through the program, but for which you did not receive an incentive? (YES, NO, don t know - If NO/don t know, go to Q2, If YES continue) 42. Did these other measures receive any rebates or incentives through a utility or government program? A. If so what were these measures and rebates? Can you tell me the efficiency and quantity? B. Did you learn about these measures through training or other information provided through the OPA new construction program? C. If you have not participated in the program, how likely is it that you would have included this measure in your project (not likely, somewhat likely, very likely) 43. Since participating in the program, have you included any energy efficient measures in projects not associated with the program? (YES, NO, don t know - If NO/don t know, go to Q4, If YES continue) A. Did you receive a rebate through a government or utility program? B. What was the measure? Quantity and efficiency? C. Did you learn about these measures through training or other information provided through the OPA new construction program? D. If you had not participated in the program, how likely is it that you would have included this measure in your project (not likely, somewhat likely, very likely) 44. Do you think your experience participating in the program has changed the way you do business in any way? Employment Effects Module 45. Did you hire any additional employees to complete projects associated with this program? A. If not, how many of your current employees devoted time to work on these projects? Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 97

98 Appendix C: RNC Participant Survey B. Could you estimate how many hours or days full-time employees worked on these projects? Conclusion 46. How would you rate your overall satisfaction with the program, on a scale of 1 to 5? Overall program satisfaction Comments: 47. Is there anything else that you would like to let us know based on the topics we covered today, including any ways to improve the program if possible or how the program has affected your use of energy efficient measures or design in projects? Thank you for your time! Your feedback will be used to improve the program and provide new opportunities for builders to save money on energy efficient projects. Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 98

99 Appendix C: RNC Participant Survey Appendix D: Updated Prescriptive Assumptions Measure Assumption Description Existing Value Updated Value Lighting: Medium and High Bay Fixtures 250 W Metal Halide Baseline Wattage (including ballast) 295 W 290 W Lighting: Medium and High Bay Fixtures 400 W Metal Halide Baseline Wattage (including ballast) 460 W 450 W ENERGY STAR Refrigerators Base Case kwh kwh 486 kwh ENERGY STAR Refrigerators Efficiency Case kwh kwh 389 kwh ENERGY STAR Clothes Washers Base Case kwh kwh 502 kwh ENERGY STAR Clothes Washers Efficiency Case kwh kwh 365 kwh ENERGY STAR Dishwashers Base Case kwh 358 kwh 151 kwh ENERGY STAR Dishwashers Efficiency Case kwh 307 kwh 126 kwh Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 99

100 Frontier Associates LLC 1515 S. Capital of Texas Hwy Suite 110 Austin, Texas Phone: (512) Fax: (512) Final Evaluation Report: High Performance New Construction and Residential New Construction Programs 100