Customer Energy Efficiency Program Measurement and Evaluation Program

Transcription

1 Customer Energy Efficiency Program Measurement and Evaluation Program EVALUATION OF PACIFIC GAS & ELECTRIC COMPANY'S PRE-1998 COMMERCIAL ENERGY EFFICIENCY INCENTIVES PROGRAM CARRY-OVER: HVAC TECHNOLOGIES PG&E Study ID number: 404B March 1, 2000 Measurement and Evaluation Customer Energy Efficiency Policy & Evaluation Section Pacific Gas and Electric Company San Francisco, California Disclaimer of Warranties and Limitation of Liabilities As part of its Customer Energy Efficiency Programs, Pacific Gas and Electdc Company (PG&E) has engaged consultants to conduct a series of studies designed to increase the certainty of and confidence in the energy savings delivered by the programs. This report describes one of those studies. It represents the findings and views of the consultant employed to conduct the study and not of PG&E itself. Furthermore, the results of the study may be applicable only to the unique geographic, meteorological, cultural, and social circumstances existing within PG&E's service area during the time frame of the study. PG&E and its employees expressly disclaim any responsibility or liability for any use of the report or any information, method, process, results or similar item contained in the report for any circumstances other than the unique circumstances existing in PG&E's service area and any other circumstances described within the parameters of the study. All inquiries should be directed to: Janice Frazier-Hampton Revenue Requirements Pacific Gas and Electric Company P. O. Box , Mail Code B9A San Francisco, CA 94177

2 Copyright 2000 Pacific Gas and Electric Company. All rights reserved. Reproduction or distribution of the whole, or any part of the contents of, this document without written permission of PG&E is prohibited. The document was prepared by PG&E for the exclusive use of its employees and its contractors. Neither PG&E nor any of its employees makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any data, information, method, product or process disclosed in this document, or represents that its use will not infringe any privately-owned rights, including but not limited to, patents, trademarks or copyrights.

3 EVALUATION OF PACIFIC GAS & ELECTRIC COMPANY'S PRE-1998 COMMERCIAL ENERGY EFFICIENCY INCENTIVES PROGRAM CARRY-OVER FOR HVAC TECHNOLOGIES PG&E Study ID number: 404B Purpose of Study This study was conducted in compliance with the requirements specified in "Protocols and Procedures for the Verification of Costs, Benefits, and Shareholders Earnings from Demand-Side Management Programs" (Protocols), as adopted by California Public Utilities Commission Decision , revised March 1998, pursuant to Decisions , , , , and This study evaluated the gross and net energy savings from HVAC energy efficiency technologies for which rebates were paid in 1998 by Pacific Gas & Electric Company's Commercial Energy Efficiency Incentive (CEEI) Programs. These retrofits were performed under CEEI programs offered from 1994 through Retrofits were performed under three different PG&E programs: the Retrofit Express (RE), Retrofit Efficiency Options (REO), and Advanced Performance Options (APO) Programs. Methodology For this evaluation, there were two types of primary data collected: telephone survey data and on-site audit data. An integrated sample design was implemented for the lighting and HVAC end uses, due to the number of participant crossover among these end uses. There were a total of 137 HVAC sites, 99 standard and 38 custom, that received a rebate from PG&E in A complete census was conducted and 81 sample points were collected. A non-participant sample was developed based upon the business type and usage strata distribution that resulted from the participant sample allocation. The HVAC end-use included 81 HVAC participant and 589 nonparticipant telephone surveys and 64 on-site audits. An integrated evaluation approach employed engineering, billing regression and net-to-gross (NTG) analyses. Engineering and statistically adjusted engineering (SAE) estimates were used to develop per participant gross energy, demand, and therm impacts for specified time-of-use costing periods. The engineering analysis combined information from telephone surveys with detailed on-site audit data to develop unadjusted engineering impacts. A billing regression analysis was employed to model the differences in customers' energy usage between pre- and

4 post-installation periods. The model was specified using actual customer billing data and independent variables that explain changes in customers' energy usage including engineering estimates of unadjusted savings. Three separate models were implemented to estimate the components of the NTG ratio (free-ridership and spillover): a model based on self-reports, a net billing analysis model applying a double inverse Mills ratio (estimating free-ridership only), and a two-stage discrete choice model. The final NTG ratios applied to the ex post gross impacts are based on the results of the self-report model. Discrete choice results were only obtained for the CAC technology segment due to the small available sample, and the results were not supported by either the Mills ratio or the self report result. To be conservative and consistent, the self-report estimates of NTG were applied to all of the HVAC technology segments. Study Results The results of the analyses for the HVAC technologies are summarized below: Gross Net Realization Net-To-Gross Realization Gross Savings Rate 1-FR Spillover NTG Ratio Net Savings Rate EX ANTE kw 3, ,376 kwh 20,671, ,525,132 Then'ns 575, ,840 EX POST kw 3, , kwh 13,659, ,865, Therms 489, , Regulatory Waivers and Filing Variances The CADMAC approved a waiver on May 20, 1999, that allows the use of self - report based algorithms to estimate free ridership and spillover effects in the event discrete choice and LIRM models fail to produce statistically reliable results. There were no E-Table variances.

5 Q QUANTUM CONSULTING EVAL UA TION OF PG&E'S PRE COMMERCIAL EEl PROGRAM CARRY-OVER HVAC TECHNOLOGIES PG&E Study ID#: 404B FINAL REPORT March 1, 2000 Submitted to Mary O'Drain Market Planning and Research Pacific Gas & Electric Co. 123 Mission Street San Francisco, CA Prepared by QUANTUM CONSUL TING INC Addison Street Berkeley, CA 94704

6 TABLE OF CONTENTS Section 1 2 EXECUTIVE SUMMARY 1.1 Evaluation Results Summary 1.2 Major Findings INTRODUCTION 2.1 Program Descriptions Retrofit Efficiency Options Program Advanced Performance Options Program Retrofit Express Program 2.2 Evaluation Overview Objectives Timing Role of Protocols 2.3 Evaluation Approach -An Overview Data Sources Analysis Elements 2.4 Report Layout METHODOLOGY 3.1 Sample Design Existing Data Sources Sample Design Overview Sample Segmentation Technology Segmentation Sample Allocation Final Sample Distribution Relative Precision Demonstration of Protocol Compliance 3.2 Engineering Analysis

7 TABLE OF CONTENTS (continued) Section Overview of the Engineering Approach Central Air-Conditioners (CAC) Adjustable Speed Drives (ASDs) for ventilation fans Custom Measures Other RE Measures Billing Regression Analysis Overview Data Sources for Billing Regression Analysis Data Aggregation and Analysis Dataset Development Analysis Periods Data Censoring Model Specification Billing Regression Analysis Results Net Billing Analysis Net-to-Gross Analysis Data Sources Self-Report Methods Discrete Choice Model Final Net-to-Gross Ratios 3-84 EVALUATION RESULTS 4.1 Ex Post Gross Impact Results 4.2 Net-to-Gross Adjustments 4.3 Ex Post Net Impacts 4.4 Realization Rates Gross Realization Rates for Energy Impacts Gross Realization Rates for Demand Impacts Gross Realization Rates for Therm Impacts Net Realization Rates 4.5 Overview of Realization Rates

8 LIST OF EXHIBITS Exhibit 1-1 Summary of Gross Evaluation Results for Commercial HVAC Applications Overall Impact Analysis Approach Commercial HVAC Segmentation and Distribution of Unique Sites 3-2 Proposed Standard Measure HVAC On-Sites 3-4 In Support of DOE-2 Model Development Available Custom Measure Sample Frame 3-5 Nonparticipant Survey Quotas, Telephone Survey Sample 3-6 Data Collected by Program and End Use 3-7 Telephone Sample Relative Precision Levels 3-8 Key Characteristics for DOE-2.1 E Prototypes 3-13 Annual Average HVAC Operating for Key Business Types 3-14 Equation for Estimating CAC Energy Savings 3-15 Equation for Estimating CAC Demand Savings 3-16 Baseline Interval Demand Estimate 3-18 Average Weekday Comparison of 3-19 kw vs. kw,o o Equation for Estimating ASD Energy Savings 3-21 Equation for Estimating ASD Demand Impacts 3-22 Billing Analysis Sample Frame 3-34 Pre-Censoring HVAC End-Use Technologies

9 LIST OF EXHIBITS (continued) Exhibit Billing Analysis Sample Frame Pre-Censoring Nonparticipants Commercial HVAC Rebated Technologies By Estimated Installation Date Distribution of Customers Removed from Billing Analysis By Data Censoring Criteria Customers with Invalid Billing Data Distribution of Customers Removed from Billing Analysis By Data Censoring Criteria Customers with Billing Aggregation Problems Billing Analysis Post-Censoring HVAC End-Use Sample Used Technologies Billing Analysis Sample Used Post-Censoring Nonparticipants Billing Regression Analysis Final Baseline Model Outputs Gross Billing Regression Analysis Final Model Outputs Commercial HVAC Gross Energy Impact SAE Coefficients Relative Precision Calculation Variables Used in HVAC Probit Model HVAC Probit Estimation Results Net Billing Regression Analysis Final Model Outputs Net Billing Regression Analysis Estimates of (1 -FR) Weighted Self-report Estimates of Free Ridership for HVAC Technology Groups Participant Out-of-Program Adoption Distribution

10 LIST OF EXHIBITS (continued) Exhibit Nonparticipant Adoption Distribution Participant Spillover Estimate Nonparticipant Spillover Estimate Purchase Model Variable Definitions Purchase Model Estimation Results Estimated Purchase Probabilities Equipment Choice Model Variable Definitions Equipment Choice Model Estimation Results Estimated NTG Ratios by Building Type Comparison of Net-to-Gross Ratios Final Net-to-Gross Ratios Ex Post Gross Energy Impacts Ex Post Gross Demand Impacts. Ex Post Gross Therm Impacts NTG Adjustments by Program and Technology Group Ex Post Net Energy Impacts Ex Post Net Demand Impacts iii

11 LIST OF EXHIBITS (continued) Exhibit Ex Post Net Therm Impacts Gross Energy Impact Realization Rates Gross Demand Impact Realization Rates Gross Therm Impact Realization Rates B'y Business Type and Technology Group For Commercial HVAC Measures Paid in 1996 Net Energy Impact Realization Rates For Commercial HVAC Measures Paid in 1996 Net Demand Impact Realization Rates For Commercial HVAC Measures Paid in 1996 Net Therm Impact Realization Rates Commercial HVAC Impact Summary By Technology Group iv

12 ATTACHMENTS TABLE OF CONTENTS Attachment CUSTOM HVAC ANALYSIS STANDARD HVAC ALGORITHM REVIEW RESU LTS TABLES PROTOCOL TABLES 6 & 7 PG&E RETROACTIVE WAIVER FOR PRE-1998 CEEI PROGRAM CARRY-OVER LIGHTING AND HVAC END USES, NET-TO-GROSS ANALYSIS A-1-1 A-2-1 A-3-1 A-4-1 A-5-1

13 SURVEY APPENDICES TABLE OF CONTENTS Appendix A B C D E F G H I J K L M PARTICIPANT SURVEY INSTRUMENT NONPARTICIPANT SURVEY INSTRUMENT CANVASS SURVEY INSTRUMENT ON-SITE INSTRUMENT PARTICIPANT SURVEY RESPONSE FREQUENCIES NONPARTICIPANT SURVEY RESPONSE FREQUENCIES CANVASS SURVEY RESPONSE FREQUENCIES PARTICIPANT SU RVEY DISPOSITION NONPARTICIPANT SURVEY DISPOSITION CANVASS SURVEY DISPOSITION PARTICIPANT SURVEY REFUSAL COMMENTS NONPARTICIPANT SURVEY REFUSAL COMMENTS CANVASS SURVEY REFUSAL COMMENTS A-1 B-1 C-1 D-1 E-1 F-1 G-1 H-1 I-1 J-1 K-1 L-1 M-1

14 1. EXECUTIVE SUMMARY This section presents a summary of the impact results for Heating, Ventilating, and Air- Conditioning (HVAC) technologies offered under Pacific Gas & Electric Company's (PG&E's) Pre-1998 Commercial Energy Efficiency Incentive (CEEI) Program Carry-Over, referred to in this report as the HVAC Program. This evaluation covers HVAC technology retrofits that were rebated during 1998, under CEEI programs offered from 1994 through These retrofits were performed under three different PG&E programs: the Retrofit Express (RE), the Retrofit Efficiency Options (REO), and the Advanced Performance Options (APO) Programs. The results are presented in two sections: Evaluation Results Summary (covering the numerical results of the study) and Major Findings. 1.1 EVALUATION RESULTS SUMMARY The evaluation results are summarized in terms of energy savings (kwh), demand savings (kw), therms impacts, and realization rates. Realization rates are defined as the ratio of the evaluation results (ex post) to the program design estimates (ex ante). All of these results are presented on a gross and net basis (i.e., before and after accounting for customer actions outside the program). Exhibit 1-1 presents the gross energy, demand and therm savings results (ex post and ex ante), together with each applicable gross realization rate. The net-to-gross ratio is comprised of free ridership, and participant and nonparticipant spillover effects. Exhibit 1-1 Summary of Gross Evaluation Results for Commercial HVAC Applications Gross Net Realization Net-To-Gross Realization Gross Savings Rate I-FR Spillover NTG Ratio Net Savings Rate EX ANTE kw 3, ,376 kwh 20,671, O ,525,132 Therms 575, ,840 EX POST kw 3, , kwh 13,659, ,865, Therms 489, , Overall, net ex post energy and therm impacts are relatively similar to ex ante estimates, while ex post net demand impacts are somewhat higher. Ex post and ex ante therm impacts are fairly consistent overall. Ex post gross energy impact estimates are measurably lower than ex ante, however the higher ex post NTG adjustment results in a net realization rate that is consistent with ex ante estimates. Ex post gross demand estimates are 12 percent higher than ex ante, which is exaggerated to 29 percent by the larger ex post NTG. Quantum Consulting, Inc. 1-1 Executive Summary

15 The ex ante numbers presented above in Exhibit 1-1 were obtained from PG&E's Marketing Decision Support System (MDSS), PG&E's program participant database. The values presented are identical to those filed in Table E-3 of the Technical Appendix of the Annual Summary Report on Demand Side Management Programs. These ex post results illustrate the following key points about the gross and net commercial HVAC impacts: Program Accomplishments: Nearly 87 percent of program energy savings are from HVAC technologies installed through the APO program. Almost all of the program therm savings are from HVAC technologies installed through the APO program, although a small therm savings was also generated in the REO program. Gross Impacts: Overall ex post gross impacts were 34 percent less than the ex ante estimates for energy, and 12 percent higher for demand. The lower energy estimates were attributable primarily to lower ex post impacts for the Water Chillers and other Custom measures within the APO and REO programs. The ex post estimates for these measures are based upon calibrated engineering results and the SAE results. The engineering analyses included a careful review of the original application calculations, an on-site audit to supplement the application information. In general, the differences between ex post impacts and ex ante estimates are due to improved information contributing to the ex post estimates or updated calculation methods. The SAE adjustment was 0.76 for these measures, contributing to the relatively low gross impact calculations relative to ex ante. Net Impacts: The net ex post impacts are lower than net ex ante estimates by 24 percent for energy, 2 percent for therms, and are 29 percent higher for demand. These results are driven by the ex ante and ex post net-to-gross (NTG) ratios. The ex ante NTG ratio was 0.75 for both demand and energy, while the ex post NTG ratio applied was much larger: 0.87 for energy and demand, and 0.90 for therms. These larger estimates measurably increase the net program effects. 1.2 MAJOR FINDINGS The key findings are summarized as follows: Overall, PG&E's ex ante estimates for demand and therm impacts for commercial HVAC technologies paid under the pre-1998 program carry-over were conservative, resulting in net realization rates exceeding one. At the same time, ex ante estimates of energy impacts were somewhat aggressive, and have a resulting net realization rate well below one. Gross ex post energy impacts were measurably lower than the ex ante estimates. This was attributable to engineering analyses of Water Chiller and other Customized APO and REO installations that found lower gross energy impacts. In addition, impacts were further reduced for these measures because the billing analysis detected less savings than predicted by engineering estimates. Larger NTG ratios resulted in larger ex post net realization rates relative to gross. For energy and therm impacts, this brought the net realization rates closer to one. For demand impacts, higher gross ex post values were exaggerated by the NTG adjustments, resulting in a net realization rate well above one. Quantum Consulting, Inc. 1-2 Executive Summary