ComEd and Nicor Gas Strategic Energy Management Program Impact Evaluation Report

Transcription

1 ComEd and Nicor Gas Strategic Energy Management Program Impact Energy Efficiency / Demand Response Plan: Electric Program Year 9 (EPY9) Gas Program Year 6 (GPY6) Presented to Commonwealth Edison Company Nicor Gas DRAFT March 2, 2018 Prepared by: Dustin Bailey Navigant Consulting Jean Rokke Navigant Consulting Navigant Consulting, Inc.

2 Submitted to: ComEd Three Lincoln Centre Oakbrook Terrace, IL Nicor Gas 1844 Ferry Road Naperville, IL Submitted by: Navigant Consulting, Inc. 150 N. Riverside, Suite 2100 Chicago, IL Contact: Randy Gunn, Managing Director Kevin Grabner, Associate Director Kevin. Jeff Erickson, Director Rob Neumann, Associate Director Disclaimer: This report was prepared by Navigant Consulting, Inc. ( Navigant ) for ComEd and Nicor Gas based upon information provided by ComEd, Nicor Gas and from other sources. Use of this report by any other party for whatever purpose should not, and does not, absolve such party from using due diligence in verifying the report s contents. Neither Navigant nor any of its subsidiaries or affiliates assumes any liability or duty of care to such parties, and hereby disclaims any such liability.

3 TABLE OF CONTENTS 1. Introduction Program Description Program Savings Program Savings by Measure Impact Analysis Findings and Recommendations Impact Parameter Estimates Other Impact Findings and Recommendations Appendix 1. Impact Analysis Methodology Verified Gross Program Savings Analysis Approach Verified Net Program Savings Analysis Approach Appendix 2. Impact Analysis Detail... 7 LIST OF TABLES AND FIGURES Table 3-1. ComEd EPY9 Total Annual Incremental Savings... 2 Table 3-2. Nicor Gas GPY6 Total Annual Incremental Savings... 2 Table 5-1. EPY9/GPY6 Volumetric Findings Detail... 5 Table 6-1. Deemed NTGR Values for EPY9/GPY Table 4-1. PY9 Energy Savings by Measures... 8 Page i

4 1. INTRODUCTION This report presents the results of the impact evaluation of ComEd s Electric Program Year 9 (EPY9) and Nicor Gas Program Year 6 (GPY6) Strategic Energy Management (SEM) Program. It presents a summary of the energy and demand impacts for the total program and broken out by relevant measure and program structure details. The appendix presents the impact analysis methodology. EPY9/GPY6 covers June 1, 2016 through December 31, PROGRAM DESCRIPTION The Strategic Energy Management (SEM) Program, managed by both ComEd and Nicor Gas, began as a pilot in EPY8/GPY5. The goal of the SEM Program is to apply a process of continuous energy management improvements that result in energy savings and demand reduction. The program seeks to educate participants in the identification of low cost and no cost measures, improve process efficiency, and reduce energy usage through behavioral changes. To encourage these savings, Nicor Gas provides an incentive of $0.10 per therm saved. In the Pilot year, ComEd provided a 10 percent bonus to rebates given on capital projects; after that year, an incentive of $0.01 per kwh saved has been given. While the utilities jointly manage the program, CLEAResult implements the day-to-day operation. The program achieves energy savings through operational and maintenance (O&M) improvements, incremental increases in capital energy efficiency projects, additional capital projects that would not otherwise have been considered (e.g., process changes, consideration of energy efficiency in all capital efforts), and improved persistence for O&M and capital projects. The SEM Program savings are calculated using site specific models developed by CLEAResult that have statistical regression analysis built into the model. The energy model uses two years of utility data prior to program participation. This data is associated with site information such as production and temperature to create baseline models that estimate a site s baseline usage based on these variables. After program participation begins, the model compares baseline energy usage to post-participation consumption, adjusted for temperature and production, and any differences attributed to SEM activities. The Pilot year began with 10 industrial participants enrolling in Cohort 1. In its second year, EPY9/GPY6, the program continued with seven of the Cohort 1 industrial participants and the addition of Cohort 2 with nine participants. Cohort 2 expanded the customer segment to include hospitals and universities in addition to the industrial segment. In August 2017, a Practitioner group was formed comprising of seven industrial and three commercial participants from Cohorts 1 and 2. Page-1

5 Table 2-1. Cohort and Practitioner Timeline Participant Group Customer Segment Time Period Cohort 1 10 Industrial November 2, 2014 October 31, 2015 Cohort 1 7 Industrial January 2, 2016 December 31, 2016 Cohort 2 2 Industrial 3 Hospitals 4 Universities June 1, 2016 May 31, 2017 Began in August 2017 with rolling Practitioner Group enrollment. The Practitioner participant s 7 Industrial usage will be re-baselined each year with 3 Commercial the savings calculated on the previous 12- month usage. Source: Navigant team analysis. ComEd s goals for SEM in EPY9 were 6 GWhs of energy savings and to develop strong customer relationships resulting in increased participation in capital projects. Similarly, Nicor Gas goals were SEM energy savings of 150,000 therms and an additional 200,000 therms of energy savings through the Business Energy Efficiency Rebate and Business Custom programs. 3. PROGRAM SAVINGS Table 3-1 and Table 3-2 summarize the incremental energy and demand savings the SEM Program achieved in PY9. The program had 21 participants in EPY9/GPY6 as shown in the following table. This program currently does not have demand savings calculators since savings are calculated using a whole building models. Savings Category Table 3-1. ComEd EPY9 Total Annual Incremental Savings Energy Savings (MWh) Source: ComEd and Nicor Gas tracking data and Navigant team analysis. Demand Savings (MW) Table 3-2. Nicor Gas GPY6 Total Annual Incremental Savings Source: ComEd and Nicor Gas tracking data and Navigant team analysis. Peak Demand Savings (MW) Ex Ante Gross Savings 13,088,906 NA NA Program Gross Realization Rate 1.22 NA NA Verified Gross Savings 15,977,950 NA NA Program Net-to-Gross Ratio (NTGR) 1 NA NA Verified Net Savings 15,977,950 NA NA Savings Category Energy Savings (Therms) Ex Ante Gross Savings 1,917,723 Program Gross Realization Rate 1 Verified Gross Savings 1,917,797 Program Net-to-Gross Ratio (NTGR) 1 Verified Net Savings 1,917,797 Page-2

6 4. PROGRAM SAVINGS BY MEASURE The SEM program tracked and evaluated savings at the site level, rather than measure level. SEM Site level detail can be found in Appendix 2. Impact Analysis Detail. Site details can be found in Section IMPACT ANALYSIS FINDINGS AND RECOMMENDATIONS 5.1 Impact Parameter Estimates As a behavioral based, model program, the program does not have standard impact parameters that are used to determine program savings. The program savings are calculated using billing regression methodologies built into the program models that are customized for each site. 5.2 Other Impact Findings and Recommendations Details regarding site by site results are shown below: Site 2 There appears to be an equation error in the SEM Tracking Models for this site. The implementer calculated an adjusted actual energy savings which subtracted a new plating line s energy from the actual energy. However, at the 3/27/2016 data point, the model s equation for adjusted actual energy stops accounting for the new plating-line. Site 5 Navigant removed one more outlier for this site than the implementer. This outlier was the data taken during 10/3/16-10/9/2016 when the total production was higher than 110% of the maximum production level in the baseline period. The implementer noted the same outlier but retained this point after conducting analysis and determining that the point was not statistically different than others around it. The implementer capped the variable at the limit of 110% the baseline maximum, allowing the site to claim a conservative amount of savings to avoid penalizing the site and removing the savings altogether. Navigant chose not to keep the data point as capping the data point creates a mismatch with the actual billing data used to estimate savings. Site 8 Navigant removed two outliers from this site. The first data point is that between 11/21/2016 and 11/26/2016 where the adjusted production of PM23 (production variable in model) was 69% of the minimum adjusted production of PM23 in the baseline period. The second data point is that between 12/26/12016 and 1/1/2017 where the adjusted production of PM23 was zero. The implementer marked both points as outliers as well by highlighting them in red but decided to keep both without explanation in their report. Navigant removed the same two outlying data points from the gas model for this site as for the electric model above. Site 9 Navigant removed one outlier from 4/2/2016 to 4/8/2016. During this period, the Mill 1 s production was more than 110% of the maximum production in the baseline. Lighting capital projects were also installed at this site in the baseline period on 10/01/2015. The implementer recorded that these lighting projects were not in place during the measurement period of the model so no savings were prorated in the Page-3

7 measurement period. However, it seems more likely that these lighting projects would have affected the measurement period. These projects were installed late in the baseline period and therefore may not have been fully considered in the baseline model. Navigant was concerned that this project may have been directly affecting the whole building site usage in the measurement period and accounted for the savings portion obtained by the lighting incentivized projects in the measurement period. Site 11 An incentivized lighting project was installed at this site in the baseline period on 7/15/2015. The implementer recorded that these lighting projects were not in place before the measurement period so no savings were prorated in the measurement period. However, it seems more likely that these lighting projects would have affected the measurement period. These projects were installed late in the baseline period and therefore may not have had been fully considered in the baseline model. Navigant was concerned that this project may have been directly affecting the whole building site usage in the measurement period. Navigant accounted for the portion savings obtained by lighting incentivized projects by adding it to the measurement period. Site 12 Navigant removed three outliers from the savings for this site. In all three cases, 7/17/2017-7/23/2016, 7/24/2016-7/30/2016, and 8/7/2016-8/13/2016, the cooling degree days CDD-70 were higher than the 110% maximum of the baseline. This resulted in the realization rate falling by 4%. Site 13 Navigant removed one outlier from the savings for this site at the 7/17/2017-7/23/2016 data point. The cooling degree days CDD-65 was much higher than the 110% maximum of the baseline. Site 15 through Site 18 Navigant removed four outliers from the savings for these projects. In three cases, 7/17/2017-7/23/2016, 7/24/2016-7/30/2016, and 8/7/2016-8/13/2016, the cooling degree days CDD-70 were higher than the 110% maximum of the baseline. One outlier occurred at 7/17/2017-7/23/2016 in which the cooling degree days CDD-55 was higher than the 110% maximum of the baseline. Site 20 Navigant removed three outliers from the savings for this site. In all three cases, 7/17/2017-7/23/2016, 7/24/2016-7/30/2016, and 8/7/2016-8/13/2016, the cooling degree days CDD-70 were higher than the 110% maximum of the baseline. Site 21 The electric savings for this site are much higher than the implementer reported. Navigant identified that a capital project that was installed by the SEM program had a negative effect on the SEM savings. The SEM program correctly removed the impact of the capital project savings by using reported ex ante savings, but between 10/20/2016 and 12/31/2016 the capital project did not seem to result in the savings equal to claimed ex ante savings resulting in the negative impact on the SEM savings. Navigant calculated the impact of this issue in several ways including adding a variable to represent this event. This variable was statistically significant and aligned with the installation of several process equipment VFD s projects. When this effect was accounted for the site realization rate increased because the ex post savings increased. Finding 1: As identified above, many sites had issues with the values of post variables falling outside of accepted standards more than 110% of maximum baseline or less than 90% of minimum baseline. In these cases, the model may not accurately represent what is occurring during these periods. Page-4

8 Recommendation 1: The implementer should continue to identify when the values of the variables fall outside of these accepted levels in the post conditions and account for them appropriately, by testing their impact or removing them. Finding 2: The capital project occurring at site 21 greatly affected the claimed SEM savings for the site and the SEM Program. This is a controls project that did not achieve savings for the initial four to five months of the project. The SEM Program correctly removed the impact of the capital project savings by using reported ex ante savings, but between 10/20/2016 and 12/31/2016 the capital project did not seem to result in the savings equal to claimed ex ante savings, resulting in the negative impact on the SEM savings. Navigant calculated the impact of this issue in several ways including adding a variable to represent this event. When this effect was accounted for the site realization rate increased because the ex post savings increased. Recommendation 2: If the models display strange results when considering installed capital projects, the implementer should coordinate with the capital project team or the utility to resolve the issue. Finding 3: Currently, when an energy model results in negative savings the program claims zero savings for the site. Navigant followed this procedure and the reported ex-post savings reflects this practice. If negative savings are included as a part of the program savings, the final savings are as follows: Table 5-1. EPY9/GPY6 Savings Including Negative Results Negative Ex Post Program Details Savings Evaluation Included Participants Total Ex Ante Savings (kwh) 13,088,906 8,646,900 Total Ex Post Savings (kwh) 15,977,950 11,585,585 Electric RR Total Ex Ante Savings (Therm) 1,917, ,679 Total Ex Post Savings (Therm) 1,917,797 1,058,656 Gas RR This practice of zeroing negative savings projects runs the risk of biasing results. The negative savings in the energy models are likely due to site s energy fluctuation: process changes, equipment changes, etc. Estimates of the standard error should be reported as noted in the recommendation. This energy fluctuation could be caused by changes that occur at the site that are not identified by the model and it could affect savings both negatively or positively. Further evaluation research into the negative savings discussion should be conducted in CY2018 to better understand this issue. Recommendation 3: Evaluation will consider and work with program leads to research the source of negative savings in CY2018 or CY2019. To further that research, the NREL Strategic Energy Management (SEM) Evaluation Protocol 1 states Evaluators should report point estimates of SEM program savings for the reporting period and standard errors or confidence intervals to indicate the program savings uncertainty. The implementer should consider including factors that indicate the level of uncertainty for program savings so that there can be more confidence in final results. 1 Page-5

9 6. APPENDIX 1. IMPACT ANALYSIS METHODOLOGY 6.1 Verified Gross Program Savings Analysis Approach Verified gross savings from the EPY9/GPY6 SEM Program were calculated using implementer provided engineering models that are grounded in site-specific data. These multi-regression models draw upon site data including energy usage, production, weather data and seasonality effects (including holidays or shutdowns). Electric and gas savings were independently evaluated using separate energy models. The verified gross savings reported includes interactive effects. With very few exceptions, the program design and calculation approach for the SEM Program does not allow us to quantify and remove the interactive effects due to the installation of multiple measures within the same timeframe. These methods closely follow the guidance of the NREL UMP protocol for SEM but the program should consider including the level of uncertainty as indicated above in Finding 3. Navigant staff carefully reviewed the models using the following procedure: A site-specific analysis approach was implemented. Because this program contains primarily behavioral-based changes, the International Performance Measurement and Verification Protocol (IPMVP) option C billing/metered data regression, was the main approach to impact evaluation. The data collection focused on verifying and/or updating the assumptions that feed into the implementer s energy model for each site. This data included: program tracking data and supporting documentation (project specifications, invoices, etc.), utility billing and interval data, Navigant-calibrated building automation system (BAS) trend logs and telephone conversations with onsite staff. This data was used with other information collected from the site to identify operating characteristics of the site both pre- and post- program participation. If major changes occurred at the site during or after the SEM activities, Navigant adjusted the energy model to account for these changes. The changes that could affect the model savings include: Change in hours of operation Change in numbers of employees Change in production Other measures installed at the site that were implemented through other Utility EE/DR programs or outside of the ComEd or Nicor Gas programs. 6.2 Verified Net Program Savings Analysis Approach Navigant calculated the verified net energy and demand savings by multiplying the verified gross savings estimates by a deemed net-to-gross ratio (NTGR). Table 6-1 shows the deemed NTGR values for EPY9 and GPY6. The deemed NTGR value of 1.00 for electric savings and 1.00 for gas savings were agreed to by stakeholders in discussions in the SAG. 2 2 Source: ComEd_NTG_History_and_PY9_Recommendations_ _Final.xlsx and Nicor_Gas_GPY6_NTG_Values_ _Final.xlsx, which are to be found on the IL SAG web site here: Page-6

10 Table 6-1. Deemed NTGR Values for EPY9/GPY6 EPY9/GPY6 Deemed Program Channel NTGR Value Electric 1.00 Natural Gas 1.00 Source: ComEd_NTG_History_and_PY9_Recommendations_ _Final.xlsx and Nicor_Gas_GPY6_NTG_Values_ _Final.xlsx, which are to be found on the 7. APPENDIX 2. IMPACT ANALYSIS DETAIL The program had an electric realization rate (RR) above 1.0 due to a site over estimating the impact of a capital project on the site s SEM savings. This was not the fault of the SEM Program, which used the reported ex ante savings provided by the utility. Instead, savings for the capital project were overestimated and not fully realized for several months which negatively impacted the SEM savings. Table 7-1 summarizes the site by site level incremental electric and gas savings the SEM Program achieved in EPY9/GPY6. A detailed site by site summary is included in Section 5.2. Page-7

11 Site Ex Ante Gross Savings (KWh) Table 7-1. PY9 Energy Savings by Measures Source: ComEd and Nicor Gas tracking data and Navigant team analysis. See site details in Section 5.2 Ex-Post Gross Savings (KWh) Verified Gross Realization Rate Ex Ante Gross Savings (Therms) Ex-Post Gross Savings (Therms) Verified Gross Realization Rate Site Site 2 852, , Site 3 2,606,757 2,606, , , Site Site 5 948, , , , Site Site 7 952, , ,633 10, Site Site 9 71,190 69, Site 10 3,175,951 3,036, Site , , Site 12 27,983 1, Site ,334 13, Site 14 1,155,823 1,155, Site , , Site ,046 69, Site Site ,019 29, Site , , ,498 25, Site , , Site 21 1,518,595 4,668, , , Total 13,088,906 15,977, ,917,723 1,917, For each site, Navigant reviewed and updated the implementer provided engineering models. Navigant staff generally followed the process below for this review: Step 1- Navigant recreated the provided energy models to ensure they aligned with the provided data. Step 2- Navigant confirmed that the model saving calculations accounted for all capital projects. Step 3- Navigant identified and accounted for any short-term effects that were occurring outside of the SEM influence. The telephone interviews with the site staff confirmed these changes. Page-8

12 Step 4- Navigant made additional changes to the model as needed. Changes may include excluding certain outlier data points or including additional variables. Several sites reported no electric or gas savings ex ante. Although activities were completed at these sites the energy model was unable to detect energy savings occurring at these site for a variety of reasons. For these sites, Navigant claimed zero savings to align with the implementer but each site model was verified and checked. The detailed site by site summary is attached below: Deatiled Summary of SEM Sites.xlsx Page-9